Hive安装问题简述
本文简要介绍Hive安装过程中的常见问题:版本下载地址、版本依赖、依赖jar包。重要的是,记录Hive的元数据库手动初始化时报错的解决方案并对其进行反思
Hive的元数据库手动初始化
当使用hive 2.x之前的版本时,也可以不做初始化,当 hive 进行第一次启动时,会自动进行初始化,只是不会生成足够多的元数据库中的表,这些表需要在使用过程中慢慢生成。
如果使用的是Hive2.x版本,则Hive的元数据库必须手动初始化。使用的命令和显示界面如下:
在初始化元数据库的过程中报错
$cd /usr/local/java/hive/hive-2.1.0/bin # 进入hive安装目录 $./schematool -dbType mysql -initSchema # 对hive进行重新初始化报错信息如下:
hadoop@ubuntu16:/usr/local/java/hive-2.1.0/bin$ ./schematool -dbType mysql -initSchema SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/usr/local/java/hive-2.1.0/lib/log4j-slf4j-impl-2.4.1.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/usr/local/java/hadoop/hadoop-2.7.1/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory] Metastore connection URL: jdbc:mysql://localhost:3306/db_hive?createDatabaseIfNotExist=true&serverTimezone=GMT Metastore Connection Driver : com.mysql.jdbc.Driver Metastore connection User: root Mon Nov 29 14:50:31 CST 2021 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification. org.apache.hadoop.hive.metastore.HiveMetaException: Failed to get schema version. Underlying cause: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException : Communications link failureThe last packet successfully received from the server was 625 milliseconds ago. The last packet sent successfully to the server was 611 milliseconds ago. SQL Error code: 0 Use --verbose for detailed stacktrace. *** schemaTool failed ***在启动hive时报错
hadoop@ubuntu16:/usr/local/java/hive-2.1.0/bin$ hive报错信息如下(仅截取部分报错信息):
hadoop@ubuntu16:/usr/local/java/hive-2.1.0/bin$ ./hive SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/usr/local/java/hive-2.1.0/lib/log4j-slf4j-impl-2.4.1.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/usr/local/java/hadoop/hadoop-2.7.1/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]Logging initialized using configuration in jar:file:/usr/local/java/hive-2.1.0/lib/hive-common-2.1.0.jar!/hive-log4j2.properties Async: true Mon Nov 29 14:18:57 CST 2021 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification. Mon Nov 29 14:18:57 CST 2021 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification. Mon Nov 29 14:18:57 CST 2021 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification. Mon Nov 29 14:18:57 CST 2021 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification. Exception in thread "main" java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClientat org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:578)at org.apache.hadoop.hive.ql.session.SessionState.beginStart(SessionState.java:518)at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:705)at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:641)at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)at java.lang.reflect.Method.invoke(Method.java:498)因为ubuntu的终端控制台都是白色字体,而且输出内容很多,很难找到报错信息,因此可以根据报错信息选择一种类型的文件,将报错信息存至文件中。因为hadoop、hive基于java语言,所以.java为后缀名的文件可以看到字体高亮。
在初始化hive元数据库时,出现hive的报错信息*** schemaTool failed ***
源于上面其他的错误信息:org.apache.hadoop.hive.metastore.HiveMetaException: Failed to get schema version.Underlying cause: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException : Communications link failure
将第一个报错信息hive.metastore.HiveMetaException: Failed to get schema version.复制并在浏览器中搜索,寻找解决方法
解决方法:设置mysql时区,useSSL1
<property><name>javax.jdo.option.ConnectionURL</name><value>jdbc:mysql://localhost:3306/db_hive?createDatabaseIfNotExist=true&serverTimezone=GMT&useSSL=false</value><description>JDBC connect string for a JDBC metastore</description> </property>反思
Hive安装过程中的其他问题
hive下载地址
APACHE官网: Index of /hive (apache.org)
推荐(版本较多):华为云开源软件镜像网站-APACHE: Index of apache-local/hive (huaweicloud.com)
hadoop和hive版本对应关系
如:hive2.3.8 依赖于hadoop2.x.y,其他版本信息见:Downloads (apache.org)
JDBC驱动下载 mysql-connector-java-x.y.z.jar
Download MySQL Connector/J (Archived Versions)
错误:org.apache.hadoop.hive.metastore.HiveMetaException: Failed to get schema version_葉蕖的博客-CSDN博客 ↩︎
总结
以上是生活随笔为你收集整理的Hive安装问题简述的全部内容,希望文章能够帮你解决所遇到的问题。
- 上一篇: hadoop fs默认操作路径
- 下一篇: 字符串是否存在