HIve项目中常见错误，及修改办法

1）SecureCRT 7.3出现乱码或者删除不掉数据，免安装版的SecureCRT 卸载或者用虚拟机直接操作或者换安装版的SecureCRT

2）连接不上mysql数据库

（1）导错驱动包，应该把mysql-connector-java-5.1.27-bin.jar导入/opt/module/hive/lib的不是这个包。错把mysql-connector-java-5.1.27.tar.gz导入hive/lib包下。

（2）修改user表中的主机名称没有都修改为%，而是修改为localhost

3）hive默认的输入格式处理是CombineHiveInputFormat，会对小文件进行合并。

hive (default)> set hive.input.format;

hive.input.format=org.apache.hadoop.hive.ql.io.CombineHiveInputFormat
1
2
3

可以采用HiveInputFormat就会根据分区数输出相应的文件。

hive (default)> set hive.input.format=org.apache.hadoop.hive.ql.io.HiveInputFormat;
1

4）不能执行mapreduce程序

可能是hadoop的yarn没开启。

5）启动mysql服务时，报MySQL server PID file could not be found! 异常。

在/var/lock/subsys/mysql路径下创建hadoop102.pid，并在文件中添加内容：4396

6）报service mysql status MySQL is not running, but lock file (/var/lock/subsys/mysql[失败])异常。

解决方案：在/var/lib/mysql 目录下创建： -rw-rw----. 1 mysql mysql 5 12月 22 16:41 hadoop102.pid 文件，并修改权限为 777。

7）JVM堆内存溢出

//描述：java.lang.OutOfMemoryError: Java heap space

//解决：在yarn-site.xml中加入如下代码

yarn.scheduler.maximum-allocation-mb 2048 
yarn.scheduler.minimum-allocation-mb 2048
yarn.nodemanager.vmem-pmem-ratio 2.1 
mapred.child.java.opts -Xmx1024m
1
2
3
4
5
6
7
8

8、执行/usr/bin/hive命令进入hive时遇到如下错误：

Logging initialized using configuration in jar:file:/opt/cloudera/parcels/CDH-5.7.3-1.cdh5.7.3.p0.5/jars/hive-common-1.1.0-cdh5.7.3.jar!/hive-log4j.properties
Exception in thread “main” java.lang.RuntimeException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:540)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:689)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:628)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1493)
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.(RetryingMetaStoreClient.java:67)
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:82)
at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:2935)
at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:2954)
at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:513)
… 8 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1491)
… 13 more
Caused by: MetaException(message:Could not connect to meta store using any of the URIs provided. Most recent failure: org.apache.thrift.transport.TTransportException: java.net.ConnectException: Connection refused
at org.apache.thrift.transport.TSocket.open(TSocket.java:187)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:427)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.(HiveMetaStoreClient.java:237)
at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.(SessionHiveMetaStoreClient.java:74)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1491)
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.(RetryingMetaStoreClient.java:67)
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:82)
at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:2935)
at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:2954)
at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:513)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:689)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:628)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:579)
at org.apache.thrift.transport.TSocket.open(TSocket.java:182)
… 21 more
)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:474)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.(HiveMetaStoreClient.java:237)
at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.(SessionHiveMetaStoreClient.java:74)
… 18 more
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63

原因：没有正常启动Hive的Metastore Server 进程

解决办法：找到Metaserver Server进程所在的节点，执行如下命令：

hive --service metastore &
1

9、执行/usr/bin/hive命令进入hive时遇到如下错误：

FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:javax.jdo.JDODataStoreException: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ‘OPTION SQL_SELECT_LIMIT=DEFAULT’ at line 1
at org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:451)
at org.datanucleus.api.jdo.JDOPersistenceManager.getDataStoreConnection(JDOPersistenceManager.java:2259)
at org.apache.hadoop.hive.metastore.MetaStoreDirectSql.trySetAnsiQuotesForMysql(MetaStoreDirectSql.java:166)
1
2
3
4

原因：数据库驱动包版本不对应

解决办法：更换对应的数据库驱动

10、在hive中执行SQL时遇到如下错误：

Diagnostic Messages for this Task:
Container launch failed for container_1431310046688_0001_02_000013 : org.apache.hadoop.yarn.exceptions.YarnException: Unauthorized request to start container.
This token is expired. current time is 1431311178863 found 1431311136701
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
1
2
3
4

原因：集群时间不同步

解决办法：同步集群时间（date -s '2018-7-1 15:08:00)

11、错误信息：[ERROR] Terminal initialization failed; falling back to unsupported

解决方案：将hadoop中share/hadoop/yarn/lib路径下的jline包换成hive中匹配的jar包。
注意需要删除share/hadoop/yarn/lib下原有的jline包

12、错误信息：Caused by: java.net.URISyntaxException: Illegal character in scheme name at index 0:

file:///usr/local/hive-0.13.0/lib/hive-hbase-handler-0.13.0.jar
解决方案：修改hive中的conf下的hive-site.xml配置文件，hive.aux.jars.path配置改成file:///。注意中间不要有换行符，也就是不要回车。多个的话直接","号分隔开。

13、错误提示：java.lang.IncompatibleClassChangeError: Found class jline.Terminal, but interface was expected

解决方案：拷贝hive的lib目录下的jline高版本jar包到，hadoop的/share/hadoop/yarn/lib目录下

14、错误提示：java.lang.IncompatibleClassChangeError: Found class jline.Terminal, but interface was expected

解决方案：拷贝hive的lib目录下的jline高版本jar包到，hadoop的/share/hadoop/yarn/lib目录下