今天 hadoop 集群任务执行失败了。报错信息如下 2013-10-26 08:00:03,229 ERROR server.TThreadPoolServer TThreadPoolServer.java:run182 - Error occurred during processing of message. at org.apache.hadoop.hive.service.HiveServer$ThriftHiveProcess
今天hadoop集群任务执行失败了。报错信息如下
<p class="devcodeoverflow"><ol><li>2013-10-26 08:00:03,229 ERROR server.TThreadPoolServer (TThreadPoolServer.java:run(182)) - Error occurred during processing of message.</li><li> at org.apache.hadoop.hive.service.HiveServer$ThriftHiveProcessorFactory.getProcessor(HiveServer.java:553)</li><li> at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:169)</li><li> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)</li><li> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)</li><li> at java.lang.Thread.run(Thread.java:662)</li><li> at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:277)</li><li> at org.apache.hadoop.hive.service.HiveServer$HiveServerHandler.init(HiveServer.java:136)</li><li> at org.apache.hadoop.hive.service.HiveServer$ThriftHiveProcessorFactory.getProcessor(HiveServer.java:550)</li><li> ... 4 more</li><li> at org.apache.hadoop.hive.ql.metadata.HiveUtils.getAuthorizeProviderManager(HiveUtils.java:199)</li><li> at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:272)</li><li> ... 6 more</li><li>Caused by: java.lang.RuntimeException: java.io.FileNotFoundException: /home/hadoop/hadoop-0.20.205.0/conf/mapred-site.xml (Too many open files)</li><li> at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1231)</li><li> at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1093)</li><li> at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1037)</li><li> at org.apache.hadoop.conf.Configuration.set(Configuration.java:438)</li><li> at org.apache.hadoop.hive.conf.HiveConf.setVar(HiveConf.java:762)</li><li> at org.apache.hadoop.hive.conf.HiveConf.setVar(HiveConf.java:770)</li><li> at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:169)</li><li> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)</li><li> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)</li><li> at java.lang.Thread.run(Thread.java:662)</li><li>Caused by: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: java.io.FileNotFoundException: /home/hadoop/hadoop-0.20.205.0/conf/core-site.xml (Too many open files)</li><li> at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:277)</li><li> at org.apache.hadoop.hive.service.HiveServer$HiveServerHandler.init(HiveServer.java:136)</li><li> at org.apache.hadoop.hive.service.HiveServer$ThriftHiveProcessorFactory.getProcessor(HiveServer.java:550)</li><li> ... 4 more</li><li>Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: java.io.FileNotFoundException: /home/hadoop/hadoop-0.20.205.0/conf/core-site.xml (Too many open files)</li><li> at org.apache.hadoop.hive.ql.metadata.HiveUtils.getAuthorizeProviderManager(HiveUtils.java:199)</li><li> at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:272)</li><li> ... 6 more</li></ol></p>
<p class="devcodeoverflow"><ol><li>ulimit -HSn 32768</li></ol></p>
原文地址:hadoop too many files异常处理, 感谢原作者分享。
每个人都需要一台速度更快、更稳定的 PC。随着时间的推移,垃圾文件、旧注册表数据和不必要的后台进程会占用资源并降低性能。幸运的是,许多工具可以让 Windows 保持平稳运行。
Copyright 2014-2025 https://www.php.cn/ All Rights Reserved | php.cn | 湘ICP备2023035733号