hadoop too many files异常处理

php中文网
发布: 2016-06-07 16:32:49
原创
1544人浏览过

今天 hadoop 集群任务执行失败了。报错信息如下 2013-10-26 08:00:03,229 ERROR server.TThreadPoolServer TThreadPoolServer.java:run182 - Error occurred during processing of message. at org.apache.hadoop.hive.service.HiveServer$ThriftHiveProcess

今天hadoop集群任务执行失败了。报错信息如下

<p class="devcodeoverflow"><ol><li>2013-10-26 08:00:03,229 ERROR server.TThreadPoolServer &#40;TThreadPoolServer.java:run&#40;182&#41;&#41; - Error occurred during processing of message.</li><li>    at org.apache.hadoop.hive.service.HiveServer$ThriftHiveProcessorFactory.getProcessor&#40;HiveServer.java:553&#41;</li><li>    at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run&#40;TThreadPoolServer.java:169&#41;</li><li>    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask&#40;ThreadPoolExecutor.java:886&#41;</li><li>    at java.util.concurrent.ThreadPoolExecutor$Worker.run&#40;ThreadPoolExecutor.java:908&#41;</li><li>    at java.lang.Thread.run&#40;Thread.java:662&#41;</li><li>    at org.apache.hadoop.hive.ql.session.SessionState.start&#40;SessionState.java:277&#41;</li><li>    at org.apache.hadoop.hive.service.HiveServer$HiveServerHandler.init&#40;HiveServer.java:136&#41;</li><li>    at org.apache.hadoop.hive.service.HiveServer$ThriftHiveProcessorFactory.getProcessor&#40;HiveServer.java:550&#41;</li><li>    ... 4 more</li><li>    at org.apache.hadoop.hive.ql.metadata.HiveUtils.getAuthorizeProviderManager&#40;HiveUtils.java:199&#41;</li><li>    at org.apache.hadoop.hive.ql.session.SessionState.start&#40;SessionState.java:272&#41;</li><li>    ... 6 more</li><li>Caused by: java.lang.RuntimeException: java.io.FileNotFoundException: /home/hadoop/hadoop-0.20.205.0/conf/mapred-site.xml &#40;Too many open files&#41;</li><li>    at org.apache.hadoop.conf.Configuration.loadResource&#40;Configuration.java:1231&#41;</li><li>    at org.apache.hadoop.conf.Configuration.loadResources&#40;Configuration.java:1093&#41;</li><li>    at org.apache.hadoop.conf.Configuration.getProps&#40;Configuration.java:1037&#41;</li><li>    at org.apache.hadoop.conf.Configuration.set&#40;Configuration.java:438&#41;</li><li>    at org.apache.hadoop.hive.conf.HiveConf.setVar&#40;HiveConf.java:762&#41;</li><li>    at org.apache.hadoop.hive.conf.HiveConf.setVar&#40;HiveConf.java:770&#41;</li><li>    at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run&#40;TThreadPoolServer.java:169&#41;</li><li>    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask&#40;ThreadPoolExecutor.java:886&#41;</li><li>    at java.util.concurrent.ThreadPoolExecutor$Worker.run&#40;ThreadPoolExecutor.java:908&#41;</li><li>    at java.lang.Thread.run&#40;Thread.java:662&#41;</li><li>Caused by: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: java.io.FileNotFoundException: /home/hadoop/hadoop-0.20.205.0/conf/core-site.xml &#40;Too many open files&#41;</li><li>    at org.apache.hadoop.hive.ql.session.SessionState.start&#40;SessionState.java:277&#41;</li><li>    at org.apache.hadoop.hive.service.HiveServer$HiveServerHandler.init&#40;HiveServer.java:136&#41;</li><li>    at org.apache.hadoop.hive.service.HiveServer$ThriftHiveProcessorFactory.getProcessor&#40;HiveServer.java:550&#41;</li><li>    ... 4 more</li><li>Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: java.io.FileNotFoundException: /home/hadoop/hadoop-0.20.205.0/conf/core-site.xml &#40;Too many open files&#41;</li><li>    at org.apache.hadoop.hive.ql.metadata.HiveUtils.getAuthorizeProviderManager&#40;HiveUtils.java:199&#41;</li><li>    at org.apache.hadoop.hive.ql.session.SessionState.start&#40;SessionState.java:272&#41;</li><li>    ... 6 more</li></ol></p>
登录后复制

debian系统解决方式
<p class="devcodeoverflow"><ol><li>ulimit -HSn 32768</li></ol></p>
登录后复制

最佳 Windows 性能的顶级免费优化软件
最佳 Windows 性能的顶级免费优化软件

每个人都需要一台速度更快、更稳定的 PC。随着时间的推移,垃圾文件、旧注册表数据和不必要的后台进程会占用资源并降低性能。幸运的是,许多工具可以让 Windows 保持平稳运行。

下载
来源:php中文网
本文内容由网友自发贡献,版权归原作者所有,本站不承担相应法律责任。如您发现有涉嫌抄袭侵权的内容,请联系admin@php.cn
最新问题
开源免费商场系统广告
热门教程
更多>
最新下载
更多>
网站特效
网站源码
网站素材
前端模板
关于我们 免责申明 举报中心 意见反馈 讲师合作 广告合作 最新更新
php中文网:公益在线php培训,帮助PHP学习者快速成长!
关注服务号 技术交流群
PHP中文网订阅号
每天精选资源文章推送

Copyright 2014-2025 https://www.php.cn/ All Rights Reserved | php.cn | 湘ICP备2023035733号