我嘗試Hive在 MR 執(zhí)行中使用SQL，但它中途失敗，錯(cuò)誤如下：Application?application_1570514228864_0001?failed?2?times?due?to?AM?Container?for?appattempt_1570514228864_0001_000002?exited?with?exitCode:?-1000
Failing?this?attempt.Diagnostics:?[2019-10-08?13:57:49.272]Failed?to?download?resource?{?{?s3a://tpcds/tmp/hadoop-yarn/staging/root/.staging/job_1570514228864_0001/libjars,?1570514262820,?FILE,?null?},pending,[(container_1570514228864_0001_02_000001)],1132444167207544,DOWNLOADING}?java.io.IOException:?Resource?s3a://tpcds/tmp/hadoop-yarn/staging/root/.staging/job_1570514228864_0001/libjars?changed?on?src?filesystem?(expected?1570514262820,?was?1570514269265從我的角度來看，錯(cuò)誤日志中的關(guān)鍵消息是libjars changed on src filesystem (expected 1570514262820, was 1570514269265。SO 有幾個(gè)關(guān)于此問題的線程，但尚未得到答復(fù)，例如thread1和thread2。我從apache jira和redhat bugzilla中發(fā)現(xiàn)了一些有價(jià)值的東西。NTP我通過所有相關(guān)節(jié)點(diǎn)同步時(shí)鐘。但同樣的問題仍然存在。歡迎任何評(píng)論，謝謝。
3 回答
慕尼黑5688855
TA貢獻(xiàn)1848條經(jīng)驗(yàn) 獲得超2個(gè)贊
我仍然不知道為什么資源文件的時(shí)間戳不一致，并且沒有辦法通過配置方式修復(fù)它，AFAIK。
但是，我設(shè)法找到了解決方法來跳過該問題。讓我在這里為可能遇到同樣問題的人總結(jié)一下。
通過檢查錯(cuò)誤日志并在源代碼中搜索Hadoop，我們可以在hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/FSDownload.java.
只需刪除異常拋出語句，
private void verifyAndCopy(Path destination)
throws IOException, YarnException {
final Path sCopy;
try {
sCopy = resource.getResource().toPath();
} catch (URISyntaxException e) {
throw new IOException("Invalid resource", e);
}
FileSystem sourceFs = sCopy.getFileSystem(conf);
FileStatus sStat = sourceFs.getFileStatus(sCopy);
if (sStat.getModificationTime() != resource.getTimestamp()) {
/**
throw new IOException("Resource " + sCopy +
" changed on src filesystem (expected " + resource.getTimestamp() +
", was " + sStat.getModificationTime());
**/
LOG.debug("[Gearon][Info] The timestamp is not consistent among resource files.\n" +
"Stop throwing exception . It doesn't affect other modules. ");
}
if (resource.getVisibility() == LocalResourceVisibility.PUBLIC) {
if (!isPublic(sourceFs, sCopy, sStat, statCache)) {
throw new IOException("Resource " + sCopy +
" is not publicly accessible and as such cannot be part of the" +
" public cache.");
}
}
downloadAndUnpack(sCopy, destination);
}
構(gòu)建hadoop-yarn-project并復(fù)制“hadoop-yarn-common-xxxjar to$HADOOP_HOME/share/hadoop/yarn”。
將此線程留在這里，并感謝您對(duì)如何在不更改hadoop源代碼的情況下修復(fù)它的任何進(jìn)一步解釋。
慕慕森
TA貢獻(xiàn)1856條經(jīng)驗(yàn) 獲得超17個(gè)贊
我必須做同樣的事情，這應(yīng)該是可配置的，即使很小的延遲也會(huì)導(dǎo)致執(zhí)行失敗，如果將hadoop文件系統(tǒng)更改為使用s3并運(yùn)行MR程序，則可能會(huì)發(fā)生這種情況，注意*請(qǐng)確保，您使用的是相同的jdk version 來生成 jar，如 apache hadoop 文檔中所述，適用于您的 hadoop 版本，否則您可能會(huì)遇到錯(cuò)誤。
搖曳的薔薇
TA貢獻(xiàn)1793條經(jīng)驗(yàn) 獲得超6個(gè)贊
實(shí)際上，您只需手動(dòng)將日期設(shè)置回之前的值即可解決此問題。例如，使用帶有或選項(xiàng)的touch命令：-t STAMP-d DATE
sudo?touch?-d?'07?Apr?2022?11:12:30.000?+0000'?'<path_to_file>'
添加回答
舉報(bào)
0/150
提交
取消