Win7 eclipse 调试远程Hadoop Failed to set permissions of path

我不是女神ヾ 2021-06-24 14:00 270阅读 0赞

hadoop1.2.1在windows远程调试linux的程序时，报错:

12/04/24 15:32:44 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable  
12/04/24 15:32:44 ERROR security.UserGroupInformation: PriviledgedActionException as:Administrator cause:java.io.IOException: Failed to set permissions of path: \\tmp\\hadoop-Administrator\\mapred\\staging\\Administrator-519341271\\.staging to 0700  
Exception in thread "main" java.io.IOException: Failed to set permissions of path: \\tmp\\hadoop-Administrator\\mapred\\staging\\Administrator-519341271\\.staging to 0700  
    at org.apache.hadoop.fs.FileUtil.checkReturnValue(FileUtil.java:682)  
    at org.apache.hadoop.fs.FileUtil.setPermission(FileUtil.java:655)  
    at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:509)  
    at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:344)  
    at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:189)  
    at org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:116)  
    at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:856)  
    at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:850)  
    at java.security.AccessController.doPrivileged(Native Method)  
    at javax.security.auth.Subject.doAs(Subject.java:396)  
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1093)  
    at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:850)  
    at org.apache.hadoop.mapreduce.Job.submit(Job.java:500)  
    at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:530)  
    at com.hadoop.learn.test.WordCountTest.main(WordCountTest.java:85)

这个是Windows下文件权限问题，在Linux下可以正常运行，不存在这样的问题。

解决方法是，修改/hadoop-1.2.1/src/core/org/apache/hadoop/fs/FileUtil.java里面的checkReturnValue，注释掉即可（有些粗暴，在Window下，可以不用检查）

......  
      private static void checkReturnValue(boolean rv, File p,  
                                           FsPermission permission  
                                           ) throws IOException {  
        /** 
    if (!rv) { 
    throw new IOException("Failed to set permissions of path: " + p + 
    " to " + 
    String.format("%04o", permission.toShort())); 
    } 
    **/  
      }  
    ......

重新编译生成class文件后替换掉hadoop-core-1.2.1.jar里面的原class即可。替换之后，刷新项目，设置好正确的jar包依赖，现在再运行WordCountTest，即可。