16°

hdfs append org.apache.hadoop.hdfs.protocol.AlreadyBeingCreatedException

hdfs append 时,如果文件不存在要县创建文件

代码:

        Path path = new Path(file);
        if (!fs.exists(path)) {
           fs.create(path);
        } 
        FSDataOutputStream out = fs.append(path); 
        out.write(content.getBytes("UTF-8"));
        out.flush();
        out.close();

报错:

Exception in thread "main" org.apache.hadoop.ipc.RemoteException: org.apache.hadoop.hdfs.protocol.AlreadyBeingCreatedException: failed to create file /huangq/dailyRolling/mommy-dailyRolling for DFSClient_-1456545217 on client 10.1.85.243 because current leaseholder is trying to recreate file.
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.recoverLeaseInternal(FSNamesystem.java:1374)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFileInternal(FSNamesystem.java:1246)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.appendFile(FSNamesystem.java:1426)
at org.apache.hadoop.hdfs.server.namenode.NameNode.append(NameNode.java:643)
at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)

异常的原因:

fs句柄的原因。HDFS(及大多数分布式文件系统)不支持文件并发写,Lease是HDFS用于保证唯一写的手段

解决办法:

创建完文件之后,关闭流fs,再次获得一次新的fs。

FileSystem fs = FileSystem.get(conf);
Path dstPath = new Path(dst);
if (!fs.exists(dstPath)) {
    fs.create(dstPath);
    fs.close();
    fs = FileSystem.get(conf);
} 
FSDataOutputStream fsout = fs.append(dstPath);

 


 

本文由【张】发布于开源中国,原文链接:https://my.oschina.net/u/2000675/blog/3137002

全部评论: 0

    我有话说: