Long day ... let me try again. Here is what I see on my FS mount: /hadoop/wxxxs:
0-15:58 djh@c24-03-06 ~> *ls /hadoop/wxxxs/data/* # No files listed 0-16:01 djh@c24-03-06 ~> *touch /hadoop/wxxxs/data/foo* 0-16:01 djh@c24-03-06 ~> *ls /hadoop/wxxxs/data/* foo packed-hbfs/ raw/ tmp/ 0-16:01 djh@c24-03-06 ~> *rm /hadoop/wxxxs/data/foo* 0-16:01 djh@c24-03-06 ~> *ls /hadoop/wxxxs/data/* packed-hbfs/ raw/ tmp/ In the log, I see a lot of: 2020-06-19 16:01:35,281 ERROR org.apache.hadoop.hdfs.nfs.nfs3.RpcProgramNfs3: cookieverf mismatch. request cookieverf: 1591897331315 dir cookieverf: 1592428367587 2020-06-19 16:01:35,287 ERROR org.apache.hadoop.hdfs.nfs.nfs3.RpcProgramNfs3: cookieverf mismatch. request cookieverf: 1591897331315 dir cookieverf: 1592428367587 2020-06-19 16:01:35,454 ERROR org.apache.hadoop.hdfs.nfs.nfs3.RpcProgramNfs3: cookieverf mismatch. request cookieverf: 1591897331315 dir cookieverf: 1592428367587 On Fri, Jun 19, 2020 at 4:04 PM Daniel Howard <[email protected]> wrote: > Here is another clue from my NFS mount: the directory entry comes up empty > until I write to it: > 0-16:01 djh@c24-03-06 ~> *touch /hadoop/wxxxs/data/foo* > 0-16:01 djh@c24-03-06 ~> *ls /hadoop/wxxxs/data/* > foo packed-hbfs/ raw/ tmp/ > 0-16:01 djh@c24-03-06 ~> *rm /hadoop/wxxxs/data/foo* > 0-16:01 djh@c24-03-06 ~> *ls /hadoop/wxxxs/data/* > packed-hbfs/ raw/ tmp/ > > On Fri, Jun 12, 2020 at 11:51 AM Daniel Howard <[email protected]> > wrote: > >> Hello, >> >> I wanted to provide NFS access on a cluster. Since the NFS server is >> lightweight, and since I use config management, I figured I have every node >> run its local nfs3 service, and have the nodes mount localhost:/. >> >> This strategy appears to work fine, at first, but before long, I get into >> a state where each system is seeing a different truth on NFS. >> >> Example: >> >> 0-11:45 djh@c24-03-06 ~> *hadoop fs -ls /data/* >> Found 2 items >> drwxrwxr-x+ - djh djh 0 2020-06-10 20:51 /data/packed-hbfs >> drwxrwxr-x+ - djh djh 0 2020-06-11 14:06 /data/tmp >> 0-11:43 djh@c24-03-06 ~> *ls /hadoop/wxxxs/data/* >> 0-11:43 djh@c24-03-06 ~> *ls /hadoop/wxxxs/data/tmp* >> allocation_2020-06-10_1591799995129_0.json >> allocation_2020-06-10_1591809732060_0.json >> allocation_2020-06-10_1591856129529_0.json >> allocation_2020-06-10_1591856640093_0.json >> allocation_2020-06-10_1591858477153_0.json >> allocation_2020-06-11_1591884434468_0.json >> allocation_2020-06-11_1591886358283_0.json >> allocation_2020-06-11_1591896890160_0.json >> >> 0-11:43 djh@c24-03-08 ~> *ls /hadoop/wxxxs/data* >> packed-hbfs/ tmp/ >> 0-11:43 djh@c24-03-08 ~> *ls /hadoop/wxxxs/data/tmp* >> allocation_2020-06-10_1591799995129_0.json >> allocation_2020-06-10_1591809732060_0.json >> allocation_2020-06-10_1591856129529_0.json >> allocation_2020-06-10_1591856640093_0.json >> allocation_2020-06-10_1591858477153_0.json >> allocation_2020-06-11_1591884434468_0.json >> allocation_2020-06-11_1591886358283_0.json >> allocation_2020-06-11_1591896890160_0.json >> >> My best guess is that the NameNode assumes there is only one NFS, so the >> NameNode only sends updates to one node? >> >> Is what I am doing supported? Alternatively, is there a decent FUSE >> implementation for Apache Hadoop? I can switch to having a single NFS >> server if that is what is supported. >> >> Thanks, >> -danny >> >> >> -- >> http://dannyman.toldme.com >> > > > -- > http://dannyman.toldme.com > -- http://dannyman.toldme.com
