Got any snapshots? On Fri, Nov 8, 2019, 09:38 David M <[email protected]> wrote:
> All, > > > > I’m working on a cluster that is running Hadoop 2.7.3. I have one folder > in particular where the command hdfs dfs -du is giving me strange results. > If I query the folder and ask for a summary, it tells me 10 GB. If I don’t > ask for a summary, all of the folders underneath don’t even add up to 1 GB, > much less 10 GB. > > > > I’ve verified this is true over time and is true using the hdfs user or > any other user. We are on an HDP cluster, so we are using Ranger for HDFS > security, and Kerberos for authentication. We see similar results in > -count, where the size and counts are both different. We have not seen this > behavior in any other folders. > > > > See below for a sample output we are seeing. I’ve replaced the full path > with a fake path to protect the data we have on the cluster. Does anyone > know anything that would cause this behavior? Thanks! > > > > $ hdfs dfs -du -h /randomFolder > > 119.9 M /randomFolder/bug > > 1.0 M /randomFolder/commitment > > 86.8 K /randomFolder/customfield > > 31.3 M /randomFolder/epic > > 10.3 M /randomFolder/feature > > 4.0 M /randomFolder/insprintbug > > 372.9 K /randomFolder/project > > 15.1 K /randomFolder/projectstatus > > 330.9 M /randomFolder/story > > 256.3 M /randomFolder/subtask > > 74.7 K /randomFolder/subtemplate > > 89.6 M /randomFolder/task > > 7.4 M /randomFolder/techdebt > > 117.7 K /randomFolder/template > > 617.9 K /randomFolder/tempomember > > 8.2 K /randomFolder/tempoteam > > 1.4 M /randomFolder/tempoworklog > > > > $ hdfs dfs -du -h -s /randomFolder > > 10.6 G /randomFolder > > > > David McGinnis > > >
