Hello Sébastien, I think hflush() on a sequence file should be working according to existing unit tests that we have. What is the underlying file system you are using (HDFS, S3A, etc.)? Support for hflush() varies depending on the file system implementation.
Chris Nauroth On Mon, Jan 27, 2025 at 6:06 AM Sébastien Rebecchi <[email protected]> wrote: > Hello, > > I got this issue with both hadoop 2.9.1 and 3.4.1. > For SequenceFile.Writer class, after doing hflush(), data is not visible > to new readers, it is visible only after doing close(). > The doc asserts that data must be visible to readers > https://hadoop.apache.org/docs/r3.4.1/hadoop-project-dist/hadoop-common/filesystem/outputstream.html > > What should we do for this? > > Thanks, > Sébastien. >
