dlmarion commented on code in PR #8065:
URL: https://github.com/apache/hadoop/pull/8065#discussion_r2571985726
##########
hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml:
##########
@@ -1779,6 +1779,21 @@
</description>
</property>
+<property>
+ <name>dfs.datanode.synconclose</name>
+ <value>false</value>
+ <description>
+ If this configuration is enabled, the datanode will instruct the
+ operating system to sync data block files contents to disk when the
+ blockfile is closed.
+
+ This has the benefit of ensuring that data is written to disk
+ immediately when a block is closed, avoiding the data being held
+ in the operating system cache. This can help data loss in the event
+ of a malfunction such as a power failure.
Review Comment:
> ..., avoiding the data being held in the operating system cache.
Does the sync of the dirty block in the page cache force the removal of the
block from the page cache? I'm not sure that it does. The user can advise the
OS, where supported, that it doesn't need the block in the page cache through
the `dfs.datanode.drop.cache.behind.writes` property or related flags when
opening the file.
##########
hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml:
##########
@@ -1779,6 +1779,21 @@
</description>
</property>
+<property>
+ <name>dfs.datanode.synconclose</name>
+ <value>false</value>
+ <description>
+ If this configuration is enabled, the datanode will instruct the
Review Comment:
```suggestion
If this property is enabled, the datanode will instruct the
```
##########
hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml:
##########
@@ -1779,6 +1779,21 @@
</description>
</property>
+<property>
+ <name>dfs.datanode.synconclose</name>
+ <value>false</value>
+ <description>
+ If this configuration is enabled, the datanode will instruct the
+ operating system to sync data block files contents to disk when the
+ blockfile is closed.
Review Comment:
```suggestion
blockfile is closed. This can help data loss in the event of a
malfunction such as a power
failure.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
