[
https://issues.apache.org/jira/browse/HADOOP-13968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15815840#comment-15815840
]
Steve Loughran commented on HADOOP-13968:
-----------------------------------------
+when creating a path {{.temp_pending_put/dest.txt}}, there must be no existing
entry there. we will need to rely on a consistent FS for this, and may also
want to do some PUT operations which bypass the s3a FS APIs, but still work
with s3guard or an existing consistent object store. (e.g an atomic PUT/mkdir.
> S3a FS to support ".temp_pending_put" path for pending put operations
> ---------------------------------------------------------------------
>
> Key: HADOOP-13968
> URL: https://issues.apache.org/jira/browse/HADOOP-13968
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: HADOOP-13345
> Reporter: Steve Loughran
>
> S3AFileSystem to add support for a special path, such as
> {{.temp_pending_put/}} or similar, which, when used as the base of a path,
> indicates that the file is actually to be saved to the parent dir, but only
> via a delayed put commit operation.
> At the same time, we may need blocks on some normal fileIO ops under these
> dirs, especially rename and delete, as this would cause serious problems
> including data loss and large bills for pending data.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
