[
https://issues.apache.org/jira/browse/HADOOP-9565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15410866#comment-15410866
]
Steve Loughran commented on HADOOP-9565:
----------------------------------------
...OK, so the multipart is incremental, but and that final commit triggers the
artifacts coming into existence.
one thing we don't have in committers is an operation to explicitly abort the
final commit, so that if a speculative task knows that it mustn't commit, it
backs off. I think EMR had something like that in the distant past; I don't
know about today.
w.r.t. network partitions, loss of access to S3 from inside an EC2 farm is
going to be pretty rare and traumatic; I don't know about other deployments. I
would discard the use case "commit to long haul dest" as an unusual —though
distcp kind of disproves that, doesn't it
> Add a Blobstore interface to add to blobstore FileSystems
> ---------------------------------------------------------
>
> Key: HADOOP-9565
> URL: https://issues.apache.org/jira/browse/HADOOP-9565
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs, fs/s3, fs/swift
> Affects Versions: 2.6.0
> Reporter: Steve Loughran
> Assignee: Pieter Reuse
> Attachments: HADOOP-9565-001.patch, HADOOP-9565-002.patch,
> HADOOP-9565-003.patch, HADOOP-9565-004.patch, HADOOP-9565-005.patch,
> HADOOP-9565-006.patch, HADOOP-9565-branch-2-007.patch
>
>
> We can make the fact that some {{FileSystem}} implementations are really
> blobstores, with different atomicity and consistency guarantees, by adding a
> {{Blobstore}} interface to add to them.
> This could also be a place to add a {{Copy(Path,Path)}} method, assuming that
> all blobstores implement at server-side copy operation as a substitute for
> rename.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
