[
https://issues.apache.org/jira/browse/HADOOP-15076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16324290#comment-16324290
]
Steve Loughran commented on HADOOP-15076:
-----------------------------------------
bq. Isn't the commit protocol being slow part of "performance"? Can this be
rephrased?
more "people complain think their jobs slow", when they should be worrying that
the data may actually be invalid. That is, yes, slow is the performance, but
the real problem is that commit algorithms based on atomic renames don't work
if renames aren't atomic, even if they are fast.
Attached Patch 5
* intro has a table
* moved commit section down past I/O
* fadvise options reordered
* added a bit on good code too
* spell check
> Enhance s3a troubleshooting docs, add perf section
> --------------------------------------------------
>
> Key: HADOOP-15076
> URL: https://issues.apache.org/jira/browse/HADOOP-15076
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: documentation, fs/s3
> Affects Versions: 2.8.2
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Attachments: HADOOP-15076-001.patch, HADOOP-15076-002.patch,
> HADOOP-15076-003.patch, HADOOP-15076-004.patch, HADOOP-15076-005.patch
>
>
> A recurrent theme in s3a-related JIRAs, support calls etc is "tried upgrading
> the AWS SDK JAR and then I got the error ...". We know here "don't do that",
> but its not something immediately obvious to lots of downstream users who
> want to be able to drop in the new JAR to fix things/add new features
> We need to spell this out quite clearlyi "you cannot safely expect to do
> this. If you want to upgrade the SDK, you will need to rebuild the whole of
> hadoop-aws with the maven POM updated to the latest version, ideally
> rerunning all the tests to make sure something hasn't broken.
> Maybe near the top of the index.md file, along with "never share your AWS
> credentials with anyone"
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
