steveloughran commented on issue #654: HADOOP-15183 S3Guard store becomes inconsistent after partial failure of rename URL: https://github.com/apache/hadoop/pull/654#issuecomment-491055414 Test failure: interesting. Will fix. For people watching this, rename seems happy, but I don't like the way every put() is creating the entire list of parent entries and pushing them out, even for bulk operations like rename into a directory tree or commits into a tree. Both of these generate needless write load, one of O(files + depth(files) plan: the notion of a rename operation is expanded to cover a "bulk add operation" which is also initiated for the commits" -it'll track which entries have already been created and not reissue them. There's a separate JIRA for commit load; there's enough commonality to cover it here too. Not going to touch this until May 14; doing review of other people's work and some backporting until then ## reviews of this PR as is welcome!
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
