[
https://issues.apache.org/jira/browse/HADOOP-17130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17158531#comment-17158531
]
Steve Loughran commented on HADOOP-17130:
-----------------------------------------
I believe it is in {{updatePropertiesWIthDeprecatedKeys()}} because that does
update the map
{code}
private void updatePropertiesWIthDeprecatedKeys(
DeprecationContext deprecations, String[] newNames) {
for (String newName : newNames) {
String deprecatedKey =
deprecations.getReverseDeprecatedKeyMap().get(newName);
if (deprecatedKey != null && !getProps().containsKey(newName)) {
String deprecatedValue = getProps().getProperty(deprecatedKey);
if (deprecatedValue != null) {
getProps().setProperty(newName, deprecatedValue); /* HERE */
}
}
}
}
{code}
> Configuration.getValByRegex() shouldn't update the results while fetching.
> --------------------------------------------------------------------------
>
> Key: HADOOP-17130
> URL: https://issues.apache.org/jira/browse/HADOOP-17130
> Project: Hadoop Common
> Issue Type: Bug
> Components: common
> Affects Versions: 3.1.3
> Reporter: Mukund Thakur
> Assignee: Mukund Thakur
> Priority: Major
>
> We have seen this stacktrace while using ABFS file system. After analysing
> the stack trace we can see that getValByRegex() is reading the properties and
> substituting the value in the same call. This may cause the
> ConcurrentModificationException.
> {code:java}
> Caused by: java.util.concurrent.ExecutionException:
> java.util.ConcurrentModificationException at
> java.util.concurrent.FutureTask.report(FutureTask.java:122) at
> java.util.concurrent.FutureTask.get(FutureTask.java:192) at
> org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.generateSplitsInfo(OrcInputFormat.java:1877)
> ... 18 more Caused by: java.util.ConcurrentModificationException at
> java.util.Hashtable$Enumerator.next(Hashtable.java:1387) at
> org.apache.hadoop.conf.Configuration.getValByRegex(Configuration.java:3855)
> at
> org.apache.hadoop.fs.azurebfs.AbfsConfiguration.validateStorageAccountKeys(AbfsConfiguration.java:689)
> at
> org.apache.hadoop.fs.azurebfs.AbfsConfiguration.<init>(AbfsConfiguration.java:237)
> at
> org.apache.hadoop.fs.azurebfs.AzureBlobFileSystemStore.<init>(AzureBlobFileSystemStore.java:154)
> at
> org.apache.hadoop.fs.azurebfs.AzureBlobFileSystem.initialize(AzureBlobFileSystem.java:113)
> at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:3396) at
> org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:158) at
> org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:3456) at
> org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:3424) at
> org.apache.hadoop.fs.FileSystem.get(FileSystem.java:518) at
> org.apache.hadoop.fs.Path.getFileSystem(Path.java:361)
>
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
