[
https://issues.apache.org/jira/browse/HADOOP-17130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mukund Thakur updated HADOOP-17130:
-----------------------------------
Description:
We have seen this stacktrace while using ABFS file system. After analysing the
stack trace we can see that getValByRegex() is reading the properties and
substituting the value in the same call. This may cause the
ConcurrentModificationException.
{code:java}
Caused by: java.util.concurrent.ExecutionException:
java.util.ConcurrentModificationException at
java.util.concurrent.FutureTask.report(FutureTask.java:122) at
java.util.concurrent.FutureTask.get(FutureTask.java:192) at
org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.generateSplitsInfo(OrcInputFormat.java:1877)
... 18 more Caused by: java.util.ConcurrentModificationException at
java.util.Hashtable$Enumerator.next(Hashtable.java:1387) at
org.apache.hadoop.conf.Configuration.getValByRegex(Configuration.java:3855) at
org.apache.hadoop.fs.azurebfs.AbfsConfiguration.validateStorageAccountKeys(AbfsConfiguration.java:689)
at
org.apache.hadoop.fs.azurebfs.AbfsConfiguration.<init>(AbfsConfiguration.java:237)
at
org.apache.hadoop.fs.azurebfs.AzureBlobFileSystemStore.<init>(AzureBlobFileSystemStore.java:154)
at
org.apache.hadoop.fs.azurebfs.AzureBlobFileSystem.initialize(AzureBlobFileSystem.java:113)
at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:3396) at
org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:158) at
org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:3456) at
org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:3424) at
org.apache.hadoop.fs.FileSystem.get(FileSystem.java:518) at
org.apache.hadoop.fs.Path.getFileSystem(Path.java:361)
{code}
was:
{code:java}
Caused by: java.util.concurrent.ExecutionException:
java.util.ConcurrentModificationException at
java.util.concurrent.FutureTask.report(FutureTask.java:122) at
java.util.concurrent.FutureTask.get(FutureTask.java:192) at
org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.generateSplitsInfo(OrcInputFormat.java:1877)
... 18 more Caused by: java.util.ConcurrentModificationException at
java.util.Hashtable$Enumerator.next(Hashtable.java:1387) at
org.apache.hadoop.conf.Configuration.getValByRegex(Configuration.java:3855) at
org.apache.hadoop.fs.azurebfs.AbfsConfiguration.validateStorageAccountKeys(AbfsConfiguration.java:689)
at
org.apache.hadoop.fs.azurebfs.AbfsConfiguration.<init>(AbfsConfiguration.java:237)
at
org.apache.hadoop.fs.azurebfs.AzureBlobFileSystemStore.<init>(AzureBlobFileSystemStore.java:154)
at
org.apache.hadoop.fs.azurebfs.AzureBlobFileSystem.initialize(AzureBlobFileSystem.java:113)
at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:3396) at
org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:158) at
org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:3456) at
org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:3424) at
org.apache.hadoop.fs.FileSystem.get(FileSystem.java:518) at
org.apache.hadoop.fs.Path.getFileSystem(Path.java:361)
{code}
> Configuration.getValByRegex() shouldn't be updating the results while
> fetching.
> -------------------------------------------------------------------------------
>
> Key: HADOOP-17130
> URL: https://issues.apache.org/jira/browse/HADOOP-17130
> Project: Hadoop Common
> Issue Type: Bug
> Components: common
> Affects Versions: 3.1.3
> Reporter: Mukund Thakur
> Assignee: Mukund Thakur
> Priority: Major
>
> We have seen this stacktrace while using ABFS file system. After analysing
> the stack trace we can see that getValByRegex() is reading the properties and
> substituting the value in the same call. This may cause the
> ConcurrentModificationException.
> {code:java}
> Caused by: java.util.concurrent.ExecutionException:
> java.util.ConcurrentModificationException at
> java.util.concurrent.FutureTask.report(FutureTask.java:122) at
> java.util.concurrent.FutureTask.get(FutureTask.java:192) at
> org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.generateSplitsInfo(OrcInputFormat.java:1877)
> ... 18 more Caused by: java.util.ConcurrentModificationException at
> java.util.Hashtable$Enumerator.next(Hashtable.java:1387) at
> org.apache.hadoop.conf.Configuration.getValByRegex(Configuration.java:3855)
> at
> org.apache.hadoop.fs.azurebfs.AbfsConfiguration.validateStorageAccountKeys(AbfsConfiguration.java:689)
> at
> org.apache.hadoop.fs.azurebfs.AbfsConfiguration.<init>(AbfsConfiguration.java:237)
> at
> org.apache.hadoop.fs.azurebfs.AzureBlobFileSystemStore.<init>(AzureBlobFileSystemStore.java:154)
> at
> org.apache.hadoop.fs.azurebfs.AzureBlobFileSystem.initialize(AzureBlobFileSystem.java:113)
> at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:3396) at
> org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:158) at
> org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:3456) at
> org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:3424) at
> org.apache.hadoop.fs.FileSystem.get(FileSystem.java:518) at
> org.apache.hadoop.fs.Path.getFileSystem(Path.java:361)
>
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
