[
https://issues.apache.org/jira/browse/HDFS-17630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18063981#comment-18063981
]
ASF GitHub Bot commented on HDFS-17630:
---------------------------------------
balodesecurity opened a new pull request, #8329:
URL: https://github.com/apache/hadoop/pull/8329
## Summary
- `PacketReceiver.MAX_PACKET_SIZE` was set via a `static { new
HdfsConfiguration(); ... }` initializer. When config resources are loaded from
HDFS URLs (e.g., during JAR loading via `URLJarFile`), the resulting
class-loading chain can instantiate `BlockReaderRemote` before `PacketReceiver`
finishes initializing. The JVM then returns the partially-initialized class
with `MAX_PACKET_SIZE == 0`, causing every subsequent block read to throw
`IOException: Incorrect value for packet payload size`.
- Fix: initialize `MAX_PACKET_SIZE` directly to the compile-time default
constant (eliminating the static initializer) and add a static
`setMaxPacketSize(Configuration)` method that `DFSClient` calls during
construction to apply any operator-configured override. This breaks the
circular class-loading dependency while preserving configurability.
## Test plan
- [ ] New tests `TestPacketReceiver#testMaxPacketSizeDefaultIsNonZero` and
`TestPacketReceiver#testSetMaxPacketSizeFromConfig` pass
- [ ] All existing `TestPacketReceiver` tests continue to pass
> Avoid PacketReceiver#MAX_PACKET_SIZE Initialized to 0
> -----------------------------------------------------
>
> Key: HDFS-17630
> URL: https://issues.apache.org/jira/browse/HDFS-17630
> Project: Hadoop HDFS
> Issue Type: Bug
> Affects Versions: 3.4.0
> Reporter: dzcxzl
> Priority: Major
> Labels: pull-request-available
>
> There are nested calls, causing the MAX_PACKET_SIZE of PacketReceiver to be 0.
>
> {code:java}
> java.io.IOException: Incorrect value for packet payload size: 1014776
> at
> org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:167)
> at
> org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:112)
> at
> org.apache.hadoop.hdfs.client.impl.BlockReaderRemote.readNextPacket(BlockReaderRemote.java:187)
> at
> org.apache.hadoop.hdfs.client.impl.BlockReaderRemote.read(BlockReaderRemote.java:146)
> at
> org.apache.hadoop.hdfs.ByteArrayStrategy.readFromBlock(ReaderStrategy.java:118)
> at
> org.apache.hadoop.hdfs.DFSInputStream.readBuffer(DFSInputStream.java:789)
> at
> org.apache.hadoop.hdfs.DFSInputStream.readWithStrategy(DFSInputStream.java:855)
> at org.apache.hadoop.hdfs.DFSInputStream.read(DFSInputStream.java:919)
> at java.base/java.io.DataInputStream.read(DataInputStream.java:158)
> at java.base/java.io.InputStream.transferTo(InputStream.java:796)
> at java.base/java.nio.file.Files.copy(Files.java:3151)
> at
> java.base/sun.net.www.protocol.jar.URLJarFile$1.run(URLJarFile.java:216)
> at
> java.base/sun.net.www.protocol.jar.URLJarFile$1.run(URLJarFile.java:212)
> at
> java.base/java.security.AccessController.doPrivileged(AccessController.java:571)
> at
> org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:1319)
> at org.apache.hadoop.conf.Configuration.getInt(Configuration.java:1545)
> at
> org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.<clinit>(PacketReceiver.java:82)
> at
> org.apache.hadoop.hdfs.client.impl.BlockReaderRemote.<init>(BlockReaderRemote.java:101)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
