anmolanmol1234 commented on code in PR #8611:
URL: https://github.com/apache/hadoop/pull/8611#discussion_r3664939759
##########
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsBlobClient.java:
##########
@@ -1673,30 +1750,101 @@ public Hashtable<String, String>
getXMSProperties(AbfsHttpOperation result)
@Override
public ListResponseData parseListPathResults(AbfsHttpOperation result, URI
uri)
throws AzureBlobFileSystemException {
+ final ListBlobResponseParser parser = ResponseParserFactory.getParser(
+ result.getResponseHeaderIgnoreCase(CONTENT_TYPE),
+ getBaseUrl().toString(),
+ () -> saxParserThreadLocal.get(),
+ getAbfsConfiguration().getPhotonArrowMemoryLimit());
try (InputStream stream = result.getListResultStream()) {
try {
- BlobListResultSchema listResultSchema;
- final SAXParser saxParser = saxParserThreadLocal.get();
- saxParser.reset();
- listResultSchema = new BlobListResultSchema();
- saxParser.parse(stream,
- new BlobListXmlParser(listResultSchema, getBaseUrl().toString()));
+ BlobListResultSchema listResultSchema = parser.parse(stream);
Review Comment:
parser.parse() on the Arrow path can throw unchecked exceptions
(org.apache.arrow.memory.OutOfMemoryException,
IllegalArgumentException/IllegalStateException on schema or vector-type
mismatch). These skip the inner catch (IOException) and land in the outer catch
(Exception), which logs"Unable to get stream for list results". The resulting
exception and metric are correct, but the log message is misleading — it points
at response-stream retrieval when the body was read successfully and it was
parsing that failed. On a Photon rollout this is the line on-call will grep.
Should catch RuntimeException alongside IOException in the inner block so parse
failures are wrapped and logged as parse failures:
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
