[
https://issues.apache.org/jira/browse/HDFS-17966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zita Dombi updated HDFS-17966:
------------------------------
Description:
WebHDFS operations that accepts a path, entity name, or metadata string as an
HTTP query parameter (e.g. RENAME, CONCAT, etc.) and has special characters in
the target path parameters (such as &, <, >, ", ') are automatically
HTML-encoded. E.g. renaming a file to a path containing "&" results in "&"
written literally to HDFS. Native WebHDFS on the NameNode handles these
operations correctly without corrupting the path.
1. HttpFS
{code:java}
curl -X PUT
"https://<host>:14000/webhdfs/v1/user/admin/httpfstest1?op=RENAME&destination=%2Fuser%2Fadmin%2Fhttpfstest1%26"
hdfs dfs -ls /user/admin/ | grep httpfstest
# Result: /user/admin/httpfstest1&
{code}
2. Native WebHDFS
{code:java}
curl -X PUT
"https://<active-namenode>:20102/webhdfs/v1/user/admin/nativetest1?op=RENAME&destination=%2Fuser%2Fadmin%2Fnativetest1%26"
hdfs dfs -ls /user/admin/ | grep nativetest
# Result: /user/admin/nativetest1&
{code}
HttpServer2 registers a global XSS protection filter, QuotingInputFilter,
`quoteHtmlChars()` explicitly converts reserved characters. Native WebHDFS does
not apply this filter and parses parameters directly via JAX-RS, keeping the
original path intact.
was:
WebHDFS operations that accepts a path, entity name, or metadata string as an
HTTP query parameter (e.g. RENAME) via HttpFS and has special characters in the
target path parameters (such as &, <, >, ", ') are automatically HTML-encoded.
E.g. renaming a file to a path containing "&" results in "&" written literally
to HDFS. Native WebHDFS on the NameNode handles these operations correctly
without corrupting the path.
1. HttpFS
{code:java}
curl -X PUT
"https://<host>:14000/webhdfs/v1/user/admin/httpfstest1?op=RENAME&destination=%2Fuser%2Fadmin%2Fhttpfstest1%26"
hdfs dfs -ls /user/admin/ | grep httpfstest
# Result: /user/admin/httpfstest1&
{code}
2. Native WebHDFS
{code:java}
curl -X PUT
"https://<active-namenode>:20102/webhdfs/v1/user/admin/nativetest1?op=RENAME&destination=%2Fuser%2Fadmin%2Fnativetest1%26"
hdfs dfs -ls /user/admin/ | grep nativetest
# Result: /user/admin/nativetest1&
{code}
HttpServer2 registers a global XSS protection filter, QuotingInputFilter,
`quoteHtmlChars()` explicitly converts reserved characters. Native WebHDFS does
not apply this filter and parses parameters directly via JAX-RS, keeping the
original path intact.
> HttpFS: Path parameters with special characters are HTML-encoded by
> QuotingInputFilter, corrupting paths
> --------------------------------------------------------------------------------------------------------
>
> Key: HDFS-17966
> URL: https://issues.apache.org/jira/browse/HDFS-17966
> Project: Hadoop HDFS
> Issue Type: Bug
> Reporter: Zita Dombi
> Priority: Major
>
> WebHDFS operations that accepts a path, entity name, or metadata string as an
> HTTP query parameter (e.g. RENAME, CONCAT, etc.) and has special characters
> in the target path parameters (such as &, <, >, ", ') are automatically
> HTML-encoded. E.g. renaming a file to a path containing "&" results in
> "&" written literally to HDFS. Native WebHDFS on the NameNode handles
> these operations correctly without corrupting the path.
> 1. HttpFS
> {code:java}
> curl -X PUT
> "https://<host>:14000/webhdfs/v1/user/admin/httpfstest1?op=RENAME&destination=%2Fuser%2Fadmin%2Fhttpfstest1%26"
> hdfs dfs -ls /user/admin/ | grep httpfstest
> # Result: /user/admin/httpfstest1&
> {code}
> 2. Native WebHDFS
> {code:java}
> curl -X PUT
> "https://<active-namenode>:20102/webhdfs/v1/user/admin/nativetest1?op=RENAME&destination=%2Fuser%2Fadmin%2Fnativetest1%26"
> hdfs dfs -ls /user/admin/ | grep nativetest
> # Result: /user/admin/nativetest1&
> {code}
> HttpServer2 registers a global XSS protection filter, QuotingInputFilter,
> `quoteHtmlChars()` explicitly converts reserved characters. Native WebHDFS
> does not apply this filter and parses parameters directly via JAX-RS, keeping
> the original path intact.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
