[
https://issues.apache.org/jira/browse/SOLR-14653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17159325#comment-17159325
]
Uwe Schindler commented on SOLR-14653:
--------------------------------------
Thanks for opening this issue! Indeed we should try to use path-modifying
methods coming from zookeeper client library directly. Maybe they were added
since Mark Miller implemented this back in time.
{{java.net.URI}} would be my fallback, the above example with leaderPath could
be written as (untested):
{code:java}
String parent = new URI(null, null, leaderPath, null).resolve("..").getPath();
{code}
> Replace usages of org.apache.hadoop.fs.Path that are used to construct
> Zookeeper paths
> --------------------------------------------------------------------------------------
>
> Key: SOLR-14653
> URL: https://issues.apache.org/jira/browse/SOLR-14653
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Reporter: Erick Erickson
> Assignee: Erick Erickson
> Priority: Major
>
> See SOLR-13939. We construct Zookeeper paths at least in
> ShardLeaderElectionContextBase.java by using methods in
> org.apache.hadoop.fs.Path e.g. "String parent = new
> Path(leaderPath).getParent().toString()". This works because the Hadoop
> method uses forward slashes even on Windows machines.
> If those calls are changed to use nio Paths.get or similar, we can't run on
> Windows. Relying on hdfs classes to construct these paths in code unrelated
> to HDFS is suspicious at best.
> We should examine all inclusions of org.apache.hadoop.fs.Path and replace
> them with something else. Uwe suggests java.net.URI as one possibility.
> We should check to see what Zookeeper provides.
> There are 5 files that import org.apache.hadoop... that do _not_ have HDFS in
> their name, and 4 of those are probably legitimate (SolrSnapshotsTool for
> instance) so checking all of them won't take long.
> From Uwe's comment in SOLR-13939:
> {quote}"Hi,
> Windows is happy again. So your fix helped. I responded to mailing list,
> here again:
> In Zookeeper all paths are forward slashes (as it is a "path" inside the
> zookeeper environment). If you replace that with a NIO Windows Path it
> creates backslashes. This makes Zookeeper fail (of course).
> Actually using the Hadoop class is broken workaround, it may be better to use
> the more abstract java.net.URI class for correct "path" transformations. The
> risk with java.nio.file.Path is also that if the path exists in file system
> it handles the resolving different. So if some user has for some reason a
> local path exactly looking like a zookeper node path it would use different
> behaviour.
> So my suggestion: Use java.net.URI to build the path and use URI.getPath() or
> similar to make a String out of it. The alternative to replace all
> backslashes in the path to forwards slashes is a bad idea for the reasons I
> explained before.
> Now we should replace the hadoop class IMHO by java.net.URI.
> {quote}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
