Are you running with solr.lock.type=hdfs
?
Have you defined your DirectoryFactory - something like:
<directoryFactory name="DirectoryFactory"
class="solr.HdfsDirectoryFactory">
<bool name="solr.hdfs.blockcache.enabled">true</bool>
<bool name="solr.hdfs.blockcache.global">true</bool>
<int name="solr.hdfs.blockcache.slab.count">43</int>
<bool
name="solr.hdfs.blockcache.direct.memory.allocation">true</bool>
<int name="solr.hdfs.blockcache.blocksperbank">16384</int>
<bool name="solr.hdfs.blockcache.read.enabled">true</bool>
<bool name="solr.hdfs.nrtcachingdirectory.enable">true</bool>
<int name="solr.hdfs.nrtcachingdirectory.maxmergesizemb">128</int>
<int name="solr.hdfs.nrtcachingdirectory.maxcachedmb">1024</int>
<str name="solr.hdfs.home">hdfs://nameservice1:8020/solr8.2.0</str>
<str name="solr.hdfs.confdir">/etc/hadoop/conf.cloudera.hdfs1</str>
</directoryFactory>
-Joe
On 8/20/2020 2:30 AM, Prashant Jyoti wrote:
Hi Joe,
These are the errors I am running into:
org.apache.solr.common.SolrException: Error CREATEing SolrCore
'newcollsolr2_shard1_replica_n1': Unable to create core
[newcollsolr2_shard1_replica_n1] Caused by: Illegal char <:> at index
4:
hdfs://hn1-pjhado.tvbhpqtgh3judk1e5ihrx2k21d.tx.internal.cloudapp.net:8020/user/solr-data/newcollsolr2/core_node3/data\
<http://hn1-pjhado.tvbhpqtgh3judk1e5ihrx2k21d.tx.internal.cloudapp.net:8020/user/solr-data/newcollsolr2/core_node3/data\>
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1256)
at
org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:93)
at
org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:362)
at
org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:397)
at
org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:181)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:211)
at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:842)
at
org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:808)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:559)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:420)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:352)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1596)
at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:545)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:590)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235)
at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1607)
at
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1297)
at
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
at
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:485)
at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1577)
at
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1212)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:221)
at
org.eclipse.jetty.server.handler.InetAccessHandler.handle(InetAccessHandler.java:177)
at
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at
org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:322)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.eclipse.jetty.server.Server.handle(Server.java:500)
at
org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383)
at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:547)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375)
at
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:270)
at
org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117)
at
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336)
at
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313)
at
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
at
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129)
at
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:388)
at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806)
at
org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.solr.common.SolrException: Unable to create core
[newcollsolr2_shard1_replica_n1]
at
org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1327)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1217)
... 47 more
Caused by: org.apache.solr.common.SolrException: Illegal char <:> at
index 4:
hdfs://hn1-pjhado.tvbhpqtgh3judk1e5ihrx2k21d.tx.internal.cloudapp.net:8020/user/solr-data/newcollsolr2/core_node3/data\
<http://hn1-pjhado.tvbhpqtgh3judk1e5ihrx2k21d.tx.internal.cloudapp.net:8020/user/solr-data/newcollsolr2/core_node3/data\>
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1072)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:901)
at
org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1306)
... 48 more
Caused by: java.nio.file.InvalidPathException: Illegal char <:> at
index 4:
hdfs://hn1-pjhado.tvbhpqtgh3judk1e5ihrx2k21d.tx.internal.cloudapp.net:8020/user/solr-data/newcollsolr2/core_node3/data\
<http://hn1-pjhado.tvbhpqtgh3judk1e5ihrx2k21d.tx.internal.cloudapp.net:8020/user/solr-data/newcollsolr2/core_node3/data\>
at sun.nio.fs.WindowsPathParser.normalize(WindowsPathParser.java:182)
at sun.nio.fs.WindowsPathParser.parse(WindowsPathParser.java:153)
at sun.nio.fs.WindowsPathParser.parse(WindowsPathParser.java:77)
at sun.nio.fs.WindowsPath.parse(WindowsPath.java:94)
at sun.nio.fs.WindowsFileSystem.getPath(WindowsFileSystem.java:255)
at sun.nio.fs.AbstractPath.resolve(AbstractPath.java:53)
at org.apache.solr.core.SolrCore.initUpdateLogDir(SolrCore.java:1380)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:958)
... 50 more
Attaching it as a file too. Thanks!
On Wed, Aug 19, 2020 at 9:37 PM Joe Obernberger
<joseph.obernber...@gmail.com <mailto:joseph.obernber...@gmail.com>>
wrote:
Your exception didn't come across - can you paste it in?
-Joe
On 8/19/2020 10:50 AM, Prashant Jyoti wrote:
> You're right Andrew. Even I read about that. But there's a use
case for
> which we want to configure the said case.
>
> Are you also aware of what feature we are moving towards instead
of HDFS?
> Will you be able to help me with the error that I'm running into?
>
> Thanks in advance!
>
>
> On Wed, 19 Aug, 2020, 5:24 pm Andrew MacKay,
<andrew.mac...@superna.net <mailto:andrew.mac...@superna.net>>
> wrote:
>
>> I believe HDFS support is being deprecated in Solr. Not sure
you want to
>> continue configuration if support will disappear.
>>
>> On Wed, Aug 19, 2020 at 7:52 AM Prashant Jyoti
<jtprash...@gmail.com <mailto:jtprash...@gmail.com>>
>> wrote:
>>
>>> Hi all,
>>> Hope you are healthy and safe.
>>>
>>> Need some help with HDFS configuration.
>>>
>>> Could anybody of you share an example of the configuration
with which you
>>> are running Solr with HDFS in any of your production/dev
environments?
>>> I am interested in the parts of SolrConfig.xml /
Solr.in.cmd/sh which you
>>> may have modified. Obviously with the security parts obfuscated.
>>>
>>> I am stuck at an error and unable to move ahead. Attaching the
exception
>>> log if anyone is interested to look at the error.
>>>
>>> Thanks!
>>>
>>> --
>>> Regards,
>>> Prashant.
>>>
>> --
>> CONFIDENTIALITY NOTICE: The information contained in this email is
>> privileged and confidential and intended only for the use of the
>> individual
>> or entity to whom it is addressed. If you receive this
message in error,
>> please notify the sender immediately at 613-729-1100 and
destroy the
>> original message and all copies. Thank you.
>>
>
--
Regards,
Prashant.
<http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient>
Virus-free. www.avg.com
<http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient>
<#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
