Re: Is that solr supports multi version operations?
Thanks for your explaination. @Alexandre Rafalovitch @Walter Underwood
My case is use SOLR as an Index Service of some NoSQL systems，it is
a common Requirement to
guarantee the consistency of index&source data .
There maybe TWO ways to write source data/index：
1. write index to solr first， then write source data to NoSQL system.
if write NoSQL failed，I want to rollback solr update。due to solr
not support rollback，I have ever
think to use multi-version to implement this feature， but ，feel
disappointed。
2. write source data first， then wirte index to solr. this is my
current implementation。and I found it fit for me。
Alexandre Rafalovitch 于2018年9月19日周三 下午1:41写道：
> I think if you try hard enough, it is possible to get Solr to keep
> multiple documents that would normally keep only the latest version.
> They will just have different internal lucene id.
>
> This may of course break a lot of other things like SolrCloud and
> possibly facet counts.
>
> So, I would ask the actual business case first. It is entirely
> possible that there are other ways to achieve the desired objectives.
>
> Regards,
>Alex.
>
> On 19 September 2018 at 00:17, Walter Underwood
> wrote:
> > No. Solr only has one version of a document. It is not a multi-version
> database.
> >
> > Each replica will return the newest version it has.
> >
> > wunder
> > Walter Underwood
> > wun...@wunderwood.org
> > http://observer.wunderwood.org/ (my blog)
> >
> >> On Sep 18, 2018, at 7:11 PM, zhenyuan wei wrote:
> >>
> >> Hi all,
> >>add solr document with overwrite=false will keepping multi version
> >> documents，
> >> My question is ：
> >>1. How to search newest documents？with what options？
> >>2. How to delete old version < newest version documents?
> >>
> >> for example:
> >> {
> >>"id":"1002",
> >>"name":["james"],
> >>"_version_":1611998319085617152,
> >>"name_str":["james"]},
> >> {
> >>"id":"1002",
> >>"name":["lily"],
> >>"_version_":1611998307815522304,
> >>"name_str":["lily"]},
> >> {
> >>"id":"1002",
> >>"name":["lucy"],
> >>"_version_":1611998248265842688,
> >>"name_str":["lucy"]}]
> >>
> >> 1. curl http://localhost:8983/solr/collection001/query?q=*:* return
> all
> >> ，
> >>how to search to make response return the newest one？
> >> 2. how to delete document of version
> >> [1611998307815522304，1611998248265842688] ,
> >> which is older then 1611998319085617152.
> >
>
Re: Implementing NeuralNetworkModel RankNet in Solr LTR
Hi Koji, Thanks for your reply and provide the information. Just to check, is this supported in Solr 7.4.0? Regards, Edwin On Wed, 19 Sep 2018 at 11:02, Koji Sekiguchi wrote: > Hi, > > > https://github.com/airalcorn2/Solr-LTR#RankNet > > > > Has anyone tried on this before? And what is the format of the training > > data that this model requires? > > I haven't tried it, but I'd like to inform you that there is another > project of LTR we've been > developed: > > https://github.com/LTR4L/ltr4l > > It has many LTR algorithms based on neural network, SVM and boosting. > > Koji > > On 2018/09/12 11:44, Zheng Lin Edwin Yeo wrote: > > Hi, > > > > I am working on to implementing Solr LTR in Solr 7.4.0 by using the > > NeuralNetworkModel for the feature selection and model training, and I > have > > found this site which uses RankNet: > > https://github.com/airalcorn2/Solr-LTR#RankNet > > > > Has anyone tried on this before? And what is the format of the training > > data that this model requires? > > > > Regards, > > Edwin > > >
CACHE -> fieldValueCache usage
Hi all, sorry if I bothered you all but in these days I'm just struggling what's going on with my production servers... Looking at Solr Admin Panel I've found the CACHE -> fieldValueCache tab where all the values are 0. class:org.apache.solr.search.FastLRUCache description:Concurrent LRU Cache(maxSize=1, initialSize=10, minSize=9000, acceptableSize=9500, cleanupThread=false) stats: CACHE.searcher.fieldValueCache.cumulative_evictions:0 CACHE.searcher.fieldValueCache.cumulative_hitratio:0 CACHE.searcher.fieldValueCache.cumulative_hits:0 CACHE.searcher.fieldValueCache.cumulative_inserts:0 CACHE.searcher.fieldValueCache.cumulative_lookups:0 CACHE.searcher.fieldValueCache.evictions:0 CACHE.searcher.fieldValueCache.hitratio:0 CACHE.searcher.fieldValueCache.hits:0 CACHE.searcher.fieldValueCache.inserts:0 CACHE.searcher.fieldValueCache.lookups:0 CACHE.searcher.fieldValueCache.size:0 CACHE.searcher.fieldValueCache.warmupTime:0 what do you thing, is that normal? Given that this stats comes from a production server I suppose to have some number here and looking at solrconfig.xml I don't see any configuration regarding the fieldValueCache. Don't should I see something here? Cheers, Vincenzo -- Vincenzo D'Amore
Re: Is that solr supports multi version operations?
You are doing the right thing. Always write to the repository first, then
write to Solr. The repository is the single source of truth.
We write to the repository, then have a process that copies new items
to Solr.
wunder
Walter Underwood
wun...@wunderwood.org
http://observer.wunderwood.org/ (my blog)
> On Sep 19, 2018, at 3:03 AM, zhenyuan wei wrote:
>
> Thanks for your explaination. @Alexandre Rafalovitch @Walter Underwood
>
>My case is use SOLR as an Index Service of some NoSQL systems，it is
> a common Requirement to
> guarantee the consistency of index&source data .
>There maybe TWO ways to write source data/index：
> 1. write index to solr first， then write source data to NoSQL system.
>if write NoSQL failed，I want to rollback solr update。due to solr
> not support rollback，I have ever
> think to use multi-version to implement this feature， but ，feel
> disappointed。
>
> 2. write source data first， then wirte index to solr. this is my
> current implementation。and I found it fit for me。
>
>
>
>
>
>
>
>
>
>
> Alexandre Rafalovitch 于2018年9月19日周三 下午1:41写道：
>
>> I think if you try hard enough, it is possible to get Solr to keep
>> multiple documents that would normally keep only the latest version.
>> They will just have different internal lucene id.
>>
>> This may of course break a lot of other things like SolrCloud and
>> possibly facet counts.
>>
>> So, I would ask the actual business case first. It is entirely
>> possible that there are other ways to achieve the desired objectives.
>>
>> Regards,
>> Alex.
>>
>> On 19 September 2018 at 00:17, Walter Underwood
>> wrote:
>>> No. Solr only has one version of a document. It is not a multi-version
>> database.
>>>
>>> Each replica will return the newest version it has.
>>>
>>> wunder
>>> Walter Underwood
>>> wun...@wunderwood.org
>>> http://observer.wunderwood.org/ (my blog)
>>>
On Sep 18, 2018, at 7:11 PM, zhenyuan wei wrote:
Hi all,
add solr document with overwrite=false will keepping multi version
documents，
My question is ：
1. How to search newest documents？with what options？
2. How to delete old version < newest version documents?
for example:
{
"id":"1002",
"name":["james"],
"_version_":1611998319085617152,
"name_str":["james"]},
{
"id":"1002",
"name":["lily"],
"_version_":1611998307815522304,
"name_str":["lily"]},
{
"id":"1002",
"name":["lucy"],
"_version_":1611998248265842688,
"name_str":["lucy"]}]
1. curl http://localhost:8983/solr/collection001/query?q=*:* return
>> all
，
how to search to make response return the newest one？
2. how to delete document of version
[1611998307815522304，1611998248265842688] ,
which is older then 1611998319085617152.
>>>
>>
Re: Is that solr supports multi version operations?
On 9/18/2018 8:11 PM, zhenyuan wei wrote: Hi all, add solr document with overwrite=false will keepping multi version documents， My question is ： 1. How to search newest documents？with what options？ 2. How to delete old version < newest version documents? When Solr is compiling results, it will only return one copy of a particular document (based on uniqueKey value).All other copies will be removed. I suspect (but do not know for sure) that which document will be returned is not defined. On a multi-shard index, if different copies are in different shards, which one is returned will be decided by which shard answers the query first, or maybe which one answers last. If multiple copies exist in the same core, that's probably more deterministic, but it might not be the copy you wanted. Solr isn't designed to have multiple versions of the same uniqueKey in the index. Lucene itself doesn't care -- it's going to return all of them -- but if you want to be sure which one is returned, you'd need to write the Lucene-based software yourself instead of using Solr. As you mentioned in the last message, writing to your true data store and then writing to Solr if that succeeds is a better option. Or you could simply write to your data store and then have your indexing software detect and read the new records from there. Thanks, Shawn
RE: weird error for accessing solr
A little update. For the client machine where solr admin page behaves differently, it turns out that the requests on the page like */admin/* were never served. I think it is related to the server setting that might prevent these urls with "/admin/" from being sent. In essence, it is not a solr issue. Thanks a lot Steve -Original Message- From: Erick Erickson Sent: Tuesday, September 18, 2018 4:18 PM To: solr-user Subject: Re: weird error for accessing solr bq. can you share *ALL* of... from both machines! On Tue, Sep 18, 2018 at 12:40 PM Shawn Heisey wrote: > > On 9/18/2018 12:23 PM, Gu, Steve (CDC/OD/OADS) (CTR) wrote: > > I have set up my solr as a standalone service and the its url is > > http://solr.server:8983/solr. I opened 8983 on solr.server to anyone, and > > solr can be accessed from laptops/desktops. But when I tried to access the > > solr from some servers, I got the error of SolrCore Initialization > > Failures. The left nav on the page is shown but indicates that the solr is > > set up as SolrCloud, which is not. > > On the dashboard when you see the Cloud tab, can you share *ALL* of > what's under JVM in the Args section? > > Thanks, > Shawn >
Re: weird error for accessing solr
Thanks for bringing closure to this, Whew! On Wed, Sep 19, 2018 at 8:04 AM Gu, Steve (CDC/OD/OADS) (CTR) wrote: > > A little update. > > For the client machine where solr admin page behaves differently, it turns > out that the requests on the page like */admin/* were never served. I think > it is related to the server setting that might prevent these urls with > "/admin/" from being sent. > > In essence, it is not a solr issue. > > Thanks a lot > Steve > > -Original Message- > From: Erick Erickson > Sent: Tuesday, September 18, 2018 4:18 PM > To: solr-user > Subject: Re: weird error for accessing solr > > bq. can you share *ALL* of... > > from both machines! > On Tue, Sep 18, 2018 at 12:40 PM Shawn Heisey wrote: > > > > On 9/18/2018 12:23 PM, Gu, Steve (CDC/OD/OADS) (CTR) wrote: > > > I have set up my solr as a standalone service and the its url is > > > http://solr.server:8983/solr. I opened 8983 on solr.server to anyone, > > > and solr can be accessed from laptops/desktops. But when I tried to > > > access the solr from some servers, I got the error of SolrCore > > > Initialization Failures. The left nav on the page is shown but indicates > > > that the solr is set up as SolrCloud, which is not. > > > > On the dashboard when you see the Cloud tab, can you share *ALL* of > > what's under JVM in the Args section? > > > > Thanks, > > Shawn > >
Re: SOLR 7.0 DIH out of memory issue with sqlserver
Has this ever worked? IOW, is this something that's changed or has just never worked? The obvious first step is to start Solr with more than 1G of memory. Solr _likes_ memory and a 1G heap is quite small. But you say: "Increasing the heap size further doesnt start SOLR instance itself.". How much RAM do you have on your machine? What other programs are running? You should be able to increase the heap and start Solr if you have the RAM on your machine so I'd figure out what's behind that issue first. I regularly start Solr with 16 or 32G of memory on my local machines, I know of installations running Solr with 60G heaps so this points to something really odd about your environment. When you "increase it further", exactly _how_ does Solr fail to start? What appears in the Solr logs? etc. Really, troubleshoot that issue first I'd recommend. If DIH still needs a ridiculous amount of memory, it's usually the JDBC driver trying to read all the rows into memory at once and you'll have to explore the jdbc driver settings in detail. Best, Erick On Tue, Sep 18, 2018 at 11:16 PM Tanya Bompi wrote: > > Hi, > I am using the Microsoft Jdbc driver 6.4 version in Solr 7.4.0 . I have > tried removing the selectMethod=Cursor and still it runs out of heap space. > Do we have anyone who has faced similar issue. > > Thanks > Tanya > > > On Tue, Sep 18, 2018 at 6:38 PM Shawn Heisey wrote: > > > On 9/18/2018 4:48 PM, Tanya Bompi wrote: > > >I have the SOLR 7.0 setup with the DataImportHandler connecting to the > > > sql server db. I keep getting OutOfMemory: Java Heap Space when doing a > > > full import. The size of the records is around 3 million so not very > > huge. > > > I tried the following steps and nothing helped thus far. > > > > See this wiki page: > > > > https://wiki.apache.org/solr/DataImportHandlerFaq > > > > You already have the suggested fix -- setting responseBuffering to > > adaptive. You might try upgrading the driver. If that doesn't work, > > you're probably going to need to talk to Microsoft about what you need > > to do differently on the JDBC url. > > > > I did find this page: > > > > > > https://docs.microsoft.com/en-us/sql/connect/jdbc/using-adaptive-buffering?view=sql-server-2017 > > > > This says that when using adaptive buffering, you should avoid using > > selectMethod=cursor. So you should try removing that parameter. > > > > Thanks, > > Shawn > > > >
Seeking a simple way to test my index.
I've got a Solr instance which crawls roughly 3,500 seed pages, depth of 1, at 240 institutions, all but 1 of which I don't control. I recrawl once a month or so. Naturally if one of the sites I crawl changes, then I need to know to update my seed URLs. I've been checking this by hand, which was tenable when my site was smaller, but is now completely unreasonable. Is there a way to test my index without actually having to run a lot of manual searches? Perhaps an output I could skim? Any suggestions would be helpful. Thanks, Chip
Re: Seeking a simple way to test my index.
Have you looked at Apache Nutch? Seems like the direct match for your - growing - requirements and it does integrate with Solr. Or one of the other solutions, like http://stormcrawler.net/ http://www.norconex.com/collectors/ Otherwise, this does not really feel like a Solr question. Regards, Alex. On 19 September 2018 at 14:01, Chip Calhoun wrote: > I've got a Solr instance which crawls roughly 3,500 seed pages, depth of 1, > at 240 institutions, all but 1 of which I don't control. I recrawl once a > month or so. Naturally if one of the sites I crawl changes, then I need to > know to update my seed URLs. I've been checking this by hand, which was > tenable when my site was smaller, but is now completely unreasonable. > > > Is there a way to test my index without actually having to run a lot of > manual searches? Perhaps an output I could skim? Any suggestions would be > helpful. > > > Thanks, > > Chip
Re: Seeking a simple way to test my index.
I do use Nutch as my crawler, but just as my crawler, so I hadn't thought to look for an answer there. I will do so. thank you. Chip From: Alexandre Rafalovitch Sent: Wednesday, September 19, 2018 2:05:41 PM To: solr-user Subject: Re: Seeking a simple way to test my index. Have you looked at Apache Nutch? Seems like the direct match for your - growing - requirements and it does integrate with Solr. Or one of the other solutions, like http://stormcrawler.net/ http://www.norconex.com/collectors/ Otherwise, this does not really feel like a Solr question. Regards, Alex. On 19 September 2018 at 14:01, Chip Calhoun wrote: > I've got a Solr instance which crawls roughly 3,500 seed pages, depth of 1, > at 240 institutions, all but 1 of which I don't control. I recrawl once a > month or so. Naturally if one of the sites I crawl changes, then I need to > know to update my seed URLs. I've been checking this by hand, which was > tenable when my site was smaller, but is now completely unreasonable. > > > Is there a way to test my index without actually having to run a lot of > manual searches? Perhaps an output I could skim? Any suggestions would be > helpful. > > > Thanks, > > Chip
UI fields signification
Hi dear SOLR community. On this page of the documentation: https://lucene.apache.org/solr/guide/6_6/core-specific-tools.html what are the fields "current" and "gen" referring to? I have not been able to find that anywhere :( Thanks, JMS
Re: UI fields signification
I would say this is the relevant page for the "current" and "generation" https://lucene.apache.org/solr/guide/6_6/index-replication.html And I think generation refers to the actual Lucene index, so is explained further here: https://lucene.apache.org/core/6_6_0/core/org/apache/lucene/codecs/lucene62/package-summary.html#package.description Regards, Alex. On 19 September 2018 at 14:33, Jean-Marc Spaggiari wrote: > Hi dear SOLR community. > > On this page of the documentation: > https://lucene.apache.org/solr/guide/6_6/core-specific-tools.html what are > the fields "current" and "gen" referring to? I have not been able to find > that anywhere :( > > Thanks, > > JMS
Unnecessary Components
I am helping implement solr for a "downloadable library" of sorts. The objective is that communities without internet access will be able to access a library's worth of information on a small, portable device. As such, I am working within strict space constraints. What are some non-essential components of solr that can be cut to conserve space for more information? -- Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Re: Seeking a simple way to test my index.
Chip: Another thing that might work for you are the streaming/export capabilities. It has the capacity to efficiently return some data (docValues only) for very large result sets. You'd have to have some automated way to feed it what to look for. But that's a fallback, I'd first look at Nutch as I bet someone's had a similar problem before in Nutch-land ;) On Wed, Sep 19, 2018 at 11:18 AM Chip Calhoun wrote: > > I do use Nutch as my crawler, but just as my crawler, so I hadn't thought to > look for an answer there. I will do so. thank you. > > > Chip > > > From: Alexandre Rafalovitch > Sent: Wednesday, September 19, 2018 2:05:41 PM > To: solr-user > Subject: Re: Seeking a simple way to test my index. > > Have you looked at Apache Nutch? Seems like the direct match for your > - growing - requirements and it does integrate with Solr. Or one of > the other solutions, like http://stormcrawler.net/ > http://www.norconex.com/collectors/ > > Otherwise, this does not really feel like a Solr question. > > Regards, >Alex. > > On 19 September 2018 at 14:01, Chip Calhoun wrote: > > I've got a Solr instance which crawls roughly 3,500 seed pages, depth of 1, > > at 240 institutions, all but 1 of which I don't control. I recrawl once a > > month or so. Naturally if one of the sites I crawl changes, then I need to > > know to update my seed URLs. I've been checking this by hand, which was > > tenable when my site was smaller, but is now completely unreasonable. > > > > > > Is there a way to test my index without actually having to run a lot of > > manual searches? Perhaps an output I could skim? Any suggestions would be > > helpful. > > > > > > Thanks, > > > > Chip
Re: SOLR 7.0 DIH out of memory issue with sqlserver
Hi Erick, Thank you for the follow-up. I have resolved the issue with the increase in heapSize and I am able to set the SOLR VM to initialize with a 3G heap size and the subset of 1 mil records was fetched successfully. Although it fails with the entire 3 mil records. So something is off with the adaptive buffering setting as I see it is not helping. And I also set the autosoftcommit param. I might have to increase the heap size further to see if it helps. I will keep posted if my issue doesnt resolve. Thanks, Tanya On Wed, Sep 19, 2018 at 8:22 AM Erick Erickson wrote: > Has this ever worked? IOW, is this something that's changed or has > just never worked? > > The obvious first step is to start Solr with more than 1G of memory. > Solr _likes_ memory and a 1G heap is quite small. But you say: > "Increasing the heap size further doesnt start SOLR instance itself.". > How much RAM do you have on your machine? What other programs are > running? You should be able to increase the heap and start Solr if you > have the RAM on your machine so I'd figure out what's behind that > issue first. I regularly start Solr with 16 or 32G of memory on my > local machines, I know of installations running Solr with 60G heaps so > this points to something really odd about your environment. > > When you "increase it further", exactly _how_ does Solr fail to start? > What appears in the Solr logs? etc. Really, troubleshoot that issue > first I'd recommend. > > If DIH still needs a ridiculous amount of memory, it's usually the > JDBC driver trying to read all the rows into memory at once and you'll > have to explore the jdbc driver settings in detail. > > Best, > Erick > On Tue, Sep 18, 2018 at 11:16 PM Tanya Bompi > wrote: > > > > Hi, > > I am using the Microsoft Jdbc driver 6.4 version in Solr 7.4.0 . I have > > tried removing the selectMethod=Cursor and still it runs out of heap > space. > > Do we have anyone who has faced similar issue. > > > > Thanks > > Tanya > > > > > > On Tue, Sep 18, 2018 at 6:38 PM Shawn Heisey > wrote: > > > > > On 9/18/2018 4:48 PM, Tanya Bompi wrote: > > > >I have the SOLR 7.0 setup with the DataImportHandler connecting > to the > > > > sql server db. I keep getting OutOfMemory: Java Heap Space when > doing a > > > > full import. The size of the records is around 3 million so not very > > > huge. > > > > I tried the following steps and nothing helped thus far. > > > > > > See this wiki page: > > > > > > https://wiki.apache.org/solr/DataImportHandlerFaq > > > > > > You already have the suggested fix -- setting responseBuffering to > > > adaptive. You might try upgrading the driver. If that doesn't work, > > > you're probably going to need to talk to Microsoft about what you need > > > to do differently on the JDBC url. > > > > > > I did find this page: > > > > > > > > > > https://docs.microsoft.com/en-us/sql/connect/jdbc/using-adaptive-buffering?view=sql-server-2017 > > > > > > This says that when using adaptive buffering, you should avoid using > > > selectMethod=cursor. So you should try removing that parameter. > > > > > > Thanks, > > > Shawn > > > > > > >
Re: Unnecessary Components
On 9/19/2018 1:48 PM, oddtyme wrote: I am helping implement solr for a "downloadable library" of sorts. The objective is that communities without internet access will be able to access a library's worth of information on a small, portable device. As such, I am working within strict space constraints. What are some non-essential components of solr that can be cut to conserve space for more information? For basic functionality, the entire contrib directory could probably be removed. That's more than half of the download right there. Some of the jars in solr-webapp/webapp/WEB-INF/lib can likely be removed. Chances are that you won't need the jars starting with "hadoop" - those are for HDFS support. That's another 11 MB. If you don't need either HDFS or SolrCloud, you can remove the zookeeper jar, and I think you can also remove the curator jars. If you're not accessing Solr with a JDBC driver, you won't need the calcite jars. If you're not dealing with oriental characters (and sometimes even if you ARE), you can probably do without lucene-analyzers-kuromoji. With careful code analysis, you can probably find other jars that aren't needed, but there's not a huge amount of space saving to be gained with most of the others. Thanks, Shawn
Re: UI fields signification
Hi Alex, Thanks for replying. I also found this: http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201404.mbox/%3c53483062.2000...@elyograg.org%3E where it says "s basically means that Lucene has detected an index state where something has made changes to the index, but those changes are not yet visible. To make them visible and return this status to 'true', do a commit or soft commit with openSearcher enabled." That is interesting because my "current" flag almost never turns green, and indeed my openSearcher is disabled... So I'm a bit lost about what it should be ad if having "current" always red is really an issue or not... :-/ JMS Le mer. 19 sept. 2018 à 14:44, Alexandre Rafalovitch a écrit : > I would say this is the relevant page for the "current" and > "generation" > https://lucene.apache.org/solr/guide/6_6/index-replication.html > > And I think generation refers to the actual Lucene index, so is > explained further here: > > https://lucene.apache.org/core/6_6_0/core/org/apache/lucene/codecs/lucene62/package-summary.html#package.description > > Regards, >Alex. > > On 19 September 2018 at 14:33, Jean-Marc Spaggiari > wrote: > > Hi dear SOLR community. > > > > On this page of the documentation: > > https://lucene.apache.org/solr/guide/6_6/core-specific-tools.html what > are > > the fields "current" and "gen" referring to? I have not been able to find > > that anywhere :( > > > > Thanks, > > > > JMS >
RE: sharding and placement of replicas
I am still wondering whether anyone has ever seen any examples of this actually working (has anyone ever seen any example of SPLITSHARD on a two-node SolrCloud placing replicas of the each shard on different hosts than other replicas of the same shards)? Anyone? -Original Message- From: Oakley, Craig (NIH/NLM/NCBI) [C] Sent: Friday, August 10, 2018 12:54 PM To: solr-user@lucene.apache.org Subject: RE: sharding and placement of replicas Note that I usually create collections with commands which contain (for example) solr/admin/collections?action=CREATE&name=collectest&collection.configName=collectest&numShards=1&replicationFactor=1&createNodeSet= I give one node in the createNodeSet and then ADDREPLICA to the other node. In case this were related, I now tried it a different way, using a command which contains solr/admin/collections?action=CREATE&name=collectest5&collection.configName=collectest&numShards=1&replicationFactor=2&createNodeSet= I gave both nodes in the createNodeSet in this case. It created one replica on each node (each node being on a different host at the same port). This is what I would consider the expected behavior (refraining from putting two replicas of the same one shard on the same node) After this I ran a command including solr/admin/collections?action=SPLITSHARD&collection=collectest5&shard=shard1&indent=on&async=test20180810h The result was still the same: one of the four new shards was on one node and the other three were all together on the node from which I issued this command (including putting two replicas of the same shard on the same node). I am wondering whether there are any examples of this actually working (any examples of SPLITSHARD occasionally placing replicas of the each shard on different hosts than other replicas of the same shards) -Original Message- From: Oakley, Craig (NIH/NLM/NCBI) [C] [mailto:craig.oak...@nih.gov] Sent: Thursday, August 09, 2018 5:08 PM To: solr-user@lucene.apache.org Subject: RE: sharding and placement of replicas Okay, I've tried again with two nodes running Solr7.4 on different hosts. Before SPLITSHARD, collectest2_shard1_replica_n1 was on the host nosqltest22, and collectest2_shard1_replica_n3 was on the host nosqltest11 After running SPLITSHARD (on the nosqltest22 node), only collectest2_shard1_0_replica0 was added to nosqltest11; nosqltest22 became the location for collectest2_shard1_0_replica_n5 and collectest2_shard1_1_replica_n6 and collectest2_shard1_1_replica0 (and so if nosqltest22 were to be down, shard1_1 would not be available). -Original Message- From: Erick Erickson [mailto:erickerick...@gmail.com] Sent: Tuesday, July 31, 2018 5:16 PM To: solr-user Subject: Re: sharding and placement of replicas Right, two JVMs on the same physical host with different ports are "different Solrs" by default. If you had two replicas per shard and both were on either Solr instance (same port) that would be unexpected. Problem is that this would have been a bug clear back in the Solr 4x days so the fact that you say you saw it on 6.6 would be unexpected. Of course if you have three replicas and two instances, I'd absolutely expect that two replicas would be on one of them for each shard. Best, Erick On Tue, Jul 31, 2018 at 12:24 PM, Oakley, Craig (NIH/NLM/NCBI) [C] wrote: > In my case, when trying on Solr7.4 (in response to Shawn Heisey's 6/19/18 > comment "If this is a provable and reproducible bug, and it's still a problem > in the current stable branch"), I had only installed Solr7.4 on one host, and > so I was testing with two nodes on the same host (different port numbers). I > had previously had the same symptom when the two nodes were on different > hosts, but that was with Solr6.6 -- I can try it again with Solr7.4 with two > hosts and report back. > > -Original Message- > From: Shawn Heisey [mailto:apa...@elyograg.org] > Sent: Tuesday, July 31, 2018 2:26 PM > To: solr-user@lucene.apache.org > Subject: Re: sharding and placement of replicas > > On 7/27/2018 8:26 PM, Erick Erickson wrote: >> Yes with some fiddling as far as "placement rules", start here: >> https://lucene.apache.org/solr/guide/6_6/rule-based-replica-placement.html >> >> The idea (IIUC) is that you provide a snitch" that identifies what >> "rack" the Solr instance is on and can define placement rules that >> define "don't put more than one thingy on the same rack". "Thingy" >> here is replica, shard, whatever as defined by other placement rules. > > I'd like to see an improvement in Solr's behavior when nothing has been > configured in auto-scaling or rule-based replica placement. Configuring > those things is certainly an option, but I think we can do better even > without that config. > > I believe that Solr already has some default intelligence that keeps > multiple replicas from ending up on the same *node* when possible ... I > would like this to also be aware of *hosts*. > > Craig
Re: local "q.op=AND" ignored for edismax query
Thanks, Shawn.
We made a change to add q.op=AND as a separate param and found a few issues.
For example, we have a query that filters out guest users in our product.
It boils down to:
select?q=myname*&q.op=AND&fq=(-(site_role:"Guest"))
debugQuery shows this is parsed as the following, which does not work:
+(-(+site_role:Guest))
Previously, because we were setting q.op=AND as a local param, i.e.
"q={!q.op=AND}myname*", my understanding is the effective operator for the
fq param was OR. So the fq param parsed as:
-site_role:Guest
I feel like we can fix the issue by just removing the outer parenthesis.
But I haven't been able to figure out why the operator change (OR to AND)
has this effect.
Any insight appreciated!
--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Re: Implementing NeuralNetworkModel RankNet in Solr LTR
Hi Edwin, > Just to check, is this supported in Solr 7.4.0? Yes, it is. https://github.com/LTR4L/ltr4l/blob/master/ltr4l-solr/ivy-jars.properties#L17 Koji On 2018/09/19 19:40, Zheng Lin Edwin Yeo wrote: Hi Koji, Thanks for your reply and provide the information. Just to check, is this supported in Solr 7.4.0? Regards, Edwin On Wed, 19 Sep 2018 at 11:02, Koji Sekiguchi wrote: Hi, > https://github.com/airalcorn2/Solr-LTR#RankNet > > Has anyone tried on this before? And what is the format of the training > data that this model requires? I haven't tried it, but I'd like to inform you that there is another project of LTR we've been developed: https://github.com/LTR4L/ltr4l It has many LTR algorithms based on neural network, SVM and boosting. Koji On 2018/09/12 11:44, Zheng Lin Edwin Yeo wrote: Hi, I am working on to implementing Solr LTR in Solr 7.4.0 by using the NeuralNetworkModel for the feature selection and model training, and I have found this site which uses RankNet: https://github.com/airalcorn2/Solr-LTR#RankNet Has anyone tried on this before? And what is the format of the training data that this model requires? Regards, Edwin
Re: Implementing NeuralNetworkModel RankNet in Solr LTR
Ok, thank you. Regards, Edwin On Thu, 20 Sep 2018 at 08:39, Koji Sekiguchi wrote: > Hi Edwin, > > > Just to check, is this supported in Solr 7.4.0? > > Yes, it is. > > > https://github.com/LTR4L/ltr4l/blob/master/ltr4l-solr/ivy-jars.properties#L17 > > Koji > > On 2018/09/19 19:40, Zheng Lin Edwin Yeo wrote: > > Hi Koji, > > > > Thanks for your reply and provide the information. > > Just to check, is this supported in Solr 7.4.0? > > > > Regards, > > Edwin > > > > On Wed, 19 Sep 2018 at 11:02, Koji Sekiguchi < > koji.sekigu...@rondhuit.com> > > wrote: > > > >> Hi, > >> > >> > https://github.com/airalcorn2/Solr-LTR#RankNet > >> > > >> > Has anyone tried on this before? And what is the format of the > training > >> > data that this model requires? > >> > >> I haven't tried it, but I'd like to inform you that there is another > >> project of LTR we've been > >> developed: > >> > >> https://github.com/LTR4L/ltr4l > >> > >> It has many LTR algorithms based on neural network, SVM and boosting. > >> > >> Koji > >> > >> On 2018/09/12 11:44, Zheng Lin Edwin Yeo wrote: > >>> Hi, > >>> > >>> I am working on to implementing Solr LTR in Solr 7.4.0 by using the > >>> NeuralNetworkModel for the feature selection and model training, and I > >> have > >>> found this site which uses RankNet: > >>> https://github.com/airalcorn2/Solr-LTR#RankNet > >>> > >>> Has anyone tried on this before? And what is the format of the training > >>> data that this model requires? > >>> > >>> Regards, > >>> Edwin > >>> > >> > > >
Re: Is that solr supports multi version operations?
Yeah~, writing to true data store first, then write to solr. I found it is simple to guarantee the finally consistency, only handling two main exception bellow: 1. If failed to write to true data store，then client samply retry its request。 2. If write true data store success，and failed to write to solr， it will retry to solr infinitely。 If write to solr is failed，and server was kill，I can use the transaction log of the true data store to replay and write to solr again。 Shawn Heisey 于2018年9月19日周三 下午10:38写道： > On 9/18/2018 8:11 PM, zhenyuan wei wrote: > > Hi all, > > add solr document with overwrite=false will keepping multi version > > documents， > > My question is ： > > 1. How to search newest documents？with what options？ > > 2. How to delete old version < newest version documents? > > When Solr is compiling results, it will only return one copy of a > particular document (based on uniqueKey value).All other copies will be > removed. > > I suspect (but do not know for sure) that which document will be > returned is not defined. On a multi-shard index, if different copies > are in different shards, which one is returned will be decided by which > shard answers the query first, or maybe which one answers last. If > multiple copies exist in the same core, that's probably more > deterministic, but it might not be the copy you wanted. > > Solr isn't designed to have multiple versions of the same uniqueKey in > the index. Lucene itself doesn't care -- it's going to return all of > them -- but if you want to be sure which one is returned, you'd need to > write the Lucene-based software yourself instead of using Solr. > > As you mentioned in the last message, writing to your true data store > and then writing to Solr if that succeeds is a better option. Or you > could simply write to your data store and then have your indexing > software detect and read the new records from there. > > Thanks, > Shawn > >
Re: SOLR 7.0 DIH out of memory issue with sqlserver
Tanya: Good to hear. You probably want to configure hard as well, and in your case perhaps with openSearcher=true Indexing is only half the problem. It's quite possible that what's happening is your index is just growing and that's pushing the boundaries of Java heap. What I'm thinking is that DIH may be importing small batches, but as more docs are added the underlying index grows by enough to hit an OOM with batch N+1. You could just move the indexing off to a client which would give you a way to see which it is (index growth or jdbc config). Here's a sample: https://lucidworks.com/2012/02/14/indexing-with-solrj/ Best, Erick On Wed, Sep 19, 2018 at 1:22 PM Tanya Bompi wrote: > > Hi Erick, > Thank you for the follow-up. I have resolved the issue with the increase > in heapSize and I am able to set the SOLR VM to initialize with a 3G heap > size and the subset of 1 mil records was fetched successfully. Although it > fails with the entire 3 mil records. So something is off with the adaptive > buffering setting as I see it is not helping. And I also set the > autosoftcommit param. I might have to increase the heap size further to see > if it helps. I will keep posted if my issue doesnt resolve. > > Thanks, > Tanya > > On Wed, Sep 19, 2018 at 8:22 AM Erick Erickson > wrote: > > > Has this ever worked? IOW, is this something that's changed or has > > just never worked? > > > > The obvious first step is to start Solr with more than 1G of memory. > > Solr _likes_ memory and a 1G heap is quite small. But you say: > > "Increasing the heap size further doesnt start SOLR instance itself.". > > How much RAM do you have on your machine? What other programs are > > running? You should be able to increase the heap and start Solr if you > > have the RAM on your machine so I'd figure out what's behind that > > issue first. I regularly start Solr with 16 or 32G of memory on my > > local machines, I know of installations running Solr with 60G heaps so > > this points to something really odd about your environment. > > > > When you "increase it further", exactly _how_ does Solr fail to start? > > What appears in the Solr logs? etc. Really, troubleshoot that issue > > first I'd recommend. > > > > If DIH still needs a ridiculous amount of memory, it's usually the > > JDBC driver trying to read all the rows into memory at once and you'll > > have to explore the jdbc driver settings in detail. > > > > Best, > > Erick > > On Tue, Sep 18, 2018 at 11:16 PM Tanya Bompi > > wrote: > > > > > > Hi, > > > I am using the Microsoft Jdbc driver 6.4 version in Solr 7.4.0 . I have > > > tried removing the selectMethod=Cursor and still it runs out of heap > > space. > > > Do we have anyone who has faced similar issue. > > > > > > Thanks > > > Tanya > > > > > > > > > On Tue, Sep 18, 2018 at 6:38 PM Shawn Heisey > > wrote: > > > > > > > On 9/18/2018 4:48 PM, Tanya Bompi wrote: > > > > >I have the SOLR 7.0 setup with the DataImportHandler connecting > > to the > > > > > sql server db. I keep getting OutOfMemory: Java Heap Space when > > doing a > > > > > full import. The size of the records is around 3 million so not very > > > > huge. > > > > > I tried the following steps and nothing helped thus far. > > > > > > > > See this wiki page: > > > > > > > > https://wiki.apache.org/solr/DataImportHandlerFaq > > > > > > > > You already have the suggested fix -- setting responseBuffering to > > > > adaptive. You might try upgrading the driver. If that doesn't work, > > > > you're probably going to need to talk to Microsoft about what you need > > > > to do differently on the JDBC url. > > > > > > > > I did find this page: > > > > > > > > > > > > > > https://docs.microsoft.com/en-us/sql/connect/jdbc/using-adaptive-buffering?view=sql-server-2017 > > > > > > > > This says that when using adaptive buffering, you should avoid using > > > > selectMethod=cursor. So you should try removing that parameter. > > > > > > > > Thanks, > > > > Shawn > > > > > > > > > >
SolrCoreInitializationException after restart of one solr node
Hi Guys
I have a 3 zookeeper ensemble and 3 solr nodes running version 7.4.0.
Recently I had to restart one node and after I did that it started throwing
this exception.
{
"error":{
"metadata":[
"error-class","org.apache.solr.core.SolrCoreInitializationException",
"root-error-class","org.apache.solr.common.SolrException"],
"msg":"SolrCore 'articles' is not available due to init failure: ",
"trace":"org.apache.solr.core.SolrCoreInitializationException: SolrCore
'articles' is not available due to init failure: \n\tat
org.apache.solr.core.CoreContainer.getCore(CoreContainer.java:1590)\n\tat
org.apache.solr.servlet.HttpSolrCall.init(HttpSolrCall.java:249)\n\tat
org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:469)\n\tat
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:377)\n\tat
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:323)\n\tat
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1634)\n\tat
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)\n\tat
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)\n\tat
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)\n\tat
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)\n\tat
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)\n\tat
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)\n\tat
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)\n\tat
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1253)\n\tat
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)\n\tat
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)\n\tat
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)\n\tat
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)\n\tat
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1155)\n\tat
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)\n\tat
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:219)\n\tat
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:126)\n\tat
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)\n\tat
org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335)\n\tat
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)\n\tat
org.eclipse.jetty.server.Server.handle(Server.java:531)\n\tat
org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:352)\n\tat
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260)\n\tat
org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:281)\n\tat
org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)\n\tat
org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118)\n\tat
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)\n\tat
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)\n\tat
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)\n\tat
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)\n\tat
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)\n\tat
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:760)\n\tat
org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:678)\n\tat
java.lang.Thread.run(Thread.java:748)\nCaused by:
org.apache.solr.common.SolrException: \n\tat
org.apache.solr.cloud.ZkController.preRegister(ZkController.java:1685)\n\tat
org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1128)\n\tat
org.apache.solr.core.CoreContainer.lambda$load$13(CoreContainer.java:681)\n\tat
com.codahale.metrics.InstrumentedExecutorService$InstrumentedCallable.call(InstrumentedExecutorService.java:197)\n\tat
java.util.concurrent.FutureTask.run(FutureTask.java:266)\n\tat
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209)\n\tat
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)\n\tat
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)\n\t...
1 more\nCaused by: org.apache.solr.common.SolrException: No coreNodeName
for
CoreDescriptor[name=articles;instanceDir=/opt/solr/server/solr/configsets/articles]\n\tat
org.apache.solr.cloud.ZkController.checkStateInZk(ZkController.java:1716)\n\tat
org.apache.solr.cloud.ZkController.preRegister(ZkController.java:1654)\n\t...
8 more\n",
"code":500}}
I tried dropping all
Re: SolrCoreInitializationException after restart of one solr node
On 9/19/2018 8:22 PM, Schaum Mallik wrote: I have a 3 zookeeper ensemble and 3 solr nodes running version 7.4.0. Recently I had to restart one node and after I did that it started throwing this exception. Caused by: org.apache.solr.common.SolrException: No coreNodeName for CoreDescriptor[name=articles;instanceDir=/opt/solr/server/solr/configsets/articles] It is VERY weird for instanceDir to be under the configsets directory. Especially for SolrCloud. I know you're in cloud mode because the ZK-related classes are heavily mentioned in the stacktrace. Can you share the entire solr.log file after a restart? If that directory really is the instanceDir, there should be a core.properties file. What are its contents? Did you use the service installer script? If you did, having an instanceDir underneath configsets is even MORE strange. Thanks, Shawn
Re: SolrCoreInitializationException after restart of one solr node
The data and index get stored under /opt/solr/server/solr/articles_shard1_replica_n1. The config directory when the collection was created, that time the path to the config was given as '/opt/solr/server/solr/configsets/articles'. I didn't use the service installer script. The other two solr nodes are working without any issue. Any ideas how I can resolve this. Below is the solr.log after restart 2018-09-20 03:01:32.853 INFO (main) [ ] o.e.j.u.log Logging initialized @950ms to org.eclipse.jetty.util.log.Slf4jLog 2018-09-20 03:01:33.015 INFO (main) [ ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 1.8.0_171-b11 2018-09-20 03:01:33.044 INFO (main) [ ] o.e.j.d.p.ScanningAppProvider Deployment monitor [file:///opt/solr/server/contexts/] at interval 0 2018-09-20 03:01:33.334 INFO (main) [ ] o.e.j.w.StandardDescriptorProcessor NO JSP Support for /solr, did not find org.apache.jasper.servlet.JspServlet 2018-09-20 03:01:33.343 INFO (main) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0 2018-09-20 03:01:33.343 INFO (main) [ ] o.e.j.s.session No SessionScavenger set, using defaults 2018-09-20 03:01:33.345 INFO (main) [ ] o.e.j.s.session node0 Scavenging every 66ms 2018-09-20 03:01:33.400 INFO (main) [ ] o.a.s.u.c.SSLCredentialProviderFactory Processing SSL Credential Provider chain: env;sysprop 2018-09-20 03:01:33.436 INFO (main) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory 2018-09-20 03:01:33.443 INFO (main) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.4.0 2018-09-20 03:01:33.447 INFO (main) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port 8983 2018-09-20 03:01:33.447 INFO (main) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: /opt/solr 2018-09-20 03:01:33.448 INFO (main) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|Start time: 2018-09-20T03:01:33.448Z 2018-09-20 03:01:33.476 INFO (main) [ ] o.a.s.c.SolrResourceLoader Using system property solr.solr.home: /opt/solr/server/solr 2018-09-20 03:01:33.551 INFO (zkConnectionManagerCallback-2-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected 2018-09-20 03:01:33.558 INFO (main) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper) 2018-09-20 03:01:33.561 INFO (main) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /opt/solr/server/solr/solr.xml 2018-09-20 03:01:33.648 INFO (main) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@15d9bc04, but no JMX reporters were configured - adding default JMX reporter. 2018-09-20 03:01:34.479 INFO (main) [ ] o.a.s.c.ZkContainer Zookeeper client=zook1.local:2181,zook2.local:2182,zook3.local:2183 2018-09-20 03:01:34.503 INFO (zkConnectionManagerCallback-6-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected 2018-09-20 03:01:34.603 WARN (main) [ ] o.a.s.c.ZkController The _default configset could not be uploaded. Please provide 'solr.default.confdir' parameter that points to a configset intended to be the default. Current 'solr.default.confdir' value: /opt/solr/server/solr/configsets/_default/conf 2018-09-20 03:01:34.608 INFO (main) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2) 2018-09-20 03:01:34.636 INFO (main) [ ] o.a.s.c.Overseer Overseer (id=null) closing 2018-09-20 03:01:34.652 INFO (main) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores 2018-09-20 03:01:34.652 INFO (main) [ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/solr1:8983_solr 2018-09-20 03:01:34.663 INFO (zkCallback-5-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3) 2018-09-20 03:01:34.666 INFO (main) [ ] o.a.s.c.CoreContainer Initializing authorization plugin: solr.RuleBasedAuthorizationPlugin 2018-09-20 03:01:34.675 INFO (main) [ ] o.a.s.c.CoreContainer Initializing authentication plugin: solr.BasicAuthPlugin 2018-09-20 03:01:34.680 INFO (main) [ ] o.a.s.c.CoreContainer PKIAuthenticationPlugin is managing internode requests 2018-09-20 03:01:34.844 INFO (zkConnectionManagerCallback-13-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected 2018-09-20 03:01:34.846 INFO (main) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3) 2018-09-20 03:01:34.849 INFO (main) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at zook1.local:2181,zook2.local:2182,zook3.local:2183 ready 2018-09-20 03:01:34.870 INFO (main) [ ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory. 2018-09-20 03:01:34.940 INFO (main) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com
Re: SolrCoreInitializationException after restart of one solr node
I also want to add one other things. I had moved from a single core solr instance on solr 6.6 to the solr cloud few months back. I had ran the indexupgrader tool on the indexes before I moved them to the solr cloud. On Wed, Sep 19, 2018 at 7:29 PM Shawn Heisey wrote: > On 9/19/2018 8:22 PM, Schaum Mallik wrote: > > I have a 3 zookeeper ensemble and 3 solr nodes running version 7.4.0. > > Recently I had to restart one node and after I did that it started > throwing > > this exception. > > > Caused by: org.apache.solr.common.SolrException: No coreNodeName > > for > > > CoreDescriptor[name=articles;instanceDir=/opt/solr/server/solr/configsets/articles] > > It is VERY weird for instanceDir to be under the configsets directory. > Especially for SolrCloud. I know you're in cloud mode because the > ZK-related classes are heavily mentioned in the stacktrace. > > Can you share the entire solr.log file after a restart? If that > directory really is the instanceDir, there should be a core.properties > file. What are its contents? > > Did you use the service installer script? If you did, having an > instanceDir underneath configsets is even MORE strange. > > Thanks, > Shawn > >
