Re: Delete from Solr console fails
Thanks Dominique! I just tried deleting a single document using its id. I tried this: id123 and this: id:id123 In each case, I still get the same "Solr connection lost" error. I checked that the Solr instance has enough RAM (it was using 73% of the RAM), but it was using 110% CPU. Could this be a CPU under-allocation problem (the Solr container has 4 cores allocated to it)? Thanks Goutham On Fri, Sep 25, 2020 at 7:41 PM Dominique Bejean wrote: > Hi Goutham, > > I agree with Rahul, avoid large deletebyquery. > It you can, prefere one query to get all the ids first than use ids with > deletebyid > > Regards > > Dominique > > > Le ven. 25 sept. 2020 à 06:50, Goutham Tholpadi a > écrit : > > > I spoke too soon. I am getting the "Connection lost" error again. > > > > I have never faced this problem when there are a small number of docs in > > the index. I was wondering if the size of the index (30M docs) has > anything > > to do with this. > > > > Thanks > > Goutham > > > > On Fri, Sep 25, 2020 at 9:55 AM Goutham Tholpadi > > wrote: > > > > > Thanks for your response Rahul! > > > > > > Yes, all the fields I tried with were indexed=true, but it did not > work. > > > > > > Btw, when I try to today, I am no longer getting the "Connection lost" > > > error. The delete command returns with status=success, however the > > document > > > is not actually deleted when I check in the search console again. > > > > > > I tried using Document Type as XML just now and I see the same > behaviour > > > as above. > > > > > > Thanks > > > Goutham > > > > > > On Fri, Sep 25, 2020 at 7:17 AM Rahul Goswami > > > wrote: > > > > > >> Goutham, > > >> Is the field you are trying to delete by indexed=true in the schema ? > > >> If the uniqueKey is indexed=true, does delete by id work for you? > > >> ( uniqueKey:value) > > >> Also, instead of "Solr Command" if you choose the Document type as > > "XML" > > >> does it make any difference? > > >> > > >> Rahul > > >> > > >> On Thu, Sep 24, 2020 at 1:04 PM Goutham Tholpadi > > > >> wrote: > > >> > > >> > Hi, > > >> > > > >> > Setup: > > >> > We have a stand-alone Solr (v7.2) with around 30 million documents > and > > >> with > > >> > 4 cores, 38G of RAM, and a 1TB disk. The documents were not directly > > >> > indexed but came from a restore of a back from another Solr > instance. > > >> > > > >> > Problem: > > >> > Search queries seem to be working fine. However, when I try to > delete > > >> > documents from the Solr console, I get a "Connection to Solr lost" > > >> error. I > > >> > am trying by navigating to the "Documents" section of the chosen > core, > > >> > using "Solr Command" as the "Document Type", and entering something > > >> this in > > >> > the box below: > > >> > > > >> > > > >> > field:value > > >> > > > >> > > > >> > > > >> > I tried with the field being the unique key, and otherwise. I also > > tried > > >> > with values containing wild cards. I got the error in all cases. > > >> > > > >> > Any pointers on this? > > >> > > > >> > Thanks > > >> > Goutham > > >> > > > >> > > > > > >
Re: Delete from Solr console fails
You mention high CPU usage...Can you share the thread dump (using jstack) for both the delete by id and delete by query? Also, an output of /solr//schema executed on the host? Lastly, is this standalone Solr or SolrCloud? Attachments won’t make it to the list, so I would recommend sharing a link to any file sharing service. On a side note, I have observed the UI timing out requests after a certain point even though the actual request is still being processed. In case something like that is happening here, did you try the delete by id as an HTTP request through a curl or Postman? Having said that I would still expect delete by id to execute in reasonable time, so I would start by looking at what is s eating up the CPU in your request. -Rahul On Sat, Sep 26, 2020 at 4:50 AM Goutham Tholpadi wrote: > Thanks Dominique! I just tried deleting a single document using its id. I > > tried this: > > > > id123 > > > > > > and this: > > > > id:id123 > > > > > > In each case, I still get the same "Solr connection lost" error. I checked > > that the Solr instance has enough RAM (it was using 73% of the RAM), but it > > was using 110% CPU. Could this be a CPU under-allocation problem (the Solr > > container has 4 cores allocated to it)? > > > > Thanks > > Goutham > > > > On Fri, Sep 25, 2020 at 7:41 PM Dominique Bejean < > dominique.bej...@eolya.fr> > > wrote: > > > > > Hi Goutham, > > > > > > I agree with Rahul, avoid large deletebyquery. > > > It you can, prefere one query to get all the ids first than use ids with > > > deletebyid > > > > > > Regards > > > > > > Dominique > > > > > > > > > Le ven. 25 sept. 2020 à 06:50, Goutham Tholpadi a > > > écrit : > > > > > > > I spoke too soon. I am getting the "Connection lost" error again. > > > > > > > > I have never faced this problem when there are a small number of docs > in > > > > the index. I was wondering if the size of the index (30M docs) has > > > anything > > > > to do with this. > > > > > > > > Thanks > > > > Goutham > > > > > > > > On Fri, Sep 25, 2020 at 9:55 AM Goutham Tholpadi > > > > wrote: > > > > > > > > > Thanks for your response Rahul! > > > > > > > > > > Yes, all the fields I tried with were indexed=true, but it did not > > > work. > > > > > > > > > > Btw, when I try to today, I am no longer getting the "Connection > lost" > > > > > error. The delete command returns with status=success, however the > > > > document > > > > > is not actually deleted when I check in the search console again. > > > > > > > > > > I tried using Document Type as XML just now and I see the same > > > behaviour > > > > > as above. > > > > > > > > > > Thanks > > > > > Goutham > > > > > > > > > > On Fri, Sep 25, 2020 at 7:17 AM Rahul Goswami > > > > > > wrote: > > > > > > > > > >> Goutham, > > > > >> Is the field you are trying to delete by indexed=true in the schema > ? > > > > >> If the uniqueKey is indexed=true, does delete by id work for you? > > > > >> ( uniqueKey:value) > > > > >> Also, instead of "Solr Command" if you choose the Document type as > > > > "XML" > > > > >> does it make any difference? > > > > >> > > > > >> Rahul > > > > >> > > > > >> On Thu, Sep 24, 2020 at 1:04 PM Goutham Tholpadi < > gtholp...@gmail.com > > > > > > > > >> wrote: > > > > >> > > > > >> > Hi, > > > > >> > > > > > >> > Setup: > > > > >> > We have a stand-alone Solr (v7.2) with around 30 million documents > > > and > > > > >> with > > > > >> > 4 cores, 38G of RAM, and a 1TB disk. The documents were not > directly > > > > >> > indexed but came from a restore of a back from another Solr > > > instance. > > > > >> > > > > > >> > Problem: > > > > >> > Search queries seem to be working fine. However, when I try to > > > delete > > > > >> > documents from the Solr console, I get a "Connection to Solr lost" > > > > >> error. I > > > > >> > am trying by navigating to the "Documents" section of the chosen > > > core, > > > > >> > using "Solr Command" as the "Document Type", and entering > something > > > > >> this in > > > > >> > the box below: > > > > >> > > > > > >> > > > > > >> > field:value > > > > >> > > > > > >> > > > > > >> > > > > > >> > I tried with the field being the unique key, and otherwise. I also > > > > tried > > > > >> > with values containing wild cards. I got the error in all cases. > > > > >> > > > > > >> > Any pointers on this? > > > > >> > > > > > >> > Thanks > > > > >> > Goutham > > > > >> > > > > > >> > > > > > > > > > > > > > >
Re: solr performance with >1 NUMAs
Thanks Shawn! Currently we are still using the CMS collector for solr with Java 8. When last evaluated with Solr 7, CMS performs better than G1 for our case. When using G1, is it better to upgrade from Java 8 to Java 11? >From https://lucene.apache.org/solr/guide/8_4/solr-system-requirements.html, seems Java 14 is not officially supported for Solr 8. Best, Wei On Fri, Sep 25, 2020 at 5:50 PM Shawn Heisey wrote: > On 9/23/2020 7:42 PM, Wei wrote: > > Recently we deployed solr 8.4.1 on a batch of new servers with 2 NUMAs. I > > noticed that query latency almost doubled compared to deployment on > single > > NUMA machines. Not sure what's causing the huge difference. Is there any > > tuning to boost the performance on multiple NUMA machines? Any pointer is > > appreciated. > > If you're running with standard options, Solr 8.4.1 will start using the > G1 garbage collector. > > As of Java 14, G1 has gained the ability to use the -XX:+UseNUMA option, > which makes better decisions about memory allocations and multiple > NUMAs. If you're running a new enough Java, it would probably be > beneficial to add this to the garbage collector options. Solr itself is > unaware of things like NUMA -- Java must handle that. > > https://openjdk.java.net/jeps/345 > > Thanks, > Shawn >
RE: Solr 8.6.2 UI issue
Thanks Alexandre. It was cache issue. Regards Manisha Rahatadkar -Original Message- From: Alexandre Rafalovitch Sent: Friday, September 25, 2020 6:12 PM To: solr-user Subject: Re: Solr 8.6.2 UI issue Sounds strange. If you had Solr installed previously, it could be cached Javascript. Force-reload or try doing it in an anonymous window. Also try starting with an example (solr/start -e techproducts). Finally, if you are up to it, see if there are any serious errors in the Browser's developer console's log. If all else fails, try an earlier version of Solr, just to check whether it could be something about the latest version (unlikely). Regards, Alex On Fri, 25 Sep 2020 at 18:43, Manisha Rahatadkar wrote: > > Hello All > > I downloaded 8.6.2 and running it on windows 10 machine. Solr starts on 8983 > port but whenever I click on any menu like Logging, Core Admin, Query it > always shows only dashboard screen. > Has anyone experienced this issue? > > > Regards > Manisha Rahatadkar > > Confidentiality Notice > > This email message, including any attachments, is for the sole use of the > intended recipient and may contain confidential and privileged information. > Any unauthorized view, use, disclosure or distribution is prohibited. If you > are not the intended recipient, please contact the sender by reply email and > destroy all copies of the original message. Anju Software, Inc. 4500 S. > Lakeshore Drive, Suite 620, Tempe, AZ USA 85282. Confidentiality Notice This email message, including any attachments, is for the sole use of the intended recipient and may contain confidential and privileged information. Any unauthorized view, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message. Anju Software, Inc. 4500 S. Lakeshore Drive, Suite 620, Tempe, AZ USA 85282.
What does current mean?
Hello, Solr community: I would like to ask a question about the current icon on the core-overview under statistics. I thought previously that the current tag tells users whether it is searchable or not (commit or not commit) because if I send a commit request, it becomes an OK-ish icon from NG-ish icon. If anyone knows the meaning of the icon, I would like to hear about. -- Sincerely, Kaya github: https://github.com/28kayak
Re: What does current mean?
My understanding is that current means whether there is data pending to be committed. Best, Wei On Sat, Sep 26, 2020 at 5:09 PM Kayak28 wrote: > Hello, Solr community: > > > > I would like to ask a question about the current icon on the core-overview > > under statistics. > > I thought previously that the current tag tells users whether it is > > searchable or not (commit or not commit) because if I send a > > commit request, it becomes an OK-ish icon from NG-ish icon. > > > > If anyone knows the meaning of the icon, I would like to hear about. > > > > > > > > > > > > -- > > > > Sincerely, > > Kaya > > github: https://github.com/28kayak > >
Re: solr performance with >1 NUMAs
On 9/26/2020 1:39 PM, Wei wrote: Thanks Shawn! Currently we are still using the CMS collector for solr with Java 8. When last evaluated with Solr 7, CMS performs better than G1 for our case. When using G1, is it better to upgrade from Java 8 to Java 11? From https://lucene.apache.org/solr/guide/8_4/solr-system-requirements.html, seems Java 14 is not officially supported for Solr 8. It has been a while since I was working with Solr every day, and when I was, Java 11 did not yet exist. I have no idea whether Java 11 improves things beyond Java 8. That said ... all software evolves and usually improves as time goes by. It is likely that the newer version has SOME benefit. Regarding whether or not Java 14 is supported: There are automated tests where all the important code branches are run with all major versions of Java, including pre-release versions, and those tests do include various garbage collectors. Somebody notices when a combination doesn't work, and big problems with newer Java versions are something that gets discussed on our mailing lists. Java 14 has been out for a while, with no big problems being discussed so far. So it is likely that it works with Solr. Can I say for sure? No. I haven't tried it myself. I don't have any hardware available where there is more than one NUMA, or I would look deeper into this myself. It would be interesting to find out whether the -XX:+UseNUMA option makes a big difference in performance. Thanks, Shawn
