Re: omitNorms for short searchable fields and ID field

2017-08-25 Thread Atita Arora
Hi Chaula, Omitnorms are basically used for index time boost & field length normalization saying that I meant when you do omitNorms=true for any field it stops storing additional stats regarding terms , length , boosts etc for that field and hence drastically reduces the size of index. It infact i

Re: edismax, pf2 and use of both AND and OR parameter

2017-08-25 Thread Atita Arora
Hi, I am in the middle of the similar use case as provided , we have three different fields on UI for searchany, searchall and searchexcept respectively for OR,AND and NOT query , I need to know how do I make them work along with Edismax. We can expect any/all of the fields to have free text. Any

Re: SPLITSHARD in SOLR 5.5.1

2017-08-25 Thread Vannia Rajan
Hi, I figured out when this issue happens, by observing the patterns with a small set of data. SPLITSHARD issues a soft-commit (with some of the files still not fully written to disk). If I restart SOLR without issuing an explicit , the index directory is not fully written and the process is kill

Fwd: Machine Learning for search

2017-08-25 Thread Joel Bernstein
I forgot to include the users list in my response below: --- Interesting. I've been meaning to test the classifier in a similar way but haven't had the time. Basically what you did is created two classes: 1) A positive class 2) A very noisy negative class of "other stuff" It was unc

Re: Custom StoredFieldVisitor in Solr

2017-08-25 Thread Jamie Johnson
Hi Rick. The use case is we use payloads to determine if a particular user can or can't see a field, as of right now we have the query piece working so that fields the user can't see don't contribute to the score but we wanted to use a custom stored field visitor as well so that we can remove fiel

Search by similarity?

2017-08-25 Thread Darko Todoric
Hi, I have 90.000.000 documents in Solr and I need to compare "title" of this document and get all documents with more than 80% similarity. PHP have "similar_text" but it's not so smart inserting 90m documents in the array... Can I do some query in Solr which will give me the more the 80% si

RE: Search by similarity?

2017-08-25 Thread Junte Zhang
If you already have the title of the document, then you could run that title as a new query against the whole index and exclude the source document from the results as a filter. You could use the DisMax query parser: https://cwiki.apache.org/confluence/display/solr/The+DisMax+Query+Parser And

RE: Search by similarity?

2017-08-25 Thread Markus Jelsma
Yes, that is roughly how MLT works as well. You can also do a full OR-search on the terms using LuceneQParser. Markus -Original message- > From:Junte Zhang > Sent: Friday 25th August 2017 18:38 > To: solr-user@lucene.apache.org > Subject: RE: Search by similarity? > > If you alread

Re: autoSoftCommit doesn't work as expected / documented

2017-08-25 Thread Shawn Heisey
On 8/24/2017 4:59 AM, Angel Todorov wrote: > I also tested, of course, by setting a value of 0, expecting that it would > work in the way I expect it to , but unfortunately - it doesn't. Nothing is > committed in that case. Settings of zero turn that part of the automatic commit off.  That's a way

solr 6.6.1: Lock held by this virtual machine

2017-08-25 Thread Nawab Zada Asad Iqbal
Hi, I am getting this error. i have deleted the file and restarted the server, but this error doesn't go away. What should I do to fix it? Caused by: org.apache.solr.common.SolrException: Error opening new searcher at org.apache.solr.core.SolrCore.(SolrCore.java:977) at org.apa

Re: solr 6.6.1: Lock held by this virtual machine

2017-08-25 Thread Nawab Zada Asad Iqbal
Ok, after looking at the logs for some more time, i found that there are more than one threads trying to load the core at startup time. This doesn't make sense to me, is it configurable? Is there any reason why this is even an option? Aug 25, 2017 12:04:37 PM INFO (main) [ ] o.e.j.s.Server jet

Re: solr 6.6.1: Lock held by this virtual machine

2017-08-25 Thread Erick Erickson
In that case you probably have two different cores pointing to the _same_ data directory. Examine your core.properties files and see if any dataDir variables are set Best, Erick On Fri, Aug 25, 2017 at 1:12 PM, Nawab Zada Asad Iqbal wrote: > Ok, after looking at the logs for some more time,

Re: autoSoftCommit doesn't work as expected / documented

2017-08-25 Thread Erick Erickson
You might find this useful: https://lucidworks.com/2013/08/23/understanding-transaction-logs-softcommit-and-commit-in-sorlcloud/ bq: So I can never have soft auto commit after each update ? This sounds like a bug to me. No, rather a fundamental misunderstanding of the difference between an ACID

Re: What is the org.apache.solr.uninverting.FieldCacheImpl?

2017-08-25 Thread Erick Erickson
You need to enable docValues on the field (and completely reindex). The standard inverted index structure is great for answering "for term X in field Y, what docs does it appear in?". It's rotten for the "uninverted" case: "For doc X, what is the value of field Y?". This latter question is the one

Re: write.lock file appears and solr wont open

2017-08-25 Thread Erick Erickson
Odd. The way core discovery works, it starts at SOLR_HOME and recursively descends the directories. Whenever the recursion finds a "core.properties" file it says "Aha, this must be a core". From there it assumes the data directory is immediately below where it found the core.properties file in the

Re: Error when using IndexMergeTool

2017-08-25 Thread Erick Erickson
You classpath is not correct. Unless it's a typo, :"C:solr/server/solr-webapp/webapp/WEB-INF/lib/lucene-misc-6.5.1.jar" is wrong, it should be: "C:/solr/server/solr-webapp/webapp/WEB-INF/lib/lucene-misc-6.5.1.jar", note the slash before "solr". If that's not just a typo you need to look further as

Re: solr 6.6.1: Lock held by this virtual machine

2017-08-25 Thread Nawab Zada Asad Iqbal
Thanks Erik I expected that but it is really not the case . I have only one core per solr installation . Though i run 3 solr processes on each host. If you see the thread names they are :coreLoadExecutor and qtp761960786-31 . If it was the case of two core pointing to one index (though it does not

Re: Error when using IndexMergeTool

2017-08-25 Thread Zheng Lin Edwin Yeo
Hi Erick, Thanks for your reply. I have check, it was not due to the typo. The same error occurs even with the slash. Regards, Edwin On 26 August 2017 at 05:16, Erick Erickson wrote: > You classpath is not correct. Unless it's a typo, > :"C:solr/server/solr-webapp/webapp/WEB-INF/lib/lucene-mis

Re: Error when using IndexMergeTool

2017-08-25 Thread Leonardo Perez Pulido
Hi, Isn't the Windows classpath separator a semi-colon? Like this: cene-core-6.5.1.jar"*;*"C:solr/serve Just saying. Regards. On Fri, Aug 25, 2017 at 8:33 PM, Zheng Lin Edwin Yeo wrote: > Hi Erick, > > Thanks for your reply. > I have check, it was not due to the typo. The same error occurs even

Correct approach to copy index between solr clouds?

2017-08-25 Thread Wei
Hi, In our set up there are two solr clouds: Cloud A: production cloud serves both writes and reads Cloud B: back up cloud serves only writes Cloud A and B have the same shard configuration. Write requests are sent to both cloud A and B. In certain circumstances when Cloud A's update lags be

Re: solr 6.6.1: Lock held by this virtual machine

2017-08-25 Thread Nawab Zada Asad Iqbal
Hi Erick, I spent some more time on this and found that if I modify 'core.properties' to contain the following values (my core.propreties file is empty otherwise and only being used for shard discovery), then the solr server works fine. loadOnStartup=false transient=false The fact is that shards