RE: Query over migrating a solr database from 7.7.1 to 8.7.0

2021-02-01 Thread Flowerday, Matthew J
only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all devices. -Original Message- From: Flowerday, Matthew J Sent: 15 January 2021 11:18 To: solr-user@lucene.apache.org Subject: RE: Query over migrat

Re: Query is timing out.

2021-01-28 Thread Modassar Ather
Hi, The above boolean query works fine when the rows fetched are smaller like 10/20 but when it is increased to a bigger number it slows down. Is document collection very expensive? Is there any configuration I am missing? *Solr setup details:* Mode : SolrCloud Number of Shards : 12 Index size :

RE: Query over migrating a solr database from 7.7.1 to 8.7.0

2021-01-15 Thread Flowerday, Matthew J
Business Park | Wavendon | Milton Keynes | MK17 8LX <http://www.unisys.com/> THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is for use only by the intended recipient. If you received this in error, please contact the sender and delete the

RE: Query over migrating a solr database from 7.7.1 to 8.7.0

2021-01-13 Thread Dyer, Jim
I think if you have _root_ in schema.xml you should look elsewhere. My memory is merely adding this one line to schema.xml took care of our problem. From: Flowerday, Matthew J Sent: Tuesday, January 12, 2021 3:23 AM To: solr-user@lucene.apache.org Subject: RE: Query over migrating a solr

RE: Query over migrating a solr database from 7.7.1 to 8.7.0

2021-01-12 Thread Flowerday, Matthew J
Jim Sent: 11 January 2021 22:58 To: solr-user@lucene.apache.org Subject: RE: Query over migrating a solr database from 7.7.1 to 8.7.0 EXTERNAL EMAIL - Be cautious of all links and attachments. When we upgraded from 7.x to 8.x, I ran into an issue similar to yours: when updating an existing documen

RE: Query over migrating a solr database from 7.7.1 to 8.7.0

2021-01-11 Thread Dyer, Jim
eature was added for nested documents, this field somehow became mandatory in order for updates to work properly, at least in some cases. From: Flowerday, Matthew J Sent: Saturday, January 9, 2021 4:44 AM To: solr-user@lucene.apache.org Subject: RE: Query over migrating a solr database from 7.7.

Re: Query over migrating a solr database from 7.7.1 to 8.7.0

2021-01-09 Thread matthew sporleder
Did you commit? > On Jan 9, 2021, at 5:44 AM, Flowerday, Matthew J > wrote: > >  > Hi There > > As a test I stopped Solr and ran the IndexUpgrader tool on the database to > see if this might fix the issue. It completed OK but unfortunately the issue > still occurs – a new version of the re

RE: Query over migrating a solr database from 7.7.1 to 8.7.0

2021-01-09 Thread Flowerday, Matthew J
Hi There As a test I stopped Solr and ran the IndexUpgrader tool on the database to see if this might fix the issue. It completed OK but unfortunately the issue still occurs - a new version of the record on solr is created rather than updating the original record. It looks to me as if the r

Re: Query generation is different for search terms with and without "-"

2020-11-25 Thread Walter Underwood
Ages ago at Netflix, I fixed this with a few hundred synonyms. If you are working with a fixed vocabulary (movie titles, product names), that can work just fine. babysitter, baby-sitter, baby sitter fullmetal, full-metal, full metal manhunter, man-hunter, man hunter spiderman, spider-man, spider

Re: Query generation is different for search terms with and without "-"

2020-11-25 Thread Erick Erickson
Parameters, no. You could use a PatternReplaceCharFilterFactory. NOTE: *FilterFactory are _not_ what you want in this case, they are applied to individual tokens after parsing *CharFiterFactory are invoked on the entire input to the field, although I can’t say for certain that even that’s early

Re: Query generation is different for search terms with and without "-"

2020-11-24 Thread Samuel Gutierrez
Are there any good workarounds/parameters we can use to fix this so it doesn't have to be solved client side? On Tue, Nov 24, 2020 at 7:50 AM matthew sporleder wrote: > Is the normal/standard solution here to regex remove the '-'s and > combine them into a single token? > > On Tue, Nov 24, 2020

Re: Query generation is different for search terms with and without "-"

2020-11-24 Thread matthew sporleder
Is the normal/standard solution here to regex remove the '-'s and combine them into a single token? On Tue, Nov 24, 2020 at 8:00 AM Erick Erickson wrote: > > This is a common point of confusion. There are two phases for creating a > query, > query _parsing_ first, then the analysis chain for the

Re: Query generation is different for search terms with and without "-"

2020-11-24 Thread Erick Erickson
This is a common point of confusion. There are two phases for creating a query, query _parsing_ first, then the analysis chain for the parsed result. So what e-dismax sees in the two cases is: Name_enUS:“high tech” -> two tokens, since there are two of them pf2 comes into play. Name_enUS:“high-

Re: Query in quotes cannot find results

2020-07-11 Thread Shawn Heisey
On 6/30/2020 12:07 PM, Permakoff, Vadim wrote: Regarding removing the stopwords, I agree, there are many cases when you don't want to remove the stopwords, but there is one very compelling case when you want them to be removed. Imagine, you have one document with the following text: 1. "to exp

RE: Query in quotes cannot find results

2020-06-30 Thread Permakoff, Vadim
Thank you Walter, I'll look into “mm” (minimum match) parameter. Best Regards, Vadim Permakoff -Original Message- From: Walter Underwood Sent: Tuesday, June 30, 2020 2:31 PM To: solr-user@lucene.apache.org Subject: Re: Query in quotes cannot find results This is exactly why th

Re: Query in quotes cannot find results

2020-06-30 Thread Walter Underwood
m Permakoff > > > -----Original Message- > From: Erick Erickson > Sent: Tuesday, June 30, 2020 12:15 PM > To: solr-user@lucene.apache.org > Subject: Re: Query in quotes cannot find results > > Well, the first thing is that you haven’t include FlattenGraphFilterFactory > in t

RE: Query in quotes cannot find results

2020-06-30 Thread Permakoff, Vadim
uot; Best Regards, Vadim Permakoff -Original Message- From: Walter Underwood Sent: Tuesday, June 30, 2020 12:50 PM To: solr-user@lucene.apache.org Subject: Re: Query in quotes cannot find results Removing stopwords is a dumb requirement. “Doctor, it hurts when I shove hedgehogs up my a

RE: Query in quotes cannot find results

2020-06-30 Thread Permakoff, Vadim
ay, June 30, 2020 12:15 PM To: solr-user@lucene.apache.org Subject: Re: Query in quotes cannot find results Well, the first thing is that you haven’t include FlattenGraphFilterFactory in the index analysis chain, see: https://urldefense.proofpoint.com/v2/url?u=https-3A__lucene.apache.org_sol

Re: Query in quotes cannot find results

2020-06-30 Thread Walter Underwood
d. > Any thoughts? > > Best Regards, > Vadim Permakoff > > > -Original Message- > From: Erick Erickson > Sent: Monday, June 29, 2020 10:19 PM > To: solr-user@lucene.apache.org > Subject: Re: Query in quotes cannot find results > > Looks like you’re removi

Re: Query in quotes cannot find results

2020-06-30 Thread Erick Erickson
querying > path, but the exact match result still cannot be found! This is weird. > Any thoughts? > > Best Regards, > Vadim Permakoff > > > -Original Message- > From: Erick Erickson > Sent: Monday, June 29, 2020 10:19 PM > To: solr-user@lucene.apache.o

RE: Query in quotes cannot find results

2020-06-30 Thread Permakoff, Vadim
al Message- From: Erick Erickson Sent: Monday, June 29, 2020 10:19 PM To: solr-user@lucene.apache.org Subject: Re: Query in quotes cannot find results Looks like you’re removing stopwords. Stopwords cause issues like this with the positions being off. It’s becoming more and more common to _N

Re: Query in quotes cannot find results

2020-06-29 Thread Erick Erickson
dex and > the query, the order is also correct, but the word "methods" in moved one > position, I guess that's why the result is not found. > > Best Regards, > Vadim Permakoff > > > > > -Original Message- > From: Shawn Heisey > Sent: Monday, June 29, 20

RE: Query in quotes cannot find results

2020-06-29 Thread Permakoff, Vadim
in moved one position, I guess that's why the result is not found. Best Regards, Vadim Permakoff -----Original Message- From: Shawn Heisey Sent: Monday, June 29, 2020 6:28 PM To: solr-user@lucene.apache.org Subject: Re: Query in quotes cannot find results On 6/29/2020 3:34 PM, Perm

Re: Query in quotes cannot find results

2020-06-29 Thread Shawn Heisey
On 6/29/2020 3:34 PM, Permakoff, Vadim wrote: The basic query q=expand the methods <<< finds the document, the query (in quotes) q="expand the methods" <<< cannot find the document Am I doing something wrong, or is it known bug (I saw similar issues discussed in the past, but not for exact

Re: Query takes more time in Solr 8.5.1 compare to 6.1.0 version

2020-06-22 Thread vishal patel
Is there any other option? Sent from Outlook<http://aka.ms/weboutlook> From: Mikhail Khludnev Sent: Sunday, May 24, 2020 3:24 AM To: solr-user Subject: Re: Query takes more time in Solr 8.5.1 compare to 6.1.0 version Unfortunately {!terms} doesn

Re: Query takes more time in Solr 8.5.1 compare to 6.1.0 version

2020-05-23 Thread Mikhail Khludnev
ery? > > Regards, > Vishal Patel > ________ > From: Jason Gerlowski > Sent: Friday, May 22, 2020 2:59 AM > To: solr-user@lucene.apache.org > Subject: Re: Query takes more time in Solr 8.5.1 compare to 6.1.0 version > > Hi Jay, > > I can&#x

Re: Query takes more time in Solr 8.5.1 compare to 6.1.0 version

2020-05-23 Thread vishal patel
w I got 30 seconds for above query. How can I use the "terms" query parser in my query? Regards, Vishal Patel From: Jason Gerlowski Sent: Friday, May 22, 2020 2:59 AM To: solr-user@lucene.apache.org Subject: Re: Query takes more time in Solr 8.5.1 compa

Re: Query takes more time in Solr 8.5.1 compare to 6.1.0 version

2020-05-21 Thread Jason Gerlowski
is no issue related to memory. > > >Maybe also you need to increase the corresponding caches in the config. > We are not using cache in both version. > > Both version have same configuration. > > Regards, > Jay Harkhani. > > > From: Jörn

Re: Query takes more time in Solr 8.5.1 compare to 6.1.0 version

2020-05-21 Thread jay harkhani
ay, May 21, 2020 7:05 PM To: solr-user@lucene.apache.org Subject: Re: Query takes more time in Solr 8.5.1 compare to 6.1.0 version Did you create Solrconfig.xml for the collection from scratch after upgrading and reindexing? Was it based on the latest template? If not then please try this. M

Re: Query takes more time in Solr 8.5.1 compare to 6.1.0 version

2020-05-21 Thread Jörn Franke
Did you create Solrconfig.xml for the collection from scratch after upgrading and reindexing? Was it based on the latest template? If not then please try this. Maybe also you need to increase the corresponding caches in the config. What happens if you reexecute the query? Are there other proces

Re: Query takes more time in Solr 8.5.1 compare to 6.1.0 version

2020-05-21 Thread vishal patel
Any one is looking this issue? I got same issue. Regards, Vishal Patel From: jay harkhani Sent: Wednesday, May 20, 2020 7:39 PM To: solr-user@lucene.apache.org Subject: Query takes more time in Solr 8.5.1 compare to 6.1.0 version Hello, Currently I upgrade S

Re: Query confusion - solr cloud 8.2.0

2020-04-08 Thread Erick Erickson
The easiest way to answer questions like this is an under-appreciated parameter “explainOther” when submitted with “debug=true”. It’ll return an explanation of how the doc identified by the “explainOther” parameter was scored. See: https://lucene.apache.org/solr/guide/8_1/common-query-parameters

Re: Query is taking a time in Solr 6.1.0

2020-03-15 Thread vishal patel
How can I use the tokenizing differently? Sent from Outlook<http://aka.ms/weboutlook> From: Erik Hatcher Sent: Friday, March 13, 2020 6:20 PM To: solr-user@lucene.apache.org Subject: Re: Query is taking a time in Solr 6.1.0 Looks like you have two, maybe

Re: Query is taking a time in Solr 6.1.0

2020-03-13 Thread Erik Hatcher
Looks like you have two, maybe three, wildcard/prefix clauses in there. Consider tokenizing differently so you can optimize the queries to not need wildcards - thats my first observation and suggestion. Erik > On Mar 13, 2020, at 05:56, vishal patel wrote: > > Some query is taking tim

Re: Re: Re: Re: Re: Query Autocomplete Evaluation

2020-02-28 Thread Audrey Lorberfeld - audrey.lorberf...@ibm.com
are given 0s) / total suggestions displayed ? > > If the above is true, wouldn't Selection to Display be binary? I.e. it's > either 1/# of suggestions displayed (assuming this is a constant) or 0? > > Best, > Audrey > > > __

Re: Re: Re: Re: Query Autocomplete Evaluation

2020-02-28 Thread Paras Lehana
suggestions displayed (assuming this is a constant) or 0? > > Best, > Audrey > > > > From: Paras Lehana > Sent: Thursday, February 27, 2020 2:58:25 AM > To: solr-user@lucene.apache.org > Subject: [EXTERNAL] Re: Re: Re: Query Autocomplete Eval

Re: Re: Re: Re: Query Autocomplete Evaluation

2020-02-27 Thread Audrey Lorberfeld - audrey.lorberf...@ibm.com
, wouldn't Selection to Display be binary? I.e. it's either 1/# of suggestions displayed (assuming this is a constant) or 0? Best, Audrey From: Paras Lehana Sent: Thursday, February 27, 2020 2:58:25 AM To: solr-user@lucene.apache.org Subject: [EXTERNA

Re: Re: Re: Query Autocomplete Evaluation

2020-02-26 Thread Paras Lehana
Hi Audrey, For MRR, we assume that if a suggestion is selected, it's relevant. It's also assumed that the user will always click the highest relevant suggestion. Thus, we calculate position selection for each selection. If still, I'm not understanding your question correctly, feel free to contact

Re: Re: Re: Query Autocomplete Evaluation

2020-02-25 Thread Audrey Lorberfeld - audrey.lorberf...@ibm.com
This article http://wwwconference.org/proceedings/www2011/proceedings/p107.pdf also indicates that MRR needs binary relevance labels, p. 114: "To this end, we selected a random sample of 198 (query, context) pairs from the set of 7,311 pairs, and manually tagged each of them as related (i.e., th

Re: Re: Query Autocomplete Evaluation

2020-02-25 Thread Audrey Lorberfeld - audrey.lorberf...@ibm.com
Thank you, Walter & Paras! So, from the MRR equation, I was under the impression the suggestions all needed a binary label (0,1) indicating relevance.* But it's great to know that you guys use proxies for relevance, such as clicks. *The reason I think MRR has to have binary relevance labels is

Re: Query Autocomplete Evaluation

2020-02-24 Thread Walter Underwood
Here is a blog article with a worked example for MRR based on customer clicks. https://observer.wunderwood.org/2016/09/12/measuring-search-relevance-with-mrr/ At my place of work, we compare the CTR and MRR of queries using suggestions to those that do not use suggestions. Solr autosuggest based

Re: Re: Query Autocomplete Evaluation

2020-02-24 Thread Paras Lehana
Hey Audrey, I assume MRR is about the ranking of the intended suggestion. For this, no human judgement is required. We track position selection - the position (1-10) of the selected suggestion. For example, this is our recent numbers: Position 1 Selected (B3) 107,699 Position 2 Selected (B4) 58,7

Re: Re: Query Autocomplete Evaluation

2020-02-24 Thread Audrey Lorberfeld - audrey.lorberf...@ibm.com
Hi Paras, This is SO helpful, thank you. Quick question about your MRR metric -- do you have binary human judgements for your suggestions? If no, how do you label suggestions successful or not? Best, Audrey On 2/24/20, 2:27 AM, "Paras Lehana" wrote: Hi Audrey, I work for Auto-S

Re: Query Autocomplete Evaluation

2020-02-23 Thread Paras Lehana
Hi Audrey, I work for Auto-Suggest at IndiaMART. Although we don't use the Suggester component, I think you need evaluation metrics for Auto-Suggest as a business product and not specifically for Solr Suggester which is the backend. We use edismax parser with EdgeNGrams Tokenization. Every week,

Re: Query Regarding SOLR cross collection join

2020-01-29 Thread Mikhail Khludnev
It's time to enforce and document field type constraints https://issues.apache.org/jira/browse/SOLR-14230. On Mon, Jan 27, 2020 at 4:12 PM Doss wrote: > @ Alessandro Benedetti , Thanks for your input! > > @ Mikhail Khludnev , I made docValues="true" for from & to and did a index > rotation, now

Re: Query Regarding SOLR cross collection join

2020-01-27 Thread Doss
@ Alessandro Benedetti , Thanks for your input! @ Mikhail Khludnev , I made docValues="true" for from & to and did a index rotation, now the score join works perfectly! Saw 7x performance increase. Thanks! On Thu, Jan 23, 2020 at 9:53 PM Mikhail Khludnev wrote: > On Wed, Jan 22, 2020 at 4:27

Re: Query Regarding SOLR cross collection join

2020-01-23 Thread Mikhail Khludnev
On Wed, Jan 22, 2020 at 4:27 PM Doss wrote: > HI, > > SOLR version 8.3.1 (10 nodes), zookeeper ensemble (3 nodes) > > Read somewhere that the score join parser will be faster, but for me it > produces no results. I am using string type fields for from and to. > That's odd. Can you try to enable

Re: Query Regarding SOLR cross collection join

2020-01-23 Thread Alessandro Benedetti
>From the Join Query Parser code: "// most of these statistics are only used for the enum method int fromSetSize; // number of docs in the fromSet (that match the from query) long resultListDocs; // total number of docs collected int fromTermCount; long fromTermTotalDf; int fromTerm

Re: Query Containing Multiple Parsers

2019-12-17 Thread Chris Hostetter
: Is there a way to construct a query that needs two different parsers? : Example: : q={!xmlparser}Hello : AND : q={!edismax}text_en:"foo bar"~4 The easies way to do what you're asking about would be to choose one of those queries for "storking" purposes, and put the other one in an "fq" simpl

Re: Query More Than One Core

2019-11-13 Thread rhys J
On Wed, Nov 13, 2019 at 3:16 PM Jörn Franke wrote: > You can use nested indexing and Index both types of documents in one core. > > https://lucene.apache.org/solr/guide/8_1/indexing-nested-documents.html I had read that, but it doesn't really fit our needs right now. I figured out how to do a

Re: Query More Than One Core

2019-11-13 Thread Jörn Franke
You can use nested indexing and Index both types of documents in one core. https://lucene.apache.org/solr/guide/8_1/indexing-nested-documents.html However, what is the use case for Solr if you have already a database? > Am 13.11.2019 um 20:50 schrieb rhys J : > > I have more than one core. Eac

Re: Query regarding truncated Date Sort

2019-11-07 Thread Erick Erickson
The easiest and most efficient would be to store the date (or a copy) at day resolution and sort on that field instead. > On Nov 7, 2019, at 3:00 AM, Paras Lehana wrote: > > Hi Inderjeet, > > Wouldn't sorting on the default format will yield documents date-wise > sorted? The time won't impact

Re: Query regarding truncated Date Sort

2019-11-07 Thread Paras Lehana
Hi Inderjeet, Wouldn't sorting on the default format will yield documents date-wise sorted? The time won't impact the date order or do you have different timezones also? On Thu, 7 Nov 2019 at 12:52, Inderjeet Singh wrote: > Hi > > I am currently using solr 7.1.0. I have indexed a few documents

Re: Query regarding Path Hierarchy tokenizer

2019-11-04 Thread Erick Erickson
> But while querying those tokens, solr is not able to parse the > query. You need to show us exactly the query and the error message. At a guess, since you haven’t provided the information, you need to escape the colon since it’s part of the query syntax, i.e. something like q=field:c\:/usr B

Re: Query on stemming

2019-11-03 Thread Shubham Goswami
Hi Paras/Jorn Thanks for your support. I am using Solr version 7.5. I tried with SnowballPorterFilterFactory class and its working fine. Thanks for your help. On Fri, Nov 1, 2019 at 5:52 PM Paras Lehana wrote: > Hi Shubham, > > I am getting the ClassNotFoundException while indexing after makin

Re: Query on stemming

2019-11-01 Thread Paras Lehana
Hi Shubham, I am getting the ClassNotFoundException while indexing after making the > changes. - Please describe the error properly with full details. We need to see what is exactly causing ClassNotFoundException in the trace. - This seems to be a Java syntax error due to *EnglishP

Re: Query on stemming

2019-11-01 Thread Paras Lehana
Hey Jorn, Ok yes but I assumed that he is referring to a newer version as no version > was mentioned. You are right, this is probably the case! Shubham, have you just upgraded or is it the first time you are using the filter? Switch to Snowball and everything will be sorted. On Fri, 1 Nov 2019

Re: Query on stemming

2019-11-01 Thread Jörn Franke
Ok yes but I assumed that he is referring to a newer version as no version was mentioned > Am 01.11.2019 um 12:43 schrieb Paras Lehana : > > Hi Jorn, > > It looks to me that you write the name of the class wrongly. > > > *EnglishPorterFilterFactory* did exist in older Solr versions >

Re: Query on stemming

2019-11-01 Thread Paras Lehana
Hi Jorn, It looks to me that you write the name of the class wrongly. *EnglishPorterFilterFactory* did exist in older Solr versions . Since it was deprecated long ago, I think that's the reason it's no

Re: Query on stemming

2019-11-01 Thread Jörn Franke
https://lucene.apache.org/solr/guide/8_2/filter-descriptions.html It looks to me that you write the name of the class wrongly > Am 01.11.2019 um 11:12 schrieb Shubham Goswami : > > Hi Jorn > > Thanks for your response. > Following is my field type definition and i am getting the > ClassNotFou

Re: Query on stemming

2019-11-01 Thread Shubham Goswami
Hi Jorn Thanks for your response. Following is my field type definition and i am getting the ClassNotFoundException while indexing after making the changes. On Fri, Nov 1, 2019 at 2:10 PM Jörn Franke wrote: > Ho

Re: Query on stemming

2019-11-01 Thread Jörn Franke
How did you define the field type? Probably you have syntax errors there. I recommend to use the schema rest api instead of schema xml as it will give you better feedback on what is wrong and it allows you also better versioning of the schema in a source code repository. https://lucene.apache.o

Re: Query on changing FieldType

2019-10-25 Thread Shubham Goswami
Hello Erick/Emir Thanks for your valuable suggestions. I will it keep in mind while doing such operations. Best, Shubham On Wed, Oct 23, 2019 at 5:56 PM Erick Erickson wrote: > Really, just don’t do this. Please. As others have pointed out, it may > look like it works, but it won’t. I’ve spent

Re: Query on changing FieldType

2019-10-23 Thread Erick Erickson
Really, just don’t do this. Please. As others have pointed out, it may look like it works, but it won’t. I’ve spent many hours tracking down why clients got weird errors after making changes like this, sometimes weeks later. Or more accurately, if you choose to change field types without reindex

Re: Query on changing FieldType

2019-10-23 Thread Emir Arnautović
Hi Shubham, My guess that it might be working for text because it uses o.toString() so there are no runtime errors while in case of others, it has to assume some class so it does class casting. You can check in logs what sort of error happens. But in any case, like Jason pointed out, that is a p

Re: Query on changing FieldType

2019-10-22 Thread Shubham Goswami
Hi Jason Thanks for the response. You are right that re-indexing is required after making any changes to Schema even i am re-indexing the docs in which i have changed the fieldtypes, but here Emir is talking about full re-indexing i.e. deleting the existing/core and creating new one that is time c

Re: Query on changing FieldType

2019-10-22 Thread Jason Gerlowski
Hi Shubbham, Emir gave you accurate advice - you cannot (safely) change field types without reindexing. You may avoid errors for a time, and searches may even return the results you expect. But the type-change is still a ticking time bomb...Solr might try to merge segments down the road or do so

Re: Query on changing FieldType

2019-10-22 Thread Shubham Goswami
Hi Emir As you have mentioned above we cannot change field type after indexing once and we have to do dull re-indexing again, I tried to change field type from plong to pint which has implemented class solr.LongPointField and solr.IntPointField respectively and it was showing error as expected.

Re: Query on changing FieldType

2019-10-22 Thread Emir Arnautović
Hi Shubham, No you cannot. What you can do is to use copy field or update request processor to store is as some other field and use that in your query and ignore the old one that will eventually disappear as the result of segment merges. HTH, Emir -- Monitoring - Log Management - Alerting - Anom

Re: Query on changing FieldType

2019-10-22 Thread Shubham Goswami
Hi Emir Thanks for the reply, i got your point. But is there any other way to do like one field could have two or more different types defined ? or if i talk about my previous query, can we index some data for the same field with different unique id after replacing the type ? Thanks again Shubha

Re: Query on changing FieldType

2019-10-22 Thread Emir Arnautović
Hi Shubham, Changing type is not allowed without full reindexing. If you do something like that, Solr will end up with segments with different types for the same field. Remember that segments are immutable and that reindexing some document will be in new segment, but old segment will still be th

Re: Query regarding positionIncrementGap

2019-10-18 Thread Paras Lehana
Hi Shubham, In other words, *you specify a large positionIncrementGap to make sure that your queries don't match across multiple values of a field*. For example, for a query like title:"paper plate making machine", you don't want it to match with doc having two values for title:"paper plate", "ma

Re: Query regarding positionIncrementGap

2019-10-18 Thread Erick Erickson
I really don’t understand the question. The field has to be multiValued, but there’s no other restriction. It’s all about whether a document you input has the same field name specified more than once, i.e. is multiValued. That’s why the example I gave has sentence one sentence two sentenc

Re: Query regarding positionIncrementGap

2019-10-17 Thread Shubham Goswami
Hi Erick Thanks for reply and your example is very helpful. But i think we can only use this attribute if we are getting data from a single field which has the copy of all data from every field. Please correct me if i am wrong. Thanks for your great support. Shubham On Thu, Oct 17, 2019 at 5:56

Re: Query regarding positionIncrementGap

2019-10-17 Thread Erick Erickson
First, it only counts if you add multiple entries for the field. Consider the following a b c def where the field has a positionIncrementGap of 100. The term positions of the entries are a:1 b:2 c:3 d:103 e:104 f:105 Now consider the doc where there’s only one field: a b c d e f T

Re: Query on autoGeneratePhraseQueries

2019-10-16 Thread Shubham Goswami
Hi Michael/Shawn Thanks for the response. Michael you are right, autoGeneratePhraseQueries works for the query like Black-company with the setting of Sow=true. Thanks for your great support. Best Shubham On Wed, Oct 16, 2019 at 9:22 PM Michael Gibney wrote: > Going to back to the initial quest

Re: Query on autoGeneratePhraseQueries

2019-10-16 Thread Michael Gibney
Going to back to the initial question, the wording is a little ambiguous and it occurs to me that it's possible there's a misunderstanding of what autoGeneratePhraseQueries does. It really only auto-generates phrase *subqueries*. To use the example from the initial request, a query like (black comp

Re: Query on autoGeneratePhraseQueries

2019-10-16 Thread Shawn Heisey
On 10/16/2019 7:14 AM, Shubham Goswami wrote: I have implemented the sow=false property with eDismax Query parser but still it does not has any effect on the query as it is still parsing as separate terms instead of phrased one. We have seen reports that when sow=false, which is the default set

Re: Re: Query on autoGeneratePhraseQueries

2019-10-16 Thread Shubham Goswami
Hi Rohan/Audrey I have implemented the sow=false property with eDismax Query parser but still it does not has any effect on the query as it is still parsing as separate terms instead of phrased one. On Tue, Oct 15, 2019 at 8:25 PM Rohan Kasat wrote: > Also check , > pf , pf2 , pf3 > ps , ps2, p

Re: Query related APACHE SOLR 8.2.0

2019-10-16 Thread sasarun
Hi Rohit, Solr bundle comes with a Jetty server by default and does not require a tomcat instance to run. Even though earlier version of Solr was in the form of war file, Solr 5.0 and higher versions no longer supports user defined containers. Details of the same are available in the link below f

Re: Re: Query on autoGeneratePhraseQueries

2019-10-15 Thread Rohan Kasat
Also check , pf , pf2 , pf3 ps , ps2, ps3 parameters for phrase searches. Regards, Rohan K On Tue, Oct 15, 2019 at 6:41 AM Audrey Lorberfeld - audrey.lorberf...@ibm.com wrote: > I'm not sure how your config file is setup, but I know that the way we do > multi-token synonyms is to have the sow (

Re: Re: Query on autoGeneratePhraseQueries

2019-10-15 Thread Audrey Lorberfeld - audrey.lorberf...@ibm.com
I'm not sure how your config file is setup, but I know that the way we do multi-token synonyms is to have the sow (split on whitespace) parameter set to False while using the edismax parser. I'm not sure if this would work with PhraseQueries , but it might be worth a try! In our config file we

Re: Query on autoGeneratePhraseQueries

2019-10-15 Thread Shubham Goswami
Hi kshitij Thanks for the reply! I tried to debug it and found that raw query(black company) has parsed as two separate queries black and company and returning the results based on black query instead of this it should have got parsed as a single phrase query like("black company") because i am usi

Re: Query on autoGeneratePhraseQueries

2019-10-15 Thread kshitij tyagi
Hi, Try debugging your solr query and understand how it gets parsed. Try using "debug=true" for the same On Tue, Oct 15, 2019 at 12:58 PM Shubham Goswami wrote: > *Hi all,* > > I am a beginner to solr framework and I am trying to implement > *autoGeneratePhraseQueries* property in a fieldtype o

Re: Query terms and the match state

2019-09-08 Thread Scott Stults
Lucene has a SynonymQuery and a BlendedTermQuery that do something like you want in different ways. However, if you want to keep your existing schema and do this through Solr you can use the constant score syntax in edismax on each term: q=name:(corsair)^=1.0 name:(ddr)^=1.0 manu:(corsair)^=1.0 ma

Re: Query field alias - issue with circular reference

2019-09-08 Thread David Smiley
No but this seems like a decent enhancement request. ~ David Smiley Apache Lucene/Solr Search Developer http://www.linkedin.com/in/davidwsmiley On Fri, Aug 9, 2019 at 3:07 AM Jaroslaw Rozanski wrote: > Hi Folks, > > > > Question about query field aliases. > > > > Assuming one has fields: > >

Re: Query regarding Solr Cloud Setup

2019-09-06 Thread Erick Erickson
Ok, you can set it as a sysvar when starting solr. Or you can change your solrconfig.xml to either use classic schema (schema.xml) or take out the add-unknown-fields... from the update processor chain. You can also set a cluster property IIRC. Better to use one of the supported options... On Fri,

RE: Query regarding Solr Cloud Setup

2019-09-06 Thread Porritt, Ian
Hi Jörn/Erick/Shawn thanks for your responses. @Jörn - much apprecaited for the heads up on Kerberos authentication its something we havent really considered at the moment, more production this may well be the case. With regards to the Solr Nodes 3 is something we are looking as a minimum, when

Re: Query regarding Solr Cloud Setup

2019-09-03 Thread Jörn Franke
If you have a properly secured cluster eg with Kerberos then you should not update files in ZK directly. Use the corresponding Solr REST interfaces then you also less likely to mess something up. If you want to have HA you should have at least 3 Solr nodes and replicate the collection to all t

Re: Query regarding Solr Cloud Setup

2019-09-03 Thread Erick Erickson
Having custom core.properties files is “fraught”. First of all, that file can be re-written. Second, the collections ADDREPLICA command will create a new core.properties file. Third, any mistakes you make when hand-editing the file can have grave consequences. What change exactly do you want to

Re: Query regarding Solr Cloud Setup

2019-09-03 Thread Shawn Heisey
On 9/3/2019 7:22 AM, Porritt, Ian wrote: We have a schema which I have managed to upload to Zookeeper along with the Solrconfig, how do I get the system to recognise both a lib/.jar extension and a custom core.properties file? I bypassed the issue of the core.properties by amending the update.a

Re: Query-time synonyms without indexing

2019-08-29 Thread Erick Erickson
Ah, thanks for letting us know. Erick > On Aug 29, 2019, at 9:20 AM, Bjarke Buur Mortensen > wrote: > > The section without type is the one getting picked up for the > index-time chain, so that wasn't my problem. > > It turns out that because of > https://issues.apache.org/jira/browse/LUCEN

Re: Query-time synonyms without indexing

2019-08-29 Thread Bjarke Buur Mortensen
The section without type is the one getting picked up for the index-time chain, so that wasn't my problem. It turns out that because of https://issues.apache.org/jira/browse/LUCENE-8134, I needed to add a omitTermFreqAndPositions="true" to the declaration. This has to do with defaults for a stri

Re: Query-time synonyms without indexing

2019-08-28 Thread Erick Erickson
Not sure. You have an section and section. Frankly I’m not sure which one will be used for the index-time chain. Why don’t you just try it? change to reload and go. It’d take you 5 minutes and you’d have your answer. Best, Erick > On Aug 28, 2019, at 1:57 AM, Bjarke Buur Mortensen > w

Re: Query-time synonyms without indexing

2019-08-27 Thread Bjarke Buur Mortensen
Yes, but isn't that what I am already doing in this case (look at the fieldType in the original mail)? Is there some other way to specify that field type and achieve what I want? Thanks, Bjarke On Tue, Aug 27, 2019, 17:32 Erick Erickson wrote: > You can have separate index and query time analys

Re: Query-time synonyms without indexing

2019-08-27 Thread Erick Erickson
You can have separate index and query time analysis chains, there are many examples in the stock Solr schemas. Best, Erick > On Aug 27, 2019, at 8:48 AM, Bjarke Buur Mortensen > wrote: > > We have a solr file of type "string". > It turns out that we need to do synonym expansion on query time

Re: Query number of Lucene documents using Solr?

2019-08-27 Thread Erick Erickson
Bram: If you optimize (Solr 7.4 and earlier), that may be part of the “stuff” as an index with a single segment can accumulate far more deleted documents. Shot in the dark. See: https://lucidworks.com/post/segment-merging-deleted-documents-optimize-may-bad/ Plus the linked article to how Solr

Re: Query number of Lucene documents using Solr?

2019-08-27 Thread Bram Van Dam
On 26/08/2019 23:12, Shawn Heisey wrote: > The numbers shown in Solr's LukeRequestHandler come directly from > Lucene.  This is the URL endpoint it will normally be at, for core XXX: > > http://host:port/solr/XXX/admin/luke Thanks Shawn, that's a great entry point! > The specific error you encou

Re: Query number of Lucene documents using Solr?

2019-08-26 Thread Bernd Fehling
You might use the Lucene internal CheckIndex included in lucene core. It should tell you everything you need. At least a good starting point for writing your own tool. Copy lucene-core-x.y.z-SNAPSHOT.jar and lucene-misc-x.y.z-SNAPSHOT.jar to a local directory. java -cp lucene-core-x.y.z-SNAPSHOT

Re: Query number of Lucene documents using Solr?

2019-08-26 Thread Shawn Heisey
On 8/26/2019 2:19 PM, Bram Van Dam wrote: Possibly somewhat unusual question: I'm looking for a way to query the number of *lucene documents* from within Solr. This can be different from the number of Solr documents (because of unmerged deletes/updates/ etc). As a bit of background; we recently

  1   2   3   4   5   6   7   8   9   10   >