Re: Solr6.6 Issue/Bug
Why would you need to start Solr as root? You should definitely not do this, there's no reason for that. And even if you *really* want this: What's so bad about the -force option? -Michael Am 06.09.2017 um 07:26 schrieb Kasim Jinwala: > Dear team, > I am using solr 5.0 last 1 year, now we are planning to upgrade > solr 6.6. > While trying to start solr using root user, we need to pass -force > parameter to start solr forcefully, > please help to start solr using root user without -force command. > > Regards > Kasim J. >
Re: Phrase boosting on multiple fields
Just pass qf=field1,field2,field3&pf=field1,firld2,field3 Like that you can add as many field as you want On Wed, 6 Sep 2017 at 2:41 AM, ritesh kumar wrote: > I have a situation where I have to apply phrase boosting on multiple > fields. I am using Edismax as query parser. > For instance, I am accepting a keyword (phrase) from the user and I want > the doc with the exact phrase to be on the top of the resultant list. The > problem is, there are multiple fields to be queried upon. > I could find some assistance from the code below but again it works on a > single field only. > > q=java+design+patterns&defType=edismax&qf=name&pf2=name^30&ps=0 > > Any kind of suggestion would be helpful. > > Best, > > Ritesh Kumar > > >
Streaming expression API innerJoin on multi-valued field
Dear SOLR users, I want to use streaming expression innerJoin using a multi-valued field to do the join by equality, that is having any child documents (of type "child") and one parent document (of type "parent") join these according to equality of id_s and children_ids Parent * id_s = "p123" * type_s = "parent" * children_ids_ss = "c1,c2" Child * id_s = "c1" * type_s = "child" Child * id_s = "c2" * type_s = "child" innerJoin( search(collection,q="type_s:child",fl="id_s",sort="id_s ASC"), search(collection,q="type_s:parent",fl="id_s,children_ids_ss",sort="id_s ASC"), on="id_s=children_ids_ss" ) This seems to be impossible, I am getting the following exception "java.util.ArrayList cannot be cast to java.lang.Comparable". Using a GraphQuery with from and to this relationship traversal along multi-valued fields worked (however not between shards, this is why I switched to streaming expressions). Is there any mechanism to flatten the tuples with the multi-valued field into new tuples with single-valued fields to get the join working ? Or any other tweak. Note: The relationship between Parent and Child is many-to-many, thus moving the foreign-keys to the children as single-valued fields is not possible. The issue is related tot he following issue: http://lucene.472066.n3.nabble.com/Using-multi-valued-field-in-solr-cloud-Graph-Traversal-Query-td4324379.html Thanks a lot in advance for any assistance, Marc Dr. Marc Röttig Software Developer EMail: marc.roet...@vico-research.com Telefon: +49(0)711. 78 78 29-290 Fax +49(0)711. 78 78 29-10 VICO Research & Consulting GmbH Friedrich-List-Strasse 46 / 70771 Leinfelden-Echterdingen Homepage: www.vico-research.com/ Blog: www.vico-research.com/expert-talk Twitter: www.twitter.com/vico_news Facebook: www.facebook.com/vico.friend Sitz der Gesellschaft: Leinfelden-Echterdingen Amtsgericht Stuttgart, HRB 720896 Geschäftsführer: Marc Trömel
facet query when using word 'AND'
Hi,all
I use facet query,but I found it dose not work when using 'AND'.
I woud like to use facet query using 'AND' as not Operator but simple word.
At first,Solr Config is as below.
suggest_dict
solr.Suggester
AnalyzingLookupFactory
suggest
suggest_ja
true
true
text_ja_romaji
suggest
AND
0
true
true
suggest
1000
1
true
suggest_dict
10
true
30
10
true
suggest_ja
I executed the query below,but Solr gave unexpected result.
$ curl
"http://localhost:8983/solr/kms/suggest_ja?wt=json&indent=true&q=\"AND\"&facet=true";
{
"response":{"numFound":0,"start":0,"maxScore":0.0,"docs":[]
},
"facet_counts":{
"facet_queries":{},
"facet_fields":{
"suggest":[]},
"facet_ranges":{},
"facet_intervals":{},
"facet_heatmaps":{}},
"spellcheck":{
"suggestions":[],
"collations":[]}}
I'd like to use facet search including the word "AND",so I surrounded "AND"
by
double quotes and then appended the escappe parameter befoe
dobule quote as below.
\"AND\"
The Document included word "AND"(I have a pen AND an apple) is already
indexed.The evidence is as below.
$ curl "http://localhost:8983/solr/kms/select?wt=json&indent=true&q=\"AND\"";
{
"responseHeader":{
"zkConnected":true,
"status":0,
"QTime":9,
"params":{
"q":"\"AND\"",
"indent":"true",
"wt":"json"}},
"response":{"numFound":1,"start":0,"maxScore":0.2770272,"docs":[
{
"pub_date":"2017-03-09T12:34:56.789Z",
"body":"I have a pen AND an apple",
"title":"test",
"url":"http://10.16.44.180:8080/#/management/two/";,
"system_cd":"hoge",
"document_id":"001",
"id":"hoge001",
"content_type":"doc",
"_version_":157862221496320}]
}}
Therefore I expected the result as below.
{
"response":{"numFound":1,"start":0,"maxScore":0.66747504,"docs":[]
},
"facet_counts":{
"facet_queries":{},
"facet_fields":{
"suggest":[
"AND",1,
"apple",1,
"pen",1,
]},
"facet_ranges":{},
"facet_intervals":{},
"facet_heatmaps":{}},
"spellcheck":{
"suggestions":[],
"collations":[]}}
Actually facet fields includes nothing.
How do I solve this?
--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Solr list operator
Hi, I have a custom field "listOfIDs" = "1,2,4,33" I want the equivalent of: select * where '1' IN (listOfIDs) --> should get a match select * where '33' IN (listOfIDs) --> should get a match select * where '3' IN (listOfIDs) --> should NOT get a match Can anyone help me out please as I can't seem to find any documentation on this. Thanks very much in advance. Kind regards, Nick Way
Re: Solr list operator
https://www.youtube.com/watch?v=pNe1wWeaHOU&list=PLYI8318YYdkCsZ7dsYV01n6TZhXA6Wf9i&index=1 https://www.youtube.com/watch?v=pNe1wWeaHOU&list=PLYI8318YYdkCsZ7dsYV01n6TZhXA6Wf9i&index=1 http://audiobible.life CHECK IT OUT! On Wed, Sep 6, 2017 at 5:57 PM, Nick Way wrote: > Hi, I have a custom field "listOfIDs" = "1,2,4,33" > > I want the equivalent of: > > select * where '1' IN (listOfIDs) --> should get a match > > select * where '33' IN (listOfIDs) --> should get a match > > select * where '3' IN (listOfIDs) --> should NOT get a match > > > Can anyone help me out please as I can't seem to find any documentation on > this. Thanks very much in advance. > > Kind regards, > > > Nick Way
Re: Unified highlighter returns an error when hl.fl param has undefined fields
On 9/4/2017 9:49 PM, Yasufumi Mizoguchi wrote: > I understood what you are saying. However, at least, I think it > strange that UnifiedSolrHighlighter > returns the same error when choosing ", " as the field delimiter in > hl.fl (e.g. hl.fl=name,%20manu). > This is because UnifiedSolrHighlighter detects that there is a > zero-length string between "," and " ", > and treats the string as a field name. > Is this a correct behavior? I have never used any highlighter functionality, so I am clueless about it ... but if the parsing of hl.fl leads to a situation where it thinks an empty string is one of the field names that has been requested, that sounds like a bug in the argument parser that needs to be fixed. Thanks, Shawn
Re: write.lock file appears and solr wont open
On 9/4/2017 5:53 PM, Erick Erickson wrote:
> Gah, thanks for letting us know. I can't tell you how often
> permissions issues have tripped me up. You're right, it does seem like
> there could be a better error message though.
I see this code in NativeFSLockFactory, code that completely ignores any
problems creating the lockfile, right before the point in the
obtainFSLock method where Phil's exception came from:
try {
Files.createFile(lockFile);
} catch (IOException ignore) {
// we must create the file to have a truly canonical path.
// if it's already created, we don't care. if it cant be created,
it will fail below.
}
I think that if we replaced that code with the following code, the
*reason* for ignoring the creation problem (file already exists) will be
preserved. Any creation problem (like permissions) would throw a
(hopefully understandable) standard Java exception that propagates up
into what Solr logs:
// If the lockfile already exists, we're going to do nothing.
// If there are problems with that lockfile, they will be caught later.
// If we *do* create the file here, exceptions will propagate upward.
if (Files.notExists(lockFile))
{
Files.createFile(lockFile);
}
The method signature already includes IOException, so this doesn't
represent an API change.
Thanks,
Shawn
Re: Solr6.6 Issue/Bug
Try to utilize the steps mentioned here at http://lucene.apache.org/solr/guide/6_6/taking-solr-to-production.html On Wed, Sep 6, 2017 at 3:52 AM, Michael Kuhlmann wrote: > Why would you need to start Solr as root? You should definitely not do > this, there's no reason for that. > > And even if you *really* want this: What's so bad about the -force option? > > -Michael > > Am 06.09.2017 um 07:26 schrieb Kasim Jinwala: > > Dear team, > > I am using solr 5.0 last 1 year, now we are planning to upgrade > > solr 6.6. > > While trying to start solr using root user, we need to pass -force > > parameter to start solr forcefully, > > please help to start solr using root user without -force command. > > > > Regards > > Kasim J. > > > >
Re: Solr list operator
Nick, checkout terms query parser http://lucene.apache.org/solr/guide/6_6/other-parsers.html or streaming expressions. Thnx On Wed, Sep 6, 2017 at 8:33 AM, alex goretoy wrote: > https://www.youtube.com/watch?v=pNe1wWeaHOU&list= > PLYI8318YYdkCsZ7dsYV01n6TZhXA6Wf9i&index=1 > https://www.youtube.com/watch?v=pNe1wWeaHOU&list= > PLYI8318YYdkCsZ7dsYV01n6TZhXA6Wf9i&index=1 > > http://audiobible.life CHECK IT OUT! > > > On Wed, Sep 6, 2017 at 5:57 PM, Nick Way > wrote: > > Hi, I have a custom field "listOfIDs" = "1,2,4,33" > > > > I want the equivalent of: > > > > select * where '1' IN (listOfIDs) --> should get a match > > > > select * where '33' IN (listOfIDs) --> should get a match > > > > select * where '3' IN (listOfIDs) --> should NOT get a match > > > > > > Can anyone help me out please as I can't seem to find any documentation > on > > this. Thanks very much in advance. > > > > Kind regards, > > > > > > Nick Way >
deep paging in parallel sql
Dear list, Is it possible to enable deep paging when querying data through Parallel SQL? Regards, Imran Sent from Mail for Windows 10
Re: facet query when using word 'AND'
On 9/6/2017 3:48 AM, Noriyuki TAKEI wrote: > I use facet query,but I found it dose not work when using 'AND'. > > I woud like to use facet query using 'AND' as not Operator but simple word. With the standard or edismax parser, AND in all uppercase is interpreted as an operator. There are two ways to deal with this. One is to change the word to lowercase, which might not do what you want depending on your text analysis, the other is to escape part of it -- use A\ND instead of AND. I have verified that escaping one of the letters in the word *does* work. If you're using the edismax query parser and the lowercase option, be sure that the lowercaseOperators parameter is set to false. The default setting depends on luceneMatchVersion. It's false when that's 7.0.0 or later, true if it's lower. Which means that until version 7.0 is released, it will default to true. https://issues.apache.org/jira/browse/SOLR-4646 Thanks, Shawn
Re: Solr list operator
Use a multivalued field. Search for listOfIds:1. Or search for listOfIds:33. This is one of the simplest things that Solr can do. wunder Walter Underwood wun...@wunderwood.org http://observer.wunderwood.org/ (my blog) > On Sep 6, 2017, at 6:07 AM, Susheel Kumar wrote: > > Nick, checkout terms query parser > http://lucene.apache.org/solr/guide/6_6/other-parsers.html or streaming > expressions. > > Thnx > > On Wed, Sep 6, 2017 at 8:33 AM, alex goretoy wrote: > >> https://www.youtube.com/watch?v=pNe1wWeaHOU&list= >> PLYI8318YYdkCsZ7dsYV01n6TZhXA6Wf9i&index=1 >> https://www.youtube.com/watch?v=pNe1wWeaHOU&list= >> PLYI8318YYdkCsZ7dsYV01n6TZhXA6Wf9i&index=1 >> >> http://audiobible.life CHECK IT OUT! >> >> >> On Wed, Sep 6, 2017 at 5:57 PM, Nick Way >> wrote: >>> Hi, I have a custom field "listOfIDs" = "1,2,4,33" >>> >>> I want the equivalent of: >>> >>> select * where '1' IN (listOfIDs) --> should get a match >>> >>> select * where '33' IN (listOfIDs) --> should get a match >>> >>> select * where '3' IN (listOfIDs) --> should NOT get a match >>> >>> >>> Can anyone help me out please as I can't seem to find any documentation >> on >>> this. Thanks very much in advance. >>> >>> Kind regards, >>> >>> >>> Nick Way >>
Re: Streaming expression API innerJoin on multi-valued field
The cartesianProduct Stream can be wrapped around the stream with the multi-value field. The cartesianProduct function is available in Solr 6.6 but since this was a late addition the documentation does not appear to Solr 7.0. Here is a link to the docs in github: https://github.com/apache/lucene-solr/blob/branch_7_0/solr/solr-ref-guide/src/stream-decorators.adoc The first stream decorator is the docs the cartesianProduct. Since you can't sort on the multi-valued field though you'll have use a hashJoin to do the join. Joel Bernstein http://joelsolr.blogspot.com/ On Wed, Sep 6, 2017 at 4:08 AM, Marc Röttig wrote: > Dear SOLR users, > > I want to use streaming expression innerJoin using a multi-valued field to > do the join by equality, that is having any child documents (of type > "child") and > one parent document (of type "parent") join these according to equality of > id_s and children_ids > > Parent > * id_s = "p123" > * type_s = "parent" > * children_ids_ss = "c1,c2" > > Child > * id_s = "c1" > * type_s = "child" > > Child > * id_s = "c2" > * type_s = "child" > > innerJoin( >search(collection,q="type_s:child",fl="id_s",sort="id_s ASC"), >search(collection,q="type_s:parent",fl="id_s,children_ids_ss",sort="id_s > ASC"), >on="id_s=children_ids_ss" > ) > > This seems to be impossible, I am getting the following exception > "java.util.ArrayList cannot be cast to java.lang.Comparable". Using a > GraphQuery with from and to > this relationship traversal along multi-valued fields worked (however not > between shards, this is why I switched to streaming expressions). > > Is there any mechanism to flatten the tuples with the multi-valued field > into new tuples with single-valued fields to get the join working ? Or any > other tweak. > > Note: The relationship between Parent and Child is many-to-many, thus > moving the foreign-keys to the children as single-valued fields is not > possible. > > The issue is related tot he following issue: http://lucene.472066.n3. > nabble.com/Using-multi-valued-field-in-solr-cloud-Graph- > Traversal-Query-td4324379.html > > Thanks a lot in advance for any assistance, > Marc > > > Dr. Marc Röttig > Software Developer > EMail: marc.roet...@vico-research.com > Telefon: +49(0)711. 78 78 29-290 > Fax +49(0)711. 78 78 29-10 > > VICO Research & Consulting GmbH > Friedrich-List-Strasse 46 / 70771 Leinfelden-Echterdingen > > Homepage: www.vico-research.com/ > Blog: www.vico-research.com/expert-talk > Twitter: www.twitter.com/vico_news > Facebook:www.facebook.com/vico.friend > Sitz der Gesellschaft: Leinfelden-Echterdingen > Amtsgericht Stuttgart, HRB 720896 > Geschäftsführer: Marc Trömel > >
Re: deep paging in parallel sql
Parallel SQL supports unlimited SELECT statements which return the entire result set. The documentation discusses the differences between the limited and unlimited SELECT statements. Other then the LIMIT clause there is not yet support for paging. Joel Bernstein http://joelsolr.blogspot.com/ On Wed, Sep 6, 2017 at 9:11 AM, Imran Rajjad wrote: > Dear list, > > Is it possible to enable deep paging when querying data through Parallel > SQL? > > Regards, > Imran > > Sent from Mail for Windows 10 > >
Customizing JSON response of a query
Hello all, After a week of research I've come to the conclusion that there is no mechanism within solr where I can create a nested json response like this: https://pastebin.com/XavvUP94 . I am able to get something like this https://pastebin.com/FeXRqG59. 1. Am I right in assuming that within solr this is not possbile? 2. Assuming it is, I imagine I would need to write custom response writer in Java to customize the response. I am having a hard time locating the right resource to get me started on writing this. Any ideas? Thanks, Sarvo
Re: Solr list operator
You'll have to split up the input on commas if you don't just do it the multiValued way Walter suggests, perhaps one of the pattern tokenizers mentioned here: https://cwiki.apache.org/confluence/display/solr/Tokenizers Best, Erick On Wed, Sep 6, 2017 at 6:29 AM, Walter Underwood wrote: > Use a multivalued field. Search for listOfIds:1. Or search for listOfIds:33. > This is one of the simplest things that Solr can do. > > wunder > Walter Underwood > wun...@wunderwood.org > http://observer.wunderwood.org/ (my blog) > > >> On Sep 6, 2017, at 6:07 AM, Susheel Kumar wrote: >> >> Nick, checkout terms query parser >> http://lucene.apache.org/solr/guide/6_6/other-parsers.html or streaming >> expressions. >> >> Thnx >> >> On Wed, Sep 6, 2017 at 8:33 AM, alex goretoy wrote: >> >>> https://www.youtube.com/watch?v=pNe1wWeaHOU&list= >>> PLYI8318YYdkCsZ7dsYV01n6TZhXA6Wf9i&index=1 >>> https://www.youtube.com/watch?v=pNe1wWeaHOU&list= >>> PLYI8318YYdkCsZ7dsYV01n6TZhXA6Wf9i&index=1 >>> >>> http://audiobible.life CHECK IT OUT! >>> >>> >>> On Wed, Sep 6, 2017 at 5:57 PM, Nick Way >>> wrote: Hi, I have a custom field "listOfIDs" = "1,2,4,33" I want the equivalent of: select * where '1' IN (listOfIDs) --> should get a match select * where '33' IN (listOfIDs) --> should get a match select * where '3' IN (listOfIDs) --> should NOT get a match Can anyone help me out please as I can't seem to find any documentation >>> on this. Thanks very much in advance. Kind regards, Nick Way >>> >
Re: facet query when using word 'AND'
I typed the below then noticed that the field that has "I have a pen AND an apple" is called "body" and you're faceting and searching on a field called "suggest". The below is still relevant if there is still a problem though ;) Your problem isn't faceting, right? It's that you aren't matching any docs at all when you expect to. Once we solve that then we can move on to faceting ;) First, I'd add &debug=query and see what the parsed query looks like. Does it fit your expectations? See "parsedQuery_toString" in the debug output. Next, use the admin/analysis page to see how the input is analyzed at index and query time. The first thing to verify is that the "and" is not being removed since it's often considered a stopword. Do you get facets when searching for "apple" (no quotes)? If you do is "and" included as a facet? If not then you need to modify the stopwords associated with the field in your schema. You can use the admin//schema browser to see the actual terms in the index that'll help you verify that the _tokens_ in the index include what you expect. Faceting works on indexed tokens BTW. This is different than returning the field, since when you specify fl=suggest you're getting back the _stored_ values, not the searchable indexed tokens. This fools everybody at first. Stored values are a verbatim copy of the input, no analysis at all. However, what you can search/facet/etc. on is the tokenized version, which are _not_ returned via the "fl" field. Best, Erick P.S. My compliments for including enough data to offer a diagnosis the first time. It's refreshing not to have to go through 3 or 4 exchanges before having enough information to say anything even potentially useful. On Wed, Sep 6, 2017 at 6:13 AM, Shawn Heisey wrote: > On 9/6/2017 3:48 AM, Noriyuki TAKEI wrote: >> I use facet query,but I found it dose not work when using 'AND'. >> >> I woud like to use facet query using 'AND' as not Operator but simple word. > > With the standard or edismax parser, AND in all uppercase is interpreted > as an operator. There are two ways to deal with this. One is to change > the word to lowercase, which might not do what you want depending on > your text analysis, the other is to escape part of it -- use A\ND > instead of AND. I have verified that escaping one of the letters in the > word *does* work. > > If you're using the edismax query parser and the lowercase option, be > sure that the lowercaseOperators parameter is set to false. The default > setting depends on luceneMatchVersion. It's false when that's 7.0.0 or > later, true if it's lower. Which means that until version 7.0 is > released, it will default to true. > > https://issues.apache.org/jira/browse/SOLR-4646 > > Thanks, > Shawn >
Re: write.lock file appears and solr wont open
Or only catch the specific exception and only swallow that? But yeah,
this is something that should change as I see this "in the field" and
a more specific error message would short-circuit a lot of unnecessary
pain.
see: LUCENE-7959
Erick
On Wed, Sep 6, 2017 at 5:49 AM, Shawn Heisey wrote:
> On 9/4/2017 5:53 PM, Erick Erickson wrote:
>> Gah, thanks for letting us know. I can't tell you how often
>> permissions issues have tripped me up. You're right, it does seem like
>> there could be a better error message though.
>
> I see this code in NativeFSLockFactory, code that completely ignores any
> problems creating the lockfile, right before the point in the
> obtainFSLock method where Phil's exception came from:
>
> try {
> Files.createFile(lockFile);
> } catch (IOException ignore) {
> // we must create the file to have a truly canonical path.
> // if it's already created, we don't care. if it cant be created,
> it will fail below.
> }
>
> I think that if we replaced that code with the following code, the
> *reason* for ignoring the creation problem (file already exists) will be
> preserved. Any creation problem (like permissions) would throw a
> (hopefully understandable) standard Java exception that propagates up
> into what Solr logs:
>
> // If the lockfile already exists, we're going to do nothing.
> // If there are problems with that lockfile, they will be caught later.
> // If we *do* create the file here, exceptions will propagate upward.
> if (Files.notExists(lockFile))
> {
> Files.createFile(lockFile);
> }
>
> The method signature already includes IOException, so this doesn't
> represent an API change.
>
> Thanks,
> Shawn
>
Re: Could not find configName error
Ah, then you don't have the bin/solr scripts. There are zkCli and zkcli scripts that will allow you to get/put the config files, see the manual. I recommend you download the reference guide for 4.10 from here: http://archive.apache.org/dist/lucene/solr/ref-guide/ Best, Erick On Tue, Sep 5, 2017 at 11:49 PM, Wael Kader wrote: > i am using SOLR 4.10.3 > > I am not sure I have them in source control. I don't actually know what > that is. > I am using SOLR on a pre-setup VM. > > On Tue, Sep 5, 2017 at 5:26 PM, Erick Erickson > wrote: > >> What version of Solr? >> >> bin/solr zk -help >> >> In particular upconfig can be used to move configsets up to Zookeeper >> (or back down or whatever) in relatively recent versions of Solr. Yo >> are keeping them in source control right? ;) >> >> Best, >> Erick >> >> On Mon, Sep 4, 2017 at 11:27 PM, Wael Kader wrote: >> > Hi, >> > >> > I had some issues in SOLR shutting down on a single node application on >> > Hadoop. >> > >> > After starting up i got the error: >> > Could not find configName for collection XXX found. >> > >> > I know the issue is that the configs has issues in Zookeeper but I would >> > like to know how I can push this configuration back to get the index >> > running. >> > >> > -- >> > Regards, >> > Wael >> > > > > -- > Regards, > Wael
RE: Customizing JSON response of a query
It should be possible with a custom response handler. -Original Message- From: Sarvothaman Madhavan [mailto:relad...@gmail.com] Sent: Wednesday, September 06, 2017 10:17 AM To: solr-user@lucene.apache.org Subject: Customizing JSON response of a query Hello all, After a week of research I've come to the conclusion that there is no mechanism within solr where I can create a nested json response like this: https://pastebin.com/XavvUP94 . I am able to get something like this https://pastebin.com/FeXRqG59. 1. Am I right in assuming that within solr this is not possbile? 2. Assuming it is, I imagine I would need to write custom response writer in Java to customize the response. I am having a hard time locating the right resource to get me started on writing this. Any ideas? Thanks, Sarvo
Solr with Alluxio
Hi, I’m trying to access index files of a core stored in S3 by using Alluxio with Solr. Did anyone try integrating Solr with Alluxio? Can someone give me any pointers? Also, I’ve come across this session in the Solr Conference around next week - https://lucenesolrrevolution2017.sched.com/event/BAwU/running-solr-at-memory-speed-with-alluxio. I’m interested in knowing more about this. By chance, is there any git repo I can take a look at? Thanks, Shashank
RE: Customizing JSON response of a query
Sarvo, What are you trying to achieve? Describe the use case. Cheers -- Rick On September 6, 2017 12:36:08 PM EDT, "Davis, Daniel (NIH/NLM) [C]" wrote: >It should be possible with a custom response handler. > >-Original Message- >From: Sarvothaman Madhavan [mailto:relad...@gmail.com] >Sent: Wednesday, September 06, 2017 10:17 AM >To: solr-user@lucene.apache.org >Subject: Customizing JSON response of a query > >Hello all, >After a week of research I've come to the conclusion that there is no >mechanism within solr where I can create a nested json response like >this: >https://pastebin.com/XavvUP94 . I am able to get something like this >https://pastebin.com/FeXRqG59. >1. Am I right in assuming that within solr this is not possbile? >2. Assuming it is, I imagine I would need to write custom response >writer in Java to customize the response. I am having a hard time >locating the right resource to get me started on writing this. > >Any ideas? > >Thanks, >Sarvo -- Sorry for being brief. Alternate email is rickleir at yahoo dot com
Bulk query Apache Solr
Hi, I indexed documents in Solr using dataImportHandler. Now when I want to query using the below URL, it gives me the results i want. http://localhost:8983/solr/mycore/select?indent=on&q=id:7fd326e23ffa8d1cb9c0a7b4fc5c4269&wt=json Can Solr handle bulk queries if I send over more than 10,000 ids in the query and get their result instead of sending 1 id each time to solr-server ? I am using Solr-5.2.1 Any pointers would help. Thanks!
NumberFormatException for multvalue, pint
Can't get a multi-valued pint field to update. The schema defines the field: I get the exception on this input: 7780386,7313483 Caused by: java.lang.NumberFormatException: For input string: "7780386, 7313483" at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65) at java.lang.Integer.parseInt(Integer.java:580) at java.lang.Integer.parseInt(Integer.java:615) at org.apache.solr.schema.IntPointField.createField(IntPointField.java:181) at org.apache.solr.schema.PointField.createFields(PointField.java:216) at org.apache.solr.update.DocumentBuilder.addField(DocumentBuilder.java:72) at org.apache.solr.update.DocumentBuilder.toDocument(DocumentBuilder.java:179) Not sure why the parser thinks the values are strings. I don't see any non-numeric extraneous characters. Do I need docValues and multivalued in my field definition, since they are defined on the pints field type? Thanks. -Steve
Re: NumberFormatException for multvalue, pint
On Wed, Sep 6, 2017 at 4:09 PM, Steve Pruitt wrote: > Can't get a multi-valued pint field to update. > > The schema defines the field: multiValued="true" required="false" docValues="true" stored="true"/> > > I get the exception on this input: 7780386,7313483 > > Caused by: java.lang.NumberFormatException: For input string: "7780386, > 7313483" Try two separate values: 7780386 7313483 Or in JSON you can do: dnis:[7780386,7313483] -Yonik > at > java.lang.NumberFormatException.forInputString(NumberFormatException.java:65) > at java.lang.Integer.parseInt(Integer.java:580) > at java.lang.Integer.parseInt(Integer.java:615) > at > org.apache.solr.schema.IntPointField.createField(IntPointField.java:181) > at org.apache.solr.schema.PointField.createFields(PointField.java:216) > at > org.apache.solr.update.DocumentBuilder.addField(DocumentBuilder.java:72) > at > org.apache.solr.update.DocumentBuilder.toDocument(DocumentBuilder.java:179) > > Not sure why the parser thinks the values are strings. I don't see any > non-numeric extraneous characters. > > Do I need docValues and multivalued in my field definition, since they are > defined on the pints field type? > > Thanks. > > -Steve
Re: Customizing JSON response of a query
Rick,
My use case is this :
I have a set of documents each of which have "sub documents" associated
with it. I have this in the json format and I am able to load this into a
solr collection. When I search within this set of documents using solr, I
want the response in "grouped" json format
i.e
{
"key": "value",
"sub_doc": [
{
"inner_key": "inner_value"
}
]
}
instead of solrs default flat json format:
i.e.
{
"key":"value",
"subdoc.inner_key"= ["inner_value"]
}
I think the "grouped" json format will be much more intuitive to my end
users who are going to use the search
P.S: Just to be clear I am not having any trouble querying children/parent
document since I have all of this stored using fully qualified names in
each document in the collection.
Regards,
Sarvo
On Wed, Sep 6, 2017 at 3:52 PM, Rick Leir wrote:
> Sarvo,
> What are you trying to achieve? Describe the use case.
> Cheers -- Rick
>
> On September 6, 2017 12:36:08 PM EDT, "Davis, Daniel (NIH/NLM) [C]" <
> daniel.da...@nih.gov> wrote:
> >It should be possible with a custom response handler.
> >
> >-Original Message-
> >From: Sarvothaman Madhavan [mailto:relad...@gmail.com]
> >Sent: Wednesday, September 06, 2017 10:17 AM
> >To: solr-user@lucene.apache.org
> >Subject: Customizing JSON response of a query
> >
> >Hello all,
> >After a week of research I've come to the conclusion that there is no
> >mechanism within solr where I can create a nested json response like
> >this:
> >https://pastebin.com/XavvUP94 . I am able to get something like this
> >https://pastebin.com/FeXRqG59.
> >1. Am I right in assuming that within solr this is not possbile?
> >2. Assuming it is, I imagine I would need to write custom response
> >writer in Java to customize the response. I am having a hard time
> >locating the right resource to get me started on writing this.
> >
> >Any ideas?
> >
> >Thanks,
> >Sarvo
>
> --
> Sorry for being brief. Alternate email is rickleir at yahoo dot com
Re: NumberFormatException for multvalue, pint
You're making a common mistake as to the meaning of multiValued. The input doc should look something like (xml format) 1 2 Each "mv_int_field" is a separate, complete single integer. But there can be a many of them. when you specify 1,2,3 you're telling Solr that the _single_ value of the field is "1,2,3" which, of course, doesn't parse as an integer. Best, Erick On Wed, Sep 6, 2017 at 1:09 PM, Steve Pruitt wrote: > Can't get a multi-valued pint field to update. > > The schema defines the field: multiValued="true" required="false" docValues="true" stored="true"/> > > I get the exception on this input: 7780386,7313483 > > Caused by: java.lang.NumberFormatException: For input string: "7780386, > 7313483" > at > java.lang.NumberFormatException.forInputString(NumberFormatException.java:65) > at java.lang.Integer.parseInt(Integer.java:580) > at java.lang.Integer.parseInt(Integer.java:615) > at > org.apache.solr.schema.IntPointField.createField(IntPointField.java:181) > at org.apache.solr.schema.PointField.createFields(PointField.java:216) > at > org.apache.solr.update.DocumentBuilder.addField(DocumentBuilder.java:72) > at > org.apache.solr.update.DocumentBuilder.toDocument(DocumentBuilder.java:179) > > Not sure why the parser thinks the values are strings. I don't see any > non-numeric extraneous characters. > > Do I need docValues and multivalued in my field definition, since they are > defined on the pints field type? > > Thanks. > > -Steve
Re: Bulk query Apache Solr
Yes, but that'll be expensive. Use the TermsQueryParser, that's what it's built for here: https://lucene.apache.org/solr/guide/6_6/other-parsers.html Note the plural, TermQueryParser only takes a single value, TermsQueryParser can take a list. Best, Erick On Wed, Sep 6, 2017 at 1:01 PM, Deeksha Sharma wrote: > Hi, > > > I indexed documents in Solr using dataImportHandler. > > Now when I want to query using the below URL, it gives me the results i want. > > > http://localhost:8983/solr/mycore/select?indent=on&q=id:7fd326e23ffa8d1cb9c0a7b4fc5c4269&wt=json > > > Can Solr handle bulk queries if I send over more than 10,000 ids in the query > and get their result instead of sending 1 id each time to solr-server ? > > > I am using Solr-5.2.1 > > Any pointers would help. > > > > Thanks! > >
Consecutive calls to a query give different results
I am using Solr 6.2.0 configured as a solr cloud with 2 shards and 4 replicas (total of 4 nodes). If I run the query multiple times I see the three different top scoring results. No data load is running, all data has been commited I get these three different hits with their scores: copperiinitratehemipentahydrate2325919004194430.61722 copperiinitrateoncelite1234598765 432.44238 copperiinitratehydrate18756anhydrousbasis13778319 428.24185 How is it that the same search against the same data can give different responses? I looked at the specific cores they look OK the numdocs for the replicas in a shard match This is the query: http://ae1c-ecomdev-msc01.sial.com:8983/solr/sial-catalog-product/select?defType=edismax&fl=searchmv_en_keywords,%20searchmv_keywords,searchmv_pno,%20searchmv_en_s_pri_name,%20search_en_p_pri_name,%20search_pno%20[explain%20style=nl]&group.field=id_s&group.limit=30&group=true&group.sort=sort_ds%20asc&indent=on&mm=2%3C-25%25&q.op=OR&q=copper%20nitrate&qf=search_pid ^500%20search_concat_pno^400%20searchmv_concat_sku^400%20searchmv_pno^300%20search_concat_pno_genr^100%20searchmv_pno_genr%20searchmv_p_skus_genr%20searchmv_user_term^200%20search_lform^190%20searchmv_en_acronym^180%20search_en_root_name^170%20searchmv_en_s_pri_name^160%20search_en_p_pri_name^150%20searchmv_en_synonyms^145%20searchmv_en_keywords^140%20search_en_sortkey^120%20searchmv_p_skus^100%20searchmv_chem_comp^90%20searchmv_en_name_suf%20searchmv_cas_number^80%20searchmv_component_cas^70%20search_beilstein^50%20search_color_idx^40%20search_ecnumber^30%20search_egecnumber^30%20search_femanumber^20%20searchmv_isbn^10%20search_mdl_number%20searchmv_en_page_title%20searchmv_en_descriptions%20searchmv_en_attributes%20searchmv_rtecs%20searchmv_lookahead_terms%20searchmv_xref_comparable_pno%20searchmv_xref_comparable_sku%20searchmv_xref_equivalent_pno%20searchmv_xref_exact_pno%20searchmv_xref_exact_sku%20searchmv_component_molform&rows=30&sort=score%20desc,sort_en_name%20asc,sort_ds%20asc,search_pid%20asc&wt=json -- This message and any attachment are confidential and may be privileged or otherwise protected from disclosure. If you are not the intended recipient, you must not copy this message or attachment or disclose the contents to any other person. If you have received this transmission in error, please notify the sender immediately and delete the message and any attachment from your system. Merck KGaA, Darmstadt, Germany and any of its subsidiaries do not accept liability for any omissions or errors in this message which may arise as a result of E-Mail-transmission or for damages resulting from any unauthorized changes of the content of this message and any attachment thereto. Merck KGaA, Darmstadt, Germany and any of its subsidiaries do not guarantee that this message is free of viruses and does not accept liability for any damages caused by any virus transmitted therewith. Click http://www.emdgroup.com/disclaimer to access the German, French, Spanish and Portuguese versions of this disclaimer.
Re: Customizing JSON response of a query
Sarvo,
I hope the users do not read JSON. I would have thought you'd have a web app in
front of Solr and some Javascript in the browser. Either would be able to
transform Solr's output into a display format. But I suspect there is more to
the problem, and I do not understand it all.
Cheers -- Rick
On September 6, 2017 4:42:03 PM EDT, Sarvothaman Madhavan
wrote:
>Rick,
>
>My use case is this :
>
>I have a set of documents each of which have "sub documents" associated
>with it. I have this in the json format and I am able to load this into
>a
>solr collection. When I search within this set of documents using solr,
>I
>want the response in "grouped" json format
>
>i.e
>
>{
>
> "key": "value",
>
> "sub_doc": [
>
>{
>
> "inner_key": "inner_value"
>
>}
>
> ]
>
>}
>
>
>
>instead of solrs default flat json format:
>
>i.e.
>
>{
>
> "key":"value",
>
> "subdoc.inner_key"= ["inner_value"]
>
>}
>
>
>
>I think the "grouped" json format will be much more intuitive to my end
>users who are going to use the search
>
>
>
>P.S: Just to be clear I am not having any trouble querying
>children/parent
>document since I have all of this stored using fully qualified names in
>each document in the collection.
>
>
>
>
>
>Regards,
>
>Sarvo
>
>
>
>On Wed, Sep 6, 2017 at 3:52 PM, Rick Leir wrote:
>
>> Sarvo,
>> What are you trying to achieve? Describe the use case.
>> Cheers -- Rick
>>
>> On September 6, 2017 12:36:08 PM EDT, "Davis, Daniel (NIH/NLM) [C]" <
>> daniel.da...@nih.gov> wrote:
>> >It should be possible with a custom response handler.
>> >
>> >-Original Message-
>> >From: Sarvothaman Madhavan [mailto:relad...@gmail.com]
>> >Sent: Wednesday, September 06, 2017 10:17 AM
>> >To: solr-user@lucene.apache.org
>> >Subject: Customizing JSON response of a query
>> >
>> >Hello all,
>> >After a week of research I've come to the conclusion that there is
>no
>> >mechanism within solr where I can create a nested json response like
>> >this:
>> >https://pastebin.com/XavvUP94 . I am able to get something like this
>> >https://pastebin.com/FeXRqG59.
>> >1. Am I right in assuming that within solr this is not possbile?
>> >2. Assuming it is, I imagine I would need to write custom response
>> >writer in Java to customize the response. I am having a hard time
>> >locating the right resource to get me started on writing this.
>> >
>> >Any ideas?
>> >
>> >Thanks,
>> >Sarvo
>>
>> --
>> Sorry for being brief. Alternate email is rickleir at yahoo dot com
--
Sorry for being brief. Alternate email is rickleir at yahoo dot com
Re: Unified highlighter returns an error when hl.fl param has undefined fields
Hi Shawn, Thank you for your reply. > that sounds like a bug in the argument parser that needs to be fixed. I have created a JIRA about this. https://issues.apache.org/jira/browse/SOLR-11334 Thanks, Yasufumi On 2017/09/06 9:48 PM, Shawn Heisey wrote: On 9/4/2017 9:49 PM, Yasufumi Mizoguchi wrote: I understood what you are saying. However, at least, I think it strange that UnifiedSolrHighlighter returns the same error when choosing ", " as the field delimiter in hl.fl (e.g. hl.fl=name,%20manu). This is because UnifiedSolrHighlighter detects that there is a zero-length string between "," and " ", and treats the string as a field name. Is this a correct behavior? I have never used any highlighter functionality, so I am clueless about it ... but if the parsing of hl.fl leads to a situation where it thinks an empty string is one of the field names that has been requested, that sounds like a bug in the argument parser that needs to be fixed. Thanks, Shawn
Re: Consecutive calls to a query give different results
Different replicas of the same shard can have different numbers of deleted documents (really just marked as deleted), and deleted documents are irrelevant to term statistics (like the number of documents a term appears in). Documents marked for deletion stop contributing to corpus statistics when they are actually removed (via expunge deletes, merges, optimizes). -Yonik On Wed, Sep 6, 2017 at 5:51 PM, Webster Homer wrote: > I am using Solr 6.2.0 configured as a solr cloud with 2 shards and 4 > replicas (total of 4 nodes). > > If I run the query multiple times I see the three different top scoring > results. > No data load is running, all data has been commited > > I get these three different hits with their scores: > copperiinitratehemipentahydrate2325919004194430.61722 > copperiinitrateoncelite1234598765 432.44238 > copperiinitratehydrate18756anhydrousbasis13778319 428.24185 > > How is it that the same search against the same data can give different > responses? > I looked at the specific cores they look OK the numdocs for the replicas in > a shard match > > This is the query: > http://ae1c-ecomdev-msc01.sial.com:8983/solr/sial-catalog-product/select?defType=edismax&fl=searchmv_en_keywords,%20searchmv_keywords,searchmv_pno,%20searchmv_en_s_pri_name,%20search_en_p_pri_name,%20search_pno%20[explain%20style=nl]&group.field=id_s&group.limit=30&group=true&group.sort=sort_ds%20asc&indent=on&mm=2%3C-25%25&q.op=OR&q=copper%20nitrate&qf=search_pid > ^500%20search_concat_pno^400%20searchmv_concat_sku^400%20searchmv_pno^300%20search_concat_pno_genr^100%20searchmv_pno_genr%20searchmv_p_skus_genr%20searchmv_user_term^200%20search_lform^190%20searchmv_en_acronym^180%20search_en_root_name^170%20searchmv_en_s_pri_name^160%20search_en_p_pri_name^150%20searchmv_en_synonyms^145%20searchmv_en_keywords^140%20search_en_sortkey^120%20searchmv_p_skus^100%20searchmv_chem_comp^90%20searchmv_en_name_suf%20searchmv_cas_number^80%20searchmv_component_cas^70%20search_beilstein^50%20search_color_idx^40%20search_ecnumber^30%20search_egecnumber^30%20search_femanumber^20%20searchmv_isbn^10%20search_mdl_number%20searchmv_en_page_title%20searchmv_en_descriptions%20searchmv_en_attributes%20searchmv_rtecs%20searchmv_lookahead_terms%20searchmv_xref_comparable_pno%20searchmv_xref_comparable_sku%20searchmv_xref_equivalent_pno%20searchmv_xref_exact_pno%20searchmv_xref_exact_sku%20searchmv_component_molform&rows=30&sort=score%20desc,sort_en_name%20asc,sort_ds%20asc,search_pid%20asc&wt=json > > -- > > > This message and any attachment are confidential and may be privileged or > otherwise protected from disclosure. If you are not the intended recipient, > you must not copy this message or attachment or disclose the contents to > any other person. If you have received this transmission in error, please > notify the sender immediately and delete the message and any attachment > from your system. Merck KGaA, Darmstadt, Germany and any of its > subsidiaries do not accept liability for any omissions or errors in this > message which may arise as a result of E-Mail-transmission or for damages > resulting from any unauthorized changes of the content of this message and > any attachment thereto. Merck KGaA, Darmstadt, Germany and any of its > subsidiaries do not guarantee that this message is free of viruses and does > not accept liability for any damages caused by any virus transmitted > therewith. > > Click http://www.emdgroup.com/disclaimer to access the German, French, > Spanish and Portuguese versions of this disclaimer.
Re: Consecutive calls to a query give different results
bq: and deleted documents are irrelevant to term statistics... Did you mean "relevant"? Or do I have to adjust my thinking _again_? Erick On Wed, Sep 6, 2017 at 7:48 PM, Yonik Seeley wrote: > Different replicas of the same shard can have different numbers of > deleted documents (really just marked as deleted), and deleted > documents are irrelevant to term statistics (like the number of > documents a term appears in). Documents marked for deletion stop > contributing to corpus statistics when they are actually removed (via > expunge deletes, merges, optimizes). > -Yonik > > > On Wed, Sep 6, 2017 at 5:51 PM, Webster Homer wrote: >> I am using Solr 6.2.0 configured as a solr cloud with 2 shards and 4 >> replicas (total of 4 nodes). >> >> If I run the query multiple times I see the three different top scoring >> results. >> No data load is running, all data has been commited >> >> I get these three different hits with their scores: >> copperiinitratehemipentahydrate2325919004194430.61722 >> copperiinitrateoncelite1234598765 432.44238 >> copperiinitratehydrate18756anhydrousbasis13778319 428.24185 >> >> How is it that the same search against the same data can give different >> responses? >> I looked at the specific cores they look OK the numdocs for the replicas in >> a shard match >> >> This is the query: >> http://ae1c-ecomdev-msc01.sial.com:8983/solr/sial-catalog-product/select?defType=edismax&fl=searchmv_en_keywords,%20searchmv_keywords,searchmv_pno,%20searchmv_en_s_pri_name,%20search_en_p_pri_name,%20search_pno%20[explain%20style=nl]&group.field=id_s&group.limit=30&group=true&group.sort=sort_ds%20asc&indent=on&mm=2%3C-25%25&q.op=OR&q=copper%20nitrate&qf=search_pid >> ^500%20search_concat_pno^400%20searchmv_concat_sku^400%20searchmv_pno^300%20search_concat_pno_genr^100%20searchmv_pno_genr%20searchmv_p_skus_genr%20searchmv_user_term^200%20search_lform^190%20searchmv_en_acronym^180%20search_en_root_name^170%20searchmv_en_s_pri_name^160%20search_en_p_pri_name^150%20searchmv_en_synonyms^145%20searchmv_en_keywords^140%20search_en_sortkey^120%20searchmv_p_skus^100%20searchmv_chem_comp^90%20searchmv_en_name_suf%20searchmv_cas_number^80%20searchmv_component_cas^70%20search_beilstein^50%20search_color_idx^40%20search_ecnumber^30%20search_egecnumber^30%20search_femanumber^20%20searchmv_isbn^10%20search_mdl_number%20searchmv_en_page_title%20searchmv_en_descriptions%20searchmv_en_attributes%20searchmv_rtecs%20searchmv_lookahead_terms%20searchmv_xref_comparable_pno%20searchmv_xref_comparable_sku%20searchmv_xref_equivalent_pno%20searchmv_xref_exact_pno%20searchmv_xref_exact_sku%20searchmv_component_molform&rows=30&sort=score%20desc,sort_en_name%20asc,sort_ds%20asc,search_pid%20asc&wt=json >> >> -- >> >> >> This message and any attachment are confidential and may be privileged or >> otherwise protected from disclosure. If you are not the intended recipient, >> you must not copy this message or attachment or disclose the contents to >> any other person. If you have received this transmission in error, please >> notify the sender immediately and delete the message and any attachment >> from your system. Merck KGaA, Darmstadt, Germany and any of its >> subsidiaries do not accept liability for any omissions or errors in this >> message which may arise as a result of E-Mail-transmission or for damages >> resulting from any unauthorized changes of the content of this message and >> any attachment thereto. Merck KGaA, Darmstadt, Germany and any of its >> subsidiaries do not guarantee that this message is free of viruses and does >> not accept liability for any damages caused by any virus transmitted >> therewith. >> >> Click http://www.emdgroup.com/disclaimer to access the German, French, >> Spanish and Portuguese versions of this disclaimer.
RE: deep paging in parallel sql
My only concern is the performance as the cursor moves forward in resultset with approximately 2 billion records Regards, Imran Sent from Mail for Windows 10 From: Joel Bernstein Sent: Wednesday, September 6, 2017 7:04 PM To: solr-user@lucene.apache.org Subject: Re: deep paging in parallel sql Parallel SQL supports unlimited SELECT statements which return the entire result set. The documentation discusses the differences between the limited and unlimited SELECT statements. Other then the LIMIT clause there is not yet support for paging. Joel Bernstein http://joelsolr.blogspot.com/ On Wed, Sep 6, 2017 at 9:11 AM, Imran Rajjad wrote: > Dear list, > > Is it possible to enable deep paging when querying data through Parallel > SQL? > > Regards, > Imran > > Sent from Mail for Windows 10 > >
