Searcher is opening twice on Reload

[Aman Tandon]

Hi,

Please help me here, when I am doing the reload of core, my searcher is
being opening twice. I am also attaching the logs output, please suggest me
what wrong I am doing here or this is default behavior on reload.

May 14, 2015 12:47:38 PM org.apache.solr.spelling.DirectSolrSpellChecker
> INFO: init:
> {name=default,field=titlews,classname=solr.DirectSolrSpellChecker,distanceMeasure=internal,accuracy=0.5,maxEdits=1,minPrefix=1,maxInspections=5,minQueryLength=5,maxQueryFrequency=100.0,thresholdTokenFrequency=100.0}
> May 14, 2015 12:47:38 PM
> org.apache.solr.handler.component.SpellCheckComponent
> INFO: No queryConverter defined, using default converter
> May 14, 2015 12:47:38 PM
> org.apache.solr.handler.component.QueryElevationComponent
> INFO: Loading QueryElevation from data dir: elevate.xml
> May 14, 2015 12:47:38 PM org.apache.solr.handler.ReplicationHandler
> INFO: Commits will be reserved for  1
> May 14, 2015 12:47:38 PM org.apache.solr.core.QuerySenderListener
> INFO: QuerySenderListener sending requests to Searcher@41dc3c83[IM-Search]
> main{StandardDirectoryReader(segments_dd4:82296:nrt
> _jdq(4.8):C5602938/2310052:delGen=3132
> _jkq(4.8):C6860454/1398005:delGen=2992
> _jx2(4.8):C5237053/1505048:delGen=3241
> _joo(4.8):C5825253/1599671:delGen=3323
> _k4d(4.8):C5860360/1916531:delGen=3150
> _o27(4.8):C5290435/1018865:delGen=370
> _mju(4.8):C5074973/1602707:delGen=1474
> _jka(4.8):C5172599/1774839:delGen=3202
> _nik(4.8):C4698916/1512091:delGen=804 _o8y(4.8):C1137592/521423:delGen=190
> _oeu(4.8):C469094/86291:delGen=29 _odq(4.8):C217505/65596:delGen=55
> _ogd(4.8):C50454/4155:delGen=5 _oea(4.8):C40833/7192:delGen=37
> _ofy(4.8):C73614/7273:delGen=13 _ogx(4.8):C395681/1388:delGen=4
> _ogh(4.8):C7676/70:delGen=2 _ohf(4.8):C108769/21:delGen=2
> _ogc(4.8):C24435/384:delGen=4 _ogi(4.8):C23088/158:delGen=3
> _ogj(4.8):C4217/2:delGen=1 _ohs(4.8):C7 _oh6(4.8):C20509/205:delGen=5
> _oh7(4.8):C3171 _oho(4.8):C6/1:delGen=1 _ohq(4.8):C1
> _ohv(4.8):C10484/996:delGen=2 _ohx(4.8):C500 _ohy(4.8):C1 _ohz(4.8):C1)}
> ^[OFMay 14, 2015 12:47:43 PM org.apache.solr.core.SolrCore
> INFO: [IM-Search] webapp=/solr path=/select
> params={spellcheck=true&lon=0&q=q&wt=json&qt=opsview.monitor&lat=0&rows=0&ps=1}
> hits=6 status=0 QTime=1
> May 14, 2015 12:47:44 PM org.apache.solr.core.SolrCore
> INFO: [IM-Search] webapp=null path=null
> params={start=0&event=firstSearcher&q=rice&distrib=false&qt=im.search.intent&rows=25}
> hits=42749 status=0 QTime=5667
> May 14, 2015 12:47:58 PM org.apache.solr.request.UnInvertedField
> INFO: UnInverted multi-valued field
> {field=city,memSize=209216385,tindexSize=11029,time=3904,phase1=3783,nTerms=77614,bigTerms=3,termInstances=31291566,uses=0}
> May 14, 2015 12:48:01 PM org.apache.solr.request.UnInvertedField
> INFO: UnInverted multi-valued field
> {field=biztype,memSize=208847178,tindexSize=40,time=1318,phase1=1193,nTerms=9,bigTerms=4,termInstances=1607459,uses=0}
> May 14, 2015 12:48:01 PM org.apache.solr.core.SolrCore
> INFO: [IM-Search] webapp=null path=null
> params={start=0&event=firstSearcher&q=rice&distrib=false&qt=im.search&rows=25}
> hits=57619 status=0 QTime=17194
> May 14, 2015 12:48:04 PM org.apache.solr.core.SolrCore
> INFO: [IM-Search] webapp=null path=null
> params={start=0&event=firstSearcher&q=potassium+cyanide&distrib=false&qt=eto.search.offer&rows=20}
> hits=443 status=0 QTime=3272
> May 14, 2015 12:48:09 PM org.apache.solr.core.SolrCore
> INFO: [IM-Search] webapp=null path=null
> params={start=0&event=firstSearcher&q=motor+spare+parts&distrib=false&qt=im.search&fq=attribs:(locprefglobal+locprefnational+locprefcity)&rows=20}
> hits=107297 status=0 QTime=5254
> May 14, 2015 12:48:09 PM org.apache.solr.core.QuerySenderListener
> INFO: QuerySenderListener done.
> May 14, 2015 12:48:09 PM
> org.apache.solr.handler.component.SpellCheckComponent$SpellCheckerListener
> INFO: Loading spell index for spellchecker: default
> May 14, 2015 12:48:09 PM
> org.apache.solr.handler.component.SpellCheckComponent$SpellCheckerListener
> INFO: Loading spell index for spellchecker: wordbreak
> May 14, 2015 12:48:09 PM org.apache.solr.core.SolrCore
> INFO: [IM-Search] Registered new searcher Searcher@41dc3c83[IM-Search]
> main{StandardDirectoryReader(segments_dd4:82296:nrt
> _jdq(4.8):C5602938/2310052:delGen=3132
> _jkq(4.8):C6860454/1398005:delGen=2992
> _jx2(4.8):C5237053/1505048:delGen=3241
> _joo(4.8):C5825253/1599671:delGen=3323
> _k4d(4.8):C5860360/1916531:delGen=3150
> _o27(4.8):C5290435/1018865:delGen=370
> _mju(4.8):C5074973/1602707:delGen=1474
> _jka(4.8):C5172599/1774839:delGen=3202
> _nik(4.8):C4698916/1512091:delGen=804 _o8y(4.8):C1137592/521423:delGen=190
> _oeu(4.8):C469094/86291:delGen=29 _odq(4.8):C217505/65596:delGen=55
> _ogd(4.8):C50454/4155:delGen=5 _oea(4.8):C40833/7192:delGen=37
> _ofy(4.8):C73614/7273:delGen=13 _ogx(4.8):C395681/1388:delGen=4
> _ogh(4.8):C7676/70:delGen=2 _ohf(4.8):C108769/21:delGen=2
> _ogc(4.8):C24435/384:delGen=4 _ogi(4.8):C23088/158:

Real-Time get and Dynamic Fields: possible bug.

[Luis Cappa Banda]

Hi there,

I have the following dynamicFields definition in my schema.xml:




  


I' ve seen that when fetching documents with /select?q=id:whateverId, the
results returned include both i18n* and *_facet fields filled. However,
when using real-time request handler (/get?ids:whateverIds) the result
fetched include only i18n* dynamic fields, but *_facet ones are not
included.

I have the impression during /get RequestHandler the server-side regular
expression used when parsing fields and fields values to return documents
with existing dynamic fields seems to be wrong. From the client side, I' ve
checked that the class DocField.java that parses SolrDocument to Bean ones
uses the following matcher:

 } else if (annotation.value().indexOf('*') >= 0) { // dynamic fields are
annotated as @Field("categories_*")

// if the field was annotated as a dynamic field, convert the name into a
pattern

// the wildcard (*) is supposed to be either a prefix or a suffix, hence
the use of replaceFirst

name = annotation.value().replaceFirst("\\*", "\\.*");

dynamicFieldNamePatternMatcher = Pattern.compile("^" + name + "$");

 } else {

name = annotation.value();

 }

So maybe a similar behavior from the server-side is wrong. That' s the only
reason I find to understand why when using /select all fields are returned
but when using /get those that matches *_facet regexp are not.

If you can confirm that this is a bug (because maybe is the expected
behavior, but after some years using Solr I think it is not) I can create
the JIRA issue and debug it more deeply to apply a patch with the aim to
help.


Regards,


-- 
- Luis Cappa

Re: Wiki new user

[Erik Hatcher]

You have to create it first, then let us know.   Create your account here: 
https://wiki.apache.org/solr , from the login 
page.  The let us know the username you create.





> On May 14, 2015, at 2:49 AM, Sergio Velasco  wrote:
> 
> Hi Erik,
>  
> SergioVelasco is a good one? If not, SergioMitula or womething like this.
>  
> Thank you, guy!
>  
> 
> www.mitula.com    
>  Sergio Velasco   |   Dpto. de Desarrollo
>  Contáctame:
>  ser...@mitula.com    |   Tfno. +34 917 08 21 47   
> |   Fax +34 917 08 21 56
>  Síguenos en:
>  Facebook.com/mitula.es.latam    |   
> @mitula_es    |   Linkedin.com/mitula 
>    |   Blog 
>    
> 
> El contenido de este correo electrónico puede ser confidencial o 
> privilegiado. Si ha recibido este mensaje por error, por favor, no lo reenvíe 
> a nadie. Le rogamos que borre todas las copias, mensajes adjuntos y por favor 
> comuníquenos que lo ha recibido la persona equivocada. Gracias
> Antes de imprimir este mensaje, asegúrese de que es necesario. El medio 
> ambiente está en nuestra mano
>  
>  
> De: Erik Hatcher [mailto:erik.hatc...@gmail.com 
> ] 
> Enviado el: miércoles, 13 de mayo de 2015 16:44
> Para: solr-user@lucene.apache.org 
> Asunto: Re: Wiki new user
>  
> Sergio - what is your wiki username?   We can add you as an editor once you 
> provide the username. 
>  
> Erik
> 
> On May 13, 2015, at 10:33, Sergio Velasco  > wrote:
> 
>> Hi,
>>  
>> I would like to become a member of the Solr wiki. I have requested it to the 
>> Solr user lists and they have send me to this list to request access to the 
>> wiki.
>>  
>> I am the Mitula CTO and we have been using Solr from the very beginning, 6 
>> years ago. I think I can contribute a lot to this wiki.
>>  
>> Thank you.
>>  
>>  
>>  
>>  
>> 
>> www.mitula.com   
>>  Sergio Velasco   |   Dpto. de Desarrollo
>>  Contáctame:
>>  ser...@mitula.com    |   Tfno. +34 917 08 21 47   
>> |   Fax +34 917 08 21 56
>>  Síguenos en:
>>  Facebook.com/mitula.es.latam    |  
>>  @mitula_es    |   Linkedin.com/mitula 
>>    |   Blog 
>>    
>> 
>> El contenido de este correo electrónico puede ser confidencial o 
>> privilegiado. Si ha recibido este mensaje por error, por favor, no lo 
>> reenvíe a nadie. Le rogamos que borre todas las copias, mensajes adjuntos y 
>> por favor comuníquenos que lo ha recibido la persona equivocada. Gracias
>> Antes de imprimir este mensaje, asegúrese de que es necesario. El medio 
>> ambiente está en nuestra mano
>>  
>>  
>>  
>> 
>>   
>> El software de antivirus Avast ha analizado este correo electrónico en busca 
>> de virus. 
>> www.avast.com 
>>  
> 
> 
>    
> El software de antivirus Avast ha analizado este correo electrónico en busca 
> de virus. 
> www.avast.com 
> 

Cannot get "uf" (user fields) to work

[Steven White]

Hi Everyone,

I'm trying to utilize "uf" but it doesn't work.  My reading of it per:
https://cwiki.apache.org/confluence/display/solr/The+Extended+DisMax+Query+Parser
is I can use it to limit into which fields a search is performed.  E.g.: If
my index has F1, F2, F3, F4, and I set "uf" to F3, than I should get hits
on F3 only.

Here is my request handler:

  
 
   explicit
   20
   edismax
   F1 F2 F3 F4
   F3
   type,id,score
   xml
   true
 
  

I also tried passing "uf" on the URL and it too didn't work.  I'm getting
hits on terms that are in other fields.

Is my usage of "uf" wrong?

Thanks

Steve

Re: Cannot get "uf" (user fields) to work

[Emir Arnautovic]

Hi Steve,
You use qf to limit search on some fields.

Thanks,
Emir

On 14.05.2015 13:49, Steven White wrote:

Hi Everyone,

I'm trying to utilize "uf" but it doesn't work.  My reading of it per:
https://cwiki.apache.org/confluence/display/solr/The+Extended+DisMax+Query+Parser
is I can use it to limit into which fields a search is performed.  E.g.: If
my index has F1, F2, F3, F4, and I set "uf" to F3, than I should get hits
on F3 only.

Here is my request handler:

   
  
explicit
20
edismax
F1 F2 F3 F4
F3
type,id,score
xml
true
  
   

I also tried passing "uf" on the URL and it too didn't work.  I'm getting
hits on terms that are in other fields.

Is my usage of "uf" wrong?

Thanks

Steve



--
Monitoring * Alerting * Anomaly Detection * Centralized Log Management
Solr & Elasticsearch Support * http://sematext.com/

Re: Cannot get "uf" (user fields) to work

[Steven White]

Hi Emir,

Are you saying I should remove "qf" if I want to use "uf"?  I just tried
that and got exception:

java.lang.RuntimeException at
org.apache.solr.search.ExtendedDismaxQParser$ExtendedDismaxConfiguration.(ExtendedDismaxQParser.java:1497)
at
org.apache.solr.search.ExtendedDismaxQParser.createConfiguration(ExtendedDismaxQParser.java:270)
at
org.apache.solr.search.ExtendedDismaxQParser.(ExtendedDismaxQParser.java:132)
at
org.apache.solr.search.ExtendedDismaxQParserPlugin.createParser(ExtendedDismaxQParserPlugin.java:37)
at org.apache.solr.search.QParser.getParser(QParser.java:315) at
org.apache.solr.handler.component.QueryComponent.prepare(QueryComponent.java:156)
at
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:196)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:143)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1984) at

Thanks

Steve

On Thu, May 14, 2015 at 8:00 AM, Emir Arnautovic <
emir.arnauto...@sematext.com> wrote:

> Hi Steve,
> You use qf to limit search on some fields.
>
> Thanks,
> Emir
>
>
> On 14.05.2015 13:49, Steven White wrote:
>
>> Hi Everyone,
>>
>> I'm trying to utilize "uf" but it doesn't work.  My reading of it per:
>>
>> https://cwiki.apache.org/confluence/display/solr/The+Extended+DisMax+Query+Parser
>> is I can use it to limit into which fields a search is performed.  E.g.:
>> If
>> my index has F1, F2, F3, F4, and I set "uf" to F3, than I should get hits
>> on F3 only.
>>
>> Here is my request handler:
>>
>>
>>   
>> explicit
>> 20
>> edismax
>> F1 F2 F3 F4
>> F3
>> type,id,score
>> xml
>> true
>>   
>>
>>
>> I also tried passing "uf" on the URL and it too didn't work.  I'm getting
>> hits on terms that are in other fields.
>>
>> Is my usage of "uf" wrong?
>>
>> Thanks
>>
>> Steve
>>
>>
> --
> Monitoring * Alerting * Anomaly Detection * Centralized Log Management
> Solr & Elasticsearch Support * http://sematext.com/
>
>

Re: Cannot get "uf" (user fields) to work

[Jack Krupansky]

The uf parameter applies to the user query string, the q parameter - the q
parameter may only references fields from uf in the q string. The qf
parameter specifies fields to be queried behind the scenes and is not seen
by the user. The term user refers to the person who is specifying the text
that goes into the q parameter, not the client app that is forming the
overall Solr query request.


-- Jack Krupansky

On Thu, May 14, 2015 at 7:49 AM, Steven White  wrote:

> Hi Everyone,
>
> I'm trying to utilize "uf" but it doesn't work.  My reading of it per:
>
> https://cwiki.apache.org/confluence/display/solr/The+Extended+DisMax+Query+Parser
> is I can use it to limit into which fields a search is performed.  E.g.: If
> my index has F1, F2, F3, F4, and I set "uf" to F3, than I should get hits
> on F3 only.
>
> Here is my request handler:
>
>   
>  
>explicit
>20
>edismax
>F1 F2 F3 F4
>F3
>type,id,score
>xml
>true
>  
>   
>
> I also tried passing "uf" on the URL and it too didn't work.  I'm getting
> hits on terms that are in other fields.
>
> Is my usage of "uf" wrong?
>
> Thanks
>
> Steve
>

RE: Transactional Behavior

[Amr Ali]

Hello guys,

Any expert advice this issue?


--
Regards,
Amr Ali

City stars capital 8 - 3rd floor, Nasr city, Cairo, Egypt
Ext: 278



-Original Message-
From: Amr Ali 
Sent: Wednesday, May 13, 2015 10:58 AM
To: solr-user@lucene.apache.org
Subject: RE: Transactional Behavior

Hello Emir,

But this is not a transaction because if some of the bulk I need to add is 
committed; they will be searchable. In a transaction I need to insert a bulk of 
data (all bulk data will be searchable once) or roll it back according to some 
business scenarios.

--
Regards,
Amr Ali

City stars capital 8 - 3rd floor, Nasr city, Cairo, Egypt
Ext: 278



-Original Message-
From: Emir Arnautovic [mailto:emir.arnauto...@sematext.com]
Sent: Tuesday, May 12, 2015 10:46 PM
To: solr-user@lucene.apache.org
Subject: Re: Transactional Behavior

Hi Amr,
One option is to include transaction id in your documents and do delete in case 
of failed transaction. It is not cheap option - additional field if you don't 
have something to use to identify transaction. Assuming rollback will not 
happen to often deleting is not that big issue.

Thanks,
Emir

--
Monitoring * Alerting * Anomaly Detection * Centralized Log Management Solr & 
Elasticsearch Support * http://sematext.com/



On 12.05.2015 22:37, Amr Ali wrote:
> Please check this
>
> https://lucene.apache.org/solr/4_1_0/solr-solrj/org/apache/solr/client
> /solrj/SolrServer.html#rollback() "Note that this is not a true 
> rollback as in databases. Content you have previously added may have been 
> committed due to autoCommit, buffer full, other client performing a commit 
> etc."
>
> It is not a real rollback if you have two threads T1 and T2 that are adding. 
> If T1 is adding 500 and T2 is adding 3 then T2 will commit its 3 document 
> PLUS the documents added by T1 (because T2 will finish add/commit before T2 
> due to the documents number). Solr transactions are "server side" only.
>
>
> --
> Regards,
> Amr Ali
>
> City stars capital 8 - 3rd floor, Nasr city, Cairo, Egypt
> Ext: 278
>
>
>
> -Original Message-
> From: Jack Krupansky [mailto:jack.krupan...@gmail.com]
> Sent: Tuesday, May 12, 2015 10:24 PM
> To: solr-user@lucene.apache.org
> Subject: Re: Transactional Behavior
>
> Solr does have a  command, but it is an expert feature and not so 
> clear how it works in SolrCloud.
>
> See:
> https://cwiki.apache.org/confluence/display/solr/Uploading+Data+with+I
> ndex+Handlers
> and
> https://wiki.apache.org/solr/UpdateXmlMessages#A.22rollback.22
>
>
> -- Jack Krupansky
>
> On Tue, May 12, 2015 at 12:58 PM, Amr Ali  wrote:
>
>> Hello,
>>
>> I have a business case in which I need to be able for the rollback.
>> When I tried add/commit I was not able to prevent other threads that 
>> write to a given Solr core from committing everything. I also tried 
>> indexwriter but Solr did not get changes until we restart it.
>>
>>
>> --
>> Regards,
>> Amr Ali
>>
>> City stars capital 8 - 3rd floor, Nasr city, Cairo, Egypt
>> Ext: 278
>>
>>
>>

--
Monitoring * Alerting * Anomaly Detection * Centralized Log Management Solr & 
Elasticsearch Support * http://sematext.com/

Re: Cannot get "uf" (user fields) to work

[Emir Arnautovic]

Hi Steve,
No - you should use qf to limit search to fields you want. If you do not 
plan to allow users to explicitly state fields they want to search, you 
don't have to use uf parameter. So, drop uf, and set qf to F3.


Thanks,
Emir

On 14.05.2015 14:09, Steven White wrote:

Hi Emir,

Are you saying I should remove "qf" if I want to use "uf"?  I just tried
that and got exception:

java.lang.RuntimeException at
org.apache.solr.search.ExtendedDismaxQParser$ExtendedDismaxConfiguration.(ExtendedDismaxQParser.java:1497)
at
org.apache.solr.search.ExtendedDismaxQParser.createConfiguration(ExtendedDismaxQParser.java:270)
at
org.apache.solr.search.ExtendedDismaxQParser.(ExtendedDismaxQParser.java:132)
at
org.apache.solr.search.ExtendedDismaxQParserPlugin.createParser(ExtendedDismaxQParserPlugin.java:37)
at org.apache.solr.search.QParser.getParser(QParser.java:315) at
org.apache.solr.handler.component.QueryComponent.prepare(QueryComponent.java:156)
at
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:196)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:143)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1984) at

Thanks

Steve

On Thu, May 14, 2015 at 8:00 AM, Emir Arnautovic <
emir.arnauto...@sematext.com> wrote:


Hi Steve,
You use qf to limit search on some fields.

Thanks,
Emir


On 14.05.2015 13:49, Steven White wrote:


Hi Everyone,

I'm trying to utilize "uf" but it doesn't work.  My reading of it per:

https://cwiki.apache.org/confluence/display/solr/The+Extended+DisMax+Query+Parser
is I can use it to limit into which fields a search is performed.  E.g.:
If
my index has F1, F2, F3, F4, and I set "uf" to F3, than I should get hits
on F3 only.

Here is my request handler:


   
 explicit
 20
 edismax
 F1 F2 F3 F4
 F3
 type,id,score
 xml
 true
   


I also tried passing "uf" on the URL and it too didn't work.  I'm getting
hits on terms that are in other fields.

Is my usage of "uf" wrong?

Thanks

Steve



--
Monitoring * Alerting * Anomaly Detection * Centralized Log Management
Solr & Elasticsearch Support * http://sematext.com/




--
Monitoring * Alerting * Anomaly Detection * Centralized Log Management
Solr & Elasticsearch Support * http://sematext.com/

Re: Cannot get "uf" (user fields) to work

[Steven White]

Hi Jack,

I don't follow.  Can you give me an example on how and when to use "uf"?

Thanks,

Steve

On Thu, May 14, 2015 at 8:15 AM, Jack Krupansky 
wrote:

> The uf parameter applies to the user query string, the q parameter - the q
> parameter may only references fields from uf in the q string. The qf
> parameter specifies fields to be queried behind the scenes and is not seen
> by the user. The term user refers to the person who is specifying the text
> that goes into the q parameter, not the client app that is forming the
> overall Solr query request.
>
>
> -- Jack Krupansky
>
> On Thu, May 14, 2015 at 7:49 AM, Steven White 
> wrote:
>
> > Hi Everyone,
> >
> > I'm trying to utilize "uf" but it doesn't work.  My reading of it per:
> >
> >
> https://cwiki.apache.org/confluence/display/solr/The+Extended+DisMax+Query+Parser
> > is I can use it to limit into which fields a search is performed.  E.g.:
> If
> > my index has F1, F2, F3, F4, and I set "uf" to F3, than I should get hits
> > on F3 only.
> >
> > Here is my request handler:
> >
> >   
> >  
> >explicit
> >20
> >edismax
> >F1 F2 F3 F4
> >F3
> >type,id,score
> >xml
> >true
> >  
> >   
> >
> > I also tried passing "uf" on the URL and it too didn't work.  I'm getting
> > hits on terms that are in other fields.
> >
> > Is my usage of "uf" wrong?
> >
> > Thanks
> >
> > Steve
> >
>

Re: Cannot get "uf" (user fields) to work

[Steven White]

Hi Emir,

Like I replied to Jack, I don't get it.  I need to see some concrete
examples.

Thanks

Steve

On Thu, May 14, 2015 at 8:30 AM, Emir Arnautovic <
emir.arnauto...@sematext.com> wrote:

> Hi Steve,
> No - you should use qf to limit search to fields you want. If you do not
> plan to allow users to explicitly state fields they want to search, you
> don't have to use uf parameter. So, drop uf, and set qf to F3.
>
> Thanks,
> Emir
>
>
> On 14.05.2015 14:09, Steven White wrote:
>
>> Hi Emir,
>>
>> Are you saying I should remove "qf" if I want to use "uf"?  I just tried
>> that and got exception:
>>
>> java.lang.RuntimeException at
>>
>> org.apache.solr.search.ExtendedDismaxQParser$ExtendedDismaxConfiguration.(ExtendedDismaxQParser.java:1497)
>> at
>>
>> org.apache.solr.search.ExtendedDismaxQParser.createConfiguration(ExtendedDismaxQParser.java:270)
>> at
>>
>> org.apache.solr.search.ExtendedDismaxQParser.(ExtendedDismaxQParser.java:132)
>> at
>>
>> org.apache.solr.search.ExtendedDismaxQParserPlugin.createParser(ExtendedDismaxQParserPlugin.java:37)
>> at org.apache.solr.search.QParser.getParser(QParser.java:315) at
>>
>> org.apache.solr.handler.component.QueryComponent.prepare(QueryComponent.java:156)
>> at
>>
>> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:196)
>> at
>>
>> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:143)
>> at org.apache.solr.core.SolrCore.execute(SolrCore.java:1984) at
>>
>> Thanks
>>
>> Steve
>>
>> On Thu, May 14, 2015 at 8:00 AM, Emir Arnautovic <
>> emir.arnauto...@sematext.com> wrote:
>>
>>  Hi Steve,
>>> You use qf to limit search on some fields.
>>>
>>> Thanks,
>>> Emir
>>>
>>>
>>> On 14.05.2015 13:49, Steven White wrote:
>>>
>>>  Hi Everyone,

 I'm trying to utilize "uf" but it doesn't work.  My reading of it per:


 https://cwiki.apache.org/confluence/display/solr/The+Extended+DisMax+Query+Parser
 is I can use it to limit into which fields a search is performed.  E.g.:
 If
 my index has F1, F2, F3, F4, and I set "uf" to F3, than I should get
 hits
 on F3 only.

 Here is my request handler:

 

  explicit
  20
  edismax
  F1 F2 F3 F4
  F3
  type,id,score
  xml
  true

 

 I also tried passing "uf" on the URL and it too didn't work.  I'm
 getting
 hits on terms that are in other fields.

 Is my usage of "uf" wrong?

 Thanks

 Steve


  --
>>> Monitoring * Alerting * Anomaly Detection * Centralized Log Management
>>> Solr & Elasticsearch Support * http://sematext.com/
>>>
>>>
>>>
> --
> Monitoring * Alerting * Anomaly Detection * Centralized Log Management
> Solr & Elasticsearch Support * http://sematext.com/
>
>

Re: Cannot get "uf" (user fields) to work

[Emir Arnautovic]

Steve,
Your config should be like:


   
 explicit
 20
 edismax
 F3
 type,id,score
 xml
 true
   


This means when you query with "something" it will be equivalent to F3:something if you 
use standard parser. If you specify uf, than it will allow query to contain that field i.e. even 
you said in qf=F1 F2 F3, if you have in uf=F1, you will be able to define q=F1:something and it 
will search only F1. Otherwise it will not - did not use it so not sure if it will throw parsing 
error or will threat "F1:something" as query and search it in F1, F2 and F3.

Thanks,
Emir



On 14.05.2015 14:37, Steven White wrote:

Hi Emir,

Like I replied to Jack, I don't get it.  I need to see some concrete
examples.

Thanks

Steve

On Thu, May 14, 2015 at 8:30 AM, Emir Arnautovic <
emir.arnauto...@sematext.com> wrote:


Hi Steve,
No - you should use qf to limit search to fields you want. If you do not
plan to allow users to explicitly state fields they want to search, you
don't have to use uf parameter. So, drop uf, and set qf to F3.

Thanks,
Emir


On 14.05.2015 14:09, Steven White wrote:


Hi Emir,

Are you saying I should remove "qf" if I want to use "uf"?  I just tried
that and got exception:

java.lang.RuntimeException at

org.apache.solr.search.ExtendedDismaxQParser$ExtendedDismaxConfiguration.(ExtendedDismaxQParser.java:1497)
at

org.apache.solr.search.ExtendedDismaxQParser.createConfiguration(ExtendedDismaxQParser.java:270)
at

org.apache.solr.search.ExtendedDismaxQParser.(ExtendedDismaxQParser.java:132)
at

org.apache.solr.search.ExtendedDismaxQParserPlugin.createParser(ExtendedDismaxQParserPlugin.java:37)
at org.apache.solr.search.QParser.getParser(QParser.java:315) at

org.apache.solr.handler.component.QueryComponent.prepare(QueryComponent.java:156)
at

org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:196)
at

org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:143)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1984) at

Thanks

Steve

On Thu, May 14, 2015 at 8:00 AM, Emir Arnautovic <
emir.arnauto...@sematext.com> wrote:

  Hi Steve,

You use qf to limit search on some fields.

Thanks,
Emir


On 14.05.2015 13:49, Steven White wrote:

  Hi Everyone,

I'm trying to utilize "uf" but it doesn't work.  My reading of it per:


https://cwiki.apache.org/confluence/display/solr/The+Extended+DisMax+Query+Parser
is I can use it to limit into which fields a search is performed.  E.g.:
If
my index has F1, F2, F3, F4, and I set "uf" to F3, than I should get
hits
on F3 only.

Here is my request handler:

 

  explicit
  20
  edismax
  F1 F2 F3 F4
  F3
  type,id,score
  xml
  true

 

I also tried passing "uf" on the URL and it too didn't work.  I'm
getting
hits on terms that are in other fields.

Is my usage of "uf" wrong?

Thanks

Steve


  --

Monitoring * Alerting * Anomaly Detection * Centralized Log Management
Solr & Elasticsearch Support * http://sematext.com/




--
Monitoring * Alerting * Anomaly Detection * Centralized Log Management
Solr & Elasticsearch Support * http://sematext.com/




--
Monitoring * Alerting * Anomaly Detection * Centralized Log Management
Solr & Elasticsearch Support * http://sematext.com/

Re: Cannot get "uf" (user fields) to work

[Alexandre Rafalovitch]

One example:

https://github.com/arafalov/solr-indexing-book/blob/master/published/languages/conf/solrconfig.xml#L36

This is from my book:
https://www.packtpub.com/big-data-and-business-intelligence/instant-apache-solr-indexing-data-how-instant

But I am not sure if it affects the specific concern you have,
hopefully other emails will.

Regards,
   Alex.


Solr Analyzers, Tokenizers, Filters, URPs and even a newsletter:
http://www.solr-start.com/

On 14 May 2015 at 22:37, Steven White  wrote:
> Hi Emir,
>
> Like I replied to Jack, I don't get it.  I need to see some concrete
> examples.
>
> Thanks
>
> Steve

Re: Transactional Behavior

[Emir Arnautovic]

Hi Amr,
As far as I am aware, SOLR does not support transaction isolation. If 
you really need it, you will have to isolate it yourself. Easiest way is 
to sacrifice concurrent updates.
Other way of achieving it would be to to index in separate indices 
(maybe in memory) concurrently and than merge it with your main index in 
isolation. There are probably better solutions but this is from the top 
of my head.


Thanks,
Emir

On 14.05.2015 14:21, Amr Ali wrote:

Hello guys,

Any expert advice this issue?


--
Regards,
Amr Ali

City stars capital 8 - 3rd floor, Nasr city, Cairo, Egypt
Ext: 278



-Original Message-
From: Amr Ali
Sent: Wednesday, May 13, 2015 10:58 AM
To: solr-user@lucene.apache.org
Subject: RE: Transactional Behavior

Hello Emir,

But this is not a transaction because if some of the bulk I need to add is 
committed; they will be searchable. In a transaction I need to insert a bulk of 
data (all bulk data will be searchable once) or roll it back according to some 
business scenarios.

--
Regards,
Amr Ali

City stars capital 8 - 3rd floor, Nasr city, Cairo, Egypt
Ext: 278



-Original Message-
From: Emir Arnautovic [mailto:emir.arnauto...@sematext.com]
Sent: Tuesday, May 12, 2015 10:46 PM
To: solr-user@lucene.apache.org
Subject: Re: Transactional Behavior

Hi Amr,
One option is to include transaction id in your documents and do delete in case 
of failed transaction. It is not cheap option - additional field if you don't 
have something to use to identify transaction. Assuming rollback will not 
happen to often deleting is not that big issue.

Thanks,
Emir

--
Monitoring * Alerting * Anomaly Detection * Centralized Log Management Solr & 
Elasticsearch Support * http://sematext.com/



On 12.05.2015 22:37, Amr Ali wrote:

Please check this

https://lucene.apache.org/solr/4_1_0/solr-solrj/org/apache/solr/client
/solrj/SolrServer.html#rollback() "Note that this is not a true
rollback as in databases. Content you have previously added may have been committed 
due to autoCommit, buffer full, other client performing a commit etc."

It is not a real rollback if you have two threads T1 and T2 that are adding. If T1 is 
adding 500 and T2 is adding 3 then T2 will commit its 3 document PLUS the documents added 
by T1 (because T2 will finish add/commit before T2 due to the documents number). Solr 
transactions are "server side" only.


--
Regards,
Amr Ali

City stars capital 8 - 3rd floor, Nasr city, Cairo, Egypt
Ext: 278



-Original Message-
From: Jack Krupansky [mailto:jack.krupan...@gmail.com]
Sent: Tuesday, May 12, 2015 10:24 PM
To: solr-user@lucene.apache.org
Subject: Re: Transactional Behavior

Solr does have a  command, but it is an expert feature and not so 
clear how it works in SolrCloud.

See:
https://cwiki.apache.org/confluence/display/solr/Uploading+Data+with+I
ndex+Handlers
and
https://wiki.apache.org/solr/UpdateXmlMessages#A.22rollback.22


-- Jack Krupansky

On Tue, May 12, 2015 at 12:58 PM, Amr Ali  wrote:


Hello,

I have a business case in which I need to be able for the rollback.
When I tried add/commit I was not able to prevent other threads that
write to a given Solr core from committing everything. I also tried
indexwriter but Solr did not get changes until we restart it.


--
Regards,
Amr Ali

City stars capital 8 - 3rd floor, Nasr city, Cairo, Egypt
Ext: 278




--
Monitoring * Alerting * Anomaly Detection * Centralized Log Management Solr & 
Elasticsearch Support * http://sematext.com/



--
Monitoring * Alerting * Anomaly Detection * Centralized Log Management
Solr & Elasticsearch Support * http://sematext.com/

Re: Cannot get "uf" (user fields) to work

[Shawn Heisey]

On 5/14/2015 6:33 AM, Steven White wrote:
> I don't follow.  Can you give me an example on how and when to use "uf"?

The uf parameter controls which fields are allowed in the query string. 
If you have "uf=f1 f2" then "q=f3:foo" won't work the way you would
expect.  If f3 is in the qf parameter, then it will still be used by the
parser.

I tried the equivalent of q=f3:foo on my own index with uf set to fields
that did not include that field.  When a field is disallowed, but used
in the query, there is no error message.  Instead, the data is
interpreted as raw input, so that example would work like "q=f3\:foo"
instead.  The colon becomes part of the actual query text.

Testing shows that uf does *NOT* affect filter queries, only the main
query.  I think this is because filter queries default to the lucene
parser, not the parser specified in defType.

Thanks,
Shawn

Re: Transactional Behavior

[Walter Underwood]

Solr does not do transactions at all. The “commit” is to signal the end of a 
batch load and start the processing.

You can hack some things, but there is no way to roll back a delete or a update.

If you need this, you need to use a database, not a search engine. I recommend 
MarkLogic. It is a full-featured database that includes search features.

wunder
Walter Underwood
wun...@wunderwood.org
http://observer.wunderwood.org/  (my blog)


On May 14, 2015, at 6:12 AM, Emir Arnautovic  
wrote:

> Hi Amr,
> As far as I am aware, SOLR does not support transaction isolation. If you 
> really need it, you will have to isolate it yourself. Easiest way is to 
> sacrifice concurrent updates.
> Other way of achieving it would be to to index in separate indices (maybe in 
> memory) concurrently and than merge it with your main index in isolation. 
> There are probably better solutions but this is from the top of my head.
> 
> Thanks,
> Emir
> 
> On 14.05.2015 14:21, Amr Ali wrote:
>> Hello guys,
>> 
>> Any expert advice this issue?
>> 
>> 
>> --
>> Regards,
>> Amr Ali
>> 
>> City stars capital 8 - 3rd floor, Nasr city, Cairo, Egypt
>> Ext: 278
>> 
>> 
>> 
>> -Original Message-
>> From: Amr Ali
>> Sent: Wednesday, May 13, 2015 10:58 AM
>> To: solr-user@lucene.apache.org
>> Subject: RE: Transactional Behavior
>> 
>> Hello Emir,
>> 
>> But this is not a transaction because if some of the bulk I need to add is 
>> committed; they will be searchable. In a transaction I need to insert a bulk 
>> of data (all bulk data will be searchable once) or roll it back according to 
>> some business scenarios.
>> 
>> --
>> Regards,
>> Amr Ali
>> 
>> City stars capital 8 - 3rd floor, Nasr city, Cairo, Egypt
>> Ext: 278
>> 
>> 
>> 
>> -Original Message-
>> From: Emir Arnautovic [mailto:emir.arnauto...@sematext.com]
>> Sent: Tuesday, May 12, 2015 10:46 PM
>> To: solr-user@lucene.apache.org
>> Subject: Re: Transactional Behavior
>> 
>> Hi Amr,
>> One option is to include transaction id in your documents and do delete in 
>> case of failed transaction. It is not cheap option - additional field if you 
>> don't have something to use to identify transaction. Assuming rollback will 
>> not happen to often deleting is not that big issue.
>> 
>> Thanks,
>> Emir
>> 
>> --
>> Monitoring * Alerting * Anomaly Detection * Centralized Log Management Solr 
>> & Elasticsearch Support * http://sematext.com/
>> 
>> 
>> 
>> On 12.05.2015 22:37, Amr Ali wrote:
>>> Please check this
>>> 
>>> https://lucene.apache.org/solr/4_1_0/solr-solrj/org/apache/solr/client
>>> /solrj/SolrServer.html#rollback() "Note that this is not a true
>>> rollback as in databases. Content you have previously added may have been 
>>> committed due to autoCommit, buffer full, other client performing a commit 
>>> etc."
>>> 
>>> It is not a real rollback if you have two threads T1 and T2 that are 
>>> adding. If T1 is adding 500 and T2 is adding 3 then T2 will commit its 3 
>>> document PLUS the documents added by T1 (because T2 will finish add/commit 
>>> before T2 due to the documents number). Solr transactions are "server side" 
>>> only.
>>> 
>>> 
>>> --
>>> Regards,
>>> Amr Ali
>>> 
>>> City stars capital 8 - 3rd floor, Nasr city, Cairo, Egypt
>>> Ext: 278
>>> 
>>> 
>>> 
>>> -Original Message-
>>> From: Jack Krupansky [mailto:jack.krupan...@gmail.com]
>>> Sent: Tuesday, May 12, 2015 10:24 PM
>>> To: solr-user@lucene.apache.org
>>> Subject: Re: Transactional Behavior
>>> 
>>> Solr does have a  command, but it is an expert feature and not 
>>> so clear how it works in SolrCloud.
>>> 
>>> See:
>>> https://cwiki.apache.org/confluence/display/solr/Uploading+Data+with+I
>>> ndex+Handlers
>>> and
>>> https://wiki.apache.org/solr/UpdateXmlMessages#A.22rollback.22
>>> 
>>> 
>>> -- Jack Krupansky
>>> 
>>> On Tue, May 12, 2015 at 12:58 PM, Amr Ali  wrote:
>>> 
 Hello,
 
 I have a business case in which I need to be able for the rollback.
 When I tried add/commit I was not able to prevent other threads that
 write to a given Solr core from committing everything. I also tried
 indexwriter but Solr did not get changes until we restart it.
 
 
 --
 Regards,
 Amr Ali
 
 City stars capital 8 - 3rd floor, Nasr city, Cairo, Egypt
 Ext: 278
 
 
 
>> --
>> Monitoring * Alerting * Anomaly Detection * Centralized Log Management Solr 
>> & Elasticsearch Support * http://sematext.com/
>> 
> 
> -- 
> Monitoring * Alerting * Anomaly Detection * Centralized Log Management
> Solr & Elasticsearch Support * http://sematext.com/
> 

Re: Transactional Behavior

[Shawn Heisey]

On 5/12/2015 10:58 AM, Amr Ali wrote:
> I have a business case in which I need to be able for the rollback. When I 
> tried add/commit I was not able to prevent other threads that write to a 
> given Solr core from committing everything. I also tried indexwriter but Solr 
> did not get changes until we restart it.

This is how Lucene is designed.  The bulk of Solr's functionality is
provided directly by Lucene, including the way that commits work.

Adding transaction support with individual rollback capability would be
a VERY fundamental and low-level change to Lucene.  I bet you would find
that major pieces of Lucene functionality are dependent on the current
non-transactional design for commits, which means that changing it would
probably require changes in every major subsystem found in the code, and
that general performance might be very adversely affected.  Verifying
correctness and fixing problems could take weeks or months.

I don't think the benefit is worth the pain, especially when there's a
simple workaround -- don't overlap changes that need to be treated as
different transactions -- do them at different times.

Thanks,
Shawn

Re: Real-Time get and Dynamic Fields: possible bug.

[Yonik Seeley]

Are the _facet fields the target of a copyField in the schema?
Realtime get either gets the values from the transaction log (and if
you didn't send it the values, they won't be there) or gets them from
the index to try and reconstruct what was sent in.

It's generally not recommended to have copyField targets "stored", or
have a mix of explicitly set values and copyField values in the same
field.

-Yonik

On Thu, May 14, 2015 at 7:17 AM, Luis Cappa Banda  wrote:
> Hi there,
>
> I have the following dynamicFields definition in my schema.xml:
>
>
> 
>
>"true" stored="true" multiValued="true" />
>
>
> I' ve seen that when fetching documents with /select?q=id:whateverId, the
> results returned include both i18n* and *_facet fields filled. However,
> when using real-time request handler (/get?ids:whateverIds) the result
> fetched include only i18n* dynamic fields, but *_facet ones are not
> included.
>
> I have the impression during /get RequestHandler the server-side regular
> expression used when parsing fields and fields values to return documents
> with existing dynamic fields seems to be wrong. From the client side, I' ve
> checked that the class DocField.java that parses SolrDocument to Bean ones
> uses the following matcher:
>
>  } else if (annotation.value().indexOf('*') >= 0) { // dynamic fields are
> annotated as @Field("categories_*")
>
> // if the field was annotated as a dynamic field, convert the name into a
> pattern
>
> // the wildcard (*) is supposed to be either a prefix or a suffix, hence
> the use of replaceFirst
>
> name = annotation.value().replaceFirst("\\*", "\\.*");
>
> dynamicFieldNamePatternMatcher = Pattern.compile("^" + name + "$");
>
>  } else {
>
> name = annotation.value();
>
>  }
>
> So maybe a similar behavior from the server-side is wrong. That' s the only
> reason I find to understand why when using /select all fields are returned
> but when using /get those that matches *_facet regexp are not.
>
> If you can confirm that this is a bug (because maybe is the expected
> behavior, but after some years using Solr I think it is not) I can create
> the JIRA issue and debug it more deeply to apply a patch with the aim to
> help.
>
>
> Regards,
>
>
> --
> - Luis Cappa

Re: Real-Time get and Dynamic Fields: possible bug.

[Luis Cappa Banda]

Ehem, *_target ---> *_facet.

2015-05-14 16:47 GMT+02:00 Luis Cappa Banda :

> Hi Yonik,
>
> Yes, they are the target from copyFields in the schema.xml. This *_target
> fields are suposed to be used in some specific searchable (thus, tokenized)
> fields that in the future are candidates to be faceted to return some
> stats. For example, imagine that you have a field storing a directory path
> and you want to search by. Also, you may want to facet by the whole
> directory path value (not just their terms). Thats why I' m storing both
> field values: searchable and tokenized one, string and 'facet candidate'
> one.
>
> What I do not understand is that both i18n* and *_target are dynamic,
> indexed and stored values. The only difference is that *_target one is
> multivalued. Does it have some sense?
>
>
> Regards
>
>
> - Luis Cappa
>
> 2015-05-14 16:42 GMT+02:00 Yonik Seeley :
>
>> Are the _facet fields the target of a copyField in the schema?
>> Realtime get either gets the values from the transaction log (and if
>> you didn't send it the values, they won't be there) or gets them from
>> the index to try and reconstruct what was sent in.
>>
>> It's generally not recommended to have copyField targets "stored", or
>> have a mix of explicitly set values and copyField values in the same
>> field.
>>
>> -Yonik
>>
>> On Thu, May 14, 2015 at 7:17 AM, Luis Cappa Banda 
>> wrote:
>> > Hi there,
>> >
>> > I have the following dynamicFields definition in my schema.xml:
>> >
>> >
>> > 
>> >
>> > > />  > indexed=
>> > "true" stored="true" multiValued="true" />
>> >
>> >
>> > I' ve seen that when fetching documents with /select?q=id:whateverId,
>> the
>> > results returned include both i18n* and *_facet fields filled. However,
>> > when using real-time request handler (/get?ids:whateverIds) the result
>> > fetched include only i18n* dynamic fields, but *_facet ones are not
>> > included.
>> >
>> > I have the impression during /get RequestHandler the server-side regular
>> > expression used when parsing fields and fields values to return
>> documents
>> > with existing dynamic fields seems to be wrong. From the client side,
>> I' ve
>> > checked that the class DocField.java that parses SolrDocument to Bean
>> ones
>> > uses the following matcher:
>> >
>> >  } else if (annotation.value().indexOf('*') >= 0) { // dynamic fields
>> are
>> > annotated as @Field("categories_*")
>> >
>> > // if the field was annotated as a dynamic field, convert the name into
>> a
>> > pattern
>> >
>> > // the wildcard (*) is supposed to be either a prefix or a suffix, hence
>> > the use of replaceFirst
>> >
>> > name = annotation.value().replaceFirst("\\*", "\\.*");
>> >
>> > dynamicFieldNamePatternMatcher = Pattern.compile("^" + name + "$");
>> >
>> >  } else {
>> >
>> > name = annotation.value();
>> >
>> >  }
>> >
>> > So maybe a similar behavior from the server-side is wrong. That' s the
>> only
>> > reason I find to understand why when using /select all fields are
>> returned
>> > but when using /get those that matches *_facet regexp are not.
>> >
>> > If you can confirm that this is a bug (because maybe is the expected
>> > behavior, but after some years using Solr I think it is not) I can
>> create
>> > the JIRA issue and debug it more deeply to apply a patch with the aim to
>> > help.
>> >
>> >
>> > Regards,
>> >
>> >
>> > --
>> > - Luis Cappa
>>
>
>
>
> --
> - Luis Cappa
>



-- 
- Luis Cappa

Re: Real-Time get and Dynamic Fields: possible bug.

[Luis Cappa Banda]

Hi Yonik,

Yes, they are the target from copyFields in the schema.xml. This *_target
fields are suposed to be used in some specific searchable (thus, tokenized)
fields that in the future are candidates to be faceted to return some
stats. For example, imagine that you have a field storing a directory path
and you want to search by. Also, you may want to facet by the whole
directory path value (not just their terms). Thats why I' m storing both
field values: searchable and tokenized one, string and 'facet candidate'
one.

What I do not understand is that both i18n* and *_target are dynamic,
indexed and stored values. The only difference is that *_target one is
multivalued. Does it have some sense?


Regards


- Luis Cappa

2015-05-14 16:42 GMT+02:00 Yonik Seeley :

> Are the _facet fields the target of a copyField in the schema?
> Realtime get either gets the values from the transaction log (and if
> you didn't send it the values, they won't be there) or gets them from
> the index to try and reconstruct what was sent in.
>
> It's generally not recommended to have copyField targets "stored", or
> have a mix of explicitly set values and copyField values in the same
> field.
>
> -Yonik
>
> On Thu, May 14, 2015 at 7:17 AM, Luis Cappa Banda 
> wrote:
> > Hi there,
> >
> > I have the following dynamicFields definition in my schema.xml:
> >
> >
> > 
> >
> > 
>   indexed=
> > "true" stored="true" multiValued="true" />
> >
> >
> > I' ve seen that when fetching documents with /select?q=id:whateverId, the
> > results returned include both i18n* and *_facet fields filled. However,
> > when using real-time request handler (/get?ids:whateverIds) the result
> > fetched include only i18n* dynamic fields, but *_facet ones are not
> > included.
> >
> > I have the impression during /get RequestHandler the server-side regular
> > expression used when parsing fields and fields values to return documents
> > with existing dynamic fields seems to be wrong. From the client side, I'
> ve
> > checked that the class DocField.java that parses SolrDocument to Bean
> ones
> > uses the following matcher:
> >
> >  } else if (annotation.value().indexOf('*') >= 0) { // dynamic fields are
> > annotated as @Field("categories_*")
> >
> > // if the field was annotated as a dynamic field, convert the name into a
> > pattern
> >
> > // the wildcard (*) is supposed to be either a prefix or a suffix, hence
> > the use of replaceFirst
> >
> > name = annotation.value().replaceFirst("\\*", "\\.*");
> >
> > dynamicFieldNamePatternMatcher = Pattern.compile("^" + name + "$");
> >
> >  } else {
> >
> > name = annotation.value();
> >
> >  }
> >
> > So maybe a similar behavior from the server-side is wrong. That' s the
> only
> > reason I find to understand why when using /select all fields are
> returned
> > but when using /get those that matches *_facet regexp are not.
> >
> > If you can confirm that this is a bug (because maybe is the expected
> > behavior, but after some years using Solr I think it is not) I can create
> > the JIRA issue and debug it more deeply to apply a patch with the aim to
> > help.
> >
> >
> > Regards,
> >
> >
> > --
> > - Luis Cappa
>



-- 
- Luis Cappa

Re: Searcher is opening twice on Reload

[Aman Tandon]

Please help.

The solr version is 4.8.1

With Regards
Aman Tandon

On Thu, May 14, 2015 at 12:53 PM, Aman Tandon 
wrote:

> Hi,
>
> Please help me here, when I am doing the reload of core, my searcher is
> being opening twice. I am also attaching the logs output, please suggest me
> what wrong I am doing here or this is default behavior on reload.
>
> May 14, 2015 12:47:38 PM org.apache.solr.spelling.DirectSolrSpellChecker
>> INFO: init:
>> {name=default,field=titlews,classname=solr.DirectSolrSpellChecker,distanceMeasure=internal,accuracy=0.5,maxEdits=1,minPrefix=1,maxInspections=5,minQueryLength=5,maxQueryFrequency=100.0,thresholdTokenFrequency=100.0}
>> May 14, 2015 12:47:38 PM
>> org.apache.solr.handler.component.SpellCheckComponent
>> INFO: No queryConverter defined, using default converter
>> May 14, 2015 12:47:38 PM
>> org.apache.solr.handler.component.QueryElevationComponent
>> INFO: Loading QueryElevation from data dir: elevate.xml
>> May 14, 2015 12:47:38 PM org.apache.solr.handler.ReplicationHandler
>> INFO: Commits will be reserved for  1
>> May 14, 2015 12:47:38 PM org.apache.solr.core.QuerySenderListener
>> INFO: QuerySenderListener sending requests to Searcher@41dc3c83[IM-Search]
>> main{StandardDirectoryReader(segments_dd4:82296:nrt
>> _jdq(4.8):C5602938/2310052:delGen=3132
>> _jkq(4.8):C6860454/1398005:delGen=2992
>> _jx2(4.8):C5237053/1505048:delGen=3241
>> _joo(4.8):C5825253/1599671:delGen=3323
>> _k4d(4.8):C5860360/1916531:delGen=3150
>> _o27(4.8):C5290435/1018865:delGen=370
>> _mju(4.8):C5074973/1602707:delGen=1474
>> _jka(4.8):C5172599/1774839:delGen=3202
>> _nik(4.8):C4698916/1512091:delGen=804 _o8y(4.8):C1137592/521423:delGen=190
>> _oeu(4.8):C469094/86291:delGen=29 _odq(4.8):C217505/65596:delGen=55
>> _ogd(4.8):C50454/4155:delGen=5 _oea(4.8):C40833/7192:delGen=37
>> _ofy(4.8):C73614/7273:delGen=13 _ogx(4.8):C395681/1388:delGen=4
>> _ogh(4.8):C7676/70:delGen=2 _ohf(4.8):C108769/21:delGen=2
>> _ogc(4.8):C24435/384:delGen=4 _ogi(4.8):C23088/158:delGen=3
>> _ogj(4.8):C4217/2:delGen=1 _ohs(4.8):C7 _oh6(4.8):C20509/205:delGen=5
>> _oh7(4.8):C3171 _oho(4.8):C6/1:delGen=1 _ohq(4.8):C1
>> _ohv(4.8):C10484/996:delGen=2 _ohx(4.8):C500 _ohy(4.8):C1 _ohz(4.8):C1)}
>> ^[OFMay 14, 2015 12:47:43 PM org.apache.solr.core.SolrCore
>> INFO: [IM-Search] webapp=/solr path=/select
>> params={spellcheck=true&lon=0&q=q&wt=json&qt=opsview.monitor&lat=0&rows=0&ps=1}
>> hits=6 status=0 QTime=1
>> May 14, 2015 12:47:44 PM org.apache.solr.core.SolrCore
>> INFO: [IM-Search] webapp=null path=null
>> params={start=0&event=firstSearcher&q=rice&distrib=false&qt=im.search.intent&rows=25}
>> hits=42749 status=0 QTime=5667
>> May 14, 2015 12:47:58 PM org.apache.solr.request.UnInvertedField
>> INFO: UnInverted multi-valued field
>> {field=city,memSize=209216385,tindexSize=11029,time=3904,phase1=3783,nTerms=77614,bigTerms=3,termInstances=31291566,uses=0}
>> May 14, 2015 12:48:01 PM org.apache.solr.request.UnInvertedField
>> INFO: UnInverted multi-valued field
>> {field=biztype,memSize=208847178,tindexSize=40,time=1318,phase1=1193,nTerms=9,bigTerms=4,termInstances=1607459,uses=0}
>> May 14, 2015 12:48:01 PM org.apache.solr.core.SolrCore
>> INFO: [IM-Search] webapp=null path=null
>> params={start=0&event=firstSearcher&q=rice&distrib=false&qt=im.search&rows=25}
>> hits=57619 status=0 QTime=17194
>> May 14, 2015 12:48:04 PM org.apache.solr.core.SolrCore
>> INFO: [IM-Search] webapp=null path=null
>> params={start=0&event=firstSearcher&q=potassium+cyanide&distrib=false&qt=eto.search.offer&rows=20}
>> hits=443 status=0 QTime=3272
>> May 14, 2015 12:48:09 PM org.apache.solr.core.SolrCore
>> INFO: [IM-Search] webapp=null path=null
>> params={start=0&event=firstSearcher&q=motor+spare+parts&distrib=false&qt=im.search&fq=attribs:(locprefglobal+locprefnational+locprefcity)&rows=20}
>> hits=107297 status=0 QTime=5254
>> May 14, 2015 12:48:09 PM org.apache.solr.core.QuerySenderListener
>> INFO: QuerySenderListener done.
>> May 14, 2015 12:48:09 PM
>> org.apache.solr.handler.component.SpellCheckComponent$SpellCheckerListener
>> INFO: Loading spell index for spellchecker: default
>> May 14, 2015 12:48:09 PM
>> org.apache.solr.handler.component.SpellCheckComponent$SpellCheckerListener
>> INFO: Loading spell index for spellchecker: wordbreak
>> May 14, 2015 12:48:09 PM org.apache.solr.core.SolrCore
>> INFO: [IM-Search] Registered new searcher Searcher@41dc3c83[IM-Search]
>> main{StandardDirectoryReader(segments_dd4:82296:nrt
>> _jdq(4.8):C5602938/2310052:delGen=3132
>> _jkq(4.8):C6860454/1398005:delGen=2992
>> _jx2(4.8):C5237053/1505048:delGen=3241
>> _joo(4.8):C5825253/1599671:delGen=3323
>> _k4d(4.8):C5860360/1916531:delGen=3150
>> _o27(4.8):C5290435/1018865:delGen=370
>> _mju(4.8):C5074973/1602707:delGen=1474
>> _jka(4.8):C5172599/1774839:delGen=3202
>> _nik(4.8):C4698916/1512091:delGen=804 _o8y(4.8):C1137592/521423:delGen=190
>> _oeu(4.8):C469094/86291:delGen=29 _odq(4.8):C217505/65596:delGen=55
>> _ogd(4.8):C50454/4155:delGen=5

High fieldNorm values causing really odd results

[Jorge Luis Betancourt González]

Hi everyone:

>From the last couple of week I'm noting some really odd results in my Solr 
>server, searching for the root cause the one thing I can point out is a very 
>high value of the fieldNorm parameter in the score calculation, an snippet of 
>the debug info:

{
   "match":true,
   "value":4907799.5,
   "description":"fieldWeight in 5316, product of:",
   "details":[
  {
 "match":true,
 "value":2,
 "description":"tf(freq=4.0), with freq of:",
 "details":[
{
   "match":true,
   "value":4,
   "description":"termFreq=4.0"
}
 ]
  },
  {
 "match":true,
 "value":3.7443538,
 "description":"idf(docFreq=49449, maxDocs=769173)"
  },
  {
 "match":true,
 "value":655360,
 "description":"fieldNorm(doc=5316)"
  }
   ]
}


This match is in the "title" field, which has 119669 total terms (which isn't 
such big number) and the total document count in this index is also rather 
small: 613772. 

>From [1] the fieldNorm encapsulates several parameters in its calculation:
  - Field boost (not used by us)
  - lengthNorm (which I'm suspecting is the culprit) 

I'm thinking on using omitNorms, but first I would like to discover why is this 
happening.

Any help on this issue?

Regards,

A Synonym Searching for Phrase?

[Ryan Yacyshyn]

Hi All,

I'm running into an issue where I have some tokens that really mean the
same thing as two. For example, there are a couple ways users might want to
search for certain type of visa called the "s pass", but they might query
for spass or s-pass.

I thought I could add a line in my synonym file to solve this, such as:

s-pass, spass => s pass

This doesn't seem to work. I found an Auto Phrase TokenFilter (
https://github.com/LucidWorks/auto-phrase-tokenfilter) that looks like it
might help, but it sounds like it needs to use a specific query parser as
well (we're using edismax).

Has anyone came across this specific problem before? Would really
appreciate your suggestions / help.

We're using Solr 4.8.x (and lucidWorks 2.9).

Thanks!
Ryan

Re: Real-Time get and Dynamic Fields: possible bug.

[Yonik Seeley]

On Thu, May 14, 2015 at 10:47 AM, Luis Cappa Banda  wrote:
> Hi Yonik,
>
> Yes, they are the target from copyFields in the schema.xml. This *_target
> fields are suposed to be used in some specific searchable (thus, tokenized)
> fields that in the future are candidates to be faceted to return some
> stats. For example, imagine that you have a field storing a directory path
> and you want to search by. Also, you may want to facet by the whole
> directory path value (not just their terms). Thats why I' m storing both
> field values: searchable and tokenized one, string and 'facet candidate'
> one.

OK, but you don't need to *store* the values in _facet, right?
-Yonik

Re: Real-Time get and Dynamic Fields: possible bug.

[Luis Cappa Banda]

That is something I didin' t know, but I thought it was mandatory. I' ll
try to explain step by step my (I think) logical way to understand it:

   - If a field is indexed, you can search by it.
   - When faceting, you have to index the field (because it can be
   tokenized and then you would like to facet by their terms). Then, you need
   to mark as indexed those fields you want to facet by.
   - If you mark as stored a field, you can return its value with the
   'original value' it was stored.
   - If you facet, you are searching, counting terms and returning values
   and their counters. Thus, that "returning their values" step is what I
   thought where 'stored=true' was necessary.

If you don' t mark as stored a field indexed and 'facetable', I was
expecting to not be able to return their values, so faceting has no sense.
Thats what I thought, of course. If it is not necessary, thats perfect: the
lighter the data, the better, and one more thing I' ve learned, :-)

Anyway, I think that the question is still open: both are dynamic fields,
stored (it is not necessary, OK) and indexed. When applying real time
requestHandler, i18n* dynamic fields are returned but those *_facet are
not. However, when applying the default /select requestHandler and finding
by the document id, both i18n* and *_facet fields are returned. You can try
it with Solr 5.1, the version I' m currently using.

The only differences between them are:

   - Regular expression: i18n* VS *_facet
   - Multivalued: *_facet are multivalued.


Regards,


- Luis Cappa

2015-05-14 18:32 GMT+02:00 Yonik Seeley :

> On Thu, May 14, 2015 at 10:47 AM, Luis Cappa Banda 
> wrote:
> > Hi Yonik,
> >
> > Yes, they are the target from copyFields in the schema.xml. This *_target
> > fields are suposed to be used in some specific searchable (thus,
> tokenized)
> > fields that in the future are candidates to be faceted to return some
> > stats. For example, imagine that you have a field storing a directory
> path
> > and you want to search by. Also, you may want to facet by the whole
> > directory path value (not just their terms). Thats why I' m storing both
> > field values: searchable and tokenized one, string and 'facet candidate'
> > one.
>
> OK, but you don't need to *store* the values in _facet, right?
> -Yonik
>



-- 
- Luis Cappa

Re: Real-Time get and Dynamic Fields: possible bug.

[Yonik Seeley]

On Thu, May 14, 2015 at 12:49 PM, Luis Cappa Banda  wrote:
> If you don' t mark as stored a field indexed and 'facetable', I was
> expecting to not be able to return their values, so faceting has no sense.

Faceting does not use or retrieve stored field values.  The labels
faceting returns are from the indexed values.

"If you want the value returned, it needs to be stored" only applies
to fields in the main document list (the fields that are retrieved for
the top ranked documents).

-Yonik

Re: Real-Time get and Dynamic Fields: possible bug.

[Luis Cappa Banda]

Yep, but those dynamic fields had a field type "string", so the unique
indexed therm will be the entire field value and the faceted terms counted
will match with exactly with each field value. Thats why I was confused.
Typically I use faceting with string non tokenized field values for simple
stats and this kind of things.

Do you think the behavior explained (I mean, ghost dynamic field values
when using real-time request handler) can be a bug? I don' t mind
investigating it this weekend and trying to patch it.

2015-05-14 18:59 GMT+02:00 Yonik Seeley :

> On Thu, May 14, 2015 at 12:49 PM, Luis Cappa Banda 
> wrote:
> > If you don' t mark as stored a field indexed and 'facetable', I was
> > expecting to not be able to return their values, so faceting has no
> sense.
>
> Faceting does not use or retrieve stored field values.  The labels
> faceting returns are from the indexed values.
>
> "If you want the value returned, it needs to be stored" only applies
> to fields in the main document list (the fields that are retrieved for
> the top ranked documents).
>
> -Yonik
>



-- 
- Luis Cappa

Re: A Synonym Searching for Phrase?

[Alessandro Benedetti]

There was a similar scission few days ago, take a look here :

I found this very interesting article that I think can help in better
understanding the problem :
http://lucidworks.com/blog/solution-for-multi-term-synonyms-in-lucenesolr-using-the-auto-phrasing-tokenfilter/

And this :
http://opensourceconnections.com/blog/2013/10/27/why-is-multi-term-synonyms-so-hard-in-solr/

Take a look and let me know !

2015-05-14 17:05 GMT+01:00 Ryan Yacyshyn :

> Hi All,
>
> I'm running into an issue where I have some tokens that really mean the
> same thing as two. For example, there are a couple ways users might want to
> search for certain type of visa called the "s pass", but they might query
> for spass or s-pass.
>
> I thought I could add a line in my synonym file to solve this, such as:
>
> s-pass, spass => s pass
>
> This doesn't seem to work. I found an Auto Phrase TokenFilter (
> https://github.com/LucidWorks/auto-phrase-tokenfilter) that looks like it
> might help, but it sounds like it needs to use a specific query parser as
> well (we're using edismax).
>
> Has anyone came across this specific problem before? Would really
> appreciate your suggestions / help.
>
> We're using Solr 4.8.x (and lucidWorks 2.9).
>
> Thanks!
> Ryan
>



-- 
--

Benedetti Alessandro
Visiting card : http://about.me/alessandro_benedetti

"Tyger, tyger burning bright
In the forests of the night,
What immortal hand or eye
Could frame thy fearful symmetry?"

William Blake - Songs of Experience -1794 England

Re: Searcher is opening twice on Reload

[Chris Hostetter]

I suspect you aren't doing anything wrong, i think it's the same as this 
bug...

https://issues.apache.org/jira/browse/SOLR-7035


: Date: Thu, 14 May 2015 12:53:34 +0530
: From: Aman Tandon 
: Reply-To: solr-user@lucene.apache.org
: To: "solr-user@lucene.apache.org" 
: Subject: Searcher is opening twice on Reload
: 
: Hi,
: 
: Please help me here, when I am doing the reload of core, my searcher is
: being opening twice. I am also attaching the logs output, please suggest me
: what wrong I am doing here or this is default behavior on reload.
: 
: May 14, 2015 12:47:38 PM org.apache.solr.spelling.DirectSolrSpellChecker
: > INFO: init:
: > 
{name=default,field=titlews,classname=solr.DirectSolrSpellChecker,distanceMeasure=internal,accuracy=0.5,maxEdits=1,minPrefix=1,maxInspections=5,minQueryLength=5,maxQueryFrequency=100.0,thresholdTokenFrequency=100.0}
: > May 14, 2015 12:47:38 PM
: > org.apache.solr.handler.component.SpellCheckComponent
: > INFO: No queryConverter defined, using default converter
: > May 14, 2015 12:47:38 PM
: > org.apache.solr.handler.component.QueryElevationComponent
: > INFO: Loading QueryElevation from data dir: elevate.xml
: > May 14, 2015 12:47:38 PM org.apache.solr.handler.ReplicationHandler
: > INFO: Commits will be reserved for  1
: > May 14, 2015 12:47:38 PM org.apache.solr.core.QuerySenderListener
: > INFO: QuerySenderListener sending requests to Searcher@41dc3c83[IM-Search]
: > main{StandardDirectoryReader(segments_dd4:82296:nrt
: > _jdq(4.8):C5602938/2310052:delGen=3132
: > _jkq(4.8):C6860454/1398005:delGen=2992
: > _jx2(4.8):C5237053/1505048:delGen=3241
: > _joo(4.8):C5825253/1599671:delGen=3323
: > _k4d(4.8):C5860360/1916531:delGen=3150
: > _o27(4.8):C5290435/1018865:delGen=370
: > _mju(4.8):C5074973/1602707:delGen=1474
: > _jka(4.8):C5172599/1774839:delGen=3202
: > _nik(4.8):C4698916/1512091:delGen=804 _o8y(4.8):C1137592/521423:delGen=190
: > _oeu(4.8):C469094/86291:delGen=29 _odq(4.8):C217505/65596:delGen=55
: > _ogd(4.8):C50454/4155:delGen=5 _oea(4.8):C40833/7192:delGen=37
: > _ofy(4.8):C73614/7273:delGen=13 _ogx(4.8):C395681/1388:delGen=4
: > _ogh(4.8):C7676/70:delGen=2 _ohf(4.8):C108769/21:delGen=2
: > _ogc(4.8):C24435/384:delGen=4 _ogi(4.8):C23088/158:delGen=3
: > _ogj(4.8):C4217/2:delGen=1 _ohs(4.8):C7 _oh6(4.8):C20509/205:delGen=5
: > _oh7(4.8):C3171 _oho(4.8):C6/1:delGen=1 _ohq(4.8):C1
: > _ohv(4.8):C10484/996:delGen=2 _ohx(4.8):C500 _ohy(4.8):C1 _ohz(4.8):C1)}
: > ^[OFMay 14, 2015 12:47:43 PM org.apache.solr.core.SolrCore
: > INFO: [IM-Search] webapp=/solr path=/select
: > 
params={spellcheck=true&lon=0&q=q&wt=json&qt=opsview.monitor&lat=0&rows=0&ps=1}
: > hits=6 status=0 QTime=1
: > May 14, 2015 12:47:44 PM org.apache.solr.core.SolrCore
: > INFO: [IM-Search] webapp=null path=null
: > 
params={start=0&event=firstSearcher&q=rice&distrib=false&qt=im.search.intent&rows=25}
: > hits=42749 status=0 QTime=5667
: > May 14, 2015 12:47:58 PM org.apache.solr.request.UnInvertedField
: > INFO: UnInverted multi-valued field
: > 
{field=city,memSize=209216385,tindexSize=11029,time=3904,phase1=3783,nTerms=77614,bigTerms=3,termInstances=31291566,uses=0}
: > May 14, 2015 12:48:01 PM org.apache.solr.request.UnInvertedField
: > INFO: UnInverted multi-valued field
: > 
{field=biztype,memSize=208847178,tindexSize=40,time=1318,phase1=1193,nTerms=9,bigTerms=4,termInstances=1607459,uses=0}
: > May 14, 2015 12:48:01 PM org.apache.solr.core.SolrCore
: > INFO: [IM-Search] webapp=null path=null
: > 
params={start=0&event=firstSearcher&q=rice&distrib=false&qt=im.search&rows=25}
: > hits=57619 status=0 QTime=17194
: > May 14, 2015 12:48:04 PM org.apache.solr.core.SolrCore
: > INFO: [IM-Search] webapp=null path=null
: > 
params={start=0&event=firstSearcher&q=potassium+cyanide&distrib=false&qt=eto.search.offer&rows=20}
: > hits=443 status=0 QTime=3272
: > May 14, 2015 12:48:09 PM org.apache.solr.core.SolrCore
: > INFO: [IM-Search] webapp=null path=null
: > 
params={start=0&event=firstSearcher&q=motor+spare+parts&distrib=false&qt=im.search&fq=attribs:(locprefglobal+locprefnational+locprefcity)&rows=20}
: > hits=107297 status=0 QTime=5254
: > May 14, 2015 12:48:09 PM org.apache.solr.core.QuerySenderListener
: > INFO: QuerySenderListener done.
: > May 14, 2015 12:48:09 PM
: > org.apache.solr.handler.component.SpellCheckComponent$SpellCheckerListener
: > INFO: Loading spell index for spellchecker: default
: > May 14, 2015 12:48:09 PM
: > org.apache.solr.handler.component.SpellCheckComponent$SpellCheckerListener
: > INFO: Loading spell index for spellchecker: wordbreak
: > May 14, 2015 12:48:09 PM org.apache.solr.core.SolrCore
: > INFO: [IM-Search] Registered new searcher Searcher@41dc3c83[IM-Search]
: > main{StandardDirectoryReader(segments_dd4:82296:nrt
: > _jdq(4.8):C5602938/2310052:delGen=3132
: > _jkq(4.8):C6860454/1398005:delGen=2992
: > _jx2(4.8):C5237053/1505048:delGen=3241
: > _joo(4.8):C5825253/1599671:delGen=3323
: > _k4d(4.8):C5860360/1916531:delGen=3150
: > _o27(4.8):C5290435/101

re: A Synonym Searching for Phrase?

[Chris Morley]

I have implemented that but it's not open sourced yet.  It will be soon.
  
 -Chris.
  
  
  


 From: "Ryan Yacyshyn" 
Sent: Thursday, May 14, 2015 12:07 PM
To: solr-user@lucene.apache.org
Subject: A Synonym Searching for Phrase?   
Hi All,

I'm running into an issue where I have some tokens that really mean the
same thing as two. For example, there are a couple ways users might want 
to
search for certain type of visa called the "s pass", but they might query
for spass or s-pass.

I thought I could add a line in my synonym file to solve this, such as:

s-pass, spass => s pass

This doesn't seem to work. I found an Auto Phrase TokenFilter (
https://github.com/LucidWorks/auto-phrase-tokenfilter) that looks like it
might help, but it sounds like it needs to use a specific query parser as
well (we're using edismax).

Has anyone came across this specific problem before? Would really
appreciate your suggestions / help.

We're using Solr 4.8.x (and lucidWorks 2.9).

Thanks!
Ryan
 

Re: Cannot get "uf" (user fields) to work

[Steven White]

Thanks Shawn and all for the insight.  This was very helpful.

Steve

On Thu, May 14, 2015 at 10:19 AM, Shawn Heisey  wrote:

> On 5/14/2015 6:33 AM, Steven White wrote:
> > I don't follow.  Can you give me an example on how and when to use "uf"?
>
> The uf parameter controls which fields are allowed in the query string.
> If you have "uf=f1 f2" then "q=f3:foo" won't work the way you would
> expect.  If f3 is in the qf parameter, then it will still be used by the
> parser.
>
> I tried the equivalent of q=f3:foo on my own index with uf set to fields
> that did not include that field.  When a field is disallowed, but used
> in the query, there is no error message.  Instead, the data is
> interpreted as raw input, so that example would work like "q=f3\:foo"
> instead.  The colon becomes part of the actual query text.
>
> Testing shows that uf does *NOT* affect filter queries, only the main
> query.  I think this is because filter queries default to the lucene
> parser, not the parser specified in defType.
>
> Thanks,
> Shawn
>
>

Re: Suggester

[lreg]

Could you please explain how you solved the problem I am having the same
issue :(



--
View this message in context: 
http://lucene.472066.n3.nabble.com/Suggester-tp4201287p4205460.html
Sent from the Solr - User mailing list archive at Nabble.com.

NPE when Faceting with MoreLikeThis handler in Solr 5.1.0

[Tim Hearn]

Hi everyone,

Recently I upgraded to solr 5.1.0.  When trying to generate facets using
the more like this handler, I now get a a NullPointerException.  I never
got this exception while using Solr 4.10.0 Details are below:

Stack Trace:
at
org.apache.solr.request.SimpleFacets.getHeatmapCounts(SimpleFacets.java:1555)
at
org.apache.solr.request.SimpleFacets.getFacetCounts(SimpleFacets.java:284)
at
org.apache.solr.handler.MoreLikeThisHandler.handleRequestBody(MoreLikeThisHandler.java:233)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:143)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1984)
at
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:829)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:446)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:220)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
at
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557)
at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1075)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
at
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255)
at
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
at org.eclipse.jetty.server.Server.handle(Server.java:368)
at
org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
at
org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)
at
org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:942)
at
org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1004)
at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:640)
at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
at
org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)
at
org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)
at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
at
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
at java.lang.Thread.run(Thread.java:745)


Query:
qt=/mlt&
q=id:545dbb57b54c2403f286050e546dcdcab54cf2d074e5a2f7&
mlt.mindf=5&
mlt.mintf=1&
mlt.minwl=3&
mlt.boost=true&
fq=storeid:546dcdcab54cf2d074e5a2f7&
mlt.fl=overview_mlt,abstract_mlt,description_mlt,company_profile_mlt,bio_mlt&
mlt.interestingTerms=details&
fl=conceptid,score
&sort=score desc&
start=0&
rows=2&
facet=true&
facet.field=tags&
facet.field=locations&
facet.mincount=1&
facet.method=enum&
facet.limit=-1&
facet.sort=count

Schema.xml(relevant parts):
   

   

   


solrconfig.xml(relevant parts):
  
  

Re: High fieldNorm values causing really odd results

[Chris Hostetter]

:   {
:  "match":true,
:  "value":655360,
:  "description":"fieldNorm(doc=5316)"
:   }
...
: This match is in the "title" field, which has 119669 total terms (which 
: isn't such big number) and the total document count in this index is 

that smells like a bug -- by the looks of it an overflow bug?

can you please provide some details on the version of solr you are using, 
and the specifics of your schema: what field type, what similarity 
configuration you have (if any) etc...


-Hoss
http://www.lucidworks.com/

Re: Is copyField a must?

[Steven White]

Anyone knows the answer to Shawn's question?  Does Solr support POST
request and is the format the same as GET?

If it does than it means I don't have to create multiple request handlers.

Thanks

Steve

On Wed, May 13, 2015 at 6:02 PM, Shawn Heisey  wrote:

> On 5/13/2015 3:36 PM, Steven White wrote:
> > Note, I want to avoid a URL base solution (sending the list of fields
> over
> > HTTP) because the list of fields could be large (1000+) and thus I will
> > exceed GET limit quickly (does Solr support POST for searching, if so,
> than
> > I can use URL base solution?)
>
> Solr does indeed support a query sent as the body in a POST request.
> I'm not completely positive, but I think you'd use the same format as
> you put on the URL:
>
> q=foo&rows=1&fq=bar
>
> If anyone knows for sure what should be in the POST body, please let me
> and Steven know.  In particular, should the content be URL escaped, as
> might be required for a GET?
>
> Thanks,
> Shawn
>
>

RE: Is copyField a must?

[Garth Grimm]

Yes, it does support POST.  As to format, I believe that's handled by the 
container.  So if you're url-encoding the parameter values, you'll probably 
need to set Content-Type: application/x-www-form-urlencoded for the HTTP POST 
header.

-Original Message-
From: Steven White [mailto:swhite4...@gmail.com] 
Sent: Thursday, May 14, 2015 3:12 PM
To: solr-user@lucene.apache.org
Subject: Re: Is copyField a must?

Anyone knows the answer to Shawn's question?  Does Solr support POST request 
and is the format the same as GET?

If it does than it means I don't have to create multiple request handlers.

Thanks

Steve

On Wed, May 13, 2015 at 6:02 PM, Shawn Heisey  wrote:

> On 5/13/2015 3:36 PM, Steven White wrote:
> > Note, I want to avoid a URL base solution (sending the list of 
> > fields
> over
> > HTTP) because the list of fields could be large (1000+) and thus I 
> > will exceed GET limit quickly (does Solr support POST for searching, 
> > if so,
> than
> > I can use URL base solution?)
>
> Solr does indeed support a query sent as the body in a POST request.
> I'm not completely positive, but I think you'd use the same format as 
> you put on the URL:
>
> q=foo&rows=1&fq=bar
>
> If anyone knows for sure what should be in the POST body, please let 
> me and Steven know.  In particular, should the content be URL escaped, 
> as might be required for a GET?
>
> Thanks,
> Shawn
>
>

Re: [MASSMAIL]Re: High fieldNorm values causing really odd results

[Jorge Luis Betancourt González]

Hi Hoss,

First of all, thank you for your reply.

Sorry for leaving the Solr version out in my previous email, I'm using Solr 
4.10.3 running on Centos7, with the following JRE: Oracle Corporation OpenJDK 
64-Bit Server VM (1.7.0_75 24.75-b04)

This are the relevant portions of my schema.xml















In this particular case I'm not using any special features, just a typical text 
field. I'm using the default similarity class provided by Solr, this is a 
pretty straightforward setup :)

Regards,

- Original Message -
From: "Chris Hostetter" 
To: solr-user@lucene.apache.org
Sent: Thursday, May 14, 2015 4:08:36 PM
Subject: [MASSMAIL]Re: High fieldNorm values causing really odd results


:   {
:  "match":true,
:  "value":655360,
:  "description":"fieldNorm(doc=5316)"
:   }
...
: This match is in the "title" field, which has 119669 total terms (which 
: isn't such big number) and the total document count in this index is 

that smells like a bug -- by the looks of it an overflow bug?

can you please provide some details on the version of solr you are using, 
and the specifics of your schema: what field type, what similarity 
configuration you have (if any) etc...


-Hoss
http://www.lucidworks.com/

Re: [MASSMAIL]Re: High fieldNorm values causing really odd results

[Chris Hostetter]

: Sorry for leaving the Solr version out in my previous email, I'm using 
: Solr 4.10.3 running on Centos7, with the following JRE: Oracle 
: Corporation OpenJDK 64-Bit Server VM (1.7.0_75 24.75-b04)

I can't reproduce Using Solr 4.10.3 (or 4.10.4 - mistread your email the 
first time)

Are you certain you didn't *build* this index with a different Similarity 
configured? or did you perhaps build it with an older version of Solr that 
might have had a bug in it?

Here's what i tried...

applied this patch to the example configs based on the fieldType you 
specified...

hossman@tray:~/lucene/lucene_solr_4_10_3_tag$ svn diff
Index: solr/example/solr/collection1/conf/schema.xml
===
--- solr/example/solr/collection1/conf/schema.xml   (revision 1679472)
+++ solr/example/solr/collection1/conf/schema.xml   (working copy)
@@ -46,6 +46,21 @@
 -->
 
 
+
+
+
+
+
+
+
+
+
+
+
+
+
+  
   

Re: Suggester

[Erick Erickson]

This might help: http://lucidworks.com/blog/solr-suggester/

Best,
Erick

On Thu, May 14, 2015 at 12:43 PM, lreg  wrote:
> Could you please explain how you solved the problem I am having the same
> issue :(
>
>
>
> --
> View this message in context: 
> http://lucene.472066.n3.nabble.com/Suggester-tp4201287p4205460.html
> Sent from the Solr - User mailing list archive at Nabble.com.

Full Copy during cluster restarts

[Paddy Krishnamoorthy]

We are running a solr 4.10.2 cluster with dozens of nodes. When ever we
restart the cluster we see that 2-3 nodes goes in to a fullCopy mode, which
takes for ever to recover. Because of the recovery our entire cluster runs
very slow and eventually becomes unresponsive.

Any thoughts?


Thanks
Paddy
-- 
Did you breathe today?
Learn from here: http://us.artofliving.org/index.html

Re: [MASSMAIL]Re: High fieldNorm values causing really odd results

[Jorge Luis Betancourt González]

Regarding the experiment, sorry If I explained myself in the wrong way, the 
indexed document doesn't have 119669 terms have a lot less terms (less than a 
1000 terms, I don't have the exact number here now), instead 119669 is the 
number of distinct terms reported by luke (Top-terms total in the admin 
interface) on the title field. 

This index was built from scratch using 4.10.3 if I'm no remembering 
incorrectly. Perhaps part of the data could be indexed using 4.10.2, but we 
updated our box quite some time ago and this problem didn't appear until 
recently. The more strange issue is that this was working fine until a week or 
so ago, the only thing I found strange is that the root partition in our Solr 
box got out of space; basically we've Solr deployed in Tomcat, which is 
installed in the root partition but the cores and all Solr related data is 
stored in a separated partition mounted in /opt with plenty of space to grow; 
could this be the cause of this behavior? 

We're thinking on rebuilding our index, but would love to avoid it if possible 
and more importantly find the root cause if this issue (if is possible at all).

As I said before very grateful for your responses,

- Original Message -
From: "Chris Hostetter" 
To: solr-user@lucene.apache.org
Sent: Thursday, May 14, 2015 7:11:08 PM
Subject: Re: [MASSMAIL]Re: High fieldNorm values causing really odd results


: Sorry for leaving the Solr version out in my previous email, I'm using 
: Solr 4.10.3 running on Centos7, with the following JRE: Oracle 
: Corporation OpenJDK 64-Bit Server VM (1.7.0_75 24.75-b04)

I can't reproduce Using Solr 4.10.3 (or 4.10.4 - mistread your email the 
first time)

Are you certain you didn't *build* this index with a different Similarity 
configured? or did you perhaps build it with an older version of Solr that 
might have had a bug in it?

Here's what i tried...

applied this patch to the example configs based on the fieldType you 
specified...

hossman@tray:~/lucene/lucene_solr_4_10_3_tag$ svn diff
Index: solr/example/solr/collection1/conf/schema.xml
===
--- solr/example/solr/collection1/conf/schema.xml   (revision 1679472)
+++ solr/example/solr/collection1/conf/schema.xml   (working copy)
@@ -46,6 +46,21 @@
 -->
 
 
+
+
+
+
+
+
+
+
+
+
+
+
+
+