UIMA Dictionary Annotator

[Darx Oman]

Hi there
I trying to use UIMA Dictionary Annotator with Solr 4.10.0

I did the following:


1) added a field to the schema




2)Modified solrConfig as follow

   
 uima
   
  

  
  
  
   
  DictionaryAnnotator.xml

  
false

  text

  

  

  org.apache.uima.DictionaryEntry
  
tokenType
uimaKeyWords
  

  

  
  
  
  
  


3) copied the following jar files to   solr\collection1\lib folder
 lucene-analyzers-uima-4.10.0.jar
 solr-uima-4.10.0.jar
 uima-an-dictionary.jar
 uimaj-core-2.3.1.jar
 WhitespaceTokenizer-2.3.1.jar
 xmlbeans-2.4.0.jar


4)Added some entries to dictionary.xml
  
iPod
   
  
samsung
  

but when I indexed xml documents from  example\exampledocs
no annotation happened

this is the xml response I got


IW-02
iPod & iPod Mini USB 2.0 Cable
Belkin
belkin

  electronics
  connector


  car power adapter for iPod, white

2.0
11.5
11.50,USD
1
false
37.7752,-122.4232
2006-02-14T23:55:59Z
1479845823070076928
  


what might went wrong?

am I missing something?

with wildcard-source?

[Clemens Wyss DEV]

is there a way to use  with a wildcard-source? 
For example to copy all fields of a certain dynamic field type:


 ...

Re: running solr in debug through eclipse

[Anurag Sharma]

Hi All,

Thanks a lot for your suggestions. Shalin, your direction quickly took me
to the issue, it was very insightful and helpful.
Finally am able to understand the issue I was working on and run particular
unit test class AtomicUpdatesTest around it.

On running the Solr in debug mode, I am still not able to start solr in
debug mode using:
java -Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=7666
-jar start.jar
(Ref: http://wiki.apache.org/solr/HowToConfigureEclipse)
The command wait for hours and the server never comes up on windows without
giving any error/info message.

Please suggest if someone faced this issue. I tried restarting windows,
ensured no process running on 7666 port. In previous trials to start, I
used to get msg "Debugger failed to attach: handshake failed - received
>GET /solr/ HTT< - expect" but it stopped coming now.

Thanks
Anurag




On Fri, Sep 19, 2014 at 8:21 PM, Erick Erickson 
wrote:

> Yeah, it's usually pretty daunting to know where to start, the
> codebase is kinda big. Even "start from junit test" is often daunting,
> there are a lot of them too.
>
> Others have given you good places to start, good luck!
>
> Erick
>
> On Fri, Sep 19, 2014 at 12:23 AM, Bernd Fehling
>  wrote:
> > Just start at the UpdateHandler and follow it down the line.
> >
> > I would start at org/apache/solr/update/UpdateHandler.java
> >
> > If you already know if it is add, delete or update then start with
> > AddUpdateCommand.java, DeleteUpdateCommand.java or UpdateCommand.java.
> >
> > Just follow the red line :-)
> >
> > Regards
> > Bernd
> >
> >
> > Am 19.09.2014 um 08:47 schrieb Anurag Sharma:
> >> Thanks Bernd for your insight.
> >> As of now, I am focussing to fix the issue in the updater but not able
> to
> >> localize which code to look in for it.
> >>
> >> Regards,
> >> Anurag
> >>
> >> On Fri, Sep 19, 2014 at 12:09 PM, Bernd Fehling <
> >> bernd.fehl...@uni-bielefeld.de> wrote:
> >>
> >>> It depends on what you are going to do.
> >>>
> >>> If you are adding/modifying code and Junit tests use Junit test cases.
> >>> If you are debugging runtime problems under load use remote debugging.
> >>> If you are going for in deep debugging (even into Jetty and Java) use
> >>> RunJettyRun for Eclipse.
> >>>
> >>> Regards
> >>> Bernd
> >>>
> >>>
> >>> Am 18.09.2014 um 20:50 schrieb Anurag Sharma:
>  Dear Solr users,
> 
>  I am new to Solr dev community and trying to setup eclipse to debug a
>  running solr server. Please suggest if anyone of you have tried doing
> the
>  same.
> 
>  Once above is done. Also suggest the entry point in code where
> breakpoint
>  can be placed.
> 
>  Thanks
>  Anurag
> 
> >>>
> >>>
> >>
>

Re: with wildcard-source?

[Alexandre Rafalovitch]

copyField - Solr is case sensitive, yet does not complain when sees
wrong/misspelt directives (this is being fixed slowly)

https://cwiki.apache.org/confluence/display/solr/Copying+Fields

Regards,
   Alex.

Personal: http://www.outerthoughts.com/ and @arafalov
Solr resources and newsletter: http://www.solr-start.com/ and @solrstart
Solr popularizers community: https://www.linkedin.com/groups?gid=6713853


On 21 September 2014 05:47, Clemens Wyss DEV  wrote:
> is there a way to use  with a wildcard-source?
> For example to copy all fields of a certain dynamic field type:
> 
> 
>  ...
> 

Re: Solr(j) API for manipulating the schema(.xml)?

[Erick Erickson]

Wait, wait, wait. I cannot _believe_ I wrote this:

bq: As far as changing the schema, there's no good way to do that that

Utter nonsense. There's the whole "managed schema" that was added in
the not too distant past whose _purpose_ is to change the schema via a
REST API both in stand-alone and in SolrCloud.

Siiigggh. Senility creeps up unnoticed.

Best
er...@onthewaytoaresthome.org

On Sat, Sep 20, 2014 at 9:34 PM, Jorge Luis Betancourt Gonzalez
 wrote:
> Basically you could create a bunch of dynamic fields (according to your 
> needs) so basically creating a dynamic field for each type of data (and 
> several combinations) and then you can create a small wrapper around Solrj 
> that will wrap the patterns defined on your schema.xml in a more 
> understandable way. Like this you will be able to abstract the manipulation 
> of the schema.xml file and only introduce it when is really needed i.e a new 
> field type with new analyzers, etc.
>
> On Sep 18, 2014, at 3:16 AM, Clemens Wyss DEV  wrote:
>
>> as our framework so far only knows a few field types "dynamic field"s may be 
>> the way to go... And if there are new fieldtypes the new schema can be 
>> distributed through ZooKeeper
>>
>> -Ursprüngliche Nachricht-
>> Von: Erick Erickson [mailto:erickerick...@gmail.com]
>> Gesendet: Mittwoch, 17. September 2014 19:56
>> An: solr-user@lucene.apache.org
>> Betreff: Re: Solr(j) API for manipulating the schema(.xml)?
>>
>> Right, you can create new cores over the rest api.
>>
>> As far as changing the schema, there's no good way to do that that I know of 
>> programmatically. In the SolrCloud world, you can upload the schema to 
>> ZooKeeper and have it automatically distributed to all the nodes though.
>>
>> Best,
>> Erick
>>
>> On Wed, Sep 17, 2014 at 2:28 AM, Clemens Wyss DEV  
>> wrote:
>>> Is there an API to manipulate/consolidate the schema(.xml) of a Solr-core? 
>>> Through SolrJ?
>>>
>>> Context:
>>> We already have a generic indexing/searching framework (based on lucene) 
>>> where any component can act as a so called IndexDataPorvider. This provider 
>>> delivers the field-types and also the entities to be (converted into 
>>> documents and then) indexed. Each of these IndexProviders has ist own 
>>> lucene index.
>>> So we kind of have the information for the Solr schema.xml.
>>>
>>> Hope the intention is clear. And yes the manipulation of the schema.xml is 
>>> basically only needed when the field types change. Thats why I am looking 
>>> for a way to consolidate the schema.xml (upon boot, initialization oft he 
>>> IndexDataProviders ...).
>>> In 99,999% it won't change, But I'd like to keep the possibility of an 
>>> IndexDataProvider to hand in "its schema".
>>>
>>> Also, again driven by the dynamic nature of our framework, can I easily 
>>> create new cores over Sorj or the Solr-REST API ?
>
> Concurso "Mi selfie por los 5". Detalles en 
> http://justiciaparaloscinco.wordpress.com

Re: running solr in debug through eclipse

[Erick Erickson]

It's doing exactly what you tell it to:

java -Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=7666
-jar start.jar

Specifically suspend=y' means it will sit there, very patiently, until
you connect to it with a debugger and tell it to go. This is _very_
useful to debug initialization errors, but can sometimes be a bit
puzzling.

I'd recommend you actually attach with the debugger (i.e. a "remote"
session). In IntelliJ, (I'm sure there are analogous ways in Eclipse),
you create, quite literally, a "remote session" that you give the URL
of the server you started above and the port. You start your server as
above and then start your remote session in your IDE and you'll be in
the debugger, attached to the running Solr instance. You can set
breakpoints or just hit the "go" button and the server should start
up. My setup usually just has "localhost" and 7666 for the URL/port.

You do not have to attach a debugger first, just specify 'suspend=n'
instead. But starting with 'suspend=y' insures you have actually
attached to the server and have all the parts in place.

Best,
Erick

On Sun, Sep 21, 2014 at 3:51 AM, Anurag Sharma  wrote:
> Hi All,
>
> Thanks a lot for your suggestions. Shalin, your direction quickly took me
> to the issue, it was very insightful and helpful.
> Finally am able to understand the issue I was working on and run particular
> unit test class AtomicUpdatesTest around it.
>
> On running the Solr in debug mode, I am still not able to start solr in
> debug mode using:
> java -Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=7666
> -jar start.jar
> (Ref: http://wiki.apache.org/solr/HowToConfigureEclipse)
> The command wait for hours and the server never comes up on windows without
> giving any error/info message.
>
> Please suggest if someone faced this issue. I tried restarting windows,
> ensured no process running on 7666 port. In previous trials to start, I
> used to get msg "Debugger failed to attach: handshake failed - received
>>GET /solr/ HTT< - expect" but it stopped coming now.
>
> Thanks
> Anurag
>
>
>
>
> On Fri, Sep 19, 2014 at 8:21 PM, Erick Erickson 
> wrote:
>
>> Yeah, it's usually pretty daunting to know where to start, the
>> codebase is kinda big. Even "start from junit test" is often daunting,
>> there are a lot of them too.
>>
>> Others have given you good places to start, good luck!
>>
>> Erick
>>
>> On Fri, Sep 19, 2014 at 12:23 AM, Bernd Fehling
>>  wrote:
>> > Just start at the UpdateHandler and follow it down the line.
>> >
>> > I would start at org/apache/solr/update/UpdateHandler.java
>> >
>> > If you already know if it is add, delete or update then start with
>> > AddUpdateCommand.java, DeleteUpdateCommand.java or UpdateCommand.java.
>> >
>> > Just follow the red line :-)
>> >
>> > Regards
>> > Bernd
>> >
>> >
>> > Am 19.09.2014 um 08:47 schrieb Anurag Sharma:
>> >> Thanks Bernd for your insight.
>> >> As of now, I am focussing to fix the issue in the updater but not able
>> to
>> >> localize which code to look in for it.
>> >>
>> >> Regards,
>> >> Anurag
>> >>
>> >> On Fri, Sep 19, 2014 at 12:09 PM, Bernd Fehling <
>> >> bernd.fehl...@uni-bielefeld.de> wrote:
>> >>
>> >>> It depends on what you are going to do.
>> >>>
>> >>> If you are adding/modifying code and Junit tests use Junit test cases.
>> >>> If you are debugging runtime problems under load use remote debugging.
>> >>> If you are going for in deep debugging (even into Jetty and Java) use
>> >>> RunJettyRun for Eclipse.
>> >>>
>> >>> Regards
>> >>> Bernd
>> >>>
>> >>>
>> >>> Am 18.09.2014 um 20:50 schrieb Anurag Sharma:
>>  Dear Solr users,
>> 
>>  I am new to Solr dev community and trying to setup eclipse to debug a
>>  running solr server. Please suggest if anyone of you have tried doing
>> the
>>  same.
>> 
>>  Once above is done. Also suggest the entry point in code where
>> breakpoint
>>  can be placed.
>> 
>>  Thanks
>>  Anurag
>> 
>> >>>
>> >>>
>> >>
>>

Re: with wildcard-source?

[Erick Erickson]

What have you tried? Because it works just fine for me.

It's _really_ helpful to tell us what you've tried and what
you think isn't operating correctly, otherwise
there's not much to go on except guesswork.

Best,
Erick

On Sun, Sep 21, 2014 at 2:47 AM, Clemens Wyss DEV  wrote:
> is there a way to use  with a wildcard-source?
> For example to copy all fields of a certain dynamic field type:
> 
> 
>  ...
> 

Static Fields Performance vs Dynamic Fields Performance

[mark12345]

Are there any performance differences in using Dynamic Fields over Static
Fields?

I read an earlier post on Aug 12, 2008 that suggested there is nothing
significant (See link below).  Is this still the case?

http://lucene.472066.n3.nabble.com/Static-Fields-vs-Dynamic-Fields-td487639.html





--
View this message in context: 
http://lucene.472066.n3.nabble.com/Static-Fields-Performance-vs-Dynamic-Fields-Performance-tp4160316.html
Sent from the Solr - User mailing list archive at Nabble.com.

RE: Help on custom sort

[Scott Smith]

There are likely several hundred groups.  Also, new groups will be added and 
some groups will be deleted.  So, I don't think putting a field in the docs 
works.  Having to add a new group price into 100 million+ documents doesn't 
seem reasonable.

Right now I'm looking at 
http://sujitpal.blogspot.com/2011/05/custom-sorting-in-solr-using-external.html.
  This reference a much older version of solr (the blog is from 2011) and so I 
will need to update the classes referenced.

-Original Message-
From: Erick Erickson [mailto:erickerick...@gmail.com] 
Sent: Saturday, September 20, 2014 11:58 AM
To: solr-user@lucene.apache.org
Subject: Re: Help on custom sort

How many different groups are there? And can user A ever be part of more than 
one group?
If
1> there are a reasonably small number of groups (< 100 or so as a
place to start)
and
2> a user is always part of a single group

then you could store separate prices in each document by group, thus you'd have 
some fields like
price_group_a: $100
price_group_b: $101

then sorting  becomes trivial, you just specify a sort_group_a for users in 
group A etc. If the number of groups is unknown-but-not-huge dynamic fields 
could be used.

If that's not the case, then you might be able to get clever with sorting by 
function, here's a place to start:
https://cwiki.apache.org/confluence/display/solr/Function+Queries

These can be arbitrarily complex, but I'm thinking something where the price 
returned by the function respects the group the user is in, perhaps even the 
min/max of all the groups the user is in. I admit I haven't really thought that 
through well though...

Best,
Erick

On Sat, Sep 20, 2014 at 9:26 AM, Scott Smith  wrote:
> I need to provide a custom sort option for sorting by price and I would like 
> some suggestions.  It's not the straightforward "just sort by a price field 
> in the document" scenario or I wouldn't be asking for help.  Here's the 
> scenario I'm dealing with.
>
> I have 100 million+ documents (so multi-sharded).  Users search for documents 
> they are interested in using a standard keyword search.  They then purchase 
> documents they are interested in.  So far, nothing hard.
>
> Here's where things get "interesting".  The documents come from multiple 
> suppliers.  Each supplier sets a price for his documents and different 
> suppliers will provide different pricing.
>
> That wouldn't be difficult except that *users* are divided up into different 
> groups and depending on which group they are in, the supplier will charge the 
> user a different price.  So, user A may pay one price for a document and user 
> B may pay a different price for the same document just because user A and 
> user B are in different groups.  I don't even know if the relative order or 
> pricing is the same between different groups (e.g., if document X is more 
> expensive than document Y for a user in group M, it may not be more expensive 
> for a user in group N).  The one thing that may make this doable is that 
> supplier A will likely have the same price for all of his documents for each 
> of the user groups.  So, a user in group A will pay the same price regardless 
> of which document he buys from supplier 1.  A user in group B will also pay 
> the same price for any document from supplier 1; it's just that a user in 
> group B will likely pay a different price than a user in group A.  So, within 
> a supplier, the price varies based on user group, not the document.
>
> To summarize, one of the requirements for the system is that we provide the 
> ability to sort search results based on price.  This would be easy except 
> that the price a user pays not only depends on what he wants to buy, but on 
> what group the he is in.
>
> I suspect there is some kind of custom solr module I'm going to have to 
> write.  I'm thinking that the user group gets passed in as a custom solr 
> parameter (I'm assuming that's possible??).  Then I'm thinking that there has 
> to be some kind of in memory database that tracks pricing based on user group 
> and document supplier).
>
> I'm happy to go read code, documents, links, etc if someone can point me in 
> the right direction.  What kind of solr module am I likely going to write 
> (extend) and are there some examples somewhere?  Maybe there's a way to do 
> this without having to extend a solr module??
>
> Hope this makes sense.  Any help is appreciated.
>
> Scott
>
>

Re: Static Fields Performance vs Dynamic Fields Performance

[Alexandre Rafalovitch]

We had this discussion a week ago.

http://lucene.472066.n3.nabble.com/Solr-Dynamic-Field-Performance-td4158737.html

Regards,
Alex.
Personal: http://www.outerthoughts.com/ and @arafalov
Solr resources and newsletter: http://www.solr-start.com/ and @solrstart
Solr popularizers community: https://www.linkedin.com/groups?gid=6713853


On 21 September 2014 16:04, mark12345  wrote:
> Are there any performance differences in using Dynamic Fields over Static
> Fields?
>
> I read an earlier post on Aug 12, 2008 that suggested there is nothing
> significant (See link below).  Is this still the case?
>
> http://lucene.472066.n3.nabble.com/Static-Fields-vs-Dynamic-Fields-td487639.html
>
>
>
>
>
> --
> View this message in context: 
> http://lucene.472066.n3.nabble.com/Static-Fields-Performance-vs-Dynamic-Fields-Performance-tp4160316.html
> Sent from the Solr - User mailing list archive at Nabble.com.

Re: running solr in debug through eclipse

[Anurag Sharma]

Hey Eric,

It works like charm :).
Thanks a lot for pin pointing the issue. My bad I was using the suspend=y
option blindly.

Thanks again,
Anurag

On Sun, Sep 21, 2014 at 10:03 PM, Erick Erickson 
wrote:

> It's doing exactly what you tell it to:
>
> java -Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=7666
> -jar start.jar
>
> Specifically suspend=y' means it will sit there, very patiently, until
> you connect to it with a debugger and tell it to go. This is _very_
> useful to debug initialization errors, but can sometimes be a bit
> puzzling.
>
> I'd recommend you actually attach with the debugger (i.e. a "remote"
> session). In IntelliJ, (I'm sure there are analogous ways in Eclipse),
> you create, quite literally, a "remote session" that you give the URL
> of the server you started above and the port. You start your server as
> above and then start your remote session in your IDE and you'll be in
> the debugger, attached to the running Solr instance. You can set
> breakpoints or just hit the "go" button and the server should start
> up. My setup usually just has "localhost" and 7666 for the URL/port.
>
> You do not have to attach a debugger first, just specify 'suspend=n'
> instead. But starting with 'suspend=y' insures you have actually
> attached to the server and have all the parts in place.
>
> Best,
> Erick
>
> On Sun, Sep 21, 2014 at 3:51 AM, Anurag Sharma  wrote:
> > Hi All,
> >
> > Thanks a lot for your suggestions. Shalin, your direction quickly took me
> > to the issue, it was very insightful and helpful.
> > Finally am able to understand the issue I was working on and run
> particular
> > unit test class AtomicUpdatesTest around it.
> >
> > On running the Solr in debug mode, I am still not able to start solr in
> > debug mode using:
> > java -Xdebug
> -Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=7666
> > -jar start.jar
> > (Ref: http://wiki.apache.org/solr/HowToConfigureEclipse)
> > The command wait for hours and the server never comes up on windows
> without
> > giving any error/info message.
> >
> > Please suggest if someone faced this issue. I tried restarting windows,
> > ensured no process running on 7666 port. In previous trials to start, I
> > used to get msg "Debugger failed to attach: handshake failed - received
> >>GET /solr/ HTT< - expect" but it stopped coming now.
> >
> > Thanks
> > Anurag
> >
> >
> >
> >
> > On Fri, Sep 19, 2014 at 8:21 PM, Erick Erickson  >
> > wrote:
> >
> >> Yeah, it's usually pretty daunting to know where to start, the
> >> codebase is kinda big. Even "start from junit test" is often daunting,
> >> there are a lot of them too.
> >>
> >> Others have given you good places to start, good luck!
> >>
> >> Erick
> >>
> >> On Fri, Sep 19, 2014 at 12:23 AM, Bernd Fehling
> >>  wrote:
> >> > Just start at the UpdateHandler and follow it down the line.
> >> >
> >> > I would start at org/apache/solr/update/UpdateHandler.java
> >> >
> >> > If you already know if it is add, delete or update then start with
> >> > AddUpdateCommand.java, DeleteUpdateCommand.java or UpdateCommand.java.
> >> >
> >> > Just follow the red line :-)
> >> >
> >> > Regards
> >> > Bernd
> >> >
> >> >
> >> > Am 19.09.2014 um 08:47 schrieb Anurag Sharma:
> >> >> Thanks Bernd for your insight.
> >> >> As of now, I am focussing to fix the issue in the updater but not
> able
> >> to
> >> >> localize which code to look in for it.
> >> >>
> >> >> Regards,
> >> >> Anurag
> >> >>
> >> >> On Fri, Sep 19, 2014 at 12:09 PM, Bernd Fehling <
> >> >> bernd.fehl...@uni-bielefeld.de> wrote:
> >> >>
> >> >>> It depends on what you are going to do.
> >> >>>
> >> >>> If you are adding/modifying code and Junit tests use Junit test
> cases.
> >> >>> If you are debugging runtime problems under load use remote
> debugging.
> >> >>> If you are going for in deep debugging (even into Jetty and Java)
> use
> >> >>> RunJettyRun for Eclipse.
> >> >>>
> >> >>> Regards
> >> >>> Bernd
> >> >>>
> >> >>>
> >> >>> Am 18.09.2014 um 20:50 schrieb Anurag Sharma:
> >>  Dear Solr users,
> >> 
> >>  I am new to Solr dev community and trying to setup eclipse to
> debug a
> >>  running solr server. Please suggest if anyone of you have tried
> doing
> >> the
> >>  same.
> >> 
> >>  Once above is done. Also suggest the entry point in code where
> >> breakpoint
> >>  can be placed.
> >> 
> >>  Thanks
> >>  Anurag
> >> 
> >> >>>
> >> >>>
> >> >>
> >>
>

Re: UIMA Dictionary Annotator

[Darx Oman]

It was some configuration errors

this is the new configuration


  
  
  
   
  AggregateAE.xml

  
false

  name

  

  

  org.apache.uima.DictionaryEntry
  
coveredText
uimaKeyWords
  

  

  
  
  
  
  

it works fine

On Sun, Sep 21, 2014 at 1:23 PM, Darx Oman  wrote:

> Hi there
> I trying to use UIMA Dictionary Annotator with Solr 4.10.0
>
> I did the following:
>
>
> 1) added a field to the schema
>  multiValued="true" />
>
>
>
> 2)Modified solrConfig as follow
> 
>
>  uima
>
>   
>
>   
>class="org.apache.solr.uima.processor.UIMAUpdateRequestProcessorFactory">
>   
>
>   DictionaryAnnotator.xml
>
>   
> false
> 
>   text
> 
>   
>
>   
> 
>   org.apache.uima.DictionaryEntry
>   
> tokenType
> uimaKeyWords
>   
> 
>   
>
>   
>   
>   
>   
>   
>
>
> 3) copied the following jar files to   solr\collection1\lib folder
>  lucene-analyzers-uima-4.10.0.jar
>  solr-uima-4.10.0.jar
>  uima-an-dictionary.jar
>  uimaj-core-2.3.1.jar
>  WhitespaceTokenizer-2.3.1.jar
>  xmlbeans-2.4.0.jar
>
>
> 4)Added some entries to dictionary.xml
>   
> iPod
>
>   
> samsung
>   
>
> but when I indexed xml documents from  example\exampledocs
> no annotation happened
>
> this is the xml response I got
>
> 
> IW-02
> iPod & iPod Mini USB 2.0 Cable
> Belkin
> belkin
> 
>   electronics
>   connector
> 
> 
>   car power adapter for iPod, white
> 
> 2.0
> 11.5
> 11.50,USD
> 1
> false
> 37.7752,-122.4232
> 2006-02-14T23:55:59Z
> 1479845823070076928
>   
>
>
> what might went wrong?
>
> am I missing something?
>
>
>

AW: with wildcard-source?

[Clemens Wyss DEV]

I haven't tried anything so far. It was a plain conceptual question ;) 
All I have at hand is "Solr in Action" which doesn't (didn't) mention the 
copyField-wildcards...

Thx!

-Ursprüngliche Nachricht-
Von: Erick Erickson [mailto:erickerick...@gmail.com] 
Gesendet: Sonntag, 21. September 2014 18:44
An: solr-user@lucene.apache.org
Betreff: Re:  with wildcard-source?

What have you tried? Because it works just fine for me.

It's _really_ helpful to tell us what you've tried and what you think isn't 
operating correctly, otherwise there's not much to go on except guesswork.

Best,
Erick

On Sun, Sep 21, 2014 at 2:47 AM, Clemens Wyss DEV  wrote:
> is there a way to use  with a wildcard-source?
> For example to copy all fields of a certain dynamic field type:
> 
>   ...
> 

Disable caching in sort

[Apoorva Gaurav]

Hello All,

We are trying to provide a personalized sort order for each user. We've a
per-computed list of user to products and if it matches the solr result set
those products need to be shown upfront. One way can be handling this in
application but pagination becomes tricky. Another way we are exploring
this is via a custom value source where we'll pass productid to
custom-score and sort based on this. We've been able to manipulate result
set using this, but sort order is getting cached. One way can be using
{!cache=false} but that would lead to performance degradation. Any other
way of achieving this?

-- 
Thanks & Regards,
Apoorva

Re: Help on custom sort

[Apoorva Gaurav]

Try using a custom value source parser and pass the "formula" of computing
the price to solr; something like this
http://java.dzone.com/articles/connecting-redis-solr-boosting

On Mon, Sep 22, 2014 at 1:38 AM, Scott Smith 
wrote:

> There are likely several hundred groups.  Also, new groups will be added
> and some groups will be deleted.  So, I don't think putting a field in the
> docs works.  Having to add a new group price into 100 million+ documents
> doesn't seem reasonable.
>
> Right now I'm looking at
> http://sujitpal.blogspot.com/2011/05/custom-sorting-in-solr-using-external.html.
> This reference a much older version of solr (the blog is from 2011) and so
> I will need to update the classes referenced.
>
> -Original Message-
> From: Erick Erickson [mailto:erickerick...@gmail.com]
> Sent: Saturday, September 20, 2014 11:58 AM
> To: solr-user@lucene.apache.org
> Subject: Re: Help on custom sort
>
> How many different groups are there? And can user A ever be part of more
> than one group?
> If
> 1> there are a reasonably small number of groups (< 100 or so as a
> place to start)
> and
> 2> a user is always part of a single group
>
> then you could store separate prices in each document by group, thus you'd
> have some fields like
> price_group_a: $100
> price_group_b: $101
>
> then sorting  becomes trivial, you just specify a sort_group_a for users
> in group A etc. If the number of groups is unknown-but-not-huge dynamic
> fields could be used.
>
> If that's not the case, then you might be able to get clever with sorting
> by function, here's a place to start:
> https://cwiki.apache.org/confluence/display/solr/Function+Queries
>
> These can be arbitrarily complex, but I'm thinking something where the
> price returned by the function respects the group the user is in, perhaps
> even the min/max of all the groups the user is in. I admit I haven't really
> thought that through well though...
>
> Best,
> Erick
>
> On Sat, Sep 20, 2014 at 9:26 AM, Scott Smith 
> wrote:
> > I need to provide a custom sort option for sorting by price and I would
> like some suggestions.  It's not the straightforward "just sort by a price
> field in the document" scenario or I wouldn't be asking for help.  Here's
> the scenario I'm dealing with.
> >
> > I have 100 million+ documents (so multi-sharded).  Users search for
> documents they are interested in using a standard keyword search.  They
> then purchase documents they are interested in.  So far, nothing hard.
> >
> > Here's where things get "interesting".  The documents come from multiple
> suppliers.  Each supplier sets a price for his documents and different
> suppliers will provide different pricing.
> >
> > That wouldn't be difficult except that *users* are divided up into
> different groups and depending on which group they are in, the supplier
> will charge the user a different price.  So, user A may pay one price for a
> document and user B may pay a different price for the same document just
> because user A and user B are in different groups.  I don't even know if
> the relative order or pricing is the same between different groups (e.g.,
> if document X is more expensive than document Y for a user in group M, it
> may not be more expensive for a user in group N).  The one thing that may
> make this doable is that supplier A will likely have the same price for all
> of his documents for each of the user groups.  So, a user in group A will
> pay the same price regardless of which document he buys from supplier 1.  A
> user in group B will also pay the same price for any document from supplier
> 1; it's just that a user in group B will likely pay a different price than
> a user in group A.  So, within a supplier, the price varies based on user
> group, not the document.
> >
> > To summarize, one of the requirements for the system is that we provide
> the ability to sort search results based on price.  This would be easy
> except that the price a user pays not only depends on what he wants to buy,
> but on what group the he is in.
> >
> > I suspect there is some kind of custom solr module I'm going to have to
> write.  I'm thinking that the user group gets passed in as a custom solr
> parameter (I'm assuming that's possible??).  Then I'm thinking that there
> has to be some kind of in memory database that tracks pricing based on user
> group and document supplier).
> >
> > I'm happy to go read code, documents, links, etc if someone can point me
> in the right direction.  What kind of solr module am I likely going to
> write (extend) and are there some examples somewhere?  Maybe there's a way
> to do this without having to extend a solr module??
> >
> > Hope this makes sense.  Any help is appreciated.
> >
> > Scott
> >
> >
>



-- 
Thanks & Regards,
Apoorva

max across documents?

[William Bell]

Is there an easy way to get max() across documents?

i.e.

1. Age = multivalued
56
67
89

2. Next document
5
43
23

How do I get this ?

http://localhost:8983/solr/people/select?q=*:*&fl=sum(max_across_document(age),
6.0, max(age))

I need the max value cross all docs and then add 6 and the max(inside the
field).

Thanks!!




-- 
Bill Bell
billnb...@gmail.com
cell 720-256-8076

AW: AW: AW: slorj -> httpclient 4, but we already have httpclient 3 in use

[Clemens Wyss DEV]

> and hopefully there isn't a stick
I'll let you know ... so hopefully no follower to this post ;)

-Ursprüngliche Nachricht-
Von: Guido Medina [mailto:guido.med...@temetra.com] 
Gesendet: Freitag, 19. September 2014 10:29
An: solr-user@lucene.apache.org
Betreff: Re: AW: AW: slorj -> httpclient 4, but we already have httpclient 3 in 
use

Hi Clemens,

You just found the carrot, let us know when you find the stick :D

We had to do some refactor due to new HTTP client way of handling multiple 
connections to a host, also BASIC authentication, from HTTP client 3 to 4 is 
just put the jar there and it would just work.

We also use HTTP client to connect to other services besides offering SolrJ 
client an HTTP client.

Anyway, wish the best and hopefully there isn't a stick :D

Guido.

On 19/09/14 08:51, Siegfried Goeschl wrote:
> Lucky you :-)
>
> Siegfried Goeschl
>
> On 19.09.14 07:31, Clemens Wyss DEV wrote:
>> I'd like to mention, that substituting the httpcore.jar  with the 
>> latest (4.3) "sufficed"...
>>
>> -Ursprüngliche Nachricht-
>> Von: Guido Medina [mailto:guido.med...@temetra.com]
>> Gesendet: Donnerstag, 18. September 2014 18:20
>> An: solr-user@lucene.apache.org
>> Betreff: Re: AW: slorj -> httpclient 4, but we already have 
>> httpclient 3 in use
>>
>> SolrJ client after 4.8 I think requires HTTP client 4.3.x so why not 
>> just start there as base version?
>>
>> Guido.
>>
>> On 18/09/14 16:49, Siegfried Goeschl wrote:
>>> AFAIK even the different minor versions are source/binary compatible 
>>> so you might need to tinker with the right "version" to get your 
>>> server running
>>>
>>> Cheers,
>>>
>>> Siegfried Goeschl
>>>
>>> On 18.09.14 17:45, Guido Medina wrote:
 Hi Clemens,

 If you are going thru the effort of migrating from SolrJ 3 to 4 and 
 HTTP client 3 to 4 make sure you do it using HTTP client 4.3.x 
 (Latest is
 4.3.5) since there are deprecations and stuff from 3.x to 4.0.x, to 
 4.1.x, to ..., to 4.3.x

 It will be painful but it is better do it one time and not later 
 needed to do it again. I was on a similar situation (well my 
 company) and I had to suffer such migration (not my company but 
 myself since I'm the one that keeps all those things up to date)

 Best regards,

 Guido.

 On 18/09/14 16:14, Clemens Wyss DEV wrote:
> I guess you are right ;)
>
> -Ursprüngliche Nachricht-
> Von: Siegfried Goeschl [mailto:sgoes...@gmx.at]
> Gesendet: Donnerstag, 18. September 2014 16:38
> An: solr-user@lucene.apache.org
> Betreff: Re: slorj -> httpclient 4, but we already have httpclient 
> 3 in use
>
> Hi Clemens,
>
> I think you need to upgrade you framework
>
> * AFAIK is httpclient 3 & 4 uses the same package names - which is 
> slightly unfortunate
> * assuming that they are using the same package name it is 
> non-deterministic which httpclient library is loaded - might work 
> on your local box but not on the production server or might change 
> to a change in the project
>
> Cheers,
>
> Siegfried Goeschl
>
>
> On 18.09.14 15:08, Clemens Wyss DEV wrote:
>> I doing initial steps with solrj which is based on httpclient 4.
>> Unfortunately parts of our framework are based on httpclient 3.
>> So when I instantiate an HttpSolrServer I run into:
>>
>> java.lang.VerifyError: Cannot inherit from final class ...
>>  at
>> org.apache.http.impl.client.DefaultHttpClient.createHttpParams(De
>> fa
>> ultHttpClient.java:157)
>>
>>
>>  at
>> org.apache.http.impl.client.AbstractHttpClient.getParams(Abstract
>> Ht
>> tpClient.java:447)
>>
>>
>>  at
>> org.apache.solr.client.solrj.impl.HttpClientUtil.setFollowRedirec
>> ts
>> (Ht
>> tpClientUtil.java:255)
>> ...
>>
>> Can these be run side-by-side at all?
>>

>>>
>>
>

Re: Ping handler during initial wamup

[William Bell]

I put in a fix a long time ago, so it wouldn't be too hard to redo.

1. Send 500 error until core > 0 numFound.
2. Give it a list of cores.

localhost:8983/solr/ping?check=core1,core2,core3

Then the Amazon load balancer can just check this url with HTTP check


On Thu, Sep 18, 2014 at 3:57 AM, Ere Maijala 
wrote:

> So, is it possible to configure a ping handler to return quickly with
>>> non-OK status if a search handler is not yet available? This would
>>> allow the load balancer to quickly fail over to another server. I
>>> couldn't find anything like this in the docs, but I'm still hopeful.
>>>
>>> I'm aware of the possibility of using a health state file, but I'd
>>> rather have a way of doing this automatically.
>>>
>>
>> If it's not horribly messy to implement, returning a non-OK status
>> immediately when there is no available searcher seems like a good idea.
>> Please file an improvement issue in Jira.
>>
>
> Thanks, I've filed .
>
> --Ere
>
>


-- 
Bill Bell
billnb...@gmail.com
cell 720-256-8076

Re: MergeReduceIndexerTool takes a lot of time for a limited number of documents

[rulinma]

-D 'mapred.child.java.opts=-Xmx500m' 
set this as your needs.
I  think it will work well.



--
View this message in context: 
http://lucene.472066.n3.nabble.com/MergeReduceIndexerTool-takes-a-lot-of-time-for-a-limited-number-of-documents-tp4138163p4160362.html
Sent from the Solr - User mailing list archive at Nabble.com.