multicores issue..

[dudes dudes]

thanks Erik, 
I'm experiencing some problems here.. 

solr version:  solr-2008-07-07
Out of the box multicores directory: apache-solr-nightly/example/multicore
multcore.xml: shown bellow:


  
  



I can easily see the admin page http://localhost:8983/solr/core0/admin/ and 
http://localhost:8983/solr/core1/admin/ .However, if I click on schema or 
config it says 

HTTP ERROR: 404

NOT_FOUND

RequestURI=/solr/admin/file/


So edited the main solrconfg.xml file by uncommenting 
 

But I still have the problem... 

Any thoughts on this ?

thanks all, 
ak


> From: [EMAIL PROTECTED]
> To: solr-user@lucene.apache.org
> Subject: Re: 2 IDs in schema.xml
> Date: Wed, 16 Jul 2008 04:50:18 -0400
> 
> 
> On Jul 16, 2008, at 3:29 AM, dudes dudes wrote:
>> and also I suppose it is not possible to have 2 schemas on one solr  ?
> 
> You can have multiple cores per Solr webapp (see MultiCore in the  
> wiki).  But each core has a single schema.
> 
>   Erik
> 

_
100’s of Nikon cameras to be won with Live Search
http://clk.atdmt.com/UKM/go/101719808/direct/01/

Re: Using custom Similarity class

[Sébastien Rainville]

I'm using solr-1.2.0. I didn't have a no-arg constructor. I just tried with
one and it doesn't fix it. My servlet container is Jetty.

Sebastien



On Wed, Jul 16, 2008 at 10:22 PM, Erik Hatcher <[EMAIL PROTECTED]>
wrote:

> What version of Solr are you using?   Seems to be not quite trunk at least,
> as IndexSchema.java:449 isn't the the similarity stuff currently).
>
> Does your CustomSimilarity have a no-arg constructor?
>
>Erik
>
>
>
>
> On Jul 16, 2008, at 9:19 PM, Sébastien Rainville wrote:
>
>  Hi,
>>
>> I'm trying to make solr use my custom similarity class:
>>
>> import org.apache.lucene.search.DefaultSimilarity;
>>
>> public class CustomSimilarity extends DefaultSimilarity {
>> }
>>
>> but I keep getting this error:
>>
>> com.example.CustomSimilarity cannot be cast to
>> org.apache.lucene.search.Similarity
>>   at org.apache.solr.schema.IndexSchema.readConfig(IndexSchema.java:449)
>>   ... 28 more
>>
>> and in schema.xml I have this:
>> 
>>
>> I even tried to extend Similarity directly just to make sure and it's not
>> working either. It means that it finds the class, it's able to instantiate
>> it but somehow it's not the right kind of object?!? What am I missing?
>>
>> Thanks in advance,
>> Sebastien
>>
>
>

RE: multicores issue..

[dudes dudes]

it's ok guys,,, fixed the problem :) I have 2 different schemas up and running 

thanks anyway
ak

> From: [EMAIL PROTECTED]
> To: solr-user@lucene.apache.org
> Subject: multicores issue..
> Date: Thu, 17 Jul 2008 09:55:04 +0100
> 
> 
> thanks Erik, 
> I'm experiencing some problems here.. 
> 
> solr version:  solr-2008-07-07
> Out of the box multicores directory: apache-solr-nightly/example/multicore
> multcore.xml: shown bellow:
> 
> 
>   
>   
> 
> 
> 
> I can easily see the admin page http://localhost:8983/solr/core0/admin/ and 
> http://localhost:8983/solr/core1/admin/ .However, if I click on schema or 
> config it says 
> 
> HTTP ERROR: 404
> 
> NOT_FOUND
> 
> RequestURI=/solr/admin/file/
> 
> 
> So edited the main solrconfg.xml file by uncommenting 
>  
> 
> But I still have the problem... 
> 
> Any thoughts on this ?
> 
> thanks all, 
> ak
> 
> 
>> From: [EMAIL PROTECTED]
>> To: solr-user@lucene.apache.org
>> Subject: Re: 2 IDs in schema.xml
>> Date: Wed, 16 Jul 2008 04:50:18 -0400
>> 
>> 
>> On Jul 16, 2008, at 3:29 AM, dudes dudes wrote:
>>> and also I suppose it is not possible to have 2 schemas on one solr  ?
>> 
>> You can have multiple cores per Solr webapp (see MultiCore in the  
>> wiki).  But each core has a single schema.
>> 
>>  Erik
>> 
> 
> _
> 100’s of Nikon cameras to be won with Live Search
> http://clk.atdmt.com/UKM/go/101719808/direct/01/

_
Find the best and worst places on the planet
http://clk.atdmt.com/UKM/go/101719807/direct/01/

Re: Multiple query fields in DisMax handler

[chris sleeman]

Thanks a lot..this is, more or less, what i was looking for.

However, is there a way to pre-configure the dismax query parser, with
parameters like qf, pf, boost etc., in solr-config.xml, rather than doing so
at query time. So my actual query would look like - <
http://localhost:8983/solr/select?q=
query&fq={!dismaxL}CA&debugQuery=true>,
where dismaxL refers to a query parser defined in solrconfig, with all the
necessary parameters. The q parameter would then use the default dismax
parser defined for the handler and fq would use dismaxL.

Regards,
Chris

On Thu, Jul 17, 2008 at 5:15 AM, Erik Hatcher <[EMAIL PROTECTED]>
wrote:

> On Jul 16, 2008, at 7:38 PM, Ryan McKinley wrote:
>
>> (assuming you are using 1.3-dev), you could use the dismax query parser
>> syntax for the fq param.  I think it is something like:
>> fq=your query
>>
>
> The latest committed syntax is:
>
>   {!dismax qf=""}your query
>
> For example, with the sample data: <
> http://localhost:8983/solr/select?q=*:*&fq={!dismax%20qf=%22name%22}ipod&debugQuery=true
> >
>
>  I can't find the syntax now (Yonik?)
>>
>> but I don't know how you could pull out the qf,pf,etc fields for the fq
>> portion vs the q portion.
>>
>
> You can add parameters like the qf above, within the {!dismax ... } area.
>
>Erik
>
>


-- 
Bill Cosby  - "Advertising is the most fun you can have with your clothes
on."

Re: Multiple query fields in DisMax handler

[Preetam Rao]

If I understand the question correctly, you can provide init params, default
params and invariant params in the appropriate request handler section in
solrconfig.xml.
So you can create a standard request handler with name dismaxL, whose
defType is dismax and set all parameters in defaults section.


Preetam

On Thu, Jul 17, 2008 at 4:35 PM, chris sleeman <[EMAIL PROTECTED]>
wrote:

> Thanks a lot..this is, more or less, what i was looking for.
>
> However, is there a way to pre-configure the dismax query parser, with
> parameters like qf, pf, boost etc., in solr-config.xml, rather than doing
> so
> at query time. So my actual query would look like - <
> http://localhost:8983/solr/select?q=<
> http://localhost:8983/solr/select?q=*:*&fq=%7B%21dismax%20qf=%22name%22%7Dipod&debugQuery=true
> >
> query&fq={!dismaxL}CA&debugQuery=true<
> http://localhost:8983/solr/select?q=*:*&fq=%7B%21dismax%20qf=%22name%22%7Dipod&debugQuery=true
> >>,
> where dismaxL refers to a query parser defined in solrconfig, with all the
> necessary parameters. The q parameter would then use the default dismax
> parser defined for the handler and fq would use dismaxL.
>
> Regards,
> Chris
>
> On Thu, Jul 17, 2008 at 5:15 AM, Erik Hatcher <[EMAIL PROTECTED]>
> wrote:
>
> > On Jul 16, 2008, at 7:38 PM, Ryan McKinley wrote:
> >
> >> (assuming you are using 1.3-dev), you could use the dismax query parser
> >> syntax for the fq param.  I think it is something like:
> >> fq=your query
> >>
> >
> > The latest committed syntax is:
> >
> >   {!dismax qf=""}your query
> >
> > For example, with the sample data: <
> >
> http://localhost:8983/solr/select?q=*:*&fq={!dismax%20qf=%22name%22}ipod&debugQuery=true
> <
> http://localhost:8983/solr/select?q=*:*&fq=%7B%21dismax%20qf=%22name%22%7Dipod&debugQuery=true
> >
> > >
> >
> >  I can't find the syntax now (Yonik?)
> >>
> >> but I don't know how you could pull out the qf,pf,etc fields for the fq
> >> portion vs the q portion.
> >>
> >
> > You can add parameters like the qf above, within the {!dismax ... } area.
> >
> >Erik
> >
> >
>
>
> --
> Bill Cosby  - "Advertising is the most fun you can have with your clothes
> on."
>

Re: Multiple query fields in DisMax handler

[chris sleeman]

What I actually meant was whether or not I could create a configuration for
a dismax query parser and then refer to it in my filter query. I already
have a standard request handler with a "dismax" deftype for my query field.
I wanted to use another dismax parser for the fq param, on the lines of what
Ryan and Erik had suggested. Just dont want to specify all the params for
this dismax at query time.

My actual query would then simply look like - "
http://localhost:8983/solr/select?q=*:*&fq={!dismaxL}CA";, instead of
specifying all the qf, pf, etc fields as part of the dismax syntax within
the query.

Regards,
Chris

On Thu, Jul 17, 2008 at 5:18 PM, Preetam Rao <[EMAIL PROTECTED]>
wrote:

> If I understand the question correctly, you can provide init params,
> default
> params and invariant params in the appropriate request handler section in
> solrconfig.xml.
> So you can create a standard request handler with name dismaxL, whose
> defType is dismax and set all parameters in defaults section.
>
> 
> Preetam
>
> On Thu, Jul 17, 2008 at 4:35 PM, chris sleeman <[EMAIL PROTECTED]>
> wrote:
>
> > Thanks a lot..this is, more or less, what i was looking for.
> >
> > However, is there a way to pre-configure the dismax query parser, with
> > parameters like qf, pf, boost etc., in solr-config.xml, rather than doing
> > so
> > at query time. So my actual query would look like - <
> > http://localhost:8983/solr/select?q=<
> >
> http://localhost:8983/solr/select?q=*:*&fq=%7B%21dismax%20qf=%22name%22%7Dipod&debugQuery=true
> > >
> > query&fq={!dismaxL}CA&debugQuery=true<
> >
> http://localhost:8983/solr/select?q=*:*&fq=%7B%21dismax%20qf=%22name%22%7Dipod&debugQuery=true
> > >>,
> > where dismaxL refers to a query parser defined in solrconfig, with all
> the
> > necessary parameters. The q parameter would then use the default dismax
> > parser defined for the handler and fq would use dismaxL.
> >
> > Regards,
> > Chris
> >
> > On Thu, Jul 17, 2008 at 5:15 AM, Erik Hatcher <
> [EMAIL PROTECTED]>
> > wrote:
> >
> > > On Jul 16, 2008, at 7:38 PM, Ryan McKinley wrote:
> > >
> > >> (assuming you are using 1.3-dev), you could use the dismax query
> parser
> > >> syntax for the fq param.  I think it is something like:
> > >> fq=your query
> > >>
> > >
> > > The latest committed syntax is:
> > >
> > >   {!dismax qf=""}your query
> > >
> > > For example, with the sample data: <
> > >
> >
> http://localhost:8983/solr/select?q=*:*&fq={!dismax%20qf=%22name%22}ipod&debugQuery=true
> <
> http://localhost:8983/solr/select?q=*:*&fq=%7B%21dismax%20qf=%22name%22%7Dipod&debugQuery=true
> >
> > <
> >
> http://localhost:8983/solr/select?q=*:*&fq=%7B%21dismax%20qf=%22name%22%7Dipod&debugQuery=true
> > >
> > > >
> > >
> > >  I can't find the syntax now (Yonik?)
> > >>
> > >> but I don't know how you could pull out the qf,pf,etc fields for the
> fq
> > >> portion vs the q portion.
> > >>
> > >
> > > You can add parameters like the qf above, within the {!dismax ... }
> area.
> > >
> > >Erik
> > >
> > >
> >
> >
> > --
> > Bill Cosby  - "Advertising is the most fun you can have with your clothes
> > on."
> >
>



-- 
Yogi Berra  - "A nickel ain't worth a dime anymore."

Re: Multiple query fields in DisMax handler

[Erik Hatcher]

A custom QParserPlugin could be created and implement an  
#init(NamedList) which you could parameterize via it's solrconfig.xml  
configuration.   That would be one way.   Another trick, I think,  
would be to use request parameter substitution.  The javadocs here  
might lead you to what you're after:





I've not tinkered with this stuff myself other than a bit of trying to  
grok the capabilities Yonik built into this stuff, so having folks  
post back their experiences would be helpful to us all :)


Erik


On Jul 17, 2008, at 8:11 AM, chris sleeman wrote:
What I actually meant was whether or not I could create a  
configuration for
a dismax query parser and then refer to it in my filter query. I  
already
have a standard request handler with a "dismax" deftype for my query  
field.
I wanted to use another dismax parser for the fq param, on the lines  
of what
Ryan and Erik had suggested. Just dont want to specify all the  
params for

this dismax at query time.

My actual query would then simply look like - "
http://localhost:8983/solr/select?q=*:*&fq={!dismaxL}CA";, instead of
specifying all the qf, pf, etc fields as part of the dismax syntax  
within

the query.

Regards,
Chris

On Thu, Jul 17, 2008 at 5:18 PM, Preetam Rao <[EMAIL PROTECTED]>
wrote:


If I understand the question correctly, you can provide init params,
default
params and invariant params in the appropriate request handler  
section in

solrconfig.xml.
So you can create a standard request handler with name dismaxL, whose
defType is dismax and set all parameters in defaults section.


Preetam

On Thu, Jul 17, 2008 at 4:35 PM, chris sleeman <[EMAIL PROTECTED] 
>

wrote:


Thanks a lot..this is, more or less, what i was looking for.

However, is there a way to pre-configure the dismax query parser,  
with
parameters like qf, pf, boost etc., in solr-config.xml, rather  
than doing

so
at query time. So my actual query would look like - <
http://localhost:8983/solr/select?q=<


http://localhost:8983/solr/select?q=*:*&fq=%7B%21dismax%20qf=%22name%22%7Dipod&debugQuery=true



query&fq={!dismaxL}CA&debugQuery=true<


http://localhost:8983/solr/select?q=*:*&fq=%7B%21dismax%20qf=%22name%22%7Dipod&debugQuery=true

,
where dismaxL refers to a query parser defined in solrconfig, with  
all

the
necessary parameters. The q parameter would then use the default  
dismax

parser defined for the handler and fq would use dismaxL.

Regards,
Chris

On Thu, Jul 17, 2008 at 5:15 AM, Erik Hatcher <

[EMAIL PROTECTED]>

wrote:


On Jul 16, 2008, at 7:38 PM, Ryan McKinley wrote:


(assuming you are using 1.3-dev), you could use the dismax query

parser

syntax for the fq param.  I think it is something like:
fq=your query



The latest committed syntax is:

 {!dismax qf=""}your query

For example, with the sample data: <



http://localhost:8983/solr/select?q=*:*&fq={!dismax%20qf=%22name 
%22}ipod&debugQuery=true

<
http://localhost:8983/solr/select?q=*:*&fq=%7B%21dismax%20qf=%22name%22%7Dipod&debugQuery=true


<


http://localhost:8983/solr/select?q=*:*&fq=%7B%21dismax%20qf=%22name%22%7Dipod&debugQuery=true






I can't find the syntax now (Yonik?)


but I don't know how you could pull out the qf,pf,etc fields for  
the

fq

portion vs the q portion.



You can add parameters like the qf above, within the {!dismax ... }

area.


  Erik





--
Bill Cosby  - "Advertising is the most fun you can have with your  
clothes

on."







--
Yogi Berra  - "A nickel ain't worth a dime anymore."

Re: Solr stops responding

[Doug Steigerwald]

It happened again last night.  I cronned a script that ran jstack on  
the process every 5 minutes just to see what was going on.  Here's a  
snippet:


"btpool0-2668" prio=10 tid=0x2aac3a905800 nid=0x76ed waiting for  
monitor entry [0x5e584000..0x5e585a10]

   java.lang.Thread.State: BLOCKED (on object monitor)
at org.apache.solr.search.LRUCache.get(LRUCache.java:129)
- waiting to lock <0x2aaabcdd9450> (a  
org.apache.solr.search.LRUCache$1)
at  
org 
.apache 
.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:730)
at  
org 
.apache 
.solr.search.SolrIndexSearcher.getDocList(SolrIndexSearcher.java:693)
at  
org.apache.solr.search.CollapseFilter.(CollapseFilter.java:137)
at  
org 
.apache 
.solr 
.handler.component.CollapseComponent.process(CollapseComponent.java:97)
at  
org 
.apache 
.solr 
.handler.component.SearchHandler.handleRequestBody(SearchHandler.java: 
148)
at  
org 
.apache 
.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java: 
117)

at org.apache.solr.core.SolrCore.execute(SolrCore.java:942)
at  
org 
.apache 
.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:280)
at  
org 
.apache 
.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:237)


During this log, there were 547 threads active (going by occurrences  
of Thread.State in the log).


Here's some more:

"btpool0-2051" prio=10 tid=0x2aac39144c00 nid=0x4012 waiting for  
monitor entry [0x45bfc000..0x45bfdd90]

   java.lang.Thread.State: BLOCKED (on object monitor)
at java.util.Vector.size(Unknown Source)
- waiting to lock <0x2aaac0af0ea0> (a java.util.Vector)
at java.util.AbstractList.listIterator(Unknown Source)
at java.util.AbstractList.listIterator(Unknown Source)
at java.util.AbstractList.equals(Unknown Source)
at java.util.Vector.equals(Unknown Source)
- locked <0x2aaac0ae8d30> (a java.util.Vector)
at  
org.apache.lucene.search.PhraseQuery.equals(PhraseQuery.java:286)

at java.util.AbstractList.equals(Unknown Source)
at  
org 
.apache 
.lucene.search.DisjunctionMaxQuery.equals(DisjunctionMaxQuery.java:243)
at  
org.apache.lucene.search.BooleanClause.equals(BooleanClause.java:102)

at java.util.AbstractList.equals(Unknown Source)
at  
org.apache.lucene.search.BooleanQuery.equals(BooleanQuery.java:461)

at java.util.HashMap.getEntry(Unknown Source)
at java.util.LinkedHashMap.get(Unknown Source)
at org.apache.solr.search.LRUCache.get(LRUCache.java:129)
- locked <0x2aaabcdd51f8> (a  
org.apache.solr.search.LRUCache$1)


This was also at the bottom of the jstack dump:

Found one Java-level deadlock:
=
"btpool0-2782":
  waiting to lock monitor 0x41e6c568 (object  
0x2aaabcdd9450, a org.apache.solr.search.LRUCache$1),

  which is held by "btpool0-2063"
"btpool0-2063":
  waiting to lock monitor 0x41e6b068 (object  
0x2aaac0ae8d30, a java.util.Vector),

  which is held by "btpool0-2051"
"btpool0-2051":
  waiting to lock monitor 0x41e6b110 (object  
0x2aaac0af0ea0, a java.util.Vector),

  which is held by "btpool0-2063"

Java stack information for the threads listed above:
===
"btpool0-2782":
at org.apache.solr.search.LRUCache.get(LRUCache.java:129)
- waiting to lock <0x2aaabcdd9450> (a  
org.apache.solr.search.LRUCache$1)
at  
org 
.apache 
.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:730)
at  
org 
.apache 
.solr.search.SolrIndexSearcher.getDocList(SolrIndexSearcher.java:693)
at  
org.apache.solr.search.CollapseFilter.(CollapseFilter.java:137)
at  
org 
.apache 
.solr 
.handler.component.CollapseComponent.process(CollapseComponent.java:97)
at  
org 
.apache 
.solr 
.handler.component.SearchHandler.handleRequestBody(SearchHandler.java: 
148)
at  
org 
.apache 
.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java: 
117)

at org.apache.solr.core.SolrCore.execute(SolrCore.java:942)
at  
org 
.apache 
.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:280)
at  
org 
.apache 
.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:237)



Thanks.
Doug

On Jul 15, 2008, at 11:39 AM, Noble Paul നോബിള്‍  
नोब्ळ् wrote:



Can we collect more information. It would be nice to know what the
threads are doing when it hangs.
If you are using *nix  issue kill -3 
it would print out the stacktrace of all the threads in the VM . That
may tell us what is the state of each thread which could help us
suggest something


On Tue, Jul 15, 2008 at 8:59 PM, Fuad Efendi <[EMAIL PROTECTED]> wrote:
I constantly have the same problem; sometimes I have  
OutOfMemoryError i

Re: Multiple query fields in DisMax handler

[Preetam Rao]

I see that a QParser takes local params (those given via {!...} )as well as
request params. It sets the lookup chain as local followed be request
params. AFAIK, the request param lookup chain  is set up as -
those given in the url explicitly, then invariants, then defaults gievn in
solrconfig for the corresponding request handler.

Since you are not using dismax params for the main query and just want them
to be available for the Dismax parser, there are no conflicts and I think
you can set the qf, bf etc in the named standard request handler that you
are configuring in solrconfig and dismax query parser will automatically
pick it up.

--
Preetam


On Thu, Jul 17, 2008 at 5:48 PM, Erik Hatcher <[EMAIL PROTECTED]>
wrote:

> A custom QParserPlugin could be created and implement an #init(NamedList)
> which you could parameterize via it's solrconfig.xml configuration.   That
> would be one way.   Another trick, I think, would be to use request
> parameter substitution.  The javadocs here might lead you to what you're
> after:
>
> <
> http://lucene.apache.org/solr/api/org/apache/solr/search/NestedQParserPlugin.html
> >
>
> I've not tinkered with this stuff myself other than a bit of trying to grok
> the capabilities Yonik built into this stuff, so having folks post back
> their experiences would be helpful to us all :)
>
>Erik
>
>
>
> On Jul 17, 2008, at 8:11 AM, chris sleeman wrote:
>
>> What I actually meant was whether or not I could create a configuration
>> for
>> a dismax query parser and then refer to it in my filter query. I already
>> have a standard request handler with a "dismax" deftype for my query
>> field.
>> I wanted to use another dismax parser for the fq param, on the lines of
>> what
>> Ryan and Erik had suggested. Just dont want to specify all the params for
>> this dismax at query time.
>>
>> My actual query would then simply look like - "
>> http://localhost:8983/solr/select?q=*:*&fq={!dismaxL}CA",
>> instead of
>> specifying all the qf, pf, etc fields as part of the dismax syntax within
>> the query.
>>
>> Regards,
>> Chris
>>
>> On Thu, Jul 17, 2008 at 5:18 PM, Preetam Rao <[EMAIL PROTECTED]>
>> wrote:
>>
>>  If I understand the question correctly, you can provide init params,
>>> default
>>> params and invariant params in the appropriate request handler section in
>>> solrconfig.xml.
>>> So you can create a standard request handler with name dismaxL, whose
>>> defType is dismax and set all parameters in defaults section.
>>>
>>> 
>>> Preetam
>>>
>>> On Thu, Jul 17, 2008 at 4:35 PM, chris sleeman <[EMAIL PROTECTED]>
>>> wrote:
>>>
>>>  Thanks a lot..this is, more or less, what i was looking for.

 However, is there a way to pre-configure the dismax query parser, with
 parameters like qf, pf, boost etc., in solr-config.xml, rather than
 doing
 so
 at query time. So my actual query would look like - <
 http://localhost:8983/solr/select?q=<


>>> http://localhost:8983/solr/select?q=*:*&fq=%7B%21dismax%20qf=%22name%22%7Dipod&debugQuery=true
>>>

>  query&fq={!dismaxL}CA&debugQuery=true<


>>> http://localhost:8983/solr/select?q=*:*&fq=%7B%21dismax%20qf=%22name%22%7Dipod&debugQuery=true
>>>
 ,
>>
> where dismaxL refers to a query parser defined in solrconfig, with all

>>> the
>>>
 necessary parameters. The q parameter would then use the default dismax
 parser defined for the handler and fq would use dismaxL.

 Regards,
 Chris

 On Thu, Jul 17, 2008 at 5:15 AM, Erik Hatcher <

>>> [EMAIL PROTECTED]>
>>>
 wrote:

  On Jul 16, 2008, at 7:38 PM, Ryan McKinley wrote:
>
>  (assuming you are using 1.3-dev), you could use the dismax query
>>
> parser
>>>
 syntax for the fq param.  I think it is something like:
>> fq=your query
>>
>>
> The latest committed syntax is:
>
>  {!dismax qf=""}your query
>
> For example, with the sample data: <
>
>
  
 http://localhost:8983/solr/select?q=*:*&fq={!dismax%20qf=%22name
>>> %22}ipod&debugQuery=true<
>>> http://localhost:8983/solr/select?q=*:*&fq=%7B%21dismax%20qf=%22name%22%7Dipod&debugQuery=true
>>> >
>>> <
>>>
>>> http://localhost:8983/solr/select?q=*:*&fq=%7B%21dismax%20qf=%22name%22%7Dipod&debugQuery=true
>>>

 <


>>> http://localhost:8983/solr/select?q=*:*&fq=%7B%21dismax%20qf=%22name%22%7Dipod&debugQuery=true
>>>

>
>>
> I can't find the syntax now (Yonik?)
>
>>
>> but I don't know how you could pull out the qf,pf,etc fields for the
>>
> fq
>>>
 portion vs the q portion.
>>
>>
> You can add parameters like the qf above, within the {!dismax ... }
>
 area.
>>>

>  Erik
>
>
>

 --
 Bill Cosby  - "Advertis

Re: Multiple query fields in DisMax handler

[Preetam Rao]

Oops.. this will only help you configure only the defaults common to the
main dismax query as well as the fq dismax query.

For creating two sets of dismax parsers which are named and want to read
params from solrconfig, I think one can extend the dismaxQParser's currently
empty init() method to setup the param hierarchy during the query time such
that the lookup chain becomes - local params, then qparser init params and
then request params. It is along similar lines to Eric's suggestion. But I
am not sure if we want to create a custom parser for this or make it a
general behavior..

---
preetam

On Thu, Jul 17, 2008 at 6:21 PM, Preetam Rao <[EMAIL PROTECTED]>
wrote:

> I see that a QParser takes local params (those given via {!...} )as well as
> request params. It sets the lookup chain as local followed be request
> params. AFAIK, the request param lookup chain  is set up as -
> those given in the url explicitly, then invariants, then defaults gievn in
> solrconfig for the corresponding request handler.
>
> Since you are not using dismax params for the main query and just want them
> to be available for the Dismax parser, there are no conflicts and I think
> you can set the qf, bf etc in the named standard request handler that you
> are configuring in solrconfig and dismax query parser will automatically
> pick it up.
>
> --
> Preetam
>
>
>
> On Thu, Jul 17, 2008 at 5:48 PM, Erik Hatcher <[EMAIL PROTECTED]>
> wrote:
>
>> A custom QParserPlugin could be created and implement an #init(NamedList)
>> which you could parameterize via it's solrconfig.xml configuration.   That
>> would be one way.   Another trick, I think, would be to use request
>> parameter substitution.  The javadocs here might lead you to what you're
>> after:
>>
>> <
>> http://lucene.apache.org/solr/api/org/apache/solr/search/NestedQParserPlugin.html
>> >
>>
>> I've not tinkered with this stuff myself other than a bit of trying to
>> grok the capabilities Yonik built into this stuff, so having folks post back
>> their experiences would be helpful to us all :)
>>
>>Erik
>>
>>
>>
>> On Jul 17, 2008, at 8:11 AM, chris sleeman wrote:
>>
>>> What I actually meant was whether or not I could create a configuration
>>> for
>>> a dismax query parser and then refer to it in my filter query. I already
>>> have a standard request handler with a "dismax" deftype for my query
>>> field.
>>> I wanted to use another dismax parser for the fq param, on the lines of
>>> what
>>> Ryan and Erik had suggested. Just dont want to specify all the params for
>>> this dismax at query time.
>>>
>>> My actual query would then simply look like - "
>>> http://localhost:8983/solr/select?q=*:*&fq={!dismaxL}CA",
>>> instead of
>>> specifying all the qf, pf, etc fields as part of the dismax syntax within
>>> the query.
>>>
>>> Regards,
>>> Chris
>>>
>>> On Thu, Jul 17, 2008 at 5:18 PM, Preetam Rao <[EMAIL PROTECTED]>
>>> wrote:
>>>
>>>  If I understand the question correctly, you can provide init params,
 default
 params and invariant params in the appropriate request handler section
 in
 solrconfig.xml.
 So you can create a standard request handler with name dismaxL, whose
 defType is dismax and set all parameters in defaults section.

 
 Preetam

 On Thu, Jul 17, 2008 at 4:35 PM, chris sleeman <[EMAIL PROTECTED]
 >
 wrote:

  Thanks a lot..this is, more or less, what i was looking for.
>
> However, is there a way to pre-configure the dismax query parser, with
> parameters like qf, pf, boost etc., in solr-config.xml, rather than
> doing
> so
> at query time. So my actual query would look like - <
> http://localhost:8983/solr/select?q=<
>
>
 http://localhost:8983/solr/select?q=*:*&fq=%7B%21dismax%20qf=%22name%22%7Dipod&debugQuery=true

>
>>  query&fq={!dismaxL}CA&debugQuery=true<
>
>
 http://localhost:8983/solr/select?q=*:*&fq=%7B%21dismax%20qf=%22name%22%7Dipod&debugQuery=true

>  ,
>>>
>> where dismaxL refers to a query parser defined in solrconfig, with all
>
 the

> necessary parameters. The q parameter would then use the default dismax
> parser defined for the handler and fq would use dismaxL.
>
> Regards,
> Chris
>
> On Thu, Jul 17, 2008 at 5:15 AM, Erik Hatcher <
>
 [EMAIL PROTECTED]>

> wrote:
>
>  On Jul 16, 2008, at 7:38 PM, Ryan McKinley wrote:
>>
>>  (assuming you are using 1.3-dev), you could use the dismax query
>>>
>> parser

>  syntax for the fq param.  I think it is something like:
>>> fq=your query
>>>
>>>
>> The latest committed syntax is:
>>
>>  {!dismax qf=""}your query
>>
>> For example, with the sample data: <
>>
>>
>  
> http://localhost:8983/solr/select?q=*:*&fq={!dismax

Re: Multiple query fields in DisMax handler

[Yonik Seeley]

On Thu, Jul 17, 2008 at 8:11 AM, chris sleeman <[EMAIL PROTECTED]> wrote:
> What I actually meant was whether or not I could create a configuration for
> a dismax query parser and then refer to it in my filter query. I already
> have a standard request handler with a "dismax" deftype for my query field.
> I wanted to use another dismax parser for the fq param, on the lines of what
> Ryan and Erik had suggested. Just dont want to specify all the params for
> this dismax at query time.

To separate the configuration from the client, you could at least grab
the parameter values from config (you still would need to specify the
parameter names though).  For a filter, some of the params you would
simply want to zero out.

fq={!dismax qf=$filter_qf pf= bf=}CA
  and set filter_qf as a default in solrconfig.xml
  OR separating out the actual query value to make this easier to compose:
fq={!dismax qf=$filter_qf pf= bf= v=$q2}&q2=CA

That's currently the closest you can get out of the box I think.

-Yonik

> My actual query would then simply look like - "
> http://localhost:8983/solr/select?q=*:*&fq={!dismaxL}CA";, instead of
> specifying all the qf, pf, etc fields as part of the dismax syntax within
> the query.
>
> Regards,
> Chris
>
> On Thu, Jul 17, 2008 at 5:18 PM, Preetam Rao <[EMAIL PROTECTED]>
> wrote:
>
>> If I understand the question correctly, you can provide init params,
>> default
>> params and invariant params in the appropriate request handler section in
>> solrconfig.xml.
>> So you can create a standard request handler with name dismaxL, whose
>> defType is dismax and set all parameters in defaults section.
>>
>> 
>> Preetam
>>
>> On Thu, Jul 17, 2008 at 4:35 PM, chris sleeman <[EMAIL PROTECTED]>
>> wrote:
>>
>> > Thanks a lot..this is, more or less, what i was looking for.
>> >
>> > However, is there a way to pre-configure the dismax query parser, with
>> > parameters like qf, pf, boost etc., in solr-config.xml, rather than doing
>> > so
>> > at query time. So my actual query would look like - <
>> > http://localhost:8983/solr/select?q=<
>> >
>> http://localhost:8983/solr/select?q=*:*&fq=%7B%21dismax%20qf=%22name%22%7Dipod&debugQuery=true
>> > >
>> > query&fq={!dismaxL}CA&debugQuery=true<
>> >
>> http://localhost:8983/solr/select?q=*:*&fq=%7B%21dismax%20qf=%22name%22%7Dipod&debugQuery=true
>> > >>,
>> > where dismaxL refers to a query parser defined in solrconfig, with all
>> the
>> > necessary parameters. The q parameter would then use the default dismax
>> > parser defined for the handler and fq would use dismaxL.
>> >
>> > Regards,
>> > Chris
>> >
>> > On Thu, Jul 17, 2008 at 5:15 AM, Erik Hatcher <
>> [EMAIL PROTECTED]>
>> > wrote:
>> >
>> > > On Jul 16, 2008, at 7:38 PM, Ryan McKinley wrote:
>> > >
>> > >> (assuming you are using 1.3-dev), you could use the dismax query
>> parser
>> > >> syntax for the fq param.  I think it is something like:
>> > >> fq=your query
>> > >>
>> > >
>> > > The latest committed syntax is:
>> > >
>> > >   {!dismax qf=""}your query
>> > >
>> > > For example, with the sample data: <
>> > >
>> >
>> http://localhost:8983/solr/select?q=*:*&fq={!dismax%20qf=%22name%22}ipod&debugQuery=true
>> <
>> http://localhost:8983/solr/select?q=*:*&fq=%7B%21dismax%20qf=%22name%22%7Dipod&debugQuery=true
>> >
>> > <
>> >
>> http://localhost:8983/solr/select?q=*:*&fq=%7B%21dismax%20qf=%22name%22%7Dipod&debugQuery=true
>> > >
>> > > >
>> > >
>> > >  I can't find the syntax now (Yonik?)
>> > >>
>> > >> but I don't know how you could pull out the qf,pf,etc fields for the
>> fq
>> > >> portion vs the q portion.
>> > >>
>> > >
>> > > You can add parameters like the qf above, within the {!dismax ... }
>> area.
>> > >
>> > >Erik
>> > >
>> > >
>> >
>> >
>> > --
>> > Bill Cosby  - "Advertising is the most fun you can have with your clothes
>> > on."
>> >
>>
>
>
>
> --
> Yogi Berra  - "A nickel ain't worth a dime anymore."
>

Specifying explicit FacetQuery w/ a normal query?

[Jon Baer]

Ive gone from a complex multicore setup back to a single solrconfig  
setup and using a doctype field (since the index is pretty small),  
however there are a few spots where items are laid out in tabs and  
each tab has a count of docs associated, ie:


News (123) | Images (345) | Video (678) | Blogs (901)

Unfortunately the tab controlling is server side and Im trying to grab  
a facet count on doctype w/ a filter query and can't seem to do it w/o  
having to send the small facet query (for the counts on all items) and  
the filter query itself.  Is there any way to do this in a single  
request w/ any params Im missing?  (Using SolrJ if that helps).


Thanks.

- Jon 
 

Re: Using custom Similarity class

[Koji Sekiguchi]

> com.example.CustomSimilarity cannot be cast to
> org.apache.lucene.search.Similarity
> at org.apache.solr.schema.IndexSchema.readConfig(IndexSchema.java:449)
> ... 28 more

I think you've got a class loader problem.
If you have solr-1.2.0 source code, see the line 499 of IndexSchema.java:

 similarity = 
(Similarity)Config.newInstance(node.getNodeValue().trim());


Change the line to:

Object o = Config.newInstance(node.getNodeValue().trim());
System.err.println("(1) The object " + o + " classloader is " + 
o.getClass().getClassLoader());
System.err.println("(2) Class Similarity class loader is " + 
Similarity.class.getClassLoader());

similarity = (Similarity) o;

You will see different classloader between (1) and (2).
I'm not familiar with jetty classloader, but you can
google "jetty classloader" to get a solution.

Cheers,

Koji

Re: Multiple query fields in DisMax handler

[chris sleeman]

Thanks a ton...this is quite useful.

Regards,
Chris

On Thu, Jul 17, 2008 at 6:42 PM, Yonik Seeley <[EMAIL PROTECTED]> wrote:

> On Thu, Jul 17, 2008 at 8:11 AM, chris sleeman <[EMAIL PROTECTED]>
> wrote:
> > What I actually meant was whether or not I could create a configuration
> for
> > a dismax query parser and then refer to it in my filter query. I already
> > have a standard request handler with a "dismax" deftype for my query
> field.
> > I wanted to use another dismax parser for the fq param, on the lines of
> what
> > Ryan and Erik had suggested. Just dont want to specify all the params for
> > this dismax at query time.
>
> To separate the configuration from the client, you could at least grab
> the parameter values from config (you still would need to specify the
> parameter names though).  For a filter, some of the params you would
> simply want to zero out.
>
> fq={!dismax qf=$filter_qf pf= bf=}CA
>  and set filter_qf as a default in solrconfig.xml
>  OR separating out the actual query value to make this easier to compose:
> fq={!dismax qf=$filter_qf pf= bf= v=$q2}&q2=CA
>
> That's currently the closest you can get out of the box I think.
>
> -Yonik
>
> > My actual query would then simply look like - "
> > http://localhost:8983/solr/select?q=*:*&fq={!dismaxL}CA",
> instead of
> > specifying all the qf, pf, etc fields as part of the dismax syntax within
> > the query.
> >
> > Regards,
> > Chris
> >
> > On Thu, Jul 17, 2008 at 5:18 PM, Preetam Rao <[EMAIL PROTECTED]>
> > wrote:
> >
> >> If I understand the question correctly, you can provide init params,
> >> default
> >> params and invariant params in the appropriate request handler section
> in
> >> solrconfig.xml.
> >> So you can create a standard request handler with name dismaxL, whose
> >> defType is dismax and set all parameters in defaults section.
> >>
> >> 
> >> Preetam
> >>
> >> On Thu, Jul 17, 2008 at 4:35 PM, chris sleeman <[EMAIL PROTECTED]
> >
> >> wrote:
> >>
> >> > Thanks a lot..this is, more or less, what i was looking for.
> >> >
> >> > However, is there a way to pre-configure the dismax query parser, with
> >> > parameters like qf, pf, boost etc., in solr-config.xml, rather than
> doing
> >> > so
> >> > at query time. So my actual query would look like - <
> >> > http://localhost:8983/solr/select?q=<
> >> >
> >>
> http://localhost:8983/solr/select?q=*:*&fq=%7B%21dismax%20qf=%22name%22%7Dipod&debugQuery=true
> >> > >
> >> > query&fq={!dismaxL}CA&debugQuery=true<
> >> >
> >>
> http://localhost:8983/solr/select?q=*:*&fq=%7B%21dismax%20qf=%22name%22%7Dipod&debugQuery=true
> >> > >>,
> >> > where dismaxL refers to a query parser defined in solrconfig, with all
> >> the
> >> > necessary parameters. The q parameter would then use the default
> dismax
> >> > parser defined for the handler and fq would use dismaxL.
> >> >
> >> > Regards,
> >> > Chris
> >> >
> >> > On Thu, Jul 17, 2008 at 5:15 AM, Erik Hatcher <
> >> [EMAIL PROTECTED]>
> >> > wrote:
> >> >
> >> > > On Jul 16, 2008, at 7:38 PM, Ryan McKinley wrote:
> >> > >
> >> > >> (assuming you are using 1.3-dev), you could use the dismax query
> >> parser
> >> > >> syntax for the fq param.  I think it is something like:
> >> > >> fq=your query
> >> > >>
> >> > >
> >> > > The latest committed syntax is:
> >> > >
> >> > >   {!dismax qf=""}your query
> >> > >
> >> > > For example, with the sample data: <
> >> > >
> >> >
> >>
> http://localhost:8983/solr/select?q=*:*&fq={!dismax%20qf=%22name%22}ipod&debugQuery=true
> <
> http://localhost:8983/solr/select?q=*:*&fq=%7B%21dismax%20qf=%22name%22%7Dipod&debugQuery=true
> >
> >> <
> >>
> http://localhost:8983/solr/select?q=*:*&fq=%7B%21dismax%20qf=%22name%22%7Dipod&debugQuery=true
> >> >
> >> > <
> >> >
> >>
> http://localhost:8983/solr/select?q=*:*&fq=%7B%21dismax%20qf=%22name%22%7Dipod&debugQuery=true
> >> > >
> >> > > >
> >> > >
> >> > >  I can't find the syntax now (Yonik?)
> >> > >>
> >> > >> but I don't know how you could pull out the qf,pf,etc fields for
> the
> >> fq
> >> > >> portion vs the q portion.
> >> > >>
> >> > >
> >> > > You can add parameters like the qf above, within the {!dismax ... }
> >> area.
> >> > >
> >> > >Erik
> >> > >
> >> > >
> >> >
> >> >
> >> > --
> >> > Bill Cosby  - "Advertising is the most fun you can have with your
> clothes
> >> > on."
> >> >
> >>
> >
> >
> >
> > --
> > Yogi Berra  - "A nickel ain't worth a dime anymore."
> >
>



-- 
Yogi Berra  - "A nickel ain't worth a dime anymore."

Re: Solr stops responding

[Fuad Efendi]

Thanks Doug,

Would be nice to have stacktrace for btpool0-2051...

I analyzed LUCache several times, of course it has strange  
synchronization statements like



  public void warm(SolrIndexSearcher searcher, SolrCache old) throws  
IOException {

...
LRUCache other = (LRUCache)old;
...
  synchronized ( ??? other.map ??? ) {
...


/** A Query that matches documents containing a particular sequence of terms.
 * A PhraseQuery is built by QueryParser for input like "new  
york".

 *
 * This query may be combined with other terms or queries with a  
[EMAIL PROTECTED] BooleanQuery}.

 */
org.apache.lucene.search.PhraseQuery
  /** Returns true iff o is equal to this. */
  public boolean equals(Object o) {
if (!(o instanceof PhraseQuery))
  return false;
PhraseQuery other = (PhraseQuery)o;
return (this.getBoost() == other.getBoost()) //*PhraseQuery.java:286*//
  && (this.slop == other.slop)
  &&  this.terms.equals(other.terms)
  && this.positions.equals(other.positions);
  }


terms & positions: java.util.Vector (synchronized)


- I don't think it may cause deadlock... at least we can stress-test  
using "New York"



Lucene & SOLR developers should avoid using java.util.Vector which is  
extremely prone to deadlocks in trivial cases...



P.S.
72 hours runtime: OOM problem with BEA JRockit R27:  
jrockit-R27.4.0-jdk1.6.0_02 (AMD Opteron, 64bit, SLES 10 SP1, Tomcat  
5.5.26). 100k queries a day...


Jul 17, 2008 11:08:07 AM org.apache.solr.common.SolrException log
SEVERE: java.lang.OutOfMemoryError: allocLargeObjectOrArray - Object  
size: 3149016, Num elements: 393625

at org.apache.solr.util.OpenBitSet.(OpenBitSet.java:86)
at  
org.apache.solr.search.DocSetHitCollector.collect(DocSetHitCollector.java:63)
at  
org.apache.solr.search.SolrIndexSearcher$9.collect(SolrIndexSearcher.java:1072)
at  
org.apache.lucene.search.BooleanScorer2.score(BooleanScorer2.java:320)
at  
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:146)

at org.apache.lucene.search.Searcher.search(Searcher.java:118)
at org.apache.lucene.search.Searcher.search(Searcher.java:97)
at  
org.apache.solr.search.SolrIndexSearcher.getDocListAndSetNC(SolrIndexSearcher.java:1069)
at  
org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:804)
at  
org.apache.solr.search.SolrIndexSearcher.getDocListAndSet(SolrIndexSearcher.java:1245)
at  
org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:96)
at  
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:148)
at  
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:117)

at org.apache.solr.core.SolrCore.execute(SolrCore.java:902)
at  
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:280)
at  
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:237)
at  
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
at  
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
at  
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)
at  
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:174)
at  
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at  
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117)
at  
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:108)
at  
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:174)
at  
org.apache.coyote.http11.Http11AprProcessor.process(Http11AprProcessor.java:834)
at  
org.apache.coyote.http11.Http11AprProtocol$Http11ConnectionHandler.process(Http11AprProtocol.java:640)
at  
org.apache.tomcat.util.net.AprEndpoint$Worker.run(AprEndpoint.java:1286)

at java.lang.Thread.run(Thread.java:619)


P.P.S.
I'll send thread dump in separate Email



Quoting Doug Steigerwald <[EMAIL PROTECTED]>:


It happened again last night.  I cronned a script that ran jstack on
the process every 5 minutes just to see what was going on.  Here's a
snippet:

"btpool0-2668" prio=10 tid=0x2aac3a905800 nid=0x76ed waiting for
monitor entry [0x5e584000..0x5e585a10]
   java.lang.Thread.State: BLOCKED (on object monitor)
at org.apache.solr.search.LRUCache.get(LRUCache.java:129)
- waiting to lock <0x2aaabcdd9450> (a
org.apache.solr.search.LRUCache$1)
at
org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:730)
at
org.apache.solr.search.SolrIndexSearcher.getDocList(SolrIndexSearcher.java:693)
at
org.apache.solr.search.CollapseFilter.(CollapseFilter.java:137)
a

Re: Using custom Similarity class

[Sébastien Rainville]

Thanks for the tip but I solved it but using the old way of loading custom
libs into solr: unpack the war file, add the custom library to WEB-INF/lib
and repack it. It definetly is a class loader problem, as the wiki specifies
that the new way of loading custom jars is using a custom class loader that
*might* not work...

Sebastien




On Thu, Jul 17, 2008 at 10:35 AM, Koji Sekiguchi <[EMAIL PROTECTED]> wrote:

> > com.example.CustomSimilarity cannot be cast to
> > org.apache.lucene.search.Similarity
> > at org.apache.solr.schema.IndexSchema.readConfig(IndexSchema.java:449)
> > ... 28 more
>
> I think you've got a class loader problem.
> If you have solr-1.2.0 source code, see the line 499 of IndexSchema.java:
>
> similarity =
> (Similarity)Config.newInstance(node.getNodeValue().trim());
>
> Change the line to:
>
> Object o = Config.newInstance(node.getNodeValue().trim());
> System.err.println("(1) The object " + o + " classloader is " +
> o.getClass().getClassLoader());
> System.err.println("(2) Class Similarity class loader is " +
> Similarity.class.getClassLoader());
> similarity = (Similarity) o;
>
> You will see different classloader between (1) and (2).
> I'm not familiar with jetty classloader, but you can
> google "jetty classloader" to get a solution.
>
> Cheers,
>
> Koji
>
>

Re: Solr stops responding

[Doug Steigerwald]

I included part of it in my last email, but here's the full one:

"btpool0-2051":
at java.util.Vector.size(Unknown Source)
- waiting to lock <0x2aaac0af0ea0> (a java.util.Vector)
at java.util.AbstractList.listIterator(Unknown Source)
at java.util.AbstractList.listIterator(Unknown Source)
at java.util.AbstractList.equals(Unknown Source)
at java.util.Vector.equals(Unknown Source)
- locked <0x2aaac0ae8d30> (a java.util.Vector)
at  
org.apache.lucene.search.PhraseQuery.equals(PhraseQuery.java:286)

at java.util.AbstractList.equals(Unknown Source)
at  
org 
.apache 
.lucene.search.DisjunctionMaxQuery.equals(DisjunctionMaxQuery.java:243)
at  
org.apache.lucene.search.BooleanClause.equals(BooleanClause.java:102)

at java.util.AbstractList.equals(Unknown Source)
at  
org.apache.lucene.search.BooleanQuery.equals(BooleanQuery.java:461)

at java.util.HashMap.getEntry(Unknown Source)
at java.util.LinkedHashMap.get(Unknown Source)
at org.apache.solr.search.LRUCache.get(LRUCache.java:129)
- locked <0x2aaabcdd51f8> (a  
org.apache.solr.search.LRUCache$1)
at  
org 
.apache.solr.search.SolrIndexSearcher.getDocSet(SolrIndexSearcher.java: 
621)
at  
org 
.apache 
.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:796)
at  
org 
.apache 
.solr.search.SolrIndexSearcher.getDocListAndSet(SolrIndexSearcher.java: 
1258)
at  
org 
.apache 
.solr 
.handler.component.CollapseComponent.process(CollapseComponent.java:102)
at  
org 
.apache 
.solr 
.handler.component.SearchHandler.handleRequestBody(SearchHandler.java: 
148)
at  
org 
.apache 
.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java: 
117)

at org.apache.solr.core.SolrCore.execute(SolrCore.java:942)
at  
org 
.apache 
.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:280)
at  
org 
.apache 
.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:237)
at org.mortbay.jetty.servlet.ServletHandler 
$CachedChain.doFilter(ServletHandler.java:1089)
at  
org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:365)
at  
org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java: 
216)
at  
org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181)
at  
org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:712)
at  
org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:405)
at  
org 
.mortbay 
.jetty 
.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java: 
211)
at  
org 
.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java: 
114)
at  
org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:139)

at org.mortbay.jetty.Server.handle(Server.java:285)
at  
org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:502)
at org.mortbay.jetty.HttpConnection 
$RequestHandler.headerComplete(HttpConnection.java:821)

at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:513)
at  
org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:208)
at  
org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:378)
at org.mortbay.jetty.bio.SocketConnector 
$Connection.run(SocketConnector.java:226)
at org.mortbay.thread.BoundedThreadPool 
$PoolThread.run(BoundedThreadPool.java:442)



Doug

On Jul 17, 2008, at 11:13 AM, Fuad Efendi wrote:


Thanks Doug,

Would be nice to have stacktrace for btpool0-2051...

I analyzed LUCache several times, of course it has strange  
synchronization statements like



 public void warm(SolrIndexSearcher searcher, SolrCache old) throws  
IOException {

...
   LRUCache other = (LRUCache)old;
...
 synchronized ( ??? other.map ??? ) {
...


/** A Query that matches documents containing a particular sequence  
of terms.
* A PhraseQuery is built by QueryParser for input like "new  
york".

*
* This query may be combined with other terms or queries with a  
[EMAIL PROTECTED] BooleanQuery}.

*/
org.apache.lucene.search.PhraseQuery
 /** Returns true iff o is equal to this. */
 public boolean equals(Object o) {
   if (!(o instanceof PhraseQuery))
 return false;
   PhraseQuery other = (PhraseQuery)o;
   return (this.getBoost() == other.getBoost()) //*PhraseQuery.java: 
286*//

 && (this.slop == other.slop)
 &&  this.terms.equals(other.terms)
 && this.positions.equals(other.positions);
 }


terms & positions: java.util.Vector (synchronized)


- I don't think it may cause deadlock... at least we can stress-test  
using "New York"



Lucene & SOLR developers should avoid using java.util.Vector which  
is extremely prone to deadlocks in trivial cases...



P.S.
72 hours runtime: OOM problem with BEA JRockit R27: jrockit-R27.4.0- 
jdk1

spellchecker problems (bugs)

[r.nieto]

Hi users and developers,

 

I'm having some problems with the spellchecker component (I think there are
bugs). A few weeks ago Geoff spoke about some of them. 

 

If I restart tomcat without re-issuing spellcheck.build=true then the
spellcheck starts to fail. 

Another thing is that if I have 3 cores they work independently. If one
fails the others can continue working  Are they being allocated in memory?

Other problem is that if I made one spellcheck.build=true before another
finish I break the index.

 

Anyone can tell me what part of the code I must analyze to solve this?

 

Thanks for your attention

 

 

Re: Solr stops responding

[Fuad Efendi]

Try to use Tomcat... I never had SOLR-deadlocks (I believe) on 4-CPU  
SLES 10 server; only OutOfMemoryExceptions (possibly due to faceting,  
and significant memory fragmentation)


Recent OOM happened because I run MySQL on same server, and I overloaded it...

OpenBitSet & Faceting is main memory constraint for SOLR. OOM may  
happen even with huge but fragmented free memory available. GC can't  
optimize blocks locked by 'synchronized' statements (possibly...)

OutOfMemoryError - Quick Fix: Increase HashDocSet

[Fuad Efendi]

Change it to higher value, for instance, 3. OpenBitSet is created  
for larger values and requires a lot of memory...





Jul 17, 2008 11:09:02 AM org.apache.solr.common.SolrException log
SEVERE: java.lang.OutOfMemoryError: allocLargeObjectOrArray - Object  
size: 3149016, Num elements: 393625

at org.apache.solr.util.OpenBitSet.(OpenBitSet.java:86)
	at  
org.apache.solr.search.DocSetHitCollector.collect(DocSetHitCollector.java:63)
	at  
org.apache.solr.search.SolrIndexSearcher$9.collect(SolrIndexSearcher.java:1072)

at 
org.apache.lucene.search.BooleanScorer2.score(BooleanScorer2.java:320)
at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:146)
at org.apache.lucene.search.Searcher.search(Searcher.java:118)
at org.apache.lucene.search.Searcher.search(Searcher.java:97)
	at  
org.apache.solr.search.SolrIndexSearcher.getDocListAndSetNC(SolrIndexSearcher.java:1069)
	at  
org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:804)
	at  
org.apache.solr.search.SolrIndexSearcher.getDocListAndSet(SolrIndexSearcher.java:1245)
	at  
org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:96)
	at  
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:148)
	at  
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:117)

at org.apache.solr.core.SolrCore.execute(SolrCore.java:902)
	at  
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:280)
	at  
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:237)
	at  
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
	at  
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
	at  
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)
	at  
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:174)
	at  
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
	at  
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117)
	at  
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:108)
	at  
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:174)
	at  
org.apache.coyote.http11.Http11AprProcessor.process(Http11AprProcessor.java:834)
	at  
org.apache.coyote.http11.Http11AprProtocol$Http11ConnectionHandler.process(Http11AprProtocol.java:640)

at 
org.apache.tomcat.util.net.AprEndpoint$Worker.run(AprEndpoint.java:1286)
at java.lang.Thread.run(Thread.java:619)

Re: spellchecker problems (bugs)

[Shalin Shekhar Mangar]

The problems you described in the spellchecker are noted in
https://issues.apache.org/jira/browse/SOLR-622 -- I shall create an issue to
synchronize spellcheck.build so that the index is not corrupted.

As for multiple cores, yes, they should work independently of each other.

On Thu, Jul 17, 2008 at 9:29 PM, <[EMAIL PROTECTED]> wrote:

> Hi users and developers,
>
>
>
> I'm having some problems with the spellchecker component (I think there are
> bugs). A few weeks ago Geoff spoke about some of them.
>
>
>
> If I restart tomcat without re-issuing spellcheck.build=true then the
> spellcheck starts to fail.
>
> Another thing is that if I have 3 cores they work independently. If one
> fails the others can continue working  Are they being allocated in memory?
>
> Other problem is that if I made one spellcheck.build=true before another
> finish I break the index.
>
>
>
> Anyone can tell me what part of the code I must analyze to solve this?
>
>
>
> Thanks for your attention
>
>
>
>
>
>


-- 
Regards,
Shalin Shekhar Mangar.

OutOfMemoryError - Lucene

[Fuad Efendi]

How much memory Lucene needs? 100Mb for each such query...


Jul 16, 2008 8:38:43 PM org.apache.solr.common.SolrException log
SEVERE: java.lang.OutOfMemoryError: allocLargeObjectOrArray - Object  
size: 100767936, Num elements: 25191979
	at  
org.apache.lucene.search.FieldCacheImpl$10.createValue(FieldCacheImpl.java:360)

at 
org.apache.lucene.search.FieldCacheImpl$Cache.get(FieldCacheImpl.java:72)
	at  
org.apache.lucene.search.FieldCacheImpl.getStringIndex(FieldCacheImpl.java:352)
	at  
org.apache.lucene.search.FieldSortedHitQueue.comparatorString(FieldSortedHitQueue.java:348)
	at  
org.apache.lucene.search.FieldSortedHitQueue$1.createValue(FieldSortedHitQueue.java:201)

at 
org.apache.lucene.search.FieldCacheImpl$Cache.get(FieldCacheImpl.java:72)
	at  
org.apache.lucene.search.FieldSortedHitQueue.getCachedComparator(FieldSortedHitQueue.java:168)
	at  
org.apache.lucene.search.FieldSortedHitQueue.(FieldSortedHitQueue.java:56)
	at  
org.apache.solr.search.SolrIndexSearcher.getDocListAndSetNC(SolrIndexSearcher.java:1036)
	at  
org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:804)
	at  
org.apache.solr.search.SolrIndexSearcher.getDocListAndSet(SolrIndexSearcher.java:1245)
	at  
org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:96)
	at  
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:148)
	at  
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:117)

at org.apache.solr.core.SolrCore.execute(SolrCore.java:902)
	at  
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:280)
	at  
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:237)
	at  
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
	at  
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
	at  
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)
	at  
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:174)
	at  
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
	at  
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117)
	at  
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:108)
	at  
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:174)
	at  
org.apache.coyote.http11.Http11AprProcessor.process(Http11AprProcessor.java:834)
	at  
org.apache.coyote.http11.Http11AprProtocol$Http11ConnectionHandler.process(Http11AprProtocol.java:640)

at 
org.apache.tomcat.util.net.AprEndpoint$Worker.run(AprEndpoint.java:1286)
at java.lang.Thread.run(Thread.java:619)

Re: OutOfMemoryError - Quick Fix: Increase HashDocSet

[Mike Klaas]

On 17-Jul-08, at 10:28 AM, Fuad Efendi wrote:

   
   

Change it to higher value, for instance, 3. OpenBitSet is  
created for larger values and requires a lot of memory...


Careful--hash sets of that size can be quite slow.  It does make sense  
to bump up the value to 6000 or so for large (multi-million) indices.


-Mike

Re: OutOfMemoryError - Quick Fix: Increase HashDocSet

[Fuad Efendi]

Thanks Mike, I have 25 millions docs indexed, faceted on simple fields  
(cardinality: 5 for country field and 1 for host field)

8192Mb, JRockit R27 (Java 6)
Unpredictable OOMs...

I set HashDocSet/max to 30,000, don't see any performance degradation  
yet (the same response times for faceted id:[* TO *] and some queries).



Cache is small (trying to avoid OOM):











Quoting Mike Klaas <[EMAIL PROTECTED]>:



On 17-Jul-08, at 10:28 AM, Fuad Efendi wrote:


  
  

Change it to higher value, for instance, 3. OpenBitSet is   
created for larger values and requires a lot of memory...


Careful--hash sets of that size can be quite slow.  It does make sense
to bump up the value to 6000 or so for large (multi-million) indices.

-Mike

Re: Using custom Similarity class

[Chris Hostetter]

: Thanks for the tip but I solved it but using the old way of loading custom
: libs into solr: unpack the war file, add the custom library to WEB-INF/lib
: and repack it. It definetly is a class loader problem, as the wiki specifies
: that the new way of loading custom jars is using a custom class loader that
: *might* not work...

Hmmm... the "might not work" comment on that wiki is pretty old, it's been 
fairly well vetted at this point, and so far I haven't seen anyone have 
any serious problems with it.

I know you've got things working fine for you know, but whould you mind 
helping us diagnose the root problem incase it happens to other people who 
aren't as familiar with building their own war?

 * which servlet container are you using?
 * which JVM ?
before you rolled your own war...
 * what did the full directory structure of your solr home look like?
 * with logging at at least the INFO level, did you see any log messages 
   that mentioned "Solr classloader" or "solr lib class loader" ?


-Hoss

what do you do to track clickthroughs?

[pdovyda2]

Hey Guys,

I've been putting together a search engine using Solr/Lucene on the backend
and GWT for the UI.  I am looking to track clickthroughs on my results to
construct relevance judgements for the dataset I'm using - but I'm not sure
of a good way to go about this.

Obviously how you do this depends entirely on how your front end is set up,
but I would be interested to hear how other people using Solr have
implemented this so I can get some ideas.  Is it as simple as having a click
execute some code server-side before redirecting the user, and just logging
the information to a database?  How does this scale?

- Paul
-- 
View this message in context: 
http://www.nabble.com/what-do-you-do-to-track-clickthroughs--tp18517015p18517015.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: what do you do to track clickthroughs?

[Jacob Singh]

Hi Paul,

I actually use google analytics for this, since it is setup to do it.
In fact, you can configure your GA profile to treat your search page as
a search page and track the effectiveness of searches and even some
support for filters!

Check it out.

-J
pdovyda2 wrote:
> Hey Guys,
> 
> I've been putting together a search engine using Solr/Lucene on the backend
> and GWT for the UI.  I am looking to track clickthroughs on my results to
> construct relevance judgements for the dataset I'm using - but I'm not sure
> of a good way to go about this.
> 
> Obviously how you do this depends entirely on how your front end is set up,
> but I would be interested to hear how other people using Solr have
> implemented this so I can get some ideas.  Is it as simple as having a click
> execute some code server-side before redirecting the user, and just logging
> the information to a database?  How does this scale?
> 
> - Paul

Updated version of SOLR?

[Sunny Bassan]

I have noticed that SOLR is stuck on Version 1.2, although this is quite
fine because it works well, I see that Lucene is now up at Version 2.3.2
well beyond the version that is supporting SOLR 1.2. Does anybody know
if there is an updated SOLR version working with the new Lucene
foundation, or is this something we have to integrate ourselves? 
 
I do hope that integrating ourselves is not an option.
 
Sunny Singh Bassan

Re: Updated version of SOLR?

[Otis Gospodnetic]

Hi,

1.3-dev (nightly) uses the latest Lucene.

Otis
--
Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch



- Original Message 
> From: Sunny Bassan <[EMAIL PROTECTED]>
> To: solr-user@lucene.apache.org
> Sent: Friday, July 18, 2008 1:20:36 AM
> Subject: Updated version of SOLR?
> 
> I have noticed that SOLR is stuck on Version 1.2, although this is quite
> fine because it works well, I see that Lucene is now up at Version 2.3.2
> well beyond the version that is supporting SOLR 1.2. Does anybody know
> if there is an updated SOLR version working with the new Lucene
> foundation, or is this something we have to integrate ourselves? 
> 
> I do hope that integrating ourselves is not an option.
> 
> Sunny Singh Bassan

Re: Updated version of SOLR?

[Sunny Bassan]

Thank you

Re: Filter by Type increases search results.

[chris sleeman]

> btw, this *seems* to only work for me with standard search handler. dismax
and fq: dont' seem to get along nicely...

Wouldnt the dismax parser consider the filter query parameter as "type idea"
and not value "idea" for solr field - "type"?
I guess thats the reason this query doesnt work with dismax, the way it
works with the standard search handler. You can add a "debugQuery=true"
parameter to check the actual parsed query.

-Chris

On Tue, Jul 15, 2008 at 10:47 PM, Yonik Seeley <[EMAIL PROTECTED]> wrote:

> On Tue, Jul 15, 2008 at 11:10 AM, Norberto Meijome <[EMAIL PROTECTED]>
> wrote:
> > On Tue, 15 Jul 2008 18:07:43 +0530
> > "Preetam Rao" <[EMAIL PROTECTED]> wrote:
> >
> >> When I say filter, I meant q=fish&fq=type:idea
> >
> > btw, this *seems* to only work for me with standard search handler.
> dismax and fq: dont' seem to get along nicely... but maybe, it is just late
> and i'm not testing it properly..
>
> It should work the same... the only thing dismax does differently now
> is change the type of the base query to "dismax".
>
> -Yonik
>



-- 
Bill Cosby  - "Advertising is the most fun you can have with your clothes
on."