Re: ShardHandler - distribution to non-default request handler doesn't work

2012-10-28 Thread AlexeyK
The only way I succeeded to forward to the right request handler was:
1. shard.qt = /suggest (shards.qt=%2Fsuggest actually) in query
2.handleSelect='true' in solrconfig
3. NO /select handler in solrconfig

Only this combination forces 2 things - shard handler forwards qt=/suggest
parameter to other shards AND qt is handled by filter. (Otherwise qt is
ignored and the query gets forwarded to the /select handler)

Is there a better way of accomplishing this? How else can I retrieve
suggestions using a distinct handler?

Thanks,
Alexey



--
View this message in context: 
http://lucene.472066.n3.nabble.com/ShardHandler-distribution-to-non-default-request-handler-doesn-t-work-tp4015855p4016401.html
Sent from the Solr - User mailing list archive at Nabble.com.


Re: ShardHandler - distribution to non-default request handler doesn't work

2012-10-28 Thread AlexeyK
Correction:
shard.qt is sufficient, but you cannot define only spellcheck component in
requestHandler as it doesn't create shard requests, seems like 'query'
handler is a must if you want distributed processing.



--
View this message in context: 
http://lucene.472066.n3.nabble.com/ShardHandler-distribution-to-non-default-request-handler-doesn-t-work-tp4015855p4016409.html
Sent from the Solr - User mailing list archive at Nabble.com.


Delete query puzzle

2012-10-28 Thread Eric Grobler
Hi

I am a bit confused why the server sometimes takes 80 seconds to respond
when I specify an id to delete than does not even exist in the index.

If I loop this query and send a bogus id to delete every minute.
03:27:38   125 ms  bogusidthatdoesnotexist commit
03:28:38   125 ms  bogusidthatdoesnotexist commit
03:29:38 69125 ms  bogusidthatdoesnotexist commit
03:30:38   124 ms  bogusidthatdoesnotexist commit
03:31:38 84141 ms  bogusidthatdoesnotexist commit
03:33:38   125 ms  bogusidthatdoesnotexist commit
03:34:38   141 ms  bogusidthatdoesnotexist commit
03:35:43 55476 ms  bogusidthatdoesnotexist commit
03:36:38   141 ms  bogusidthatdoesnotexist commit
This was at 3am and the server only has about 200,000 documents and is not
busy, average query time is a constant < 5ms.

If the server takes 80 seconds when it needs to update the index I would
understand it.
*But in this case the id does not exists, so the server should just return
immediately?*
I then must assume that the delete command must be in some low priority
queue and waits for some exclusive lock?
When I look at the stats it seems that it was only my loop that did
cumulative_deletesById every minute.

What settings in the solrconfig.xml would effect this behaviour?

Thank you & Regards
Ericz


Re: Delete query puzzle

2012-10-28 Thread Erick Erickson
That is very weird. What version of Solr are you using, and is there
any way you could get a stack trace when this is happening?

Best
Erick

On Sun, Oct 28, 2012 at 6:22 AM, Eric Grobler  wrote:
> Hi
>
> I am a bit confused why the server sometimes takes 80 seconds to respond
> when I specify an id to delete than does not even exist in the index.
>
> If I loop this query and send a bogus id to delete every minute.
> 03:27:38   125 ms  bogusidthatdoesnotexist commit
> 03:28:38   125 ms  bogusidthatdoesnotexist commit
> 03:29:38 69125 ms  bogusidthatdoesnotexist commit
> 03:30:38   124 ms  bogusidthatdoesnotexist commit
> 03:31:38 84141 ms  bogusidthatdoesnotexist commit
> 03:33:38   125 ms  bogusidthatdoesnotexist commit
> 03:34:38   141 ms  bogusidthatdoesnotexist commit
> 03:35:43 55476 ms  bogusidthatdoesnotexist commit
> 03:36:38   141 ms  bogusidthatdoesnotexist commit
> This was at 3am and the server only has about 200,000 documents and is not
> busy, average query time is a constant < 5ms.
>
> If the server takes 80 seconds when it needs to update the index I would
> understand it.
> *But in this case the id does not exists, so the server should just return
> immediately?*
> I then must assume that the delete command must be in some low priority
> queue and waits for some exclusive lock?
> When I look at the stats it seems that it was only my loop that did
> cumulative_deletesById every minute.
>
> What settings in the solrconfig.xml would effect this behaviour?
>
> Thank you & Regards
> Ericz


Exception while getting Field info in Lucene

2012-10-28 Thread adityab
Hi
Deployed 4.0 and while investigating the schema Browser for seeing the
unique term count for each field observed following error. The top term
shows "10/-1". its -1 all the time. Any idea what might be wrong. 

thanks
Aditya

2012-10-28 10:48:42,017 SEVERE [org.apache.solr.servlet.SolrDispatchFilter]
(ajp-0.0.0.0-8009-12) null:java.lang.NullPointerException
at
org.apache.solr.handler.admin.LukeRequestHandler.getDetailedFieldInfo(LukeRequestHandler.java:602)
at
org.apache.solr.handler.admin.LukeRequestHandler.getIndexedFieldsInfo(LukeRequestHandler.java:371)
at
org.apache.solr.handler.admin.LukeRequestHandler.handleRequestBody(LukeRequestHandler.java:159)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1699)
at
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:455)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:276)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at
org.jboss.web.tomcat.filters.ReplyHeaderFilter.doFilter(ReplyHeaderFilter.java:96)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:235)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
at
org.jboss.web.tomcat.security.SecurityAssociationValve.invoke(SecurityAssociationValve.java:190)
at
org.jboss.web.tomcat.security.JaccContextValve.invoke(JaccContextValve.java:92)
at
org.jboss.web.tomcat.security.SecurityContextEstablishmentValve.process(SecurityContextEstablishmentValve.java:126)
at
org.jboss.web.tomcat.security.SecurityContextEstablishmentValve.invoke(SecurityContextEstablishmentValve.java:70)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
at
org.jboss.web.tomcat.service.jca.CachedConnectionValve.invoke(CachedConnectionValve.java:158)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:330)
at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:436)
at
org.apache.coyote.ajp.AjpProtocol$AjpConnectionHandler.process(AjpProtocol.java:384)
at
org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:447)
at java.lang.Thread.run(Thread.java:662)




--
View this message in context: 
http://lucene.472066.n3.nabble.com/Exception-while-getting-Field-info-in-Lucene-tp4016448.html
Sent from the Solr - User mailing list archive at Nabble.com.


Re: Delete query puzzle

2012-10-28 Thread Eric Grobler
Hi Erick,

It is Solr 1.41 (a Drupal installation) running on Jetty.
How can one get a stack trace? (there is no exception/error)

Could it be that solr does something like this?
start delete job
   cannot find bogus id to delete
   does some reindex or optimization anyway regardless which takes 80
seconds
end delete job

Anyway, does it sound like Solr is just waiting 80 seconds for some
exclusive lock or is it actually doing something in a background thread?. I
do not know what kind of calls drupal is doing.

Thanks & Regards
Ericz




On Sun, Oct 28, 2012 at 3:08 PM, Erick Erickson wrote:

> That is very weird. What version of Solr are you using, and is there
> any way you could get a stack trace when this is happening?
>
> Best
> Erick
>
> On Sun, Oct 28, 2012 at 6:22 AM, Eric Grobler 
> wrote:
> > Hi
> >
> > I am a bit confused why the server sometimes takes 80 seconds to respond
> > when I specify an id to delete than does not even exist in the index.
> >
> > If I loop this query and send a bogus id to delete every minute.
> > 03:27:38   125 ms  bogusidthatdoesnotexist
> commit
> > 03:28:38   125 ms  bogusidthatdoesnotexist
> commit
> > 03:29:38 69125 ms  bogusidthatdoesnotexist
> commit
> > 03:30:38   124 ms  bogusidthatdoesnotexist
> commit
> > 03:31:38 84141 ms  bogusidthatdoesnotexist
> commit
> > 03:33:38   125 ms  bogusidthatdoesnotexist
> commit
> > 03:34:38   141 ms  bogusidthatdoesnotexist
> commit
> > 03:35:43 55476 ms  bogusidthatdoesnotexist
> commit
> > 03:36:38   141 ms  bogusidthatdoesnotexist
> commit
> > This was at 3am and the server only has about 200,000 documents and is
> not
> > busy, average query time is a constant < 5ms.
> >
> > If the server takes 80 seconds when it needs to update the index I would
> > understand it.
> > *But in this case the id does not exists, so the server should just
> return
> > immediately?*
> > I then must assume that the delete command must be in some low priority
> > queue and waits for some exclusive lock?
> > When I look at the stats it seems that it was only my loop that did
> > cumulative_deletesById every minute.
> >
> > What settings in the solrconfig.xml would effect this behaviour?
> >
> > Thank you & Regards
> > Ericz
>


Re: Delete query puzzle

2012-10-28 Thread Erick Erickson
Oh, 1.4.1. You're probably on your own here. I'd be surprised if
people were willing to work on code of that vintage. Are
you sure you can't upgrade at least to 3.6?

Best
Erick

On Sun, Oct 28, 2012 at 12:43 PM, Eric Grobler
 wrote:
> Hi Erick,
>
> It is Solr 1.41 (a Drupal installation) running on Jetty.
> How can one get a stack trace? (there is no exception/error)
>
> Could it be that solr does something like this?
> start delete job
>cannot find bogus id to delete
>does some reindex or optimization anyway regardless which takes 80
> seconds
> end delete job
>
> Anyway, does it sound like Solr is just waiting 80 seconds for some
> exclusive lock or is it actually doing something in a background thread?. I
> do not know what kind of calls drupal is doing.
>
> Thanks & Regards
> Ericz
>
>
>
>
> On Sun, Oct 28, 2012 at 3:08 PM, Erick Erickson 
> wrote:
>
>> That is very weird. What version of Solr are you using, and is there
>> any way you could get a stack trace when this is happening?
>>
>> Best
>> Erick
>>
>> On Sun, Oct 28, 2012 at 6:22 AM, Eric Grobler 
>> wrote:
>> > Hi
>> >
>> > I am a bit confused why the server sometimes takes 80 seconds to respond
>> > when I specify an id to delete than does not even exist in the index.
>> >
>> > If I loop this query and send a bogus id to delete every minute.
>> > 03:27:38   125 ms  bogusidthatdoesnotexist
>> commit
>> > 03:28:38   125 ms  bogusidthatdoesnotexist
>> commit
>> > 03:29:38 69125 ms  bogusidthatdoesnotexist
>> commit
>> > 03:30:38   124 ms  bogusidthatdoesnotexist
>> commit
>> > 03:31:38 84141 ms  bogusidthatdoesnotexist
>> commit
>> > 03:33:38   125 ms  bogusidthatdoesnotexist
>> commit
>> > 03:34:38   141 ms  bogusidthatdoesnotexist
>> commit
>> > 03:35:43 55476 ms  bogusidthatdoesnotexist
>> commit
>> > 03:36:38   141 ms  bogusidthatdoesnotexist
>> commit
>> > This was at 3am and the server only has about 200,000 documents and is
>> not
>> > busy, average query time is a constant < 5ms.
>> >
>> > If the server takes 80 seconds when it needs to update the index I would
>> > understand it.
>> > *But in this case the id does not exists, so the server should just
>> return
>> > immediately?*
>> > I then must assume that the delete command must be in some low priority
>> > queue and waits for some exclusive lock?
>> > When I look at the stats it seems that it was only my loop that did
>> > cumulative_deletesById every minute.
>> >
>> > What settings in the solrconfig.xml would effect this behaviour?
>> >
>> > Thank you & Regards
>> > Ericz
>>


Re: Exception while getting Field info in Lucene

2012-10-28 Thread Erick Erickson
I'm afraid you have to give more details, this works fine for me with
the example docs and the example schema.

Best
Erick

On Sun, Oct 28, 2012 at 11:02 AM, adityab  wrote:
> Hi
> Deployed 4.0 and while investigating the schema Browser for seeing the
> unique term count for each field observed following error. The top term
> shows "10/-1". its -1 all the time. Any idea what might be wrong.
>
> thanks
> Aditya
>
> 2012-10-28 10:48:42,017 SEVERE [org.apache.solr.servlet.SolrDispatchFilter]
> (ajp-0.0.0.0-8009-12) null:java.lang.NullPointerException
> at
> org.apache.solr.handler.admin.LukeRequestHandler.getDetailedFieldInfo(LukeRequestHandler.java:602)
> at
> org.apache.solr.handler.admin.LukeRequestHandler.getIndexedFieldsInfo(LukeRequestHandler.java:371)
> at
> org.apache.solr.handler.admin.LukeRequestHandler.handleRequestBody(LukeRequestHandler.java:159)
> at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:1699)
> at
> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:455)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:276)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
> at
> org.jboss.web.tomcat.filters.ReplyHeaderFilter.doFilter(ReplyHeaderFilter.java:96)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
> at
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:235)
> at
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
> at
> org.jboss.web.tomcat.security.SecurityAssociationValve.invoke(SecurityAssociationValve.java:190)
> at
> org.jboss.web.tomcat.security.JaccContextValve.invoke(JaccContextValve.java:92)
> at
> org.jboss.web.tomcat.security.SecurityContextEstablishmentValve.process(SecurityContextEstablishmentValve.java:126)
> at
> org.jboss.web.tomcat.security.SecurityContextEstablishmentValve.invoke(SecurityContextEstablishmentValve.java:70)
> at
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
> at
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
> at
> org.jboss.web.tomcat.service.jca.CachedConnectionValve.invoke(CachedConnectionValve.java:158)
> at
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
> at
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:330)
> at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:436)
> at
> org.apache.coyote.ajp.AjpProtocol$AjpConnectionHandler.process(AjpProtocol.java:384)
> at
> org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:447)
> at java.lang.Thread.run(Thread.java:662)
>
>
>
>
> --
> View this message in context: 
> http://lucene.472066.n3.nabble.com/Exception-while-getting-Field-info-in-Lucene-tp4016448.html
> Sent from the Solr - User mailing list archive at Nabble.com.


Re: Delete query puzzle

2012-10-28 Thread Eric Grobler
Hi Erick

> You're probably on your own here. I'd be surprised if  people were
willing to work on code of that vintage.
Yes, this is not a vintage wine!

I just hoped someone would say, "ah, we had this issue before and..." :-)

I think best is to just upgrade like you suggested.

Thanks for your time
Ericz


On Sun, Oct 28, 2012 at 6:34 PM, Erick Erickson wrote:

> Oh, 1.4.1. You're probably on your own here. I'd be surprised if
> people were willing to work on code of that vintage. Are
> you sure you can't upgrade at least to 3.6?
>
> Best
> Erick
>
> On Sun, Oct 28, 2012 at 12:43 PM, Eric Grobler
>  wrote:
> > Hi Erick,
> >
> > It is Solr 1.41 (a Drupal installation) running on Jetty.
> > How can one get a stack trace? (there is no exception/error)
> >
> > Could it be that solr does something like this?
> > start delete job
> >cannot find bogus id to delete
> >does some reindex or optimization anyway regardless which takes 80
> > seconds
> > end delete job
> >
> > Anyway, does it sound like Solr is just waiting 80 seconds for some
> > exclusive lock or is it actually doing something in a background
> thread?. I
> > do not know what kind of calls drupal is doing.
> >
> > Thanks & Regards
> > Ericz
> >
> >
> >
> >
> > On Sun, Oct 28, 2012 at 3:08 PM, Erick Erickson  >wrote:
> >
> >> That is very weird. What version of Solr are you using, and is there
> >> any way you could get a stack trace when this is happening?
> >>
> >> Best
> >> Erick
> >>
> >> On Sun, Oct 28, 2012 at 6:22 AM, Eric Grobler <
> impalah...@googlemail.com>
> >> wrote:
> >> > Hi
> >> >
> >> > I am a bit confused why the server sometimes takes 80 seconds to
> respond
> >> > when I specify an id to delete than does not even exist in the index.
> >> >
> >> > If I loop this query and send a bogus id to delete every minute.
> >> > 03:27:38   125 ms  bogusidthatdoesnotexist
> >> commit
> >> > 03:28:38   125 ms  bogusidthatdoesnotexist
> >> commit
> >> > 03:29:38 69125 ms  bogusidthatdoesnotexist
> >> commit
> >> > 03:30:38   124 ms  bogusidthatdoesnotexist
> >> commit
> >> > 03:31:38 84141 ms  bogusidthatdoesnotexist
> >> commit
> >> > 03:33:38   125 ms  bogusidthatdoesnotexist
> >> commit
> >> > 03:34:38   141 ms  bogusidthatdoesnotexist
> >> commit
> >> > 03:35:43 55476 ms  bogusidthatdoesnotexist
> >> commit
> >> > 03:36:38   141 ms  bogusidthatdoesnotexist
> >> commit
> >> > This was at 3am and the server only has about 200,000 documents and is
> >> not
> >> > busy, average query time is a constant < 5ms.
> >> >
> >> > If the server takes 80 seconds when it needs to update the index I
> would
> >> > understand it.
> >> > *But in this case the id does not exists, so the server should just
> >> return
> >> > immediately?*
> >> > I then must assume that the delete command must be in some low
> priority
> >> > queue and waits for some exclusive lock?
> >> > When I look at the stats it seems that it was only my loop that did
> >> > cumulative_deletesById every minute.
> >> >
> >> > What settings in the solrconfig.xml would effect this behaviour?
> >> >
> >> > Thank you & Regards
> >> > Ericz
> >>
>


Re: Occasional Solr performance issues

2012-10-28 Thread Dotan Cohen
On Fri, Oct 26, 2012 at 11:04 PM, Shawn Heisey  wrote:
> Warming doesn't seem to be a problem here -- all your warm times are zero,
> so I am going to take a guess that it may be a heap/GC issue.  I would
> recommend starting with the following additional arguments to your JVM.
> Since I have no idea how solr gets started on your server, I don't know
> where you would add these:
>
> -Xmx4096M -Xms4096M -XX:NewRatio=1 -XX:+UseParNewGC -XX:+UseConcMarkSweepGC
> -XX:+CMSParallelRemarkEnabled
>

Thanks. I've added those flags to the Solr line that I use to start
Solr. Those are Java flags, not Solr, correct? I'm googling the flags
now, but I find it interesting that I cannot find a canonical
reference for them.


> This allocates 4GB of RAM to java, sets up a larger than normal Eden space
> in the heap, and uses garbage collection options that usually fare better in
> a server environment than the default.Java memory management options are
> like religion to some people ... I may start a flamewar with these
> recommendations. ;)  The best I can tell you about these choices: They made
> a big difference for me.
>

Thanks. I will experiment with them empirically. The first step is to
learn to read the debug info, though. I've been googing for days, but
I must be missing something. Where is the information that I pasted in
pastebin documented?


> I would also recommend switching to a Sun/Oracle jvm.  I have heard that
> previous versions of Solr were not happy on variants like OpenJDK, I have no
> idea whether that might still be the case with 4.0.  If you choose to do
> this, you probably have package choices in Ubuntu.  I know that in Debian,
> the package is called sun-java6-jre ... Ubuntu is probably something
> similar. Debian has a CLI command 'update-java-alternatives' that will
> quickly switch between different java implementations that are installed.
> Hopefully Ubuntu also has this.  If not, you might need the following
> command instead to switch the main java executable:
>
> update-alternatives --config java
>

Thanks, I will take a look at the current Oracle JVM.


-- 
Dotan Cohen

http://gibberish.co.il
http://what-is-what.com


Re: throttle segment merging

2012-10-28 Thread Lance Norskog
1) Do you use compound files (CFS)? This adds a lot of overhead to merging.
2) Does ES use the same merge policy code as Solr?

In solrconfig.xml, here are the lines that control segment merging. You can 
probably set mergeFactor to 20 and cut the amount of disk I/O.



   







- Original Message -
| From: "Radim Kolar" 
| To: solr-user@lucene.apache.org
| Sent: Saturday, October 27, 2012 7:44:46 PM
| Subject: Re: throttle segment merging
| 
| Dne 26.10.2012 3:47, Tomás Fernández Löbbe napsal(a):
| >> Is there way to set-up logging to output something when segment
| >> merging
| >> runs?
| >>
| > I think segment merging is logged when you enable infoStream
| > logging (you
| > should see it commented in the solrconfig.xml)
| no, segment merging is not logged at info level. it needs customized
| log
| config.
| 
| >
| >> Can be segment merges throttled?
|  > You can change when and how segments are merged with the merge
| policy, maybe it's enough for you changing the initial settings
| (mergeFactor for example)?
| 
| I am now researching elasticsearch, it can do it, its lucene 3.6
| based
| 


Re: org.apache.lucene.queryparser.classic.ParseException - a Bug?

2012-10-28 Thread deniz
hello iorixxx,

i have just tried url encoding because the raw format was also giving the
same error/exception... i was curious if it could fix...

anyone has any ideas on the exception? i still couldnt find a way to
overcome this 



-
Zeki ama calismiyor... Calissa yapar...
--
View this message in context: 
http://lucene.472066.n3.nabble.com/org-apache-lucene-queryparser-classic-ParseException-a-Bug-tp4015763p4016550.html
Sent from the Solr - User mailing list archive at Nabble.com.


Re: Search Suggest for full content with Filter option

2012-10-28 Thread Sujatha Arun
Any Suggestions on this?



On Sun, Oct 28, 2012 at 1:30 PM, Sujatha Arun  wrote:

> Hello,
>
> I want  suggestions for full content of several books with a filter
> that restricts suggestions to a single book .However the best options of
>  suggester and terms component  do not support filter.
>
> That leaves the facets and  ngram Indexing.I indexed entire content by
> splitting on white space as the suggestions should work for any words in
> the index, But I find this query
>
> /select?q=tes&facets=true&facet.field=ph_su&facet.prefix=tes&facet.limit=5
> extremely time consuming .This could be because of the number of unique
> terms in the full Index.
>
> For an ngram Indexing ,If were to Index the entire content as tokenized
> into a field and store the same  ,for any token for the document ,I would
> get the entire stored content as suggestion .How can I get the only the
> correct keyword as suggestion using an non-suggester based n gram Indexing
> such that it can be filtered?
>
>
> Regards
> Sujatha
>


Re: Occasional Solr performance issues

2012-10-28 Thread Shawn Heisey

On 10/28/2012 2:28 PM, Dotan Cohen wrote:

On Fri, Oct 26, 2012 at 11:04 PM, Shawn Heisey  wrote:

Warming doesn't seem to be a problem here -- all your warm times are zero,
so I am going to take a guess that it may be a heap/GC issue.  I would
recommend starting with the following additional arguments to your JVM.
Since I have no idea how solr gets started on your server, I don't know
where you would add these:

-Xmx4096M -Xms4096M -XX:NewRatio=1 -XX:+UseParNewGC -XX:+UseConcMarkSweepGC
-XX:+CMSParallelRemarkEnabled

Thanks. I've added those flags to the Solr line that I use to start
Solr. Those are Java flags, not Solr, correct? I'm googling the flags
now, but I find it interesting that I cannot find a canonical
reference for them.


They are indeed Java options.  The first two control the maximum and 
starting heap sizes.  NewRatio controls the relative size of the young 
and old generations, making the young generation considerably larger 
than it is by default.  The others are garbage collector options.  This 
seems to be a good summary:


http://www.petefreitag.com/articles/gctuning/

Here's the official Sun (Oracle) documentation on GC tuning:

http://www.oracle.com/technetwork/java/javase/gc-tuning-6-140523.html

Thanks,
Shawn



Management of solr.xml on master server

2012-10-28 Thread maneesha
Hi,

As suggested by the Solr Enterprise book, I have separate strategies for
updating the solr core (e.g. blah) when I need to do incremental
updates(every day) VS create a fresh index from scratch(once in a 4 months).
Assuming that the dataDir for the core "blah" is /home/blah/data/defaultData
in the solr.xml.

For creating the full index for my core every quarter from scratch:
 - I create a new core e.g. "blahNov2012" using admin url with option
action=CREATE and I give it a new dataDir property e.g.
/home/blah/data/Nov2012.
- I do a full import on blahNov2012 to populate the new core "blah-Nov2012"
and test it. 
- If all is good, I run the admin url with option action=SWAP to swap (blah
with blahNov2012).
- Since I have persistent="true" in the solr.xml, it updates the dataDir for
the  core "blah" to point to the new directory /home/blah/data/Nov2012.



  
  
  


  


My first question: is this the pattern most people use to create fresh index
(e.g. create a new tmp core, test, and swap).

My second question is if I need to make further unrelated changes the
solr.xml in next releases and I must update the solr.xml on the production
system, I need to manually change the "blah"'s data directory from
"/home/blah/data/defaultData" to "/home/blah/data/Nov2012". Is that what
needs to be done? Do people automate this step somehow in their production
releases..?

-maneesha




--
View this message in context: 
http://lucene.472066.n3.nabble.com/Management-of-solr-xml-on-master-server-tp4016570.html
Sent from the Solr - User mailing list archive at Nabble.com.


Re: Any way to by pass the checking on QueryElevationComponent

2012-10-28 Thread Amit Nithian
Is the goal to have the elevation data read from somewhere else? In
other words, why don't you want the elevate.xml to exist locally?

If you want to read the data from somewhere else, could you put a
dummy elevate.xml locally and subclass the QueryElevationComponent and
override the loadElevationMap() to read this data from your own custom
location?

On Fri, Oct 26, 2012 at 6:47 PM, James Ji  wrote:
> Hi there
>
> We are currently working on having Solr files read from HDFS. We extended
> some of the classes so as to avoid modifying the original Solr code and
> make it compatible with the future release. So here comes the question, I
> found in QueryElevationComponent, there is a piece of code checking whether
> elevate.xml exists at local file system. I am wondering if there is a way
> to by pass this?
> QueryElevationComponent.inform(){
> 
> File fC = new File(core.getResourceLoader().getConfigDir(), f);
> File fD = new File(core.getDataDir(), f);
> if (fC.exists() == fD.exists()) { throw new
> SolrException(SolrException.ErrorCode.SERVER_ERROR,
> "QueryElevationComponent missing config file: '" + f + "\n" + "either: " +
> fC.getAbsolutePath() + " or " + fD.getAbsolutePath() + " must exist, but
> not both."); }
> if (fC.exists()) { exists = true; log.info("Loading QueryElevation from:
> "+fC.getAbsolutePath()); Config cfg = new Config(core.getResourceLoader(),
> f); elevationCache.put(null, loadElevationMap(cfg)); }
> 
> }
>
> --
> Jiayu (James) Ji,
>
> ***
>
> Cell: (312)823-7393
> Website: https://sites.google.com/site/jiayuji/
>
> ***


Re: org.apache.lucene.queryparser.classic.ParseException - a Bug?

2012-10-28 Thread deniz
well i found the problem... it was because of my code that process the query
request before sending it to the server... and because fq thing has two =
signs, the parsing was ruined... fixed now...



-
Zeki ama calismiyor... Calissa yapar...
--
View this message in context: 
http://lucene.472066.n3.nabble.com/org-apache-lucene-queryparser-classic-ParseException-a-Bug-tp4015763p4016579.html
Sent from the Solr - User mailing list archive at Nabble.com.