Re: Debug Solr in Netbeans..

2009-06-16 Thread noor

Yes Mark Miller,
I changed the Connector specification in attach-debugger dialog,
from SocketAttach TO socketListen.

Now i got the netbeans debugger-console output as,
Listening on 5005

But, nothing will be display after that, but the progress shows, 
attaching debugger


What is the problem 

thanks and regards,
Noor

noor wrote:

Addition to my previous reply::

I am running solr by start.jar file. It has my custom class jar file 
in its lib folder.

Also in netbeans, custom class source has the checkpoint to debug.;
and in the project folder, i created the build.properties.
So i set the AttachDebugger settings; but it gives the connection 
refused error:


I don't know, i am doing correctly 
Please anyone help me to solve this ...

thanks and regards
Noorulla

noor wrote:

Now, i put that build.properties file in the solr location tooo.
But still i am getting.

Attaching to localhost:5005
Connection refused

Note:
Solr lib folder contains, my custom class's jar file.
But in netbeans, i am doing the attach-debugger processing.
And in browser, i am accessing that class setting as,
http://localhost:8983/solr/custom?q=searchText&debugQuery=true
Browser page also gives Null error.

Is this way correct.

For your information,
pls see about my custom handler settings on the following page:
http://markmail.org/message/uvm5xp3ld5mmd5or?q=custom+solr+handler+error: 





Mark Miller wrote:
If you don't see that, you may have build.properties in the wrong 
place.


When you run 'solr-example' in debug mode, Listening for transport 
dt_socket at address: 5005 will be printed to STD out.


Once you have that working correctly, you want to attach to port 
5005, not 8983. Solr runs on 8983, but the debugger is listening on 
5005.



- Mark

noor wrote:

No.
In netbeans, debugger-console output shows,

Attaching to localhost:8983
handshake failed - connection prematurally closed

i dont know where the problem is ?

Mark Miller wrote:
Do you see the following printed to std out when you start solr 
(using 'run-example')?


Listening for transport dt_socket at address: 5005

noor wrote:

Addition to the previous reply:
I built my custom project and put into solr webapps lib folder.
And starts running solr.
In netbeans, i made the changes as i said before.
But it shows connection refused error.

anybody please give me the solution...

noor wrote:

Solr starts running in the port 8983,
i created build.properties in the project folder, where the 
build.xml is.

And in that empty build.properties file,
i added,
example.debug=true
only.
And in Netbeans, Debug -> Attach Debugger,
- Debugger is JavaDebugger(JPDA);
- Connector is SocketAttach(Attaches by socket to other VMs)
- HOST is localhost;
- Port as 5005;
and Timeout is empty.

During solr running, i set this, but in the output screen shows 
"Connection is refused."


Is my changes are correct ??? or i need to change anything else...


thanks and regards,
Noor


Mark Miller wrote:

noor wrote:

hi, i am new to apache solr.
i got the solr source code, and i created my own (custom) 
classes.
Also, i make the request reference to that newly created 
classes in solr-config.xml.


now i need to debug my code, when the solr search calls my 
class..

So, for this, i dont know how to debug my code?

Please anybody help me to achieve this.

thanks and regards,
Noor



Make a file next to build.xml called build.properties.

Add to the empty file: example.debug=true

Run the ant target 'run-example' in build.xml.

Solr will run with the ability to connect with a remote 
debugger on port 5005.


In Netbeans, from the main menu, select Debug > Attach 
Debugger... (In NetBeans 6.1 and older select Run > Attach 
Debugger...).


Follow the dialogue box prompts to connect to the running Solr 
example.


























Re: Custom Request handler Error:

2009-06-16 Thread noor

how to create a custom update handler, to update my fields... ?
Also, how to make the update, through url, i mean,

http://localhost:8983/solr/update/../ like that... ???




noor wrote:

Yes, i changed "custom" into "/custom", now it calls my class.
But, in browser, It shows
Null RequestHandler null.

So, i need to accept my changes by solr. For that, what i need to do,
pls guide me to acheive this.


Noble Paul wrote:

register is as follows
class="org.apache.solr.my.MyCustomHandler">


the request must be made to the uri "/custom" only then the requests
would come to your handler

On Sat, Jun 13, 2009 at 5:49 PM, noor wrote:
 

Yes, i changed requestHandler name as,
class="org.apache.solr.my.MyCustomHandler">

.

Then also,
In statistics page, my custom handler under QueryHandler's request 
count

remains 0. It shows that, the webrequest is not coming to my class

Noble Paul wrote:
   

register your handler in some other name and fire a request to that

On Fri, Jun 12, 2009 at 8:07 PM, noor wrote:

 

I solved this NullPointerException, by the following changes.

In java code:
public void handleRequestBody(SolrQueryRequest request, 
SolrQueryResponse

response) throws Exception {
SolrCore coreToRequest =
request.getCore();//coreContainer.getCore("core2");
.
}

and in solr-config.xml:


explicit
tandem
true



Now, my webapp runs fine by,
http://localhost:8983/mysearch
searching also working fine.
But, these are not run through my custom handler. So i felt, it 
wrongly

doing searching.
Because, in solr admin statistics page,
my custom handler under QueryHandler's request count remains 0, it
doesn't
get incremented, when i search something. Rather, 
statndardReqHandler's

request count is incremented.

And another thing, how do we debug solr. ???
Please anybody help me to solve this ...

Thanks in advance.

Noble Paul ??? ?? wrote:

   

is there any error on the console?

On Fri, Jun 12, 2009 at 4:26 PM, Noor 
wrote:



 

hi,
 i am new to apache solr.
I need to create a custom request handler class. So i create a 
new one

and
changed the solr-config.xml file as,
 

explicit
tandem
true

 

And in my java class, the code is,

public class MyCustomHandler extends RequestHandlerBase{
 public CoreContainer coreContainer;
 public void handleRequestBody(SolrQueryRequest request,
SolrQueryResponse
response) throws Exception {
SolrCore coreToRequest = coreContainer.getCore("core2");
ModifiableSolrParams params = new ModifiableSolrParams();
params.set("echoParams", "explicit");
params.set("q", "text");
params.set("debugQuery", "true");
request = new LocalSolrQueryRequest(coreToRequest, params);
// SolrRequestHandler reqHandler =
coreToRequest.getRequestHandler("/select");
coreToRequest.execute(reqHandler, request, response);
coreToRequest.close();
request.close();
 }
 // the abstract methods - getDescription(), getSourceId(),
getSource(),
getVersion() are //overrided... but these methods doesn't have any
implementations.
}


But, if i search any text in my webapp from browser, gots the 
HTTP 500

error.
i dont know how SolrContainer is intialized
Pls anyone give me the solution...

thanks and regards,
Mohamed






  




  





  







Re: fq vs. q

2009-06-16 Thread Fergus McMenemie
>Fergus McMenemie schrieb:
>
>> The article could explain the difference between fq= and
>> facet.query= and when you should use one in preference to
>> the other.
>
>My understanding is that while these query modifiers rely on the
>same implementation (cached filters) to boost performance, they
>simply and obviously differ in that "fq" limits the result set to
>your filter criterion whereas "facet.query" does not restrict the
>result but instead enhances it with statistical information gained
>from applying set intersection of result and facet query filters.
>
>It looks like "facet.query" is just a more flexible means of
>defining a filter than possible using a mere "facet.field".
>
>Would that be approximately correct?
Yes. While q= and fq= affect the "results" portion of a search 
response. The facet.query only affects the "facets" portion of a
response. facet.query(s) are only used where you want a facet
summary of your query based on some kind of complex expression
rather than the terms within a single field.

I added the comment in that I think that a wiki page discussing
fs vs q should also mention facet.query.


>It appears to me that each "facet.query" invariably leads to one
>boolean filter, so if you wanted to do range faceting for a given
>field and obtain, say, results reduced from their actual continuum
>of values to three ranges {A,B,C}, you'd have to define three
>"facet.query" parameters accordingly. A mere "facet.field", on the
>other hand, creates as many filters as there are unique values in
>the field. Is that correct?
Yes, A single facet.query on its own is probably useless. You would
need many of them. And as they have to be re-calculated after each
query I would imagine they are expensive. Also, given that facets
are used to help drive GUI options which turn drive the contents of
subsequent fq= filters, I am wondering fq= queries are not 
analyzed before the search is made but I get the impression that
facet.querys are! This could be a big pitfall.

>Michael Ludwig
Fergus.

-- 

===
Fergus McMenemie   Email:fer...@twig.me.uk
Techmore Ltd   Phone:(UK) 07721 376021

Unix/Mac/Intranets Analyst Programmer
===


How to allow case insensitive in Solr

2009-06-16 Thread chem leakhina
Hi everyone,How can I allow Solr to search for all cases like upper case or
lower case in Solr?
Best regards,
Leakhina


Re: How to allow case insensitive in Solr

2009-06-16 Thread Markus Jelsma - Buyways B.V.
hi,


You can create a field with class="Solr.TextField" which needs a
LowerCaseFilterFactory in the index and query analyzers e.g.













This will allow for case insensitive searches.

regards,

-  
Markus Jelsma  Buyways B.V. Tel. 050-3118123
Technisch ArchitectFriesestraatweg 215c Fax. 050-3118124
http://www.buyways.nl  9743 AD GroningenKvK  01074105


On Tue, 2009-06-16 at 15:18 +0700, chem leakhina wrote:

> Hi everyone,How can I allow Solr to search for all cases like upper case or
> lower case in Solr?
> Best regards,
> Leakhina


Re: How to allow case insensitive in Solr

2009-06-16 Thread chem leakhina
So when I use LowerCaseFilterFactory, I can search with upper case and lower
case right?

On Tue, Jun 16, 2009 at 3:44 PM, Markus Jelsma - Buyways B.V. <
mar...@buyways.nl> wrote:

> hi,
>
>
> You can create a field with class="Solr.TextField" which needs a
> LowerCaseFilterFactory in the index and query analyzers e.g.
>
>
>
> class="solr.WhitespaceTokenizerFactory"/>
> class="solr.LowerCaseFilterFactory"/>
>
>
>
> class="solr.WhitespaceTokenizerFactory"/>
> class="solr.LowerCaseFilterFactory"/>
>
>
>
> This will allow for case insensitive searches.
>
> regards,
>
> -
> Markus Jelsma  Buyways B.V. Tel. 050-3118123
> Technisch ArchitectFriesestraatweg 215c Fax. 050-3118124
> http://www.buyways.nl  9743 AD GroningenKvK  01074105
>
>
> On Tue, 2009-06-16 at 15:18 +0700, chem leakhina wrote:
>
> > Hi everyone,How can I allow Solr to search for all cases like upper case
> or
> > lower case in Solr?
> > Best regards,
> > Leakhina
>


Re: How to allow case insensitive in Solr

2009-06-16 Thread Markus Jelsma - Buyways B.V.
Indeed, as it will LowerCase all tokens during indextime and querytime
so the term LowerCaseFilterFactory will (aside from some tokenizer)
enter the index as lowercasefilterfactory. Then, any searchterm will
also be lowercased so the searchterm LOWERCASEFILTERFACTORY will match
because on querytime it will also pass through the filter.


-  
Markus Jelsma  Buyways B.V. Tel. 050-3118123
Technisch ArchitectFriesestraatweg 215c Fax. 050-3118124
http://www.buyways.nl  9743 AD GroningenKvK  01074105


On Tue, 2009-06-16 at 15:46 +0700, chem leakhina wrote:

> So when I use LowerCaseFilterFactory, I can search with upper case and
> lower
> case right?


Re: Custom Request handler Error:

2009-06-16 Thread Noble Paul നോബിള്‍ नोब्ळ्
please take a look at one of the updatehandlers which are already there
XmlUpdateRequestHandler
CSVRequestHandler
BinaryUpdateRequestHandler

On Tue, Jun 16, 2009 at 1:34 PM, noor wrote:
> how to create a custom update handler, to update my fields... ?
> Also, how to make the update, through url, i mean,
>
> http://localhost:8983/solr/update/../ like that... ???
>
>
>
>
> noor wrote:
>>
>> Yes, i changed "custom" into "/custom", now it calls my class.
>> But, in browser, It shows
>> Null RequestHandler null.
>>
>> So, i need to accept my changes by solr. For that, what i need to do,
>> pls guide me to acheive this.
>>
>>
>> Noble Paul wrote:
>>>
>>> register is as follows
>>> >> class="org.apache.solr.my.MyCustomHandler">
>>>
>>> the request must be made to the uri "/custom" only then the requests
>>> would come to your handler
>>>
>>> On Sat, Jun 13, 2009 at 5:49 PM, noor wrote:
>>>

 Yes, i changed requestHandler name as,
 >>> class="org.apache.solr.my.MyCustomHandler">
 .

 Then also,
 In statistics page, my custom handler under QueryHandler's request count
 remains 0. It shows that, the webrequest is not coming to my class

 Noble Paul wrote:

>
> register your handler in some other name and fire a request to that
>
> On Fri, Jun 12, 2009 at 8:07 PM, noor wrote:
>
>
>>
>> I solved this NullPointerException, by the following changes.
>>
>> In java code:
>> public void handleRequestBody(SolrQueryRequest request,
>> SolrQueryResponse
>> response) throws Exception {
>> SolrCore coreToRequest =
>> request.getCore();//coreContainer.getCore("core2");
>> .
>> }
>>
>> and in solr-config.xml:
>> 
>> 
>> explicit
>> tandem
>> true
>> 
>> 
>>
>> Now, my webapp runs fine by,
>> http://localhost:8983/mysearch
>> searching also working fine.
>> But, these are not run through my custom handler. So i felt, it
>> wrongly
>> doing searching.
>> Because, in solr admin statistics page,
>> my custom handler under QueryHandler's request count remains 0, it
>> doesn't
>> get incremented, when i search something. Rather,
>> statndardReqHandler's
>> request count is incremented.
>>
>> And another thing, how do we debug solr. ???
>> Please anybody help me to solve this ...
>>
>> Thanks in advance.
>>
>> Noble Paul ??? ?? wrote:
>>
>>
>>>
>>> is there any error on the console?
>>>
>>> On Fri, Jun 12, 2009 at 4:26 PM, Noor
>>> wrote:
>>>
>>>
>>>

 hi,
  i am new to apache solr.
 I need to create a custom request handler class. So i create a new
 one
 and
 changed the solr-config.xml file as,
  
    
        explicit
        tandem
        true
    
  

 And in my java class, the code is,

 public class MyCustomHandler extends RequestHandlerBase{
  public CoreContainer coreContainer;
  public void handleRequestBody(SolrQueryRequest request,
 SolrQueryResponse
 response) throws Exception {
    SolrCore coreToRequest = coreContainer.getCore("core2");
    ModifiableSolrParams params = new ModifiableSolrParams();
    params.set("echoParams", "explicit");
    params.set("q", "text");
    params.set("debugQuery", "true");
 request = new LocalSolrQueryRequest(coreToRequest, params);
 //             SolrRequestHandler reqHandler =
 coreToRequest.getRequestHandler("/select");
    coreToRequest.execute(reqHandler, request, response);
    coreToRequest.close();
    request.close();
  }
  // the abstract methods - getDescription(), getSourceId(),
 getSource(),
 getVersion() are //overrided... but these methods doesn't have any
 implementations.
 }


 But, if i search any text in my webapp from browser, gots the HTTP
 500
 error.
 i dont know how SolrContainer is intialized
 Pls anyone give me the solution...

 thanks and regards,
 Mohamed




>>>
>>>
>>
>>
>
>
>


>>>
>>>
>>>
>>>
>>
>>
>
>



-- 
-
Noble Paul | Principal Engineer| AOL | http://aol.com


Re: Count of facet count

2009-06-16 Thread Karsten F.

Hi,

Some questions about an integer with numFacetFound for facets in solr:

Our application "tecfinder" has a faceted browsing with paging for each
facet 
(see "Fachgebiete" in
http://tecfinder.fiz-technik.de/tecfinder/faces/facelets/search/search.jsp?query=Laser
).
Currently I am integrating solr in "tecfinder". So I need the number of
terms that have a non zero count.
e.g. if I search with
facet=true&facet.field=LG&f.LG.facet.limit=4
we need something like numFacetFound=19 for field LG (see example below).

To program an addition for SimpleFacets should be easy,
because I only have to change three methods:
 - getFacetTermEnumCounts (with respect to the line "if (--lim<0) break;")
 - getFieldCacheCounts (count non zero values of "int[] counts")
 - UnInvertedField.getCounts (again "int[] counts")

It will not be easy to change DistribFieldFacet, but I hope this can wait
("This API is experimental and subject to change").

So my questions:
Is there already an "numFacetFound" addition programed for solr?
 
Did I miss something? Is there more to change then this three methods?

What is an accurate place to expand the response with "numFacetFound"?
(Should I derive a subclass "SubList" or "OffsetList" of NamedList with an
attribute "totalAmount").

Any advice is welcome

  Karsten

p.s. the example for
facet=true&facet.field=LG&f.LG.facet.limit=4



  

  
19
592
191
82
32
  

  

 


hossman wrote:
> 
> 
> : > : how I can get count of distinct facet_fields ?
> : > : 
> : > : like numFacetFound in this example:
> : > 
> : > There's currently no way to do that.
> 
> : I need to do the same thing. Any pointers on how one would go about
> : implementing that? (in Java) Thanks.
> 
> The change would be in the SimpleFacets class, and there are a couple of 
> differnet code paths to worry about (because two different hueristics are 
> used depending on the field type) but the first step would be to define 
> what the count represents: is it just the number of terms being returned? 
> the number of terms that have a non zero count? or all of the terms in the 
> field?
> 
> the first and the last are pretty trivial, the middle one requires 
> maintaining a new count as the terms are scanned (and if i'm not mistaken, 
> there's an optimization in there to stop once we know we won't find any 
> terms better then theones we already have, and in order to return that 
> count you'd need to prevent that optimization)
> 
> 
> -Hoss
> 
> 
> 

-- 
View this message in context: 
http://www.nabble.com/Count-of-facet-count-tp16678811p24052214.html
Sent from the Solr - User mailing list archive at Nabble.com.



Re: Few Queries regarding indexes in Solr

2009-06-16 Thread Otis Gospodnetic

Hello,

 
Answers inlined.


- Original Message 

>   1. Is it possible to query from another index folder (say index1) in
> solr?

You don't really query *from* an index, so I'm not sure what exactly you mean.  
However, you can query Solr remotely, if that's what you mean, for example 
using SolrJ if your application is written in Java.

>   2. Is it possible to query 2 indexes(folders index1 and index2) stored
> in the same machine using the same port on a single solr instance?


Yes, please look for "DistributedSearch" on the Wiki.

>   3. consider a case:
>i have indexes in 2 shards, and i merge the indexes (present in 2
> shards) onto the 3rd shard
>now i add more documents into shard1 and delete some documents
> from shard 2 and update the indexes.
>is it possible to send the differences only into shard 3 and then
> merge it at shard 3?


If I understood you correctly, then no, sending only the newly added/deleted 
documents in 2 indices to the 3rd index after the fact is not doable.

Otis
--
Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch



Integration of Solr with Terracotta

2009-06-16 Thread bhushan_mahale
Hi,

I am looking for some way to integrate Solr with Terracotta.

There is already an issue logged in Solr JIRA:
http://jira.terracotta.org/jira/browse/CDV-399

But it is logged almost 2 years back.

Is the integration with Terracotta possible in the latest release (1.3.0)?

Thanks
- Bhushan

DISCLAIMER
==
This e-mail may contain privileged and confidential information which is the 
property of Persistent Systems Ltd. It is intended only for the use of the 
individual or entity to which it is addressed. If you are not the intended 
recipient, you are not authorized to read, retain, copy, print, distribute or 
use this message. If you have received this communication in error, please 
notify the sender and delete all copies of this message. Persistent Systems 
Ltd. does not accept any liability for virus infected mails.


Re: Problem with Query Parser?

2009-06-16 Thread Avlesh Singh
Can someone explain this?
+myField:"\*" +city:Mumbai gives me all results for +city:Mumbai

myField is a regular text field and "*" is not a stopword.

Cheers
Avlesh

On Tue, Jun 16, 2009 at 10:26 AM, Yonik Seeley
wrote:

> On Tue, Jun 16, 2009 at 12:28 AM, Avlesh Singh wrote:
> >>
> >> Probably the analyzer removed the "$", leaving an empty term and causing
> >> the clause to be removed altogether.
> >>
> >
> > I predicted this behavior while writing the mail yesterday, Yonik.
> > Does it sound logical and intuitive?
>
> It's intuitive in some circumstances, and not in others.  It's
> certainly not intuitive in this particular case.  I think there's
> another JIRA issue already open for this somewhere.
>
> -Yonik
> http://www.lucidimagination.com
>


Re: Few Queries regarding indexes in Solr

2009-06-16 Thread Michael Ludwig

Rakhi Khatwani schrieb:


1. Is it possible to query from another index folder (say
index1) in solr?


I think you're looking for the multi-core feature.

http://wiki.apache.org/solr/MultipleIndexes
http://wiki.apache.org/solr/CoreAdmin


2. Is it possible to query 2 indexes(folders index1 and index2)
stored in the same machine using the same port on a single solr
instance?


Sounds like multi-core.


3. consider a case: i have indexes in 2 shards, and i merge the
indexes (present in 2 shards) onto the 3rd shard now i add more
documents into shard1 and delete some documents from shard 2 and
update the indexes. is it possible to send the differences only
into shard 3 and then merge it at shard 3?


My (very limited) understanding of shards is that you repartition
your documents among shards and send each document to only one
shard. (Not sure this is correct.)

Michael Ludwig


Re: Integration of Solr with Terracotta

2009-06-16 Thread Yonik Seeley
Solr (1.4) now has a DirectoryFactory, so it would be much easier to
try this out.

-Yonik
http://www.lucidimagination.com


On Tue, Jun 16, 2009 at 8:29 AM,
bhushan_mahale wrote:
> Hi,
>
> I am looking for some way to integrate Solr with Terracotta.
>
> There is already an issue logged in Solr JIRA:
> http://jira.terracotta.org/jira/browse/CDV-399
>
> But it is logged almost 2 years back.
>
> Is the integration with Terracotta possible in the latest release (1.3.0)?
>
> Thanks
> - Bhushan
>
> DISCLAIMER
> ==
> This e-mail may contain privileged and confidential information which is the 
> property of Persistent Systems Ltd. It is intended only for the use of the 
> individual or entity to which it is addressed. If you are not the intended 
> recipient, you are not authorized to read, retain, copy, print, distribute or 
> use this message. If you have received this communication in error, please 
> notify the sender and delete all copies of this message. Persistent Systems 
> Ltd. does not accept any liability for virus infected mails.
>


Re: Problem with Query Parser?

2009-06-16 Thread Yonik Seeley
On Tue, Jun 16, 2009 at 8:33 AM, Avlesh Singh wrote:
> Can someone explain this?
> +myField:"\*" +city:Mumbai gives me all results for +city:Mumbai
>
> myField is a regular text field and "*" is not a stopword.

* and other non alphanumerics are probably being dropped by WordDelimiterFilter.

-Yonik
http://www.lucidimagination.com


Re: Problem with Query Parser?

2009-06-16 Thread Avlesh Singh
Thanks Yonik!

Cheers
Avlesh

On Tue, Jun 16, 2009 at 7:25 PM, Yonik Seeley wrote:

> On Tue, Jun 16, 2009 at 8:33 AM, Avlesh Singh wrote:
> > Can someone explain this?
> > +myField:"\*" +city:Mumbai gives me all results for +city:Mumbai
> >
> > myField is a regular text field and "*" is not a stopword.
>
> * and other non alphanumerics are probably being dropped by
> WordDelimiterFilter.
>
> -Yonik
> http://www.lucidimagination.com
>


RE: Integration of Solr with Terracotta

2009-06-16 Thread bhushan_mahale

Hey Yonik,

Thanks for the reply.

Can I use something like Compass for the integration? 
(http://www.terracotta.org/web/display/orgsite/Lucene+Integration)

The idea is to change Solr code at the places where it uses 
lucene.store.directory.
Use the TerracottaDirectory (provided by Compass) instead.

Do you suggest that there is some additional directory defined in Solr 1.4 
DirectoryFactory for the smooth integration with Terracotta?
Can you elaborate more on the required steps?

BTW When is the Solr 1.4 expected to release?

Thanks a ton,
- Bhushan

-Original Message-
From: ysee...@gmail.com [mailto:ysee...@gmail.com] On Behalf Of Yonik Seeley
Sent: Tuesday, June 16, 2009 7:06 PM
To: solr-user@lucene.apache.org
Subject: Re: Integration of Solr with Terracotta

Solr (1.4) now has a DirectoryFactory, so it would be much easier to
try this out.

-Yonik
http://www.lucidimagination.com


On Tue, Jun 16, 2009 at 8:29 AM,
bhushan_mahale wrote:
> Hi,
>
> I am looking for some way to integrate Solr with Terracotta.
>
> There is already an issue logged in Solr JIRA:
> http://jira.terracotta.org/jira/browse/CDV-399
>
> But it is logged almost 2 years back.
>
> Is the integration with Terracotta possible in the latest release (1.3.0)?
>
> Thanks
> - Bhushan
>
> DISCLAIMER
> ==
> This e-mail may contain privileged and confidential information which is the 
> property of Persistent Systems Ltd. It is intended only for the use of the 
> individual or entity to which it is addressed. If you are not the intended 
> recipient, you are not authorized to read, retain, copy, print, distribute or 
> use this message. If you have received this communication in error, please 
> notify the sender and delete all copies of this message. Persistent Systems 
> Ltd. does not accept any liability for virus infected mails.
>

DISCLAIMER
==
This e-mail may contain privileged and confidential information which is the 
property of Persistent Systems Ltd. It is intended only for the use of the 
individual or entity to which it is addressed. If you are not the intended 
recipient, you are not authorized to read, retain, copy, print, distribute or 
use this message. If you have received this communication in error, please 
notify the sender and delete all copies of this message. Persistent Systems 
Ltd. does not accept any liability for virus infected mails.


Re: Integration of Solr with Terracotta

2009-06-16 Thread Yonik Seeley
On Tue, Jun 16, 2009 at 10:17 AM,
bhushan_mahale wrote:
> The idea is to change Solr code at the places where it uses 
> lucene.store.directory.
> Use the TerracottaDirectory (provided by Compass) instead.

I thought Teracotta could work directly on RAMDirectory somehow... but
trying out the implementation from Compass seems reasonable.  I don't
know much about Teracotta - but I've been skeptical in the past that
it can automatically provide good replication at scale.  I'd love to
be proven wrong though!

> Do you suggest that there is some additional directory defined in Solr 1.4 
> DirectoryFactory for the smooth integration with Terracotta?
> Can you elaborate more on the required steps?
>
> BTW When is the Solr 1.4 expected to release?

It was supposed to have been about now - but now Lucene 2.9 looks like
it's wrapping things up too, and this particular Lucene release has
had so many changes, it may be wise to wait until we can use a
non-trunk version.

-Yonik
http://www.lucidimagination.com


Query parameter encode issue

2009-06-16 Thread Radha C.
Hello list,
 
I am having the following query,
q=+text:test +site_id:(4 ) +publishDate:[2008-05-01T00\:00\:00Z TO
2009-06-30T00\:00\:00Z]
 
If I try this query in the browser directly , it is working fine and the url
is encoded automatically in the browser when I enter as follows
 
http://localhost:8983/solr/TeamSite/select?q=+text:test%20+site_id:(4%20)%20
+publishDate:[2008-05-01T00\:00\:00Z%20TO%202009-06-30T00\:00\:00Z]
 
In my developed solr client, I am using the following code to encode,
 
qryString = "+text:test +site_id:(4 ) +publishDate:[2008-05-01T00\:00\:00Z
TO 2009-06-30T00\:00\:00Z]";
URLEncoder.encode(qryString, "UTF-8");
 
and the encoded url is like this,
 
http://localhost:8983/solr/TeamSite/select?q=%2Btext%3Atest+%2Bsite_id%3A%28
4+%29+%2BpublishDate%3A%5B2008-05-01T00%5C%3A00%5C%3A00Z+TO+2009-06-30T00%5C
%3A00%5C%3A00Z%5D
 
I am just encoding the parameter value ( +text:test +site_id:(4 )
+publishDate:[2008-05-01T00\:00\:00Z TO 2009-06-30T00\:00\:00Z] ) and not
parameter name ( q=).
 
Can anyone please tell me what mistake I have done here?
 


Searching across multivalued fields

2009-06-16 Thread MilkDud

I'm trying to prevent a search from going across multiple values in a
multivalued field and am running into an issue.  From what I've read, the
standard way to do this is with a positionIncrementGap that is larger than
the ps value.  However, I can't make this a phrase query because there is
another field that has to be searched against.

To be more specific, I'm indexing a collection of music albums that have
multiple tracks and an album artist.  So, some searches will contain both
the artist name and the track name.  I can't make this a single phrase query
as it is indexed across two separate fields.  So a small ps with a large
posIncGap doesn't do anything.  Is there any way to get past this?
-- 
View this message in context: 
http://www.nabble.com/Searching-across-multivalued-fields-tp24056297p24056297.html
Sent from the Solr - User mailing list archive at Nabble.com.



moreLikeThis fl

2009-06-16 Thread Matt Mitchell
I'd like to have a MLT query return similar docs, but the fl for those mlt
docs should be different from the main fl. For example, the main fl is "*,
score" -- but I only want the title and id in my MLT results. Is this
possible?

Matt


Re: Query parameter encode issue

2009-06-16 Thread Avlesh Singh
>
> qryString = "+text:test +site_id:(4 ) +publishDate:[2008-05-01T00\:00\:00Z
> TO 2009-06-30T00\:00\:00Z]";
> URLEncoder.encode(qryString, "UTF-8");
>

You don't have to encode the complete query string parameter. You just need
encode the values for individual query paramters.
So it should be more like qryString = "+text:" + URLEncoder.encode("test",
"UTF-8") ... and so on.

Cheers
Avlesh

On Tue, Jun 16, 2009 at 8:20 PM, Radha C.  wrote:

> Hello list,
>
> I am having the following query,
> q=+text:test +site_id:(4 ) +publishDate:[2008-05-01T00\:00\:00Z TO
> 2009-06-30T00\:00\:00Z]
>
> If I try this query in the browser directly , it is working fine and the
> url
> is encoded automatically in the browser when I enter as follows
>
>
> http://localhost:8983/solr/TeamSite/select?q=+text:test%20+site_id:(4%20)%20
> +publishDate:[2008-05-01T00
> \:00\:00Z%20TO%202009-06-30T00\:00\:00Z]
>
> In my developed solr client, I am using the following code to encode,
>
> qryString = "+text:test +site_id:(4 ) +publishDate:[2008-05-01T00\:00\:00Z
> TO 2009-06-30T00\:00\:00Z]";
> URLEncoder.encode(qryString, "UTF-8");
>
> and the encoded url is like this,
>
>
> http://localhost:8983/solr/TeamSite/select?q=%2Btext%3Atest+%2Bsite_id%3A%28
>
> 4+%29+%2BpublishDate%3A%5B2008-05-01T00%5C%3A00%5C%3A00Z+TO+2009-06-30T00%5C
> %3A00%5C%3A00Z%5D
>
> I am just encoding the parameter value ( +text:test +site_id:(4 )
> +publishDate:[2008-05-01T00\:00\:00Z TO 2009-06-30T00\:00\:00Z] ) and not
> parameter name ( q=).
>
> Can anyone please tell me what mistake I have done here?
>
>


Range queries

2009-06-16 Thread gwk

Hi,

When doing range queries it seems the query is either x:[5 TO 8] which 
means 5 <= x <= 8 or x:{5 TO 8} which means 5 < x < 8. But how do you 
get one half exclusive, the other inclusive for double fields the 
following: 5 <= x < 8? Is this possible?


Regards,

gwk


RE: Query parameter encode issue

2009-06-16 Thread Radha C.
 
Thanks for your reply.. 
 
If that is the case, I need to do as follows, 
 
"+text:" + URLEncoder.encode("test", "UTF-8") +URLEncoder.encode(" ",
"UTF-8")+"+site_id"+URLEncoder.encode(xx, "UTF-8") 
 
Do I need to encode the space between two search field also? It is difficult
for me to do like this because I am having many search fields and forming
the query string dynamically. 
 
Is there any other way I can do this? or else I will follow this then.
 
 

 
  _  

From: Avlesh Singh [mailto:avl...@gmail.com] 
Sent: Tuesday, June 16, 2009 8:38 PM
To: solr-user@lucene.apache.org; cra...@ceiindia.com
Subject: Re: Query parameter encode issue



qryString = "+text:test +site_id:(4 ) +publishDate:[2008-05-01T00\:00\:00Z
TO 2009-06-30T00\:00\:00Z]";
URLEncoder.encode(qryString, "UTF-8");



You don't have to encode the complete query string parameter. You just need
encode the values for individual query paramters.

So it should be more like qryString = "+text:" + URLEncoder.encode("test",
"UTF-8") ... and so on.

Cheers
Avlesh


On Tue, Jun 16, 2009 at 8:20 PM, Radha C.  wrote:


Hello list,

I am having the following query,
q=+text:test +site_id:(4 ) +publishDate:[2008-05-01T00\:00\:00Z TO
2009-06-30T00\:00\:00Z]

If I try this query in the browser directly , it is working fine and the url
is encoded automatically in the browser when I enter as follows

http://localhost:8983/solr/TeamSite/select?q=+text:test%20+site_id:(4%20)%20
 
+publishDate:[2008-05-01T00\:00\:00Z%20TO%202009-06-30T00\:00\:00Z]

In my developed solr client, I am using the following code to encode,

qryString = "+text:test +site_id:(4 ) +publishDate:[2008-05-01T00\:00\:00Z
TO 2009-06-30T00\:00\:00Z]";
URLEncoder.encode(qryString, "UTF-8");

and the encoded url is like this,

http://localhost:8983/solr/TeamSite/select?q=%2Btext%3Atest+%2Bsite_id%3A%28
 
4+%29+%2BpublishDate%3A%5B2008-05-01T00%5C%3A00%5C%3A00Z+TO+2009-06-30T00%5C
%3A00%5C%3A00Z%5D

I am just encoding the parameter value ( +text:test +site_id:(4 )
+publishDate:[2008-05-01T00\:00\:00Z TO 2009-06-30T00\:00\:00Z] ) and not
parameter name ( q=).

Can anyone please tell me what mistake I have done here?






MoreLikeThis and TermVectors

2009-06-16 Thread Marc Sturlese

Hey there,
As I understood documentation, to have TermVectors will help to MoreLikeThis
in performance side but the similitude of the results will be the same. Am I
correct?
Thanks in advance
-- 
View this message in context: 
http://www.nabble.com/MoreLikeThis-and-TermVectors-tp24057914p24057914.html
Sent from the Solr - User mailing list archive at Nabble.com.



Uncommitted documents coutn with autocommit enabled?

2009-06-16 Thread Ingo Renner

Hi all,

is there a way to find out how many documents are "pending" when  
having autocommit enabled?



best
Ingo

--
Ingo Renner
TYPO3 Core Developer, Release Manager TYPO3 4.2





Re: MoreLikeThis and TermVectors

2009-06-16 Thread Grant Ingersoll
They should be, assuming the Analysis process creates the same tokens  
with the same frequency as was created when the Term Vectors were  
created.


On Jun 16, 2009, at 12:24 PM, Marc Sturlese wrote:



Hey there,
As I understood documentation, to have TermVectors will help to  
MoreLikeThis
in performance side but the similitude of the results will be the  
same. Am I

correct?
Thanks in advance
--
View this message in context: 
http://www.nabble.com/MoreLikeThis-and-TermVectors-tp24057914p24057914.html
Sent from the Solr - User mailing list archive at Nabble.com.



--
Grant Ingersoll
http://www.lucidimagination.com/

Search the Lucene ecosystem (Lucene/Solr/Nutch/Mahout/Tika/Droids)  
using Solr/Lucene:

http://www.lucidimagination.com/search



Re: Joins or subselects in solr

2009-06-16 Thread Nasseam Elkarra
Thanks for the advice. Glad you like the site! Yes, we are using the  
new multi-select support to do this. At first, we wrote our own client- 
side solution to allow for multi-select which was super complex but  
moved to the built-in solr support as soon as we found it. Excellent  
feature!


-Nasseam

Solr-powered Ajax search+nav:
http://factbook.bodukai.com/

Powered by Boutique:
http://bodukai.com/boutique/


On Jun 14, 2009, at 8:16 AM, Yonik Seeley wrote:

On Fri, Jun 12, 2009 at 6:43 PM, Nasseam  
Elkarra wrote:
I am storing items in an index. Each item has a comma separated  
list of
related items. Is it possible to bring back an item and all of its  
related
items in one query? If so how and how would you distinguish between  
which

one is the main item and which are the related.


You would currently need a custom query component to do this... not
sure if it's worth it vs two requests.


Solr-powered Ajax search+nav:
http://factbook.bodukai.com/


Cool, nice multi-select!   Are you using Solr's new multi-select
support to do this?

-Yonik
http://www.lucidimagination.com




Re: Range queries

2009-06-16 Thread Otis Gospodnetic

Hi,

I think the square brackets/curly braces need to be balanced, so this is 
currently not doable with existing query parsers.

 Otis
--
Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch



- Original Message 
> From: gwk 
> To: solr-user@lucene.apache.org
> Sent: Tuesday, June 16, 2009 11:52:12 AM
> Subject: Range queries
> 
> Hi,
> 
> When doing range queries it seems the query is either x:[5 TO 8] which means 
> 5 
> <= x <= 8 or x:{5 TO 8} which means 5 < x < 8. But how do you get one half 
> exclusive, the other inclusive for double fields the following: 5 <= x < 8? 
> Is 
> this possible?
> 
> Regards,
> 
> gwk



Re: Uncommitted documents coutn with autocommit enabled?

2009-06-16 Thread Otis Gospodnetic

Ingo,

I believe that's still visible on the Statistics page of the Solr Admin UI.

 Otis
--
Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch



- Original Message 
> From: Ingo Renner 
> To: solr-user@lucene.apache.org
> Sent: Tuesday, June 16, 2009 12:33:05 PM
> Subject: Uncommitted documents coutn with autocommit enabled?
> 
> Hi all,
> 
> is there a way to find out how many documents are "pending" when having 
> autocommit enabled?
> 
> 
> best
> Ingo
> 
> --Ingo Renner
> TYPO3 Core Developer, Release Manager TYPO3 4.2



Re: moreLikeThis fl

2009-06-16 Thread Otis Gospodnetic

Matt,

Maybe I'm missing something, but can't you use mlt.fl in the URL?
Please see 
http://wiki.apache.org/solr/MoreLikeThis#head-6460069f297626f2a982f1e22ec5d1519c456b2c

Otis
--
Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch



- Original Message 
> From: Matt Mitchell 
> To: solr-user@lucene.apache.org
> Sent: Tuesday, June 16, 2009 11:08:00 AM
> Subject: moreLikeThis fl
> 
> I'd like to have a MLT query return similar docs, but the fl for those mlt
> docs should be different from the main fl. For example, the main fl is "*,
> score" -- but I only want the title and id in my MLT results. Is this
> possible?
> 
> Matt



Re: Replication problems on 1.4

2009-06-16 Thread Phil Hagelberg
Phil Hagelberg  writes:

> Noble Paul നോബിള്‍  नोब्ळ्  writes:
>
>> if you removed the files while the slave is running , then the slave
>> will not know that you removed the files (assuming it is a *nix box)
>> and it will serve the search requests. But if you restart the slave ,
>> it should have automatically picked up the current index.
>>
>> if it doesn't it is a bug
>
> I did restart the slave server in my case. If I can confirm this with
> the latest build from trunk, I will submit an issue.

Hmm... can't reproduce with a fresh checkout and recreating my indices
from that. Maybe it was something specifically misconfigured in my last
setup.

-Phil


Re: Query Filter fq with OR operator

2009-06-16 Thread brian519

This feature is very important to me .. should I post something on the dev
forum?  Not sure what the proper protocol is for adding a feature to the
roadmap

Thanks,
Brian.
-- 
View this message in context: 
http://www.nabble.com/Query-Filter-fq-with-OR-operator-tp23895837p24059181.html
Sent from the Solr - User mailing list archive at Nabble.com.



Re: Few Queries regarding indexes in Solr

2009-06-16 Thread Otis Gospodnetic

Regarding that 3rd answer below:
Yes, that's what most people do, though nothing prevents the indexing client 
from sending the same doc to multiple shards.  In some scenarios that's exactly 
what you want to do.

 Otis
--
Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch



- Original Message 
> From: Michael Ludwig 
> To: solr-user@lucene.apache.org
> Sent: Tuesday, June 16, 2009 8:36:25 AM
> Subject: Re: Few Queries regarding indexes in Solr
> 
> Rakhi Khatwani schrieb:
> 
> > 1. Is it possible to query from another index folder (say
> > index1) in solr?
> 
> I think you're looking for the multi-core feature.
> 
> http://wiki.apache.org/solr/MultipleIndexes
> http://wiki.apache.org/solr/CoreAdmin
> 
> > 2. Is it possible to query 2 indexes(folders index1 and index2)
> > stored in the same machine using the same port on a single solr
> > instance?
> 
> Sounds like multi-core.
> 
> > 3. consider a case: i have indexes in 2 shards, and i merge the
> > indexes (present in 2 shards) onto the 3rd shard now i add more
> > documents into shard1 and delete some documents from shard 2 and
> > update the indexes. is it possible to send the differences only
> > into shard 3 and then merge it at shard 3?
> 
> My (very limited) understanding of shards is that you repartition
> your documents among shards and send each document to only one
> shard. (Not sure this is correct.)
> 
> Michael Ludwig



Luke / get doc count for each term

2009-06-16 Thread Ryan McKinley
Hi-

I'm trying to use the LukeRequestHandler with an index of ~9 million
docs.  I know that counting the top / distinct terms for each field is
expensive and can take a LONG time to return.

Is there a faster way to check the number of documents for each field?
 Currently this gets the doc count for each term:

  if( sfield != null && sfield.indexed() ) {
Query q = qp.parse( fieldName+":[* TO *]" );
int docCount = searcher.numDocs( q, matchAllDocs );
...

Looking at it again, that could be replaced with:

  if( sfield != null && sfield.indexed() ) {
Query q = qp.parse( fieldName+":[* TO *]" );
int docCount = searcher.getDocSet( q ).size();
...

Is there any faster option then running a query for each field?

thanks
ryan


Re: Query Filter fq with OR operator

2009-06-16 Thread Otis Gospodnetic

Brian,

Opening a JIRA issue if it doesn't already exist is the best way.  If you can 
provide a patch, even better!

 Otis
--
Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch



- Original Message 
> From: brian519 
> To: solr-user@lucene.apache.org
> Sent: Tuesday, June 16, 2009 1:32:41 PM
> Subject: Re: Query Filter fq with OR operator
> 
> 
> This feature is very important to me .. should I post something on the dev
> forum?  Not sure what the proper protocol is for adding a feature to the
> roadmap
> 
> Thanks,
> Brian.
> -- 
> View this message in context: 
> http://www.nabble.com/Query-Filter-fq-with-OR-operator-tp23895837p24059181.html
> Sent from the Solr - User mailing list archive at Nabble.com.



Re: Query Filter fq with OR operator

2009-06-16 Thread brian519



Otis Gospodnetic wrote:
> 
> 
> Brian,
> 
> Opening a JIRA issue if it doesn't already exist is the best way.  If you
> can provide a patch, even better!
> 
>  Otis
> --
> Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch
> 
> Thanks Otis!  I've created a JIRA 
> https://issues.apache.org/jira/browse/SOLR-1223 issue  here
> 
> - Original Message 
>> From: brian519 
>> To: solr-user@lucene.apache.org
>> Sent: Tuesday, June 16, 2009 1:32:41 PM
>> Subject: Re: Query Filter fq with OR operator
>> 
>> 
>> This feature is very important to me .. should I post something on the
>> dev
>> forum?  Not sure what the proper protocol is for adding a feature to the
>> roadmap
>> 
>> Thanks,
>> Brian.
>> -- 
>> View this message in context: 
>> http://www.nabble.com/Query-Filter-fq-with-OR-operator-tp23895837p24059181.html
>> Sent from the Solr - User mailing list archive at Nabble.com.
> 
> 
> 

-- 
View this message in context: 
http://www.nabble.com/Query-Filter-fq-with-OR-operator-tp23895837p24060300.html
Sent from the Solr - User mailing list archive at Nabble.com.



Problem getting Solr statistics

2009-06-16 Thread Development Team
Hi all,
 I am stumped trying to get statistics from the Solr server. It seems
that every time I get the correct SolrInfoMBean, when I look up the proper
value (by name) in the NamedList, I get the exact same number back each
time. For example, upon start-up the server reports that "2" queries have
been performed, and any time I pull the value out of the MBean after that it
says "2" even though the stats.jsp reports an increasing number of queries
over time. What am I doing wrong?
 Here is my sample code:

public class SolrUtil {

  protected static final CoreContainer coreContainer;
  protected static final String DEFAULT_CORE_NAME = "";

  static {
CoreContainer.Initializer initializer = new CoreContainer.Initializer();
try {
  coreContainer = initializer.initialize();
}
catch (Exception e) {
  throw new ExceptionInInitializerError("Can't initialize core
container: " + e.getMessage());
}
initialize();
  }

  private static SolrCore getCore() {
return getCore(DEFAULT_CORE_NAME);
  }

  private static SolrCore getCore(String name) {
try {
  return coreContainer.getCore(name);
}
catch (Exception e) {
  e.printStackTrace();
}
return null;
  }

  public static String getSolrInfoMBeanValue(SolrInfoMBean.Category
category, String entryName, String statName) {
Map registry = getCore().getInfoRegistry();
for (Map.Entry entry : registry.entrySet()) {
  String key = entry.getKey();
  SolrInfoMBean solrInfoMBean = entry.getValue();
  if ((solrInfoMBean.getCategory() != category) ||
  (!entryName.equals(key.trim( {
continue;
  }
  NamedList nl = solrInfoMBean.getStatistics();
  if ((nl != null) && (nl.size() > 0)) {
for (int i = 0; i < nl.size(); i++) {
  if (nl.getName(i).equals(statName)) {
return nl.getVal(i).toString();
  }
}
  }
}
return null;
  }

  [...I have other methods, that also get the value as a long, etc]

}



 This code is modeled after the SolrDispatchFilter.java, _info.jsp and
stats.jsp.
 I'd appreciate any help. (And yes, my core is named "".)

Sincerely,

 Daryl.


Using Lucene Filter with SolrIndexSearcher

2009-06-16 Thread Aleksandar Bradic

Hi all,

Any ideas on how to use custom Lucene Filter (lucene.search.Filter) in  
conjuction with SolrIndexSearcher ?


The case is writing custom Filter performing arbitrary records  
selection and AND-ing it to the existing filters passed in the query.


However, the ResponseBuilder filters are Query objects, and there is  
no visible means of mixing those with Filter's in the standard  
QueryCommand object.


(There is a search(Query query, Filter filter, Sort sort) method that  
allow passing of Filter but no way of mixing it with existing query.  
Also, using this means immediate loss of all caching benefits etc.).


Any other idea on how to pass a custom filter applying arbitrary user- 
defined logic (for example "random drop of 30% records from result  
set"), to standard query would be useful as well.


Thanks,
.Alek


Re: Using Lucene Filter with SolrIndexSearcher

2009-06-16 Thread Yonik Seeley
On Tue, Jun 16, 2009 at 3:31 PM, Aleksandar Bradic wrote:
> Any ideas on how to use custom Lucene Filter (lucene.search.Filter) in
> conjuction with SolrIndexSearcher ?

You can wrap the original query with a FilteredQuery class, or create
a DocSet from the Filter via SolrIndexSearcher.convertFilter()

-Yonik
http://www.lucidimagination.com

> The case is writing custom Filter performing arbitrary records selection and
> AND-ing it to the existing filters passed in the query.
>
> However, the ResponseBuilder filters are Query objects, and there is no
> visible means of mixing those with Filter's in the standard QueryCommand
> object.
>
> (There is a search(Query query, Filter filter, Sort sort) method that allow
> passing of Filter but no way of mixing it with existing query. Also, using
> this means immediate loss of all caching benefits etc.).
>
> Any other idea on how to pass a custom filter applying arbitrary
> user-defined logic (for example "random drop of 30% records from result
> set"), to standard query would be useful as well.
>
> Thanks,
> .Alek
>


Re: Range queries

2009-06-16 Thread Peter Keegan
How about this: x:[5 TO 8] AND x:{0 TO 8}

On Tue, Jun 16, 2009 at 1:16 PM, Otis Gospodnetic <
otis_gospodne...@yahoo.com> wrote:

>
> Hi,
>
> I think the square brackets/curly braces need to be balanced, so this is
> currently not doable with existing query parsers.
>
>  Otis
> --
> Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch
>
>
>
> - Original Message 
> > From: gwk 
> > To: solr-user@lucene.apache.org
> > Sent: Tuesday, June 16, 2009 11:52:12 AM
> > Subject: Range queries
> >
> > Hi,
> >
> > When doing range queries it seems the query is either x:[5 TO 8] which
> means 5
> > <= x <= 8 or x:{5 TO 8} which means 5 < x < 8. But how do you get one
> half
> > exclusive, the other inclusive for double fields the following: 5 <= x <
> 8? Is
> > this possible?
> >
> > Regards,
> >
> > gwk
>
>


Re: Luke / get doc count for each term

2009-06-16 Thread Yonik Seeley
doc count for each term is stored directly in the index - with the big
caveat that it doesn't take deleted docs into account.  That addresses
the "get doc count for each term".

"get doc count for each field" is a different question... see below.

On Tue, Jun 16, 2009 at 1:57 PM, Ryan McKinley wrote:
> Hi-
>
> I'm trying to use the LukeRequestHandler with an index of ~9 million
> docs.  I know that counting the top / distinct terms for each field is
> expensive and can take a LONG time to return.
>
> Is there a faster way to check the number of documents for each field?
>  Currently this gets the doc count for each term:
>
>      if( sfield != null && sfield.indexed() ) {
>        Query q = qp.parse( fieldName+":[* TO *]" );
>        int docCount = searcher.numDocs( q, matchAllDocs );

That looks like it gets the doc count for each field, as opposed to each term.

> Looking at it again, that could be replaced with:
>
>      if( sfield != null && sfield.indexed() ) {
>        Query q = qp.parse( fieldName+":[* TO *]" );
>        int docCount = searcher.getDocSet( q ).size();

Correct.  Unfortunately it probably won't save you much (one set intersection).
I don't (currently) know of a way to get this info quicker.

In a specific application, the fastest way would be to index a boolean
or another single token for each document that had the field you were
interested in then count the number of docs for the single token
rather than all tokens in the field.

-Yonik
http://www.lucidimagination.com

> Is there any faster option then running a query for each field?
>
> thanks
> ryan
>


Re: Luke / get doc count for each term

2009-06-16 Thread Grant Ingersoll


On Jun 16, 2009, at 1:57 PM, Ryan McKinley wrote:



Is there a faster way to check the number of documents for each field?
Currently this gets the doc count for each term:



In the past, I've created a field that contains the names of the  
Fields present on the document.  Then, simply facet on the new Field.   
I think that gets you what you want and the mechanism is all built in  
to Solr and is quite speedy.


DisMaxRequestHandler usage

2009-06-16 Thread siping liu

Hi,

I have this standard query:

q=(field1:hello OR field2:hello) AND (field3:world)

 

Can I use dismax handler for this (applying the same search term on field1 and 
field2, but keep field3 with something separate)? If it can be done, what's the 
advantage of doing it this way over using the standard query?

 

thanks.

_
Microsoft brings you a new way to search the web.  Try  Bing™ now
http://www.bing.com?form=MFEHPG&publ=WLHMTAG&crea=TEXT_MFEHPG_Core_tagline_try 
bing_1x1

Re: Using Lucene Filter with SolrIndexSearcher

2009-06-16 Thread Aleksandar Bradic

Ah, right.
FilteredQuery works just great

Thanks, Yonik

Best,
.Alek

On Jun 16, 2009, at 9:45 PM, Yonik Seeley wrote:

On Tue, Jun 16, 2009 at 3:31 PM, Aleksandar  
Bradic wrote:
Any ideas on how to use custom Lucene Filter (lucene.search.Filter)  
in

conjuction with SolrIndexSearcher ?


You can wrap the original query with a FilteredQuery class, or create
a DocSet from the Filter via SolrIndexSearcher.convertFilter()

-Yonik
http://www.lucidimagination.com

The case is writing custom Filter performing arbitrary records  
selection and

AND-ing it to the existing filters passed in the query.

However, the ResponseBuilder filters are Query objects, and there  
is no
visible means of mixing those with Filter's in the standard  
QueryCommand

object.

(There is a search(Query query, Filter filter, Sort sort) method  
that allow
passing of Filter but no way of mixing it with existing query.  
Also, using

this means immediate loss of all caching benefits etc.).

Any other idea on how to pass a custom filter applying arbitrary
user-defined logic (for example "random drop of 30% records from  
result

set"), to standard query would be useful as well.

Thanks,
.Alek





NPE while sorting

2009-06-16 Thread Nirkhe, Chandra
Hi,
Solr is throwing Null pointer exception when soring on a field.


http://localhost:9080/solr/select?version=2.2&q=*:*&qt=geo&lat=41.883784
&long=-87.637668&radius=25&rows=100&sort=geo_distance%20top


Using 16-June build. Any pointers ??


HTTP Status 500 - null java.lang.NullPointerException at
org.apache.lucene.search.SortField.getComparator(SortField.java:483) at
org.apache.lucene.search.FieldValueHitQueue$OneComparatorFieldValueHitQu
eue.(FieldValueHitQueue.java:80) at
org.apache.lucene.search.FieldValueHitQueue.create(FieldValueHitQueue.ja
va:190) at
org.apache.lucene.search.TopFieldCollector.create(TopFieldCollector.java
:851) at
org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.
java:972) at
org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.j
ava:880) at
org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:3
37) at
org.apache.solr.search.SolrIndexSearcher.getDocList(SolrIndexSearcher.ja
va:1149) at
com.pjaol.search.solr.component.LocalSolrQueryComponent.process(LocalSol
rQueryComponent.java:286) at
org.apache.solr.handler.component.SearchHandler.handleRequestBody(Search
Handler.java:195) at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerB
ase.java:131) at
org.apache.solr.core.SolrCore.execute(SolrCore.java:1328) at
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.ja
va:341) at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.j
ava:244) at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Applica
tionFilterChain.java:215) at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilt
erChain.java:188) at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValv
e.java:213) at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValv
e.java:174) at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java
:127) at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java
:117) at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.
java:108) at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:1
51) at
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:87
4) at
org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.proc
essConnection(Http11BaseProtocol.java:665) at
org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint
.java:528) at
org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollow
erWorkerThread.java:81) at
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool
.java:689) at java.lang.Thread.run(Thread.java:595)


Re: NPE while sorting

2009-06-16 Thread Yonik Seeley
Looks like this is the local-solr patch, which isn't part of Solr yet?

-Yonik
http://www.lucidimagination.com

On Tue, Jun 16, 2009 at 6:15 PM, Nirkhe, Chandra wrote:
> Hi,
> Solr is throwing Null pointer exception when soring on a field.
>
>
> http://localhost:9080/solr/select?version=2.2&q=*:*&qt=geo&lat=41.883784
> &long=-87.637668&radius=25&rows=100&sort=geo_distance%20top
>
>
> Using 16-June build. Any pointers ??
>
>
> HTTP Status 500 - null java.lang.NullPointerException at
> org.apache.lucene.search.SortField.getComparator(SortField.java:483) at
> org.apache.lucene.search.FieldValueHitQueue$OneComparatorFieldValueHitQu
> eue.(FieldValueHitQueue.java:80) at
> org.apache.lucene.search.FieldValueHitQueue.create(FieldValueHitQueue.ja
> va:190) at
> org.apache.lucene.search.TopFieldCollector.create(TopFieldCollector.java
> :851) at
> org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.
> java:972) at
> org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.j
> ava:880) at
> org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:3
> 37) at
> org.apache.solr.search.SolrIndexSearcher.getDocList(SolrIndexSearcher.ja
> va:1149) at
> com.pjaol.search.solr.component.LocalSolrQueryComponent.process(LocalSol
> rQueryComponent.java:286) at
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(Search
> Handler.java:195) at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerB
> ase.java:131) at
> org.apache.solr.core.SolrCore.execute(SolrCore.java:1328) at
> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.ja
> va:341) at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.j
> ava:244) at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Applica
> tionFilterChain.java:215) at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilt
> erChain.java:188) at
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValv
> e.java:213) at
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValv
> e.java:174) at
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java
> :127) at
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java
> :117) at
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.
> java:108) at
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:1
> 51) at
> org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:87
> 4) at
> org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.proc
> essConnection(Http11BaseProtocol.java:665) at
> org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint
> .java:528) at
> org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollow
> erWorkerThread.java:81) at
> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool
> .java:689) at java.lang.Thread.run(Thread.java:595)
>


Re: Luke / get doc count for each term

2009-06-16 Thread Ryan McKinley


On Jun 16, 2009, at 5:21 PM, Grant Ingersoll wrote:



On Jun 16, 2009, at 1:57 PM, Ryan McKinley wrote:



Is there a faster way to check the number of documents for each  
field?

Currently this gets the doc count for each term:



In the past, I've created a field that contains the names of the  
Fields present on the document.  Then, simply facet on the new  
Field.  I think that gets you what you want and the mechanism is all  
built in to Solr and is quite speedy.



makes sense -- i like this idea.

ryan


Re: localsolr sort

2009-06-16 Thread pjaol

Haven't tried the 14th build, but the latest stable version is available on
http://www.nsshutdown.com/solr-example.tgz
Built last week
-- 
View this message in context: 
http://www.nabble.com/localsolr-sort-tp24040799p24065745.html
Sent from the Solr - User mailing list archive at Nabble.com.



what date format to pass for search in Solr?

2009-06-16 Thread chem leakhina
Hi
Does anyone know what date format pass to search in Solr?
Could you give me any examples for search with Date in solr?

Regards,
Leahina


Re: Range queries

2009-06-16 Thread Avlesh Singh
And how about this - x:{5 TO 8} AND x:5

Cheers
Avlesh

On Wed, Jun 17, 2009 at 1:57 AM, Peter Keegan wrote:

> How about this: x:[5 TO 8] AND x:{0 TO 8}
>
> On Tue, Jun 16, 2009 at 1:16 PM, Otis Gospodnetic <
> otis_gospodne...@yahoo.com> wrote:
>
> >
> > Hi,
> >
> > I think the square brackets/curly braces need to be balanced, so this is
> > currently not doable with existing query parsers.
> >
> >  Otis
> > --
> > Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch
> >
> >
> >
> > - Original Message 
> > > From: gwk 
> > > To: solr-user@lucene.apache.org
> > > Sent: Tuesday, June 16, 2009 11:52:12 AM
> > > Subject: Range queries
> > >
> > > Hi,
> > >
> > > When doing range queries it seems the query is either x:[5 TO 8] which
> > means 5
> > > <= x <= 8 or x:{5 TO 8} which means 5 < x < 8. But how do you get one
> > half
> > > exclusive, the other inclusive for double fields the following: 5 <= x
> <
> > 8? Is
> > > this possible?
> > >
> > > Regards,
> > >
> > > gwk
> >
> >
>


Re: Range queries

2009-06-16 Thread Avlesh Singh
Really sorry, this is what I meant: x:{5 TO 8} OR x:5

Cheers
Avlesh

On Wed, Jun 17, 2009 at 9:36 AM, Avlesh Singh  wrote:

> And how about this - x:{5 TO 8} AND x:5
>
> Cheers
> Avlesh
>
>
> On Wed, Jun 17, 2009 at 1:57 AM, Peter Keegan wrote:
>
>> How about this: x:[5 TO 8] AND x:{0 TO 8}
>>
>> On Tue, Jun 16, 2009 at 1:16 PM, Otis Gospodnetic <
>> otis_gospodne...@yahoo.com> wrote:
>>
>> >
>> > Hi,
>> >
>> > I think the square brackets/curly braces need to be balanced, so this is
>> > currently not doable with existing query parsers.
>> >
>> >  Otis
>> > --
>> > Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch
>> >
>> >
>> >
>> > - Original Message 
>> > > From: gwk 
>> > > To: solr-user@lucene.apache.org
>> > > Sent: Tuesday, June 16, 2009 11:52:12 AM
>> > > Subject: Range queries
>> > >
>> > > Hi,
>> > >
>> > > When doing range queries it seems the query is either x:[5 TO 8] which
>> > means 5
>> > > <= x <= 8 or x:{5 TO 8} which means 5 < x < 8. But how do you get one
>> > half
>> > > exclusive, the other inclusive for double fields the following: 5 <= x
>> <
>> > 8? Is
>> > > this possible?
>> > >
>> > > Regards,
>> > >
>> > > gwk
>> >
>> >
>>
>
>