Re: Solr MultiCore query
Thanks ahammad for the quick reply. As suggested, I am trying out multi core way of implementing the search. I am trying out the multicore example and getting stuck at an issue. Here is what I did and the issue I am facing 1) Downloaded 1.4 and started the multicore example using java -Dsolr.solr.home=multicore -jar start.jar 2) There were 2 files present under example/multicore/exampledocs/ , which I added to 2 cores respectively. ( Totally 3 docs are present in those 2 files and all have the word 'ipod' in it ) 3) When I query using http://localhost:8983/solr/core0/select?shards=localhost:8983/solr/core0,localhost:8983/solr/core1&q=*:*I get all the 3 results. But when I query using http://localhost:8983/solr/core0/select?shards=localhost:8983/solr/core0,localhost:8983/solr/core1&q= *ipod* , I get no results :( What could be the issue ? Thanks! On Fri, Jul 17, 2009 at 7:20 PM, ahammad wrote: > > Hello, > > I'm not sure what the best way is to do this, but I have done something > identical. > > I have the same requirements, ie several datasources. I also used SolrJ and > jsp for this. The way I ended up doing it was to create a multi core > environment, one core per datasource. When I do a query across several > datasources, I use shards. Solr automatically returns a "hybrid" result set > that way, sorted by solr's default scoring. > > Faceting comes in the picture when you want to show the number of documents > per datasource and have the ability to narrow down the result set. The way > I > did it was to add a field called "dataSource" to all the documents, and > injected them with a default value of the data source name (in your case, > D1, D2 ...). You can do this by adding this in the schema: > > required="true" default="D1"/> > > When you perform a query across multiple datasources, you will use shards. > Here is an example: > > > http://localhost:8080/solr/core1/select?shards=localhost:8080/solr/core1,localhost:8080/solr/core2&q=some > query > > That will search on both cores 1 and 2. > > To facet on the datasource in order to be able to categorize the result > set, > you can simply add this snippet to the query: > > &facet=on&facet.field=dataSource > > This will return the datasources that are defined with their number of > results for the query. > > Making the facet results clickable in order to narrow down the results can > be achieved by adding a filter to the query and filtering to a specific > dataSource. I actually ended uo creating a fairly intuitive front-end for > my > system with faceting, filtering, paging etc all using jsp and SolrJ. SolrJ > is powerful enough to handle all of the backend processing. > > Good luck! > > > > > > > joe_coder wrote: > > > > I missed adding some size related information in the query above. > > > > D1 and D2 would have close to 1 million records each > > D3 would have ~10 million records. > > > > Thanks! > > > > -- > View this message in context: > http://www.nabble.com/Solr-MultiCore-query-tp24534383p24534793.html > Sent from the Solr - User mailing list archive at Nabble.com. > >
Re: Solr MultiCore query
Both schema.xml ( in example/multicore/core0/conf and example/multicore/core1/conf ) already have * name* Here are the following query responses: 1) http://localhost:8983/solr/core0/select?shards=localhost:8983/solr/core0,localhost:8983/solr/core1&q=*:* 0254MA147LL/AApple 60 GB iPod with Video Playback BlackF8V7067-APL-KITBelkin Mobile Power Cord for iPod w/ DockIW-02iPod & iPod Mini USB 2.0 Cable 2) http://localhost:8983/solr/core0/select?shards=localhost:8983/solr/core0,localhost:8983/solr/core1&q= *ipod* No result 3) http://localhost:8983/solr/core0/select?shards=localhost:8983/solr/core0,localhost:8983/solr/core1&q= *name:ipod *No result* * What may be happening? Thanks! On Fri, Jul 17, 2009 at 11:37 PM, ahammad wrote: > > Hello joe_coder, > > Are you using the default example docs in your queries? > > If so, then I see that the word "ipod" appears in a field called "name". By > default, the default search field (defined in solrconfig.xml) is the field > called "text". This means that when you submit a query without specifying > which field to look for (using the field:query) notation, Solr > automatically > assumes that you are looking in the field called "text". > > If you change your query to q=name:ipod, you should get the results back. > > One way to prevent this is to change your default search field to something > else. Alternatively, if you want to search on multiple fields, you can copy > all those fields to the "text" field and go from there. This can be useful > if for example you had a book library to search through. You may need to > search on title, short summary, description etc simultaneously. You can > copy > all those things to the text field and then search on the text field, which > contains all the information that you wanted to search on. > > > joe_coder wrote: > > > > Thanks ahammad for the quick reply. > > > > As suggested, I am trying out multi core way of implementing the search. > I > > am trying out the multicore example and getting stuck at an issue. Here > is > > what I did and the issue I am facing > > > > 1) Downloaded 1.4 and started the multicore example using java > > -Dsolr.solr.home=multicore -jar start.jar > > > > 2) There were 2 files present under example/multicore/exampledocs/ , > which > > I > > added to 2 cores respectively. ( Totally 3 docs are present in those 2 > > files > > and all have the word 'ipod' in it ) > > > > 3) When I query using > > > http://localhost:8983/solr/core0/select?shards=localhost:8983/solr/core0,localhost:8983/solr/core1&q=*:*I > > get all the 3 results. > > > > But when I query using > > > http://localhost:8983/solr/core0/select?shards=localhost:8983/solr/core0,localhost:8983/solr/core1&q= > > *ipod* , I get no results :( > > > > What could be the issue ? > > > > Thanks! > > > > > > On Fri, Jul 17, 2009 at 7:20 PM, ahammad wrote: > > > >> > >> Hello, > >> > >> I'm not sure what the best way is to do this, but I have done something > >> identical. > >> > >> I have the same requirements, ie several datasources. I also used SolrJ > >> and > >> jsp for this. The way I ended up doing it was to create a multi core > >> environment, one core per datasource. When I do a query across several > >> datasources, I use shards. Solr automatically returns a "hybrid" result > >> set > >> that way, sorted by solr's default scoring. > >> > >> Faceting comes in the picture when you want to show the number of > >> documents > >> per datasource and have the ability to narrow down the result set. The > >> way > >> I > >> did it was to add a field called "dataSource" to all the documents, and > >> injected them with a default value of the data source name (in your > case, > >> D1, D2 ...). You can do this by adding this in the schema: > >> > >> >> required="true" default="D1"/> > >> > >> When you perform a query across multiple datasources, you will use > >> shards. > >> Here is an example: > >> > >> > >> > http://localhost:8080/solr/core1/select?shards=localhost:8080/solr/core1,localhost:8080/solr/core2&q=some > >> query > >> > >> That will search on both cores 1 and 2. > >> > >> To facet on the datasource in order to be able to categorize the result > >> set, > >> you can simply add this snippet to the query: > >> > >> &facet=on&facet.field=dataSource > >> > >> This will return the datasources that are defined with their number of > >> results for the query. > >> > >> Making the facet results clickable in order to narrow down the results > >> can > >> be achieved by adding a filter to the query and filtering to a specific > >> dataSource. I actually ended uo creating a fairly intuitive front-end > for > >> my > >> system with faceting, filtering, paging etc all using jsp and SolrJ. > >> SolrJ > >> is powerful enough to handle all of the backend processing. > >> > >> Good luck! > >> > >> > >> > >> > >> > >> > >> joe_coder wrote: > >> > > >> > I missed adding some size related information in the query above. > >> > > >> > D1 and D2 would have
Confusion around Binary/XML in SolrJ
I am using solr 1.4 dev in a multicore way. Each of my core's solrconfig.xml has the following lines I am using SolrJ as EmbeddedSolrServer. When I try to add a POJO ( with @Field annotations ), the data does not get indexed. Where as, if I use SolrInputDocument way, the data gets indexed. PS: Both ways I am adding data using addBean/add and then commit followed by optimize PPS: The final intention is that all the indexing and searching needs to be done in the binary format since I am running on a single machine. Could someone provide insights on this issue ? Thanks!
Re: Confusion around Binary/XML in SolrJ
Another observation:
I am even unable to delete documents using the EmbeddedSolrServer ( on a
specific core )
Steps:
1) I have 2 cores ( core0 , core1 ) Each of them have ~10 records.
2) System.setProperty("solr.solr.home",
"/home/user/projects/solr/example/multi");
File home = new File("/home/user/projects/solr/example/multi");
File f = new File(home, "solr.xml");
CoreContainer coreContainer = new CoreContainer();
coreContainer.load("/home/user/projects/solr/example/multi", f);
SolrServer server = new EmbeddedSolrServer(coreContainer, "core1");
server.deleteByQuery("*:*");
server.commit();
server.optimize();
3) When I check the status using
http://localhost:8983/solr/admin/cores?action=STATUS , I still see same
number of numDocs.
4) If I try deleting using CommonsHttpSolrServer, it works fine
String url = "http://localhost:8983/solr/core1";;
CommonsHttpSolrServer server = new CommonsHttpSolrServer(url);
server.setSoTimeout(1000); // socket read timeout
server.setConnectionTimeout(100);
server.setDefaultMaxConnectionsPerHost(100);
server.setMaxTotalConnections(100);
server.setFollowRedirects(false); // defaults to false
server.setAllowCompression(true);
server.setMaxRetries(1); // defaults to 0. > 1 not recommended.
server.setRequestWriter(new BinaryRequestWriter());
server.deleteByQuery("*:*");
server.commit();
server.optimize();
Thanks!
On Mon, Jul 20, 2009 at 3:26 PM, Code Tester <
codetester.codetes...@gmail.com> wrote:
> I am using solr 1.4 dev in a multicore way. Each of my core's
> solrconfig.xml has the following lines
>
>
> class="solr.BinaryUpdateRequestHandler" />
>
> I am using SolrJ as EmbeddedSolrServer. When I try to add a POJO ( with
> @Field annotations ), the data does not get indexed. Where as, if I use
> SolrInputDocument way, the data gets indexed.
>
> PS: Both ways I am adding data using addBean/add and then commit followed
> by optimize
>
> PPS: The final intention is that all the indexing and searching needs to be
> done in the binary format since I am running on a single machine.
>
> Could someone provide insights on this issue ?
>
> Thanks!
>
>
>
>
Re: Confusion around Binary/XML in SolrJ
Sorry everyone. Found the issue. It was because of a very stupid assumption.
My code and solr were running as 2 different processes! ( Weird part is that
when I run the code using EmbeddedSolrServer, it did not throw any exception
that there was already a server running on that port. )
Thanks!
On Mon, Jul 20, 2009 at 3:41 PM, Code Tester <
codetester.codetes...@gmail.com> wrote:
> Another observation:
>
> I am even unable to delete documents using the EmbeddedSolrServer ( on a
> specific core )
>
> Steps:
>
> 1) I have 2 cores ( core0 , core1 ) Each of them have ~10 records.
>
> 2) System.setProperty("solr.solr.home",
> "/home/user/projects/solr/example/multi");
> File home = new File("/home/user/projects/solr/example/multi");
> File f = new File(home, "solr.xml");
> CoreContainer coreContainer = new CoreContainer();
> coreContainer.load("/home/user/projects/solr/example/multi", f);
> SolrServer server = new EmbeddedSolrServer(coreContainer, "core1");
>
> server.deleteByQuery("*:*");
> server.commit();
> server.optimize();
>
> 3) When I check the status using
> http://localhost:8983/solr/admin/cores?action=STATUS , I still see same
> number of numDocs.
>
> 4) If I try deleting using CommonsHttpSolrServer, it works fine
> String url = "http://localhost:8983/solr/core1";;
> CommonsHttpSolrServer server = new CommonsHttpSolrServer(url);
> server.setSoTimeout(1000); // socket read timeout
> server.setConnectionTimeout(100);
> server.setDefaultMaxConnectionsPerHost(100);
> server.setMaxTotalConnections(100);
> server.setFollowRedirects(false); // defaults to false
> server.setAllowCompression(true);
> server.setMaxRetries(1); // defaults to 0. > 1 not recommended.
> server.setRequestWriter(new BinaryRequestWriter());
>
> server.deleteByQuery("*:*");
> server.commit();
> server.optimize();
>
> Thanks!
>
>
> On Mon, Jul 20, 2009 at 3:26 PM, Code Tester <
> codetester.codetes...@gmail.com> wrote:
>
>> I am using solr 1.4 dev in a multicore way. Each of my core's
>> solrconfig.xml has the following lines
>>
>>
>> > class="solr.BinaryUpdateRequestHandler" />
>>
>> I am using SolrJ as EmbeddedSolrServer. When I try to add a POJO ( with
>> @Field annotations ), the data does not get indexed. Where as, if I use
>> SolrInputDocument way, the data gets indexed.
>>
>> PS: Both ways I am adding data using addBean/add and then commit followed
>> by optimize
>>
>> PPS: The final intention is that all the indexing and searching needs to
>> be done in the binary format since I am running on a single machine.
>>
>> Could someone provide insights on this issue ?
>>
>> Thanks!
>>
>>
>>
>>
>
>
