Using SolrCloud on Amazon EC2
Hi together, we currently plan to setup a project based on solr cloud and amazon webservices. Our main search application is deployed using aws opsworks which works out quite good. Since we also want to provision solr to ec2 i want to ask for experiences with the different deployment/provisioning tools. By now i see the following 3 approaches. 1. Using ludic solr scale tk to setup and maintain the cluster Who is using this in production and what are your experiences? 2. Implementing own chef cookbooks for aws opsworks to install solrcloud as a custom opsworks layer Did somebody do this allready? What are you experiences? Are there any cookbooks out, where we can contribute and reuse? 3. Implementing own chef cookbooks for aws opsworks to install solrcloud as a docker container Any experiences with this? Do you see other options? Afaik elasticbeanstalk could also be an option? It would be very nice to get some experiences and recommendations? Cheers Timo
AW: grouping finds
Hi Giovanni, afaik grouping is not completly working with solr cloud. You maybe could check: https://issues.apache.org/jira/browse/SOLR-5046 In addition, documents that should be grouped, need to be in the same shard (You can use &router.field=IDCat3 to place all of your documents with the same IDCat3 in the same shard). Maybe someboy else can give some more insight's i am also interested into the topic. Cheers Timo Von: Giovanni Bricconi [giovanni.bricc...@banzai.it] Gesendet: Donnerstag, 6. November 2014 11:43 An: solr-user Betreff: grouping finds 297254 49 0 ... 1204312043SSD498
Aw: is group.query supported in solrcloud (4.8) ?
Hi Giovanni, i think there is a bug in solr (cloud only) https://issues.apache.org/jira/browse/SOLR-5046 i've prepared a patch, that needs some review and improvements (my solr core knowledge is limited here :)) Maybe you can try the patch and vote for the issue in jira. What i know by now: when you can use group.field=dd instead of group.query it works. And also it seems that this is an issue when you have one shard not matching your docst Cheers Timo Gesendet: Montag, 10. November 2014 um 12:41 Uhr Von: "Giovanni Bricconi" An: solr-user Betreff: is group.query supported in solrcloud (4.8) ? hello I have a collection 0_2014_10_11 made of three shards When I try a group.query, even specifying a single shard, i get this error "shard 0 did not set sort field values (FieldDoc.fields is null); you must pass fillFields=true to IndexSearcher.search on each shard" This is the request, ask the collection to find groups on IDCat3=922 http://src-dev-1:8080/solr/0_2014_10_11/select?q=*:*&group=true&group.query=IDCat3%3A922&shards=src-dev-1:8080/solr/0_2014_10_11_shard1_replica2 According to this page [ https://cwiki.apache.org/confluence/display/solr/Result+Grouping ] the group.query is supported. Am I missing some key parameter? Should the shards parameter be really mandatory? It seems that with group.field it is not required. Thanks Giovanni
AW: is group.query supported in solrcloud (4.8) ?
Hi Giovanni, i think there is a bug in solr (cloud only) https://issues.apache.org/jira/browse/SOLR-5046 i've prepared a patch, that needs some review and improvements (my solr core knowledge is limited here :)) Maybe you can try the patch and vote for the issue in jira. What i know by now: when you can use group.field=dd instead of group.query it works. And also it seems that this is an issue when you have one shard not matching your docst Cheers Timo Von: Giovanni Bricconi [giovanni.bricc...@banzai.it] Gesendet: Montag, 10. November 2014 12:41 An: solr-user Betreff: is group.query supported in solrcloud (4.8) ? hello I have a collection 0_2014_10_11 made of three shards When I try a group.query, even specifying a single shard, i get this error "shard 0 did not set sort field values (FieldDoc.fields is null); you must pass fillFields=true to IndexSearcher.search on each shard" This is the request, ask the collection to find groups on IDCat3=922 http://src-dev-1:8080/solr/0_2014_10_11/select?q=*:*&group=true&group.query=IDCat3%3A922&shards=src-dev-1:8080/solr/0_2014_10_11_shard1_replica2 According to this page [ https://cwiki.apache.org/confluence/display/solr/Result+Grouping ] the group.query is supported. Am I missing some key parameter? Should the shards parameter be really mandatory? It seems that with group.field it is not required. Thanks Giovanni
Set spellcheck field on query time?
Hello together, we are currently working on a mutilanguage single core setup. During that I stumbled upon the question if it is possible to define different sources for the spellcheck. For now I only see the possibility to define different request handlers. Is it somehow possible to set the source field for the DirectSolrSpellChecker on querytime? Cheers timo [cid:image001.jpg@01CE764E.E6958B90] Timo Schmidt Entwickler (Dipl. Inf. FH) AOE GmbH Borsigstr. 3 65205 Wiesbaden Germany Tel. +49 (0) 6122 70 70 7 - 234 Fax. +49 (0) 6122 70 70 7 -199 e-Mail: timo.schm...@aoemedia.de<mailto:timo.schm...@aoemedia.de> Web: http://www.aoemedia.de/ Pflichtangaben laut Handelsgesetz §37a / Aktiengesetz §35a USt-ID Nr.: DE250247455 Handelsregister: Wiesbaden B Handelsregister Nr.: 22567 Stammsitz: Wiesbaden Creditreform: 625.0209354 Geschäftsführer: Kian Toyouri Gould Diese E-Mail Nachricht enthält vertrauliche und/oder rechtlich geschützte Informationen. Wenn Sie nicht der richtige Adressat sind oder diese E-Mail irrtümlich erhalten haben, informieren Sie bitte sofort den Absender und vernichten Sie diese Mail. This e-mail message may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and destroy this e-mail.
Support of field variants in solr
Hi together, i am timo and work for a solr implementation company. During the last projects we came to know that we need to be able to generate different variants of a document. Example 1 (Language): To handle all documents in one solr core, we need a field variant for each language. content for spanish content content for german content Each of these fields can be configured in the solr schema to act optimal for the specific taget language. Example 2 (Stores): We have customers who want to sell the same product in different stores for different prices. price in frankfurt price in paris To solve this in an optimal way it would be nice when this works complely transparent inside solr by definig a „variantQuery“ A select query could look like this: select?variantQuery=fr&qf=price,content Additional the following is possible. No variant is present, behavious should be as before, so it should be relevant for all queries. The setting variant=“*“ would mean: There can be several wildcard variant defined in a commited document. This makes sence when the data type would be the same for all variants and you will have many variants (like in the price example). The same as during query time should be possible during indexing time. I know, that we can do somthing like this also with dynamic fields but then we need to resolve the concrete fields during index and querytime on the application level, what is possible but it would be nicer to have a concept like this in solr, also working with facets is easier with this approach when the concrete fieldname does not need to be populated in the application. So my questions are: What do you think about this approach? Is it better to work with dynamic fields? Is it reasonable when you have 200 variants or more of a document? What needs to be done in solr to have something like this variant attribute for fields? Do you have other approaches?
Aw: Re: Support of field variants in solr
Ok, thanks for this hint i have two further questions to understand it completly. Settingup custom request handler makes it easier to avoid all the mapping parameters in the query but it would also be possible with one request handler and all mapping in the request arguments right? What about indexing, i there also a mechanism like this or should the application deside with target field to use? Gesendet: Dienstag, 23. April 2013 um 02:32 Uhr Von: "Alexandre Rafalovitch" An: solr-user@lucene.apache.org Betreff: Re: Support of field variants in solr To route different languages, you could use different request handlers and do different alias mapping. There are two alias mapping: On the way in for eDisMax: https://wiki.apache.org/solr/ExtendedDisMax#Field_aliasing_.2BAC8_renaming On the way out: https://wiki.apache.org/solr/CommonQueryParameters#Field_alias[https://wiki.apache.org/solr/CommonQueryParameters#Field_alias] Between the two, you can make sure that all searches to /searchES map 'content' field to 'content_es' and for /searchDE map 'content' to 'content_de'. Hope this helps, Alex. Personal blog: http://blog.outerthoughts.com/[http://blog.outerthoughts.com/] LinkedIn: http://www.linkedin.com/in/alexandrerafalovitch[http://www.linkedin.com/in/alexandrerafalovitch] - Time is the quality of nature that keeps events from happening all at once. Lately, it doesn't seem to be working. (Anonymous - via GTD book) On Mon, Apr 22, 2013 at 2:31 PM, Timo Schmidt wrote: > Hi together, > > i am timo and work for a solr implementation company. During the last > projects we came to know that we need to be able to generate different > variants of a document. > > Example 1 (Language): > > To handle all documents in one solr core, we need a field variant for each > language. > > > content for spanish content > > variant=“es“ /> > > content for german content > > variant=“de“ /> > > > Each of these fields can be configured in the solr schema to act optimal for > the specific taget language. > > Example 2 (Stores): > > We have customers who want to sell the same product in different stores for > different prices. > > > price in frankfurt > > > > price in paris > > > > To solve this in an optimal way it would be nice when this works complely > transparent inside solr by definig a „variantQuery“ > > A select query could look like this: > > select?variantQuery=fr&qf=price,content > > Additional the following is possible. No variant is present, behavious should > be as before, so it should be relevant for all queries. > > The setting variant=“*“ would mean: There can be several wildcard variant > defined in a commited document. This makes sence when the data type would be > the same for all variants and you will have many variants (like in the price > example). > > The same as during query time should be possible during indexing time. > > I know, that we can do somthing like this also with dynamic fields but then > we need to resolve the concrete fields during index and querytime on the > application level, what is possible but it would be nicer to have a concept > like this in solr, also working with facets is easier with this approach when > the concrete fieldname does not need to be populated in the application. > > So my questions are: > > What do you think about this approach? > Is it better to work with dynamic fields? Is it reasonable when you have 200 > variants or more of a document? > What needs to be done in solr to have something like this variant attribute > for fields? > Do you have other approaches?
Maintain stopwords.txt and synonyms.txt
Hello together, i am currently developing a search solution, based on Apache Solr. Currently I have the problem that I want to offer the user the possibility to maintain synonyms and stopwords in a userfriendy tool. But currently I could not find any possibility to write the stopwords.txt or synonyms.txt. Are there any other solutions? Currently I have some ideas how to handle it: 1.Implement another SynonymFilterFactory to allow other datasources like databases. I already saw approaches for that but no solutions yet. 2.Implement a fileWriter request handler to write the stopwords.txt Are there other solutions which are maybe already implemented? Thanks and best regards Timo Timo Schmidt Entwickler (Diplom Informatiker FH) AOE media GmbH Borsigstr. 3 65205 Wiesbaden Germany Tel. +49 (0) 6122 70 70 7 - 234 Fax. +49 (0) 6122 70 70 7 -199 e-Mail: timo.schm...@aoemedia.de Web: http://www.aoemedia.de/ Pflichtangaben laut Handelsgesetz §37a / Aktiengesetz §35a USt-ID Nr.: DE250247455 Handelsregister: Wiesbaden B Handelsregister Nr.: 22567 Stammsitz: Wiesbaden Creditreform: 625.0209354 Geschäftsführer: Kian Toyouri Gould Diese E-Mail Nachricht enthält vertrauliche und/oder rechtlich geschützte Informationen. Wenn Sie nicht der richtige Adressat sind oder diese E-Mail irrtümlich erhalten haben, informieren Sie bitte sofort den Absender und vernichten Sie diese Mail. This e-mail message may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and destroy this e-mail.
AW: Maintain stopwords.txt and synonyms.txt
Hi Stefan, i allready thought about that. Maybe some php service or something like that. But this would mean, that I need additional software on that server like a normal Apache installation, which needs to be maintained. That's why I thought a solution that is build into solr would be nice. Thanks Timo Schmidt Entwickler (Diplom Informatiker FH) AOE media GmbH Borsigstr. 3 65205 Wiesbaden Germany Tel. +49 (0) 6122 70 70 7 - 234 Fax. +49 (0) 6122 70 70 7 -199 e-Mail: timo.schm...@aoemedia.de Web: http://www.aoemedia.de/ Pflichtangaben laut Handelsgesetz §37a / Aktiengesetz §35a USt-ID Nr.: DE250247455 Handelsregister: Wiesbaden B Handelsregister Nr.: 22567 Stammsitz: Wiesbaden Creditreform: 625.0209354 Geschäftsführer: Kian Toyouri Gould -Ursprüngliche Nachricht- Von: Stefan Matheis [mailto:matheis.ste...@googlemail.com] Gesendet: Mittwoch, 9. Februar 2011 11:14 An: solr-user@lucene.apache.org Betreff: Re: Maintain stopwords.txt and synonyms.txt Timo, On Wed, Feb 9, 2011 at 11:07 AM, Timo Schmidt wrote: > But currently I could not find any possibility to write the stopwords.txt or > synonyms.txt. what about writing the Files from an external Application and reload your Solr Core!? Seemed to be the simplest way to solve your problem, not? Regards Stefan
AW: Maintain stopwords.txt and synonyms.txt
Yes we have something, but on another machine. Timo Schmidt Entwickler (Diplom Informatiker FH) AOE media GmbH Borsigstr. 3 65205 Wiesbaden Germany Tel. +49 (0) 6122 70 70 7 - 234 Fax. +49 (0) 6122 70 70 7 -199 e-Mail: timo.schm...@aoemedia.de Web: http://www.aoemedia.de/ Pflichtangaben laut Handelsgesetz §37a / Aktiengesetz §35a USt-ID Nr.: DE250247455 Handelsregister: Wiesbaden B Handelsregister Nr.: 22567 Stammsitz: Wiesbaden Creditreform: 625.0209354 Geschäftsführer: Kian Toyouri Gould -Ursprüngliche Nachricht- Von: Stefan Matheis [mailto:matheis.ste...@googlemail.com] Gesendet: Mittwoch, 9. Februar 2011 11:34 An: solr-user@lucene.apache.org Betreff: Re: Maintain stopwords.txt and synonyms.txt Hi Timo, of course - that's right. Write some JSP (i guess) which could be integrated in the already existing jetty/tomcat Server? Just wondering about, how do you perform Search-Requests to Solr? Normally, there is already any other Service running, which acts as 'proxy' to the outer world? ;) Regards Stefan On Wed, Feb 9, 2011 at 11:20 AM, Timo Schmidt wrote: > Hi Stefan, > > i allready thought about that. Maybe some php service or something like that. > But this would mean, that I need additional software on that server like a > normal > Apache installation, which needs to be maintained. That's why I thought a > solution that > is build into solr would be nice. > > Thanks > > Timo Schmidt > Entwickler (Diplom Informatiker FH) > > > AOE media GmbH > Borsigstr. 3 > 65205 Wiesbaden > Germany > Tel. +49 (0) 6122 70 70 7 - 234 > Fax. +49 (0) 6122 70 70 7 -199 > > > > e-Mail: timo.schm...@aoemedia.de > Web: http://www.aoemedia.de/ > > Pflichtangaben laut Handelsgesetz §37a / Aktiengesetz §35a > USt-ID Nr.: DE250247455 > Handelsregister: Wiesbaden B > Handelsregister Nr.: 22567 > Stammsitz: Wiesbaden > Creditreform: 625.0209354 > Geschäftsführer: Kian Toyouri Gould > > > -Ursprüngliche Nachricht- > Von: Stefan Matheis [mailto:matheis.ste...@googlemail.com] > Gesendet: Mittwoch, 9. Februar 2011 11:14 > An: solr-user@lucene.apache.org > Betreff: Re: Maintain stopwords.txt and synonyms.txt > > Timo, > > On Wed, Feb 9, 2011 at 11:07 AM, Timo Schmidt > wrote: >> But currently I could not find any possibility to write the stopwords.txt or >> synonyms.txt. > > what about writing the Files from an external Application and reload > your Solr Core!? > Seemed to be the simplest way to solve your problem, not? > > Regards > Stefan >
