List of recommendation engines with solr
Hi All, I would require list of recs engine which can be integrated with solr and also suggest best one out of this. any comments would be appriciated!! Thanks, Rohan -- View this message in context: http://lucene.472066.n3.nabble.com/List-of-recommendation-engines-with-solr-tp3818917p3818917.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: How to index doc file in solr?
Hi Erick, Thanks for the valuable comments on this. See i have few set of word docs file and i would like to index meta data part includeing the content of the page , so is there any way to complete this task? Need your comments on this. Thanks, Rohan -- View this message in context: http://lucene.472066.n3.nabble.com/How-to-index-doc-file-in-solr-tp3806543p3818938.html Sent from the Solr - User mailing list archive at Nabble.com.
RE: List of recommendation engines with solr
Hi Gora, Thanks a lot for your valuable comments, really appreciated. Yeah , You got me correctly I am exactly looking for "Mahout" as I am using Java as my business layer with Apache solr. Thanks, Rohan From: Gora Mohanty-3 [via Lucene] [mailto:ml-node+s472066n3819480...@n3.nabble.com] Sent: Monday, March 12, 2012 8:28 PM To: Rohan Ashok Kumbhar Subject: Re: List of recommendation engines with solr On 12 March 2012 16:30, Rohan <[hidden email]> wrote: > Hi All, > > I would require list of recs engine which can be integrated with solr and > also suggest best one out of this. > > any comments would be appriciated!! What exactly do you mean by that? Why is integration with Solr a requirement, and what do you expect to gain by such an integration? "Best" also probably depends on the context of your requirements. There are a variety of open-source recommendation engines. If you are looking at something from Apache, and in Java, Mahout might be a good choice. Regards, Gora If you reply to this email, your message will be added to the discussion below: http://lucene.472066.n3.nabble.com/List-of-recommendation-engines-with-solr-tp3818917p3819480.html To unsubscribe from List of recommendation engines with solr, click here<http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=3818917&code=Um9oYW5fS3VtYmhhckBpbmZvc3lzLmNvbXwzODE4OTE3fC0xMjUwNDUyNDI1>. NAML<http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml> CAUTION - Disclaimer * This e-mail contains PRIVILEGED AND CONFIDENTIAL INFORMATION intended solely for the use of the addressee(s). If you are not the intended recipient, please notify the sender by e-mail and delete the original message. Further, you are not to copy, disclose, or distribute this e-mail or its contents to any other person and any such actions are unlawful. This e-mail may contain viruses. Infosys has taken every reasonable precaution to minimize this risk, but is not liable for any damage you may sustain as a result of any virus in this e-mail. You should carry out your own virus checks before opening the e-mail or attachment. Infosys reserves the right to monitor and review the content of all messages sent to or from this e-mail address. Messages sent to or from this e-mail address may be stored on the Infosys e-mail system. ***INFOSYS End of Disclaimer INFOSYS*** -- View this message in context: http://lucene.472066.n3.nabble.com/List-of-recommendation-engines-with-solr-tp3818917p3821268.html Sent from the Solr - User mailing list archive at Nabble.com.
RE: How to index doc file in solr?
Thanks Erick ,really appreciated. From: Erick Erickson [via Lucene] [mailto:ml-node+s472066n3819585...@n3.nabble.com] Sent: Monday, March 12, 2012 9:05 PM To: Rohan Ashok Kumbhar Subject: Re: How to index doc file in solr? Consider using SolrJ, possibly combined with Tika (which is what underlies Solr Cel). http://www.lucidimagination.com/blog/2012/02/14/indexing-with-solrj/ AlthoughExtractingRequestHandler has the capability of indexing metadata as well if you map the fields. See: http://wiki.apache.org/solr/ExtractingRequestHandler Best Erick On Mon, Mar 12, 2012 at 11:09 AM, Rohan <[hidden email]> wrote: > Hi Erick, > > Thanks for the valuable comments on this. > > See i have few set of word docs file and i would like to index meta data > part includeing the content of the page , so is there any way to complete > this task? > > Need your comments on this. > > Thanks, > Rohan > > -- > View this message in context: > http://lucene.472066.n3.nabble.com/How-to-index-doc-file-in-solr-tp3806543p3818938.html > Sent from the Solr - User mailing list archive at Nabble.com. If you reply to this email, your message will be added to the discussion below: http://lucene.472066.n3.nabble.com/How-to-index-doc-file-in-solr-tp3806543p3819585.html To unsubscribe from How to index doc file in solr?, click here<http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=3806543&code=Um9oYW5fS3VtYmhhckBpbmZvc3lzLmNvbXwzODA2NTQzfC0xMjUwNDUyNDI1>. NAML<http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml> CAUTION - Disclaimer * This e-mail contains PRIVILEGED AND CONFIDENTIAL INFORMATION intended solely for the use of the addressee(s). If you are not the intended recipient, please notify the sender by e-mail and delete the original message. Further, you are not to copy, disclose, or distribute this e-mail or its contents to any other person and any such actions are unlawful. This e-mail may contain viruses. Infosys has taken every reasonable precaution to minimize this risk, but is not liable for any damage you may sustain as a result of any virus in this e-mail. You should carry out your own virus checks before opening the e-mail or attachment. Infosys reserves the right to monitor and review the content of all messages sent to or from this e-mail address. Messages sent to or from this e-mail address may be stored on the Infosys e-mail system. ***INFOSYS End of Disclaimer INFOSYS*** -- View this message in context: http://lucene.472066.n3.nabble.com/How-to-index-doc-file-in-solr-tp3806543p3821271.html Sent from the Solr - User mailing list archive at Nabble.com.
RE: List of recommendation engines with solr
HI , Sorry , I have no idea as I never worked on this . Thanks, Rohan From: Trev [via Lucene] [mailto:ml-node+s472066n3985922...@n3.nabble.com] Sent: Thursday, May 24, 2012 7:37 PM To: Rohan Ashok Kumbhar Subject: Re: List of recommendation engines with solr Have you heard of NG Data with their product called Lily? If you reply to this email, your message will be added to the discussion below: http://lucene.472066.n3.nabble.com/List-of-recommendation-engines-with-solr-tp3818917p3985922.html To unsubscribe from List of recommendation engines with solr, click here<http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=3818917&code=Um9oYW5fS3VtYmhhckBpbmZvc3lzLmNvbXwzODE4OTE3fC0xMjUwNDUyNDI1>. NAML<http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml> CAUTION - Disclaimer * This e-mail contains PRIVILEGED AND CONFIDENTIAL INFORMATION intended solely for the use of the addressee(s). If you are not the intended recipient, please notify the sender by e-mail and delete the original message. Further, you are not to copy, disclose, or distribute this e-mail or its contents to any other person and any such actions are unlawful. This e-mail may contain viruses. Infosys has taken every reasonable precaution to minimize this risk, but is not liable for any damage you may sustain as a result of any virus in this e-mail. You should carry out your own virus checks before opening the e-mail or attachment. Infosys reserves the right to monitor and review the content of all messages sent to or from this e-mail address. Messages sent to or from this e-mail address may be stored on the Infosys e-mail system. ***INFOSYS End of Disclaimer INFOSYS*** -- View this message in context: http://lucene.472066.n3.nabble.com/List-of-recommendation-engines-with-solr-tp3818917p3985927.html Sent from the Solr - User mailing list archive at Nabble.com.
java.io.IOException: read past EOF
Hi Guys, This is my first post. We are running solr with multiple Indexes, 20 Indexes. I'm facing problem with 5 one. I'm not able to run optimized on that index. I'm getting following error. Your help is really appreciated. java.io.IOException: read past EOF at org.apache.lucene.store.BufferedIndexInput.refill(BufferedIndexInput.java:89 ) at org.apache.lucene.store.BufferedIndexInput.readByte(BufferedIndexInput.java: 34) at org.apache.lucene.store.IndexInput.readVInt(IndexInput.java:57) at org.apache.lucene.index.SegmentTermPositions.readDeltaPosition(SegmentTermPo sitions.java:70) at org.apache.lucene.index.SegmentTermPositions.nextPosition(SegmentTermPositio ns.java:66) at org.apache.lucene.index.SegmentMerger.appendPostings(SegmentMerger.java:388) at org.apache.lucene.index.SegmentMerger.mergeTermInfo(SegmentMerger.java:320) at org.apache.lucene.index.SegmentMerger.mergeTermInfos(SegmentMerger.java:292) at org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:256) at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:97) at org.apache.lucene.index.IndexWriter.mergeSegments(IndexWriter.java:1835) at org.apache.lucene.index.IndexWriter.optimize(IndexWriter.java:1195) at org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.java :508) at org.apache.solr.handler.XmlUpdateRequestHandler.update(XmlUpdateRequestHandl er.java:214) at org.apache.solr.handler.XmlUpdateRequestHandler.handleRequestBody(XmlUpdateR equestHandler.java:84) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase. java:77) at org.apache.solr.core.SolrCore.execute(SolrCore.java:658) at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:1 91) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java: 159) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Application FilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterCh ain.java:206) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.ja va:233) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.ja va:175) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128 ) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102 ) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java :109) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:263) at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:844) at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http 11Protocol.java:584) at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:447) at java.lang.Thread.run(Thread.java:619) type Status reportmessage read past EOF java.io.IOException: read past EOF at org.apache.lucene.store.BufferedIndexInput.refill(BufferedIndexInput.java:89 ) at org.apache.lucene.store.BufferedIndexInput.readByte(BufferedIndexInput.java: 34) at org.apache.lucene.store.IndexInput.readVInt(IndexInput.java:57) at org.apache.lucene.index.SegmentTermPositions.readDeltaPosition(SegmentTermPo sitions.java:70) at org.apache.lucene.index.SegmentTermPositions.nextPosition(SegmentTermPositio ns.java:66) at org.apache.lucene.index.SegmentMerger.appendPostings(SegmentMerger.java:388) at org.apache.lucene.index.SegmentMerger.mergeTermInfo(SegmentMerger.java:320) at org.apache.lucene.index.SegmentMerger.mergeTermInfos(SegmentMerger.java:292) at org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:256) at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:97) at org.apache.lucene.index.IndexWriter.mergeSegments(IndexWriter.java:1835) at org.apache.lucene.index.IndexWriter.optimize(IndexWriter.java:1195) at org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.java :508) at org.apache.solr.handler.XmlUpdateRequestHandler.update(XmlUpdateRequestHandl er.java:214) at org.apache.solr.handler.XmlUpdateRequestHandler.handleRequestBody(XmlUpdateR equestHandler.java:84) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase. java:77) at org.apache.solr.core.SolrCore.execute(SolrCore.java:658) at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:1 91) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java: 159) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Application FilterChain.java:235) at org.apache.catalina.core.Applicatio
Fwd: Mobiles/Tablets for Repair
Hey there, Currently, I am part of a company* ZurePro Warranty* which deals with providing warranties over the mobiles and tablets. If you have any such product which needs to be repaired (only mobiles & tablets) you can get in touch with me. ZurePro will arrange a free pick and drop for the gadget, you will only have to bear the repair cost which will be the market price of the repairing. We are looking for faulty smartphones and tablets with any kind of hardware/software problems like motherboard issue, power port malfunction, screen/touch pad problem etc.. If you have any such product kindly let me know, ZurePro will take care of it. :) Cheers!! -- Best Regards, Rohan Thakur
help regarding custom query which returns custom output
hi all I have requirement like I have implemented fulltext search and autosuggestion and spellcorrection functionality in solr but they all are running on different cores so I have to call 3 different request handlers for getting the results which is adding the unnecessary delay so I wanted to know is there any solution that I call just one request URL and get all these three results and json feedback from solr. thanx regards rohan
query regarding running solr4.1.0 on tomcat6
hi all I had installed tomcat6 on centos redhat linux os and had configured solr with name on solrt on tomcat and It was running fine now what I did was placed another copy of solr home folder in centos and changed the tomcat directory to this new solr and now every thing is working fine like the full database import and all from the browser and query from browser but when I open the solr-example/admin(the default solr admin panel ) from browser it shows the error that : http://localhost:8080/solr-example/#/ HTTP Status 404 - -- *type* Status report *message* *description* *The requested resource () is not available.* -- Apache Tomcat/6.0.24 and other wise when I hit http://localhost:8080/solr-example/collection1/select?q=samsung%20duos&wt=json&indent=true&rows=20 its running fine and even if i hit http://localhost:8080/solr-example/dataimport?command=full-import&indent=true&clean=true its running fine and even in the tomcat manager panal I can see solr-example and when I click on it shows the same error. 404 what could be the problem with the solr admin panel help anyone. thanks regards rohan
Re: had query regarding the indexing and analysers
hi does this means that while indexing also ace is been stored as ac in solr index? thanks regards Rohan On Fri, Mar 22, 2013 at 9:49 AM, Jack Krupansky wrote: > Actually, it's the Porter Stemmer that is turning "ace" into "ac". > > Try making a copy of text_en_splitting and delete the > PorterStemFilterFactory filter from both the query and index analyzers. > > > -- Jack Krupansky > > -Original Message- From: Rohan Thakur > Sent: Wednesday, March 20, 2013 8:39 AM > > To: solr-user@lucene.apache.org > Subject: Re: had query regarding the indexing and analysers > > hi jack > > I have been using text_en_splitting initially but what it was doing is it > is changing by query aswell > for example: > if i am searching for "ace" term it is taking it as "ac" thus giving split > ac higher score... > see debug statment: > > "debug":{ >"rawquerystring":"ace", >"querystring":"ace", >"parsedquery":"(+**DisjunctionMaxQuery((title:ac^**30.0)))/no_coord", >"parsedquery_toString":"+(**title:ac^30.0)", >"explain":{ > "":"\n1.8650155 = (MATCH) weight(title:ac^30.0 in 469) > [DefaultSimilarity], result of:\n 1.8650155 = fieldWeight in 469, > product of:\n1.0 = tf(freq=1.0), with freq of:\n 1.0 = > termFreq=1.0\n4.2628927 = idf(docFreq=39, maxDocs=1045)\n > 0.4375 = fieldNorm(doc=469)\n", > "":"\n1.8650155 = (MATCH) weight(title:ac^30.0 in 470) > [DefaultSimilarity], result of:\n 1.8650155 = fieldWeight in 470, > product of:\n1.0 = tf(freq=1.0), with freq of:\n 1.0 = > termFreq=1.0\n4.2628927 = idf(docFreq=39, maxDocs=1045)\n > 0.4375 = fieldNorm(doc=470)\n", > "":"\n1.8650155 = (MATCH) weight(title:ac^30.0 in 471) > [DefaultSimilarity], result of:\n 1.8650155 = fieldWeight in 471, > product of:\n1.0 = tf(freq=1.0), with freq of:\n 1.0 = > termFreq=1.0\n4.2628927 = idf(docFreq=39, maxDocs=1045)\n > 0.4375 = fieldNorm(doc=471)\n", > "":"\n1.8650155 = (MATCH) weight(title:ac^30.0 in 472) > [DefaultSimilarity], result of:\n 1.8650155 = fieldWeight in 472, > product of:\n1.0 = tf(freq=1.0), with freq of:\n 1.0 = > termFreq=1.0\n4.2628927 = idf(docFreq=39, maxDocs=1045)\n > 0.4375 = fieldNorm(doc=472)\n", > "":"\n1.5985848 = (MATCH) weight(title:ac^30.0 in 331) > [DefaultSimilarity], result of:\n 1.5985848 = fieldWeight in 331, > product of:\n1.0 = tf(freq=1.0), with freq of:\n 1.0 = > termFreq=1.0\n4.2628927 = idf(docFreq=39, maxDocs=1045)\n0.375 > = fieldNorm(doc=331)\n", > "":"\n1.5985848 = (MATCH) weight(title:ac^30.0 in 332) > [DefaultSimilarity], result of:\n 1.5985848 = fieldWeight in 332, > product of:\n1.0 = tf(freq=1.0), with freq of:\n 1.0 = > termFreq=1.0\n4.2628927 = idf(docFreq=39, maxDocs=1045)\n0.375 > = fieldNorm(doc=332)\n", > "":"\n1.5985848 = (MATCH) weight(title:ac^30.0 in 335) > [DefaultSimilarity], result of:\n 1.5985848 = fieldWeight in 335, > product of:\n1.0 = tf(freq=1.0), with freq of:\n 1.0 = > termFreq=1.0\n4.2628927 = idf(docFreq=39, maxDocs=1045)\n0.375 > = fieldNorm(doc=335)\n", > "":"\n1.5985848 = (MATCH) weight(title:ac^30.0 in 336) > [DefaultSimilarity], result of:\n 1.5985848 = fieldWeight in 336, > product of:\n1.0 = tf(freq=1.0), with freq of:\n 1.0 = > termFreq=1.0\n4.2628927 = idf(docFreq=39, maxDocs=1045)\n0.375 > = fieldNorm(doc=336)\n", > "":"\n1.5985848 = (MATCH) weight(title:ac^30.0 in 337) > [DefaultSimilarity], result of:\n 1.5985848 = fieldWeight in 337, > product of:\n1.0 = tf(freq=1.0), with freq of:\n 1.0 = > termFreq=1.0\n4.2628927 = idf(docFreq=39, maxDocs=1045)\n0.375 > = fieldNorm(doc=337)\n", > "":"\n1.5985848 = (MATCH) weight(title:ac^30.0 in 393) > [DefaultSimilarity], result of:\n 1.5985848 = fieldWeight in 393, > product of:\n1.0 = tf(freq=1.0), with freq of:\n 1.0 = > termFreq=1.0\n4.2628927 = idf(docFreq=39, maxDocs=1045)\n0.375 > = fieldNorm(doc=393)\n", > "":"\n1.5985848 = (MATCH) weight(title:ac^30.0 in 425) > [DefaultSimilarity], result of:\n 1.5985848 = fieldWeight in 425, > product of:\n1.0 = tf(freq=1.0), with freq of:\n 1.0 = > termFreq=1.0\n4.2628927 = idf(docFreq=39, maxDocs=1045)\n0.375 > = fieldNorm(doc=425)\n", > "":"\n1.5985848 = (MATCH) weight(title:ac^30.0 in
Re: is there a way we can build spell dictionary from solr index such that it only take words leaving all`special characters
hi upayavira you mean to say that I dont have to follow this : http://wiki.apache.org/solr/SpellCheckComponent and directly I can create spell check field from copyfield and use it...I dont have to build dictionary on the fieldjust use copyfield for spell suggetions? thanks regards Rohan On Wed, Mar 13, 2013 at 12:56 PM, Upayavira wrote: > Use text analysis and copyField to create a new field that has terms as > you expect them. Then use that for your spellcheck dictionary. > > Note, since 4.0, you don't need to create a dictionary. Solr can use > your index directly. > > Upayavira > > On Wed, Mar 13, 2013, at 06:00 AM, Rohan Thakur wrote: > > while building the spell dictionary... > > > > On Wed, Mar 13, 2013 at 11:29 AM, Rohan Thakur > > wrote: > > > > > even do not want to break the words as in samsung to s a m s u n g or > sII > > > ti s II ir s2 to s 2 > > > > > > On Wed, Mar 13, 2013 at 11:28 AM, Rohan Thakur >wrote: > > > > > >> k as in like if the field I am indixing from the database like title > that > > >> has characters like () - # /n// > > >> example: > > >> > > >> Screenguard for Samsung Galaxy SII (Matt and Gloss) (with Dual > Protection, Cleaning Cloth and Bubble Remover) > > >> > > >> or > > >> samsung-galaxy-sii-screenguard-matt-and-gloss.html > > >> or > > >> /s/a/samsung_galaxy_sii_i9100_pink_.jpg > > >> or > > >> 4.27-inch Touchscreen, 3G, Android v2.3 OS, 8MP Camera with LED Flash > > >> > > >> now I do not want to build the spell dictionary to only include the > words > > >> not any of the - , _ . ( ) /s/a/ or numeric like 4.27 > > >> how can I do that? > > >> > > >> thanks > > >> regards > > >> Rohan > > >> > > >> On Tue, Mar 12, 2013 at 11:06 PM, Alexandre Rafalovitch < > > >> arafa...@gmail.com> wrote: > > >> > > >>> Sorry, leaving them where? > > >>> > > >>> Can you give a concrete example or problem. > > >>> > > >>> Regards, > > >>> Alex > > >>> On Mar 12, 2013 1:31 PM, "Rohan Thakur" > wrote: > > >>> > > >>> > hi all > > >>> > > > >>> > wanted to know is there way we can make spell dictionary from solr > > >>> index > > >>> > such that it only takes words from the index leaving all the > special > > >>> > characters and unwanted characters. > > >>> > > > >>> > thanks > > >>> > regards > > >>> > Rohan > > >>> > > > >>> > > >> > > >> > > > >
how to avoid single character to get indexed for directspellchecker dictionary
hi all I am using solr directspellcheker for spell suggestions using raw analyses for indexing but I have some fields which have single characters like l L so its is been indexed in the dictionary and when I am using this for suggestions for query like delll its suggesting de and l l l as the spell correction as my index has de and l as single characters in the fields. please help. thanks regards Rohan
Re: how to avoid single character to get indexed for directspellchecker dictionary
hi james after using this its working file for delll but not for de. what does this minbreaklength signifies? also can you tell me why am I not getting suggestions for smaller words like for del i should get dell as suggestion but its not giving any suggestions and also can I get suggestion for like complete the sentence like if I give sams it should also give samsung as in suggestion? thanks regards Rohan On Fri, Apr 5, 2013 at 12:54 AM, Dyer, James wrote: > I assume if your user queries "delll" and it breaks it into pieces like > "de l l l", then you're probably using WordBreakSolrSpellChecker in > addition to DirectSolrSpellChecker, right? If so, then you can specify > "minBreakLength" in solrconfig.xml like this: > > > ... spellcheckers here ... > > wordbreak > solr.WordBreakSolrSpellChecker > ... parameters here ... > 5 > > > > One note is that both DirectSolrSpellChecker and WordBreakSolrSpellChecker > operate directly on the terms dictionary and do not have a separate > dictionary like IndexBasedSpellChecker. The only way to prevent a word > from being in the dictionary then is to filter this out in the analysis > chain. For instance, if you use to build a field just for > spellchecking, you can use LengthFilterFactory to remove the short terms. > See > http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters#solr.LengthFilterFactory. > > James Dyer > Ingram Content Group > (615) 213-4311 > > > -Original Message- > From: Rohan Thakur [mailto:rohan.i...@gmail.com] > Sent: Thursday, April 04, 2013 1:42 PM > To: solr-user@lucene.apache.org > Subject: how to avoid single character to get indexed for > directspellchecker dictionary > > hi all > > I am using solr directspellcheker for spell suggestions using raw analyses > for indexing but I have some fields which have single characters like l L > so its is been indexed in the dictionary and when I am using this for > suggestions for query like delll its suggesting de and l l l as the spell > correction as my index has de and l as single characters in the fields. > please help. > > thanks > regards > Rohan > >
Re: how to avoid single character to get indexed for directspellchecker dictionary
hi james I have tried using length filter factory as well but it seems that it is removing the single character from the index but when I qeuery for delll it is still giving dell l in suggestions this I think is due to querying the term like dell l solr can find the result as in it will tokenise dell and l and will return the results with dell in the documents so to remove such thing do I have to use minbreaklenth? and what is the significance of minbreak length number? On Fri, Apr 5, 2013 at 12:20 PM, Rohan Thakur wrote: > hi james > > after using this its working file for delll but not for de. what does > this minbreaklength signifies? > > > also can you tell me why am I not getting suggestions for smaller words > like for del i should get dell as suggestion but its not giving any > suggestions and also can I get suggestion for like complete the sentence > like if I give sams it should also give samsung as in suggestion? > > thanks > regards > Rohan > > > > > On Fri, Apr 5, 2013 at 12:54 AM, Dyer, James > wrote: > >> I assume if your user queries "delll" and it breaks it into pieces like >> "de l l l", then you're probably using WordBreakSolrSpellChecker in >> addition to DirectSolrSpellChecker, right? If so, then you can specify >> "minBreakLength" in solrconfig.xml like this: >> >> >> ... spellcheckers here ... >> >> wordbreak >> solr.WordBreakSolrSpellChecker >> ... parameters here ... >> 5 >> >> >> >> One note is that both DirectSolrSpellChecker and >> WordBreakSolrSpellChecker operate directly on the terms dictionary and do >> not have a separate dictionary like IndexBasedSpellChecker. The only way >> to prevent a word from being in the dictionary then is to filter this out >> in the analysis chain. For instance, if you use to build a >> field just for spellchecking, you can use LengthFilterFactory to remove the >> short terms. See >> http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters#solr.LengthFilterFactory. >> >> James Dyer >> Ingram Content Group >> (615) 213-4311 >> >> >> -Original Message- >> From: Rohan Thakur [mailto:rohan.i...@gmail.com] >> Sent: Thursday, April 04, 2013 1:42 PM >> To: solr-user@lucene.apache.org >> Subject: how to avoid single character to get indexed for >> directspellchecker dictionary >> >> hi all >> >> I am using solr directspellcheker for spell suggestions using raw analyses >> for indexing but I have some fields which have single characters like l L >> so its is been indexed in the dictionary and when I am using this for >> suggestions for query like delll its suggesting de and l l l as the spell >> correction as my index has de and l as single characters in the fields. >> please help. >> >> thanks >> regards >> Rohan >> >> >
solr spell suggestions help
hi all I had some issues with solr spell suggestions. 1) first of all I wanted to know is indexbased spell suggestions better then directspell suggestions that solr 4.1 provides in any way? 2) then I wanted to know is their way I can get suggestions for words providing only few prefix for the word. like when I query sam I should get samsung as one of suggestion. 3) also I wanted to know why am I not getting suggestions for the words that have more then 2 character difference between them like if I query for wirlpool wich has 8 characters I get suggestion as whirlpool which is 9 characters and correct spelling but when I query for wirlpol which is 7 characters it says that this is false spelling but does not show any suggestions. even like if I search for pansonic(8 char) it provides panasonic(9 char) as suggestion but when I remove one more character that is is search for panonic(7 char) it does not return any suggestions?? how can I correct this? even when I search for ipo it does not return ipod as suggestions? 4) one more thing I want to get clear that when I search for microwave ovan it does not give any miss spell even when ovan is wrong it provides the result for microwave saying the query is correct...this is the case when one of the term in the query is correct while others are incorrect it does not point out the wrong spelling one but reutrns the result for correct word thats it how can I correct this? similar is the case when I query for microvave oven is shows the result for oven saying that the query is correct.. 5) one more case is when I query plntronies (correct word is: plantronics) it does not return any solution but when I query for plantronies it returns the plantronics as suggestions why is that happening? *my schema.xml is:* *my solrconfig.xml is :* *default* solr.DirectSolrSpellChecker *spell internal 0.3 1 1 5 4 0.01 * * wordbreak solr.WordBreakSolrSpellChecker spell true true 3 * * jarowinkler spell solr.DirectSolrSpellChecker org.apache.lucene.search.spell.JaroWinklerDistance * *tSpell * * * * spell default wordbreak false on true 10 5 5 true true 10 5 spellcheck * thanks in advance regards Rohan
query regarding spell suggestions for multiple word query
hi all I wanted to know how can I generate spell suggestion for more than one word query my suggestions are working well for single word correction but when I pass more then one word query and any one of them is correct and other is wrong spelled the sentence is showns as correct and the results appears based on the correctly spelled word rather than suggesting correction to the word out of them which is wrongly spelled. for example I query for q=microwave ovan in this ovan is miss spelled and must be given suggestion but rather than giving suggestion for ovan as oven solr is stating that the sentence is correct and showing results based on microwave which is indeed correct word but complete sentence is not correct as whole...since ovan is misspelledit should rather provide suggestion and when I query for q=ovan separately it gives corrected suggestion for it as oven.so sugestion is working fine how can I use it for complete sentence not just word? help please thanks regards Rohan
help needed for applying patch to solr I am using
hi all I am new to solr and wanted to apply this patch to my solr how can I do this searched on net but did not got any thing useful the patch is: https://issues.apache.org/jira/browse/SOLR-2585 I am using solr 4.1.0 on tomcat6 in redhat centos. thanks regards rohan
Re: solr spell suggestions help
hi all I have resolved all issues(its was relating to the distance measures I was using was by default lavanstine which is very basic and is not good now I am using jarowinkler distance measures which is better and now giving exact results that I was looking for) except the 4th one which I think is solrs issue and they have also released patch for that https://issues.apache.org/jira/browse/SOLR-2585 I am applying this patch now will let you know if its is working correctly. thanks regards Rohan On Fri, Apr 5, 2013 at 4:44 PM, Rohan Thakur wrote: > hi all > > I had some issues with solr spell suggestions. > > 1) first of all I wanted to know is indexbased spell suggestions better > then directspell suggestions that solr 4.1 provides in any way? > > 2) then I wanted to know is their way I can get suggestions for words > providing only few prefix for the word. like when I query sam I should get > samsung as one of suggestion. > > 3) also I wanted to know why am I not getting suggestions for the words > that have more then 2 character difference between them like if I query for > wirlpool wich has 8 characters I get suggestion as whirlpool which is 9 > characters and correct spelling but when I query for wirlpol which is 7 > characters it says that this is false spelling but does not show any > suggestions. even like if I search for pansonic(8 char) it provides > panasonic(9 char) as suggestion but when I remove one more character that > is is search for panonic(7 char) it does not return any suggestions?? how > can I correct this? even when I search for ipo it does not return ipod as > suggestions? > > 4) one more thing I want to get clear that when I search for microwave > ovan it does not give any miss spell even when ovan is wrong it provides > the result for microwave saying the query is correct...this is the case > when one of the term in the query is correct while others are incorrect it > does not point out the wrong spelling one but reutrns the result for > correct word thats it how can I correct this? similar is the case when I > query for microvave oven is shows the result for oven saying that the query > is correct.. > > 5) one more case is when I query plntronies (correct word is: plantronics) > it does not return any solution but when I query for plantronies it returns > the plantronics as suggestions why is that happening? > > *my schema.xml is:* > omitNorms="true"> > >pattern="\\\[\]\(\)\-\,\/\+" replacement=" "/> > > >words="stopwords.txt"/> > > > > > > >synonyms="synonyms.txt" ignoreCase="true" expand="true"/> >words="stopwords.txt"/> > > > > > > > > > > > *my solrconfig.xml is :* > > > > > > > > > > > *default* > > solr.DirectSolrSpellChecker > > > *spell > internal > > 0.3 > > 1 > > 1 > > 5 > > 4 > > 0.01 > > * > > > * > wordbreak > solr.WordBreakSolrSpellChecker > spell > true > true > 3 > > * > > > > * > jarowinkler > spell > solr.DirectSolrSpellChecker > name="distanceMeasure">org.apache.lucene.search.spell.JaroWinklerDistance >* > > > > > > > > > > >*tSpell > * > > > > > * class="solr.SpellingQueryConverter"/>* > > > * startup="lazy"> > > spell > > default > wordbreak > > > > false > on > true > 10 > 5 > 5 > true > true > 10 > 5 > > > spellcheck > > > * > > > > thanks in advance > regards > Rohan >
Re: help needed for applying patch to solr I am using
hi all just checked out this issue was already incorporated in solr4.0 alpha and I am using solr4.1.0 so this must have been in this as wellbut still why am I not getting suggestions for word like microvave oven its stating it to be correct and returning results based of oven wordwhy is this happening? any one please help and when I am querying it like "microvave oven" its providing corrected suggestionhow to handle this any one please help... thanks regards Rohan On Mon, Apr 8, 2013 at 1:18 PM, Rohan Thakur wrote: > hi all > > I am new to solr and wanted to apply this patch to my solr how can I do > this searched on net but did not got any thing useful > the patch is: > https://issues.apache.org/jira/browse/SOLR-2585 > > I am using solr 4.1.0 on tomcat6 in redhat centos. > > thanks > regards > rohan >
Re: help needed for applying patch to solr I am using
hi all I think I have to pass the query in inverted commas then its returning correct results as i needed. thanks regards Rohan On Mon, Apr 8, 2013 at 1:50 PM, Rohan Thakur wrote: > hi all > > just checked out this issue was already incorporated in solr4.0 alpha and > I am using solr4.1.0 so this must have been in this as wellbut still > why am I not getting suggestions for word like microvave oven its stating > it to be correct and returning results based of oven wordwhy is this > happening? > any one please help and when I am querying it like "microvave oven" its > providing corrected suggestionhow to handle this any one please help... > > thanks > regards > Rohan > > > On Mon, Apr 8, 2013 at 1:18 PM, Rohan Thakur wrote: > >> hi all >> >> I am new to solr and wanted to apply this patch to my solr how can I do >> this searched on net but did not got any thing useful >> the patch is: >> https://issues.apache.org/jira/browse/SOLR-2585 >> >> I am using solr 4.1.0 on tomcat6 in redhat centos. >> >> thanks >> regards >> rohan >> > >
spell suggestions help
hi all one thing I wanted to clear is for every other query I have got correct suggestions but these 2 cases I am not getting what suppose to be the suggestions: 1) I have kettle(doc frequency =5) and cable(doc frequecy=1) word indexed in direct solr spell cheker..but when I query for cattle I get cable as only suggestion and not kettle why is this happening i want to get kettle in suggestion as well im using jarowinkler distance according to which score for cattle => cable which is coming out to be 0.857 and for cattle => kettle which is coming out to be 0.777 kettle should also come in suggestions but its not how can I correct this any one. 2) how to query for sentence like "hand blandar & chopper" as & is delimiter for solr query and thus this query is returning error. thanks in advance regards Rohan
query regarding the use of boost across the fields in edismax query
hi all wanted to know what could be the difference between the results if I apply boost accross say 5 fields in query like for first: title^10.0 features^7.0 cat^5.0 color^3.0 root^1.0 and second settings like : title^10.0 features^5.0 cat^3.0 color^2.0 root^1.0 what could be the difference as in the weights are in same order decreasing? thanks in advance regards Rohan
Re: query regarding the use of boost across the fields in edismax query
hi otis can you explain that in some depth like If is search for led in both the cases what could be the difference in the results I get? thanks in advance regards Rohan On Tue, Apr 9, 2013 at 11:25 PM, Otis Gospodnetic < otis.gospodne...@gmail.com> wrote: > Not sure if i'm missing something but in the first case features, cat, > and color field have more weight, so matches on them with have bigger > contribution to the overall relevancy score. > > Otis > -- > Solr & ElasticSearch Support > http://sematext.com/ > > > > > > On Tue, Apr 9, 2013 at 1:52 PM, Rohan Thakur wrote: > > hi all > > > > wanted to know what could be the difference between the results if I > apply > > boost accross say 5 fields in query like for > > > > first: title^10.0 features^7.0 cat^5.0 color^3.0 root^1.0 and > > second settings like : title^10.0 features^5.0 cat^3.0 color^2.0 root^1.0 > > > > what could be the difference as in the weights are in same order > decreasing? > > > > thanks in advance > > > > regards > > Rohan >
Re: spell suggestions help
hi erick do we have to do urlencoding from the php side or does solr supports urlencode? On Thu, Apr 11, 2013 at 5:57 AM, Erick Erickson wrote: > Try URL encoding it and/or escaping the & > > On Tue, Apr 9, 2013 at 2:32 AM, Rohan Thakur wrote: > > hi all > > > > one thing I wanted to clear is for every other query I have got correct > > suggestions but these 2 cases I am not getting what suppose to be the > > suggestions: > > > > 1) I have kettle(doc frequency =5) and cable(doc frequecy=1) word indexed > > in direct solr spell cheker..but when I query for cattle I get cable as > > only suggestion and not kettle why is this happening i want to get kettle > > in suggestion as well im using jarowinkler distance according to which > > score for cattle => cable which is coming out to be 0.857 and for cattle > => > > kettle which is coming out to be 0.777 kettle should also come in > > suggestions but its not how can I correct this any one. > > > > 2) how to query for sentence like "hand blandar & chopper" as & is > > delimiter for solr query and thus this query is returning error. > > > > thanks in advance > > regards > > Rohan >
Re: spell suggestions help
urlencode replaces & with space thus resulting in results that contains even the single terms like in the case of mobile & accessories it replaces it with mobile accessories and results the document containing even accessories which i dont want. how to tackle this I tried using pattern replace filter at query time to replace & with and but it did not worked I used & => replace with "and" in this but did not worked any guess our help.. thanks regards rohan On Thu, Apr 11, 2013 at 4:39 PM, Rohan Thakur wrote: > hi erick > > do we have to do urlencoding from the php side or does solr supports > urlencode? > > > On Thu, Apr 11, 2013 at 5:57 AM, Erick Erickson > wrote: > >> Try URL encoding it and/or escaping the & >> >> On Tue, Apr 9, 2013 at 2:32 AM, Rohan Thakur >> wrote: >> > hi all >> > >> > one thing I wanted to clear is for every other query I have got correct >> > suggestions but these 2 cases I am not getting what suppose to be the >> > suggestions: >> > >> > 1) I have kettle(doc frequency =5) and cable(doc frequecy=1) word >> indexed >> > in direct solr spell cheker..but when I query for cattle I get cable as >> > only suggestion and not kettle why is this happening i want to get >> kettle >> > in suggestion as well im using jarowinkler distance according to which >> > score for cattle => cable which is coming out to be 0.857 and for >> cattle => >> > kettle which is coming out to be 0.777 kettle should also come in >> > suggestions but its not how can I correct this any one. >> > >> > 2) how to query for sentence like "hand blandar & chopper" as & is >> > delimiter for solr query and thus this query is returning error. >> > >> > thanks in advance >> > regards >> > Rohan >> > >
Re: spell suggestions help
hi jack I am using whitespace toknizer only and before this im using pattern replace to replace & with and but its not working I guess. my query analyser: wrote: > Try replacing standard tokenizer with whitespace tokenizer in your field > types. And make sure not to use any other token filters that might discard > special characters (or provide a character map if they support one.) > > Also, be side to try your test terms in the Solr Admin UI ANalyzer page to > see that the "&" is preserved or which stage in term analysis it gets > discarded. > > -- Jack Krupansky > > -Original Message- From: Rohan Thakur > Sent: Thursday, April 11, 2013 7:39 AM > To: solr-user@lucene.apache.org > Subject: Re: spell suggestions help > > > urlencode replaces & with space thus resulting in results that contains > even the single terms like in the case of mobile & accessories it replaces > it with mobile accessories and results the document containing even > accessories which i dont want. how to tackle this I tried using pattern > replace filter at query time to replace & with and but it did not worked I > used & => replace with "and" in this but did not worked any guess our > help.. > > thanks > regards > rohan > > > On Thu, Apr 11, 2013 at 4:39 PM, Rohan Thakur > wrote: > > hi erick >> >> do we have to do urlencoding from the php side or does solr supports >> urlencode? >> >> >> On Thu, Apr 11, 2013 at 5:57 AM, Erick Erickson >> **wrote: >> >> Try URL encoding it and/or escaping the & >>> >>> On Tue, Apr 9, 2013 at 2:32 AM, Rohan Thakur >>> wrote: >>> > hi all >>> > >>> > one thing I wanted to clear is for every other query I have got correct >>> > suggestions but these 2 cases I am not getting what suppose to be the >>> > suggestions: >>> > >>> > 1) I have kettle(doc frequency =5) and cable(doc frequecy=1) word >>> indexed >>> > in direct solr spell cheker..but when I query for cattle I get cable as >>> > only suggestion and not kettle why is this happening i want to get >>> kettle >>> > in suggestion as well im using jarowinkler distance according to which >>> > score for cattle => cable which is coming out to be 0.857 and for >>> cattle => >>> > kettle which is coming out to be 0.777 kettle should also come in >>> > suggestions but its not how can I correct this any one. >>> > >>> > 2) how to query for sentence like "hand blandar & chopper" as & is >>> > delimiter for solr query and thus this query is returning error. >>> > >>> > thanks in advance >>> > regards >>> > Rohan >>> >>> >> >> >
solr spell correction help
hi all I have configured solr direct spell correction on spell field most of the words solr is correcting and giving suggestions but on some words like mentioned below is giving absurd results: 1) blender(indexed) 2) kettle(indexed) 3) electric(indexed) problems: 1) when I search for blandar its giving correct result as blender but when I search for blandars its not giving correction as blender 2) for this when I search for kettle the correct spell its still showing it to be false but not giving suggestions and even the results documents are showing up. and when I search for cettle its giving correct result as kettle but when I search for cattle its not giving any suggestions. 3) for this again when I search for electric the correct spell its showing it to be false in suggestions section but not giving any suggestions and documents are also returning for this spelling as its the correct one. even if I want solr to return samsung as spell suggetion if I search for sam what could be the configuration and what could be the solution for above problems? please help. thanks in advance regards Rohan
Re: solr spell correction help
k thanks jack but then why does cattle not giving kettle as suggestions?? On Fri, Apr 12, 2013 at 6:46 PM, Jack Krupansky wrote: > "blandars its not giving correction as blender" > > They have an edit distance of 3. Direct Spell is limited to a maximum ED > of 2. > > -- Jack Krupansky > > -Original Message- From: Rohan Thakur > Sent: Friday, April 12, 2013 8:45 AM > To: solr-user@lucene.apache.org > Subject: solr spell correction help > > > hi all > > I have configured solr direct spell correction on spell field most of the > words solr is correcting and giving suggestions but on some words like > mentioned below is giving absurd results: > > 1) blender(indexed) > 2) kettle(indexed) > 3) electric(indexed) > > problems: > 1) when I search for blandar its giving correct result as blender but when > I search for blandars its not giving correction as blender > > 2) for this when I search for kettle the correct spell its still showing it > to be false but not giving suggestions and even the results documents are > showing up. and when I search for cettle its giving correct result as > kettle but when I search for cattle its not giving any suggestions. > > 3) for this again when I search for electric the correct spell its showing > it to be false in suggestions section but not giving any suggestions and > documents are also returning for this spelling as its the correct one. > > even if I want solr to return samsung as spell suggetion if I search for > sam what could be the configuration and what could be the solution for > above problems? please help. > > thanks in advance > > regards > Rohan >
Re: solr spell correction help
but jack im not using lavanstine distance measures im using jarowinker distance On Mon, Apr 15, 2013 at 11:50 AM, Rohan Thakur wrote: > k thanks jack but then why does cattle not giving kettle as suggestions?? > > > On Fri, Apr 12, 2013 at 6:46 PM, Jack Krupansky > wrote: > >> "blandars its not giving correction as blender" >> >> They have an edit distance of 3. Direct Spell is limited to a maximum ED >> of 2. >> >> -- Jack Krupansky >> >> -Original Message- From: Rohan Thakur >> Sent: Friday, April 12, 2013 8:45 AM >> To: solr-user@lucene.apache.org >> Subject: solr spell correction help >> >> >> hi all >> >> I have configured solr direct spell correction on spell field most of the >> words solr is correcting and giving suggestions but on some words like >> mentioned below is giving absurd results: >> >> 1) blender(indexed) >> 2) kettle(indexed) >> 3) electric(indexed) >> >> problems: >> 1) when I search for blandar its giving correct result as blender but when >> I search for blandars its not giving correction as blender >> >> 2) for this when I search for kettle the correct spell its still showing >> it >> to be false but not giving suggestions and even the results documents are >> showing up. and when I search for cettle its giving correct result as >> kettle but when I search for cattle its not giving any suggestions. >> >> 3) for this again when I search for electric the correct spell its showing >> it to be false in suggestions section but not giving any suggestions and >> documents are also returning for this spelling as its the correct one. >> >> even if I want solr to return samsung as spell suggetion if I search for >> sam what could be the configuration and what could be the solution for >> above problems? please help. >> >> thanks in advance >> >> regards >> Rohan >> > >
query regarding the multiple documents
hi all wanted to know that I have indexed documents for search purpose in solr and now for auto suggestion purpose I want to index new data that is the popular query term searched by users and frequency of them to get searched on websitebut as it has no relation with the product data on which I have build search...can I like index this new table in same data-config.xml within new document tag and if so then how to configure request handler for auto suggestion to search in this new index only and not the search index document. thanks regards rohan
indexing unrelated tables in single core
hi all I want to index 2 separate unrelated tables from database into single solr core and search in any one of the document separately how can I do it? please help thanks in advance regards Rohan
Re: indexing unrelated tables in single core
I am not able to index the fields from data base its getting failed... data-config.xml its showing all failed and 0 indexed On Wed, May 15, 2013 at 8:31 PM, Alexandre Rafalovitch wrote: > 1. Create a schema that accomodates both types of fields either using > optional fields or dynamic fields. > 2. Create some sort of differentiator key (e.g. schema), separately > from id (which needs to be globally unique, so possibly schema+id) > 3. Use that schema in filter queries (fq) to look only at subject of items > 4. (Optionally) define separate search request handlers that force > that schema parameter (using appends or invariants instead of > defaults) > > That should get you most of the way there. > > Regards, >Alex. > Personal blog: http://blog.outerthoughts.com/ > LinkedIn: http://www.linkedin.com/in/alexandrerafalovitch > - Time is the quality of nature that keeps events from happening all > at once. Lately, it doesn't seem to be working. (Anonymous - via GTD > book) > > > On Wed, May 15, 2013 at 7:07 AM, Rohan Thakur > wrote: > > hi all > > > > > > I want to index 2 separate unrelated tables from database into single > solr > > core and search in any one of the document separately how can I do it? > > please help > > > > thanks in advance > > regards > > Rohan >
Re: indexing unrelated tables in single core
its saying in the logs that missing required field title which is no where in the database... On Thu, May 16, 2013 at 3:08 PM, Rohan Thakur wrote: > I am not able to index the fields from data base its getting failed... > > data-config.xml > > url="jdbc:mysql://localhost/test" > user="user" password="dfsdf"/> > > > > > > > > its showing all failed and 0 indexed > > > On Wed, May 15, 2013 at 8:31 PM, Alexandre Rafalovitch > wrote: > >> 1. Create a schema that accomodates both types of fields either using >> optional fields or dynamic fields. >> 2. Create some sort of differentiator key (e.g. schema), separately >> from id (which needs to be globally unique, so possibly schema+id) >> 3. Use that schema in filter queries (fq) to look only at subject of items >> 4. (Optionally) define separate search request handlers that force >> that schema parameter (using appends or invariants instead of >> defaults) >> >> That should get you most of the way there. >> >> Regards, >>Alex. >> Personal blog: http://blog.outerthoughts.com/ >> LinkedIn: http://www.linkedin.com/in/alexandrerafalovitch >> - Time is the quality of nature that keeps events from happening all >> at once. Lately, it doesn't seem to be working. (Anonymous - via GTD >> book) >> >> >> On Wed, May 15, 2013 at 7:07 AM, Rohan Thakur >> wrote: >> > hi all >> > >> > >> > I want to index 2 separate unrelated tables from database into single >> solr >> > core and search in any one of the document separately how can I do it? >> > please help >> > >> > thanks in advance >> > regards >> > Rohan >> > >
Re: indexing unrelated tables in single core
hi I got the problem it is with the unique key defined in the schema.xml if i difine it to be query_id then while indexing it says missing mandatory key query_id which is not present in the root entity(data-config.xml) which is indexing the product from the database which has product_id as the unique key and when in schema I set product_id as the unique key then it says missing mandatory key product_id which is not present in the root entity(data-config.xml) which is indiexing the user query from another table in the database which has user_id as the unique key. how can I fix this thanks I want to index both the tables which are basically unrelated that is does not have any *Common* fields thanks rohan On Thu, May 16, 2013 at 3:24 PM, Michael Della Bitta < michael.della.bi...@appinions.com> wrote: > True, it's complaining that your Solr schema has a required field 'title' > and your query and data import config aren't providing it. > On May 16, 2013 5:51 AM, "Rohan Thakur" wrote: > > > its saying in the logs that missing required field title which is no > where > > in the database... > > > > > > On Thu, May 16, 2013 at 3:08 PM, Rohan Thakur > > wrote: > > > > > I am not able to index the fields from data base its getting failed... > > > > > > data-config.xml > > > > > > > > url="jdbc:mysql://localhost/test" > > > user="user" password="dfsdf"/> > > > > > > > > > > > > > > > > > > > > > > > > its showing all failed and 0 indexed > > > > > > > > > On Wed, May 15, 2013 at 8:31 PM, Alexandre Rafalovitch < > > arafa...@gmail.com > > > > wrote: > > > > > >> 1. Create a schema that accomodates both types of fields either using > > >> optional fields or dynamic fields. > > >> 2. Create some sort of differentiator key (e.g. schema), separately > > >> from id (which needs to be globally unique, so possibly schema+id) > > >> 3. Use that schema in filter queries (fq) to look only at subject of > > items > > >> 4. (Optionally) define separate search request handlers that force > > >> that schema parameter (using appends or invariants instead of > > >> defaults) > > >> > > >> That should get you most of the way there. > > >> > > >> Regards, > > >>Alex. > > >> Personal blog: http://blog.outerthoughts.com/ > > >> LinkedIn: http://www.linkedin.com/in/alexandrerafalovitch > > >> - Time is the quality of nature that keeps events from happening all > > >> at once. Lately, it doesn't seem to be working. (Anonymous - via GTD > > >> book) > > >> > > >> > > >> On Wed, May 15, 2013 at 7:07 AM, Rohan Thakur > > >> wrote: > > >> > hi all > > >> > > > >> > > > >> > I want to index 2 separate unrelated tables from database into > single > > >> solr > > >> > core and search in any one of the document separately how can I do > it? > > >> > please help > > >> > > > >> > thanks in advance > > >> > regards > > >> > Rohan > > >> > > > > > > > > >
Re: indexing unrelated tables in single core
I mean to say that I want to index 2 tables that is using 2 root entity in data-config.xml one is product table and other is user search table these both have no foreign key and I want to index both of them as document in my solr index what should I do...its taking either one of them and rejecting other table as document when I am taking primary key of one table as unique key in the solr schema...and vice verca.how to solve this? On Thu, May 16, 2013 at 4:24 PM, Rohan Thakur wrote: > hi > > I got the problem it is with the unique key defined in the schema.xml > if i difine it to be query_id then while indexing it says > missing mandatory key query_id which is not present in the root > entity(data-config.xml) which is indexing the product from the database > which has product_id as the unique key and when in schema I set product_id > as the unique key then it says missing mandatory key product_id which is > not present in the root entity(data-config.xml) which is indiexing the user > query from another table in the database which has user_id as the unique > key. > > how can I fix this thanks I want to index both the tables which are > basically unrelated that is does not have any *Common* fields > > thanks > rohan > > > On Thu, May 16, 2013 at 3:24 PM, Michael Della Bitta < > michael.della.bi...@appinions.com> wrote: > >> True, it's complaining that your Solr schema has a required field 'title' >> and your query and data import config aren't providing it. >> On May 16, 2013 5:51 AM, "Rohan Thakur" wrote: >> >> > its saying in the logs that missing required field title which is no >> where >> > in the database... >> > >> > >> > On Thu, May 16, 2013 at 3:08 PM, Rohan Thakur >> > wrote: >> > >> > > I am not able to index the fields from data base its getting failed... >> > > >> > > data-config.xml >> > > >> > > > > > url="jdbc:mysql://localhost/test" >> > > user="user" password="dfsdf"/> >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > its showing all failed and 0 indexed >> > > >> > > >> > > On Wed, May 15, 2013 at 8:31 PM, Alexandre Rafalovitch < >> > arafa...@gmail.com >> > > > wrote: >> > > >> > >> 1. Create a schema that accomodates both types of fields either using >> > >> optional fields or dynamic fields. >> > >> 2. Create some sort of differentiator key (e.g. schema), separately >> > >> from id (which needs to be globally unique, so possibly schema+id) >> > >> 3. Use that schema in filter queries (fq) to look only at subject of >> > items >> > >> 4. (Optionally) define separate search request handlers that force >> > >> that schema parameter (using appends or invariants instead of >> > >> defaults) >> > >> >> > >> That should get you most of the way there. >> > >> >> > >> Regards, >> > >>Alex. >> > >> Personal blog: http://blog.outerthoughts.com/ >> > >> LinkedIn: http://www.linkedin.com/in/alexandrerafalovitch >> > >> - Time is the quality of nature that keeps events from happening all >> > >> at once. Lately, it doesn't seem to be working. (Anonymous - via >> GTD >> > >> book) >> > >> >> > >> >> > >> On Wed, May 15, 2013 at 7:07 AM, Rohan Thakur >> > >> wrote: >> > >> > hi all >> > >> > >> > >> > >> > >> > I want to index 2 separate unrelated tables from database into >> single >> > >> solr >> > >> > core and search in any one of the document separately how can I do >> it? >> > >> > please help >> > >> > >> > >> > thanks in advance >> > >> > regards >> > >> > Rohan >> > >> >> > > >> > > >> > >> > >
Re: indexing unrelated tables in single core
hi mohanty I appreciate it but dint get that can you please elaborate? my dataconfig is like: my schema is like: value_id title thanks regards Rohan On Thu, May 16, 2013 at 5:11 PM, Gora Mohanty wrote: > On 16 May 2013 16:24, Rohan Thakur wrote: > > hi > > > > I got the problem it is with the unique key defined in the schema.xml > > if i difine it to be query_id then while indexing it says > > missing mandatory key query_id which is not present in the root > > entity(data-config.xml) which is indexing the product from the database > > which has product_id as the unique key and when in schema I set > product_id > > as the unique key then it says missing mandatory key product_id which is > > not present in the root entity(data-config.xml) which is indiexing the > user > > query from another table in the database which has user_id as the unique > > key. > > > > how can I fix this thanks I want to index both the tables which are > > basically unrelated that is does not have any *Common* fields > [...] > > Fix it in the SELECT statement: > SELECT product_id as id,... for one entity, and > SELECT query_id as id,... in the other > and use id as the uniqueKey for Solr. > > Regards, > Gora >
Re: indexing unrelated tables in single core
hi Mohanty I tried what you suggested of using id as common field and changing the SQL query to point to id and using id as uniqueKey it is working but now what it is doing is just keeping the id's that are not same in both the tables and discarding the id's that are same in both the tablesbut this is not correct as both the product_id and query_id has no relation as such both are representing separate things in each tables. regards Rohan On Thu, May 16, 2013 at 5:11 PM, Gora Mohanty wrote: > On 16 May 2013 16:24, Rohan Thakur wrote: > > hi > > > > I got the problem it is with the unique key defined in the schema.xml > > if i difine it to be query_id then while indexing it says > > missing mandatory key query_id which is not present in the root > > entity(data-config.xml) which is indexing the product from the database > > which has product_id as the unique key and when in schema I set > product_id > > as the unique key then it says missing mandatory key product_id which is > > not present in the root entity(data-config.xml) which is indiexing the > user > > query from another table in the database which has user_id as the unique > > key. > > > > how can I fix this thanks I want to index both the tables which are > > basically unrelated that is does not have any *Common* fields > [...] > > Fix it in the SELECT statement: > SELECT product_id as id,... for one entity, and > SELECT query_id as id,... in the other > and use id as the uniqueKey for Solr. > > Regards, > Gora >
Re: indexing unrelated tables in single core
thanks gora I got that...have to change in the DB itself... On Fri, May 17, 2013 at 6:27 PM, Gora Mohanty wrote: > On 16 May 2013 19:11, Rohan Thakur wrote: > > hi Mohanty > > > > I tried what you suggested of using id as common field and changing the > SQL > > query to point to id > > and using id as uniqueKey > > it is working but now what it is doing is just keeping the id's that are > > not same in both the tables and discarding the id's that are same in both > > the tablesbut this is not correct as both the product_id and query_id > > has no relation as such both are representing separate things in each > > tables. > [...] > > Sorry, was away from email. The last configuration that you posted > seemed fine, and as you say above things seem to work for you. > > What you are facing now is that documents where the product_id > is the same as the query_id are being overwritten, as they have the > same uniqueKey as far as Solr is concerned. Thus, Solr will update an > existing document rather than adding a new one. So, you have to > come up with a scheme that makes the IDs unique. There are various > ways of doing this depending on how your product_id/query_id are > set up. One way might be to make the 'id' field that is used as the > uniqueKey a string rather than an integer, and prefix 'P' for product_ids > and 'Q' for query_ids. > > Regards, > Gora >
Sorting solr search results using multiple fields
hi all I wanted to know is there a way I can sort the my documents based on 3 fields I have fields like pop(which is basically frequency of the term searched history) and autosug(auto suggested words) and initial_boost(copy field of autosug such that only match with initial term match having whole sentence saved as one token) Now I want the documents to be returned as: 1. initial_boost with pop of 192 2. initial_boost with pop of 156 3. initial_boost with pop of 120 4. autosug with pop of 205 5. autosug with pop of 180 6. autosug with pop of 112 I have tried using boosting the initial_boost field but without the sort it does the above boost to the initial_boost than autosug but as I add sort=pop desc documents gets sorted according to pop field disturbing the boost on the fields that I had set. help anyone... thanks in advance. regards Rohan
Re: Sorting solr search results using multiple fields
thanks gora I got that one more thing what actually I have done is made document consisting of fields: { "autosug":"galaxy", "query_id":1414, "pop":168, "initial_boost":"galaxy" "_version_":1435669695565922305, "score":1.8908522} this inital_boost is basically copy field of autosug but saved using different analysers taking whole sentence as single token and generating edge ngrams so that what I search on this field only term matching from first will match...and for any other infix term match I have autosug field so now what I want from this is to show the documents returned with initial_boost first and then the documents with autosug field sorted with pop field respectively (separately) and return the result... now from your suggestion I could do this using sort on multiple fields by separating them by commas, as described under http://wiki.apache.org/solr/CommonQueryParameters#sort but for that I would require 1 field having value greater(all equal say 2) for initial_boost field and smaller(all same say 1) for autosug field how can I do this? or is there some better solution.. thanks regards Rohan On Wed, May 22, 2013 at 6:39 PM, Gora Mohanty wrote: > On 22 May 2013 18:26, Rohan Thakur wrote: > > hi all > > > > I wanted to know is there a way I can sort the my documents based on 3 > > fields > > I have fields like pop(which is basically frequency of the term searched > > history) and autosug(auto suggested words) and initial_boost(copy field > of > > autosug such that only match with initial term match having > > whole sentence saved as one token) > [...] > > You seem to be confusing boosting with sorting. If you > sort the results, the boosts are irrelevant. > > You can sort on multiple fields by separating them by > commas, as described under > http://wiki.apache.org/solr/CommonQueryParameters#sort > > Regards, > Gora >
Re: Sorting solr search results using multiple fields
k thanks gora that what I was thinking but thought I should ask as there might be some solution in solr it self...the last option fine I would go with that only. thanks rohan On Thu, May 23, 2013 at 9:13 AM, Gora Mohanty wrote: > On 22 May 2013 19:37, Rohan Thakur wrote: > [...] > > this inital_boost is basically copy field of autosug but saved using > > different analysers taking whole sentence as single token and generating > > edge ngrams so that what I search on this field only term matching from > > first will match...and for any other infix term match I have autosug > > field > > > > so now what I want from this is to show the documents returned with > > initial_boost first and then the documents with autosug field sorted with > > pop field respectively (separately) and return the result... > > > > now from your suggestion I could do this using sort on multiple fields > by > > separating them by > > commas, as described under > > http://wiki.apache.org/solr/CommonQueryParameters#sort > > > > but for that I would require 1 field having value greater(all equal say > 2) > > for initial_boost field and smaller(all same say 1) for autosug field how > > can I do this? or is there some better solution.. > > Um, maybe it would help if you started out by explaining > what you are trying to do instead of changing the requirements > after a solution is proposed. > > The above design seems faulty. If the fields have the same > values, there is no way to sort them differently. Boosts are > not guaranteed to always promote documents in the > desired manner, and will not work with sorting in any case. > You can try to kludge things together in various ways, e.g., > * Add a prefix so that some fields always sort first, e.g., > prefix "aaa..." to a string field to try and ensure that it > sorts first > * Have not tried this, but it should be possible to use a > Solr function query with a function that sorts by the > field name: http://wiki.apache.org/solr/FunctionQuery > > As you seem to be using analysers on the fields, beware > that sorting will not work reliably on multi-valued or tokenised > fields. > > You could also index different fields to different Solr cores, > fetch the results for individual fields with separate queries, > and combine them as desired. This might be the cleanest, > if not the most efficient, solution. > > Regards, > Gora >
Re: help with getting error with mysql driver connection in solr 4.1
It worked thanks alot Arcadius.. On Fri, Feb 1, 2013 at 7:56 PM, Arcadius Ahouansou wrote: > Hi Rohan. > * > * > Solr 4.1 uses Jetty 8. > > You need to put your JDBC driver under SOLR_HOME/lib/ext > > SOLR_HOME/lib/ being where all jetty *jar sit. > You may need to create "ext" if it does not exists. > > > HTH. > > Arcadius. > > > > On 1 February 2013 13:25, Rohan Thakur wrote: > > > hi everyone > > > > I am getting problem regarding mysql driver connection im upgrading to > solr > > 4.1 was using solr 3.6.2 earlier which was working fine but after > updating > > to solr 4.1, I have done all the required configuration but getting error > > with mysql driver connection. While doing full data import i am getting > > error : > > > > Caused by: java.lang.ClassNotFoundException: com.mysql.jdbc.Driver > > at java.net.URLClassLoader$1.run(Unknown Source) > > at java.security.AccessController.doPrivileged(Native Method) > > at java.net.URLClassLoader.findClass(Unknown Source) > > at java.lang.ClassLoader.loadClass(Unknown Source) > > at java.net.FactoryURLClassLoader.loadClass(Unknown Source) > > at java.lang.ClassLoader.loadClass(Unknown Source) > > at java.lang.Class.forName0(Native Method) > > at java.lang.Class.forName(Unknown Source) > > at > > org.apache.solr.core.SolrResourceLoader.findClass(SolrResourceLoader. > > java:422) > > > > I have done all the configuration I have even set the class path for > mysql > > driver jar file but its getting this same error and import fails > everytime > > ending with this error I have even tried setting the sharedLib="lib" in > > solr.xml and putting mysql connector in lib folder but it still not > working > > i am using solr 4.1 on windows and running it on jetty container and > using > > xammp for mysql database server. It was working fine on 3.6.2 version but > > 4.1 version not working I dont know why. please help! > > > > thanks in advance > > > > regards > > Rohan > > >
Re: help with getting error with mysql driver connection in solr 4.1
hi arcadius can you also help me with partial document update...I have followed what is written in this blog but its giving me error http://solr.pl/en/2012/07/09/solr-4-0-partial-documents-update/ error im getting after this command : C:\Users\rohan>curl localhost:8983/solr/update?commit=true -H Content-type:appli cation/json -d [{"value":"samsung-s5610-gray","value":{"set":samsung-111}}] {"responseHeader":{"status":500,"QTime":3},"error":{"msg":"Expected string: char =v,position=2 BEFORE='[{v' AFTER='alue:samsung-s5610-gray,value:{set:sams'","tra ce":"org.apache.noggit.JSONParser$ParseException: Expected string: char=v,positi on=2 BEFORE='[{v' AFTER='alue:samsung-s5610-gray,value:{set:sams'\r\n\tat org.ap ache.noggit.JSONParser.err(JSONParser.java:221)\r\n\tat org.apache.noggit.JSONPa rser.nextEvent(JSONParser.java:670)\r\n\tat org.apache.solr.handler.loader.JsonL oader$SingleThreadedJsonLoader.parseField(JsonLoader.java:425)\r\n\tat org.apach e.solr.handler.loader.JsonLoader$SingleThreadedJsonLoader.parseDoc(JsonLoader.ja va:414)\r\n\tat org.apache.solr.handler.loader.JsonLoader$SingleThreadedJsonLoad er.handleAdds(JsonLoader.java:386)\r\n\tat org.apache.solr.handler.loader.JsonLo ader$SingleThreadedJsonLoader.processUpdate(JsonLoader.java:112)\r\n\tat org.apa che.solr.handler.loader.JsonLoader$SingleThreadedJsonLoader.load(JsonLoader.java :96)\r\n\tat org.apache.solr.handler.loader.JsonLoader.load(JsonLoader.java:60)\ r\n\tat org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler .java:92)\r\n\tat org.apache.solr.handler.ContentStreamHandlerBase.handleRequest Body(ContentStreamHandlerBase.java:74)\r\n\tat org.apache.solr.handler.RequestHa ndlerBase.handleRequest(RequestHandlerBase.java:135)\r\n\tat org.apache.solr.cor e.SolrCore.execute(SolrCore.java:1816)\r\n\tat org.apache.solr.servlet.SolrDispa tchFilter.execute(SolrDispatchFilter.java:448)\r\n\tat org.apache.solr.servlet.S olrDispatchFilter.doFilter(SolrDispatchFilter.java:269)\r\n\tat org.eclipse.jett y.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1307)\r\n\tat org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:453)\r\n\t at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137) \r\n\tat org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java: 560)\r\n\tat org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHan dler.java:231)\r\n\tat org.eclipse.jetty.server.handler.ContextHandler.doHandle( ContextHandler.java:1072)\r\n\tat org.eclipse.jetty.servlet.ServletHandler.doSco pe(ServletHandler.java:382)\r\n\tat org.eclipse.jetty.server.session.SessionHand ler.doScope(SessionHandler.java:193)\r\n\tat org.eclipse.jetty.server.handler.Co ntextHandler.doScope(ContextHandler.java:1006)\r\n\tat org.eclipse.jetty.server. handler.ScopedHandler.handle(ScopedHandler.java:135)\r\n\tat org.eclipse.jetty.s erver.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255) \r\n\tat org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollec tion.java:154)\r\n\tat org.eclipse.jetty.server.handler.HandlerWrapper.handle(Ha ndlerWrapper.java:116)\r\n\tat org.eclipse.jetty.server.Server.handle(Server.jav a:365)\r\n\tat org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(Abs tractHttpConnection.java:485)\r\n\tat org.eclipse.jetty.server.BlockingHttpConne ction.handleRequest(BlockingHttpConnection.java:53)\r\n\tat org.eclipse.jetty.se rver.AbstractHttpConnection.content(AbstractHttpConnection.java:937)\r\n\tat org .eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttp Connection.java:998)\r\n\tat org.eclipse.jetty.http.HttpParser.parseNext(HttpPar ser.java:856)\r\n\tat org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParse r.java:240)\r\n\tat org.eclipse.jetty.server.BlockingHttpConnection.handle(Block ingHttpConnection.java:72)\r\n\tat org.eclipse.jetty.server.bio.SocketConnector$ ConnectorEndPoint.run(SocketConnector.java:264)\r\n\tat org.eclipse.jetty.util.t hread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)\r\n\tat org.eclipse.jet ty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)\r\n\tat java.la ng.Thread.run(Unknown Source)\r\n","code":500}} and nothing is changed for the field value am I doing something wrong here? regards Rohan On Mon, Feb 4, 2013 at 11:24 AM, Rohan Thakur wrote: > It worked thanks alot Arcadius.. > > > On Fri, Feb 1, 2013 at 7:56 PM, Arcadius Ahouansou > wrote: > >> Hi Rohan. >> * >> * >> Solr 4.1 uses Jetty 8. >> >> You need to put your JDBC driver under SOLR_HOME/lib/ext >> >> SOLR_HOME/lib/ being where all jetty *jar sit. >> You may need to create "ext" if it does not exists. >> >
Re: help with getting error with mysql driver connection in solr 4.1
hi arcadius I also tried going by this blog but in this too I am not able to use curl for update now it gives can not resolve host even if I can open the host using browserplease can you help me with thisI want to do partial document update for specific field... thanks regards Rohan On Mon, Feb 4, 2013 at 1:29 PM, Rohan Thakur wrote: > hi arcadius > > can you also help me with partial document update...I have followed what > is written in this blog but its giving me error > http://solr.pl/en/2012/07/09/solr-4-0-partial-documents-update/ > > error im getting after this command : > C:\Users\rohan>curl localhost:8983/solr/update?commit=true -H > Content-type:appli > cation/json -d [{"value":"samsung-s5610-gray","value":{"set":samsung-111}}] > {"responseHeader":{"status":500,"QTime":3},"error":{"msg":"Expected > string: char > =v,position=2 BEFORE='[{v' > AFTER='alue:samsung-s5610-gray,value:{set:sams'","tra > ce":"org.apache.noggit.JSONParser$ParseException: Expected string: > char=v,positi > on=2 BEFORE='[{v' AFTER='alue:samsung-s5610-gray,value:{set:sams'\r\n\tat > org.ap > ache.noggit.JSONParser.err(JSONParser.java:221)\r\n\tat > org.apache.noggit.JSONPa > rser.nextEvent(JSONParser.java:670)\r\n\tat > org.apache.solr.handler.loader.JsonL > oader$SingleThreadedJsonLoader.parseField(JsonLoader.java:425)\r\n\tat > org.apach > > e.solr.handler.loader.JsonLoader$SingleThreadedJsonLoader.parseDoc(JsonLoader.ja > va:414)\r\n\tat > org.apache.solr.handler.loader.JsonLoader$SingleThreadedJsonLoad > er.handleAdds(JsonLoader.java:386)\r\n\tat > org.apache.solr.handler.loader.JsonLo > ader$SingleThreadedJsonLoader.processUpdate(JsonLoader.java:112)\r\n\tat > org.apa > > che.solr.handler.loader.JsonLoader$SingleThreadedJsonLoader.load(JsonLoader.java > :96)\r\n\tat > org.apache.solr.handler.loader.JsonLoader.load(JsonLoader.java:60)\ > r\n\tat > org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler > .java:92)\r\n\tat > org.apache.solr.handler.ContentStreamHandlerBase.handleRequest > Body(ContentStreamHandlerBase.java:74)\r\n\tat > org.apache.solr.handler.RequestHa > ndlerBase.handleRequest(RequestHandlerBase.java:135)\r\n\tat > org.apache.solr.cor > e.SolrCore.execute(SolrCore.java:1816)\r\n\tat > org.apache.solr.servlet.SolrDispa > tchFilter.execute(SolrDispatchFilter.java:448)\r\n\tat > org.apache.solr.servlet.S > olrDispatchFilter.doFilter(SolrDispatchFilter.java:269)\r\n\tat > org.eclipse.jett > > y.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1307)\r\n\tat > > org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:453)\r\n\t > at > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137) > \r\n\tat > org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java: > 560)\r\n\tat > org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHan > dler.java:231)\r\n\tat > org.eclipse.jetty.server.handler.ContextHandler.doHandle( > ContextHandler.java:1072)\r\n\tat > org.eclipse.jetty.servlet.ServletHandler.doSco > pe(ServletHandler.java:382)\r\n\tat > org.eclipse.jetty.server.session.SessionHand > ler.doScope(SessionHandler.java:193)\r\n\tat > org.eclipse.jetty.server.handler.Co > ntextHandler.doScope(ContextHandler.java:1006)\r\n\tat > org.eclipse.jetty.server. > handler.ScopedHandler.handle(ScopedHandler.java:135)\r\n\tat > org.eclipse.jetty.s > > erver.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255) > \r\n\tat > org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollec > tion.java:154)\r\n\tat > org.eclipse.jetty.server.handler.HandlerWrapper.handle(Ha > ndlerWrapper.java:116)\r\n\tat > org.eclipse.jetty.server.Server.handle(Server.jav > a:365)\r\n\tat > org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(Abs > tractHttpConnection.java:485)\r\n\tat > org.eclipse.jetty.server.BlockingHttpConne > ction.handleRequest(BlockingHttpConnection.java:53)\r\n\tat > org.eclipse.jetty.se > rver.AbstractHttpConnection.content(AbstractHttpConnection.java:937)\r\n\tat > org > > .eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttp > Connection.java:998)\r\n\tat > org.eclipse.jetty.http.HttpParser.parseNext(HttpPar > ser.java:856)\r\n\tat > org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParse > r.java:240)\r\n\tat > org.eclipse.jetty.server.BlockingHttpConnection.handle(Block > ingHttpConnection.java:72)\r\n\tat > org.eclipse.jetty.server.bio.SocketConnector$ > ConnectorEndPoint.run(SocketConnector
Re: help with getting error with mysql driver connection in solr 4.1
hi gora I have tried what you told but now its giving error like: C:\Users\rohan>curl 127.0.0.1:8983/solr/update?commit=true -H "Content-type:appl ication/json" -d '[{"value":"samsung-wave-s5253-silver", "value":{"set":"samsung -111"}}]' {"responseHeader":{"status":500,"QTime":1},"error":{"msg":"JSON Parse Error: cha r=',position=0 BEFORE=''' AFTER='[{value:samsung-wave-s5253-silver,'","trace":"o rg.apache.noggit.JSONParser$ParseException: JSON Parse Error: char=',position=0 BEFORE=''' AFTER='[{value:samsung-wave-s5253-silver,'\r\n\tat org.apache.noggit. JSONParser.err(JSONParser.java:221)\r\n\tat org.apache.noggit.JSONParser.next(JS ONParser.java:620)\r\n\tat org.apache.noggit.JSONParser.nextEvent(JSONParser.jav a:662)\r\n\tat org.apache.solr.handler.loader.JsonLoader$SingleThreadedJsonLoade r.processUpdate(JsonLoader.java:106)\r\n\tat org.apache.solr.handler.loader.Json Loader$SingleThreadedJsonLoader.load(JsonLoader.java:96)\r\n\tat org.apache.solr .handler.loader.JsonLoader.load(JsonLoader.java:60)\r\n\tat org.apache.solr.hand ler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:92)\r\n\tat org.apache .solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBas e.java:74)\r\n\tat org.apache.solr.handler.RequestHandlerBase.handleRequest(Requ estHandlerBase.java:135)\r\n\tat org.apache.solr.core.SolrCore.execute(SolrCore. java:1816)\r\n\tat org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispat chFilter.java:448)\r\n\tat org.apache.solr.servlet.SolrDispatchFilter.doFilter(S olrDispatchFilter.java:269)\r\n\tat org.eclipse.jetty.servlet.ServletHandler$Cac hedChain.doFilter(ServletHandler.java:1307)\r\n\tat org.eclipse.jetty.servlet.Se rvletHandler.doHandle(ServletHandler.java:453)\r\n\tat org.eclipse.jetty.server. handler.ScopedHandler.handle(ScopedHandler.java:137)\r\n\tat org.eclipse.jetty.s ecurity.SecurityHandler.handle(SecurityHandler.java:560)\r\n\tat org.eclipse.jet ty.server.session.SessionHandler.doHandle(SessionHandler.java:231)\r\n\tat org.e clipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1072)\r\ n\tat org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:382)\ r\n\tat org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.j ava:193)\r\n\tat org.eclipse.jetty.server.handler.ContextHandler.doScope(Context Handler.java:1006)\r\n\tat org.eclipse.jetty.server.handler.ScopedHandler.handle (ScopedHandler.java:135)\r\n\tat org.eclipse.jetty.server.handler.ContextHandler Collection.handle(ContextHandlerCollection.java:255)\r\n\tat org.eclipse.jetty.s erver.handler.HandlerCollection.handle(HandlerCollection.java:154)\r\n\tat org.e clipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)\r\n\t at org.eclipse.jetty.server.Server.handle(Server.java:365)\r\n\tat org.eclipse.j etty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:485 )\r\n\tat org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(Blocking HttpConnection.java:53)\r\n\tat org.eclipse.jetty.server.AbstractHttpConnection. content(AbstractHttpConnection.java:937)\r\n\tat org.eclipse.jetty.server.Abstra ctHttpConnection$RequestHandler.content(AbstractHttpConnection.java:998)\r\n\tat org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:856)\r\n\tat org.ec lipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240)\r\n\tat org.ecli pse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)\r \n\tat org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(Socket Connector.java:264)\r\n\tat org.eclipse.jetty.util.thread.QueuedThreadPool.runJo b(QueuedThreadPool.java:608)\r\n\tat org.eclipse.jetty.util.thread.QueuedThreadP ool$3.run(QueuedThreadPool.java:543)\r\n\tat java.lang.Thread.run(Unknown Source )\r\n","code":500}} curl: (6) Could not resolve host: value:set:samsung-111; Host not found even in place of 127.0.0.1 i have tried using localhostbut it still gives this error...im using this from command line in windows On Mon, Feb 4, 2013 at 2:07 PM, Gora Mohanty wrote: > On 4 February 2013 13:29, Rohan Thakur wrote: > > hi arcadius > > > > can you also help me with partial document update...I have followed what > is > > written in this blog but its giving me error > > http://solr.pl/en/2012/07/09/solr-4-0-partial-documents-update/ > > > > error im getting after this command : > > C:\Users\rohan>curl localhost:8983/solr/update?commit=true -H > > Content-type:appli > > cation/json -d > [{"value":"samsung-s5610-gray","value":{"set":samsung-111}}] > [...] > > I am guessing it is a shell-quoting problem. Please follow > the instructions in that blog carefully. You probably need to > double-quote any values that should be strings, e.g., > samsung-111, and need to put the entire argument of > -d within single quotes (do not put any other single quotes > inside this argument unless you escape them properly). > > Regards, > Gora >
Re: solr atomic update
hi were you able to do atomic update for specific field using curlim using curl from windows cmd but getting error like host can not be resolved im using this command: C:\Users\rohan>curl www.localhost.com:8983/solr/update?commit=true -H "Content-t ype:text/json" -d '[{"value":"samsung-wave-s5253-silver", "value":{"set":"samsun g-111"}}]' any help would be greatly appreciated. thanks regards Rohan On Sun, Feb 3, 2013 at 6:16 PM, Erick Erickson wrote: > Not that I know of... > > Best > Erick > > > On Thu, Jan 31, 2013 at 7:09 AM, Marcos Mendez > wrote: > > > Is there a way to do an atomic update (inc by 1) and retrieve the updated > > value in one operation? >
Re: help with getting error with mysql driver connection in solr 4.1
hi I think I have found the problem its windows which is acctualy not able to distinguish between double and single quote and thus curl is trying to resolve the host under double quote individually after -d and thus causing the error but how do I rectify this in windows that is what I am looking for any help would be appreciated. thanks regards rohan On Mon, Feb 4, 2013 at 4:28 PM, Rohan Thakur wrote: > hi gora > > I have tried what you told but now its giving error like: > > C:\Users\rohan>curl 127.0.0.1:8983/solr/update?commit=true -H > "Content-type:appl > ication/json" -d '[{"value":"samsung-wave-s5253-silver", > "value":{"set":"samsung > -111"}}]' > {"responseHeader":{"status":500,"QTime":1},"error":{"msg":"JSON Parse > Error: cha > r=',position=0 BEFORE=''' > AFTER='[{value:samsung-wave-s5253-silver,'","trace":"o > rg.apache.noggit.JSONParser$ParseException: JSON Parse Error: > char=',position=0 > BEFORE=''' AFTER='[{value:samsung-wave-s5253-silver,'\r\n\tat > org.apache.noggit. > JSONParser.err(JSONParser.java:221)\r\n\tat > org.apache.noggit.JSONParser.next(JS > ONParser.java:620)\r\n\tat > org.apache.noggit.JSONParser.nextEvent(JSONParser.jav > a:662)\r\n\tat > org.apache.solr.handler.loader.JsonLoader$SingleThreadedJsonLoade > r.processUpdate(JsonLoader.java:106)\r\n\tat > org.apache.solr.handler.loader.Json > > Loader$SingleThreadedJsonLoader.load(JsonLoader.java:96)\r\n\tat > org.apache.solr > .handler.loader.JsonLoader.load(JsonLoader.java:60)\r\n\tat > org.apache.solr.hand > > ler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:92)\r\n\tat > org.apache > > .solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBas > e.java:74)\r\n\tat > org.apache.solr.handler.RequestHandlerBase.handleRequest(Requ > estHandlerBase.java:135)\r\n\tat > org.apache.solr.core.SolrCore.execute(SolrCore. > java:1816)\r\n\tat > org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispat > chFilter.java:448)\r\n\tat > org.apache.solr.servlet.SolrDispatchFilter.doFilter(S > olrDispatchFilter.java:269)\r\n\tat > org.eclipse.jetty.servlet.ServletHandler$Cac > hedChain.doFilter(ServletHandler.java:1307)\r\n\tat > org.eclipse.jetty.servlet.Se > rvletHandler.doHandle(ServletHandler.java:453)\r\n\tat > org.eclipse.jetty.server. > handler.ScopedHandler.handle(ScopedHandler.java:137)\r\n\tat > org.eclipse.jetty.s > ecurity.SecurityHandler.handle(SecurityHandler.java:560)\r\n\tat > org.eclipse.jet > ty.server.session.SessionHandler.doHandle(SessionHandler.java:231)\r\n\tat > org.e > > > clipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1072)\r\ > n\tat > org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:382)\ > r\n\tat > org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.j > ava:193)\r\n\tat > org.eclipse.jetty.server.handler.ContextHandler.doScope(Context > Handler.java:1006)\r\n\tat > org.eclipse.jetty.server.handler.ScopedHandler.handle > (ScopedHandler.java:135)\r\n\tat > org.eclipse.jetty.server.handler.ContextHandler > Collection.handle(ContextHandlerCollection.java:255)\r\n\tat > org.eclipse.jetty.s > erver.handler.HandlerCollection.handle(HandlerCollection.java:154)\r\n\tat > org.e > > clipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)\r\n\t > > at org.eclipse.jetty.server.Server.handle(Server.java:365)\r\n\tat > org.eclipse.j > > etty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:485 > )\r\n\tat > org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(Blocking > HttpConnection.java:53)\r\n\tat > org.eclipse.jetty.server.AbstractHttpConnection. > content(AbstractHttpConnection.java:937)\r\n\tat > org.eclipse.jetty.server.Abstra > > ctHttpConnection$RequestHandler.content(AbstractHttpConnection.java:998)\r\n\tat > org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:856)\r\n\tat > org.ec > lipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240)\r\n\tat > org.ecli > > > pse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)\r > \n\tat > org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(Socket > Connector.java:264)\r\n\tat > org.eclipse.jetty.util.thread.QueuedThreadPool.runJo > b(QueuedThreadPool.java:608)\r\n\tat > org.eclipse.jetty.util.thread.QueuedThreadP > ool$3.run(QueuedThreadPool.java:543)\r\n\tat java.lang.Thread.run(Unknown > Source > )\r\n","code":500}} > curl: (6) Could not
Re: help with getting error with mysql driver connection in solr 4.1
yup I am downloading cygwin now...will be working through there let see it should work though... On Mon, Feb 4, 2013 at 5:14 PM, Gora Mohanty wrote: > On 4 February 2013 16:58, Rohan Thakur wrote: > > > > hi > > > > I think I have found the problem its windows which is acctualy not able > to > > distinguish between double and single quote and thus curl is trying to > > resolve the host under double quote individually after -d and thus > causing > > the error but how do I rectify this in windows that is what I am looking > > for any help would be appreciated. > [...] > > Hmm, for some reason I thought that you were using Cygwin > rather than the Microsoft Windows CMD client. You would be > better off asking on a WIndows-specific list, but searching > Google for "windows curl quotes" turns up some possibilities. > > Regards, > Gora >
Re: help with getting error with mysql driver connection in solr 4.1
thanks I have instaled cygwin and its running fine now thanks.. On Mon, Feb 4, 2013 at 6:14 PM, Gora Mohanty wrote: > On 4 February 2013 17:50, Rohan Thakur wrote: > > yup I am downloading cygwin now...will be working through there let see > it > > should work though... > [...] > > We are getting highly off-topic now, but f you have RAM > available on the machine, you should seriously consider > running Linux in a VM. > > Regards, > Gora >
auto trigger the delta import to update index in solr if any update in sql database
hi everyone is their any way in which we can auto trigger the delta import to update index in solr if their any update in sql database. thanks regards Rohan
Re: auto trigger the delta import to update index in solr if any update in sql database
that is gud but it will not take care of the field that is dynamical changing like and want to do realtime update for that fieldits not possible to set cron to cal DIH every second... On Tue, Feb 5, 2013 at 5:19 PM, Alexandre Rafalovitch wrote: > If you have your deltaQuery setup in DIH, that should check for updates. > Then you just ping DIH Url periodically to get it to check. > > Regards, >Alex. > > Personal blog: http://blog.outerthoughts.com/ > LinkedIn: http://www.linkedin.com/in/alexandrerafalovitch > - Time is the quality of nature that keeps events from happening all at > once. Lately, it doesn't seem to be working. (Anonymous - via GTD book) > > > On Tue, Feb 5, 2013 at 5:39 AM, Rohan Thakur wrote: > > > hi everyone > > > > is their any way in which we can auto trigger the delta import to update > > index in solr if their any update in sql database. > > > > thanks > > regards > > Rohan > > >
Re: auto trigger the delta import to update index in solr if any update in sql database
hi jp thanks can you provide me any good link for thisthanks regards Rohan On Tue, Feb 5, 2013 at 6:52 PM, jp wrote: > You could use SQL service Broker External Activation service to monitor the > changes and post the changes into the Solr Index using update request > handler in soft commit mode > > --JP > > > > -- > View this message in context: > http://lucene.472066.n3.nabble.com/auto-trigger-the-delta-import-to-update-index-in-solr-if-any-update-in-sql-database-tp4038525p4038554.html > Sent from the Solr - User mailing list archive at Nabble.com. >
Re: auto trigger the delta import to update index in solr if any update in sql database
hi thanks but I think this one is for ms sql not for mysql regards Rohan On Wed, Feb 6, 2013 at 11:53 AM, jp wrote: > The following link provides on using external activator for tracking DB > changes > http://ajitananthram.wordpress.com/2012/05/26/auditing-external-activator/ > > --JP > > > > -- > View this message in context: > http://lucene.472066.n3.nabble.com/auto-trigger-the-delta-import-to-update-index-in-solr-if-any-update-in-sql-database-tp4038525p4038715.html > Sent from the Solr - User mailing list archive at Nabble.com. >
Re: solr file based spell suggestions
hi thanks I configured that using synonym mapping its now giving sII results on searching for s2. thanks regards Rohan On Thu, Feb 7, 2013 at 7:15 PM, Jack Krupansky wrote: > Changing "x" to "y" (e.g., "s2" to "sII") is not a function of "spell > check" or "suggestion". > > Synonyms are a closer match, but can be difficult to configure properly. > Good luck. > > You may be better off preprocessing the query at the application level and > then generating the appropriate boolean logic, such as: "(s2 OR sII)". > > -- Jack Krupansky > > -Original Message- From: Rohan Thakur > Sent: Thursday, February 07, 2013 8:24 AM > To: solr-user@lucene.apache.org > Subject: solr file based spell suggestions > > > hi all > > I wanted to know how can I apply file based dictionary for spell > suggestions such that if I search for s2 in the query it would take it as > sII which also represent same thing in my indexed field...but as in search > it can also be interpreted as s2 please help anyone... > > thanks > regards > Rohan >
Re: html parsing with dataimport handler from sql column
hi yes I dont have html as documents I have data saved in sql data base in HTML format and I want to index it on solr but not as complete string that is with tags but just want to index the actual text in it...that is strip off the tags. regards Rohan On Wed, Feb 20, 2013 at 6:40 PM, Gora Mohanty wrote: > On 20 February 2013 18:31, Rohan Thakur wrote: > > hi all > > > > I have data stored in HTML format in a column in sql database and want to > > index the data from that field to solr how can I do that any one has idea > > please help. right now i am treating it as a string which is indexing > > complete HTML with tags as one string to solr. > > How do you want to process the HTML? If you simply want to > strip HTML tags, please take a look at the HTMLStripTransformer > http://wiki.apache.org/solr/DataImportHandler#HTMLStripTransformer > > Your title implies that you want to parse the HTML in some > fashion. If so, you will need to do that on your own, e.g., by > using a transformer. > > Regards, > Gora >
Re: Is their a way in which I can make spell suggestion dictionary build on specific fileds
thanks On Thu, Feb 21, 2013 at 9:41 PM, Jack Krupansky wrote: > Yes, each spellchecker (or "dictionary") in your spellcheck search > component has a "field" parameter to specify the field to be used to > generate the dictionary index for that spellchecker: > > spell > > See the Solr example solrconfig.xml and search for name="spellchecker">. > > Also see: > http://wiki.apache.org/solr/**SpellCheckComponent<http://wiki.apache.org/solr/SpellCheckComponent> > > -- Jack Krupansky > > -Original Message- From: Rohan Thakur > Sent: Thursday, February 21, 2013 2:34 AM > To: solr-user@lucene.apache.org > Subject: Is their a way in which I can make spell suggestion dictionary > build on specific fileds > > > hi all > > I wanted to know is their a way in which I have select on which indexed > field I want to build the spell suggestions dictionary? > > thanks > regards > Rohan >
Re: solr search showing irrelevant results
I am using "text_en_splitting" as while indexing is that the problem?? On Mon, Mar 4, 2013 at 7:02 PM, Rohan Thakur wrote: > hi all > > I wanted to know that why solr is showing irrelevant result as I search > for "galaxy ace" its showing result "sony bravia" that does not have either > of them galaxy or ace in the result but way down the order why is it doing > so any Idea please help I do not want that to happen. > > thanks in advance > > regards > Rohan >
Re: solr search showing irrelevant results
thanks I got the problem it was with using *text_en_splitting* filed type for indexingwhich actually includes fuzzy results aswell...but I dont know it will take fuzzy results upto this extentnow im using *text_en_splitting_tight *and its giving correct results without fuzzy results On Tue, Mar 5, 2013 at 2:32 AM, Jack Krupansky wrote: > Add &debugQuery=true to your query and look at the "explain" section for > details of why a document was scored as it was. > > Also look at the parsed query to see what fields it is actually searching. > > -- Jack Krupansky > > -Original Message- From: Rohan Thakur > Sent: Monday, March 04, 2013 8:32 AM > To: solr-user@lucene.apache.org > Subject: solr search showing irrelevant results > > > hi all > > I wanted to know that why solr is showing irrelevant result as I search for > "galaxy ace" its showing result "sony bravia" that does not have either of > them galaxy or ace in the result but way down the order why is it doing so > any Idea please help I do not want that to happen. > > thanks in advance > > regards > Rohan >
Re: query about length normalization in solr for relevance score
k thanks On Wed, Mar 6, 2013 at 5:05 PM, Otis Gospodnetic wrote: > Hi Rohan, > > It's per field. > > Otis > -- > Solr & ElasticSearch Support > http://sematext.com/ > > > > > > On Wed, Mar 6, 2013 at 3:09 AM, Rohan Thakur wrote: > > > hi all > > > > I wanted to confirm that while taking length normalization of document > into > > count while calculating the scores of the document in solr does solr take > > length of complete document or the particular field in the document that > we > > are querying on. Like I have 5 fields in each document but when I am > > querying I am searching on one default single field so I wanted to know > > does solr not takes the length of this single field in to count for the > > document or total length of the document that is adding length of all the > > fields in that document. > > > > thanks in advance. > > > > regards > > Rohan > > >
Re: query about length normalization in solr for relevance score
one more thing how does the solr handles the score when we have multiple words as query parameters which are not consecutively present in the filed that is indexed and we are querying on does it first searches for document field containing both the term and then how does is find the scoring on them taking both the words collectively or separating them and adding the scores and if adding the scores then how does is find scores individually? On Wed, Mar 6, 2013 at 5:17 PM, Rohan Thakur wrote: > k thanks > > > On Wed, Mar 6, 2013 at 5:05 PM, Otis Gospodnetic < > otis.gospodne...@gmail.com> wrote: > >> Hi Rohan, >> >> It's per field. >> >> Otis >> -- >> Solr & ElasticSearch Support >> http://sematext.com/ >> >> >> >> >> >> On Wed, Mar 6, 2013 at 3:09 AM, Rohan Thakur >> wrote: >> >> > hi all >> > >> > I wanted to confirm that while taking length normalization of document >> into >> > count while calculating the scores of the document in solr does solr >> take >> > length of complete document or the particular field in the document >> that we >> > are querying on. Like I have 5 fields in each document but when I am >> > querying I am searching on one default single field so I wanted to know >> > does solr not takes the length of this single field in to count for the >> > document or total length of the document that is adding length of all >> the >> > fields in that document. >> > >> > thanks in advance. >> > >> > regards >> > Rohan >> > >> > >
Re: query about length normalization in solr for relevance score
346 = score(doc=15693,freq=1.0 = termFreq=1.0\n), product of:\n 0.6119956 = queryWeight, product of:\n 8.345795 = idf(docFreq=38, maxDocs=60437)\n 0.073329814 = queryNorm\n 3.129673 = fieldWeight in 15693, product of:\n 1.0 = tf(freq=1.0), with freq of:\n1.0 = termFreq=1.0\n 8.345795 = idf(docFreq=38, maxDocs=60437)\n 0.375 = fieldNorm(doc=15693)\n 0.667 = coord(2/3)\n", "":"\n1.9359072 = (MATCH) product of:\n 2.9038608 = (MATCH) sum of:\n0.98851484 = (MATCH) weight(value:samsung in 15749) [DefaultSimilarity], result of:\n 0.98851484 = score(doc=15749,freq=1.0 = termFreq=1.0\n), product of:\n 0.4396593 = queryWeight, product of:\n 5.9956417 = idf(docFreq=408, maxDocs=60437)\n 0.073329814 = queryNorm\n 2.2483656 = fieldWeight in 15749, product of:\n 1.0 = tf(freq=1.0), with freq of:\n1.0 = termFreq=1.0\n 5.9956417 = idf(docFreq=408, maxDocs=60437)\n 0.375 = fieldNorm(doc=15749)\n1.915346 = (MATCH) weight(value:metallic in 15749) [DefaultSimilarity], result of:\n 1.915346 = score(doc=15749,freq=1.0 = termFreq=1.0\n), product of:\n 0.6119956 = queryWeight, product of:\n 8.345795 = idf(docFreq=38, maxDocs=60437)\n 0.073329814 = queryNorm\n 3.129673 = fieldWeight in 15749, product of:\n 1.0 = tf(freq=1.0), with freq of:\n1.0 = termFreq=1.0\n 8.345795 = idf(docFreq=38, maxDocs=60437)\n 0.375 = fieldNorm(doc=15749)\n 0.667 = coord(2/3)\n", "":"\n1.9359072 = (MATCH) product of:\n 2.9038608 = (MATCH) sum of:\n0.98851484 = (MATCH) weight(value:samsung in 16803) [DefaultSimilarity], result of:\n 0.98851484 = score(doc=16803,freq=1.0 = termFreq=1.0\n), product of:\n 0.4396593 = queryWeight, product of:\n 5.9956417 = idf(docFreq=408, maxDocs=60437)\n 0.073329814 = queryNorm\n 2.2483656 = fieldWeight in 16803, product of:\n 1.0 = tf(freq=1.0), with freq of:\n1.0 = termFreq=1.0\n 5.9956417 = idf(docFreq=408, maxDocs=60437)\n 0.375 = fieldNorm(doc=16803)\n1.915346 = (MATCH) weight(value:metallic in 16803) [DefaultSimilarity], result of:\n 1.915346 = score(doc=16803,freq=1.0 = termFreq=1.0\n), product of:\n 0.6119956 = queryWeight, product of:\n 8.345795 = idf(docFreq=38, maxDocs=60437)\n 0.073329814 = queryNorm\n 3.129673 = fieldWeight in 16803, product of:\n 1.0 = tf(freq=1.0), with freq of:\n1.0 = termFreq=1.0\n 8.345795 = idf(docFreq=38, maxDocs=60437)\n 0.375 = fieldNorm(doc=16803)\n 0.667 = coord(2/3)\n", "":"\n1.9359072 = (MATCH) product of:\n 2.9038608 = (MATCH) sum of:\n0.98851484 = (MATCH) weight(value:samsung in 21574) [DefaultSimilarity], result of:\n 0.98851484 = score(doc=21574,freq=1.0 = termFreq=1.0\n), product of:\n 0.4396593 = queryWeight, product of:\n 5.9956417 = idf(docFreq=408, maxDocs=60437)\n 0.073329814 = queryNorm\n 2.2483656 = fieldWeight in 21574, product of:\n 1.0 = tf(freq=1.0), with freq of:\n1.0 = termFreq=1.0\n 5.9956417 = idf(docFreq=408, maxDocs=60437)\n 0.375 = fieldNorm(doc=21574)\n1.915346 = (MATCH) weight(value:metallic in 21574) [DefaultSimilarity], result of:\n 1.915346 = score(doc=21574,freq=1.0 = termFreq=1.0\n), product of:\n 0.6119956 = queryWeight, product of:\n 8.345795 = idf(docFreq=38, maxDocs=60437)\n 0.073329814 = queryNorm\n 3.129673 = fieldWeight in 21574, product of:\n 1.0 = tf(freq=1.0), with freq of:\n1.0 = termFreq=1.0\n 8.345795 = idf(docFreq=38, maxDocs=60437)\n 0.375 = fieldNorm(doc=21574)\n 0.667 = coord(2/3)\n"}, thanks in advance regards Rohan On Wed, Mar 6, 2013 at 10:14 PM, Otis Gospodnetic < otis.gospodne...@gmail.com> wrote: > That's a mighty long sentence, Rohan! :) > > If you append &debugQuery=true to the query URL you will see the scoring > explanation in all its glory. > > Otis > -- > Solr & ElasticSearch Support > http://sematext.com/ > > > > > > On Wed, Mar 6, 2013 at 6:55 AM, Rohan Thakur wrote: > > > one more thing how does the solr handles the score when we have multiple > > words as query parameters which are not consecutively present in the > filed > > that is indexed and we are querying on does it first searches for > document > > field containing both the term and then how does is find the scoring on > > them taking both the words collectively or separating them and adding the > > scores and if adding the scores then how does is find scores > individually? > > > > On Wed, Mar 6, 2013 at 5:17 PM, Rohan Thakur > wrote: > > > > > k thanks > > > > > > > > > O
Re: query about length normalization in solr for relevance score
hi I think its not readable but if you open this in editor that gmail has like while replying you can se the text in proper format that is indent.I tried saving this output in other formats aswell but it is still the same...you can see this properly while replying to the mail... thanks regards Rohan On Thu, Mar 7, 2013 at 11:48 AM, Rohan Thakur wrote: > yup I got that but can you please explain me how is it going on? > I got this: > > "rawquerystring":"samsung ace metallic", > "querystring":"samsung ace metallic", > "parsedquery":"value:samsung value:ace value:metallic", > "parsedquery_toString":"value:samsung value:ace value:metallic", > "explain":{ > "":"\n4.261568 = (MATCH) sum of:\n 0.82376236 = (MATCH) > weight(value:samsung in 31646) [DefaultSimilarity], result of:\n > 0.82376236 = score(doc=31646,freq=1.0 = termFreq=1.0\n), product of:\n > 0.4396593 = queryWeight, product of:\n5.9956417 = idf(docFreq=408, > maxDocs=60437)\n0.073329814 = queryNorm\n 1.873638 = fieldWeight > in 31646, product of:\n1.0 = tf(freq=1.0), with freq of:\n > 1.0 = termFreq=1.0\n5.9956417 = idf(docFreq=408, maxDocs=60437)\n >0.3125 = fieldNorm(doc=31646)\n 1.8416842 = (MATCH) weight(value:ace in > 31646) [DefaultSimilarity], result of:\n1.8416842 = > score(doc=31646,freq=1.0 = termFreq=1.0\n), product of:\n 0.65738964 = > queryWeight, product of:\n8.964834 = idf(docFreq=20, maxDocs=60437)\n >0.073329814 = queryNorm\n 2.8015108 = fieldWeight in 31646, > product of:\n1.0 = tf(freq=1.0), with freq of:\n 1.0 = > termFreq=1.0\n8.964834 = idf(docFreq=20, maxDocs=60437)\n > 0.3125 = fieldNorm(doc=31646)\n 1.5961217 = (MATCH) weight(value:metallic in > 31646) [DefaultSimilarity], result of:\n1.5961217 = > score(doc=31646,freq=1.0 = termFreq=1.0\n), product of:\n 0.6119956 = > queryWeight, product of:\n8.345795 = idf(docFreq=38, maxDocs=60437)\n >0.073329814 = queryNorm\n 2.6080608 = fieldWeight in 31646, > product of:\n1.0 = tf(freq=1.0), with freq of:\n 1.0 = > termFreq=1.0\n8.345795 = idf(docFreq=38, maxDocs=60437)\n > 0.3125 = fieldNorm(doc=31646)\n", > "":"\n4.261568 = (MATCH) sum of:\n 0.82376236 = (MATCH) > weight(value:samsung in 31701) [DefaultSimilarity], result of:\n > 0.82376236 = score(doc=31701,freq=1.0 = termFreq=1.0\n), product of:\n > 0.4396593 = queryWeight, product of:\n5.9956417 = idf(docFreq=408, > maxDocs=60437)\n0.073329814 = queryNorm\n 1.873638 = fieldWeight > in 31701, product of:\n1.0 = tf(freq=1.0), with freq of:\n > 1.0 = termFreq=1.0\n5.9956417 = idf(docFreq=408, maxDocs=60437)\n >0.3125 = fieldNorm(doc=31701)\n 1.8416842 = (MATCH) weight(value:ace in > 31701) [DefaultSimilarity], result of:\n1.8416842 = > score(doc=31701,freq=1.0 = termFreq=1.0\n), product of:\n 0.65738964 = > queryWeight, product of:\n8.964834 = idf(docFreq=20, maxDocs=60437)\n >0.073329814 = queryNorm\n 2.8015108 = fieldWeight in 31701, > product of:\n1.0 = tf(freq=1.0), with freq of:\n 1.0 = > termFreq=1.0\n8.964834 = idf(docFreq=20, maxDocs=60437)\n > 0.3125 = fieldNorm(doc=31701)\n 1.5961217 = (MATCH) weight(value:metallic in > 31701) [DefaultSimilarity], result of:\n1.5961217 = > score(doc=31701,freq=1.0 = termFreq=1.0\n), product of:\n 0.6119956 = > queryWeight, product of:\n8.345795 = idf(docFreq=38, maxDocs=60437)\n >0.073329814 = queryNorm\n 2.6080608 = fieldWeight in 31701, > product of:\n1.0 = tf(freq=1.0), with freq of:\n 1.0 = > termFreq=1.0\n8.345795 = idf(docFreq=38, maxDocs=60437)\n > 0.3125 = fieldNorm(doc=31701)\n", > "":"\n2.843143 = (MATCH) product of:\n 4.2647142 = (MATCH) sum of:\n > 1.3180199 = (MATCH) weight(value:samsung in 1646) [DefaultSimilarity], > result of:\n 1.3180199 = score(doc=1646,freq=1.0 = termFreq=1.0\n), > product of:\n0.4396593 = queryWeight, product of:\n > 5.9956417 = idf(docFreq=408, maxDocs=60437)\n 0.073329814 = > queryNorm\n2.9978209 = fieldWeight in 1646, product of:\n > 1.0 = tf(freq=1.0), with freq of:\n1.0 = termFreq=1.0\n > 5.9956417 = idf(docFreq=408, maxDocs=60437)\n 0.5 = > fieldNorm(doc=1646)\n2.9466946 = (MATCH) weight(value:ace in 1646) > [DefaultSimilarity], result of:\n 2.9466946 = sc
Re: query about length normalization in solr for relevance score
If you can explain any one line of I will figure out othersthanks On Thu, Mar 7, 2013 at 11:48 AM, Rohan Thakur wrote: > yup I got that but can you please explain me how is it going on? > I got this: > > "rawquerystring":"samsung ace metallic", > "querystring":"samsung ace metallic", > "parsedquery":"value:samsung value:ace value:metallic", > "parsedquery_toString":"value:samsung value:ace value:metallic", > "explain":{ > "":"\n4.261568 = (MATCH) sum of:\n 0.82376236 = (MATCH) > weight(value:samsung in 31646) [DefaultSimilarity], result of:\n > 0.82376236 = score(doc=31646,freq=1.0 = termFreq=1.0\n), product of:\n > 0.4396593 = queryWeight, product of:\n5.9956417 = idf(docFreq=408, > maxDocs=60437)\n0.073329814 = queryNorm\n 1.873638 = fieldWeight > in 31646, product of:\n1.0 = tf(freq=1.0), with freq of:\n > 1.0 = termFreq=1.0\n5.9956417 = idf(docFreq=408, maxDocs=60437)\n >0.3125 = fieldNorm(doc=31646)\n 1.8416842 = (MATCH) weight(value:ace in > 31646) [DefaultSimilarity], result of:\n1.8416842 = > score(doc=31646,freq=1.0 = termFreq=1.0\n), product of:\n 0.65738964 = > queryWeight, product of:\n8.964834 = idf(docFreq=20, maxDocs=60437)\n >0.073329814 = queryNorm\n 2.8015108 = fieldWeight in 31646, > product of:\n1.0 = tf(freq=1.0), with freq of:\n 1.0 = > termFreq=1.0\n8.964834 = idf(docFreq=20, maxDocs=60437)\n > 0.3125 = fieldNorm(doc=31646)\n 1.5961217 = (MATCH) weight(value:metallic in > 31646) [DefaultSimilarity], result of:\n1.5961217 = > score(doc=31646,freq=1.0 = termFreq=1.0\n), product of:\n 0.6119956 = > queryWeight, product of:\n8.345795 = idf(docFreq=38, maxDocs=60437)\n >0.073329814 = queryNorm\n 2.6080608 = fieldWeight in 31646, > product of:\n1.0 = tf(freq=1.0), with freq of:\n 1.0 = > termFreq=1.0\n8.345795 = idf(docFreq=38, maxDocs=60437)\n > 0.3125 = fieldNorm(doc=31646)\n", > "":"\n4.261568 = (MATCH) sum of:\n 0.82376236 = (MATCH) > weight(value:samsung in 31701) [DefaultSimilarity], result of:\n > 0.82376236 = score(doc=31701,freq=1.0 = termFreq=1.0\n), product of:\n > 0.4396593 = queryWeight, product of:\n5.9956417 = idf(docFreq=408, > maxDocs=60437)\n0.073329814 = queryNorm\n 1.873638 = fieldWeight > in 31701, product of:\n1.0 = tf(freq=1.0), with freq of:\n > 1.0 = termFreq=1.0\n5.9956417 = idf(docFreq=408, maxDocs=60437)\n >0.3125 = fieldNorm(doc=31701)\n 1.8416842 = (MATCH) weight(value:ace in > 31701) [DefaultSimilarity], result of:\n1.8416842 = > score(doc=31701,freq=1.0 = termFreq=1.0\n), product of:\n 0.65738964 = > queryWeight, product of:\n8.964834 = idf(docFreq=20, maxDocs=60437)\n >0.073329814 = queryNorm\n 2.8015108 = fieldWeight in 31701, > product of:\n1.0 = tf(freq=1.0), with freq of:\n 1.0 = > termFreq=1.0\n8.964834 = idf(docFreq=20, maxDocs=60437)\n > 0.3125 = fieldNorm(doc=31701)\n 1.5961217 = (MATCH) weight(value:metallic in > 31701) [DefaultSimilarity], result of:\n1.5961217 = > score(doc=31701,freq=1.0 = termFreq=1.0\n), product of:\n 0.6119956 = > queryWeight, product of:\n8.345795 = idf(docFreq=38, maxDocs=60437)\n >0.073329814 = queryNorm\n 2.6080608 = fieldWeight in 31701, > product of:\n1.0 = tf(freq=1.0), with freq of:\n 1.0 = > termFreq=1.0\n8.345795 = idf(docFreq=38, maxDocs=60437)\n > 0.3125 = fieldNorm(doc=31701)\n", > "":"\n2.843143 = (MATCH) product of:\n 4.2647142 = (MATCH) sum of:\n > 1.3180199 = (MATCH) weight(value:samsung in 1646) [DefaultSimilarity], > result of:\n 1.3180199 = score(doc=1646,freq=1.0 = termFreq=1.0\n), > product of:\n0.4396593 = queryWeight, product of:\n > 5.9956417 = idf(docFreq=408, maxDocs=60437)\n 0.073329814 = > queryNorm\n2.9978209 = fieldWeight in 1646, product of:\n > 1.0 = tf(freq=1.0), with freq of:\n1.0 = termFreq=1.0\n > 5.9956417 = idf(docFreq=408, maxDocs=60437)\n 0.5 = > fieldNorm(doc=1646)\n2.9466946 = (MATCH) weight(value:ace in 1646) > [DefaultSimilarity], result of:\n 2.9466946 = score(doc=1646,freq=1.0 = > termFreq=1.0\n), product of:\n0.65738964 = queryWeight, product of:\n > 8.964834 = idf(docFreq=20, maxDocs=60437)\n 0.073329814 = > queryNorm\n4.482417 = fieldWeight in 1646, produc
Re: is there a way we can build spell dictionary from solr index such that it only take words leaving all`special characters
k as in like if the field I am indixing from the database like title that has characters like () - # /n// example: Screenguard for Samsung Galaxy SII (Matt and Gloss) (with Dual Protection, Cleaning Cloth and Bubble Remover) or samsung-galaxy-sii-screenguard-matt-and-gloss.html or /s/a/samsung_galaxy_sii_i9100_pink_.jpg or 4.27-inch Touchscreen, 3G, Android v2.3 OS, 8MP Camera with LED Flash now I do not want to build the spell dictionary to only include the words not any of the - , _ . ( ) /s/a/ or numeric like 4.27 how can I do that? thanks regards Rohan On Tue, Mar 12, 2013 at 11:06 PM, Alexandre Rafalovitch wrote: > Sorry, leaving them where? > > Can you give a concrete example or problem. > > Regards, > Alex > On Mar 12, 2013 1:31 PM, "Rohan Thakur" wrote: > > > hi all > > > > wanted to know is there way we can make spell dictionary from solr index > > such that it only takes words from the index leaving all the special > > characters and unwanted characters. > > > > thanks > > regards > > Rohan > > >
Re: is there a way we can build spell dictionary from solr index such that it only take words leaving all`special characters
even do not want to break the words as in samsung to s a m s u n g or sII ti s II ir s2 to s 2 On Wed, Mar 13, 2013 at 11:28 AM, Rohan Thakur wrote: > k as in like if the field I am indixing from the database like title that > has characters like () - # /n// > example: > > Screenguard for Samsung Galaxy SII (Matt and Gloss) (with Dual Protection, > Cleaning Cloth and Bubble Remover) > > or > samsung-galaxy-sii-screenguard-matt-and-gloss.html > or > /s/a/samsung_galaxy_sii_i9100_pink_.jpg > or > 4.27-inch Touchscreen, 3G, Android v2.3 OS, 8MP Camera with LED Flash > > now I do not want to build the spell dictionary to only include the words > not any of the - , _ . ( ) /s/a/ or numeric like 4.27.... > how can I do that? > > thanks > regards > Rohan > > On Tue, Mar 12, 2013 at 11:06 PM, Alexandre Rafalovitch < > arafa...@gmail.com> wrote: > >> Sorry, leaving them where? >> >> Can you give a concrete example or problem. >> >> Regards, >> Alex >> On Mar 12, 2013 1:31 PM, "Rohan Thakur" wrote: >> >> > hi all >> > >> > wanted to know is there way we can make spell dictionary from solr index >> > such that it only takes words from the index leaving all the special >> > characters and unwanted characters. >> > >> > thanks >> > regards >> > Rohan >> > >> > >
Re: is there a way we can build spell dictionary from solr index such that it only take words leaving all`special characters
while building the spell dictionary... On Wed, Mar 13, 2013 at 11:29 AM, Rohan Thakur wrote: > even do not want to break the words as in samsung to s a m s u n g or sII > ti s II ir s2 to s 2 > > On Wed, Mar 13, 2013 at 11:28 AM, Rohan Thakur wrote: > >> k as in like if the field I am indixing from the database like title that >> has characters like () - # /n// >> example: >> >> Screenguard for Samsung Galaxy SII (Matt and Gloss) (with Dual Protection, >> Cleaning Cloth and Bubble Remover) >> >> or >> samsung-galaxy-sii-screenguard-matt-and-gloss.html >> or >> /s/a/samsung_galaxy_sii_i9100_pink_.jpg >> or >> 4.27-inch Touchscreen, 3G, Android v2.3 OS, 8MP Camera with LED Flash >> >> now I do not want to build the spell dictionary to only include the words >> not any of the - , _ . ( ) /s/a/ or numeric like 4.27 >> how can I do that? >> >> thanks >> regards >> Rohan >> >> On Tue, Mar 12, 2013 at 11:06 PM, Alexandre Rafalovitch < >> arafa...@gmail.com> wrote: >> >>> Sorry, leaving them where? >>> >>> Can you give a concrete example or problem. >>> >>> Regards, >>> Alex >>> On Mar 12, 2013 1:31 PM, "Rohan Thakur" wrote: >>> >>> > hi all >>> > >>> > wanted to know is there way we can make spell dictionary from solr >>> index >>> > such that it only takes words from the index leaving all the special >>> > characters and unwanted characters. >>> > >>> > thanks >>> > regards >>> > Rohan >>> > >>> >> >> >
Re: removing all fields before full import using DIH
k thanks yes I dint checked it before I was using DIH full import directly and one day I observed that my solr search was giving duplicate results then I deleted all the entries and re index the dataand after that for ensure that this does not happen I always use delete first then do full import...k so this automatically does that... thanks for confirming. regards Rohan On Mon, Mar 18, 2013 at 1:32 PM, Gora Mohanty wrote: > On 18 March 2013 13:09, Rohan Thakur wrote: > > hi all > > > > how can I ensure that I have delete all the fields for solr before doing > > full import in DIH only? the aim is that my database is pretty small so > > full import takes only 3-4 sec. thus I do not require delta import for > now > > and I want to ensure that when ever I do full import of the database the > > duplicate field do not get indexed that is multiple instances of same > > document does not get indexed so want to delete all the documents first > and > > then reindex using full import. any one help. > > Have you tried a full-import? What you want is done by default by > DIH, unless one specifies clean=false as a query parameter to the > full-import URL. > > Regards, > Gora >
Re: wanted to know can we some how know what are the documents that are exact match in solr
yes that I know but I want to know that is there way I can separate them in search results...the exact match one? On Fri, Mar 15, 2013 at 10:18 PM, Jack Krupansky wrote: > The "explain" section that is returned if you specify the &debugQuery=true > parameter will provides the details of what terms matched for each document. > > -- Jack Krupansky > > -Original Message- From: Rohan Thakur > Sent: Friday, March 15, 2013 9:19 AM > To: solr-user@lucene.apache.org > Subject: wanted to know can we some how know what are the documents that > are exact match in solr > > > hi all > > I need to pass some variable or some flag with the exact match document > than the others like say I have 3 terms in the search query so I need to > know the documents in which all three words are found from the other > documents in which only 1 or 2 out of three terms are matched. > > any help would be great > thanks > regards > rohan >
Re: had query regarding the indexing and analysers
my default is title only I have used debug as well it shows that solr divides the query into dual and core and then searches both separately now while calculating the scores it puts the document in which both the terms appear and in my case the document containing this title: Wipro 7710U Laptop-DUAL CORE 1.4 Ghz-120GB HDD solr has found only core term not dual as I guess it is attached to laptop term not as even searching for only dual term this document doesnot show up which is why this document sshows down in the search results thus I am not able to search for partial terms for that I have to apply *dual in the query then it is searching this document but then other search scoring gets affected with this when I put * in the query terms I think I have to remove the "-" terms from the strings before indexing them point me if i am wrong any where thanks regards Rohan On Sat, Mar 16, 2013 at 7:02 PM, Erick Erickson wrote: > See admin/analysis, it's invaluable. Probably > > The terms are being searched against your default text field which I'd > guess is not "title". > > Also, try adding &debug=all to your query and look in the debug info at the > parsed form of the query to see what's actually being searched. > > Best > Erick > > > On Fri, Mar 15, 2013 at 2:52 AM, Rohan Thakur > wrote: > > > hi all > > > > wanted to know I have this string in field title : > > > > Wipro 7710U Laptop-DUAL CORE 1.4 Ghz-120GB HDD > > > > I have indexed it using text-en-splliting-tight > > > > > > and now I am searching for term like q=dual core > > > > but in the relevance part its this title is coming down the order as > > solr is not searching dual in this string its just searching core term > > from the query in this string thus multiplying the score for this field > by > > 1/2 > > decreasing the score. > > > > how can I correct this can any one help > > > > thanks > > regards > > Rohan > > >
Re: had query regarding the indexing and analysers
431, product of:\n1.0 = tf(freq=1.0), with freq of:\n 1.0 = termFreq=1.0\n4.2628927 = idf(docFreq=39, maxDocs=1045)\n0.375 = fieldNorm(doc=431)\n", "":"\n1.5985848 = (MATCH) weight(title:ac^30.0 in 433) [DefaultSimilarity], result of:\n 1.5985848 = fieldWeight in 433, product of:\n1.0 = tf(freq=1.0), with freq of:\n 1.0 = termFreq=1.0\n4.2628927 = idf(docFreq=39, maxDocs=1045)\n0.375 = fieldNorm(doc=433)\n", "":"\n1.5985848 = (MATCH) weight(title:ac^30.0 in 434) [DefaultSimilarity], result of:\n 1.5985848 = fieldWeight in 434, product of:\n1.0 = tf(freq=1.0), with freq of:\n 1.0 = termFreq=1.0\n4.2628927 = idf(docFreq=39, maxDocs=1045)\n0.375 = fieldNorm(doc=434)\n", "":"\n1.5985848 = (MATCH) weight(title:ac^30.0 in 502) [DefaultSimilarity], result of:\n 1.5985848 = fieldWeight in 502, product of:\n1.0 = tf(freq=1.0), with freq of:\n 1.0 = termFreq=1.0\n4.2628927 = idf(docFreq=39, maxDocs=1045)\n0.375 = fieldNorm(doc=502)\n", "":"\n1.332154 = (MATCH) weight(title:ac^30.0 in 411) [DefaultSimilarity], result of:\n 1.332154 = fieldWeight in 411, product of:\n1.0 = tf(freq=1.0), with freq of:\n 1.0 = termFreq=1.0\n4.2628927 = idf(docFreq=39, maxDocs=1045)\n 0.3125 = fieldNorm(doc=411)\n", "":"\n1.332154 = (MATCH) weight(title:ac^30.0 in 424) [DefaultSimilarity], result of:\n 1.332154 = fieldWeight in 424, product of:\n1.0 = tf(freq=1.0), with freq of:\n 1.0 = termFreq=1.0\n4.2628927 = idf(docFreq=39, maxDocs=1045)\n 0.3125 = fieldNorm(doc=424)\n"}, "QParser":"ExtendedDismaxQParser", On Tue, Mar 19, 2013 at 7:37 PM, Jack Krupansky wrote: > Yeah, one ambiguity in typography is whether a hyphen is internal to a > compound term (e.g., "CD-ROM") or a phrase separator as in your case. Some > people are careful to put spaces around the hyphen for a phrase delimiter, > but plenty of people still just drop it in directly adjacent to two words. > > In your case, text_en_splitting_tight is SPECIFICALLY trying to keep > "Laptop-DUAL" together as a single term, so that "wi fi" is kept distinct > from "Wi-Fi". > > Try text_en_splitting, which specifically is NOT trying to keep them > together. > > The key clue here is that the former does not have generateWordParts="1". > That is the option that is needed so that "Laptop-DUAL" will be indexed as > "laptop dual". > > -- Jack Krupansky > > -Original Message- From: Rohan Thakur > Sent: Tuesday, March 19, 2013 3:35 AM > To: solr-user@lucene.apache.org > Subject: Re: had query regarding the indexing and analysers > > > my default is title only I have used debug as well it shows that solr > divides the query into dual and core and then searches both separately now > while calculating the scores it puts the document in which both the terms > appear and in my case the document containing this title: > > Wipro 7710U Laptop-DUAL CORE 1.4 Ghz-120GB HDD > > solr has found only core term not dual as I guess it is > attached to laptop term not as even searching for only dual > term this document doesnot show up which is why this document > sshows down in the search results thus I am not able to > search for partial terms for that I have to apply *dual > in the query then it is searching this document but then > other search scoring gets affected with this when I put * in > the query terms I think I have to remove the "-" terms from > the strings before indexing them point me if i am wrong any > where > > thanks > regards > Rohan > > > On Sat, Mar 16, 2013 at 7:02 PM, Erick Erickson * > *wrote: > > See admin/analysis, it's invaluable. Probably >> >> The terms are being searched against your default text field which I'd >> guess is not "title". >> >> Also, try adding &debug=all to your query and look in the debug info at >> the >> parsed form of the query to see what's actually being searched. >> >> Best >> Erick >> >> >> On Fri, Mar 15, 2013 at 2:52 AM, Rohan Thakur >> wrote: >> >> > hi all >> > >> > wanted to know I have this string in field title : >> > >> > Wipro 7710U Laptop-DUAL CORE 1.4 Ghz-120GB HDD >> > >> > I have indexed it using text-en-splliting-tight >> > >> > >> > and now I am searching for term like q=dual core >> > >> > but in the relevance part its this title is coming down the order as >> > solr is not searching dual in this string its just searching core term >> > from the query in this string thus multiplying the score for this field >> by >> > 1/2 >> > decreasing the score. >> > >> > how can I correct this can any one help >> > >> > thanks >> > regards >> > Rohan >> > >> >> >
Solr for finding similar word between two documents
Is there a way to use solr and get similar words between two document (files). Any ideas Regards Rohan
Re: Solr for finding similar word between two documents
Lets say 1 have document(file) which is large and contains word inside it. And the 2nd document also is a text file. Problem is to find all those words in 2nd document which is present in first document when both of the files are large enough. Regards Rohan On Fri, Feb 4, 2011 at 1:01 AM, openvictor Open wrote: > Rohan : what you want to do can be done with quite little effort if your > document has a limited size (up to some Mo) with common and basic > structures > like Hasmap. > > Do you have any additional information on your problem so that we can give > you more useful inputs ? > > 2011/2/3 Gora Mohanty > > > On Thu, Feb 3, 2011 at 11:32 PM, rohan rai wrote: > > > Is there a way to use solr and get similar words between two document > > > (files). > > [...] > > > > This is *way* too vague t make any sense out of. Could you elaborate, > > as I could have sworn that what you seem to want is the essential > > function of a search engine. > > > > Regards, > > Gora > > >
DIH and EmbeddedSolr
Have been unable to use DIH for Embedded Solr Is there a way?? Regards Rohan
Re: DIH and EmbeddedSolr
The configuration is not an issue... But how doindex i invoke it... I only have known a url way to invoke it and thus import the data into index... like http://localhost:8983/solr/db/dataimport?command=full-import t But with embedded I havent been able to figure it out Regards Rohan 2009/10/10 Noble Paul നോബിള് नोब्ळ् > I guess it should be possible... what are the problems you encounter? > > On Sat, Oct 10, 2009 at 10:56 AM, rohan rai wrote: > > Have been unable to use DIH for Embedded Solr > > > > Is there a way?? > > > > Regards > > Rohan > > > > > > -- > - > Noble Paul | Principal Engineer| AOL | http://aol.com >
Re: DIH and EmbeddedSolr
ModifiableSolrParams p = new ModifiableSolrParams(); p.add("qt", "/dataimport"); p.add("command", "full-import"); server.query(p, METHOD.POST); I do this But it starts giving me this exception SEVERE: Full Import failed java.util.concurrent.RejectedExecutionException at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:1760) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:767) at java.util.concurrent.ScheduledThreadPoolExecutor.delayedExecute(ScheduledThreadPoolExecutor.java:216) at java.util.concurrent.ScheduledThreadPoolExecutor.schedule(ScheduledThreadPoolExecutor.java:366) at org.apache.solr.update.DirectUpdateHandler2$CommitTracker.scheduleCommitWithin(DirectUpdateHandler2.java:466) at org.apache.solr.update.DirectUpdateHandler2.deleteByQuery(DirectUpdateHandler2.java:322) at org.apache.solr.update.processor.RunUpdateProcessor.processDelete(RunUpdateProcessorFactory.java:69) at org.apache.solr.handler.dataimport.SolrWriter.doDeleteAll(SolrWriter.java:192) at org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:332) at org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:386) at org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:377) 2009/10/10 Noble Paul നോബിള് नोब्ळ् > you may need to extend a SolrRequest and set appropriate path > ("/dataimport") and other params > then you may invoke the request method. > > On Sat, Oct 10, 2009 at 11:07 AM, rohan rai wrote: > > The configuration is not an issue... > > But how doindex i invoke it... > > > > I only have known a url way to invoke it and thus import the data into > > index... > > like http://localhost:8983/solr/db/dataimport?command=full-import t > > But with embedded I havent been able to figure it out > > > > Regards > > Rohan > > 2009/10/10 Noble Paul നോബിള് नोब्ळ् > >> > >> I guess it should be possible... what are the problems you encounter? > >> > >> On Sat, Oct 10, 2009 at 10:56 AM, rohan rai > wrote: > >> > Have been unable to use DIH for Embedded Solr > >> > > >> > Is there a way?? > >> > > >> > Regards > >> > Rohan > >> > > >> > >> > >> > >> -- > >> - > >> Noble Paul | Principal Engineer| AOL | http://aol.com > > > > > > > > -- > - > Noble Paul | Principal Engineer| AOL | http://aol.com >
Re: DIH and EmbeddedSolr
This is pretty unstable...anyone has any clue...Sometimes it even creates index, sometimes it does not ?? But everytime time I do get this exception Regards Rohan On Sat, Oct 10, 2009 at 6:07 PM, rohan rai wrote: > ModifiableSolrParams p = new ModifiableSolrParams(); > p.add("qt", "/dataimport"); > p.add("command", "full-import"); > server.query(p, METHOD.POST); > > I do this > > But it starts giving me this exception > > SEVERE: Full Import failed > java.util.concurrent.RejectedExecutionException > at > java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:1760) > at > java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:767) > at > java.util.concurrent.ScheduledThreadPoolExecutor.delayedExecute(ScheduledThreadPoolExecutor.java:216) > at > java.util.concurrent.ScheduledThreadPoolExecutor.schedule(ScheduledThreadPoolExecutor.java:366) > at > org.apache.solr.update.DirectUpdateHandler2$CommitTracker.scheduleCommitWithin(DirectUpdateHandler2.java:466) > at > org.apache.solr.update.DirectUpdateHandler2.deleteByQuery(DirectUpdateHandler2.java:322) > at > org.apache.solr.update.processor.RunUpdateProcessor.processDelete(RunUpdateProcessorFactory.java:69) > at > org.apache.solr.handler.dataimport.SolrWriter.doDeleteAll(SolrWriter.java:192) > at > org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:332) > at > org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:386) > at > org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:377) > > > > > > 2009/10/10 Noble Paul നോബിള് नोब्ळ् > >> you may need to extend a SolrRequest and set appropriate path >> ("/dataimport") and other params >> then you may invoke the request method. >> >> On Sat, Oct 10, 2009 at 11:07 AM, rohan rai wrote: >> > The configuration is not an issue... >> > But how doindex i invoke it... >> > >> > I only have known a url way to invoke it and thus import the data into >> > index... >> > like http://localhost:8983/solr/db/dataimport?command=full-import t >> > But with embedded I havent been able to figure it out >> > >> > Regards >> > Rohan >> > 2009/10/10 Noble Paul നോബിള് नोब्ळ् >> >> >> >> I guess it should be possible... what are the problems you encounter? >> >> >> >> On Sat, Oct 10, 2009 at 10:56 AM, rohan rai >> wrote: >> >> > Have been unable to use DIH for Embedded Solr >> >> > >> >> > Is there a way?? >> >> > >> >> > Regards >> >> > Rohan >> >> > >> >> >> >> >> >> >> >> -- >> >> - >> >> Noble Paul | Principal Engineer| AOL | http://aol.com >> > >> > >> >> >> >> -- >> - >> Noble Paul | Principal Engineer| AOL | http://aol.com >> > >
Re: DIH and EmbeddedSolr
Small data set.. 11 11 11 22 22 22 33 33 33 data-config schema id name Sometime it creates sometimes it gives thread pool exception. It does not consistently creates the index. Regards Rohan On Sun, Oct 11, 2009 at 3:56 PM, Shalin Shekhar Mangar < shalinman...@gmail.com> wrote: > On Sat, Oct 10, 2009 at 7:44 PM, rohan rai wrote: > > > This is pretty unstable...anyone has any clue...Sometimes it even creates > > index, sometimes it does not ?? > > > > > Most DataImportHandler tests run Solr in an embedded-like mode and they run > fine. Can you tell us which version of Solr are you using? Also, any data > which can help us reproduce the problem would be nice. > > -- > Regards, > Shalin Shekhar Mangar. >
Re: DIH and EmbeddedSolr
Hey Any reason why it may be happening ?? Regards Rohan On Sun, Oct 11, 2009 at 9:25 PM, rohan rai wrote: > > Small data set.. > > > > 11 > 11 > 11 > > > 22 > 22 > 22 > > > 33 > 33 > 33 > > > > data-config > > > > forEach="/root/test/" > url="/home/test/test_data.xml" > > > > > > > > > > schema > > > > omitNorms="true"/> > > > >multiValued="false" required="true"/> >multiValued="false" /> >multiValued="false" /> > > > id > > name > > > > > Sometime it creates sometimes it gives thread pool exception. It does not > consistently creates the index. > > Regards > Rohan > > > On Sun, Oct 11, 2009 at 3:56 PM, Shalin Shekhar Mangar < > shalinman...@gmail.com> wrote: > >> On Sat, Oct 10, 2009 at 7:44 PM, rohan rai wrote: >> >> > This is pretty unstable...anyone has any clue...Sometimes it even >> creates >> > index, sometimes it does not ?? >> > >> > >> Most DataImportHandler tests run Solr in an embedded-like mode and they >> run >> fine. Can you tell us which version of Solr are you using? Also, any data >> which can help us reproduce the problem would be nice. >> >> -- >> Regards, >> Shalin Shekhar Mangar. >> > >
Want to start contributing.
Hi all, I am an absolute beginner (dummy) in the field of contributing open source. But I am interested in contributing to open source. How do i start? Solr is a java based search engine based on Lucene. I am good at Java and therefore chose this to start. I need guidance. Help required!!
Query to multiple collections
Hi All , I have a SolrCloud setup with multiple collections. I have created say - two collections here as the data source for the both collections are different and hence wanted to store them differently. There is a use case , where i need to query both the collections and show unified search results. The fields in the schema are same. ( say - title , description , date ) Is there any specific way i can do this directly with the collections API or something like that? Or i need to write a federator and combine results from search to the respective collections and then unify them? -- *Regards,Rohan*
Re: Query to multiple collections
Thanks Chris for the update. I was thinking on the same grounds just wanted to check if you faced any specific issues. Regards, Rohan Kasat On Mon, Oct 22, 2018 at 11:20 AM Chris Ulicny wrote: > Rohan, > > I do not remember where I came across it or what restrictions exist on it, > but it works for our use case of querying multiple archived collections > with identical schemas in the same SolrCloud cluster. The queries have the > following form: > > > http::/solr/current/select?collection=current,archive2,archive4&q=... > > > It seems like it might work for your use case, but you might need to tread > carefully depending on your requirements for the returned results. Sorting > and duplicate unique keys come to mind. > > Best, > Chris > > On Mon, Oct 22, 2018 at 1:49 PM Rohan Kasat wrote: > > > Hi All , > > > > I have a SolrCloud setup with multiple collections. > > I have created say - two collections here as the data source for the > both > > collections are different and hence wanted to store them differently. > > There is a use case , where i need to query both the collections and show > > unified search results. > > The fields in the schema are same. ( say - title , description , date ) > > Is there any specific way i can do this directly with the collections API > > or something like that? > > Or i need to write a federator and combine results from search to the > > respective collections and then unify them? > > > > -- > > > > *Regards,Rohan* > > > -- *Regards,Rohan Kasat*
Re: Query to multiple collections
Thanks Alex. I check aliases but dint focused much , will try to relate more to my use case and have a look again at the same. I guess the specification of collection in the query should be useful. Regards, Rohan Kasat On Mon, Oct 22, 2018 at 11:21 AM Alexandre Rafalovitch wrote: > Have you tried using aliases: > > http://lucene.apache.org/solr/guide/7_5/collections-api.html#collections-api > > You can also - I think - specify a collection of shards/collections > directly in the query, but there may be side edge-cases with that (not > sure). > > Regards, > Alex. > On Mon, 22 Oct 2018 at 13:49, Rohan Kasat wrote: > > > > Hi All , > > > > I have a SolrCloud setup with multiple collections. > > I have created say - two collections here as the data source for the > both > > collections are different and hence wanted to store them differently. > > There is a use case , where i need to query both the collections and show > > unified search results. > > The fields in the schema are same. ( say - title , description , date ) > > Is there any specific way i can do this directly with the collections API > > or something like that? > > Or i need to write a federator and combine results from search to the > > respective collections and then unify them? > > > > -- > > > > *Regards,Rohan* > -- *Regards,Rohan Kasat*
Re: SOLR External Id field
Piyush, can you elaborate your question for external ID ? is this the field which distinguish each record in your indexes ? Regards, Rohan Kasat On Mon, Oct 22, 2018 at 11:46 AM Rathor, Piyush (US - Philadelphia) < prat...@deloitte.com> wrote: > Hi All, > > > > We are storing data in solr. Please let me know on the following: > > * How can we set a field as external id which can be used for update. > * What operation/ query needs to sent to update the same external id > record. > > > > Thanks & Regards > > Piyush Rathor > > > > This message (including any attachments) contains confidential information > intended for a specific individual and purpose, and is protected by law. If > you are not the intended recipient, you should delete this message and any > disclosure, copying, or distribution of this message, or the taking of any > action based on it, by you is strictly prohibited. > > v.E.1 > -- *Regards,Rohan Kasat*
Re: Query to multiple collections
Thanks Chris. This help. Regards, Rohan On Mon, Oct 22, 2018 at 12:26 PM Chris Ulicny wrote: > There weren't any particular problems we ran into since the client that > makes the queries to multiple collections previously would query multiple > cores using the 'shards' parameter before we moved to solrcloud. We didn't > have any complicated sorting or scoring requirements fortunately. > > The one thing I remember looking into was what solr would do when two > documents with the same id were found in both collections. I believe it > just non-deterministically picked one, probably the one that came in first > or last. > > Depending on how many collections you need to query simultaneously, it's > worth looking into using aliases for lists of collections as Alex > mentioned. > > Unfortunately, in our use case, it wasn't worth the headache of managing > aliases for every possible combination of collections that needed to be > queried, but we would have preferred to use aliases. > > On Mon, Oct 22, 2018 at 2:27 PM Rohan Kasat wrote: > > > Thanks Alex. > > I check aliases but dint focused much , will try to relate more to my use > > case and have a look again at the same. > > I guess the specification of collection in the query should be useful. > > > > Regards, > > Rohan Kasat > > > > On Mon, Oct 22, 2018 at 11:21 AM Alexandre Rafalovitch < > arafa...@gmail.com > > > > > wrote: > > > > > Have you tried using aliases: > > > > > > > > > http://lucene.apache.org/solr/guide/7_5/collections-api.html#collections-api > > > > > > You can also - I think - specify a collection of shards/collections > > > directly in the query, but there may be side edge-cases with that (not > > > sure). > > > > > > Regards, > > > Alex. > > > On Mon, 22 Oct 2018 at 13:49, Rohan Kasat > wrote: > > > > > > > > Hi All , > > > > > > > > I have a SolrCloud setup with multiple collections. > > > > I have created say - two collections here as the data source for the > > > both > > > > collections are different and hence wanted to store them differently. > > > > There is a use case , where i need to query both the collections and > > show > > > > unified search results. > > > > The fields in the schema are same. ( say - title , description , > date ) > > > > Is there any specific way i can do this directly with the collections > > API > > > > or something like that? > > > > Or i need to write a federator and combine results from search to the > > > > respective collections and then unify them? > > > > > > > > -- > > > > > > > > *Regards,Rohan* > > > > > > > > > -- > > > > *Regards,Rohan Kasat* > > > -- *Regards,Rohan Kasat*
Re: SOLR External Id field
Hi Piyush, There can be only a single unique identifier for a particular collection. And you can index the external field as Id for already existing record and it will replace the existing record. Regards, Rohan Kasat On Mon, Oct 22, 2018 at 2:20 PM Rathor, Piyush (US - Philadelphia) < prat...@deloitte.com> wrote: > Hi Rohan, > > We need to update certain records based on external id. > Please let me know how can we do it. > > Thanks & Regards > Piyush Rathor > > Please consider the environment before printing. > > -Original Message- > From: Rohan Kasat > Sent: Monday, October 22, 2018 2:46 PM > To: solr-user@lucene.apache.org > Subject: [EXT] Re: SOLR External Id field > > Piyush, > can you elaborate your question for external ID ? > is this the field which distinguish each record in your indexes ? > > Regards, > Rohan Kasat > > On Mon, Oct 22, 2018 at 11:46 AM Rathor, Piyush (US - Philadelphia) < > prat...@deloitte.com> wrote: > > > Hi All, > > > > > > > > We are storing data in solr. Please let me know on the following: > > > > * How can we set a field as external id which can be used for update. > > * What operation/ query needs to sent to update the same external id > > record. > > > > > > > > Thanks & Regards > > > > Piyush Rathor > > > > > > > > This message (including any attachments) contains confidential > > information intended for a specific individual and purpose, and is > > protected by law. If you are not the intended recipient, you should > > delete this message and any disclosure, copying, or distribution of > > this message, or the taking of any action based on it, by you is > strictly prohibited. > > > > v.E.1 > > > > > -- > > *Regards,Rohan Kasat* > -- *Regards,Rohan Kasat*
Re: Query to multiple collections
Thanks Shawn for the update. I am going ahead with the standard aliases approach , suits my use case. Regards, Rohan Kasat On Mon, Oct 22, 2018 at 4:49 PM Shawn Heisey wrote: > On 10/22/2018 1:26 PM, Chris Ulicny wrote: > > There weren't any particular problems we ran into since the client that > > makes the queries to multiple collections previously would query multiple > > cores using the 'shards' parameter before we moved to solrcloud. We > didn't > > have any complicated sorting or scoring requirements fortunately. > > > > The one thing I remember looking into was what solr would do when two > > documents with the same id were found in both collections. I believe it > > just non-deterministically picked one, probably the one that came in > first > > or last. > > Yes, that is how it works. I do not know whether it is the first one to > respond or the last one to respond that ends up in the results. Solr is > designed to work with data where the uniqueKey field really is unique > across everything that is being queried. Results can vary when you have > the same uniqueKey value in more than one place and you query both of > them at once. > > > Depending on how many collections you need to query simultaneously, it's > > worth looking into using aliases for lists of collections as Alex > > mentioned. > > > > Unfortunately, in our use case, it wasn't worth the headache of managing > > aliases for every possible combination of collections that needed to be > > queried, but we would have preferred to use aliases. > > Aliases are the cleanest option. This syntax also works, sorta blew my > mind when somebody told me about it: > > http://host:port/solr/current,archive2,archive4/select?q=*:* > > If you're using a Solr client library, it might not be possible to > control the URL like that, but if you're building URLs yourself, you > could use it. > > I recently filed an issue related to alias handling, some unexpected > behavior: > > https://issues.apache.org/jira/browse/SOLR-12849 > > Thanks, > Shawn > > -- *Regards,Rohan Kasat*
Re: Tesseract language
Hi Martin, Are you using it For image formats , I think you can try tess4j and use give TESSDATA_PREFIX as the home for tessarct Configs. I have tried it and it works pretty well in my local machine. I have used java 8 and tesseact 3 for the same. Regards, Rohan Kasat On Fri, Oct 26, 2018 at 12:31 PM Martin Frank Hansen (MHQ) wrote: > Hi Tim, > > You were right. > > When I called `tesseract testing/eurotext.png testing/eurotext-dan -l > dan`, I got an error message so I downloaded "dan.traineddata" and added it > to the Tesseract-OCR/tessdata folder. Furthermore I added the > 'TESSDATA_PREFIX' variable to the path-variables pointing to > "Tesseract-OCR/tessdata". > > Now Tesseract works with Danish language from the CMD, but now I can't > make the code work in Java, not even with default settings (which I could > before). Am I missing something or just mixing some things up? > > > > -Original Message- > From: Tim Allison > Sent: 26. oktober 2018 19:58 > To: solr-user@lucene.apache.org > Subject: Re: Tesseract language > > Tika relies on you to install tesseract and all the language libraries > you'll need. > > If you can successfully call `tesseract testing/eurotext.png > testing/eurotext-dan -l dan`, Tika _should_ be able to specify "dan" > with your code above. > On Fri, Oct 26, 2018 at 10:49 AM Martin Frank Hansen (MHQ) > wrote: > > > > Hi again, > > > > Now I moved the OCR part to Tika, but I still can't make it work with > Danish. It works when using default language settings and it seems like > Tika is missing Danish dictionary. > > > > My java code looks like this: > > > > { > > File file = new File(pathfilename); > > > > Metadata meta = new Metadata(); > > > > InputStream stream = TikaInputStream.get(file); > > > > Parser parser = new AutoDetectParser(); > > BodyContentHandler handler = new > > BodyContentHandler(Integer.MAX_VALUE); > > > > TesseractOCRConfig config = new TesseractOCRConfig(); > > config.setLanguage("dan"); // code works if this phrase is > commented out. > > > > ParseContext parseContext = new ParseContext(); > > > > parseContext.set(TesseractOCRConfig.class, config); > > > > parser.parse(stream, handler, meta, parseContext); > > System.out.println(handler.toString()); > > } > > > > Hope that someone can help here. > > > > -Original Message- > > From: Martin Frank Hansen (MHQ) > > Sent: 22. oktober 2018 07:58 > > To: solr-user@lucene.apache.org > > Subject: SV: Tessera > <https://maps.google.com/?q=ect:+SV:+Tessera&entry=gmail&source=g>ct > language > > > > Hi Erick, > > > > Thanks for the help! I will take a look at it. > > > > > > Martin Frank Hansen, Senior Data Analytiker > > > > Data, IM & Analytics > > > > > > > > Lautrupparken 40-42, DK-2750 Ballerup > > E-mail m...@kmd.dk Web www.kmd.dk > > Mobil +4525571418 > > > > -Oprindelig meddelelse- > > Fra: Erick Erickson > > Sendt: 21. oktober 2018 22:49 > > Til: solr-user > > Emne: Re: Tesseract language > > > > Here's a skeletal program that uses Tika in a stand-alone client. Rip > the RDBMS parts out > > > > https://lucidworks.com/2012/02/14/indexing-with-solrj/ > > On Sun, Oct 21, 2018 at 1:13 PM Alexandre Rafalovitch < > arafa...@gmail.com> wrote: > > > > > > Usually, we just say to do a custom solution using SolrJ client to > > > connect. This gives you maximum flexibility and allows to integrate > > > Tika either inside your code or as a server. Latest Tika actually > > > has some off-thread handling I believe, to make it safer to embed. > > > > > > For DIH alternatives, if you want configuration over custom code, > > > you could look at something like Apache NiFI. It can push data into > Solr. > > > Obviously it is a bigger solution, but it is correspondingly more > > > robust too. > > > > > > Regards, > > >Alex. > > > On Sun, 21 Oct 2018 at 11:07, Martin Frank Hansen (MHQ) > wrote: > > > > > > > > Hi Alexandre, > > > > > > > > Thanks for your reply. > > > > > > > > Yes right now it is just for testing the possibilities of Solr and > Tesseract. > > > > > >
Re: Tesseract language
I used tess4j for image formats and Tika for scanned PDFs and images within PDFs. Regards, Rohan Kasat On Sat, Oct 27, 2018 at 12:39 AM Martin Frank Hansen (MHQ) wrote: > Hi Rohan, > > Thanks for your reply, are you using tess4j with Tika or on its own? I > will take a look at tess4j if I can't make it work with Tika alone. > > Best regards > Martin > > > -Original Message- > From: Rohan Kasat > Sent: 26. oktober 2018 21:45 > To: solr-user@lucene.apache.org > Subject: Re: Tesseract language > > Hi Martin, > > Are you using it For image formats , I think you can try tess4j and use > give TESSDATA_PREFIX as the home for tessarct Configs. > > I have tried it and it works pretty well in my local machine. > > I have used java 8 and tesseact 3 for the same. > > Regards, > Rohan Kasat > > On Fri, Oct 26, 2018 at 12:31 PM Martin Frank Hansen (MHQ) > wrote: > > > Hi Tim, > > > > You were right. > > > > When I called `tesseract testing/eurotext.png testing/eurotext-dan -l > > dan`, I got an error message so I downloaded "dan.traineddata" and > > added it to the Tesseract-OCR/tessdata folder. Furthermore I added the > > 'TESSDATA_PREFIX' variable to the path-variables pointing to > > "Tesseract-OCR/tessdata". > > > > Now Tesseract works with Danish language from the CMD, but now I can't > > make the code work in Java, not even with default settings (which I > > could before). Am I missing something or just mixing some things up? > > > > > > > > -Original Message- > > From: Tim Allison > > Sent: 26. oktober 2018 19:58 > > To: solr-user@lucene.apache.org > > Subject: Re: Tesseract language > > > > Tika relies on you to install tesseract and all the language libraries > > you'll need. > > > > If you can successfully call `tesseract testing/eurotext.png > > testing/eurotext-dan -l dan`, Tika _should_ be able to specify "dan" > > with your code above. > > On Fri, Oct 26, 2018 at 10:49 AM Martin Frank Hansen (MHQ) > > > > wrote: > > > > > > Hi again, > > > > > > Now I moved the OCR part to Tika, but I still can't make it work > > > with > > Danish. It works when using default language settings and it seems > > like Tika is missing Danish dictionary. > > > > > > My java code looks like this: > > > > > > { > > > File file = new File(pathfilename); > > > > > > Metadata meta = new Metadata(); > > > > > > InputStream stream = TikaInputStream.get(file); > > > > > > Parser parser = new AutoDetectParser(); > > > BodyContentHandler handler = new > > > BodyContentHandler(Integer.MAX_VALUE); > > > > > > TesseractOCRConfig config = new TesseractOCRConfig(); > > > config.setLanguage("dan"); // code works if this phrase > > > is > > commented out. > > > > > > ParseContext parseContext = new ParseContext(); > > > > > > parseContext.set(TesseractOCRConfig.class, config); > > > > > > parser.parse(stream, handler, meta, parseContext); > > > System.out.println(handler.toString()); > > > } > > > > > > Hope that someone can help here. > > > > > > -Original Message- > > > From: Martin Frank Hansen (MHQ) > > > Sent: 22. oktober 2018 07:58 > <https://maps.google.com/?q=tober+2018+07:58&entry=gmail&source=g> > > > To: solr-user@lucene.apache.org > > > Subject: SV: Tessera > > <https://maps.google.com/?q=ect:+SV:+Tessera&entry=gmail&source=g>ct > > language > > > > > > Hi Erick, > > > > > > Thanks for the help! I will take a look at it. > > > > > > > > > Martin Frank Hansen, Senior Data Analytiker > > > > > > Data, IM & Analytics > > > > > > > > > > > > Lautrupparken 40-42, DK-2750 Ballerup E-mail m...@kmd.dk Web > > > www.kmd.dk Mobil +4525571418 > > > > > > -Oprindelig meddelelse- > > > Fra: Erick Erickson > > > Sendt: 21. oktober 2018 22:49 > > > Til: solr-user > > > Emne: Re: Tesseract language > > > > > > Here's a skeletal program that uses Tika in a stand-alone client. > > > Rip > > the RDBMS parts out > >
Re: Solr Auto correct user query
Can you share your config file and use case ? Its difficult to guess how you have configured the component. Regards, Rohan Kasat On Wed, Feb 20, 2019 at 12:21 AM Prasad_sarada wrote: > Hi, > I want to implement solr auto correct feature, i have tried doing the spell > check one but not getting satisfying result. it's showing the top > suggestion > but not giving the result of the correct word. > ex:if i am searching for "procesor" then i should get the result of > "processor" coz the second one is the correct word. > > Please help me doing this > > Thanks, > Sarada Prasad > > > > -- > Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html > -- *Regards,Rohan Kasat*
Re: Control Solr spellcheck functionality to provide suggestions for correct word
Hi Rashi, Can you share your spellcheck configuration, it will be easy to check from the configuration? Regards, Rohan Kasat On Fri, Apr 5, 2019 at 10:29 AM rashi gandhi wrote: > HI, > > I am working on Solr spellcheck feature, and I am using index based > spellcheck dictionary as a source for spellcheck suggestions. > I observed that collated results returned by spellcheck component, provide > the suggestions for misspelled words, however also provide suggestions for > correctly spelled word in query. > > For example, > misspelled query - root priviladge to user > > *collated results (even suggestion includes the same) *- > root privilege to user, room privilege to user, root privilege to users, > rest privilege to user, root privilege to used > > It corrected word 'privilege' which was misspelled, however also provided > suggestions for 'root' or 'user', which were already correct. > > is there a way , we can tell Solr not to provide suggestions for correct > word, when using spellcheck feature. > > Please provide pointers. > -- *Regards,Rohan Kasat*
Re: Control Solr spellcheck functionality to provide suggestions for correct word
Hi Rashi, Can you try with 0 Also you can play with maxqueryfrequency value. Regards, Rohan On Sat, Apr 6, 2019 at 2:53 AM Rashi wrote: > Hi Rohan, > > Sharing the configuration: > > > text_suggest > > > > default > title > solr.DirectSolrSpellChecker > > internal > > 0.5 > > 2 > > 1 > > 5 > > 4 > > 0.01 > > > > > Also, below are the parameters, passing into Solr params: > > solrParams.add("spellcheck.dictionary", "default"); > solrParams.add("spellcheck.extendedResults", "true"); > solrParams.add("spellcheck.count", "10"); > solrParams.add("spellcheck.alternativeTermCount", "5"); > solrParams.add("spellcheck.maxResultsForSuggest", "5"); > solrParams.add("spellcheck.collate", "true"); > solrParams.add("spellcheck.collateExtendedResults", "true"); > solrParams.add("spellcheck.maxCollationTries", "10"); > solrParams.add("spellcheck.maxCollations", "5"); > solrParams.add("spellcheck.onlyMorePopular", "false"); > > is Solr always provide the suggestions for correct words also ? Any > pointers. > > > > > -- > Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html > -- *Regards,Rohan Kasat*
Re: Stopwords impact on search
So do we use stopwords filter as part of query analyzer, to avoid highlighting of these stop words ? Regards, Rohan On Fri, Apr 24, 2020 at 7:45 AM Walter Underwood wrote: > Agreed. Here is an article from 13 years ago when I accidentally turned on > stopword removal at Netflix. It caused bad problems. > > https://observer.wunderwood.org/2007/05/31/do-all-stopword-queries-matter/ > > Infoseek was not removing stopwords when I joined them in 1996. Since then, > I’ve always left stopwords in the index. Removing stop words is a desperate > speed/hack hack from the days of 16-bit machines. > > wunder > Walter Underwood > wun...@wunderwood.org > http://observer.wunderwood.org/ (my blog) > > > On Apr 24, 2020, at 5:44 AM, David Hastings < > hastings.recurs...@gmail.com> wrote: > > > > you should never use the stopword filter unless you have a very specific > > purpose > > > > On Fri, Apr 24, 2020 at 8:33 AM Steven White > wrote: > > > >> Hi everyone, > >> > >> What is, if any, the impact of stopwords in to my search ranking > quality? > >> Will my ranking improve is I do not index stopwords? > >> > >> I'm trying to figure out if I should use the stopword filter or not. > >> > >> Thanks in advanced. > >> > >> Steve > >> > > -- *Regards,Rohan Kasat*
Copy index from lower environment to Production | SolrCloud
Hi, In our setup we have two SolrCloud environments running Solr 7.5 version. Specific to the question - We have one collection with 3 shards and 3 replicas on the lower environment and a newly created mirrored collection on Production. Wanted to know on approaches to copy the index for collection from cloud on lower environment to Production solrcloud instance ? -- *Regards,Rohan Kasat*
Re: Copy index from lower environment to Production | SolrCloud
Thank you so much Erick. Will check these out. Regards, Rohan Kasat On Tue, Jun 4, 2019 at 12:54 PM Erick Erickson wrote: > > (t’s usually far easier to create a new collection in your upper > environment and index to _that_. Once the indexing is done, use the > Collections API CREATEALIAS command to point traffic to the new collection. > You can then use the old one to index to and use CREATEALIAS to point to > that one, switching back and forth as necessary. > > > If that’s not possible, try is the low-level replication API call. > WARNING, “this should work”, but I haven’t tried it.….. > > See “HTTP API Commands for the Replication Handler” here: > https://lucene.apache.org/solr/guide/7_6/index-replication.html > > You do _not_ need to set up replication in solrconfig.xml or the like. You > want to use “fetchIndex”. Tricks: > > 1> you must issue this command _to the core_ you want the index copied to. > This will be something like “ > http://blahlbah/solr/collection1_shard1_replica_n2”. Specify the > masterURL as the core you want to copy _from_, i.e. your lower environment. > > 2> Have one and only one replica for each shard running in your higher > environment. This will be “leader only”, i.e. no followers. Once you’ve > successfully moved the index, bring up any other replicas (or use > ADDREPLICA). When they come up they’ll replicate down from the leader. > > 3> Be very, very careful that you copy the indexes from exactly the same > shard, as defined by the range in state.json associated with each shard. > > An alternative is to shut down both systems and just copy the data dir > (tlog and index) files manually from the lower to upper. You’d need your > upper collection to be completely shut down. Again, copy the files to one > replica of each shard then bring _only_ that replica up. Once it’s decided > to be the leader, bring the other replicas up or ADDREPLICA, whichever is > easier. > > > > On Jun 4, 2019, at 11:36 AM, Rohan Kasat wrote: > > > > Hi, > > > > In our setup we have two SolrCloud environments running Solr 7.5 version. > > Specific to the question - We have one collection with 3 shards and 3 > > replicas on the lower environment and a newly created mirrored collection > > on Production. > > Wanted to know on approaches to copy the index for collection from cloud > on > > lower environment to Production solrcloud instance ? > > > > -- > > > > *Regards,Rohan Kasat* > > -- *Regards,Rohan Kasat*
Issue with indexing document - solr 7.5
Hi All, Am indexing json documents with parent child relationship's and have the following schema on Solr 7.5 id Document - { "id": "Year-2019", "year": 2019, "_childDocuments_": [ { "id": "5-2019", "month": 5 } ] } and the error am getting is - '_childDocuments_.id_str' is not an explicit field and doesn't match a dynamicField." Not using any dynamic fields in my schema. Have anyone faced such issue ? -- *Regards,R K*
Fwd: Issue with indexing document - solr 7.5
Hi All, Am indexing json documents with parent child relationship and have the following schema on Solr 7.5 Schema - id Document - { "id": "Year-2019", "year": 2019, "_childDocuments_": [ { "id": "5-2019", "month": 5 } ] } and the error am getting is - '_childDocuments_.id_str' is not an explicit field and doesn't match a dynamicField." Not using any dynamic fields in my schema. Have anyone faced such issue ? -- *Regards,R K*
Solr crash | GC issue
Hi All, I have a Solr Cloud setup of 3 solr servers 7.5 version. 24GB heap memory is allocated to each solr server and i have around 655 GB of data in indexes to be searched for. Few last 2-3 days, the solr servers are crashing and am able to see the heap memory is almost full but the CPU usage is just 1 %. I am attaching the gc logs from 3 servers. Can you please help in analyzing yje logs and comments to improve https://gist.github.com/rohankasat/cee8203c0c12983d9839b7a59047733b -- *Regards,Rohan Kasat*
Re: Solr crash | GC issue
Thanks Shawn and Paul. I tried using the https://gceasy.io/ but was not able to understand much. I see the OOM file getting created with "not much heap space" as the error . Shawn, i have tried your CMS settings too and now will try increasing the heap memory, hope it works this time. Any things specific i should be checking ? Regards, Rohan Kasat On Fri, Aug 16, 2019 at 12:23 PM Shawn Heisey wrote: > On 8/16/2019 8:23 AM, Rohan Kasat wrote: > > I have a Solr Cloud setup of 3 solr servers 7.5 version. > > 24GB heap memory is allocated to each solr server and i have around 655 > GB > > of data in indexes to be searched for. > > > > Few last 2-3 days, the solr servers are crashing and am able to see the > > heap memory is almost full but the CPU usage is just 1 %. > > > > I am attaching the gc logs from 3 servers. Can you please help in > analyzing > > yje logs and comments to improve > > > > https://gist.github.com/rohankasat/cee8203c0c12983d9839b7a59047733b > > These three GC logs do not indicate that all the heap is used. > > The peak heap usage during these GC logs is 18.86GB, 19.42GB, and > 18.91GB. That's quite a bit below the 24GB max. > > There are some very long GC pauses recorded. Increasing the heap size > MIGHT help with that, or it might not. > > The typical way that Solr appears to "crash" is when an OutOfMemoryError > exception is thrown, at which time a Solr instance that is running on an > OS like Linux will kill itself with a -9 signal. This scripting is not > present when starting on Windows. > > An OOME can be thrown for a resource other than memory, so despite the > exception name, it might not actually be memory that has been depleted. > The exception will need to be examined to learn why it was thrown. > > GC logs do not indicate the cause of OOME. If that information is > logged at all, and it might not be, it will be in solr.log. > > Looking at the GC logs to see how your Solr is laid out... the following > command might find the cause, if it was logged, and if the relevant log > has not been rotated out: > > grep -r OutOfMemory /apps/solr/solr_data/logs/* > > At the very least it might help you find out which log file to > investigate further. > > Thanks, > Shawn > -- *Regards,Rohan Kasat*
Re: Solr crash | GC issue
Thanks Shawn. I saw that error when the solr crashed last time. Am waiting to see if it happens again and to capture the compete error log. Regards, Rohan Kasat On Fri, Aug 16, 2019 at 2:36 PM Shawn Heisey wrote: > On 8/16/2019 11:59 AM, Rohan Kasat wrote: > > I see the OOM file getting created with "not much heap space" as the > error > > Can you get the precise error cause? I haven't ever seen that > particular text before. If you can paste the entire error (which will > be many lines), that can be helpful. > > > Shawn, i have tried your CMS settings too and now will try increasing the > > heap memory, hope it works this time. > > Changing GC tuning can never fix an OOME problem. The only way to fix > it is to increase the resource that's running out or adjust things so > less of that resource is needed. > > Thanks, > Shawn > -- *Regards,Rohan Kasat*
Re: Re: Query on autoGeneratePhraseQueries
Also check , pf , pf2 , pf3 ps , ps2, ps3 parameters for phrase searches. Regards, Rohan K On Tue, Oct 15, 2019 at 6:41 AM Audrey Lorberfeld - audrey.lorberf...@ibm.com wrote: > I'm not sure how your config file is setup, but I know that the way we do > multi-token synonyms is to have the sow (split on whitespace) parameter set > to False while using the edismax parser. I'm not sure if this would work > with PhraseQueries , but it might be worth a try! > > In our config file we do something like this: > > > > edismax > 1.0 > explicit > 100 > content_en > w3json_en > false > > > > You can read a bit about the parameter here: > https://opensourceconnections.com/blog/2018/02/20/edismax-and-multiterm-synonyms-oddities/ > > Best, > Audrey > > -- > Audrey Lorberfeld > Data Scientist, w3 Search > IBM > audrey.lorberf...@ibm.com > > > On 10/15/19, 5:50 AM, "Shubham Goswami" > wrote: > > Hi kshitij > > Thanks for the reply! > I tried to debug it and found that raw query(black company) has parsed > as > two separate queries > black and company and returning the results based on black query > instead of > this it should have > got parsed as a single phrase query like("black company") because i am > using > autoGeneratedPhraseQuery. > Do you have any idea about this please correct me if i am wrong. > > Thanks > Shubham > > On Tue, Oct 15, 2019 at 1:58 PM kshitij tyagi < > kshitij.shopcl...@gmail.com> > wrote: > > > Hi, > > > > Try debugging your solr query and understand how it gets parsed. Try > using > > "debug=true" for the same > > > > On Tue, Oct 15, 2019 at 12:58 PM Shubham Goswami < > > shubham.gosw...@hotwax.co> > > wrote: > > > > > *Hi all,* > > > > > > I am a beginner to solr framework and I am trying to implement > > > *autoGeneratePhraseQueries* property in a fieldtype of > > type=text_general, i > > > kept the property value as true and restarted the solr server but > still > > it > > > is not taking my two words query like(Black company) as a phrase > without > > > double quotes and returning the results only for Black. > > > > > > Can somebody please help me to understand what am i > missing ? > > > Following is my Schema.xml file code and i am using solr 7.5 > version. > > > > > positionIncrementGap="100" multiValued="true" > > > autoGeneratePhraseQueries="true"> > > > > > > = > > >> > ignoreCase="true"/> > > > > > > > > > > > > > > >> > ignoreCase="true"/> > > >> > ignoreCase="true" synonyms="synonyms.txt"/> > > > > > > > > > > > > > > > > > > -- > > > *Thanks & Regards* > > > Shubham Goswami > > > Enterprise Software Engineer > > > *HotWax Systems* > > > *Enterprise open source experts* > > > cell: +91-7803886288 > > > office: 0731-409-3684 > > > > https://urldefense.proofpoint.com/v2/url?u=http-3A__www.hotwaxsystems.com&d=DwIBaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=_8ViuZIeSRdQjONA8yHWPZIBlhj291HU3JpNIx5a55M&m=Zi9beGF58BzJUNUdCkeW0pwliKwq9vdTSh0V_lR0734&s=FhSkJBcmYw_bfHgq1enzuYQeOZwKHzlP9h4VwTZSL5E&e= > > > > > > > > -- > *Thanks & Regards* > Shubham Goswami > Enterprise Software Engineer > *HotWax Systems* > *Enterprise open source experts* > cell: +91-7803886288 > office: 0731-409-3684 > > https://urldefense.proofpoint.com/v2/url?u=http-3A__www.hotwaxsystems.com&d=DwIBaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=_8ViuZIeSRdQjONA8yHWPZIBlhj291HU3JpNIx5a55M&m=Zi9beGF58BzJUNUdCkeW0pwliKwq9vdTSh0V_lR0734&s=FhSkJBcmYw_bfHgq1enzuYQeOZwKHzlP9h4VwTZSL5E&e= > > > -- *Regards,Rohan Kasat*