Re: What filter to use to search with spaces omitted/included between words?

I think you should skip spaces before indexing and querying followed by a filter of EdgeNGramFilterFactory at index time. This may requre you to work on your data before indexing & querying. Point is we do not want to make tokens by breaking them into spaces. So indexing and query time should be th

Re: What filter to use to search with spaces omitted/included between words?

Additionally, if you dont want results like q=best and result=bestbuy; you can use to actually replace whitespaces with nothing. http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters#CharFilterFactories

Re: struggling with solr.WordDelimiterFilterFactory

Hi All, There were two fixes for the issue I was facing 1. By changing the version in schema form* 1.1* to *1.5* OR 2. keeping the version to 1.1 and adding *autoGeneratePhraseQueries*="false" to the field type However the issue is not completely resolved yet on searching for content:speedPost th

High memory usage on solr 3.6

Hi all, we are facing a high memory usage in our Solr 3.6 master (not in the slaves) even during "idle" (non indexing) periods. container: Tomcat 6.0.29 maxthreads: 1.5k (i think this setting is wrong, 300 would be enough) solr: solr 3.6.0 setup: multitenant environment with 120+ cores and near 5M

Re: SolrCloud Zookeeper Exception

I don't think that the ZooKeeper errors are related to the DIH problems. Can you post your data-config and dih output using http://apaste.info or other pastebin like website? On Mon, Aug 19, 2013 at 4:58 PM, Prasi S wrote: > Hi, > I have setup solrcloud with 4.4 version. There is one external zoo

Error while importing HBase data to Solr using the DataImportHandler

Hi, I want to import HBase(0.90.4) data to solr 4.3.0 using DIH (Data import handler) for this I used "https://code.google.com/p/hbase-solr-dataimport/"; project. Whenever I run data import handler " http://localhost:8080/solr/#/collection1/dataimport"; it throws following error in log. *Jar:* hb

Re: convert text file to solr document where delimiter fields are fields of document

Hi, Try this: http://wiki.apache.org/solr/DataImportHandler#FileDataSource and http://wiki.apache.org/solr/DataImportHandler#httpds -- View this message in context: http://l

Re: Error while importing HBase data to Solr using the DataImportHandler

You would need to add the jar that is missing to the Solr web-inf\lib folder. You can do that using winzip etc into the lib folder of solr.war. Then you need to redeployed the changed solr.war and restart your webcontainer. The jar is available here: http://code.google.com/p/hbase-solr-dataimport/

How to sort by the function: relevance_score*numberic_field/(relevance_score +numberic_field )

Hi: I want to rank the search result by the function: relevance_score*numberic_field/(relevance_score +numberic_field ) , this function equals to 1/((1/relevance_score)+1/numberic_field) As far as I know ,I could use function query: sort= div(1,sum(div(1,field(numberic_field)),div(1,query

Create core with user-defined the instanceDir

Hi All With the Solr-4.4 ,I try the Hdfs and SolrCloud. I create a SolrCloud core with hdfs using : http://10.7.23.125:8080/solr/admin/cores?action=CREATE&name=testcore8_1&shard=shard13&collection.configName=myconf&schema=schema.xml&config=solrconfig3.xml&collection=collection1&dataDir=/s

convert text file to solr document where delimiter fields are fields of document

I have a text file which consist of the following data andy~1234;M~64365113~2P3VWU3H10~~ mike~4152;M~64365113~2P3VWU3H10~0.6~MG lesa~4512;F,PM~~N/A~16~MG riky~7845;M,PM2~~N/A~3.99~MG I wish to convert it into a solr document in the following manner : 1. Each row is considered as 1 document in s

Common Grams Highlighting

Hi, I have gone through a lot of posts about Highlighting issues with Common Grams but I am still a little confused. Below is my requirement: - Highlighting needs to work properly with Common Grams - Phrase highlighting needs to work - Wildcard highlighting needs to work Is this possible with Ph

Re: How to sort by the function: relevance_score*numberic_field/(relevance_score +numberic_field )

Solr does not have a function query value source which is the raw document relevance score for the current query. That would be a nice Jira request. Or, a parameter to outright replace the score for each document rather than merely multiply or add to it. -- Jack Krupansky -Original Mess

Re: What filter to use to search with spaces omitted/included between words?

Also consider WordDelimterFilterFactory, which will break up the tokens on upper/lower case transitions. to get relevance, consider edismax-style query parsers and adding automatic phrase generation (with boosts usually). This one will be a problem: q=bestbuy There's no good generic way to get t

Re: struggling with solr.WordDelimiterFilterFactory

OK, here's where you can drive yourself mad with the subtle variations between how WordDelimiterFilterFactory splits and recombines the tokens. Take a look at the stock distro, you'll see that the index time and query time settings for WEFF are slightly different. The idea is that if you do things

Re: Error while importing HBase data to Solr using the DataImportHandler

Thanks tamanjit! Issue is resolved I just further added following jars as well: hbase-0.90.4.jar hadoop-core-0.20-append-r1056497.jar zookeeper-3.4.5.jar Regards, Jamshaid On Tue, Aug 20, 2013 at 2:32 PM, tamanjit.bin...@yahoo.co.in < tamanjit.bin...@yahoo.co.in> wrote: > You would need to ad

Re: High memory usage on solr 3.6

Hmmm, first time I've seen this, but are you seeing a problem or is this just a theoretical concern? There are some references that are held around until GC, these could be perfectly legitimate, unused but un-collected references. What I'd try before worrying much: 1> attach jconsole and press the

Re: convert text file to solr document where delimiter fields are fields of document

You can directly update Solr using a "CSV" file and specify an alternate separator character: http://wiki.apache.org/solr/UpdateCSV#separator -- Jack Krupansky -Original Message- From: bharat Sent: Tuesday, August 20, 2013 5:01 AM To: solr-user@lucene.apache.org Subject: convert text

Re: convert text file to solr document where delimiter fields are fields of document

Hi, Since your data is well delimited, I'd suggest using CSV Updater, with the delimiter/ separator set to: *'~*' See: http://wiki.apache.org/solr/UpdateCSV#separator Looks like you might also have to additionally split based on your second delimiter: *';'* See: http://wiki.apache.org/solr/Update

Re: Create core with user-defined the instanceDir

_How_ does it "not work"? You've given us no information to go on. Please review: http://wiki.apache.org/solr/UsingMailingLists At a guess, does your heihei directory exist and does it have the proper configuration files? Best Erick On Tue, Aug 20, 2013 at 6:38 AM, YouPeng Yang wrote: > Hi Al

Re: What filter to use to search with spaces omitted/included between words?

You could either have a synonym filter to replace "bestbuy" with "best buy" or use DictionaryCompoundWordTokenFilterFactory to do the same. See: http://lucene.apache.org/core/4_4_0/analyzers-common/org/apache/lucene/analysis/compound/DictionaryCompoundWordTokenFilterFactory.html There are some

Re: struggling with solr.WordDelimiterFilterFactory

Hi Erik, I was going to come to that. Now if I have the word *speedpost* in the index and if I dont use catenation at the query end then query for the word speedPost wont fetch me the results. It would then might make sense to remove the entire WDFF from query and search for a few possible combina

Boost by numFounds

Hi to everybody, in my index I have location names and I'd like to give boost to those that are more referred to in other documents. For example, if I query document by "name:London", I'd like to give more boost to those ids that are most referred in other fields of documents (e.g. city=1234), that

Re: Common Grams Highlighting

The answer to each of your questions is the same: Just try it. And then feel free to report back to this list how things worked out. And, hopefully, you will have learned the lesson to avoid overly complex and overly ambitious analysis schemes. The simple message: Keep things simple. If you h

Re: struggling with solr.WordDelimiterFilterFactory

Just to be clear for other readers, if you have "speedpost" in the index and you query "speedPost" using the "OR" operator and the WDF set to "catenate all", and use lower case filter, the query should work fine. If it fails in your case, well, maybe there is something else wrong... somewhere.

Re: struggling with solr.WordDelimiterFilterFactory

Hi Jack, As mentioned earliear a part of the issue was resolved by the two fixes I mentioned above and for the query u mentioned I am getting the same result as yours. What is not working though is the query *q=content:"speedPost"* with the text enclosed in inverted commas -- View this message

Re: struggling with solr.WordDelimiterFilterFactory

I just wanted other readers to be clear about what can be made to work quite easily. As to "speedPost" as a quoted phrase, that's a different beast entirely due to the semantics of phrases, which is that they are an implicit "AND" operator - all tokens must match - and you and your users must

Re: struggling with solr.WordDelimiterFilterFactory

Hi Jack, Thanks for the expalnation -- View this message in context: http://lucene.472066.n3.nabble.com/struggling-with-solr-WordDelimiterFilterFactory-tp4085021p4085661.html Sent from the Solr - User mailing list archive at Nabble.com.

Re: State sharing

got it. thank you Jack and Shalin On Aug 19, 2013, at 9:52 AM, "Jack Krupansky" wrote: > Generally, you shouldn't be trying to maintain, let alone share "state" in > Solr itself. It sounds like you need an application layer between your > application clients and Solr which could then maintain

Solr 4.4 Query ignoring null values

Hi, I have a few questions with Solr 4.4 query parser 1. http://localhost:8180/solr/collection1/select?q=business_name:Catherine AND city:""&debugQuery=yes -> 100 results In

Re: Issue in Swap Space display at Solr Admin

Stefan, I tried to execute the query what you mentioned, but http 404 error was returned. We are using multicore solr, I think query format depends on it. - Vladimir. 2013/8/19 Stefan Matheis > Vladimir > > Would you mind attaching the output of /solr/admin/system?wt=json ? The > last about 2

Re: Issue in Swap Space display at Solr Admin

Vladimir That shouldn't matter .. perhaps i did not provide enough information? depends on which host & port you have solr running .. and the path you have defined. based on the tutorial (host + port configuration) you would use something like this: http://localhost:8983/solr/admin/system?wt=j

Re: conflicting field options

If you have modified the right schema.xml then a server restart should pick it up. On Tue, Aug 20, 2013 at 9:19 PM, Brian Robinson wrote: > Hello, > I'm having problems with getting my core setup, I think. I just have the > collection1 core, or at least that's my intent, at /etc/solr/collection1/

Re: Issue in Swap Space display at Solr Admin

On 8/20/2013 9:49 AM, Stefan Matheis wrote: Vladimir That shouldn't matter .. perhaps i did not provide enough information? depends on which host & port you have solr running .. and the path you have defined. based on the tutorial (host + port configuration) you would use something like this:

conflicting field options

Hello, I'm having problems with getting my core setup, I think. I just have the collection1 core, or at least that's my intent, at /etc/solr/collection1/. When I go to the solr/admin/ page in my browser, I get the following error message: SolrCore 'collection1' is not available due to init fa

Re: How to sort by the function: relevance_score*numberic_field/(relevance_score +numberic_field )

: I want to rank the search result by the function: : relevance_score*numberic_field/(relevance_score +numberic_field ) , this : function equals to : : 1/((1/relevance_score)+1/numberic_field) your email seems to asks 2 differnet questions: 1) "sort by the function ...(containing the releva

"Path must not end with / character" error during performance tests

Hi, I have integrated SolrCloud search in some system with single shard and works fine on single tests. Recently we started to run performance test and we are getting following exception after a while. Help is really appreciated, Thanks Tanya 2013-08-20 10:45:56,128 [cTaskExecutor-1] ERROR Log

Re: Solr 4.4 Query ignoring null values

: Subject: Solr 4.4 Query ignoring null values First off: it's important that you understand there is no such thing as a "null value" in a Solr index -- the concept does not exist. You can have documents that "do not contain a value" for a field, and you can have documents that contain "the e

Re: Best way to version config files

Hi, Perhaps you could post your dreams in a form of solr usability suggestion? :) http://solrstart.uservoice.com/forums/216001-usability-contest/filters/top I hope it is accepting contributions still. What you are saying from the point of view of Solr UI in-built version-control features, I wou

Re: spatial search, geofilt does not work

Oh, man. I have been trying to figure out the problem for half day. Probably Solr could use some error msg if the query format is invalid. But, THANKS! David, you probably saved me another half day. Ming- On Mon, Aug 19, 2013 at 10:20 PM, David Smiley (@MITRE.org) < dsmi...@mitre.org> wrote:

Re: Error loading class 'solr.DisMaxRequestHandler' after upgrade from solr350 to 431

how do you solved this problem? -- View this message in context: http://lucene.472066.n3.nabble.com/Error-loading-class-solr-DisMaxRequestHandler-after-upgrade-from-solr350-to-431-tp4083477p4085699.html Sent from the Solr - User mailing list archive at Nabble.com.

Re: High memory usage on solr 3.6

Thanks for the quick answer. We are experiencing slower indexing speed (x2/x2.5 time consumed) and the memory consumed during this process oscilates between 10g (max allowed, so JVM performs full gc's) and 8gb, but never goes under 8gb. I'll try your suggestions and see how it goes. Thanks! El

Re: "Path must not end with / character" error during performance tests

It looks like you've specified your zkHost (?) as something like machine:port/solr/ rather than machine:port/solr Is that possible? Best, Erick On Tue, Aug 20, 2013 at 12:10 PM, Tanya wrote: > Hi, > > I have integrated SolrCloud search in some system with single shard and > works fine on sin

Re: Best way to version config files

Sorry, the Contest has just finished: http://blog.outerthoughts.com/2013/08/wrap-up-of-solr-usability-contest/ But there is always JIRA. Or this could probably be implemented as some sort of JS-subversion ( similar to https://github.com/creationix/js-git) + custom Admin UI plugin. Regards, A

Autosuggest on very large index

Using 4.4.0 - I would like to be able to do an autosuggest query against one of the fields in our index and have the results be limited by an fq. I can get exactly the results I want with a facet query using a facet.prefix, but the first query takes ~5 minutes to run on our QA env (~240M docs).

Flushing cache without restarting everything?

Hi All, Is there a way to flush the cache of all nodes in a Solr Cloud (by reloading all the cores, through the collection API, ...) without having to restart all nodes? Thanks

RE: Autosuggest on very large index

I am not entirely sure but the Suggester's FST uses prefixes so you may be able to prefix the value you otherwise use for the filter query when you build the suggester. -Original message- > From:Greg Preston > Sent: Tuesday 20th August 2013 20:00 > To: solr-user@lucene.apache.org > Sub

Sharing SolrCloud collection configs w/overrides

Hey guys, I have a situation where I have a lot of collections that share the same core config in Zookeeper. For each of my SolrCloud collections, 99.9% of the config (schema.xml, solrcloud.xml) are the same, only the DataImportHandler parameters are different for different database names/credenti

Re: Autosuggest on very large index

The filter query would be on a different field (clientId) than the field we want to autosuggest on (title). Or are you proposing we index a compound field that would be clientId+titleTokens so we would then prefix the suggester with clientId+userInput ? Interesting idea. -Greg On Tue, Aug 20,

Re: Flushing cache without restarting everything?

Why? What are you trying to acheive with this? --wunder On Aug 20, 2013, at 11:04 AM, Jean-Sebastien Vachon wrote: > Hi All, > > Is there a way to flush the cache of all nodes in a Solr Cloud (by reloading > all the cores, through the collection API, ...) without having to restart all > nodes?

Re: Autosuggest on very large index

Sounds like a problem for DocValues - assuming the number of unique values fits reasonably in memory to avoid I/O. How many unique values do you have or contemplate for two your billion documents? Two possibilities: 1. You need a lot more hardware. 2. You need to scale back your ambitions.

RE: Flushing cache without restarting everything?

I just want to run benchmarks and want to have the same starting conditions. > -Original Message- > From: Walter Underwood [mailto:wun...@wunderwood.org] > Sent: August-20-13 2:06 PM > To: solr-user@lucene.apache.org > Subject: Re: Flushing cache without restarting everything? > > Why? Wh

Re: Autosuggest on very large index

DocValues looks interesting, a non-inverted field. I'll play with it a bit and see how it works. Thanks for the suggestion. I don't know how many total terms we've got, but each "document" is only 2-5 words/terms on average, and there is a TON of overlap between docs. -Greg On Tue, Aug 20,

Getting the shard a document lives on in resultset

Hi, We have several shards which we're querying across using distributed search. This initial search only returns basic information to the user. When a user requests more information about a document, we do a separate query using only the uniqueID for that document. The problem is, I don't know

Distributed MLT is slow

Before I file an issue on this, I wanted to bring it up here, so I can see if there's something I'm overlooking. Distributed MLT is very very slow for me. I can make it work, but a QTime of one to two minutes in production isn't acceptable. Sending a non-distributed MLT request directly to a

Re: Distributed MLT is slow

Is distributed MLT officially released or you are using a patch? El martes, 20 de agosto de 2013, Shawn Heisey escribió: > Before I file an issue on this, I wanted to bring it up here, so I can see > if there's something I'm overlooking. > > Distributed MLT is very very slow for me. I can make i

Re: Getting the shard a document lives on in resultset

I know I've done this in a search via the admin console, but I can't remember/find the exact syntax right now... -Greg On Tue, Aug 20, 2013 at 12:56 PM, AdamP wrote: > Hi, > > We have several shards which we're querying across using distributed search. > This initial search only returns basic i

Re: Getting the shard a document lives on in resultset

Found it. Add "[shard]" to your fl. -Greg On Tue, Aug 20, 2013 at 1:24 PM, Greg Preston wrote: > I know I've done this in a search via the admin console, but I can't > remember/find the exact syntax right now... > > -Greg > > > On Tue, Aug 20, 2013 at 12:56 PM, AdamP wrote: >> Hi, >> >> We ha

Re: custom hashing across cloud & shards

But this discussion says that if you don't specify numshards, you can still have shards and you can have custom hashing http://lucene.472066.n3.nabble.com/Solr-4-1-Custom-Hashing-DIH-td4036316.html, and I'm pretty sure I've read that elsewhere as well. Does anyone have more information on that? Th

Re: Getting the shard a document lives on in resultset

That was shockingly easy. Thanks so much! -- View this message in context: http://lucene.472066.n3.nabble.com/Getting-the-shard-a-document-lives-on-in-resultset-tp4085731p4085747.html Sent from the Solr - User mailing list archive at Nabble.com.

edismax type query in different sets of fields?

Hi there, suppose I have documents with fields that can be loosely grouped by what kind of information they contain. As an example, imagine the documents representing people and some fields are related to how they're called (name, surname, nickname, etc...) and some fields are related to where th

Re: Distributed MLT is slow

On 8/20/2013 2:21 PM, Luis Cappa Banda wrote: Is distributed MLT officially released or you are using a patch? It was added by SOLR-788 to version 4.1.0. It has not received very much scrutiny yet. In addition to SOLR-5125, I have discovered SOLR-5161. Both issues have workarounds, and 51

Re: spatial search, geofilt does not work

Technically, the query was valid in that it parsed. If it didn't parse then you'd get an error. If the notion of a "default field" didn't exist, and I wish it didn't, then you would most certainly have gotten an error. Perhaps you are using the defaultSearchField feature in the schema; please do

Re: What filter to use to search with spaces omitted/included between words?

Thanks Tamanjit and Erick. I tried out the filters, most of the usecases work except "q=bestbuy". As mentioned by Erick, that is a hard one to crack. I am looking into DictionaryCompoundWordTokenFilterFactory but compound words like these: http://www.manythings.org/vocabulary/lists/a/words.php?f=c

Re: What filter to use to search with spaces omitted/included between words?

Let me take that back, this actually works. q=bestbuy matches "Best Buy" and documents are returned. I was using , replacing

Re: Create core with user-defined the instanceDir

Hi Erick Thanks for your reponse. In fact ,I create an core by send the Http request with parameters above instead of set the configuration files. http://10.7.23.125:8080/solr/admin/cores?action=CREATE&name=testcore8_1&shard=shard13&collection.configName=myconf&schema=schema.xml&config

Re: "Path must not end with / character" error during performance tests

Hi Erick,Tanya Happened I saw the mail,and I would like report anothe relative issue. I create a core through the solr/admin URL. when I set the collection name to / , the catalina.out of the tomcat send lots of exceptions -Path must not end with / character-. Worse, It seems never t

Re: Distance sort on a multi-value field

The distance sorting code in SOLR-2155 is roughly equivalent to the code that RPT uses (RPT has its lineage in SOLR-2155 after all). I just reviewed it to double-check. It's possible the behavior is slightly better in SOLR-2155 because the cache (a Solr cache) contains normal hard-references wher

Re: Solr 4.4 Query ignoring null values

Hi Hoss, Below is the debug output for the query1. We have values for physical_city. It is not an empty valued field. when i said no-value, it was that the query does not have a value. *business_name:Catherine AND physical_city:""* business_name:Catherine AND physical_city:"" (business_name:K0R

4.3 Cloud looks good on the outside, but lots of errors in the logs

Hi, I'm running a solr 4.3 cloud in a 3 machine setup that has the following configuration: each machine is running 3 zookeepers on different ports each machine is running a jetty instance PER zookeeper.. Essentially, this gives us the ability to host 3 isolated clouds across the 3 machines. 3 sh

Solr Filter Query

Hi, Is there any limit on how big a filter query can be ? What are the values that should be set properly for handling big filter queries. thanks, Prasi

Re: convert text file to solr document where delimiter fields are fields of document

Thanks all of you for quick reply. Your guidance helps me lot. I am new to Solr so may be it is basic questions : 1) Is there any way we can import csv file using post.jar? (As I am windows user) 2) I declared DataImportHandlers, I use Solr Admin (default solr UI) to import the data for DIH, so h