Re: sending results of function query to range query

> I am not sure if I can use function > queries this way. I have a query like this"attributeX:[* TO > ?]" in my DB. I replace the ? with input from the front end. > Obviously, this works fine. However, what I really want to > do is "attributeX:[* TO (3 * ?)]" Is there anyway to embed > the results

Re: fieldCache problem OOM exception

Hi Erik, I will take some memory snapshots during the next week, but how can it be to get OOMs with one query? - I started with 6g for JVM --> 1 day until OOM. - increased to 8 g --> 2 days until OOM - increased to 10g --> 3.5 days until OOM - increased to 16g --> 5 days until OOM - currently 20g

Re: omitTermFreqAndPositions in a TextField fieldType

> Does it even make sense to use omitTermFreqAndPositions for > a TextField, or am I perhaps doing something I shouldn't > be? Yes it makes sense to disable tf, positions and norms if you don't need them. For example, I have this category_name field where its definition includes: omitNorms="true

Re: about the SolrServer server = new CommonsHttpSolrServer(URL);

> SolrServer server =  new CommonsHttpSolrServer(URL); > > through out the class. How can I improve the connection, in > my case: should I need to close the server after fetching > the result or CommonsHttpSolrServer(URL); will maintain at > their end. There is other way: I can make this as static

Re: Document Level Security (SOLR-1872 ,SOLR,SOLR-1834)

You'll need to be a bit careful using joins, as the performance hit can be significant if you have lots of cross-referencing to do, which I believe you would given your scenario. Your table could be setup to use the username as the key (for fast lookup), then map these to your own data class or co

Re: Document Level Security (SOLR-1872 ,SOLR,SOLR-1834)

Just to chip in my 2 cents: You know you can increase the max number of boolean clauses in the configuration files? Depending on your situation it might not be a permanent fix, but it could provide some instant relief. Constantijn On Fri, Jun 17, 2011 at 11:19 AM, Peter Sturge wrote: > You'll

difficult sort

Is this possible in 1.4.1 Return a result set sorted by a field but within Categorical groups, limited to 1 record per group Something like: group1 xxx (bottom of sorted field within group) group2 xxx (bottom of sorted field within group) etc is the only approach to issue multiple queries and col

Search failed even if it has the keyword .

Hello, solr-search failed even if it has the keyword . I am using solr (solr3.1 on ubuntu 10.10) for Indexing the tweets. I am indexing certain tweets, but solr do'nt return any result when I search any keyword from tweet. in Solr, tweet is stored as 'text'. below is the tweet which I index : *"RT

RE: Multiple indexes

> I think there are reasons to use seperate indexes for each document type > but do combined searches on these indexes > (for example if you need separate TFs for each document type). I wonder if in this precise case it wouldn't be pertinent to have a single index with the various document types

Re: Search failed even if it has the keyword .

First check, in your schema.xml, which is your default search field. Also look if you are using WordDelimiterFilterFactory in your schema.xml for the specific field. This would tokenize your words on every capital letter, so, for the word "DescribeYourImageWithAMovieTitle" will be broken into multi

Re: Document Level Security (SOLR-1872 ,SOLR,SOLR-1834)

> Do you mean that we  have current Index as it is and have a separate core > which  has only the user-id ,product-id relation and at while querying ,do a > join between the two cores based on the user-id. Exactly. You can index user-id, product-id relation either to the same core or to different c

Re: difficult sort

I'm not sure, but have looked at Collapsing feature in SOLR yet? You may have to apply patch for 1.4.1 version, if this is what u want? -- View this message in context: http://lucene.472066.n3.nabble.com/difficult-sort-tp3075563p3075661.html Sent from the Solr - User mailing list archive at Nabb

Re: Search failed even if it has the keyword .

Hi Pravesh, this is how my schema looks for 'text' field : My default search field is 'defaultquery' and I am copy field is : And My tweet is indexed into 'title'. On 17 June 2011 15:46, pravesh wrote: > First check, in your schema.xml, which is your default search field.

Re: difficult sort

Thanks for the reply Pravesh We can't go to trunk or apply patch to production so the field collapsing goodness is out of reach for now. Is multiple queries the only way to go for this ? On 17 June 2011 11:23, pravesh wrote: > I'm not sure, but have looked at Collapsing feature in SOLR yet? You

Re: Search failed even if it has the keyword .

What is the type for the field's defaultquery & title in your schema.xml ? -- View this message in context: http://lucene.472066.n3.nabble.com/Search-failed-even-if-it-has-the-keyword-tp3075626p3075797.html Sent from the Solr - User mailing list archive at Nabble.com.

Re: difficult sort

Yes. Then I beleive you would need multiple queries -- View this message in context: http://lucene.472066.n3.nabble.com/difficult-sort-tp3075563p3075802.html Sent from the Solr - User mailing list archive at Nabble.com.

Re: Search failed even if it has the keyword .

very very thanks Parvesh. my 'title' was 'text' whereas 'defaultquery' was 'query_text'. I change my 'defaultquery' to 'text' and problem is solved. thanks again. On 17 June 2011 16:57, pravesh wrote: > What is the type for the field's defaultquery & title in your schema.xml ? > > -- > Vie

Re: fieldCache problem OOM exception

Sorry, it was late last night when I typed that... Basically, if you sort and facet on #all# the fields you mentioned, it should populate the cache in one go. If the problem is that you just have too many unique terms for all those operations, then it should go bOOM. But, frankly, that's unlikely

Re: Indexing-speed issues (chart included)

Well, it's kinda hard to say anything pertinent with so little information. How are you indexing things? What kind of documents? How are you feeding docs to Solr? You might review: http://wiki.apache.org/solr/UsingMailingLists Best Erick On Fri, Jun 17, 2011 at 8:10 AM, Mark Schoy wrote: > Hi,

Re: Controlling Tika's metadata

I have the same problem with discarding the metadata title. I thought the parameter "captureAttr" (can be provided at the solrconfig.xml and via get/post as a parameter) is responsible for that? I set it to false in in the xml and as a parameter, still, I get "not multivalued field" errors due

Re: Indexing-speed issues (chart included)

Sorry, here are some details: requestHandler: XmlUpdateRequesetHandler protocol: http (10 concurrend threads) document: 1kb size, 15 fields cpu load: 20% memory usage: 50% But generally speaking, is that normal or must be something wrong with my configuration, ... 2011/6/17 Erick Erickson > W

Re: fieldCache problem OOM exception

Hi Erik, as far as I can see with MemoryAnalyzer from the heap: - the class fieldCache has a HashMap - one entry of the HashMap is FieldCacheImpl$StringIndex which is "mister big" - FieldCacheImpl$StringIndex is a WeakHashMap - WeakHashMap has three entries -- 63.58 percent of heap -- 8.14 perce

RE: Multiple indexes

> > (for example if you need separate TFs for each document type). > > I wonder if in this precise case it wouldn't be pertinent to > have a single index with the various document types each > having each their own fields set. Isn't TF calculated field by field ? Oh, you are right :) So i will

RE: difficult sort

What if you were to set up a new field, which is the concatenation of your 'field' and 'category group', and then facet on that? How many combinations would we be talking about here? And - against what field(s) do you run your query? We did something a bit similar, where we wanted an 'author'

Re: Indexing-speed issues (chart included)

No, generally this isn't what I'd expect. There will be periodic slowdowns when segments are flushed (I'm assuming you're not using trunk, there have been speedups here, see: http://blog.jteam.nl/2011/04/01/gimme-all-resources-you-have-i-can-use-them/) Does your config have any parameters set? Y

Re: SOlR -- Out of Memory exception

I did that , but when I split them into 5 mill records, the first file went through fine, when I started processing the second file SOLR hit an OOM again: org.apache.solr.common.SolrException log SEVERE: java.lang.OutOfMemoryError: Java heap space at org.apache.lucene.index.FreqProxTermsWri

Re: SOlR -- Out of Memory exception

On Fri, Jun 17, 2011 at 1:30 AM, pravesh wrote: > If you are sending whole CSV in a single HTTP request using curl, why not > consider sending it in smaller chunks? Smaller chunks should not matter - Solr streams from the input (i.e. the whole thing is not buffered in memory). It could be relate

Re: Document Scoring

Thank you this is something that I wanted to hear. I knew the design was most likely flawed because I have never done Solr or any kind of full text searching, but needed an unbiased opinion. I think that if I were to tune the configs and pay close attention to the logs with lots of performance te

Re: Document Scoring

I think this is the way to go. When trying to minimize latency, there are two statistics to pay particular attention to on your #searchers#. 1> What is the warmup time for your caches? 2> What is your polling interval? Make sure your polling interval is, say, at least three times longer than your

Re: Controlling Tika's metadata

This is the same issue I brought up in this thread: http://search-lucene.com/m/s8sOH1YG1TP As a workaround I wrote an UpdateProcessor to copy/move fields around (SOLR-2599). I think we need a separate fmap for TIKA generated fields (say tmap), so the problem could be fixed by: tmap.title=tika

Re: Document Level Security (SOLR-1872 ,SOLR,SOLR-1834)

Thanks ,Peter . This very much seems to be the solution that I should be going forward with .Thanks for your time and clear explanation. Regards Sujatha On Fri, Jun 17, 2011 at 2:49 PM, Peter Sturge wrote: > You'll need to be a bit careful using joins, as the performance hit > can be signi

Re: Document Level Security (SOLR-1872 ,SOLR,SOLR-1834)

Constantijn, I am aware of this and we have already increased max boolean clauses to <3500> from the default <1200> for all our 200+ instances . But the requirement is that we could havenumber of products running to several thousands for each of the instances and since is not defined ,

Re: Document Level Security (SOLR-1872 ,SOLR,SOLR-1834)

Alexey , We are not planning to upgrade our solr version at the moment as all is fine with the current version so far and hence would not be able to try this solution . Regards Sujatha On Fri, Jun 17, 2011 at 3:47 PM, Alexey Serba wrote: > > Do you mean that we have current Index as it is an

Re: Document Scoring

Thank you, I will give that a shot. Zarni

Token Factory attribute in filter tag

How does this attribute "token factory" within filter work? In this link [click here],There is the usage of token factory in the synonym filter tag. Here I see the white space token at index time then a synonym filter

become solr mail list user

Hi all I want to be notified about new features or fix bugs of solr

Re: become solr mail list user

RT On 6/17/11, trang van anh wrote: > Hi all > > > > I want to be notified about new features or fix bugs of solr > > -- Sent from my mobile device

REGEX Proper Usage?

All, I am having trouble getting my regex pattern to work properly. I have tried PatternReplaceFilterFactory after the standard tokenizer and PatternReplaceCharFilterFactory before it. It looks like this should work to remove everything except letters and numbers.

Solr Highlighting and fqs

I've read in several places that the Solr Highlighting component ignores fqs, is there a way to enable it for them? I have an interface where a user essentially continues to pare down the information they are looking for by adding fqs so the initial query is usually something like "John" and a use

Re: REGEX Proper Usage?

Maybe try ([^a-z0-9]+) Sent by CarrierPigeon On 17 Jun 2011, at 20:26, Adam Estrada wrote: > All, > > I am having trouble getting my regex pattern to work properly. I have tried > PatternReplaceFilterFactory after the standard tokenizer > > replacement=" " replace="all"/> > > and PatternRep

Re: SOlR -- Out of Memory exception

I commented the autocommit option and tried uploading the file (a smaller file now 5 million records) and I hit an oom again: Jun 17, 2011 2:32:59 PM org.apache.solr.common.SolrException log SEVERE: java.lang.OutOfMemoryError: Java heap space -- View this message in context: http://lucene.472066

Re: Solr Highlighting and fqs

Hi, Vote for this feature https://issues.apache.org/jira/browse/SOLR-1926 -- Jan Høydahl, search solution architect Cominvent AS - www.cominvent.com Solr Training - www.solrtraining.com On 17. juni 2011, at 21.48, Jamie Johnson wrote: > I've read in several places that the Solr Highlighting com

Caching queries.

I'm wondering if something like this is possible. Lets say I want to query 5000 objects all pertaining to a specific search and I want to return the top 100 or something and cache the rest on my solr server. The next time I get the same query or something with a new offset (lets say start from 10

Re: become solr mail list user

Then you probably want to subscribe to the lucene_dev list at: http://lucene.apache.org/java/docs/mailinglists.html#Java Developer List Or perhaps the commit list on that page.. Best Erick On Fri, Jun 17, 2011 at 2:20 PM, trang van anh wrote: > Hi all > > > > I want to be notified about new f

Re: Solr Highlighting and fqs

This is really not what I'm looking for. While I could see it accomplishing the job I'd prefer that the highlighting component take into account the fqs when doing highlighting. Again I can get around this by simply joining all the fqs with an AND and making them the query but this is limiting si

Re: Caching queries.

Well, it depends on how you've set the parameters in solrconfig.xml for the queryResultWindowSize. Note that this size is simply the size of a list of integers, so it's not a very expensive cache. Best Erick On Fri, Jun 17, 2011 at 6:26 PM, arian487 wrote: > I'm wondering if something like this

Solr and Tag Cloud

Does anyone have details of how to generate a tag cloud of popular terms across an entire data set and then also across a query?