Re: Why Slop doens't match anything?

On Wed, Aug 3, 2011 at 1:33 AM, Alexander Ramos Jardim wrote: [...] > I am not using dismax. I didn't find the solution for the problem. I just > made a full-import and the problem ended. Still odd. [...] Maybe you changed the type of the field in question, or changed positionIncrementGap for the

Re: how to get row no. of current record

On Wed, Aug 3, 2011 at 9:35 AM, Ranveer wrote: > Hi Anshum, > > Thanks for reply. > > My requirement is to get result start from current id. For this I need to > set start rows. > I am looking something like Jonty's post : > http://lucene.472066.n3.nabble.com/previous-and-next-rows-of-current-reco

Re: Query on multi valued field

Thank you. This logic works for me. Thanks a lot. Regards, Rajani Maski On Wed, Aug 3, 2011 at 1:21 AM, Chris Hostetter wrote: > > : The query is get only those documents which have multiple elements > for > : that multivalued field. > : > : I.e, doc 2 and 3 should be returned from the

Re: PivotFaceting in solr 3.3

Hi Pranav, I know Pivot faceting is a feature in solr 4.0 But i want is there any patch that can make pivot faceting possible in solr3.3. Thanks! Isha On Wednesday 03 August 2011 10:23 AM, Pranav Prakash wrote: From what I know, this is a feature in Solr 4.0 marked as SOLR-

Re: PivotFaceting in solr 3.3

>From what I know, this is a feature in Solr 4.0 marked as SOLR-792 in JIRA. Is this what you are looking for ? https://issues.apache.org/jira/browse/SOLR-792 *Pranav Prakash* "temet nosce" Twitter | Blog | Google

PivotFaceting in solr 3.3

Hi All! Can anyone tell which patch should I apply to solr 3.3 to enable pivot faceting in it. Thanks in advance! Isha garg

Re: how to get row no. of current record

Hi Anshum, Thanks for reply. My requirement is to get result start from current id. For this I need to set start rows. I am looking something like Jonty's post : http://lucene.472066.n3.nabble.com/previous-and-next-rows-of-current-record-td3187935.html thanks Ranveer On Wednesday 03 Augus

Re: how to get row no. of current record

Hi Ranveer, I'm not really sure if you mean lucene's docid (as that's the auto increment id used in here). Why would you need that in the first place? I'd suggest you not to expose that. Let me know in case you wanted something else. Also, perhaps you could explain the exact usecase and one of us g

Re: how to get row no. of current record

any help.. On Tuesday 02 August 2011 11:22 PM, Ranveer wrote: Hi, How to know the row number of current record. i.e : suppose we have 10 million record indexed. Currently I am on 5th records and id of the this record is XYZ00234, how to know that the current record rows no is 5th.

Re: SEVERE: org.apache.solr.common.SolrException: Error loading class 'solr.ICUTokenizerFactory'

I copied the file apache-solr-analysis-extras-3.3.0.jar into solr's lib folder. Now the error is different - SEVERE: java.lang.NoClassDefFoundError: org/apache/solr/analysis/BaseTokenizerFactory Please help. Satish On Tue, Aug 2, 2011 at 5:23 PM, Robert Muir wrote: > did you add the analysis-

Solr 3.3.0 set-up difficulty

Hey, I downloaded solr 3.3.0 and followed the tutorial to check my installation. I do not see any context in the jetty server. This is all I see when I run java -jar start.jar 2011-08-02 18:06:08.439:INFO::Logging to STDERR via org.mortbay.log.StdErrLog 2011-08-02 18:06:08.603:INFO::jetty-6.1-SN

Re: IMP: indexing taking very long time

Can somebody answer this? What should be the best strategy for optimize (when million of messages we are indexing for a new registered user) Thanks Naveen On Tue, Aug 2, 2011 at 5:36 PM, Naveen Gupta wrote: > Hi > > We have a requirement where we are indexing all the messages of a a thread, >

Re: I can't pass the unit test when compile from apache-solr-3.3.0-src

On 7/29/2011 5:26 PM, Chris Hostetter wrote: Can you please be specific... * which test(s) fail for you? * what are the failures? Any time a test fails, that info appears in the "ant test" output, and the full details or all tests are written to build/test-results you can run "ant test-repo

SolrCloud: is there a programmatic way to create an ensemble

I have multiple SolrCloud instances, each running its own Zookeeper (Solr launched with -DzkRun). I would like to create an ensemble out of them. I know about -DzkHost parameter, but can I achieve the same programmatically? Either with SolrJ or REST API? Thanks, Yury

Re: Solr with many indexes

Hello, >From: Vikram Kumar > >We have a multi-tenant Solr deployment with a core for each user. > >Due to the limitations we are facing with number of cores, >lazy-loading (and associated warm-up times), we are researching about >consolidating several users into one core with queries limited by

Re: Matching queries on a per-element basis against a multivalued field

Thanks for the history and the current state of trunk, guys. It sounds like it's rather stable for serious use... in which case it's probably ready for a release, but let's not go back in circles. :) I'll give it a shot sometime. Thanks, again! -- View this message in context: http://lucene.4720

Re: Solr with many indexes

We have a multi-tenant Solr deployment with a core for each user. Due to the limitations we are facing with number of cores, lazy-loading (and associated warm-up times), we are researching about consolidating several users into one core with queries limited by user-id field. My question is about

Re: Matching queries on a per-element basis against a multivalued field

On Aug 2, 2011, at 5:47 PM, eks dev wrote: > Sure, I know..., > the point I was trying to make, "if someone serious like Lucid is > using solr 4.x as a core technology for own customers, the trunk could > not be all that bad" => release date not as far as 2012 :) Oh the current trunk is most def

Re: Matching queries on a per-element basis against a multivalued field

Sure, I know..., the point I was trying to make, "if someone serious like Lucid is using solr 4.x as a core technology for own customers, the trunk could not be all that bad" => release date not as far as 2012 :) On Tue, Aug 2, 2011 at 11:33 PM, Smiley, David W. wrote: > "LucidWorks Enterprise"

Re: Matching queries on a per-element basis against a multivalued field

"LucidWorks Enterprise" (which is more than Solr, and a modified Solr at that) isn't free; so you can't extract the Solr part of that package and use it unless you are willing to pay them. Lucid's "Certified Solr", on the other hand, is free. But they have yet to bump that to trunk/4.x; it was

Re: Matching queries on a per-element basis against a multivalued field

Well, Lucid released "LucidWorks Enterprise" with " Complete Apache Solr 4.x Release Integrated and tested with powerful enhancements" Whatever it means for solr 4.0 On Tue, Aug 2, 2011 at 11:10 PM, David Smiley (@MITRE.org) wrote: > My best guess (and it is just a guess) is between December

MultiSearcher/ParallelSearcher - searching over multiple cores?

Hi *, I searched the web for an answer whether it is possible in SOLR to make a query over several cores with all features(boosting, pagination, highlighting) and so on out of the box. In Lucene it it possible with MultiSearcher/Parallelsearcher. I do not mean "Distributed Search" or merging

Re: Matching queries on a per-element basis against a multivalued field

My best guess (and it is just a guess) is between December and March. The roots of Solr 4 which triggered the major version change is known as "flexible indexing" (or just "flex" for short amongst developers). The genesis of it was posted to JIRA as a patch on 18 November 2008 -- LUCENE-1458 (a

TikaEntityProcessor is filling logs

I want to use TikaEntityProcessor for URLs defined in "link" from the parent entity. This field can be empty as well. While the dataimport is working OK, the logging is filling up with exceptions in case link is null. Is there way to prevent this? -- View this message in context:

Re: Matching queries on a per-element basis against a multivalued field

Thanks. I saw the related jira issue but didn't follow closely enough to see the cross-core join being added later. Any idea/hint on when I can expect Solr 4 to be released? -- View this message in context: http://lucene.472066.n3.nabble.com/Matching-queries-on-a-per-element-basis-against-a-multi

Re: Why Slop doens't match anything?

Hey dude, Sorry for the long absence. (Need to check my personal email more times o0) I am not using dismax. I didn't find the solution for the problem. I just made a full-import and the problem ended. Still odd. 2011/7/27 Gora Mohanty > On Wed, Jul 27, 2011 at 8:38 PM, Alexander Ramos Jardim

Re: Query on multi valued field

: The query is get only those documents which have multiple elements for : that multivalued field. : : I.e, doc 2 and 3 should be returned from the above set.. The only way to do something like this is to add a field when you index your documents that contains the number and then filter on

Re: performance crossover between single index and sharding

On 8/2/2011 12:06 PM, Jonathan Rochkind wrote: What's the reasoning behind having three shards on one machine, instead of just combining those into one shard? Just curious. I had been thinking the point of shards was to get them on different machines, and there'd be no reason to have multiple

Re: performance crossover between single index and sharding

With low qps and multi-core servers, I believe one reason to have multiple shards on one server is to provide better parallelism for a request, and thus reduce your response time. -- Ken On Aug 2, 2011, at 11:06am, Jonathan Rochkind wrote: > What's the reasoning behind having three shards on

RE: performance crossover between single index and sharding

Hi Jonothan and Markus, >>Why 3 shards on one machine instead of one larger shard per machine? Good question! We made this architectural decision several years ago and I'm not remembering the rationale at the moment. I believe we originally made the decision due to some tests showing a sweetsp

Re: performance crossover between single index and sharding

Hi Tom, Very interesting indeed! But i keep wondering why some engineers choose to store multiple shards of the same index on the same machine, there must be significant overhead. The only reason i can think of is ease of maintenance in moving shards to a separate physical machine. I know that

Re: performance crossover between single index and sharding

What's the reasoning behind having three shards on one machine, instead of just combining those into one shard? Just curious. I had been thinking the point of shards was to get them on different machines, and there'd be no reason to have multiple shards on one machine. On 8/2/2011 1:59 PM, B

RE: performance crossover between single index and sharding

Hi Markus, Just as a data point for a very large sharded index, we have the full text of 9.3 million books with an index size of about 6+ TB spread over 12 shards on 4 machines. Each machine has 3 shards. The size of each shard ranges between 475GB and 550GB. We are definitely I/O bound. Our m

how to get row no. of current record

Hi, How to know the row number of current record. i.e : suppose we have 10 million record indexed. Currently I am on 5th records and id of the this record is XYZ00234, how to know that the current record rows no is 5th. thanks.. regards Ranveer

Re: Matching queries on a per-element basis against a multivalued field

On Aug 2, 2011, at 1:09 PM, Suk-Hyun Cho [via Lucene] wrote: > I appreciate your replies and ideas. > > SpanQuery would work, and I'll look into this further. However, what about > the original question? Is there no way to match documents on a per-element > basis against a multivalued field?

Re: Matching queries on a per-element basis against a multivalued field

I appreciate your replies and ideas. SpanQuery would work, and I'll look into this further. However, what about the original question? Is there no way to match documents on a per-element basis against a multivalued field? If not, would it perhaps make sense to create a feature request? Also, rega

Re: Jetty error message regarding EnvEntry in WebAppContext

On 8/2/2011 11:42 AM, Marian Steinbach wrote: Can anyone tell me how a working configuration for Jetty 6.1.22 would have to look like? You know that Solr distro comes with a jetty with a Solr in it, right, as an example application? Even if you don't want to use it for some reason, that would

Re: lucene/solr, raw indexing/searching

In your solr schema.xml, are the fields you are using defined as text fields with analyzers? It sounds like you want no analysis at all, which probably means you don't want text fields either, you just want string fields. That will make it impossible to search for individual tokens though, sear

RE: lucene/solr, raw indexing/searching

dhastings, my recommendation for the approaches from both sides ... Lucene: try on a whitespace analyzer for size Analyzer an = new WhitespaceAnalyzer(Version.LUCENE_31); Solr: in your /index/solr/conf/schema.xml ... -craig -Original Message

Re: performance crossover between single index and sharding

Actually, i do worry about it. Would be marvelous if someone could provide some metrics for an index of many terabytes. > [..] At some extreme point there will be diminishing > returns and a performance decrease, but I wouldn't worry about that at all > until you've got many terabytes -- I don't

Re: performance crossover between single index and sharding

That's a fantastic answer, Shawn. To more directly answer Bernd's question: Bernard, shard your data once you've done reasonable performance optimizations to your single core index setup (see Chapter 9 of my book) and the query response time isn't meeting your requirements in spite of this. Solr

Re: changing the root directory where solrCloud stores info inside zookeeper File system

Thanks A lot mark, Since My SolrCloud code was old I tried downloading and building the newest code from here https://svn.apache.org/repos/asf/lucene/dev/trunk/ I am using tomcat6 I manually created the sc sub-directory in my zooKeeper ensemble file-system I used this connection String to my ZK ens

Re: Matching queries on a per-element basis against a multivalued field

Suk, You're hitting on a well known limitation with Lucene, and the "solutions" are work-arounds that may be unacceptable depending on the specifics of your case. Solr 4.0 (trunk)'s support for Joins is definitely an up and coming option, as Mike pointed out. Kersen's suggestion of using an ind

Jetty error message regarding EnvEntry in WebAppContext

Hi! I am trying to deploy Solr under Jetty 6.1.22-1ubuntu1 (installed the "jetty" and "libjetty-extra-java" packages). However, it seems as if I can't get the webapp configuration set right. With this configuration... ... ** /solr/home /opt/exptbx-solr/solr true ... I get the e

Re: How to cut off hits with score below threshold?

I've created an issue to track this funny behaviour: https://issues.apache.org/jira/browse/SOLR-2689 On Tuesday 02 August 2011 16:46:18 Markus Jelsma wrote: > Be careful with that approach as it will return score=1.0f for all > documents (fl=*,score). This, however, doesn't affect the outcome of t

CoreContainer from CommonsHttpSolrServer

Hi everybody, I'm using Solr (with multiple cores) in a Webapp and access the differnt cores using CommonsHttpSolrServer. As I would like to know, which cores are configured and what there status is I would like to get an instance of CoreContainer. The site http://wiki.apache.org/solr/CoreAdmin t

lucene/solr, raw indexing/searching

Hello, I am trying to get lucene and solr to agree on a completely Raw indexing method. I use lucene in my indexers that write to an index on disk, and solr to search those indexes that i create, as creating the indexes without solr is much much faster than using the solr server. are there settin

Re: How to cut off hits with score below threshold?

Be careful with that approach as it will return score=1.0f for all documents (fl=*,score). This, however, doesn't affect the outcome of the frange. Feels like a bug though On Tuesday 02 August 2011 16:29:16 karsten-s...@gmx.de wrote: > Hi Otis, > > is this the same question as > http://lucene.4

Re: How to cut off hits with score below threshold?

Hi Otis, is this the same question as http://lucene.472066.n3.nabble.com/Filter-by-relevance-td1837486.html ? If yes, perhaps something like (http://search-lucene.com/m/4AHNF17wIJW1/) q={!frange l=0.85}query($qq) qq= will help? (BTW, a also would like to specify a custom Collector via API in Sol

RE: Spell Check

The most likely problem is forgetting to specify "spellcheck.build=true" on the first query since the last restart. This builds the spell check dictionary used by the IndexBasedSpellChecker. You should put this in a warming query or alternatively, specify build-on-commit or build-on-optimize.

Re: Matching queries on a per-element basis against a multivalued field

Hi Suk-Hyun Cho, if "myFriend" is the unit of retrieval you should use this as lucene document with the fields "isCool" "gender" "bloodType" ... if you realy want to insert all "myFriends" in one field like your myFriends = [ "isCool=true SOME_JUNK_HERE gender=female bloodType=O", "isCoo

How to cut off hits with score below threshold?

Hello, If one wanted to cut off hits whose score is below some threshold (I know, I know, one doesn't typically want to do this), what are the most elegant options? I can think of 2 options, but I wonder if there are better choices: 1) custom Collector (problem: one can't specify a custom Collec

Re: performance crossover between single index and sharding

On 8/2/2011 4:44 AM, Bernd Fehling wrote: Is there any knowledge on this list about the performance crossover between a single index and sharding and when to change from a single index to sharding? E.g. if index size is larger than 150GB and num of docs is more than 25 mio. then it is better to

Re: Matching queries on a per-element basis against a multivalued field

You have a few choices: 1) flatten your field structure - like your "undesirable" example, but wouldn't you want to have the document identifier as a field value also? 2) use phrase queries to make sure the key/value pairs are adjacent 3) use a join query That's all I can think of -Mike On

Re: Store complete XML record (DIH & XPathEntityProcessor)

Hi g, Hi Chantal I had the same problem. You can use XPathEntityProcessor but you have to insert an xsl. The drawback is performance "wasting": See http://lucene.472066.n3.nabble.com/DIH-Enhance-XPathRecordReader-to-deal-with-body-FLATTEN-true-and-body-h1-td2799005.html Best regards Karsten -

Re: xpath expression not working

Hi abhayd, XPathEntityProcessor does only support a subset of xpath, like div[@id="2"] but not [id=2] Take a look to https://issues.apache.org/jira/browse/SOLR-1437#commentauthor_12756469_verbose I solve this problem by using xslt a preprocessor (with full xpath). The drawback is performance "wa

Re: DIH + signature

Follow-up on this issue. I eventually found the problem. The naming scheme changed from solr 3.2 onwards. The line as it states in the documentation: dedupe should now be: dedupe https://issues.apache.org/jira/browse/SOLR-2105 -- View this message in context: http://lucene.472066.n3.nabble.

Re: Solr Master-slave master failover without data loss

Not OOB. You say that the index updates, but if the data hasn't been committed, it isn't really in the index. After the commit (which varies time-wise depending on merges etc.) the next replication from the slave should get the new index, regardless of whether the master has gone down or not. One

Re: Different options for autocomplete/autosuggestion

You have to tell us more information about what "not right" means. Please review: http://wiki.apache.org/solr/UsingMailingLists Best Erick On Wed, Jul 27, 2011 at 6:12 AM, scorpking wrote: > HI Bell, > i used autocomplete in solr 3.1. same this: > >   >     >      autocomplete >       name="cla

Re: Solr 3.3 crashes after ~18 hours?

Assuming you are running on Linux, you might want to check /var/log/messages too (the location might vary), I think the kernel logs forced process termination there. I recall that the kernel will usually picks the process consuming the most memory, there may be other factors involved too. Franç

Re: Solr 3.3 crashes after ~18 hours?

Monitor your memory usage. I use to encounter a problem like this before where nothing was in the logs and the process was just gone. Turned out my system was out odd memory and swap got used up because of another process which then forced the kernel to start killing off processes. Google OOM lin

DIH + signature

Hi, I'm using solr 3.3 and want to add a signature field to solr to later be able to deduplicate search results using field collapsing. I'm using DIH to fill solr. Extract from solrconfig.xml true false signature ctcontent solr.update.processor.Lookup3Signatu

indexing taking very long time

Hi We have a requirement where we are indexing all the messages of a a thread, a thread may have attachment too . We are adding to the solr for indexing and searching for applying few business rule. For a user, we have almost many threads (100k) in number and each thread may be having 10-20 messa

Re: SEVERE: org.apache.solr.common.SolrException: Error loading class 'solr.ICUTokenizerFactory'

did you add the analysis-extras jar itself? thats what has this factory. On Tue, Aug 2, 2011 at 5:03 AM, Satish Talim wrote: > I am using Solr 3.3 on a Windows box. > > I want to use the solr.ICUTokenizerFactory in my schema.xml and added the > fieldType name="text_icu" as per the URL - > http://

RE: changing the root directory where solrCloud stores info inside zookeeper File system

Thanks A lot mark, Since My SolrCloud code was old I tried downloading and building the newest code from here https://svn.apache.org/repos/asf/lucene/dev/trunk/ I am using tomcat6 I manually created the sc sub-directory in my zooKeeper ensemble file-system I used this connection String to my ZK ens

Re: Solr 3.3 crashes after ~18 hours?

What do you mean by it just crashes? Does the process stops execution? Does it takes too long to respond which might result in lots of 503s in your application? Does the system run out of resources? Are you indexing and serving from the same server? It happened once with us that Solr was performin

Re: Solr 3.3 crashes after ~18 hours?

Strange, anything out of the ordinary in the syslog? On Tuesday 02 August 2011 12:01:35 alexander sulz wrote: > Hello folks, > > I'm using the latest stable Solr release -> 3.3 and I encounter strange > phenomena with it. > After about 19 hours it just crashes, but I can't find anything in the >

performance crossover between single index and sharding

Is there any knowledge on this list about the performance crossover between a single index and sharding and when to change from a single index to sharding? E.g. if index size is larger than 150GB and num of docs is more than 25 mio. then it is better to change from single index to sharding and ha

Re: Solr 3.3 crashes after ~18 hours?

Nope, none :/ Am 02.08.2011 12:33, schrieb Bernd Fehling: Any JAVA_OPTS set? Do not use "-XX:+OptimizeStringConcat" or "-XX:+AggressiveOpts" flags. Am 02.08.2011 12:01, schrieb alexander sulz: Hello folks, I'm using the latest stable Solr release -> 3.3 and I encounter strange phenomena wi

Re: Solr 3.3 crashes after ~18 hours?

Any JAVA_OPTS set? Do not use "-XX:+OptimizeStringConcat" or "-XX:+AggressiveOpts" flags. Am 02.08.2011 12:01, schrieb alexander sulz: Hello folks, I'm using the latest stable Solr release -> 3.3 and I encounter strange phenomena with it. After about 19 hours it just crashes, but I can't fin

Solr 3.3 crashes after ~18 hours?

Hello folks, I'm using the latest stable Solr release -> 3.3 and I encounter strange phenomena with it. After about 19 hours it just crashes, but I can't find anything in the logs, no exceptions, no warnings, no suspicious info entries.. I have an index-job running from 6am to 8pm every 10 mi

SEVERE: org.apache.solr.common.SolrException: Error loading class 'solr.ICUTokenizerFactory'

I am using Solr 3.3 on a Windows box. I want to use the solr.ICUTokenizerFactory in my schema.xml and added the fieldType name="text_icu" as per the URL - http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters#solr.ICUTokenizerFactory I also added the following files to my apache-solr-3.3.0\