Re: Would it be nuts to store a bunch of large attachments (images, videos) in stored but-not-indexed fields

2010-10-29 Thread Shashi Kant
On Fri, Oct 29, 2010 at 6:00 PM, Ron Mayer wrote: > I have some documents with a bunch of attachments (images, thumbnails > for them, audio clips, word docs, etc); and am currently dealing with > them by just putting a path on a filesystem to them in solr; and then > jumping through hoops of keep

Re: Looking for Developers

2010-10-29 Thread Dennis Gearon
LOL! We ARE programmers, and we do like absolutes :-) Dennis Gearon Signature Warning It is always a good idea to learn from your own mistakes. It is usually a better idea to learn from others’ mistakes, so you do not have to make them yourself. from 'http://blogs.techrepublic.

Re: replication not working between 1.4.1 and 3.1-dev

2010-10-29 Thread Shawn Heisey
On 10/27/2010 8:34 PM, Shawn Heisey wrote: I started to upgrade my slave servers from 1.4.1 to 3.1-dev checked out this morning. Because of SOLR-2034 (new javabin version) the replication fails. Asking about it in comments on SOLR-2034 brought up the suggestion of switching to XML instead of

Would it be nuts to store a bunch of large attachments (images, videos) in stored but-not-indexed fields

2010-10-29 Thread Ron Mayer
I have some documents with a bunch of attachments (images, thumbnails for them, audio clips, word docs, etc); and am currently dealing with them by just putting a path on a filesystem to them in solr; and then jumping through hoops of keeping them in sync with solr. Would it be nuts to stick the i

Solr + Zookeeper Integration

2010-10-29 Thread Claudio Devecchi
Hi people, I'm trying to configure a little solr cluster but I need to shard the documents. I configured my solr with core0 (/opt/solr/core0) and installer the zookeeper (/opt/zookeeper). 1. On my solrconfig.xml I added the lines below: host1:2181 http://host1:8983/solr/core0 5000

Re: NOT keyword - doesn't work with dismax?

2010-10-29 Thread Scott K
I couldn't even get the bq= to work with negated queries, although with edismax, negated queries work with just q=-term Works: /solr/select?qt=edismax&q=-red Here is the failed attempt with dismax /solr/select?qt=dismax&rows=1&indent=true&q=-red&bq=*:*^0.001&echoParams=all&debugQuery=true { "r

Re: SolrCore.getSearcher() and postCommit()

2010-10-29 Thread Yonik Seeley
On Fri, Oct 29, 2010 at 5:36 PM, Grant Ingersoll wrote: > Is it OK to call and increment a Searcher ref (i.e. SolrCore.getSearcher()) > in a SolrEventListener.postCommit() hook as long as I decrement it when I am > done?  I need to get a handle on an IndexReader so I can dump out a portion > of

Re: How can I disable fsync()?

2010-10-29 Thread Grant Ingersoll
On Oct 29, 2010, at 2:11 PM, Igor Chudov wrote: > Thanks to all and I made Solr work very well on one newer machine. > > Now I am setting up Solr on an older server with an IDE hard drive. > > Unfortunately, populating the index takes FOREVER due to > Solr/Lucene/Tomcat calling fsync() a lot af

SolrCore.getSearcher() and postCommit()

2010-10-29 Thread Grant Ingersoll
Is it OK to call and increment a Searcher ref (i.e. SolrCore.getSearcher()) in a SolrEventListener.postCommit() hook as long as I decrement it when I am done? I need to get a handle on an IndexReader so I can dump out a portion of the index to an external process. Thanks, Grant

Re: documentCache clarification

2010-10-29 Thread Yonik Seeley
On Fri, Oct 29, 2010 at 4:21 PM, Chris Hostetter wrote: > > : > Why don't we just include the start & rows (modulo the window size) in > : > the cache key? > : > : The implementation of equals() would be rather difficult... actually > : impossible w/o abusing the semantics. > : It would also be im

Re: documentCache clarification

2010-10-29 Thread Chris Hostetter
: > Why don't we just include the start & rows (modulo the window size) in : > the cache key? : : The implementation of equals() would be rather difficult... actually : impossible w/o abusing the semantics. : It would also be impossible w/o the Map implementation guaranteeing : what object was on

Re: documentCache clarification

2010-10-29 Thread Yonik Seeley
On Fri, Oct 29, 2010 at 3:49 PM, Chris Hostetter wrote: > > : This is a limitation in the SolrCache API. > : The key into the cache does not contain rows, so the cache returns the > : first 10 docs and increments it's hit count.  Then the cache user > : (SolrIndexSearcher) looks at the entry and d

Re: Custom Sorting in Solr

2010-10-29 Thread Yonik Seeley
On Fri, Oct 29, 2010 at 3:39 PM, Ezequiel Calderara wrote: > Hi all guys! > I'm in a weird situation here. > We have index a set of documents which are ordered using a linked list (each > documents has the reference of the previous and the next). > > Is there a way when sorting in the solr search,

Re: documentCache clarification

2010-10-29 Thread Chris Hostetter
: This is a limitation in the SolrCache API. : The key into the cache does not contain rows, so the cache returns the : first 10 docs and increments it's hit count. Then the cache user : (SolrIndexSearcher) looks at the entry and determines it can't use it. Wow, I never realized that. Why don't

RE: Custom Sorting in Solr

2010-10-29 Thread Jonathan Rochkind
There's no way I know of to make Solr use that kind of data to create the sort order you want. Generally for 'custom' sorts, you want to create a field in your Solr index with possibly artificially constructed values that will 'naturally' sort the way you want. How to do that with a linked l

Custom Sorting in Solr

2010-10-29 Thread Ezequiel Calderara
Hi all guys! I'm in a weird situation here. We have index a set of documents which are ordered using a linked list (each documents has the reference of the previous and the next). Is there a way when sorting in the solr search, Use the linked list to sort? If that is not possible, how can i use

Re: documentCache clarification

2010-10-29 Thread Yonik Seeley
On Fri, Oct 29, 2010 at 2:31 PM, Jay Luker wrote: > This makes sense but still doesn't explain what I'm seeing in my cache > stats. When I issue a request with rows=10 the stats show an insert > into the queryResultCache. If I send the same query, this time with > rows=1000, I would not expect to

Re: documentCache clarification

2010-10-29 Thread Jay Luker
On Thu, Oct 28, 2010 at 7:27 PM, Chris Hostetter wrote: > The queryResultCache is keyed on and the > value is a "DocList" object ... > > http://lucene.apache.org/solr/api/org/apache/solr/search/DocList.html > > Unlike the Document objects in the documentCache, the DocLists in the > queryResultCa

How can I disable fsync()?

2010-10-29 Thread Igor Chudov
Thanks to all and I made Solr work very well on one newer machine. Now I am setting up Solr on an older server with an IDE hard drive. Unfortunately, populating the index takes FOREVER due to Solr/Lucene/Tomcat calling fsync() a lot after every write. I would like to know how to disable fsync.

Re: Stored or indexed?

2010-10-29 Thread Elizabeth L. Murnane
Hi Ron, In a nutshell - an indexed field is searchable, and a stored field has its content stored in the index so it is retrievable. Here are some examples that will hopefully give you a feel for how to set the indexed and stored options: indexed="true" stored="true" Use this for information yo

Re: Multiple indexes inside a single core

2010-10-29 Thread Valli Indraganti
Here's the Jira issue for the distributed search issue. https://issues.apache.org/jira/browse/SOLR-1632 I tried applying this patch but, get the same error that is posted in the discussion section for that issue. I will be glad to help too on this one. On Sat, Oct 23, 2010 at 2:35 PM, Erick Erick

Re: Exception while processing: attach document

2010-10-29 Thread Tommaso Teofili
I think this is a JDBC warning message since some isolation levels may not be implemented in the actual (Oracle) implementation (e.g.: READ_UNCOMMITTED). May your issue be related to some transactions updating/inserting/deleting records on your Oracle DB while trying to run DIH? Regards, Tommaso 2

Something for the weekend - Lily 0.2 is OUT ! :)

2010-10-29 Thread Steven Noels
Dear all, three months after the highly anticipated proof of architecture release, we're living up to our promises, and are releasing Lily 'CR' 0.2 today - a fully-distributed, highly scalable and highly available content repository, marrying best-of-breed database and search technology into a pow

Re: RAM increase

2010-10-29 Thread Tommaso Teofili
Hello Lance, form the command line run: > export JAVA_OPTS='-d64 -Xms128m -Xmx5g' eventually changing values of Xms and Xmx. Hope this helps. Tommaso 2010/10/29 Lance Norskog > When you start the Tomcat app, you tell it how much memory to allocate > to the JVM. I don't remember where, probably

RE: spellchecker results not as desired

2010-10-29 Thread Dyer, James
You should be building your index on a field that creates tokens on whitespace. So your dictionary would have "iphone" and "case" as separate terms instead of "iphone case" as one term. And if you query on something like "iphole case", it will give suggestions for "iphole" but not for "case" b

RE: Natural string sorting

2010-10-29 Thread Bob Sandiford
Well, you could do a magnitude notation approach. Depends on how complex the strings are, but based on your examples, this would work: 1) Identify a series of integers in the string. (This assumes lengths are no more than 9 for each series). 2) Insert the number of integers into the string be

Re: QueryElevation Component is so slow

2010-10-29 Thread Chamnap Chhorn
Thanks for reply. I'm looking for how to improve the speed of the search query. The QueryElevation Component is taking too much time which is unacceptable. The size of elevation file is only 1 Mb. I wonder other people using this component without problems (related to speed)? Am I using it the wro

Re: QueryElevation Component is so slow

2010-10-29 Thread Chamnap Chhorn
Thanks for reply. I'm looking for how to improve the speed of the search query. The QueryElevation Component is taking too much time which is unacceptable. The size of elevation file is only 1 Mb. I wonder other people using this component without problems (related to speed)? Am I using it the wro

Re: Maximum of length of a Dismax Query?

2010-10-29 Thread Swapnonil Mukherjee
Solved this issue, by setting the maxHttpHeaderSize to 65536 in tomcat/conf/server.xml file. Otherwise Tomcat was not responding. Swapnonil Mukherjee On 29-Oct-2010, at 2:43 PM, Swapnonil Mukherjee wrote: I am using the SOLRJ client to post my query, The query length is roughly 10,000 char

Re: eDismax result differs from Dismax

2010-10-29 Thread Yonik Seeley
On Fri, Oct 29, 2010 at 9:30 AM, Ryan Walker wrote: > > We are launching a new version of our job board helping returning veterans > find a civilian job, and we chose Solr and Sunspot[1] to power our search. We > really didn't consider the power users in the HR world who are trained to use > bo

eDismax result differs from Dismax

2010-10-29 Thread Ryan Walker
We are launching a new version of our job board helping returning veterans find a civilian job, and we chose Solr and Sunspot[1] to power our search. We really didn't consider the power users in the HR world who are trained to use boolean search, for example: "Engineer" AND ("Electrical" OR "M

Re: Reverse range query

2010-10-29 Thread kenf_nc
I modified the text of this hopefully to make it clearer. I wasn't sure what I was asking was coming across well. And I'm adding this comment in a shameless attempt to boost my question back to the top for people to see. Before I write a messy work around, just wanted to check the community to see

RE: Influencing scores on values in multiValue fields

2010-10-29 Thread Michael Sokolov
How about creating another field for doing exact matches (a string); searching both and boosting the string match? -Mike > -Original Message- > From: Imran [mailto:imranboho...@gmail.com] > Sent: Friday, October 29, 2010 6:25 AM > To: solr-user@lucene.apache.org > Subject: Influencing s

Re: Exception while processing: attach document

2010-10-29 Thread Bac Hoang
Could any one shed a light please? I saw in the log a message as below, but I don't think it's the root cause, because my dataSrouce, the readOnly is true Caused by: java.sql.SQLException: READ_COMMITTED and SERIALIZABLE are the only valid transaction levels A newbie Solr user ===

Influencing scores on values in multiValue fields

2010-10-29 Thread Imran
Hi All We've got an index in which we have a multiValued field per document. Assume the multivalue field values in each document to be; Doc1: bar lifters Doc2: truck tires back drops bar lifters Doc 3: iron bar lifters Doc 4: brass bar lifters iron bar lifters tire something truck something o

Re: QueryElevation Component is so slow

2010-10-29 Thread Lance Norskog
I do not know if this is accurate. There are direct tools to monitor these problems: jconsole, visualgc/visualvm, YourKit, etc. Often these counts allot many things to one place that should be spread out. On Fri, Oct 29, 2010 at 12:27 AM, Chamnap Chhorn wrote: > anyone has some suggestions to imp

Re: RAM increase

2010-10-29 Thread Lance Norskog
When you start the Tomcat app, you tell it how much memory to allocate to the JVM. I don't remember where, probably in catalina.sh. On Fri, Oct 29, 2010 at 2:56 AM, satya swaroop wrote: > Hi All, > >         Thanks for your reply.I have a doubt whether to increase the ram or > heap size to java o

Re: No response from Solr on complex request after several days

2010-10-29 Thread Xavier Schepler
On 29/10/2010 12:08, Lance Norskog wrote: There are a few problems that can happen. This is usually a sign of garbage collection problems. You can monitor the Tomcat instance with JConsole or one of the other java monitoring tools and see if there is a memory leak. Also, most people don't need t

Re: Looking for Developers

2010-10-29 Thread Lance Norskog
Then, Godwin! On Fri, Oct 29, 2010 at 3:04 AM, Toke Eskildsen wrote: > On Fri, 2010-10-29 at 10:06 +0200, Mark Allan wrote: >> For me, I simply deleted the original email, but I'm now quite >> enjoying the irony of the complaints causing more noise on the list >> than the original email!  ;-) >

Re: Sorting and filtering on fluctuating multi-currency price data?

2010-10-29 Thread Lance Norskog
ExternalFileField can only be used for boosting. It is not a "first-class" field. On Thu, Oct 28, 2010 at 11:07 AM, Chris Hostetter wrote: > > : Another approach would be to use ExternalFileField and keep the price data, > : normalized to USD, outside of the index. Every time the currency rates >

Re: No response from Solr on complex request after several days

2010-10-29 Thread Lance Norskog
There are a few problems that can happen. This is usually a sign of garbage collection problems. You can monitor the Tomcat instance with JConsole or one of the other java monitoring tools and see if there is a memory leak. Also, most people don't need to do it, but you can automatically restart i

Re: Upgrading from Solr 1.2 to 1.4.1

2010-10-29 Thread Lance Norskog
Yes, from Solr 1.2 to 1.3/Lucene 2.4.1 to 2.9 there was a change in the Porter stemmer for English. I don't know what it was. It may also affect the other language variants of the stemmer. If stemming is important for your users, you might want to try the Solr 3.x branch instead, or find Lucid's K

Re: Looking for Developers

2010-10-29 Thread Toke Eskildsen
On Fri, 2010-10-29 at 10:06 +0200, Mark Allan wrote: > For me, I simply deleted the original email, but I'm now quite > enjoying the irony of the complaints causing more noise on the list > than the original email! ;-) He he. An old classic. Next in line is the meta-meta-discussion about whet

Re: RAM increase

2010-10-29 Thread satya swaroop
Hi All, Thanks for your reply.I have a doubt whether to increase the ram or heap size to java or to tomcat where the solr is running Regards, satya

Re: Overriding Tika's field processing

2010-10-29 Thread Lance Norskog
If you change 'title' to be single-valued, the Extracting thing may or may not override it. I remember a go-round on this problem. But the ExtractingWhatsIt has code that explicitly checks for single-valued v.s. multi-valued. And this may all be different in different Solr versions. The DataImport

Re: Natural string sorting

2010-10-29 Thread Toke Eskildsen
On Fri, 2010-10-29 at 10:18 +0200, RL wrote: > Executing a query and sorting by this field leads to unnatural sorting of : > string1 > string10 > string2 That's very much natural. Numbers are not treated any different from words made up of letters. Your have to use alignment if you want to use nat

Re: Maximum of length of a Dismax Query?

2010-10-29 Thread Swapnonil Mukherjee
I am using the SOLRJ client to post my query, The query length is roughly 10,000 characters. I am using GET like this. int page = 1; int resultsPerPage = 24; ModifiableSolrParams params = new ModifiableSolrParams(); params.set("q", query); params.set("start

Re: OutOfMemory and auto-commit

2010-10-29 Thread Tommaso Teofili
If the problem is autowarming queries running in the meantime maybe you could consider changing set to true the following: false and/or change this value 2 another option would be lowering the value of autowarmCount inside the cache definitions. Hope this helps. Tommaso 2010/10/25 Jona

Re: Searching for terms on specific fields

2010-10-29 Thread Imran
Cheers Hoss. That did it for me. ~~Sent by an Android On 29 Oct 2010 00:39, "Chris Hostetter" wrote: > > The specifics of your overall goal confuse me a bit, but drilling down to > your core question... > > : I want to be able to use the dismax parser to search on both terms > : (assigning slops

Re: Possible bug in query sorting

2010-10-29 Thread Gora Mohanty
On Fri, Oct 29, 2010 at 1:47 PM, Pablo Recio wrote: > That's my schema XML: >   >     >       >       >       >       >     >     >       >       ignoreCase="true" expand="true"/> >       >       >       >     >   >   > >   [...] >   required="true" multiValued="false" omitNorms="f

Re: Natural string sorting

2010-10-29 Thread Savvas-Andreas Moysidis
I think string10 is before string2 in lexicographic order? On 29 October 2010 09:18, RL wrote: > > Just a quick question about natural sorting of strings. > > I've a simple dynamic field in my schema: > > omitNorms="true"/> > omitNorms="true"/> > > There are 3 indexed strings for example > str

org.tartarus package in lucene/solr?

2010-10-29 Thread Tharindu Mathew
Hi, How come $subject is present?? -- Regards, Tharindu

Natural string sorting

2010-10-29 Thread RL
Just a quick question about natural sorting of strings. I've a simple dynamic field in my schema: There are 3 indexed strings for example string1,string2,string10 Executing a query and sorting by this field leads to unnatural sorting of : string1 string10 string2 (Some time ago i used Lucen

Re: Possible bug in query sorting

2010-10-29 Thread Pablo Recio
That's my schema XML: link text ... 2010/10/28 Gora Mohanty > On Thu, Oct 28, 2010 at 5:18 PM, Michael McCandless > wrote: > >

Re: Looking for Developers

2010-10-29 Thread Mark Allan
For me, I simply deleted the original email, but I'm now quite enjoying the irony of the complaints causing more noise on the list than the original email! ;-) M -- The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336.

Newbie to Solr, LIKE:foo

2010-10-29 Thread MilleBii
I'm Nutch user but I'm considering to use Solr for the following reason. I need a LIKE:foo , which turns into a *foo* query. I saw the built-in prefix query parser but it does only look for foo*, if I understand it well So is there a query parser that does what I'm looking. If not how difficult is

Re: QueryElevation Component is so slow

2010-10-29 Thread Chamnap Chhorn
anyone has some suggestions to improve the search? thanks On 10/28/10, Chamnap Chhorn wrote: > Sorry for very bad pasting. I paste it again. > > Slowest Components Count Exclusive > Total > QueryElevationComponent 1 5

Maximum of length of a Dismax Query?

2010-10-29 Thread Swapnonil Mukherjee
Hi Everybody, It seems that the maximum query length supported by the Dismax Query Handler is 3534 characters. Is there anyway I can set this limit to be around 12,000? If I fire a query beyond 3534 characters, I don't even get error messages in the catalina.XXX log files. Swapnonil Mukherjee

Re: Looking for Developers

2010-10-29 Thread Gora Mohanty
On Fri, Oct 29, 2010 at 12:23 PM, scott chu (朱炎詹) wrote: > When I first saw this particular email, I wrote a letter intend to ask the > sender remove solr-user from its recepient cause I thought this should go to > solr-dev. But then I thought again, it's about 'job-offer' not 'development > of So