Re: Requests per second/minute monitor?

2007-05-09 Thread Ian Holsman
Walter Underwood wrote: > This is for monitoring -- what happened in the last 30 seconds. > Log file analysis doesn't really do that. > I would respectfully disagree. Log file analysis of each request can give you that, and a whole lot more. you could either grab the stats via a regular cron j

Question about delete

2007-05-09 Thread James liu
i use command like this curl http://localhost:8983/solr/update --data-binary 'name:DDR' curl http://localhost:8983/solr/update --data-binary '' and i get numDocs : 0 maxDoc : 1218819 when i search something which exists in before delete and find nothing. but index file size not changed

Re: Index Concurrency

2007-05-09 Thread Yonik Seeley
On 5/9/07, joestelmach <[EMAIL PROTECTED]> wrote: Does solr provide any additional concurrency control over what Lucene provides? Yes, coordination between the main index searcher, the index writer, and the index reader needed to delete other documents. In my simple testing of indexing 2,000

Re: Index Concurrency

2007-05-09 Thread joestelmach
Yonik, Thanks for your fast reply. > No, not currently. Start your implementation with just a single > index... unless it is very large, it will likely be fast enough. My index will get quite large > Solr also handles all the concurrency issues, and you should never hit > "lock access timeou

Re: Solr Sorting, merging/weighting sort fields

2007-05-09 Thread Nick Jenkin
Thanks, worked perfectly! -Nick On 5/10/07, Walter Underwood <[EMAIL PROTECTED]> wrote: No problem. Use a boost function. In a DisMaxRequestHandler spec in solrconfig.xml, specify this: popularity^0.5 This value will be added to the score before ranking. You will probably need to f

Re: Ideas for a relevance score that could be considered stable across multiple searches with the same query structure?

2007-05-09 Thread Sean Timm
Yes, for good (hopefully) or bad. -Sean Shridhar Venkatraman wrote on 5/7/2007, 12:37 AM: Interesting.. Surrogates can also bring the searcher's subjectivity (opinion and context) into it by the learning process ? shridhar Sean Timm wrote: It may not be easy or even possible withou

Re: facet.sort does not work in python output

2007-05-09 Thread Yonik Seeley
On 5/3/07, Mike Klaas <[EMAIL PROTECTED]> wrote: On 5/3/07, Jack L <[EMAIL PROTECTED]> wrote: > The Python output uses nested dictionaries for facet counts. This might be fixed in the future It's fixed in the current development version (future 1.2), already. See http://wiki.apache.org/solr/So

Re: Solr Sorting, merging/weighting sort fields

2007-05-09 Thread Walter Underwood
No problem. Use a boost function. In a DisMaxRequestHandler spec in solrconfig.xml, specify this: popularity^0.5 This value will be added to the score before ranking. You will probably need to fuss with the multiplier to get the popularity to the right proportion of the total score. I

Solr Sorting, merging/weighting sort fields

2007-05-09 Thread Nick Jenkin
Hi all, I have a popularity field in my solr index, this field is a popularity rating of a particular product (based on the number of product views etc). I want to be able to integrate this number into the search result sorting such that a product with a higher popularity rating is ranking highe

Re: Facet only support english?

2007-05-09 Thread Walter Underwood
I didn't remember that requirement, so I looked it up. It was added in XML 1.0 2nd edition. Originally, unspecified encodings were open for auto-detection. Content type trumps encoding declarations, of course, per RFC 3023 and allowed by the XML spec. wunder On 5/9/07 4:19 PM, "Mike Klaas" <[EMA

Re: Facet only support english?

2007-05-09 Thread Mike Klaas
On 5/9/07, Yonik Seeley <[EMAIL PROTECTED]> wrote: > +1 on explicit encoding declarations. Done (even though it really wasn't needed since there were no int'l chars in the example). As Mike points out, it only marginally helps... if the user adds international chars to the config and saves it

Re: Question about word treatment...

2007-05-09 Thread Yonik Seeley
On 5/4/07, escher2k <[EMAIL PROTECTED]> wrote: (2) Is it possible to force Lucene to treat a multiword (e.g. Ruby on Rails) as one word ? I am not sure if there is a mechanism to do this by creating a special text file (like the one that exists for synonyms for instance) ? Solr's SynonymFilter

Re: Facet only support english?

2007-05-09 Thread Yonik Seeley
+1 on explicit encoding declarations. Done (even though it really wasn't needed since there were no int'l chars in the example). As Mike points out, it only marginally helps... if the user adds international chars to the config and saves it as something other than UTF-8 they are still hosed.

Re: Facet only support english?

2007-05-09 Thread Koji Sekiguchi
+1 on explicit encoding declarations. Yonik Seeley wrote: On 5/9/07, Yonik Seeley <[EMAIL PROTECTED]> wrote: If you are saving the file in UTF-8 format, then try changing the first line to be this: We should probably change the example solrconfig.xml and schema.xml to be UTF-8 by default. A

Re: Facet only support english?

2007-05-09 Thread Yonik Seeley
On 5/9/07, Ryan McKinley <[EMAIL PROTECTED]> wrote: Yonik Seeley wrote: > We should probably change the example solrconfig.xml and schema.xml to > be UTF-8 by default. Any objections? > I'm for it... but if the xml parser uses getReader() does it make any difference? For Solr's XML config fi

Re: Facet only support english?

2007-05-09 Thread Ryan McKinley
Yonik Seeley wrote: On 5/9/07, Yonik Seeley <[EMAIL PROTECTED]> wrote: If you are saving the file in UTF-8 format, then try changing the first line to be this: We should probably change the example solrconfig.xml and schema.xml to be UTF-8 by default. Any objections? I'm for it... but if

Re: Index Concurrency

2007-05-09 Thread Yonik Seeley
On 5/9/07, joestelmach <[EMAIL PROTECTED]> wrote: My first intuition is to give each user their own index. My thinking here is that querying would be faster (since each user's index would be much smaller than one big index,) and, more importantly, that I would dodge any concurrency issues stemmin

Re: Facet only support english?

2007-05-09 Thread Walter Underwood
I was about to suggest the same thing. +1 on explicit encoding declarations. wunder On 5/9/07 3:26 PM, "Yonik Seeley" <[EMAIL PROTECTED]> wrote: > On 5/9/07, Yonik Seeley <[EMAIL PROTECTED]> wrote: >> If you are saving the file in UTF-8 format, then try changing the >> first line to be this: >>

Re: Facet only support english?

2007-05-09 Thread Mike Klaas
On 5/9/07, Yonik Seeley <[EMAIL PROTECTED]> wrote: On 5/9/07, Yonik Seeley <[EMAIL PROTECTED]> wrote: > If you are saving the file in UTF-8 format, then try changing the > first line to be this: > We should probably change the example solrconfig.xml and schema.xml to be UTF-8 by default. Any o

Re: Facet only support english?

2007-05-09 Thread Yonik Seeley
On 5/9/07, Yonik Seeley <[EMAIL PROTECTED]> wrote: If you are saving the file in UTF-8 format, then try changing the first line to be this: We should probably change the example solrconfig.xml and schema.xml to be UTF-8 by default. Any objections? -Yonik

Re: Facet only support english?

2007-05-09 Thread Yonik Seeley
On 5/5/07, James liu <[EMAIL PROTECTED]> wrote: Expect it to support other language like chinese. maybe solr facet can config like this when it support other language. title:"诺基亚" solrconfig.xml is a normal XML document. It currently starts off with which has no char encoding specified and

Index Concurrency

2007-05-09 Thread joestelmach
Hello, I'm a bit new to search indexing and I'm hoping some of you here can help me with an e-mail application I'm working on. I have a mail retrieval program that accesses multiple POP accounts in parallel, and parses each message into a database. I would like to add a new document to a solr i

Re: Look ahead queries

2007-05-09 Thread Yonik Seeley
You could perhaps use faceting to do this for single terms. Set the base query to whatever you want (or *:* for everything) Then use facet.field=text&facet.prefix=foo If you indexed field values as strings like "fuel consumption" (instead of breaking it up into tokens) then you could get your phr

Re: Dismax Config?

2007-05-09 Thread Chris Hostetter
: Example: : "a b"~1 will match fields with a b, a x b, or b a, but not b x a. : The last would require a slop of 2 also note that there are two "slop" params in the DisMax handler ... "qs" refers to how much "slop' will be used when querying the qf fields if the user actually types in a query st

Re: Dismax Config?

2007-05-09 Thread Yonik Seeley
On 5/9/07, Matthew Runo <[EMAIL PROTECTED]> wrote: Ah hah! After doing some research, slop is a fun term for how sloppy a match SOLR will make. Eg, slop = 0, means that only exact matches will work. Slop = 1 means that they can be off by one word... etc Yes? All terms must appear, but the pos

Re: date range search

2007-05-09 Thread Chris Hostetter
: does solr support date range searching? i've tried all the examples on : the lucene site as well as using the solr response format and a few : others that seemed nifty but so far I always get query parsing errors. : i know i can easily convert the dates to ints and do ranges that way but : all

Re: Dismax Config?

2007-05-09 Thread Matthew Runo
Ah hah! After doing some research, slop is a fun term for how sloppy a match SOLR will make. Eg, slop = 0, means that only exact matches will work. Slop = 1 means that they can be off by one word... etc Yes? ++ | Matthew Runo | Zapp

Re: Dismax Config?

2007-05-09 Thread Matthew Runo
Perfect! I had seen the wiki, but did not visit the class page since I am using Perl. What is "slop"? heh ++ | Matthew Runo | Zappos Development | [EMAIL PROTECTED] | 702-943-7833 +-

Re: Dismax Config?

2007-05-09 Thread Ryan McKinley
check: http://wiki.apache.org/solr/DisMaxRequestHandler For now, most of the docs for dismax are in the javadocs: http://lucene.apache.org/solr/api/org/apache/solr/request/DisMaxRequestHandler.html Matthew Runo wrote: I'd love to see some explanation of what's going on here, and how to confi

Dismax Config?

2007-05-09 Thread Matthew Runo
I'd love to see some explanation of what's going on here, and how to configure it for my own use. I've changed the fields to match my own columns, but it'd be great if I could actually understand it.. 243 class="solr.DisMaxRequestHandler" > 244 245 explicit 246

Re: Index corruptions?

2007-05-09 Thread Yonik Seeley
On 5/7/07, Tom Hill <[EMAIL PROTECTED]> wrote: Is the "cp-lr" in snapshot really guaranteed to be atomic? Or is it just fast, and unlikely to be interrupted? It's called from Solr within a synchronized context, and it's guaranteed that no index changes (via Solr at least) will happen concurrent

date range search

2007-05-09 Thread Will Johnson
does solr support date range searching? i've tried all the examples on the lucene site as well as using the solr response format and a few others that seemed nifty but so far I always get query parsing errors. i know i can easily convert the dates to ints and do ranges that way but all the documen

Re: Solr Update Handler Failes with Some Doc Characters

2007-05-09 Thread Yonik Seeley
On 5/9/07, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: 1000194 http://www(r)barharborinn(r)com Bar Harbor Hotels and Bar Harbor Inn near Acadia in Bar Harbor ME Bar Harbor Inn- premier oceanfront hotel in Bar Harbor ME(r) Rated Superior First Class by OHG, Best in-town location(r)Spe

Re: Solr Update Handler Failes with Some Doc Characters

2007-05-09 Thread Koji Sekiguchi
Can you use ® , instead of registered trade mark sign? On my environment, I can post both ® and reg sign successfully, though. Koji [EMAIL PROTECTED] wrote: Hi, I tried CDATA. It fails the same way. I will try to check of the utf8-example.xml works ok (I just have to change it to match my sch

Re: Solr Update Handler Failes with Some Doc Characters

2007-05-09 Thread [EMAIL PROTECTED]
Hi, I tried CDATA. It fails the same way. I will try to check of the utf8-example.xml works ok (I just have to change it to match my schema). I just ran a test by adding (R) symbolt into the XML to get the exact error message. See below. Thanks, Av *** SUBMITTED REQUEST *** (as captured by HT

Re: Solr Update Handler Failes with Some Doc Characters

2007-05-09 Thread Koji Sekiguchi
I'm not sure this is the case, but did you use CDATA section in your XML? Or try to use character reference to represent copyright symbol. I believe it is © or © . Hope this helps, Koji [EMAIL PROTECTED] wrote: > Hi, > > I run the example using Jetty on Windows 2003 machine. When I submit some

Re: Solr Update Handler Failes with Some Doc Characters

2007-05-09 Thread Brian Whitman
I see that the update handler fails even if the character is NOT right next to XML closing tag. If the character is anywhere in any of the XML tags, the update handler fails to parse the XML. Does posting the utf8-example in the exampledocs directory work?

Re: Solr Update Handler Failes with Some Doc Characters

2007-05-09 Thread [EMAIL PROTECTED]
Hi, I specify the following encoding when POSTING the data to Solr: text/xml; charset=utf-8 The encoding of the actual XML is also UTF-8. I see that the update handler fails even if the character is NOT right next to XML closing tag. If the character is anywhere in any of the XML tags, the up

Re: Solr Update Handler Failes with Some Doc Characters

2007-05-09 Thread Yonik Seeley
On 5/9/07, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: I run the example using Jetty on Windows 2003 machine. When I submit some documents containing upper ASCII characters, Solr update handler fails with an XML parsing error saying that it encountered an EOF before the closing tags. Normall

Solr Update Handler Failes with Some Doc Characters

2007-05-09 Thread [EMAIL PROTECTED]
Hi, I run the example using Jetty on Windows 2003 machine. When I submit some documents containing upper ASCII characters, Solr update handler fails with an XML parsing error saying that it encountered an EOF before the closing tags. The XML is perfectly correct and is using utf-8 encoding. It

Re: adjusting score slightly by date field

2007-05-09 Thread Yonik Seeley
On 5/9/07, mike topper <[EMAIL PROTECTED]> wrote: Is it possible for a date field to weigh in on the score slightly in some way? Or if not is there another way to push up newer records in the order of results while still maintaining the scoring? Still under construction, but see the relevancy

adjusting score slightly by date field

2007-05-09 Thread mike topper
Hello, In our application there are a lot of old records that we still want in our index but would like for them to be scored lower than some newer records. Is it possible for a date field to weigh in on the score slightly in some way? Or if not is there another way to push up newer records