Re: Date Faceting on Solr 3.1

2011-09-06 Thread Jan Høydahl
Hi, Note that if you want more control over the buckets, you may use facet.query instead. Also, under development is SOLR-2366 which will eventually give a more powerful gap specification to range facets. -- Jan Høydahl, search solution architect Cominvent AS - www.cominvent.com Solr Training -

Re: Date Faceting on Solr 3.1

2011-09-06 Thread karthik
Thanks Hoss. Whatever you described does make sense, however we are migrating over from FAST and the date range buckets work differently there. The expectations of the business users are based on the existing system. I need to reset their expectations ;-) ... Thanks for the very detailed explana

Re: Date Faceting on Solr 3.1

2011-09-04 Thread Chris Hostetter
: Can solr take the earliest date from the result set to be the value for : "facet.date.start"? I dont want to have the value 1/1/1995 hardcoded in my : application since a new data feed that gets into my index could be older : than 1995 and i might keep missing them from the facet. no, you have

Re: Date faceting per last hour, three days and last week

2011-08-10 Thread O. Klein
I would use facet queries: facet.query=date:[NOW-1DAY TO NOW] facet.query=date:[NOW-3DAY TO NOW] facet.query=date:[NOW-7DAY TO NOW] -- View this message in context: http://lucene.472066.n3.nabble.com/Date-faceting-per-last-hour-three-days-and-last-week-tp3242364p3242574.html Sent from the Solr -

Re: Date faceting +1MONTH problem

2010-10-22 Thread Yonik Seeley
On Fri, Oct 22, 2010 at 6:02 PM, Shawn Heisey wrote: > On 10/22/2010 3:01 PM, Yonik Seeley wrote: >> >> On Fri, Sep 17, 2010 at 9:51 PM, Chris Hostetter >>  wrote: >>> >>>  the default query parser >>> doesn't support range queries with mixed upper/lower bound inclusion. >> >> This has just been

Re: Date faceting +1MONTH problem

2010-10-22 Thread Shawn Heisey
On 10/22/2010 3:01 PM, Yonik Seeley wrote: On Fri, Sep 17, 2010 at 9:51 PM, Chris Hostetter wrote: the default query parser doesn't support range queries with mixed upper/lower bound inclusion. This has just been added to trunk. Things like [0 TO 100} now work. Awesome! Is it easily port

Re: Date faceting +1MONTH problem

2010-10-22 Thread Yonik Seeley
On Fri, Sep 17, 2010 at 9:51 PM, Chris Hostetter wrote: > the default query parser > doesn't support range queries with mixed upper/lower bound inclusion. This has just been added to trunk. Things like [0 TO 100} now work. -Yonik http://www.lucidimagination.com

Re: Date faceting +1MONTH problem

2010-09-17 Thread Chris Hostetter
: Reindexing with a +1MILLI hack had occurred to me and I guess that's what : I'll do in the meantime; it just seemed like something that people must have : run into before! I suppose it depends on the granularity of your people have definitely run into it before, and most of them (that i know

Re: Date faceting +1MONTH problem

2010-09-10 Thread Dennis Gearon
Crowded' Laugh at http://www.yert.com/film.php --- On Thu, 9/9/10, Liam O'Boyle wrote: > From: Liam O'Boyle > Subject: Re: Date faceting +1MONTH problem > To: solr-user@lucene.apache.org > Date: Thursday, September 9, 2010, 4:20 PM > Hi Chris, > > Yes

Re: Date faceting +1MONTH problem

2010-09-10 Thread Jan Høydahl / Cominvent
Just attended a talk at JavaZone (www.javazone.no) by Stephen Colebourne about JSR-310 which will make these kind of operations easier in future JDK, and how Joda-Time goes a great way of enabling it today. I'm not saying it would fix your GAP issue, as it's all about what definition of "month"

Re: Date faceting +1MONTH problem

2010-09-09 Thread Liam O'Boyle
Hi Chris, Yes, I saw the facet.range.include feature and briefly tried to implement it before realising that it was Solr 3.1 only :) I agree that it seems like the best solution to problem. Reindexing with a +1MILLI hack had occurred to me and I guess that's what I'll do in the meantime; it just

Re: Date faceting +1MONTH problem

2010-09-09 Thread Chris Hostetter
: I'm trying to break down the data over a year into facets by month; to avoid : overlap, I'm using -1MILLI on the start and end dates and using a gap of : +1MONTH. : : However, it seems like February completely breaks my monthly cycles, leading Yep. Everything you posted makes sense to me in ho

Re: Date faceting

2010-08-04 Thread Eric Grobler
Thanks Koji, It works :-) Have a nice day. regards ericz On Wed, Aug 4, 2010 at 12:08 PM, Koji Sekiguchi wrote: > (10/08/04 19:42), Eric Grobler wrote: > >> Hi Solr community, >> >> How do I facet on timestamp for example? >> >> I tried something like this - but I get no result. >> >> facet=t

Re: Date faceting

2010-08-04 Thread Koji Sekiguchi
(10/08/04 19:42), Eric Grobler wrote: Hi Solr community, How do I facet on timestamp for example? I tried something like this - but I get no result. facet=true facet.date=timestamp f.facet.timestamp.date.start=2010-01-01T00:00:00Z f.facet.timestamp.date.end=2010-12-31T00:00:00Z f.facet.timesta

RE: Date faceting and memory leaks

2010-05-17 Thread Yao
From: Chris Hostetter-3 [via Lucene] [mailto:ml-node+825052-1711725506-201...@n3.nabble.com] Sent: Monday, May 17, 2010 9:04 PM To: Ge, Yao (Y.) Subject: Re: Date faceting and memory leaks : Cache settings: : that's a monster filterCache ...i can easly imagine it causing an OOM if

Re: Date faceting and memory leaks

2010-05-17 Thread Yao
Chris, Just completed the re-run and your date rounding tip saved my day. I now realized the "NOW" as a timestamp is a very bad idea for query caching as it is never the same in value. NOW/DAY would at least makes a set facet queries caches re-usable for a period of time. It turns on you can help

Re: Date faceting and memory leaks

2010-05-17 Thread Chris Hostetter
: Cache settings: : that's a monster filterCache ...i can easly imagine it causing an OOM if your heap is only 5G. : The date rounding suggest is a very good one, I will need to rerun the test : and report back on the cache setting. I remember my filterCache hit ratio is : around 0.7. I did u

Re: Date faceting and memory leaks

2010-05-17 Thread Yao
Chris, Thanks for the detailed response. No I am not using Date Facet but Facet Query as for facet display. Here is the full configuration of my "dismax" query handler: dismax explicit 0.01 title text^0.5 domain^0.1 nature^0.1 author title

Re: Date faceting and memory leaks

2010-05-17 Thread Chris Hostetter
: Subject: Date faceting and memory leaks First off, just to be clear, you don't seem to be useing the "date faceting" feature, you are using the "Facet Query" feature, your queries just so happen to be on a date field. Second: to help people help you, you need to provide all the details. you

Re: Date faceting and memory leaks

2010-05-17 Thread Yao
No I still have the OOM issue with repeated facet query request on the date field. I forgot to mention that I am running 64-bit IBM 1.5 JVM. I also tried the Sun 1.6 JVM with and without your GC arguments. The GC pattern is different but the heap size does not drop as the test going on. I tested w

Re: Date faceting and memory leaks

2010-05-17 Thread Antonio Lobato
2010 2:44 PM > To: solr-user@lucene.apache.org > Subject: Re: Date faceting and memory leaks > > What garbage collection settings are you running at the command line > when starting Solr? > On May 17, 2010, at 2:41 PM, Yao wrote: > >> >> I have been running load testing using JM

RE: Date faceting and memory leaks

2010-05-17 Thread Ge, Yao (Y.)
44 PM To: solr-user@lucene.apache.org Subject: Re: Date faceting and memory leaks What garbage collection settings are you running at the command line when starting Solr? On May 17, 2010, at 2:41 PM, Yao wrote: > > I have been running load testing using JMeter on a Solr 1.4 index with ~4 >

Re: Date faceting and memory leaks

2010-05-17 Thread Antonio Lobato
What garbage collection settings are you running at the command line when starting Solr? On May 17, 2010, at 2:41 PM, Yao wrote: > > I have been running load testing using JMeter on a Solr 1.4 index with ~4 > million docs. I notice a steady JVM heap size increase as I iterator 100 > query terms

Re: Date Faceting and Double Counting

2009-09-11 Thread Chris Hostetter
: What are wild-card range searches? i'm pretty sure we was just refering to open ended range searchers, like the example he asked about... : > What does this mean? : > : >      {* TO *} : : Same thing as [* TO *] - not worth trying to make it different IMO. ...right, that's something the Sol

Re: Date Faceting and Double Counting

2009-09-11 Thread Yonik Seeley
On Fri, Sep 11, 2009 at 3:59 PM, Lance Norskog wrote: > Thanks! I had to find this in the Lucene query parser syntax- it is > not mentioned anywhere in the Solr wiki. You are right [a TO z} and {a > TO z] are obvious improvements and solve the bucket-search problem the > right way. But this collid

Re: Date Faceting and Double Counting

2009-09-11 Thread Lance Norskog
Thanks! I had to find this in the Lucene query parser syntax- it is not mentioned anywhere in the Solr wiki. You are right [a TO z} and {a TO z] are obvious improvements and solve the bucket-search problem the right way. But this collides with wild-card range searches. What does this mean?

Re: Date Faceting and Double Counting

2009-09-11 Thread Chris Hostetter
: datefield:[X TO* Y] for X to Y-0....1 : : This would be backwards-compatible. {} are used for other things and lexing You lost me there ... {} aren't used for "other things" in the query parser -- they're used for range queries that are exclusive of their end points. datefield:{X TO

Re: Date Faceting and Double Counting

2009-09-10 Thread Lance Norskog
datefield:[X TO* Y] for X to Y-0....1 This would be backwards-compatible. {} are used for other things and lexing is a dying art. Using a * causes mistakes to trigger wildcard syntaxes, which will fail loudly. On Tue, Sep 8, 2009 at 5:20 PM, Chris Hostetter wrote: > > : I ran into that p

Re: Date Faceting and Double Counting

2009-09-08 Thread Chris Hostetter
: I ran into that problem as well but the solution was provided to me by : this very list :) See : http://www.nabble.com/Range-queries-td24057317.html It's not the : cleanest solution, but as long as you know what you're doing it's not : that bad. Hmmm... yeah, that's a total hack. one of these

Re: Date Faceting and Double Counting

2009-09-02 Thread gwk
Chris Hostetter wrote: : When I added numerical faceting to my checkout of solr (solr-1240) I basically : copied date faceting and modified it to work with numbers instead of dates. : With numbers I got a lot of doulbe-counted values as well. So to fix my : problem I added an extra parameter to n

Re: Date Faceting and Double Counting

2009-09-01 Thread Chris Hostetter
: When I added numerical faceting to my checkout of solr (solr-1240) I basically : copied date faceting and modified it to work with numbers instead of dates. : With numbers I got a lot of doulbe-counted values as well. So to fix my : problem I added an extra parameter to number faceting where you

Re: Date Faceting and Double Counting

2009-09-01 Thread Chris Hostetter
: Is this a known behavior people are happy with, or should I file an issue : asking for ranges in date-facets to be constructed to subtract one second : from the end of each range (so that the effective range queries for my case It's a known anoyance, but not something that seems to anoy people e

Re: Date Faceting and Double Counting

2009-09-01 Thread gwk
Hi Stephen, When I added numerical faceting to my checkout of solr (solr-1240) I basically copied date faceting and modified it to work with numbers instead of dates. With numbers I got a lot of doulbe-counted values as well. So to fix my problem I added an extra parameter to number faceting

Re: Date Faceting and Double Counting

2009-08-31 Thread Avlesh Singh
I don't think this behavior needs to be fixed. It is justified for the data you have indexed. "date minus 1 second" should definitely work for you. Cheers Avlesh On Mon, Aug 31, 2009 at 11:37 PM, Stephen Duncan Jr < stephen.dun...@gmail.com> wrote: > If we do date faceting and start at 2009-01-0

Re: Date faceting - howto improve performance

2009-04-30 Thread Marcus Herou
Thanks should have grep'ed the source of course (like I always seem to end up with doing haha) /M On Wed, Apr 29, 2009 at 10:13 PM, Shalin Shekhar Mangar < shalinman...@gmail.com> wrote: > Some basic documentation is in the example schema.xml. Ask away if you have > specific questions. > > On T

Re: Date faceting - howto improve performance

2009-04-29 Thread Shalin Shekhar Mangar
Some basic documentation is in the example schema.xml. Ask away if you have specific questions. On Thu, Apr 30, 2009 at 1:00 AM, Marcus Herou wrote: > Aha! > > Hmm , googling wont help me I see. any hints of usages ? > > /M > > > On Tue, Apr 28, 2009 at 12:29 AM, Shalin Shekhar Mangar < > shalinm

Re: Date faceting - howto improve performance

2009-04-29 Thread Marcus Herou
Aha! Hmm , googling wont help me I see. any hints of usages ? /M On Tue, Apr 28, 2009 at 12:29 AM, Shalin Shekhar Mangar < shalinman...@gmail.com> wrote: > Sorry, I'm late in this thread. > > Did you try using Trie fields (new in 1.4)? The regular date faceting won't > work out-of-the-box for

Re: Date faceting - howto improve performance

2009-04-27 Thread Shalin Shekhar Mangar
Sorry, I'm late in this thread. Did you try using Trie fields (new in 1.4)? The regular date faceting won't work out-of-the-box for trie fields I think. But you could use facet.query to achieve the same effect. On my simple benchmarks I've found trie fields to give a huge improvement in range sear

Re: Date faceting - howto improve performance

2009-04-27 Thread Marcus Herou
pr 25, 2009 at 3:46 PM, Otis Gospodnetic < > >> otis_gospodne...@yahoo.com> wrote: > >> > >>> > >>> I should emphasize that the PR trick I mentioned is something you'd do > at > >>> the Lucene level, outside Solr, and then you'd

Re: Date faceting - howto improve performance

2009-04-27 Thread Ning Li
'd just slip the modified index >>> back into Solr. >>> Of, if you like the bleeding edge, perhaps you can make use of Ning Li's >>> Solr index merging functionality (patch in JIRA). >>> >>> >>> Otis -- >>> Sematext -- http://s

Re: Date faceting - howto improve performance

2009-04-25 Thread Marcus Herou
Of, if you like the bleeding edge, perhaps you can make use of Ning Li's >> Solr index merging functionality (patch in JIRA). >> >> >> Otis -- >> Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch >> >> >> >> - Original Message

Re: Date faceting - howto improve performance

2009-04-25 Thread Marcus Herou
into Solr. >> Of, if you like the bleeding edge, perhaps you can make use of Ning Li's >> Solr index merging functionality (patch in JIRA). >> >> >> Otis -- >> Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch >> >> >> >> -

Re: Date faceting - howto improve performance

2009-04-25 Thread Marcus Herou
; > Otis -- > Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch > > > > - Original Message > > From: Otis Gospodnetic > > To: solr-user@lucene.apache.org > > Sent: Saturday, April 25, 2009 9:41:45 AM > > Subject: Re: Date faceting - howto impro

Re: Date faceting - howto improve performance

2009-04-25 Thread Otis Gospodnetic
(patch in JIRA). Otis -- Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch - Original Message > From: Otis Gospodnetic > To: solr-user@lucene.apache.org > Sent: Saturday, April 25, 2009 9:41:45 AM > Subject: Re: Date faceting - howto improve performance > > >

Re: Date faceting - howto improve performance

2009-04-25 Thread Otis Gospodnetic
Yes, you could simply round the date, no need for a non-date type field. Yes, you can add a field after the fact by making use of ParallelReader and merging (I don't recall the details, search the ML for ParallelReader and Andrzej), I remember he once provided the working recipe. Otis -- Semat

RE: Date faceting - howto improve performance

2009-04-25 Thread Smiley, David W.
Hi Marcus. You must supply dates in the format that you are doing now -- ISO-8601 with the Z to indicate there is no time-zone offset occurring. To reduce cardinality to the day level instead of to the second that you are currently performing, the date you supply can include DateMathParser ope

Re: Date faceting problem

2007-09-04 Thread Chris Hostetter
: I'm having an issue w/ the date faceter, running solr 1.2.0 on Tomcat Welcom to the list Delsey, Date Faceting was only recently added to the Solr trunk ... there is a wiki convention of noting when docs refer to "bleeding edge" stuff not in the latest release by noting hte version it's expe