Re: SpatialRecursivePrefixTreeFieldType Spatial Searching

2013-06-05 Thread Chris Atkinson
ight of the decimal, your maxDistErr should definitely be > >> >>>smaller > >> >>> than 1 -- use something like 0.5 (given you have two levels of > >> >>>precision > >> >>> below a full day) but to be safer (more certain it's not a problem) > >> use > >> >>> 0.3 -- a little less. Please report back how that goes. > >> >>> > >> >>> ~ David > >> >>> > >> >>> On 6/3/13 7:27 AM, "Chris Atkinson" < > > > chrisacky@ > > > > wrote: > >> >>> > >> >>> >Hi, > >> >>> >I'm seeing really slow query times. 7-25 seconds when I run a > simple > >> >>> >filter > >> >>> >query that uses my SpatialRecursivePrefixTreeFieldType field. > >> >>> > > >> >>> >My index is about 30k documents. Prior to adding the Spatial field, > >> >>>the > >> >>> on > >> >>> >disk space was about 100Mb, so it's a really tiny index. Once I add > >> >>>the > >> >>> >spatial field (which is multi-values), the index size jumps up to > >> 2GB. > >> >>> (Is > >> >>> >this normal?). > >> >>> > > >> >>> >Only about 10k documents will have any spatial data. Typically, > they > >> >>>will > >> >>> >have at most 10 shapes each, but the majority are all one of two > >> >>> >rectangles. > >> >>> > > >> >>> >This is my fieldType definition. > >> >>> > > >> >>> > > > >> > > >>> >class="solr.SpatialRecursivePrefixTreeFieldType" > >> >>> >geo="false" > >> >>> >worldBounds="0 0 3650 1" > >> >>> >distErrPct="0" > >> >>> >maxDistErr="1" > >> >>> >units="degrees" > >> >>> >/> > >> >>> > > >> >>> >And the field > >> >>> > > >> >>> > > > >> > > >>> > indexed="true" stored="false" multiValued="true" /> > >> >>> > > >> >>> > > >> >>> >I am using the field to represent approximately 10 years after > >> January > >> >>> 1st > >> >>> >2013, where each day is along the X-axis. Because the availability > >> >>>starts > >> >>> >and ends at 2pm and 10am, I was using a decimal place when creating > >> my > >> >>> >shape to show that detail. (Is this approach wrong?) > >> >>> > > >> >>> >So a typical rectangle when indexed would be (minX minY maxX maxY) > >> >>> > > >> >>> >Rectangle 100.6 0 120.4 1 > >> >>> > > >> >>> >Is it wrong that my Y and X values are not of the same scale? Since > >> I > >> >>> >don't > >> >>> >care about the Y axis at all, I just set it to be of 1 height > >> always. > >> >>> > > >> >>> >I'm running Solr 4.3, with a small JVM of 768M (can be increased). > >> >>>And I > >> >>> >have 2GB RAM. (Again can be increased). > >> >>> > > >> >>> >Thanks > >> >>> > >> >>> > >> >> > >> > >> > > > > > > - > Author: > http://www.packtpub.com/apache-solr-3-enterprise-search-server/book > -- > View this message in context: > http://lucene.472066.n3.nabble.com/SpatialRecursivePrefixTreeFieldType-Spatial-Searching-tp4067778p4068216.html > Sent from the Solr - User mailing list archive at Nabble.com. >

Re: SpatialRecursivePrefixTreeFieldType Spatial Searching

2013-06-04 Thread David Smiley (@MITRE.org)
t;>> On 6/3/13 7:27 AM, "Chris Atkinson" < > chrisacky@ > > wrote: >> >>> >> >>> >Hi, >> >>> >I'm seeing really slow query times. 7-25 seconds when I run a simple >> >>> >filter >> >>&g

Re: SpatialRecursivePrefixTreeFieldType Spatial Searching

2013-06-04 Thread Chris Atkinson
Hi David, Thanks for your continued help. I think that you have nailed it on the head for me. I'm 100% sure that I had previously tried that query without success. I'm not sure if perhaps I had wrong distErrPct or maxDistErr values... It's getting late, so I'm going to call it a night (I'm on GM

Re: SpatialRecursivePrefixTreeFieldType Spatial Searching

2013-06-04 Thread Smiley, David W.
So "availability" is the absence of any other document's indexed time duration overlapping with your availability query duration. So I think you should negate an overlaps query. The overlaps query looks like: Intersects(-Inf start end Inf). And remember the slight buffering needed as described o

Re: SpatialRecursivePrefixTreeFieldType Spatial Searching

2013-06-04 Thread Chris Atkinson
Here is an example I have tried. So let's assume that I want to checkIn on the 30th day, and leave on the 115th day. My query would be: -availability_spatial:"Intersects( 30 0 3650 115 )" However, that wouldn't match anything. Here is an example document below so you can see. (I've not negat

Re: SpatialRecursivePrefixTreeFieldType Spatial Searching

2013-06-04 Thread Chris Atkinson
Thanks David. Query times are really quick and my index is only 20Mb now which is about what I would expect. I'm having some problems figuring out what type of query I want to find *Available* properties with this new points system. I'm storing bookings against each document. So I have X Y coordi

Re: SpatialRecursivePrefixTreeFieldType Spatial Searching

2013-06-03 Thread Smiley, David W.
Hi Chris: Have you read: http://wiki.apache.org/solr/SpatialForTimeDurations You're modeling your data sub-optimally. Full precision rectangles (distErrPct=0) doesn't scale well and you're seeing that. You should represent your durations as a point and it will take up a fraction of the space (se

Re: SpatialRecursivePrefixTreeFieldType Spatial Searching

2013-06-03 Thread Chris Atkinson
Also, here is a sample query, and the debugQuery output fq={!cost=200}*:* -availability_spatial:"Intersects(182.6 0 199.4 1)" Incase the formatting is bad, here is a raw past of the debugQuery: http://pastie.org/pastes/872/text?key=ksjyboect4imrha0rck8sa 0 8171 true true *:* 1370259235

SpatialRecursivePrefixTreeFieldType Spatial Searching

2013-06-03 Thread Chris Atkinson
Hi, I'm seeing really slow query times. 7-25 seconds when I run a simple filter query that uses my SpatialRecursivePrefixTreeFieldType field. My index is about 30k documents. Prior to adding the Spatial field, the on disk space was about 100Mb, so it's a really tiny index. Once I add the spatial f