Mikhail, "Join" of any nature should be chosen in last resort to using a single index (when it's possible), especially if there is minimal to no denormalization of data. In this specific case, if the average document had 200 temporal ranges to index (100 days out, 2 per day), a Join based solution would have 200+1 documents in the index. That's an explosion of the document count by 200x! Yoyzah! Obviously what we're discussing here, modeling numeric ranges as x-y points has its limits -- namely that the spatial module is limited to 2 dimensions currently. It's plausible to see it generalized, but I don't think it'll scale well beyond 4-5 dimensions. I recall a research paper talking about multi-dimensional numeric indexes seriously breaking down at about 6.
~ David ________________________________ From: Mikhail Khludnev [via Lucene] [ml-node+s472066n4025602...@n3.nabble.com] Sent: Monday, December 10, 2012 12:15 AM To: Smiley, David W. Subject: Re: Modeling openinghours using multipoints Colleagues, What are benefits of this approach at contrast to block join? Thanks 10.12.2012 3:35 пользователь "Lance Norskog" <[hidden email]<UrlBlockedError.aspx>> написал: > If these are not raw times, but quantized on-the-hour, would it be > faster to create a bit map of hours and then query across the bit > maps? > > On Sun, Dec 9, 2012 at 8:06 AM, Erick Erickson <[hidden > email]<UrlBlockedError.aspx>> > wrote: > > Thanks for the discussion, I've added this to my bag of tricks, way cool! > > > > Erick > > > > > > On Sat, Dec 8, 2012 at 10:52 PM, britske <[hidden > > email]<UrlBlockedError.aspx>> wrote: > > > >> Brilliant! Got some great ideas for this. Indeed all sorts of usecases > >> which use multiple temporal ranges could benefit.. > >> > >> Eg: Another Guy on stackoverflow asked me about this some days ago.. He > >> wants to model multiple temporary offers per product (free shopping for > >> christmas, 20% discount for Black friday , etc) .. All possible with > this > >> out of the box. Factor in 'offer category' in x and y as well for some > >> extra powerfull querying. > >> > >> Yup im enthousiastic about it , which im sure you can tell :) > >> > >> Thanks a lot David, > >> > >> Cheers, > >> Geert-Jan > >> > >> > >> > >> Sent from my iPhone > >> > >> On 9 dec. 2012, at 05:35, "David Smiley (@MITRE.org) [via Lucene]" < > >> [hidden email]<UrlBlockedError.aspx>> wrote: > >> > >> > britske wrote > >> > That's seriously awesome! > >> > > >> > Some change in the query though: > >> > You described: "To query for a business that is open during at least > some > >> > part of a given time duration" > >> > I want "To query for a business that is open during at least the > entire > >> > given time duration". > >> > > >> > Feels like a small difference but probably isn't (I'm still wrapping > my > >> > head on the intersect query I must admit) > >> > So this would be a slightly different rectangle query. Interestingly, > >> you simply swap the location in the rectangle where you put the start > and > >> end time. In summary: > >> > > >> > Indexed span CONTAINS query span: > >> > minX minY maxX maxY -> 0 end start * > >> > > >> > Indexed span INTERSECTS (i.e. OVERLAPS) query span: > >> > minX minY maxX maxY -> 0 start end * > >> > > >> > Indexed span WITHIN query span: > >> > minX minY maxX maxY -> start 0 * end > >> > > >> > I'm using '*' here to denote the max possible value. At some point I > >> may add that as a feature. > >> > > >> > That was a fun exercise! I give you credit in prodding me in this > >> direction as I'm not sure if this use of spatial would have occurred to > me > >> otherwise. > >> > > >> > britske wrote > >> > Moreover, any indication on performance? Should, say, 50.000 docs with > >> > about 100-200 points each (1 a 2 open-close spans per day) be ok? ( I > >> know > >> > 'your mileage may very' etc. but just a guestimate :) > >> > You should have absolutely no problem. The real clincher in your > favor > >> is the fact that you only need 9600 discrete time values (so you said), > not > >> Long.MAX_VALUE. Using Long.MAX_VALUE would simply not be possible with > the > >> current implementation because it's using Doubles which has 52 bits of > >> precision not the 64 that would be required to be a complete substitute > for > >> any time/date. Even given the 52 bits, a quad SpatialPrefixTree with > >> maxLevels="52" would probably not perform well or might fail; not sure. > >> Eventually when I have time to work on an implementation that can be > based > >> on a configurable number of grid cells (not unlike how you can configure > >> precisionStep on the Trie numeric fields), 52 should be no problem. > >> > > >> > I'll have to remember to refer back to this email on the approach if I > >> create a field type that wraps this functionality. > >> > > >> > ~ David > >> > > >> > britske wrote > >> > Again, this looks good! > >> > Geert-Jan > >> > > >> > 2012/12/8 David Smiley (@MITRE.org) [via Lucene] < > >> > [hidden email]> > >> > > >> > > Hello again Geert-Jan! > >> > > > >> > > What you're trying to do is indeed possible with Solr 4 out of the > box. > >> > > Other terminology people use for this is multi-value time duration. > >> This > >> > > creative solution is a pure application of spatial without the > >> geospatial > >> > > notion -- we're not using an earth or other sphere model -- it's a > flat > >> > > plane. So no need to make reference to longitude & latitude, it's > x & > >> y. > >> > > > >> > > I would put opening time into x, and closing time into y. To > express a > >> > > point, use "x y" (x space y), and supply this as a string to your > >> > > SpatialRecursivePrefixTreeFieldType based field for indexing. You > can > >> give > >> > > it multiple values and it will work correctly; this is one of RPT's > >> main > >> > > features that set it apart from Solr 3 spatial. To query for a > >> business > >> > > that is open during at least some part of a given time duration, say > >> 6-8 > >> > > o'clock, the query would look like openDuration:"Intersects(minX > minY > >> maxX > >> > > maxY)" and put 0 or minX (always), 6 for minY (start time), 8 for > maxX > >> > > (end time), and the largest possible value for maxY. You wouldn't > >> actually > >> > > use 6 & 8, you'd use the number of 15 minute intervals since your > >> epoch for > >> > > this equivalent time span. > >> > > > >> > > You'll need to configure the field correctly: geo="false" > >> worldBounds="0 0 > >> > > maxTime maxTime" substituting an appropriate value for maxTime > based on > >> > > your unit of time (number of 15 minute intervals you need) and > >> > > distErrPct="0" (full precision). > >> > > > >> > > Let me know how this works for you. > >> > > > >> > > ~ David > >> > > Author: > >> > > http://www.packtpub.com/apache-solr-3-enterprise-search-server/book > >> > Author: > >> http://www.packtpub.com/apache-solr-3-enterprise-search-server/book > >> > > >> > > >> > If you reply to this email, your message will be added to the > discussion > >> below: > >> > > >> > http://lucene.472066.n3.nabble.com/Modeling-openinghours-using-multipoints-tp4025336p4025434.html > >> > To unsubscribe from Modeling openinghours using multipoints, click > here. > >> > NAML > >> > >> > >> > >> > >> -- > >> View this message in context: > >> > http://lucene.472066.n3.nabble.com/Modeling-openinghours-using-multipoints-tp4025336p4025454.html > >> Sent from the Solr - User mailing list archive at Nabble.com. > >> > > > > -- > Lance Norskog > [hidden email]<UrlBlockedError.aspx> > ________________________________ If you reply to this email, your message will be added to the discussion below: http://lucene.472066.n3.nabble.com/Modeling-openinghours-using-multipoints-tp4025336p4025602.html To unsubscribe from Modeling openinghours using multipoints, click here<http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=4025336&code=RFNNSUxFWUBtaXRyZS5vcmd8NDAyNTMzNnwxMDE2NDI2OTUw>. NAML<http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml> ----- Author: http://www.packtpub.com/apache-solr-3-enterprise-search-server/book -- View this message in context: http://lucene.472066.n3.nabble.com/Modeling-openinghours-using-multipoints-tp4025336p4025685.html Sent from the Solr - User mailing list archive at Nabble.com.
