Re: Solr spatial search within the polygon

2014-03-10 Thread David Smiley (@MITRE.org)
You're going to have to use the Lucene-spatial module directly then. There's SpatialExample.java to get you started. javinsnc wrote > > David Smiley (@MITRE.org) wrote >> On 3/10/14, 12:56 PM, "javinsnc" < >> javiersangrador@ >> >

Re: Solr spatial search within the polygon

2014-03-10 Thread David Smiley (@MITRE.org)
Lucene has multiple modules, one of which is "spatial". You'll see it in the source tree checkout underneath the lucene directory. Javadocs: http://lucene.apache.org/core/4_7_0/spatial/index.html SpatialExample.java: https://github.com/apache/lucene-solr/blob/trunk/lucene/spatial/src/test/org/apa

Re: Issue with spatial search

2014-03-10 Thread David Smiley (@MITRE.org)
Correct, Steve. Alternatively you can also put this option in your query after the end of the last parenthesis, as in this example from the wiki: fq=geo:"IsWithin(POLYGON((-10 30, -40 40, -10 -20, 40 20, 0 0, -10 30))) distErrPct=0" ~ David Steven Bower wrote > Only points in the index.. Am I

Re: Issue with spatial search

2014-03-11 Thread David Smiley (@MITRE.org)
centage? or maybe better put how does it impact perf? steve On Mon, Mar 10, 2014 at 11:17 PM, David Smiley (@MITRE.org) < [hidden email]> wrote: > Correct, Steve. Alternatively you can also put this option in your query > after the end of the last parenthesis, as in this example from t

Re: Index JTS Point in Solr/Lucene index

2013-10-28 Thread David Smiley (@MITRE.org)
Just follow-ing up with this thread after a round of emails between Shahbaz and I… David Smiley wrote > Ooooh, I see your confusion. You looked at code in an > UpdateRequestProcessor and expected it to work on the client in SolrJ. It > won't work for the reason that the code in the URP is creat

Removing last replica from a SolrCloud collection

2014-01-31 Thread David Smiley (@MITRE.org)
Hi, If I issue either a core UNLOAD command, or a collection DELETEREPLICA command, (which both seem pretty much equivalent) it works but if there are no other replicas for the shard, then the metadata for the shard is completely gone in clusterstate.json! That's pretty disconcerting because you

Clone (or Restore) Solrcloud

2014-01-31 Thread David Smiley (@MITRE.org)
Hi, I'm attempting to come up with a SolrCloud restore / clone process for either recover to a known good state or to clone the environment for experimentation. At the moment my process involves either creating a new zookeeper environment or at least deleting the existing Collection so that I can

Re: Removing last replica from a SolrCloud collection

2014-02-02 Thread David Smiley (@MITRE.org)
Mark, I appreciate all the context. ... Mark Miller-3 wrote > What do you mean you are hosed? > > All the SolrCores should be gone, so why does it matter so much that the > collection is gone? In the days before the collections api, if you wanted > to then create the collection again, just creat

Re: Removing last replica from a SolrCloud collection

2014-02-02 Thread David Smiley (@MITRE.org)
Thanks; that is *exactly* the behavior I'm talking about. I tried to find an existing issue before posting but missed this one somehow. Ramkumar R. Aiyengar wrote > There's already an issue for this, > https://issues.apache.org/jira/browse/SOLR-5209, we were once bitten by > the > same issue, wh

Re: Clone (or Restore) Solrcloud

2014-02-02 Thread David Smiley (@MITRE.org)
te that parent shard (since it's not needed anymore; it becomes inactive). I'm not sure why this metadata is recorded because, at least after the split, I can't see why it's pertinent to anything. ~ David David Smiley (@MITRE.org) wrote > Hi, > > I'm attempti

Re: Join Scoring

2014-02-11 Thread David Smiley (@MITRE.org)
Hi Anand. Solr's JOIN query, {!join}, constant-scores. It's simpler and faster and more memory efficient (particularly the worse-case memory use) to implement the JOIN query without scoring, so that's why. Of course, you might want it to score and pay whatever penalty is involved. For that you'

Re: Adding (spatial) filter query slows down avg response time

2014-03-01 Thread David Smiley (@MITRE.org)
Hi, The reason why it's working faster for you when you put {!geofilt} into your main query is most likely because the particular field type you are using is LatLonType, which works on a per-document basis, and by putting it in the main query it will, by default, leap-frog across the documents wit

Re: Solr is NoSQL database or not?

2014-03-01 Thread David Smiley (@MITRE.org)
+1 Excellent responses, Jack. - Author: http://www.packtpub.com/apache-solr-3-enterprise-search-server/book -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-is-NoSQL-database-or-not-tp4120554p4120682.html Sent from the Solr - User mailing list archive at Nabble.com.

Re: custom field type plugin

2013-07-22 Thread David Smiley (@MITRE.org)
Like Hoss said, you're going to have to solve this using http://wiki.apache.org/solr/SpatialForTimeDurations Using PointType is *not* going to work because your durations are multi-valued per document. It would be useful to create a custom field type that wraps the capability outlined on the wiki

Re: custom field type plugin

2013-07-23 Thread David Smiley (@MITRE.org)
s, since we have more than just > human data to deal with. It would get quite messy otherwise. > > > On 7/22/13 11:50 AM, "David Smiley (@MITRE.org)" < > DSMILEY@ > > wrote: > >>Like Hoss said, you're going to have to solve this using >>http

Re: Performance question on Spatial Search

2013-07-30 Thread David Smiley (@MITRE.org)
Steve, (1) Can you give a specific example of how your are specifying the spatial query? I'm looking to ensure you are not using "IsWithin", which is not meant for point data. If your query shape is a circle or the bounding box of a circle, you should use the geofilt query parser, otherwise use

Re: Ingesting geo data into Solr very slow

2013-07-30 Thread David Smiley (@MITRE.org)
Hi Marta, Presumably you are indexing polygons -- I suspect complex ones. There isn't too much that you can do about this right now other than index them in parallel. I see you are doing this in 2 threads; try 4, or maybe even 6. Also, ensure that maxDistErr is reflective of the smallest distan

Re: Performance question on Spatial Search

2013-08-05 Thread David Smiley (@MITRE.org)
From: "Steven Bower-2 [via Lucene]" mailto:ml-node+s472066n4082569...@n3.nabble.com>> Date: Monday, August 5, 2013 9:14 AM To: "Smiley, David W." mailto:dsmi...@mitre.org>> Subject: Re: Performance question on Spatial Search So after re-feeding our data with a new boolean field that is true when

Re: Multipoint date ranges with spatial - Invalid Longitude Exception?

2013-08-12 Thread David Smiley (@MITRE.org)
Hi Dan, FYI the main reference page on this technique is here: http://wiki.apache.org/solr/SpatialForTimeDurations (note the slight buffering needed for the query shape). You got the exception you got because you separated the dimensions of your query rectangle using a comma, which got Spatia

Re: SOLR4 Spatial sorting and query string

2013-08-13 Thread David Smiley (@MITRE.org)
Hi Roy. Using the example schema and data, and copying the "store" field to "store_rpt" indexed with location_rpt field type, try this query: http://localhost:8983/solr/select?indent=true&fl=name,store&q=*:*&sort=query%28{!geofilt%20score=distance%20filter=false%20sfield=store_rpt%20pt=45.15,-93.

Re: SOLR4 Spatial sorting and query string

2013-08-15 Thread David Smiley (@MITRE.org)
Hi Roy, You'll have to calculate this client-side. I am aware of this conundrum and I put up a TODO JIRA item for it here months ago: https://issues.apache.org/jira/browse/SOLR-4633It actually shouldn't be that hard to do. ~ David roySolr wrote > Hello David, > > The first months there w

Re: SOLR4 Spatial sorting and query string

2013-08-19 Thread David Smiley (@MITRE.org)
This is a known limitation. From CHANGES.txt: * SOLR-2345: Enhanced geodist() to work with an RPT field, provided that the field is referenced via 'sfield' and the query point is constant. (David Smiley) The reason why that limitation is there relates to the fact that the function query parse

Re: spatial search, geofilt does not work

2013-08-19 Thread David Smiley (@MITRE.org)
Thank goodness for Solr's feature of echo'ing params back in the response as it helps diagnose problems like this. In your case, the filter query that Solr is seeing isn't what you (seemed) to have given on the command line: "fq":"!geofilt sfield=author_geo" Clearly wrong. Try escaping the braces

Re: Use case of Spatial search

2013-08-19 Thread David Smiley (@MITRE.org)
Shishir, Use the location_rpt type and index circles of the business and the distance they serve with this syntax: Circle(lat,lon d=degreesRadius) Your query shape is then simply a point; use bbox query parser with d=0. This approach should scale *great* at query time. Erick suggesting using fun

Re: spatial search, geofilt does not work

2013-08-20 Thread David Smiley (@MITRE.org)
trying to figure out the problem for half day. > Probably Solr could use some error msg if the query format is invalid. > > But, THANKS! David, you probably saved me another half day. > > Ming- > > > > On Mon, Aug 19, 2013 at 10:20 PM, David Smiley (@MITRE.org) <

Re: Distance sort on a multi-value field

2013-08-20 Thread David Smiley (@MITRE.org)
The distance sorting code in SOLR-2155 is roughly equivalent to the code that RPT uses (RPT has its lineage in SOLR-2155 after all). I just reviewed it to double-check. It's possible the behavior is slightly better in SOLR-2155 because the cache (a Solr cache) contains normal hard-references wher

Re: Distance sort on a multi-value field

2013-08-22 Thread David Smiley (@MITRE.org)
erage response time down from 4 seconds to about > 50ms. > > Very nice! > > > > On 8/20/13 7:37 PM, "David Smiley (@MITRE.org)" < > DSMILEY@ > > wrote: > >>The distance sorting code in SOLR-2155 is roughly equivalent to the code >>that >&g

Re: How to access latitude and longitude with only LatLonType?

2013-08-22 Thread David Smiley (@MITRE.org)
Hi Quan You claim to be using LatLonType, yet the error you posted makes it clear you are in fact using SpatialRecursivePrefixTreeFieldType (RPT). Regardless of which spatial field you use, it's not clear to me what sort of statistics could be useful on a spatial field. The stats component doesn

Re: Geo spatial clustering of points

2013-08-22 Thread David Smiley (@MITRE.org)
Hi Chris & Jeroen, Tonight I posted some tips on Solr's wiki on this subject: http://wiki.apache.org/solr/SpatialClustering ~ David Chris Atkinson wrote > Did you get any resolution for this? I'm about to implement something > identical. > On 3 Jul 2013 23:03, "Jeroen Steggink" < > jeroen@ >

Re: Change the score of a document based on the *value* of a multifield using dismax

2013-09-03 Thread David Smiley (@MITRE.org)
If you want to alter the score in a customized way based on indexed text data on a per-value basis then index Lucene payloads, and use PayloadTermQuery. See the javadocs for PayloadTermQuery in particular and follow the references. This is a bit dated but read this: http://searchhub.org/2009/08/0

Re: Solr 4.5 spatial search - distance and score

2013-09-13 Thread David Smiley (@MITRE.org)
Hi Weber, Returning the distance separately from the score is really awkward without being able to use geodist() (which is coming in Solr 4.5 for the RPT spatial field). But as you note in SOLR-4255 it is possible. If you modify the Solr example schema so that the 'store' spatial field is of type

Re: DocValues vs stored fields?

2013-04-01 Thread David Smiley (@MITRE.org)
Otis, DocValues are quite insufficient for true field updates. DocValues is a per-document value storage (hence the name); it's not uninverted/indexed. If you needed to search based on these values (e.g. find all docs that have this value or between these values) then that's not going to work.

Re: Easier way to do this?

2013-04-12 Thread David Smiley (@MITRE.org)
Bill, I responded to the issue you created about this: https://issues.apache.org/jira/browse/SOLR-4704 In summary, use {!geofilt}. ~ David Billnbell wrote > I would love for the SOLR spatial 4 to support pt so that I can run # of > results around a central point easily like in 3.6. How can I

Re: Support old syntax including geodist

2013-04-12 Thread David Smiley (@MITRE.org)
Hi Bill, FYI see https://issues.apache.org/jira/browse/SOLR-4242 Billnbell wrote > Since Spatial Lucene 4 does not seem to support geodist(), even sending > d,pt,fq={!geofilt}does not help me = I need to sort. So I end up having to > set up the sortsq. Any other ideas on how to support the old s

Re: Spatial search question

2013-04-12 Thread David Smiley (@MITRE.org)
Yup, Lance is right. But it won't always work if you have multi-valued data since it wouldn't match a document that had a point both in the ring and the hole. Another approach that internally works faster and addresses the multi-value case is to implement a custom Spatial4j Shape. In this case,

Re: Easier way to do this?

2013-04-13 Thread David Smiley (@MITRE.org)
Good question. With geofilt it's kilometers. - Author: http://www.packtpub.com/apache-solr-3-enterprise-search-server/book -- View this message in context: http://lucene.472066.n3.nabble.com/Easier-way-to-do-this-tp4055474p4055784.html Sent from the Solr - User mailing list archive at Nab

Re: Multi dimensional spatial search

2013-05-11 Thread David Smiley (@MITRE.org)
Hi Kiran. The often-forgotten PointType field type can be configured to hold a variable number of dimensions. See the "dimension" attribute of the field type's configuration in the example schema. This field type is really just a kind of a macro field type for a configurable number of numeric fi

Re: How to improve performance of geodist()

2013-05-13 Thread David Smiley (@MITRE.org)
Hi Nicholas, Given that boosting is generally inherently fuzzy / inexact thing, you can likely get away with using simpler calculations. dist() can do the Euclidean distance (i.e. the Pythagorean theorem). If your data is in just one region of the world, you can project your data into a 2-D plan

Re: How to query docs with an indexed polygon field in java?

2013-05-23 Thread David Smiley (@MITRE.org)
Hi Kevenz, kevenz wrote > ... > String sql = "indexType:219" " AND " > "geo:Contains(POINT(114.078327401257,22.5424866754136))"; > ... > Then I got an error at "java.lang.IllegalArgumentException: missing > parens: Contains". Is there any suggestion? First of all, if your query shape is a point,

Re: multivalue location_rpt field not indexing with JSON format

2013-05-24 Thread David Smiley (@MITRE.org)
Hi Barbra, Solr needs to see a String for each point value, not a 2-element array. Your doc should look like: [{"id":"054ac6377d6ca4ad387f73b063000910","keywords":["time", "trouble", "exactly"],"description":"a anno is an anno is an anno", "location":["33.44844800999897,-111.98840074003"

Re: Restaurant availability from database

2013-05-24 Thread David Smiley (@MITRE.org)
Use this reference: http://wiki.apache.org/solr/SpatialForTimeDurations Alexandre Rafalovitch wrote > On Thu, May 23, 2013 at 6:47 PM, Amit Nithian < > anithian@ > > wrote: >> Hossman did a presentation on something similar to this using spatial >> data >> at a Solr meetup some months ago. >> >

Re: exact match country

2013-05-26 Thread David Smiley (@MITRE.org)
Hi Bill. So it seems you want an exact match to be first even if it is outside the spatial region, right? Your suggested implementation suggests this. And apparently you want to sort by distance, notwithstanding the exact match being first. Although you don't have to do this as two queries, I t

Re: SOLR 4.3.0 - How to make fq optional?

2013-05-28 Thread David Smiley (@MITRE.org)
Your client needs to know to submit the proper filter query conditionally. It's not really a spatial issue, and I disagree with the idea to make bbox (and all other query parsers for that matter) do nothing if not given an expected input. ~ David bbarani wrote > I am using the SOLR geospatial c

Re: SpatialRecursivePrefixTreeFieldType Spatial Searching

2013-06-04 Thread David Smiley (@MITRE.org)
maxDistErr should be like 0.3 based on earlier parts of this discussion since your data is to one of a couple hours of the day, not whole days. If it was whole days, you would use 1. Changing this requires a re-index. So does changing worldBounds if you do so. distErrPct should be 0. Changing i

Re: multiple dateranges/timeslots per doc: modeling openinghours.

2011-09-26 Thread David Smiley (@MITRE.org)
In case anyone is curious, I responded to him with a solution using either SOLR-2155 (Geohash prefix query filter) or LSP: https://issues.apache.org/jira/browse/SOLR-2155?focusedCommentId=13115244#comment-13115244 ~ David Smiley - Author: https://www.packtpub.com/solr-1-4-enterprise-search-s

Re: VelocityResponseWriter's future

2011-12-09 Thread David Smiley (@MITRE.org)
Erik, The /browse UI pages have definitely gotten a bit long in the tooth, to the point that it's a maintenance nightmare IMO. Perhaps it was inevitable given the nature of the technology; it would be the same situation with JSP/ASP/PHP. FWIW, in my experience with Endeca (a Solr competitor) the

Re: Pattern: Is there a method of resolving multivalued date ranges into a single document?

2012-01-06 Thread David Smiley (@MITRE.org)
See https://issues.apache.org/jira/browse/SOLR-2155?focusedCommentId=13114839&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13114839 with my response following Geert-Jan's question. ~ David - Author: http://www.packtpub.com/apache-solr-3-enterprise-search-serv

Wither field compresed="true" ?

2010-06-27 Thread David Smiley (@MITRE.org)
I just noticed that field compression (e.g. compressed="true") is no longer in Solr, nor can I find why this was done. Can a committer offer an explanation? If the reason is that it eats up CPU, then I'd rather accept this tradeoff for a big-data yet small query volume use case. ~ David Smiley

Re: Problem with migration to SolrAdaptersForLuceneSpatial4

2012-11-26 Thread David Smiley (@MITRE.org)
Hi Viacheslav, 1. You don't need JTS unless you're using polygons or WKT and your examples uses neither. So you can remove the spatialContext attribute to use the default, and remove the JTS jar. But that shouldn't be related to your reported problem. 2. The units for d= in the circle are in de

RE: Problem with migration to SolrAdaptersForLuceneSpatial4

2012-11-28 Thread David Smiley (@MITRE.org)
type usage. WBR Viacheslav. On 26.11.2012, at 18:52, David Smiley (@MITRE.org) wrote: > Hi Viacheslav, > > 1. You don't need JTS unless you're using polygons or WKT and your examples > uses neither. So you can remove the spatialContext attribute to use the > default, and

Re: Modeling openinghours using multipoints

2012-12-08 Thread David Smiley (@MITRE.org)
Hello again Geert-Jan! What you're trying to do is indeed possible with Solr 4 out of the box. Other terminology people use for this is multi-value time duration. This creative solution is a pure application of spatial without the geospatial notion -- we're not using an earth or other sphere mod

Re: Modeling openinghours using multipoints

2012-12-08 Thread David Smiley (@MITRE.org)
Trie numeric fields), 52 should be no problem. I'll have to remember to refer back to this email on the approach if I create a field type that wraps this functionality. ~ David britske wrote > Again, this looks good! > Geert-Jan > > 2012/12/8 David Smiley (@MITRE.org) [via Luce

RE: Modeling openinghours using multipoints

2012-12-10 Thread David Smiley (@MITRE.org)
which im sure you can tell :) >> >> Thanks a lot David, >> >> Cheers, >> Geert-Jan >> >> >> >> Sent from my iPhone >> >> On 9 dec. 2012, at 05:35, "David Smiley (@MITRE.org) [via Lucene]" < >> [hidden email]> wr

RE: Modeling openinghours using multipoints

2012-12-10 Thread David Smiley (@MITRE.org)
Black friday , etc) .. All possible with > this > >> out of the box. Factor in 'offer category' in x and y as well for some > >> extra powerfull querying. > >> > >> Yup im enthousiastic about it , which im sure you can tell :) > >> > >

Re: Intersect Circle is matching points way outside the radius ( Solr 4 Spatial)

2012-12-10 Thread David Smiley (@MITRE.org)
Javi, The center point of your query circle and the indexed point is just under 49.9km (just under your query radius); this is why it matched. I plugged in your numbers here: http://www.movable-type.co.uk/scripts/latlong.html Perhaps you are misled by the projection you are using to view the map

Re: Intersect Circle is matching points way outside the radius ( Solr 4 Spatial)

2012-12-10 Thread David Smiley (@MITRE.org)
there is nothing wrong > with Solr and that I didn't mix the concepts, it is just as in many cases > the problem is somewhere else where you would never imagine. > > Thanks for the hint. > > Cheers, > Javier > > > > > > On 11 December 2012 02:47, D

Re: suggestion howto handle highly repetitive valued field

2012-12-11 Thread David Smiley (@MITRE.org)
The indexed="true" side is quite efficient. The stored="true" side -- not so much, but the strings you have here are pretty small and I wouldn't worry about it. Solr 4.1 (unreleased) does a great job here and compresses all the stored field data across documents. ~ David Jie Sun wrote > Hi - >

Re: modeling prices based on daterange using multipoints

2012-12-11 Thread David Smiley (@MITRE.org)
Hi Britske, This is a very interesting question! britske wrote > ... > I realize the new spatial-stuff in Solr 4 is no magic bullet, but I'm > wondering if I could model multiple prices per day as multipoints, > whereas: > > - date*duration*nr of persons*roomtype is modeled as point.x (discr

Re: modeling prices based on daterange using multipoints

2012-12-11 Thread David Smiley (@MITRE.org)
britske wrote > Hi David, > > Yeah interesting (as well as problematic as far is implementing) use-case > indeed :) > > 1. You mention "there are no special caches / memory requirements inherent > in this.". For a given user-query this would mean all hotels would have to > seach for all point.x e

Re: modeling prices based on daterange using multipoints

2012-12-12 Thread David Smiley (@MITRE.org)
britske wrote >> Ah; ok. But still, my first suggestion is still what I think you could >> do >> except that the algorithm is simpler -- return the first matching 'y' in >> the >> document where the point matches the query. Alternatively, if you're >> confident the number of matching documents (h

Re: Spatial filter in solr 4.0 - "Intersects" operation with parameters

2012-12-25 Thread David Smiley (@MITRE.org)
Hi Mladen, Despite some similarities at first glance, the Solr 4 spatial fields are not implemented with Solr query parsers, unlike Solr 3 spatial. Everything in quotes is handled by the field type. What you're looking for is for the Solr 3 geospatial functions to be adapted to support the Solr

Re: Spatial filter in solr 4.0 - "Intersects" operation with parameters

2012-12-26 Thread David Smiley (@MITRE.org)
It's unusual to have Solr be the first point of entry into a service. Usually it's fronted with a web application that has the business logic that knows how to map the request to the search back-end. Given your further questions, almost anything could work without much trouble: * A standard servl

Re: Spatial filter in solr 4.0 - "Intersects" operation with parameters

2013-01-02 Thread David Smiley (@MITRE.org)
Mladen, FYI I just committed this to 4.x: https://issues.apache.org/jira/browse/SOLR-4230 ~ David mladen micevic wrote > Hi, > I went through example for spatial search in Solr 4.0 > (http://wiki.apache.org/solr/SolrAdaptersForLuceneSpatial4) > Both indexing and searching work fine. > > Examp

Re: Using Solr Spatial in conjunction with HBASE/Hadoop

2013-01-17 Thread David Smiley (@MITRE.org)
Hi Oakstream, Coincidentally I've been thinking of porting the geohash prefixtree intersection algorithm in Lucene 4 spatial to Accumulo (another big-table system like HBase). There's a decent chance it'll happen this year, I think. That doesn't help your need right now of course so go with Otis

Re: Using Solr Spatial in conjunction with HBASE/Hadoop

2013-01-19 Thread David Smiley (@MITRE.org)
oakstream wrote > Thanks guys! > David, > > In general and in your opinion would Lucene Spatial be the way to go to > index hundreds of terabytes of spatial data that continually grows. > Mostly point data, mostly structured, however, could be polygons. The > searches would be within or contains

Re: Distance Range query

2013-01-21 Thread David Smiley (@MITRE.org)
Hi Stefano. I answered someone's question on stackoverflow which is basically the same question you have: http://stackoverflow.com/questions/13723891/lucene-4-0-spatial-calculate-max-distance-dynamically-using-indexed-documets/13764793#13764793 Essentially, you should index circles and then searc

Re: Spatial+Dataimport full import results in OutOfMemory for a rectangle defining a line

2013-01-21 Thread David Smiley (@MITRE.org)
Javier, Your minX is slightly greater than maxX, which is interpreted as a line that wraps nearly the entire globe. Is that what you intended? If this is what you intended, then you got bitten by this unfixed bug: https://issues.apache.org/jira/browse/LUCENE-4550 As a work-around, you could spli

Re: Using Solr Spatial in conjunction with HBASE/Hadoop

2013-01-21 Thread David Smiley (@MITRE.org)
What good is a key-value store in the context of oakstream's question? ~ David - Author: http://www.packtpub.com/apache-solr-3-enterprise-search-server/book -- View this message in context: http://lucene.472066.n3.nabble.com/Using-Solr-Spatial-in-conjunction-with-HBASE-Hadoop-tp4034307p40

Re: Spatial+Dataimport full import results in OutOfMemory for a rectangle defining a line

2013-01-21 Thread David Smiley (@MITRE.org)
hat multivalued field that > intersects with the given area? > > Thanks, > Javier > > > > > > On 22 January 2013 05:43, David Smiley (@MITRE.org) < > DSMILEY@ > >wrote: > >> Javier, >> >> Your minX is slightly greater tha

Re: Spatial+Dataimport full import results in OutOfMemory for a rectangle defining a line

2013-01-22 Thread David Smiley (@MITRE.org)
Javier Molina wrote > This very wide rectangle will cause an OutOfMemoryError > > -180 3 180 3.016668 > > While this one, slightly taller will work fine. > > -180 3 180 3.5 Due to the bug, the accuracy computing algorithm believes the width for both of these is 0. That algorithm also l

Re: Distance Range query

2013-01-22 Thread David Smiley (@MITRE.org)
I'm glad to be of help. This is all possible using Solr 4 without programatic customization. As always remember the docs: http://wiki.apache.org/solr/SolrAdaptersForLuceneSpatial4#IndexingThat has an example of index a circle. Not much to it. The only aspect of the SpatialRecursivePrefixTree

RE: solr 4.1 spatial with JTS - spatial query withitin a WKT polygon contained within another query ...

2013-02-27 Thread David Smiley (@MITRE.org)
Pires, Guilherme wrote > Hello David, > > Thanks for your response. > I'm working deeply in this and it's fully decided that solr 4.1 + JTS is > going to be supporting the map navigation for a 'public facing' GIS > solution, i.e. will deliver the objects return by a bounding box > intersection. In

Re: geodist() spatial sorting: sort param could not be parsed as a query, and is not a field that exists in the index: geodist()

2013-02-28 Thread David Smiley (@MITRE.org)
Strange. The code in Solr that has that error string passes an an additional exception that will have its own error message that is more detailed, and you'll see that in the stack trace in the Solr logs; perhaps in your error response too but I'm not sure. If you remove the sorting, are the searc

Re: Solr 4.1: problems with Spatial Search.

2013-03-05 Thread David Smiley (@MITRE.org)
Hmm; weird. It looks right. Does it work without the sort? -- i.e. does the filter work? Are there more interesting looking error messages output by Solr? Rakudten wrote > Hello! > > I´m trying to sort by geodist() distance, but it seems that I can´t: > > *The query:* > > http://192.168.1.1

Re: Solr 4.1: problems with Spatial Search.

2013-03-06 Thread David Smiley (@MITRE.org)
s Cappa > > > 2013/3/5 Chris Hostetter < > hossman_lucene@ > > > >> >> 1) which version of solr are you using? >> 2) what is the field & fieldtype for "geolocation" >> 2) can you try changing your query to "q={!func}geodist(

Re: Solr 4.1: problems with Spatial Search.

2013-03-06 Thread David Smiley (@MITRE.org)
entation is a little bit confusing, but know It works perfectly. > > Regards, > > - Luis Cappa > > 2013/3/6 David Smiley (@MITRE.org) <[hidden > email]> > > Ah; bingo! >> >> The top error in the log is what Solr reports in the HTTP response

Re: Migrate Solr 3.4 w/ solr-1255 GeoHash to Solr 4

2013-03-06 Thread David Smiley (@MITRE.org)
Hi Harley, See: http://wiki.apache.org/solr/SolrAdaptersForLuceneSpatial4 In SOLR-2155 you had to explicitly specify the prefix encoding length, whereas in Solr 4 you specify how much precision you need and it figures out what the length is that satisfies that. When you first use the field, it'll

RE: Migrate Solr 3.4 w/ solr-1255 GeoHash to Solr 4

2013-03-08 Thread David Smiley (@MITRE.org)
s Parks, GISP > Booz | Allen | Hamilton > Geospatial Visualization Web Developer > > WEB: https://www.apan.org > > USPACOM J73/APAN > Pacific Warfighting Center Ford Island > > p: 808.472.7752 > c: 808.377.0632 > apan: > harley.parks@ > nipr:   > harley.pa

RE: Migrate Solr 3.4 w/ solr-1255 GeoHash to Solr 4

2013-03-08 Thread David Smiley (@MITRE.org)
You're supposed to add geo point data in "latitude, longitude" format, although some other variations work. Is your updating process supplying a geohash instead? If so you could write a simple Solr UpdateRequestProcessor to convert it to the expected format. But that doesn't help the fact that a

Re: InvalidShapeException when using SpatialRecursivePrefixTreeFieldType with custom worldBounds

2013-03-08 Thread David Smiley (@MITRE.org)
Hi Jon. If you're able to trigger an IndexOutOfBoundsException out of the prefix tree then please file a bug (to the Lucene project, not Solr). I'll look into it when I have time. I need to add a Wiki page on the use of spatial for time ranges; there are some tricks to it. Nevertheless you've de

Re: Solr 4.1: problems with Spatial Search.

2013-03-09 Thread David Smiley (@MITRE.org)
Rakudten wrote > Hello again! > > Uhm, so if I have understood then if I´m writing/reading from the index at > the same time (in other words, indexing operations are executing at the > same that that other ones are querying) the performance goes down, isn´t > it? Committing is the problem (soft o

Re: Improving performance for SOLR geo queries?

2012-02-12 Thread David Smiley (@MITRE.org)
Mathias, For what it's worth, someone using LSP (Lucene Spatial Playground)'s RecursivePrefixTreeFieldType (which uses geohash encoding by default) quoted a 2x performance increase over Solr's built-in LatLonType. To boost the performance further, there are a couple parameters you can tweak. One

Re: Geospatial search with multivalued field

2012-02-12 Thread David Smiley (@MITRE.org)
Hi Marian. I'm the author of Solr2155. SpatialQueryable is an interface provided by Solr. Where did you put the SOLR-2155 built jar file? Perhaps you put it where it doesn't belong like in your servlet engine's lib directory (varies). You can put this in a variety of places like embedding it

Re: Spatial4j

2012-04-23 Thread David Smiley (@MITRE.org)
Yes, I definitely think so. At a minimum, I expect there will at least be a patch or built jar file for you to get going by 1 June. - Author: http://www.packtpub.com/apache-solr-3-enterprise-search-server/book -- View this message in context: http://lucene.472066.n3.nabble.com/Spatial4j-tp3

Re: Which version of Solr?

2011-02-13 Thread David Smiley (@MITRE.org)
Maven support got a major shot in the arm for releases going forward. If you want to start with 3x, then get the source for that branch and do a build. You should see some ant tasks which build maven artifacts. My guestimate on the 3x release date is 3-4 weeks from now. ~ David Smiley (author o

Re: Which version of Solr?

2011-02-14 Thread David Smiley (@MITRE.org)
Wow; I'm glad you figured it out -- sort of. FYI, in the future, don't hijack email threads to talk about a new subject. Start a new thread. ~ David p.s. yes, I'm working on the 2nd edition. - Author: https://www.packtpub.com/solr-1-4-enterprise-search-server/book -- View this message in

Re: [ANNOUNCE] Web Crawler

2011-03-01 Thread David Smiley (@MITRE.org)
Dominique, The obvious number one question is of course why you re-invented this wheel when there are several existing crawlers to choose from. Your website says the reason is that the UIs on existing crawlers (e.g. Nutch, Heritrix, ...) weren't sufficiently user-friendly or had the site-specific

Re: Custom request handler/plugin

2011-03-13 Thread David Smiley (@MITRE.org)
Your approach is problematic. It probably doesn't factor in things like faceting, highlighting, ... Can you say why it is that the original search can't include a filter query to filter out results the user shouldn't have access to? That's the standard approach to implement access control. ~ Davi

Re: Custom request handler/plugin

2011-03-13 Thread David Smiley (@MITRE.org)
*If* you decide to check each result found in your search results against some 3rd party so-called "entitlement service" which is essentially a black box for this discussion, then your search results are going to be disastrously slow; no? Especially if it's a SOAP one ;-) (I'm looking at your use

Re: Sorting on multiValued fields via function query

2011-03-15 Thread David Smiley (@MITRE.org)
Hi Harish. Did sorting on multiValued fields actually work correctly for you before? I'd be surprised if so. I could be wrong but I think you previously always got the sorting affects of whatever was the last indexed value. It is indeed the case that the FieldCache only supports up to one indexed

Re: noobie question: sorting

2011-03-15 Thread David Smiley (@MITRE.org)
Hi. Where did you find such an obtuse example? Recently, Solr supports sorting by function query. One such function is named "query" which takes a query and uses the score of the result of that query as the function's result. Due to constraints of where this query is placed within a function qu

Re: MoreLikeThis with document that has not been indexed

2011-03-30 Thread David Smiley (@MITRE.org)
Ben, It's absolutely possible for MLT to find documents similar to another indexed document. That's its primary use case. For externally supplied data, you will need to supply one blob of text. You could derive this by concatenating applicable parts of your structured data before handing to Solr.

Re: difference between geospatial search from database angle and from solr angle

2011-04-06 Thread David Smiley (@MITRE.org)
Sean, Geospatial search in Lucene/Solr is of course implemented based on Lucene's underlying index technology. That technology was originally just for text but it's been adapted very successfully for numerics and querying ranges too. The only mature geospatial field type in Solr 3.1 is LatLonTy

Re: KStemmer for Solr 3.x +

2011-04-08 Thread David Smiley (@MITRE.org)
I see no reason why it would not be compatible. - Author: https://www.packtpub.com/solr-1-4-enterprise-search-server/book -- View this message in context: http://lucene.472066.n3.nabble.com/KStemmer-for-Solr-3-x-tp2796594p2798213.html Sent from the Solr - User mailing list archive at Nabble.

Re: Pattern: Is there a method of resolving multivalued date ranges into a single document?

2011-06-11 Thread David Smiley (@MITRE.org)
Judioo, This is an interesting problem I've contemplated before. I've been working on geospatial support for Solr using a prefix/tree/grid technique that supports multi-valued fields. Geo/spatial might be 2D (x,y), but your problem is 1D (time) which means you can use spatial techniques just by ig

Re: intersecting map extent with solr spatial documents

2011-06-24 Thread David Smiley (@MITRE.org)
Very cool! What you've essentially described is a way of indexing & searching lat-lon box shapes, and the cool thing is that you were able to do this without custom coding / hacking of Solr. Sweet! I do have some observations about this approach: 1. Doesn't support variable number of shapes per

Re: intersecting map extent with solr spatial documents

2011-06-27 Thread David Smiley (@MITRE.org)
Steve, I like your geospatial boosting algorithm; it makes sense. FYI I'm collaborating with a couple other Lucene/Solr committers on a new geospatial module here: http://code.google.com/p/lucene-spatial-playground/ It is very much in-progress. There is variable-points per document support (ba

Re: bbox query syntax

2011-07-01 Thread David Smiley (@MITRE.org)
Hi. By the way, your uses of parenthesis are completely superfluous. You can't just plop that "{!" syntax anywhere you please, it only works at the beginning of a query to establish the query parser for the rest of the string and/or to set "local-params". There is a sub-query hacky syntax: ... AN

Re: SolrJ & Spatial Search

2011-07-10 Thread David Smiley (@MITRE.org)
Saïd, The misunderstanding you have is that you are confusing the user query (the q parameter) with the URL that Solr sees. Well actually the part of the URL that is called the query string -- that which is after the "?". SolrQuery has various setters for well-known parameters, and others just us

Solr Ecosystem / Integration wiki pages

2011-07-10 Thread David Smiley (@MITRE.org)
Hi folks. I've been keeping a categorized list of software that relates/integrates with Solr in some way. I've dubbed this the "Solr Ecosystem". This Solr ecosystem concept was going to be an appendix in an upcoming 2nd edition of my book http://www.packtpub.com/apache-solr-3-enterprise-search-se

  1   2   >