RE: Spatial Search on Postal Code
re: Postal Codes and polygons. I've heard of basic techniques that use Commerce Department (or was it Census within Commerce??) that give the basic points, but the real run is deciding what the "center" of that polygon is. There is likely a commercial solution available, and certainly you can buy a spreadsheet with the zipcodes and their guestimated center. Fun project! > Subject: Re: Spatial Search on Postal Code > To: solr-user@lucene.apache.org > From: emir.arnauto...@sematext.com > Date: Fri, 4 Mar 2016 21:18:10 +0100 > > Hi Manohar, > I don't think there is such functionality in Solr - you need to do it on > client side: > 1. find some postal code polygons (you can use open street map - > http://wiki.openstreetmap.org/wiki/Key:postal_code) > 2. create zip to polygon lookup > 3. create code that will expand zip code polygon by some distance (you > can use JTS buffer api) > > On query time you get zip code and distance: > 1. find polygon for zip > 2. expand polygon > 3. send resulting polygon to Solr and use Intersects function to filter > results > > Regards, > Emir > > On 04.03.2016 19:49, Manohar Sripada wrote: > > Thanks Emir, > > > > Obviously #2 approach is much better. I know its not straight forward. But, > > is it really acheivable in Solr? Like building a polygon for a postal code. > > If so, can you throw some light how to do? > > > > Thanks, > > Manohar > > > > On Friday, March 4, 2016, Emir Arnautovic > > wrote: > > > >> Hi Manohar, > >> This depends on your requirements/usecase. If postal code is interpreted > >> as point than it is expected to have radius that is significantly larger > >> than postal code diameter. In such case you can go with first approach. In > >> order to avoid missing results from postal code in case of small search > >> radius and large postal code, you can reverse geocode records and store > >> postal code with each document. > >> If you need to handle distance from postal code precisely - distance from > >> its border, you have to get postal code polygon, expand it by search > >> distance and use resulting polygon to find matches. > >> > >> HTH, > >> Emir > >> > >> On 04.03.2016 13:09, Manohar Sripada wrote: > >> > >>> Here's my requirement - User enters postal code and provides the radius. > >>> I > >>> need to find the records with in the radius from the provided postal code. > >>> > >>> There are few ways I thought through after going through the "Spatial > >>> Search" Solr wiki > >>> > >>> 1. As Latitude and Longitude positions are required for spatial search. > >>> Get > >>> Latitude Longitude position (may be using GeoCoding API) of a postal code > >>> and use "LatLonType" field type and query accordingly. As the GeoCoding > >>> API > >>> returns one point and if the postal code area is too big, then I may end > >>> up > >>> not getting any results (apart from the records from the same postal code) > >>> if the radius provided is small. > >>> > >>> 2. Get the latitude longitude points of the postal code which forms a > >>> border (not sure yet on how to get) and build a polygon (using RPT). While > >>> querying use this polygon and provide the distance. Can this be achieved? > >>> Or Am I ruminating too much? :( > >>> > >>> Appreciate any help on this. > >>> > >>> Thanks > >>> > >>> > >> -- > >> Monitoring * Alerting * Anomaly Detection * Centralized Log Management > >> Solr & Elasticsearch Support * http://sematext.com/ > >> > >> > > -- > Monitoring * Alerting * Anomaly Detection * Centralized Log Management > Solr & Elasticsearch Support * http://sematext.com/ >
Re: What is the best way to index 15 million documents of total size 425 GB?
Thanks Toke, Charlie and Evert, I will reconsider using a light weight program to read and index data from database to Solr..! Hi Walter, no we have not yet started with atomic updates; just the plan is in progress. Current issue is with respect to indexing the document as such in bulk. On Fri, Mar 4, 2016 at 10:11 PM, Evert R. wrote: > I have worked with Pentaho and I believe your problem might be there > > Try to settled a quick php script and you might get better results with it. > There is no need for Data Integration on that. > > Just a tip. > Em 04/03/2016 13:12, "Walter Underwood" escreveu: > > > > > > On Mar 3, 2016, at 9:54 AM, Aneesh Mon N wrote: > > > > > > To be noted that all the fields are stored so as to support the atomic > > > updates. > > > > Are you doing all of these updates as atomic? That could be slow. If you > > are supplying all the fields, then just do a regular add. > > > > wunder > > Walter Underwood > > wun...@wunderwood.org > > http://observer.wunderwood.org/ (my blog) > > > > > -- Regards, Aneesh Mon N Bangalore +91-8197-188-588
Re: Question about Solr logs
Thanks Shawn.
To make sure I get this right, I see two methods on UpdateResponse class,
is getElapsedTime the client time and getQTime Solr's time? If so, than
getElapsedTime is how long my call was blocked, right? And getQTime will
have the value of 1174 (per the log of my example), right?
Steve
On Sat, Mar 5, 2016 at 1:33 AM, Shawn Heisey wrote:
> On 3/4/2016 10:21 PM, Steven White wrote:
> > org.apache.solr.update.processor.LogUpdateProcessor; [test]
> > webapp=/solr path=/update params={wt=xml&version=2.2} {add=[5539783
> > (1527883353280217088), 5539867 (1527883353296994304), , ... (101 adds)]}
> 0
> > 1174
> >
> > What does this log tell me? Is "1174" the time (in milliseconds) it took
> > Solr to process those 101 documents? Does this mean "solrConn.add(docs,
> > 1)" was blocked for "1174" milliseconds?
>
> Yes, the QTime on the request was 1174 milliseconds. The UpdateResponse
> object has a getElapsedTime method that will tell you how long the
> request took from the client's point of view. Depending on which
> SolrClient implementation you used, as well as other performance
> factors, it may block for more or less time than what Solr reports in
> the QTime parameter.
>
> Thanks,
> Shawn
>
>
Disk Usage anomoly across shards/replicas
Hi, I have an index with 65m docs spread across 2 shards, each with 1 replica. The replica1 of shard2 is using up nearly double the amount of disk space as the other shards/replicas. Could there be a reason/fix for this? /home/s123/solr/data/de_shard1_replica1 = 72G numDocs:34,786,026 maxDoc:45,825,444 deletedDocs:11,039,418 /home/s123/solr/data/de_shard1_replica2 = 70G numDocs:34,786,026 maxDoc:46,914,095 deletedDocs:12,128,069 /home/s123/solr/data/de_shard2_replica1 = 138G numDocs:34,775,193 maxDoc:45,409,362 deletedDocs:10,634,169 /home/s123/solr/data/de_shard2_replica2 = 66G numDocs:34,775,193 maxDoc:44,181,734 deletedDocs:9,406,541 Thanks, Rob
Re: Disk Usage anomoly across shards/replicas
Have you executed an optimize across that particular shard? On Sat, 5 Mar 2016, 18:39 Robert Brown, wrote: > Hi, > > I have an index with 65m docs spread across 2 shards, each with 1 replica. > > The replica1 of shard2 is using up nearly double the amount of disk > space as the other shards/replicas. > > Could there be a reason/fix for this? > > > /home/s123/solr/data/de_shard1_replica1 = 72G > > numDocs:34,786,026 > maxDoc:45,825,444 > deletedDocs:11,039,418 > > > > /home/s123/solr/data/de_shard1_replica2 = 70G > > numDocs:34,786,026 > maxDoc:46,914,095 > deletedDocs:12,128,069 > > > > /home/s123/solr/data/de_shard2_replica1 = 138G > > numDocs:34,775,193 > maxDoc:45,409,362 > deletedDocs:10,634,169 > > > > /home/s123/solr/data/de_shard2_replica2 = 66G > > numDocs:34,775,193 > maxDoc:44,181,734 > deletedDocs:9,406,541 > > > > Thanks, > Rob > > > > > > -- Regards, Binoy Dalal
Re: Disk Usage anomoly across shards/replicas
Nope, we never run optimise. Would there be some tell-tale files in the index dir to indicate if someone else had ran an optimise? On 05/03/16 13:11, Binoy Dalal wrote: Have you executed an optimize across that particular shard? On Sat, 5 Mar 2016, 18:39 Robert Brown, wrote: Hi, I have an index with 65m docs spread across 2 shards, each with 1 replica. The replica1 of shard2 is using up nearly double the amount of disk space as the other shards/replicas. Could there be a reason/fix for this? /home/s123/solr/data/de_shard1_replica1 = 72G numDocs:34,786,026 maxDoc:45,825,444 deletedDocs:11,039,418 /home/s123/solr/data/de_shard1_replica2 = 70G numDocs:34,786,026 maxDoc:46,914,095 deletedDocs:12,128,069 /home/s123/solr/data/de_shard2_replica1 = 138G numDocs:34,775,193 maxDoc:45,409,362 deletedDocs:10,634,169 /home/s123/solr/data/de_shard2_replica2 = 66G numDocs:34,775,193 maxDoc:44,181,734 deletedDocs:9,406,541 Thanks, Rob -- Regards, Binoy Dalal
Re: Spatial Search on Postal Code
Another path to consider is doing this point-in-zipcode-poly lookup at index time and enriching the document with a zipcode field (possibly multi-valued if there is doubt). On Sat, Mar 5, 2016 at 4:05 AM steve shepard wrote: > re: Postal Codes and polygons. I've heard of basic techniques that use > Commerce Department (or was it Census within Commerce??) that give the > basic points, but the real run is deciding what the "center" of that > polygon is. There is likely a commercial solution available, and certainly > you can buy a spreadsheet with the zipcodes and their guestimated center. > Fun project! > > > Subject: Re: Spatial Search on Postal Code > > To: solr-user@lucene.apache.org > > From: emir.arnauto...@sematext.com > > Date: Fri, 4 Mar 2016 21:18:10 +0100 > > > > Hi Manohar, > > I don't think there is such functionality in Solr - you need to do it on > > client side: > > 1. find some postal code polygons (you can use open street map - > > http://wiki.openstreetmap.org/wiki/Key:postal_code) > > 2. create zip to polygon lookup > > 3. create code that will expand zip code polygon by some distance (you > > can use JTS buffer api) > > > > On query time you get zip code and distance: > > 1. find polygon for zip > > 2. expand polygon > > 3. send resulting polygon to Solr and use Intersects function to filter > > results > > > > Regards, > > Emir > > > > On 04.03.2016 19:49, Manohar Sripada wrote: > > > Thanks Emir, > > > > > > Obviously #2 approach is much better. I know its not straight forward. > But, > > > is it really acheivable in Solr? Like building a polygon for a postal > code. > > > If so, can you throw some light how to do? > > > > > > Thanks, > > > Manohar > > > > > > On Friday, March 4, 2016, Emir Arnautovic < > emir.arnauto...@sematext.com> > > > wrote: > > > > > >> Hi Manohar, > > >> This depends on your requirements/usecase. If postal code is > interpreted > > >> as point than it is expected to have radius that is significantly > larger > > >> than postal code diameter. In such case you can go with first > approach. In > > >> order to avoid missing results from postal code in case of small > search > > >> radius and large postal code, you can reverse geocode records and > store > > >> postal code with each document. > > >> If you need to handle distance from postal code precisely - distance > from > > >> its border, you have to get postal code polygon, expand it by search > > >> distance and use resulting polygon to find matches. > > >> > > >> HTH, > > >> Emir > > >> > > >> On 04.03.2016 13:09, Manohar Sripada wrote: > > >> > > >>> Here's my requirement - User enters postal code and provides the > radius. > > >>> I > > >>> need to find the records with in the radius from the provided postal > code. > > >>> > > >>> There are few ways I thought through after going through the "Spatial > > >>> Search" Solr wiki > > >>> > > >>> 1. As Latitude and Longitude positions are required for spatial > search. > > >>> Get > > >>> Latitude Longitude position (may be using GeoCoding API) of a postal > code > > >>> and use "LatLonType" field type and query accordingly. As the > GeoCoding > > >>> API > > >>> returns one point and if the postal code area is too big, then I may > end > > >>> up > > >>> not getting any results (apart from the records from the same postal > code) > > >>> if the radius provided is small. > > >>> > > >>> 2. Get the latitude longitude points of the postal code which forms a > > >>> border (not sure yet on how to get) and build a polygon (using RPT). > While > > >>> querying use this polygon and provide the distance. Can this be > achieved? > > >>> Or Am I ruminating too much? :( > > >>> > > >>> Appreciate any help on this. > > >>> > > >>> Thanks > > >>> > > >>> > > >> -- > > >> Monitoring * Alerting * Anomaly Detection * Centralized Log Management > > >> Solr & Elasticsearch Support * http://sematext.com/ > > >> > > >> > > > > -- > > Monitoring * Alerting * Anomaly Detection * Centralized Log Management > > Solr & Elasticsearch Support * http://sematext.com/ > > > -- Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker LinkedIn: http://linkedin.com/in/davidwsmiley | Book: http://www.solrenterprisesearchserver.com
Re: How to use geospatial search to find the locations within polygon
Yes you're right Jack. The ref guide page on this needs more.
On Fri, Mar 4, 2016 at 11:36 AM Jack Krupansky
wrote:
> It would be nice for the doc to say that - describe when IsWithin is and
> isn't appropriate. And give some examples as well for people to copy/mimic.
>
> -- Jack Krupansky
>
> On Fri, Mar 4, 2016 at 10:20 AM, david.w.smi...@gmail.com <
> david.w.smi...@gmail.com> wrote:
>
> > First of all, assuming this is a standard point-in-polygon situation, use
> > the Intersects predicate -- with point data it's semantically the same as
> > IsWithin and Intersects is much faster. I don't know why you used
> > isDisjointTo in your 2nd example; maybe you want to find when they don't
> > touch? Any way, one problem right away I saw is that the first point in
> > the polygon is not repeated in the last. That's what the WKT spec
> demands.
> >
> >
> > On Fri, Mar 4, 2016 at 1:37 AM Pradeepchandra Mulpuru <
> > prade...@infologitech.in> wrote:
> >
> > > Hi Sir,
> > >
> > > I have a question on Apache Solr Spatial search. I have a json type
> data
> > > of City, Latitude & Longitude. I indexed those fields with locm_place
> of
> > > the type location_rpt. Now I want to give a polygon as a filter query
> in
> > > order to get the City names located in that polygon. I don't have any
> > idea
> > > of doing that.
> > >
> > > I tried with this:
> > >
> > >
> > >
> >
> http://localhost:8983/solr/loopback/select?fl=City&wt=json&q=*:*&fq=locm_place
> > :"IsWithin(POLYGON((16.762467717941604
> > > 78.94775390625,16.99375545289456 78.11279296875%20,17.31917640744285
> > > 77.98095703125,17.80099604766698 78.72802734375))) distErrPct=0"
> > >
> > > It is showing the result like:
> > >
> > >
> >
> {"responseHeader":{"status":400,"QTime":4,"params":{"fl":"City","q":"*:*","wt":"json","fq":"locm_place:\"IsWithin(POLYGON((16.762467717941604
> > 78.94775390625, 16.99375545289456 78.11279296875 , 17.31917640744285
> > 77.98095703125 , 17.80099604766698 78.72802734375)))
> > distErrPct=0\""}},"error":{"msg":"Couldn't parse shape
> > 'POLYGON((16.762467717941604 78.94775390625, 16.99375545289456
> > 78.11279296875 , 17.31917640744285 77.98095703125 , 17.80099604766698
> > 78.72802734375))' because: Unknown Shape definition
> > [POLYGON((16.762467717941604 78.94775390625, 16.99375545289456
> > 78.11279296875 , 17.31917640744285 77.98095703125 ,
> > 17.80099604...]","code":400}}
> > >
> > >
> > > I tried with this:
> > >
> > >
> >
> http://localhost:8983/solr/loopback/select?fl=City&wt=json&q=*:*&fq=geo:%22IsDisjointTo(POLYGON((16.762467717941604%2078.94775390625,%2016.99375545289456%2078.11279296875,17.31917640744285%2077.98095703125,17.80099604766698%2078.72802734375)))%22
> > >
> > > It is showing the result like:
> > >
> > >
> > >
> >
> {"responseHeader":{"status":400,"QTime":21,"params":{"fl":"City","q":"*:*","wt":"json","fq":"geo:\"IsDisjointTo(POLYGON((16.762467717941604
> > 78.94775390625, 16.99375545289456 78.11279296875,17.31917640744285
> > 77.98095703125,17.80099604766698
> > 78.72802734375)))\""}},"error":{"msg":"Couldn't parse shape
> > 'POLYGON((16.762467717941604 78.94775390625, 16.99375545289456
> > 78.11279296875,17.31917640744285 77.98095703125,17.80099604766698
> > 78.72802734375))' because: java.lang.IllegalArgumentException: points
> must
> > form a closed linestring","code":400}}
> > >
> > >
> > > Kindly tell me what I have to change/configure. I am attaching the json
> > file,schema.xml and a screenshot of Solr admin total result query.
> > >
> > >
> > > Thanks and regards,
> > >
> > > M Pradeep Chandra
> > >
> > > --
> > Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
> > LinkedIn: http://linkedin.com/in/davidwsmiley | Book:
> > http://www.solrenterprisesearchserver.com
> >
>
--
Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
LinkedIn: http://linkedin.com/in/davidwsmiley | Book:
http://www.solrenterprisesearchserver.com
Re: Question about Solr logs
On 3/5/2016 5:33 AM, Steven White wrote: > Thanks Shawn. > > To make sure I get this right, I see two methods on UpdateResponse class, > is getElapsedTime the client time and getQTime Solr's time? If so, than > getElapsedTime is how long my call was blocked, right? And getQTime will > have the value of 1174 (per the log of my example), right? Correct. getQTime just pulls out the QTime value calculated by Solr and sent back with the response. The client also calculates time for every request and puts it into the response object, getElapsedTime returns that value. Thanks, Shawn
Re: Disk Usage anomoly across shards/replicas
On 3/5/2016 6:09 AM, Robert Brown wrote: > I have an index with 65m docs spread across 2 shards, each with 1 > replica. > > The replica1 of shard2 is using up nearly double the amount of disk > space as the other shards/replicas. I *very* occasionally see some of the shards in my non-SolrCloud index show this behavior. Usually if I fully rebuild the index (which takes several hours), the problem will correct itself. I have no idea what causes it. I do not recall seeing it before upgrading from 3.5 to late 4.x. I do have some 5.x indexes ... I have not been running them for very long, so I do not know whether that version is having the same problem. Thanks, Shawn
Re: Disk Usage anomoly across shards/replicas
Thanks Shawn, I'm just about to remove that node and rebuild it, at least there won't be any actual downtime. On 05/03/16 14:44, Shawn Heisey wrote: On 3/5/2016 6:09 AM, Robert Brown wrote: I have an index with 65m docs spread across 2 shards, each with 1 replica. The replica1 of shard2 is using up nearly double the amount of disk space as the other shards/replicas. I *very* occasionally see some of the shards in my non-SolrCloud index show this behavior. Usually if I fully rebuild the index (which takes several hours), the problem will correct itself. I have no idea what causes it. I do not recall seeing it before upgrading from 3.5 to late 4.x. I do have some 5.x indexes ... I have not been running them for very long, so I do not know whether that version is having the same problem. Thanks, Shawn
Re: How to use geospatial search to find the locations within polygon
A Java NoClassDefFoundError of something in com.vividsolutions.jts means
you don't have JTS on your classpath. You should put the JTS jar file in
server/lib/. You can download it from maven-central. Here's a search for
JTS with the 1.14 version:
http://search.maven.org/#artifactdetails%7Ccom.vividsolutions%7Cjts-core%7C1.14.0%7Cjar
p.s. Nabble.com seems increasingly glitchy. I attempted to reply earlier
but Nabble returned a failure.
On Sat, Mar 5, 2016 at 1:39 AM Pradeep Chandra [via Lucene] <
ml-node+s472066n4261824...@n3.nabble.com> wrote:
> Thank u for your reply sirNow, I gave the ending point as starting
> point to close the polygon
>
> It is showing this error:
>
> {"error":{"msg":"java.lang.NoClassDefFoundError:
> com/vividsolutions/jts/geom/Lineal","trace":"java.lang.RuntimeException:
> java.lang.NoClassDefFoundError: com/vividsolutions/jts/geom/Lineal\n\tat
> org.apache.solr.servlet.HttpSolrCall.sendError(HttpSolrCall.java:618)\n\tat
> org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:477)\n\tat
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:210)\n\tat
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:179)\n\tat
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)\n\tat
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)\n\tat
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)\n\tat
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)\n\tat
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)\n\tat
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)\n\tat
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)\n\tat
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)\n\tat
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)\n\tat
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)\n\tat
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)\n\tat
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)\n\tat
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)\n\tat
> org.eclipse.jetty.server.Server.handle(Server.java:499)\n\tat
> org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)\n\tat
> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)\n\tat
> org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)\n\tat
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)\n\tat
> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)\n\tat
> java.lang.Thread.run(Thread.java:745)\nCaused by:
> java.lang.NoClassDefFoundError: com/vividsolutions/jts/geom/Lineal\n\tat
> com.spatial4j.core.shape.jts.JtsGeometry.(JtsGeometry.java:104)\n\tat
> com.spatial4j.core.context.jts.JtsSpatialContext.makeShape(JtsSpatialContext.java:203)\n\tat
> com.spatial4j.core.io.jts.JtsWktShapeParser.makeShapeFromGeometry(JtsWktShapeParser.java:252)\n\tat
> com.spatial4j.core.io.jts.JtsWktShapeParser.parsePolygonShape(JtsWktShapeParser.java:133)\n\tat
> com.spatial4j.core.io.jts.JtsWktShapeParser.parseShapeByType(JtsWktShapeParser.java:89)\n\tat
> com.spatial4j.core.io.WktShapeParser.parseIfSupported(WktShapeParser.java:114)\n\tat
> com.spatial4j.core.io.WktShapeParser.parse(WktShapeParser.java:86)\n\tat
> com.spatial4j.core.context.SpatialContext.readShapeFromWkt(SpatialContext.java:294)\n\tat
> org.apache.solr.schema.AbstractSpatialFieldType.parseShape(AbstractSpatialFieldType.java:231)\n\tat
> org.apache.solr.schema.AbstractSpatialFieldType$1.parseShape(AbstractSpatialFieldType.java:174)\n\tat
> org.apache.lucene.spatial.query.SpatialArgsParser.parse(SpatialArgsParser.java:90)\n\tat
> org.apache.solr.schema.AbstractSpatialFieldType.parseSpatialArgs(AbstractSpatialFieldType.java:317)\n\tat
> org.apache.solr.schema.AbstractSpatialFieldType.getFieldQuery(AbstractSpatialFieldType.java:312)\n\tat
> org.apache.solr.parser.SolrQueryParserBase.getFieldQuery(SolrQueryParserBase.java:757)\n\tat
> org.apache.solr.parser.SolrQueryParserBase.getFieldQuery(SolrQueryParserBase.java:383)\n\tat
> org.apache.solr.parser.SolrQueryParserBase.handleQuotedTerm(SolrQueryParserBase.java:560)\n\tat
> org.apache.solr.parser.QueryParser.Term(QueryParser.java:403)\n\tat
> org.apache.solr.parser.QueryParser.Clause(QueryParser.java:185)\n\tat
> org.apache.solr.parser.QueryParser.Query(QueryParser.java:107)\n\tat
> org.apache.solr.parser.QueryParser.TopLevelQuery(QueryParser.java:96)\n\tat
> org.apache.solr.parser.SolrQueryParserBase.parse(SolrQueryParserBase.java:151)\n\tat
> org.apache.solr.search.LuceneQParser.parse(LuceneQParser.java:50)\n\tat
> org.apache.solr.search.QParser.
Re: How to use geospatial search to find the locations within polygon
The doc does indeed say "JTS... It's a JAR file that you need to put on
Solr's classpath (but not via the standard solrconfig.xml mechanisms)", but
that is a little vague and nonspecific. It should probably be a labeled
section in the doc, like "Configuring JTS for Polygon Search", and have the
spatialContextFactory property (called a "setting" for some reason there
although elsewhere in the Solr doc XML attributes are referred to as
properties) point to that section. The "old" wiki has some more info, but
whether that is sufficient to fully configure JTS is unknown to me.
-- Jack Krupansky
On Sat, Mar 5, 2016 at 11:12 AM, david.w.smi...@gmail.com <
david.w.smi...@gmail.com> wrote:
> A Java NoClassDefFoundError of something in com.vividsolutions.jts means
> you don't have JTS on your classpath. You should put the JTS jar file in
> server/lib/. You can download it from maven-central. Here's a search for
> JTS with the 1.14 version:
>
> http://search.maven.org/#artifactdetails%7Ccom.vividsolutions%7Cjts-core%7C1.14.0%7Cjar
>
> p.s. Nabble.com seems increasingly glitchy. I attempted to reply earlier
> but Nabble returned a failure.
>
> On Sat, Mar 5, 2016 at 1:39 AM Pradeep Chandra [via Lucene] <
> ml-node+s472066n4261824...@n3.nabble.com> wrote:
>
> > Thank u for your reply sirNow, I gave the ending point as starting
> > point to close the polygon
> >
> > It is showing this error:
> >
> > {"error":{"msg":"java.lang.NoClassDefFoundError:
> > com/vividsolutions/jts/geom/Lineal","trace":"java.lang.RuntimeException:
> > java.lang.NoClassDefFoundError: com/vividsolutions/jts/geom/Lineal\n\tat
> >
> org.apache.solr.servlet.HttpSolrCall.sendError(HttpSolrCall.java:618)\n\tat
> > org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:477)\n\tat
> >
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:210)\n\tat
> >
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:179)\n\tat
> >
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)\n\tat
> >
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)\n\tat
> >
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)\n\tat
> >
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)\n\tat
> >
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)\n\tat
> >
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)\n\tat
> >
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)\n\tat
> >
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)\n\tat
> >
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)\n\tat
> >
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)\n\tat
> >
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)\n\tat
> >
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)\n\tat
> >
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)\n\tat
> > org.eclipse.jetty.server.Server.handle(Server.java:499)\n\tat
> > org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)\n\tat
> >
> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)\n\tat
> >
> org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)\n\tat
> >
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)\n\tat
> >
> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)\n\tat
> > java.lang.Thread.run(Thread.java:745)\nCaused by:
> > java.lang.NoClassDefFoundError: com/vividsolutions/jts/geom/Lineal\n\tat
> >
> com.spatial4j.core.shape.jts.JtsGeometry.(JtsGeometry.java:104)\n\tat
> >
> com.spatial4j.core.context.jts.JtsSpatialContext.makeShape(JtsSpatialContext.java:203)\n\tat
> >
> com.spatial4j.core.io.jts.JtsWktShapeParser.makeShapeFromGeometry(JtsWktShapeParser.java:252)\n\tat
> >
> com.spatial4j.core.io.jts.JtsWktShapeParser.parsePolygonShape(JtsWktShapeParser.java:133)\n\tat
> >
> com.spatial4j.core.io.jts.JtsWktShapeParser.parseShapeByType(JtsWktShapeParser.java:89)\n\tat
> >
> com.spatial4j.core.io.WktShapeParser.parseIfSupported(WktShapeParser.java:114)\n\tat
> > com.spatial4j.core.io.WktShapeParser.parse(WktShapeParser.java:86)\n\tat
> >
> com.spatial4j.core.context.SpatialContext.readShapeFromWkt(SpatialContext.java:294)\n\tat
> >
> org.apache.solr.schema.AbstractSpatialFieldType.parseShape(AbstractSpatialFieldType.java:231)\n\tat
> >
> org.apache.solr.schema.AbstractSpatialFieldType$1.parseShape(AbstractSpatialFieldType.java:174)\n\tat
> >
> org.apache.lucene.spatial.query.SpatialArgsParser.parse(SpatialArgsParser.java:90)\n\tat
> >
> org.apache.solr.schema.AbstractSpatialFieldType.parseSpatialArgs(AbstractSpatialFieldType.java:317)\n\tat
> >
> org
Re: Field exclusion from fl and hl.fl
I have yet to find any workaround so far.Still have to list out all the remaining fields one by one. Does anyone else has any suggestions? Regards, Edwin On 18 February 2016 at 17:07, Anil wrote: > I am looking for the same. please do let me know just in case you find > workaround. > > On 18 February 2016 at 14:18, Zheng Lin Edwin Yeo > wrote: > > > Hi, > > > > Would like to find out, is there already a way to exclude field from the > > Solr response. I did came across SOLR-3191 which was created about 4 > years > > ago, but could not find any workable solutions from there. > > > > As my collections can have more than 100 fields, and I would need to > return > > the majority of then except for one or two, so if there is a way to > exclude > > the fields would be good, if not I have to list all the remaining fields > > (which can be more than 100 for each collections). > > > > I am using Solr 5.4.0. > > > > Regards, > > Edwin > > >
Re: Field exclusion from fl and hl.fl
it used to support fl=*,-field Does that not work now? On Sat, Mar 5, 2016 at 7:37 PM, Zheng Lin Edwin Yeo wrote: > I have yet to find any workaround so far.Still have to list out all the > remaining fields one by one. > > Does anyone else has any suggestions? > > Regards, > Edwin > > > On 18 February 2016 at 17:07, Anil wrote: > > > I am looking for the same. please do let me know just in case you find > > workaround. > > > > On 18 February 2016 at 14:18, Zheng Lin Edwin Yeo > > wrote: > > > > > Hi, > > > > > > Would like to find out, is there already a way to exclude field from > the > > > Solr response. I did came across SOLR-3191 which was created about 4 > > years > > > ago, but could not find any workable solutions from there. > > > > > > As my collections can have more than 100 fields, and I would need to > > return > > > the majority of then except for one or two, so if there is a way to > > exclude > > > the fields would be good, if not I have to list all the remaining > fields > > > (which can be more than 100 for each collections). > > > > > > I am using Solr 5.4.0. > > > > > > Regards, > > > Edwin > > > > > > -- Bill Bell billnb...@gmail.com cell 720-256-8076
High Cpu sys usage
Hi We are using Solr Cloud 4.6 in our production for searching service since 2 years ago.And now it has 700GB in one cluster which is comprised of 3 machines with ssd. At beginning ,everything go well,while more and more business services interfered with our searching service .And a problem which we haunted with is just like a nightmare . That is the cpu sys usage is often growing up to over 10% even higher, and as a result the machine will hang down because system resources have be drained out.We have to restart the machine manually. We have done some google,and we also know the high cpu sys usage is due to system kernel.But we do not have further more information about the problem other than the solution. Is it perhaps because of the increase of the search concurrence? Do you guys have experiences like us.How do you solve it? Please any suggestion will be appreciated.
Re: Disk Usage anomoly across shards/replicas
Hi Robert, Within the shard directory there should be multiple directories - "tlog" "index." . Do you see multiple "index.*" directories in there for the shard which has more data on disk? On Sat, Mar 5, 2016 at 6:39 PM, Robert Brown wrote: > Hi, > > I have an index with 65m docs spread across 2 shards, each with 1 replica. > > The replica1 of shard2 is using up nearly double the amount of disk space > as the other shards/replicas. > > Could there be a reason/fix for this? > > > /home/s123/solr/data/de_shard1_replica1 = 72G > > numDocs:34,786,026 > maxDoc:45,825,444 > deletedDocs:11,039,418 > > > > /home/s123/solr/data/de_shard1_replica2 = 70G > > numDocs:34,786,026 > maxDoc:46,914,095 > deletedDocs:12,128,069 > > > > /home/s123/solr/data/de_shard2_replica1 = 138G > > numDocs:34,775,193 > maxDoc:45,409,362 > deletedDocs:10,634,169 > > > > /home/s123/solr/data/de_shard2_replica2 = 66G > > numDocs:34,775,193 > maxDoc:44,181,734 > deletedDocs:9,406,541 > > > > Thanks, > Rob > > > > > > -- Regards, Varun Thacker
