Re: DateDiff

2013-11-06 Thread Aloke Ghoshal
Hi Adam, With FunctionQuery (http://wiki.apache.org/solr/FunctionQuery) & DateMath ( http://lucene.apache.org/solr/4_5_1/solr-core/org/apache/solr/util/DateMathParser.html) - round to the day level, subtract & divide by milliseconds_in_a_day (86400K). ?q=*floor*(*div*(*sub*(*ms*(NOW/DAY),*ms*(NOW

Grouping on MLT results

2013-11-06 Thread vineeth mohan
Hi , I need to find similar documents to a document and also group the result based on a field say category. I could find the MLT handler and I could find the grouping feature. - MLT handler - http://wiki.apache.org/solr/MoreLikeThisHandler - Grouping - http://wiki.apache.org/solr/FieldCol

Re: get min-max prices as facets

2013-11-06 Thread PeterKerk
Thanks! I know how to fire a range query. However, what I want is to provide the visitor with a range filter. In this range filter the minimum and maximum value are already set to the lowest and highest price of the current resultset. e.g. I sell cars. My cheapest car is 1,000 and the most expensi

SOLRCloud - Small Index - Full Index Strategy

2013-11-06 Thread ade-b
Hi We are moving from running the Endeca search engine to SOLRcloud. We have a comparatively small index compared to a lot of companies (approx 150,000 records). To potentially keep things simple in the first release to the production environment we are considering running a full index every 15 m

Re: solr sort facets by name

2013-11-06 Thread PeterKerk
That works, thanks! -- View this message in context: http://lucene.472066.n3.nabble.com/solr-sort-facets-by-name-tp4099499p4099572.html Sent from the Solr - User mailing list archive at Nabble.com.

solr cloud bad replication

2013-11-06 Thread Emmanuel GOURAUD
Hi all, I have a solr cloud (solr 4.3.1) configuration that is composed of 8 instances: 4 shards with their replicas (solrcloud replication, not legacy one) Due to a wrong assignment by solrcloud(the leader shard and its replica was on the same physical server), i must set the shardId parameter

Data Import Handler

2013-11-06 Thread Ramesh
Hi Folks, Can anyone suggest me how can customize dataconfig.xml file I want to provide database details like( db_url,uname,password ) from my own properties file instead of dataconfig.xaml file

Question regarding Indexsize with Spatial4j Rectangulars

2013-11-06 Thread Dennis Reichelt
Hi, we are testing Solr and index a huge amount of files. We integrated a Spatial4j field which is only used to index rectangulars so we removed the JTS dependency. However we had some problems with this. At first Solr seems to get a GC OutOfMemory error which seems to be fixed with more memo

Group.ngroup query slower with docValues

2013-11-06 Thread vicky desai
Hi, I am planning to use the docValues feature of Solr. I have added "docValues= true" parameter to a few fields in my schema on which there is heavy faceting and grouping query involved. While I noticed a considerable improvement in faceting queries. I didnt get any improvements in grouping query

Re: Data Import Handler

2013-11-06 Thread Peter Keegan
I've done this by adding an attribute to the entity element (e.g. myconfig="myconfig.xml"), and reading it in the 'init' method with context.getResolvedEntityAttribute("myconfig"). Peter On Wed, Nov 6, 2013 at 8:25 AM, Ramesh wrote: > Hi Folks, > > > > Can anyone suggest me how can customize d

Re: Data Import Handler

2013-11-06 Thread Giovanni
I configured a data source in tomcat and referenced it by its jdbc name. So dev and production sites shares the same config file but uses different dbs I hope this helps > Il giorno 06/nov/2013, alle ore 13:25, "Ramesh" > ha scritto: > > Hi Folks, > > > > Can anyone suggest me how can cu

RE: Data Import Handler

2013-11-06 Thread Dyer, James
If you prepend the variable name with "dataimporter.request", you can include variables like these as request parameters: /dih?driver=some.driver.class&url=jdbc:url:something If you want to include these in solrcore.properties, you can additionally add each property to solrconfig.xml like thi

Limit single field length in solr response via the request url

2013-11-06 Thread PeterKerk
I'm requesting fields like so: http://localhost:8983/solr/test/select/?indent=on&facet=true&wt=json&start=0&rows=20&fl= id,title,description,pricerange However, the field description might be more than 4000 characters long, so I want to limit it to a maximum of 100 characters and then cut it off.

Re: Replication: slow first query after replication.

2013-11-06 Thread Shawn Heisey
On 11/5/2013 10:45 PM, Luis Cappa wrote: > I have seen that when disabling replication and executing queries the time > responses are good. Interesting... I can't ser the solution, then, because > slow replication tomes are needed to almost always get 'fresh' documents in > slaves to search by,

Re: solr cloud bad replication

2013-11-06 Thread Shawn Heisey
On 11/6/2013 5:52 AM, Emmanuel GOURAUD wrote: > I have a solr cloud (solr 4.3.1) configuration that is composed of 8 > instances: > > 4 shards with their replicas (solrcloud replication, not legacy one) > > Due to a wrong assignment by solrcloud(the leader shard and its replica was > on the sam

Re: Group.ngroup query slower with docValues

2013-11-06 Thread Shawn Heisey
On 11/6/2013 6:28 AM, vicky desai wrote: > I am planning to use the docValues feature of Solr. I have added "docValues= > true" parameter to a few fields in my schema on which there is heavy > faceting and grouping query involved. While I noticed a considerable > improvement in faceting queries. I

Re: solr cloud bad replication

2013-11-06 Thread Emmanuel GOURAUD
Hi shawn, I configure 2 solr instances per server. Concerning the file, i mean shard by shard (shard1 leader to shard 1 replica, shard2 leader to shard2 replica, etc...) and not shard2 leader to shard3 replica :) It seams that having a shardId forced make a difference (because it is the only

Creating a replica by copying index

2013-11-06 Thread michael.boom
Is it possible to create a replica of a shard (collection1_shard1_replica1), in SolrCloud, by copying the collection1_shard1_replica1/ directory to the new server, updating core.properties and restarting solr on that machine? Would this be faster than using the CoreAPI to create a new core and spe

Performance Question: 'facets.missing'

2013-11-06 Thread andres
I'm debating whether or not to set the 'facets.missing' parameter to true by default when faceting. What is the performance impact of setting 'facets.missing' to true? -- View this message in context: http://lucene.472066.n3.nabble.com/Performance-Question-facets-missing-tp4099602.html Sent fr

Re: Performance Question: 'facets.missing'

2013-11-06 Thread Yonik Seeley
On Wed, Nov 6, 2013 at 12:07 PM, andres wrote: > I'm debating whether or not to set the 'facets.missing' parameter to true by > default when faceting. What is the performance impact of setting > 'facets.missing' to true? It really depends on the faceting method. For some faceting methods (like e

Re: get min-max prices as facets

2013-11-06 Thread Raymond Wiker
One way of getting the min/max values is to fire off two additional queries, using the current query and filter params, a field list ("fl") containing just the field you want to find min/max for, asking for a single hit (rows=1), and sorting by that field - asc in one query, desc in the other. T

Re: geo/spatial search performance comparison using different methods

2013-11-06 Thread Smiley, David W.
Hi Kuro, I don't know of any benchmarks featuring distance-sort performance. Presumably you are using SOLR-2155 because you have multi-valued spatial fields? If so, LatLonType is not an option. SOLR-2155 sorting performance is *probably* about the same as the equivalent in Solr 4 RPT. If you ac

Re: get min-max prices as facets

2013-11-06 Thread PeterKerk
Hi Raymond, Thanks. This caching of the main query you mention, is that done automatically or do I have to do something for that? -- View this message in context: http://lucene.472066.n3.nabble.com/get-min-max-prices-as-facets-tp4099501p4099613.html Sent from the Solr - User mailing list archi

Re: Question regarding Indexsize with Spatial4j Rectangulars

2013-11-06 Thread Smiley, David W.
Hi Dennis, I would not expect the index growth to be quite linear as the number of shapes grows, but nonetheless it may be significant. Indexing non-point shapes will index more term data than it ideally should: LUCENE-4942 I need to find the time/priority to do it. Probably within the next cou

Re: solr 4.3 solrj generating search terms that return no results

2013-11-06 Thread dboychuck
Thanks Shawn! That makes sense now. I appreciate the response. -- View this message in context: http://lucene.472066.n3.nabble.com/solr-4-3-solrj-generating-search-terms-that-return-no-results-tp4077137p4099615.html Sent from the Solr - User mailing list archive at Nabble.com.

eDisMax and Boolean operator case-sensitivity

2013-11-06 Thread Tom Mortimer
Hi, I'm using eDisMax query parser, and need to support Boolean operators AND and OR. It seems from testing that these are *not* case sensitive, e.g. setting mm to 0, "oscar AND wilde" returns the same results as "oscar and wilde" (15 hits) while "oscar foo wilde" returns the same results as "osca

Re: unable to load core after cluster restart

2013-11-06 Thread kaustubh147
Hi All, I have further investigated the difference between both the environments. We have JDK 1.6.0_17 (VM 14.3-b01)on UAT and JDK 1.6.0_33 (VM 20.8-b03)on QA1. Can it be the reason behind this error? Is there a recommended jdk version for SolrCloud ? Thanks, Kaustubh -- View this message i

Re: Example of join using Solr/Lucene

2013-11-06 Thread Alvaro Cabrerizo
You can check these links: http://wiki.apache.org/solr/Join http://stackoverflow.com/questions/12665797/is-solr-4-0-capable-of-using-join-for-multiple-core Regards. On Wed, Nov 6, 2013 at 1:00 AM, Tech Id wrote: > Hi Alvaro, > > Could you please point me to some link from where I can see ho

Re: get min-max prices as facets

2013-11-06 Thread Raymond Wiker
As far as I understand, the filter terms (fq parameters) should be automatically cached. On 06 Nov 2013, at 19:08 , PeterKerk wrote: > Hi Raymond, > > Thanks. > This caching of the main query you mention, is that done automatically or do > I have to do something for that? > > > > -- > View

Re: eDisMax and Boolean operator case-sensitivity

2013-11-06 Thread Shawn Heisey
On 11/6/2013 11:46 AM, Tom Mortimer wrote: I'm using eDisMax query parser, and need to support Boolean operators AND and OR. It seems from testing that these are *not* case sensitive, e.g. setting mm to 0, "oscar AND wilde" returns the same results as "oscar and wilde" (15 hits) while "oscar foo

Re: unable to load core after cluster restart

2013-11-06 Thread Shawn Heisey
On 11/6/2013 11:53 AM, kaustubh147 wrote: Hi All, I have further investigated the difference between both the environments. We have JDK 1.6.0_17 (VM 14.3-b01)on UAT and JDK 1.6.0_33 (VM 20.8-b03)on QA1. Can it be the reason behind this error? Is there a recommended jdk version for SolrCloud ?

Re: eDisMax and Boolean operator case-sensitivity

2013-11-06 Thread Tom Mortimer
Oh, good grief - I was just reading that page, how did I miss that? *derp* Thanks Shawn!!! Tom On 6 November 2013 18:59, Shawn Heisey wrote: > On 11/6/2013 11:46 AM, Tom Mortimer wrote: > >> I'm using eDisMax query parser, and need to support Boolean operators AND >> and OR. It seems from tes

New to Solr usage - Issue with SolrConfig.xml loading

2013-11-06 Thread Kiran Bandaru
Hi, I am trying to setup Solr in WebSphere 7.1 version with RAD 8. Downloaded Solr 4.5.1 Used below link to build EAR file with Solr WAR to make it work in WebSphere 7.1 version. https://wiki.base22.com/display/btg/How+to+install+and+configure+Solr+v4.3.1+on+IBM+WebSphere+Application+Server+v8

Re: unable to load core after cluster restart

2013-11-06 Thread kaustubh147
Hi, Here is my solr.xml ${host:} 28081 /solr IP1:2181,IP2:2181,IP3:2181/mysolr 15000 ${genericCoreNodeNames:true} ${socketTimeout:0} ${connTimeout:0} --

Re: geo/spatial search performance comparison using different methods

2013-11-06 Thread T. Kuro Kurosaka
Thank you, David. I believe the field doesn't need to be multivalued. Can you give me some idea how much query-time performance gain we can expect by switching to LatLonType from Solr-2155? On 11/06/2013 09:56 AM, Smiley, David W. wrote: Hi Kuro, I don't know of any benchmarks featuring distanc

Re: Example of join using Solr/Lucene

2013-11-06 Thread Alvaro Cabrerizo
For example, a dummy schema (if we index the former data in separate documents) should look like this: ... ...

Re: Limit single field length in solr response via the request url

2013-11-06 Thread Erick Erickson
Here are some options: http://grokbase.com/t/lucene/solr-user/10c3t860zn/limit-number-of-characters-returned Best, Erick On Wed, Nov 6, 2013 at 10:58 AM, PeterKerk wrote: > I'm requesting fields like so: > > > http://localhost:8983/solr/test/select/?indent=on&facet=true&wt=json&start=0&rows=2

Re: Creating a replica by copying index

2013-11-06 Thread Erick Erickson
bq: Would this be faster than using the CoreAPI to create a new core and specifying the collection and shard? Not really. Under the covers, SolrCloud uses the old-style replication process if there are a large number of changes in the index. Basically, it's copying the files from the leader to the

Re: unable to load core after cluster restart

2013-11-06 Thread Chris Geeringh
--- In the case of Java 7, 1.7.0_40 and 1.7.0_45 have known problems with Lucene/Solr. Shawn, this is interesting. What are the problems, where are the documented? On 6 November 2013 20:18, kaustubh147 wrote: > Hi, > > Here is my > solr.xml > > > > > > > ${host:} > 28081 > /sol

Inconsistent number of hits returned by two solr instances (from the same index!)

2013-11-06 Thread Roman Chyla
Hello, We have two solr searchers/instances (read-only). They read the same index, but they did not return the same #hits for a particular query Log is below, but to summarize: first server always returns 576 hits, the second server returns: 440, 440, 576, 576... These are just few seconds apart

Re: unable to load core after cluster restart

2013-11-06 Thread Shawn Heisey
On 11/6/2013 2:03 PM, Chris Geeringh wrote: --- In the case of Java 7, 1.7.0_40 and 1.7.0_45 have known problems with Lucene/Solr. Shawn, this is interesting. What are the problems, where are the documented? https://issues.apache.org/jira/browse/LUCENE-5212 The issue comments say that one of

Re: geo/spatial search performance comparison using different methods

2013-11-06 Thread Smiley, David W.
Any guesses would be wild ones, but I'm pretty sure you'll notice it, assuming the result size isn't trivially small. Also, LatLonType will use much less memory and be more real-time search friendly (i.e. Commit warming will be faster, assuming you do warming queries as everyone should do). To be

Re: unable to load core after cluster restart

2013-11-06 Thread Shawn Heisey
On 11/6/2013 1:18 PM, kaustubh147 wrote: I am attaching a small log file with debug option enabled.. log shows following process 1. first start of solr cluster 2. create collection "collection1" 3. shutdown cluster 4. start cluster again error is in only the 4th step...and it is coming after so

Re: Inconsistent number of hits returned by two solr instances (from the same index!)

2013-11-06 Thread Michael Della Bitta
Does the header in the response indicate you're getting partialResults? http://help.websolr.com/kb/common-problems/why-am-i-getting-partial-results Michael Della Bitta Applications Developer o: +1 646 532 3062 | c: +1 917 477 7906 appinions inc. “The Science of Influence Marketing” 18 East

Solr cloud : Changing properties of alreadt loaded collection

2013-11-06 Thread sriram
Hi, Is it possible to change the properties of collection after it has been created in solr cloud? I want to change the number of replicas, number of shards, number of shards per node for an already existing collection. I changed those properties while creating a collection using the steps mentio

Re: Example of join using Solr/Lucene

2013-11-06 Thread Alvaro Cabrerizo
You can also go further and ask: Give me all the people that drive a toyota (company->car->people): *q=*:*&fq=*:* AND _query_:"{!join to=id from=ownerId}id:* AND _query_:{!join to=companyId from=id}name:toyo*"* If we divide this query in fragments we got: - *q=*:** which is restriction ove

Re: Inconsistent number of hits returned by two solr instances (from the same index!)

2013-11-06 Thread Roman Chyla
No, and I should add that this query was not against shards, just a one single index (and we dont use timeouts). --roman On Wed, Nov 6, 2013 at 5:28 PM, Michael Della Bitta < michael.della.bi...@appinions.com> wrote: > Does the header in the response indicate you're getting partialResults? > >

Re: unable to load core after cluster restart

2013-11-06 Thread kaustubh147
Hi, I tried "simple" lock type too. It is throwing similar error... Caused by: org.apache.lucene.store.LockObtainFailedException: Lock obtain timed out: SimpleFSLock@/mnt/emc/app_name/data-prod-refresh/SolrCloud/SolrHome1/solr/collection1_shard1_replica2/data/index/write.lock at org.apa

Re: Inconsistent number of hits returned by two solr instances (from the same index!)

2013-11-06 Thread Michael Della Bitta
Wow, that's pretty weird. Have you tried turning logging down to debug and seeing if anything interesting shakes out? Michael Della Bitta Applications Developer o: +1 646 532 3062 | c: +1 917 477 7906 appinions inc. “The Science of Influence Marketing” 18 East 41st Street New York, NY 10017

Jetty 9?

2013-11-06 Thread William Bell
When are we moving Solr to Jetty 9? -- Bill Bell billnb...@gmail.com cell 720-256-8076

Re: Group.ngroup query slower with docValues

2013-11-06 Thread vicky desai
Hi Shawn, Thanks for the reply. But the issue you pointed out talk about general performance issue of ngroups. However what i noticed is that after using docValues the performance of group.ngroups had degraded about 2-3 times. This is stopping me from using docValues which otherwise in case of fac

Multi-core support for indexing multiple servers

2013-11-06 Thread Rob Veliz
Trying to find specific information to support the following scenario: - I have one site running on one server with marketing content, blog, etc. I want to index. - I have another site running on Magento on a different server with ecommerce content (products). - Both servers live in completely dif

Re: Multi-core support for indexing multiple servers

2013-11-06 Thread Shawn Heisey
On 11/6/2013 11:38 PM, Rob Veliz wrote: > Trying to find specific information to support the following scenario: > > - I have one site running on one server with marketing content, blog, etc. > I want to index. > - I have another site running on Magento on a different server with > ecommerce conte

Re: Multi-core support for indexing multiple servers

2013-11-06 Thread Rob Veliz
Great feedback, thanks. So the multi-core structure I have then is a single Solr server set up, essentially hosted by one domain owner (but to be used by both). My question is how does that Solr server connect to the 2 Web applications to create the 1 master index (to be used when searching on ei

Re: Multi-core support for indexing multiple servers

2013-11-06 Thread Shawn Heisey
On 11/7/2013 12:07 AM, Rob Veliz wrote: > Great feedback, thanks. So the multi-core structure I have then is a > single Solr server set up, essentially hosted by one domain owner (but to > be used by both). My question is how does that Solr server connect to the > 2 Web applications to create the

Help to find BaseTokenFilterFactory to write a Custom TokenFilter

2013-11-06 Thread Dileepa Jayakody
Hi All, I am writing a custom TokenFilter to post a token value to Apache Stanbol for enhancement. In this Custom TokenFilter I'm trying to retrieve the response from Stanbol and index it as a new document in Solr. I'm following [1] to write a custom filter, but I'm having trouble locating BaseTo

Re: Multi-core support for indexing multiple servers

2013-11-06 Thread Rob Veliz
I've been reading about Solarium--definitely useful. Could you elaborate here: If you are planning a single master index, that's not multicore. Having more than one document type in a single index is possible, they just have to overlap on at least one field - whatever field is the uniqueKey for

Re: get min-max prices as facets

2013-11-06 Thread manju16832003
Hey Peter, Now I understood your requirement :-). I come across this http://wiki.apache.org/solr/StatsComponent have a look if it helps you :-). -- View this message in context: http://lucene.472066.n3.nabble.com/get-min-max-prices-as-facets-tp4099501p4099737.html Sent from the Solr - User ma

How to set default values for int fields

2013-11-06 Thread manju16832003
How do I set default value for int fields ex While indexing lets say if I have not set the value for mileage, and solr tries to assign default value and it fails. Because *mileage* is of type int not string. It throws following exception 3656198 [http-bio-8080-exec-4] ERROR org.apache.solr.serv