Re: best way to contribute solr??

2014-04-09 Thread Aman Tandon
I will also try to do in the perl as well, this is going to be something great, i am excited :D Thanks a ton!! Thanks Aman Tandon On Thu, Apr 10, 2014 at 12:22 PM, Aman Tandon wrote: > Thank you so much sir :) > > Can i try in java as well? > > Thanks > Aman Tandon > > > On Thu, Apr 10, 2014 a

Re: best way to contribute solr??

2014-04-09 Thread Aman Tandon
Thank you so much sir :) Can i try in java as well? Thanks Aman Tandon On Thu, Apr 10, 2014 at 12:15 PM, Alexandre Rafalovitch wrote: > Great, Solr + Perl + Geospatial. > > There are two Perl clients for Solr listed on the Wiki: > http://wiki.apache.org/solr/IntegratingSolr . Are there any mor

Re: Tomcat creates a thread for each SOLR core

2014-04-09 Thread Alexandre Rafalovitch
Are you using all those cores at once? If not, there is a recent settings to allow solr to load cores on demand. If you are using them all, perhaps you need to look into splitting them to different machines (horizontal scaling). What about your caches? How many additional structures you have conf

Re: Tomcat creates a thread for each SOLR core

2014-04-09 Thread Aman Tandon
I guess this is definitely due to the firstsearcher defined in solrconfig.xml, you must make some tweaks in that I hope it will help. We are using the same typo which you just mentioned here but we are using the indexing server separately and replicating data to our other two server so that it won'

Re: best way to contribute solr??

2014-04-09 Thread Alexandre Rafalovitch
Great, Solr + Perl + Geospatial. There are two Perl clients for Solr listed on the Wiki: http://wiki.apache.org/solr/IntegratingSolr . Are there any more? If yes, add them to the Wiki (need to ask permission to edit Wiki). Are those two listed clients dead or alive? Do they work with Solr 4.7.1?

Re: Ranking code

2014-04-09 Thread Aman Tandon
For the better analysis for ranking of documents, you should need to query the index with these extra parameters(in bold) eg...whole_query*&debug=true&wt=xml.* Copy that xml and and paste it to http://explain.solr.pl/ you can then easily find out the ranking alalysis in the forms of the pie charts

Tomcat creates a thread for each SOLR core

2014-04-09 Thread Atanas Atanasov
Hi, guys, I need some help. After updating to SOLR 4.4 the tomcat process is consuming about 2GBs of memory, the CPU usage is about 40% after the start for about 10 minutes. However, the bigger problem is, I have about 1000 cores and seems that for each core a thread is created. The process has mo

Re: best way to contribute solr??

2014-04-09 Thread Aman Tandon
Okay sir i will mail to solr-user only, I am feeling so thankful to you for all you help, i am java developer with a good knowledge of perl, working on solr, actually just started working on solr for the geospatial search(not using JTS) only, To be very frank I learned about faceting from Mr Yonik'

Re: Facet search and growing memory usage

2014-04-09 Thread Toke Eskildsen
On Thu, 2014-04-10 at 04:23 +0200, Damien Kamerman wrote: > What I have found with Solr 4.6.0 to 4.7.1 is that memory usage continues > to grow with facet queries. It allocates (potentially significant) temporary structures, yes. > Then I tried to determine a safe limit at which the search would

Re: best way to contribute solr??

2014-04-09 Thread Alexandre Rafalovitch
Hi Aman, Nice of you to want to help. Let's keep the discussion in the user mailing list as opposed to the developer one (most of the people are on both). What is your skill set? Are you familiar with particular languages? If so, the easiest way to contribute would be the following: 1) Find all t

best way to contribute solr??

2014-04-09 Thread Aman Tandon
Can anybody please explain me that how should i start contributing to solr, i am novice here as well in this technology as well, but i am learning solr day by day. So how should i start ? Thanks Aman Tandon

RE: No route to host

2014-04-09 Thread Suresh Soundararajan
You are running the solr in the built in jetty server or tomcat ? First check http://:8080/ is working. If that working then check with http://:8080/solr, which will display the solr admin page. From this page you can check the collection1 core is available or not and also you can view the log

Re: boost results within 250km

2014-04-09 Thread Aman Tandon
Thanks Bill, Actually i have to return the distances for every results that how far it is from the searched location thats why i am calculating the distance using geodist separately and then using that distance for passing to fl as well as in boosting for 250km. *.&distance=geodist($lat,$lon,l

Re: WordDelimiterFilter issue and suggested fix

2014-04-09 Thread Erick Erickson
This really doesn't seem necessary. What is your actual field definition? I also think your cut/paste is messed up, this is wrong: bdeut_nullablevaluebdeutnullablevaluebdeutnullablevalue the vertical bars in the admin/analysis page are quite important here. Either the cut/paste is doing weird thin

Re: boost results within 250km

2014-04-09 Thread Bill Bell
Just take geodist and use the map function and send to bf or boost Bill Bell Sent from mobile > On Apr 9, 2014, at 8:26 AM, Erick Erickson wrote: > > Why do you want to do this? This sounds like an XY problem, you're > asking how to do something specific without explaining why you care, > per

Facet search and growing memory usage

2014-04-09 Thread Damien Kamerman
Hi All, What I have found with Solr 4.6.0 to 4.7.1 is that memory usage continues to grow with facet queries. Originally I saw the issue with 40 facets over 60 collections (distributed search). Memory usage would spike and solr would become unresponsive like https://issues.apache.org/jira/browse/

WordDelimiterFilter issue and suggested fix

2014-04-09 Thread Anirudha Jadhav
indexed term: bdeut_NullableValue query term 1 : bdeut_nullablevalue (lowercase) query term 2 : nullablevalue (lowercase) current field type definition in order: Whitespace Tokenizer Word Delimiter Filter Lowercase filter current analysis output: [INDEX]schema_field WT text bdeut_NullableValue

Re: Were changes made to facetting on multivalued fields recently?

2014-04-09 Thread Shawn Heisey
On 4/9/2014 2:15 PM, Erick Erickson wrote: Right, but the response in the doc when you make a request is almost, but not quite totally, unrelated to how facet values are tallied. It's all about what tokens are actually in your index, which you can see in the "schema browser"... Supplement to wh

Solr 4.3.1 snapshot ate up disk space

2014-04-09 Thread Darrell Burgan
Hello, just a quick question about the expected behavior of the SnapShooter. We're running Solr 4.3.1 in a SolrCloud configuration, with two separate virtual machines running Solr and three Zookeepers in various places. Our search index is about 70GB in size. Today I took a snapshot of just one

Re: MapReduceIndexerTool does not respect Lucene version in solrconfig Was: converting 4.7 index to 4.3.1

2014-04-09 Thread Wolfgang Hoschek
There is a current limitation in that the code doesn’t actually look into solrconfig.xml for the version. We should fix this, indeed. See https://github.com/apache/lucene-solr/blob/trunk/solr/contrib/map-reduce/src/java/org/apache/solr/hadoop/TreeMergeOutputFormat.java#L100-101 Wolfgang. On Apr

Highlighting bug with edgegrams

2014-04-09 Thread Software Dev
In 3.5.0 we have the following. If we searched for "c" with highlighting enabled we would get back results such as: cdat crocdile cool beans But in the latest Solr (4.7.1) we get the full words highlighted back.

Re: No route to host

2014-04-09 Thread Rallavagu
Sorry. I should have mentioned earlier. I have removed the original host name on purpose. Thanks. On 4/9/14, 1:42 PM, Siegfried Goeschl wrote: Hi folks, the URL looks wrong (misconfigured) http://:8080/solr/collection1 Cheers, Siegfried Goeschl On 09 Apr 2014, at 14:28, Rallavagu wrote:

Re: No route to host

2014-04-09 Thread Siegfried Goeschl
Hi folks, the URL looks wrong (misconfigured) http://:8080/solr/collection1 Cheers, Siegfried Goeschl On 09 Apr 2014, at 14:28, Rallavagu wrote: > All, > > I see the following error in the log file. The host that it is trying to find > is itself. Wondering if anybody experienced this befor

Re: No route to host

2014-04-09 Thread Rallavagu
Agreed. But, it is failing to find route to itself. Weird. On 4/9/14, 1:34 PM, Greg Walters wrote: This doesn't looks like a solr-specfic issue. Be sure to check your routes and your firewall. I've seen firewalls refuse packets and return a special flag that results in a no route to host error

Re: No route to host

2014-04-09 Thread Greg Walters
This doesn't looks like a solr-specfic issue. Be sure to check your routes and your firewall. I've seen firewalls refuse packets and return a special flag that results in a no route to host error. Thanks, Greg On Apr 9, 2014, at 3:28 PM, Rallavagu wrote: > All, > > I see the following error

No route to host

2014-04-09 Thread Rallavagu
All, I see the following error in the log file. The host that it is trying to find is itself. Wondering if anybody experienced this before or any other info would helpful. Thanks. 709703139 [http-bio-8080-exec-43] ERROR org.apache.solr.update.SolrCmdDistributor – org.apache.solr.client.sol

Re: create heat maps

2014-04-09 Thread Aman Tandon
Thanks sir, i read your post but i am confused how can i visualize the heatmap in the interface? By the way i am waiting for your book which will launch in july, all the best for that :) On Wed, Apr 9, 2014 at 6:06 PM, david.w.smi...@gmail.com < david.w.smi...@gmail.com> wrote: > No movement her

Re: boost results within 250km

2014-04-09 Thread Aman Tandon
Thanks Erik, the problem was that i was boost the results closer to the city via geodist, but i also have to boost cities within 250km extra as well. i did it by using the map(distance,0,250,10,1) along with boosting cities via geodist query, problem solved thanks for replying. On Thu, Apr 10, 2

Re: boost results within 250km

2014-04-09 Thread Aman Tandon
Thanks Erik, the problem was that i was boost the results closer to the city via geodist, but i also have to boost cities within 250km extra as well. i did it by using the map(distance,0,250,10,1) along with boosting cities via geodist query, problem solved thanks for replying. On Wed, Apr 9, 20

multiple analyzers for one field

2014-04-09 Thread Michael Sokolov
I think I would like to do something like copyfield from a bunch of fields into a single field, but with different analysis for each source, and I'm pretty sure that's not a thing. Is there some alternate way to accomplish my goal? Which is to have a suggester that suggests words from my full

Re: Were changes made to facetting on multivalued fields recently?

2014-04-09 Thread Erick Erickson
Right, but the response in the doc when you make a request is almost, but not quite totally, unrelated to how facet values are tallied. It's all about what tokens are actually in your index, which you can see in the "schema browser"... Let me know what the results are Erick On Wed, Apr 9, 2014 at

Re: Commit Within and /update/extract handler

2014-04-09 Thread Jamie Johnson
Thanks Shawn, I appreciate the information. On Wed, Apr 9, 2014 at 10:27 AM, Shawn Heisey wrote: > On 4/9/2014 7:47 AM, Jamie Johnson wrote: > > This is being triggered by adding the commitWithin param to > > ContentStreamUpdateRequest (request.setCommitWithin(1);). My > > configuration ha

solr-user@lucene.apache.org

2014-04-09 Thread Furkan KAMACI
Hi; I have developed a Search API for such kind of cases and generate Solr query within that API. I have also have my own query syntax. When a search query comes into my API I generate query and does not allow for something like *:*. On the other hand I escape query string and append the appropri

RE: Were changes made to facetting on multivalued fields recently?

2014-04-09 Thread Jean-Sebastien Vachon
Thanks Erick I will check this as soon as I can. In the meantime, here is a sample query and how it looks in our index. It looks good to me (at least that what is showing up as well in our other and older indexes) http://10.0.5.227:8201/solr/Current/select?q=*:*&fl=ad_job_type_id&fq=ad_job_type

Re: Were changes made to facetting on multivalued fields recently?

2014-04-09 Thread Erick Erickson
That is...um...very strange. It looks to me like you have somehow indexed a bunch of new values. I'm guessing here, but it's suspicious that you have a value "4,1" should that have been indexed as "4" and "1" as separate tokens? So here's what I'd do 1> take a look at the solr/admin/schema browser

Were changes made to facetting on multivalued fields recently?

2014-04-09 Thread Jean-Sebastien Vachon
Hi All, We just discovered that the response from Solr (4.7.1) when faceting on one of our multi-valued fields has changed considerably. In the past (4.6.1 and prior versions as well) we used to have something like this: (there are 7 possible values for this attribute) 11454652 11387070 20

Re: Boosting vs. sorting

2014-04-09 Thread Ugo Matrangolo
Hi, What I meant for "not working properly" is that while the partition still works the sorting inside the partitions (by position in the sale) is not. Interesting enough it stopped to work only for all the items belonging to the sold out items. Ugo On Apr 9, 2014 6:07 PM, "Erick Erickson" wrote

Re: Boosting vs. sorting

2014-04-09 Thread Erick Erickson
What does "not working properly" mean? An example would help. What does adding &debug=query show in the two (debug=all in 3.x)? My suspicion is that the second boost definition may be replacing the first in 4.x, but that's only a guess. Best, Erick On Wed, Apr 9, 2014 at 9:32 AM, Ugo Matrangolo

Boosting vs. sorting

2014-04-09 Thread Ugo Matrangolo
Hi, we are using SOLR to power a sale listing page where each product has an availability and a position in the sale. We would like SOLR to partition the products based on their sold_out={true|false} state and to sort the two partitions by product position. We started with appending to the query

Re: Solr relevancy tuning

2014-04-09 Thread Doug Turnbull
Hey Giovanni, nice to meet you. I'm the person that did the Test Driven Relevancy talk. We've got a product Quepid (http://quepid.com) that lets you gather good/bad results for queries and do a sort of test driven development against search relevancy. Sounds similar to your existing scripted appro

solr-user@lucene.apache.org

2014-04-09 Thread Erick Erickson
Note that when I mentioned "filter these characters out" I had something like PatternReplaceCharFilterFactory or LowerCaseTokenizer in mind rather than you having to do it manually. Doesn't help figuring out what to escape on the URL though. Best, Erick On Wed, Apr 9, 2014 at 8:05 AM, Shawn Heise

Re: WordDelimiterFilterFactory splits up hyphenated terms although splitOnNumerics, generateWordParts and generateNumberParts are set to 0 (false)

2014-04-09 Thread Erick Erickson
In think you're confusing initial tokenization with post-tokenization operations. From here (and I admit it's a little opaque): http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters#solr.WordDelimiterFilterFactory bq: Splits words into subwords and performs optional transformations on subwo

solr-user@lucene.apache.org

2014-04-09 Thread Shawn Heisey
On 4/9/2014 8:39 AM, Philip Durbin wrote: > Filtering out special characters sounds like a good idea, or possibly > escaping some of them. I definitely want to avoid brittleness. > > Right now I'm passing the query relatively "as is" which means users > can type "title:foo" to find documents that

AW: WordDelimiterFilterFactory splits up hyphenated terms although splitOnNumerics, generateWordParts and generateNumberParts are set to 0 (false)

2014-04-09 Thread Malte Hübner
> -Ursprüngliche Nachricht- > Von: Erick Erickson [mailto:erickerick...@gmail.com] > Gesendet: Samstag, 29. März 2014 16:09 > An: solr-user@lucene.apache.org > Betreff: Re: WordDelimiterFilterFactory splits up hyphenated terms although > splitOnNumerics, generateWordParts and generateNumber

Re: Solr relevancy tuning

2014-04-09 Thread Giovanni Bricconi
Thank you for the links. The book is really useful, I will definitively have to spend some time reformatting the logs to to access number of result founds, session id and much more. I'm also quite happy that my test cases produces similar results to the precision reports shown at the beginning of

solr-user@lucene.apache.org

2014-04-09 Thread Philip Durbin
Filtering out special characters sounds like a good idea, or possibly escaping some of them. I definitely want to avoid brittleness. Right now I'm passing the query relatively "as is" which means users can type "title:foo" to find documents that have "foo" in the "title" field. But a query for jus

Re: Solr join and lucene scoring

2014-04-09 Thread Alvaro Cabrerizo
Right, The examples posted before had a trick (both weren't pure join queries thats why the results had score). Having a pure join query (q={!join from=id to=id}printer) gives all the scores as 1 as Moritz said). Regards. On Wed, Apr 9, 2014 at 3:13 PM, Mikhail Khludnev wrote: > On Wed

Re: Commit Within and /update/extract handler

2014-04-09 Thread Shawn Heisey
On 4/9/2014 7:47 AM, Jamie Johnson wrote: > This is being triggered by adding the commitWithin param to > ContentStreamUpdateRequest (request.setCommitWithin(1);). My > configuration has autoCommit max time of 15s and openSearcher set to false. > I'm assuming that changing openSeracher to tru

Re: boost results within 250km

2014-04-09 Thread Erick Erickson
Why do you want to do this? This sounds like an XY problem, you're asking how to do something specific without explaining why you care, perhaps there are other ways to do this. Best, Erick On Tue, Apr 8, 2014 at 11:30 PM, Aman Tandon wrote: > How can i gave the more boost to the results within 2

Re: Commit Within and /update/extract handler

2014-04-09 Thread Jamie Johnson
This is being triggered by adding the commitWithin param to ContentStreamUpdateRequest (request.setCommitWithin(1);). My configuration has autoCommit max time of 15s and openSearcher set to false. I'm assuming that changing openSeracher to true should address this, and adding the softCommit =

Re: Solr join and lucene scoring

2014-04-09 Thread Mikhail Khludnev
On Wed, Apr 9, 2014 at 1:33 PM, wrote: > Hello Mikhail, > > thx for the clarification. I'm a little bit confused by the answer of > Alvaro, but my own tests didn't result in a proper score, so I think you're > right and it's still not implemented. > > What do you mean with the "impedance between

Re: create heat maps

2014-04-09 Thread david.w.smi...@gmail.com
No movement here lately but this is something to read if you're venturing down this path: http://wiki.apache.org/solr/SpatialClustering On Wed, Apr 9, 2014 at 1:19 AM, Aman Tandon wrote: > How can we create the heat maps using solr spatial search? > > Thanks > Aman Tandon >

Re: Solr relevancy tuning

2014-04-09 Thread Ahmet Arslan
Hi Giovanni, Here are some relevant pointers :  http://www.lucenerevolution.org/2013/Test-Driven-Relevancy-How-to-Work-with-Content-Experts-to-Optimize-and-Maintain-Search-Relevancy   http://rosenfeldmedia.com/books/search-analytics/  http://www.sematext.com/search-analytics/index.html Ahmet

Re: Solr join and lucene scoring

2014-04-09 Thread mm
Hello Mikhail, thx for the clarification. I'm a little bit confused by the answer of Alvaro, but my own tests didn't result in a proper score, so I think you're right and it's still not implemented. What do you mean with the "impedance between Lucene and Solr"? Why isn't the possibility of

Solr relevancy tuning

2014-04-09 Thread Giovanni Bricconi
It is about one year I'm working on an e-commerce site, and unfortunately I have no "information retrieval" background, so probably I am missing some important practices about relevance tuning and search engines. During this period I had to fix many "bugs" about bad search results, which I have sol

Re: How are you handling "killer queries" with solr?

2014-04-09 Thread Sohan Kalsariya
So what is the issue and the Solution? Do i need to change any configurations in my solrconfig.xml ? So I have attached my solrconfig.xml have a look. On Wed, Apr 9, 2014 at 1:32 AM, Toke Eskildsen wrote: > Shawn Heisey [s...@elyograg.org] wrote: > > Are you using the Jetty that comes with Solr,

Re: create heat maps

2014-04-09 Thread Aman Tandon
no i don't have any idea of it. On Wed, Apr 9, 2014 at 1:08 PM, Mikhail Khludnev wrote: > Hello, > Have you checked pivot facets? > 09.04.2014 9:19 пользователь "Aman Tandon" > написал: > > > How can we create the heat maps using solr spatial search? > > > > Thanks > > Aman Tandon > > >

Re: create heat maps

2014-04-09 Thread Mikhail Khludnev
Hello, Have you checked pivot facets? 09.04.2014 9:19 пользователь "Aman Tandon" написал: > How can we create the heat maps using solr spatial search? > > Thanks > Aman Tandon >