Re: Poll: Master-Slave or SolrCloud?

2017-04-26 Thread Stephen Weiss
We run both, and we are running the latest versions for both. There are different use cases for each one. Where we are using solrcloud, it only has to operate in one datacenter, and sharding is incredibly important because we have billions and billions of documents. In a separate group of ser

Re: DateRangeField and Faceting

2017-04-21 Thread Stephen Weiss
se case (I don't think I can even do what I'm doing with old facets). Why would daterangefield work with the old faceting system and not the new? -- Steve On Fri, Apr 21, 2017 at 11:50 AM, Stephen Weiss mailto:steve.we...@wgsn.com>> wrote: Hi everyone, Just trying to do a sense che

DateRangeField and Faceting

2017-04-21 Thread Stephen Weiss
Hi everyone, Just trying to do a sense check on this. I'm trying to do a facet based off a DateRangeField and I'm hitting this error: Error from server at http://172.20.141.150:8983/solr/instock_au_shard1_replica0: Unable to range facet on field:sku_history.date_range{type=daterange,properti

Re: Challenges with new Solrcloud Backup/Restore functionality

2016-11-08 Thread Stephen Weiss
s.apache.org/jira/browse/SOLR-9527> (on issue #2), but it appears to occur without the patch as well. I haven't seen this issue with solr 6.1.0 despite extensive testing. I haven't confirmed if this occurs on the official 6.2.0 release build. I will try to confirm and gather more dat

Re: Challenges with new Solrcloud Backup/Restore functionality

2016-09-26 Thread Stephen Weiss
egarding 1st issue, does backup command succeeds? Also do you see any warning/error log messages? How about the restore command? Thanks Hrishikesh On Sat, Sep 24, 2016 at 12:14 PM, Stephen Weiss mailto:steve.we...@wgsn.com>> wrote: Hi everyone, We're very excited about SolrCloud'

Challenges with new Solrcloud Backup/Restore functionality

2016-09-24 Thread Stephen Weiss
Hi everyone, We're very excited about SolrCloud's new backup / restore collection APIs, which should introduce some major new efficiencies into our indexing workflow. Unfortunately, we've run into some snags with it that are preventing us from moving into production. I was hoping someone on t

Re: SolrCloud replicas consistently out of sync

2016-05-17 Thread Stephen Weiss
etting file length for [segments_4o] > java.nio.file.NoSuchFileException: > /var/solr/data/instock_shard5_replica1/data/index.20160516230059221/segments_4o > > -- > Steve > > > On Tue, May 17, 2016 at 5:07 PM, Stephen Weiss > mailto:steve.we...@wgsn.com><mail

Re: SolrCloud replicas consistently out of sync

2016-05-17 Thread Stephen Weiss
index.20160516230059221/segments_4o -- Steve On Tue, May 17, 2016 at 5:07 PM, Stephen Weiss mailto:steve.we...@wgsn.com>> wrote: OK, so we did as you suggest, read through that article, and we reconfigured the autocommit to: ${solr.autoCommit.maxTime:3} false ${solr.autoSoftCommit.max

Re: SolrCloud replicas consistently out of sync

2016-05-17 Thread Stephen Weiss
ch for each shard. Here's a long blog on commits: https://lucidworks.com/blog/2013/08/23/understanding-transaction-logs-softcommit-and-commit-in-sorlcloud/ Best, Erick On Tue, May 17, 2016 at 8:18 AM, Stephen Weiss mailto:steve.we...@wgsn.com>> wrote: > Yes, after startup there was

Re: SolrCloud replicas consistently out of sync

2016-05-17 Thread Stephen Weiss
#x27;s the only way I can imagine that Solr "healed" itself. I've got to point back to the Solr logs. Are they showing any anomalies? Are any nodes in recovery when you restart? Best, Erick On Mon, May 16, 2016 at 4:14 PM, Stephen Weiss mailto:steve.we...@wgsn.com>> wrote: &

Re: SolrCloud replicas consistently out of sync

2016-05-16 Thread Stephen Weiss
all nodes? Even if we had to trigger that manually after indexing, it would be fine. It's a very controlled indexing workflow that only happens once a day. -- Steve On Mon, May 16, 2016 at 6:52 PM, Stephen Weiss mailto:steve.we...@wgsn.com>> wrote: Each node has one JVM with 16GB o

Re: SolrCloud replicas consistently out of sync

2016-05-16 Thread Stephen Weiss
ee exceptions in your Solr logs that will shed light on the issue. Best, Erick On Mon, May 16, 2016 at 8:03 AM, Stephen Weiss mailto:steve.we...@wgsn.com>> wrote: > Hi everyone, > > I'm running into a problem with SolrCloud replicas and thought I would ask > the list to se

SolrCloud replicas consistently out of sync

2016-05-16 Thread Stephen Weiss
Hi everyone, I'm running into a problem with SolrCloud replicas and thought I would ask the list to see if anyone else has seen this / gotten past it. Right now, we are running with only one replica per shard. This is obviously a problem because if one node goes down anywhere, the whole collec

Re: DateRangeField Query throws NPE

2015-08-04 Thread Stephen Weiss
ues. What's the version of Solr? Can't it happen you inherited schema.xml from the older version of Solr? Which "Web-based queries work fine," you run exactly and what's the output with debugQuery=true? How exactly do you obtain dateRangeField? are you sure that "sku_history.d

DateRangeField Query throws NPE

2015-08-03 Thread Stephen Weiss
Hi everyone, I'm running into a trouble building a query with DateRangeField. Web-based queries work fine, but this code throws an NPE: dateRangeQuery = dateRangeField.getRangeQuery(null, SidxS.getSchema().getField("sku_history.date_range"), start_date_str, end_date_str, true, true); ERROR

Re: How to use BitDocSet within a PostFilter

2015-08-03 Thread Stephen Weiss
cs are numbered from zero. So to check them against the full index bitset, I'd be doing Bitset.exists(indexBase + docid) Just one thing to check Roman On Aug 3, 2015 1:24 AM, "Stephen Weiss" > wrote: > Hi everyone, > > I'm trying to write a PostFilter for Solr 5.

How to use BitDocSet within a PostFilter

2015-08-02 Thread Stephen Weiss
Hi everyone, I'm trying to write a PostFilter for Solr 5.1.0, which is meant to crawl through grandchild documents during a search through the parents and filter out documents based on statistics gathered from aggregating the grandchildren together. I've been successful in getting the logic co

Re: Problem using curl in PHP to get Solr results

2010-12-15 Thread Stephen Weiss
Forgive me if this seems like a dumb question but have you tried the Apache_Solr_Service class? http://www.ibm.com/developerworks/library/os-php-apachesolr/index.html It's really quite good at handling the nuts and bolts of making the HTTP requests and decoding the responses for PHP. I almost

Re: Color search for images

2010-09-15 Thread Stephen Weiss
There's a project out there called LIRE (I heard about it on this list) that's supposed to create a lucene-based CIBR index for images. I wonder if this could be integrated with Solr? Personally I don't really care about the flower part, I'm more worried about searching whether the flower is r

SOLR Memory Usage - Where does it go?

2010-07-23 Thread Stephen Weiss
We have been having problems with SOLR on one project lately. Forgive me for writing a novel here but it's really important that we identify the root cause of this issue. It is becoming unavailable at random intervals, and the problem appears to be memory related. There are basically two

Re: If you could have one feature in Solr...

2010-02-27 Thread Stephen Weiss
I think an examples page would be a good idea. We've already implemented search in Chinese, Japanese, and Spanish back with 1.3, but it was not really very well laid out how it was supposed to work - I had to dig through bits and pieces of people's configs left in the mailing list archives

Re: If you could have one feature in Solr...

2010-02-26 Thread Stephen Weiss
+1 I have several projects backburnered in the hope realtime search will come to solr soon... [m] On Feb 26, 2010, at 8:37 PM, Don Werve wrote: Realtime search, hands down.

Re: NullPointerException in ReplicationHandler.postCommit + question about compression

2010-01-21 Thread Stephen Weiss
Hi Shalin, Thanks for your reply. Please see below. On Jan 18, 2010, at 4:19 AM, Shalin Shekhar Mangar wrote: On Wed, Jan 13, 2010 at 12:51 AM, Stephen Weiss wrote: ... When we replicate manually (via the admin page) things seem to go well. However, when replication is triggered by a

NullPointerException in ReplicationHandler.postCommit + question about compression

2010-01-12 Thread Stephen Weiss
Hi Solr List, We're trying to set up java-based replication with Solr 1.4 (dist tarball). We are running this to start with on a pair of test servers just to see how things go. There's one major problem we can't seem to get past. When we replicate manually (via the admin page) things se

Re: Solr 1.4 - stats page slow

2010-01-06 Thread Stephen Weiss
Sorry, know I'm a little late in replying but the LukeRequestHandler tip was just what I needed! Thank you so much. -- Steve On Dec 25, 2009, at 2:03 AM, Chris Hostetter wrote: : I've noticed this as well, usually when working with a large field cache. I : haven't done in-depth analysis

Solr 1.4 - stats page slow

2009-12-23 Thread Stephen Weiss
We've been using Solr 1.4 for a few days now and one slight downside we've noticed is the stats page comes up very slowly for some reason - sometimes more than 10 seconds. We call this programmatically to retrieve the last commit date so that we can keep users from committing too frequentl

Re: Field Collapsing (was Re: Schema for group/child entity setup)

2009-09-09 Thread Stephen Weiss
I just noticed this and it reminded me of an issue I've had with collapsed faceting with an older version of the patch in Solr 1.3. Would it be possible, if we can get the terms for all the collapsed documents on a field, to then facet each collapsed document on the unique terms it has col

Can solr return documents which don't match a query?

2009-09-07 Thread Stephen Weiss
Hi everyone, I have what's probably a loaded question about how Solr parses queries. Basically, the client wants to restructure the way they handle permissions. Instead of assigning specific permissions ids (as we've done up until this point), they want to structure permissions by adding

Re: Results from Solr

2009-08-20 Thread Stephen Weiss
If I'm not mistaken, you should index name as field type "string" - right now you are probably using "text" so it is tokenizing on the Uppercase characters. If you use "string" type this shouldn't happen. You could use a copyField to make a separate "name_string" field so that you can do

Re: Slowness during submit the index

2009-06-20 Thread Stephen Weiss
Isn't it possible that the production equipment is simply under much higher load (given that, since it's in production, your various users are all actually using it), vs the QA equipment, which is only in use by the people doing QA? We've found the same thing at one point - we had a very sm

Re: Destemming snafu

2009-06-18 Thread Stephen Weiss
t). Check out http://wiki.apache.org/solr/ AnalyzersTokenizersTokenFilters and the example config for the Porter Stemmer: protected="protwords.txt" /> HTH Brendan On Jun 18, 2009, at 4:38 PM, Stephen Weiss wrote: Hi, I've hit a bit of a problem with destemming and

Destemming snafu

2009-06-18 Thread Stephen Weiss
Hi, I've hit a bit of a problem with destemming and could use some advice. Right now there is a word in the index called "Stylesight" and another word "Stylesightings", which was just added. When users search for "Stylesightings", the client really only wants them to get results that matc

Re: Facet query return zero hits if category facet value has 2 or more words (looks like space does not work)

2009-06-05 Thread Stephen Weiss
Thanh, Do the categories have id numbers? What we do is we index the ID numbers of our tags and categories, and then on display we pull the real text out of the database. In our situation, since the tags and categories are multilingual, it's a lot easier than converting from English to

Re: Birthday (that's "day" not "date") search query?

2009-04-07 Thread Stephen Weiss
If someone's birthday falls on a leap year, in most countries their birthday is considered to be February 28th unless it happens to be a leap year. You could make the field a float, encode the day number as 59.5, so it will match where it should, and write special handling along these line

Re: Search transparently with Solr with multiple cores, different indexes, common response type

2009-03-26 Thread Stephen Weiss
I have a very similar setup and that's precisely what we do - except with JSON. 1) Request comes into PHP 2) PHP runs the search against several different cores (in a multicore setup) - ours are a little more than "slightly" different 3) PHP constructs a new object with the responseHeader a

Re: solr 1.3 - did something with deleting documents change?

2009-02-28 Thread Stephen Weiss
Yeah honestly I don't know how it ever worked either. The notes in the wiki seem to indicate that syntax (with multiple nodes) will be supported in Solr 1.4, not 1.3 - but I guess it really just means that you can't combine those with a node yet. I'll miss the deletesPending stat, I think

Re: SOLR newbie question: How to filter the results based on my Unique Key

2009-02-28 Thread Stephen Weiss
There's an experimental patch for this I've had pretty good success with: https://issues.apache.org/jira/browse/SOLR-236 If you don't particularly need faceting support to work 100% it's already pretty perfect. Officially I guess they want it to make it in for version 1.5?? But in the me

solr 1.3 - did something with deleting documents change?

2009-02-25 Thread Stephen Weiss
Hi, This might be a very silly question that's documented everywhere but I just can't find an answer right now. When we first implemented solr we used version 1.2 (of course, 1.3 was released days afterwards). In the brief period we used 1.2, whenever we wanted to delete a bunch of docum

Re: why don't we have a forum for discussion?

2009-02-18 Thread Stephen Weiss
Like an earlier poster, my issue isn't on the laptop, it's with my mobile device. The sheer volume of e-mail overwhelms the thing sometimes (right now, for instance). There's really no option for moving the e-mail off to some other folder, it just all goes to one place. Perhaps that mea

Re: why don't we have a forum for discussion?

2009-02-18 Thread Stephen Weiss
I third the motion SOLR is the second largest contributor to my e- mail glut (my company's marketing is #1). I often have no idea what area of Solr I'm actually asking about when I have a question, so I would disagree and say a general forum provides a place to post when you don't reall

Re: Exact match search problem

2009-02-02 Thread Stephen Weiss
Try using fieldtype "string" instead of "text" for the UserName field. Then it will not be tokenized so it should only give exact matches. -- Steve On Feb 2, 2009, at 2:27 AM, mahendra mahendra wrote: Hi, I have indexed my data as "custom123, customer, custom" for the "UserName" fiel

Re: Help with Solr 1.3 lockups?

2009-01-15 Thread Stephen Weiss
I've been wondering about this one myself - most of the services we have installed work this way, if they crash out for whatever reason they restart automatically (Apache, MySQL, even the OS itself). Failures are detected and corrected by the load balancers and also in some cases by the ma

Re: Database permissions integration and Sub documents

2009-01-12 Thread Stephen Weiss
On Jan 11, 2009, at 10:08 PM, Mike Shredder wrote: Hi , I'm new to Solr .. I've been able to get Solr up & running. But got some quick questions. 1) How do I filter results based on permissions from an external database system ? --> Should I implement a queryfilter which will

Re: Memory size error

2008-12-28 Thread Stephen Weiss
Actually I think this is a PHP error, not a Java error (I certainly recognize it from PHP, in Java the error is usually more like OutOfMemoryException). Check the server's php.ini file for a line like this: memory_limit = 16M and try setting it to something more reasonable: memory_limit =

Re: umlaut index รถ == o == oe Poss ible?

2008-12-16 Thread Stephen Weiss
I believe the german porter stemmer should handle this. I haven't used it with SOLR but I've used it with other projects, and basically, when the word is parsed, the umlauts and also accented vowels are converted to plain vowels. I guess with SOLR you use solr.SnowballPorterFilterFactory:

Re: Please help me articulate this query

2008-12-15 Thread Stephen Weiss
I think in this case you would want to index each question with the possible referrers ( by title might be too imprecise, I'd go with filename or ID) and then do a search like this (assuming in this case it's by filename) q=(referring:TomCruise.html) OR (question: Tom AND Cruise) Which see

Re: Applying Field Collapsing Patch

2008-12-11 Thread Stephen Weiss
s around that. -John On Dec 11, 2008, at 3:05 PM, Stephen Weiss wrote: Yes, only ivan patch 2 (and before, only ivan patch 1), my sense was these patches were meant to be used in isolation (there were no notes saying to apply any other patches first). Are you using patches for any oth

Re: Applying Field Collapsing Patch

2008-12-11 Thread Stephen Weiss
thanks for the advice. I just downloaded a completely clean version, haven't even tried to build it yet. Applied the same, and I received exactly the same results. Do you only apply the ivan patch 2? What version of patch are you running? -John On Dec 11, 2008, at 2:10 PM, Stephen We

Re: Problems with SOLR-236 (field collapsing)

2008-12-11 Thread Stephen Weiss
to only execute when builder.isNeedDocSet=true. In the field collapsing patch that I'm using, I've got code to remove a previous 'response' from the builder.rsp so we don't have duplicates. Now, if I could get field collapsing to work properly with a docSet/ docList

Re: Applying Field Collapsing Patch

2008-12-11 Thread Stephen Weiss
Are you sure you have a clean copy of the source? Every time I've applied his patch I grab a fresh copy of the tarball and run the exact same command, it always works for me. Now, whether the collapsing actually works is a different matter... -- Steve On Dec 11, 2008, at 1:29 PM, John Mart

Re: Problems with SOLR-236 (field collapsing)

2008-12-09 Thread Stephen Weiss
ginning of the year, so I'm somewhat familiar. I'll try and get a look this weekend. It's the earliest I''m likely to get spare cycles. I'll post any results. Tracy On Dec 9, 2008, at 4:18 PM, Stephen Weiss wrote: Hi, I'm trying to use field collapsi

Problems with SOLR-236 (field collapsing)

2008-12-09 Thread Stephen Weiss
Hi, I'm trying to use field collapsing with our SOLR but I just can't seem to get it to do anything. I've downloaded a dist copy of solr 1.3 and applied Ivan de Prado's patch - reading through the source code, the patch definitely was applied successfully (all the changes are in the right

Field collapsing (SOLR-236) and Solr 1.3.0 release version

2008-11-19 Thread Stephen Weiss
Hi, A requirement has come up in a project where we're going to need to group by a field in the result set. I looked into the SOLR-236 patch and it seems there are a couple versions out now that are supposed to work against the Solr 1.3.0 release. This is a production site, it really can

Re: Solr with Wordpress - Anyone doing this?

2008-11-10 Thread Stephen Weiss
import for indexing (I think) and then it's just a matter of pointing the search box at the Solr instance, I suppose. On Nov 6, 2008, at 11:19 PM, Stephen Weiss wrote: Hi, We recently implemented Solr for one major search component of our site, and now that this is complete we'

Solr with Wordpress - Anyone doing this?

2008-11-06 Thread Stephen Weiss
Hi, We recently implemented Solr for one major search component of our site, and now that this is complete we're turning to other areas of our site to see where Solr can help us improve results relevancy and performance. One major area where I think Solr could do a lot of good is to repla

Re: Max Number of Facets

2008-10-30 Thread Stephen Weiss
I've actually seen cases on our site where it's possible to bring up over 30,000 facets for one query. And they actually come up quickly - like, 3 seconds. It takes longer for the browser to render them. -- Steve On Oct 30, 2008, at 6:04 PM, Ryan McKinley wrote: the only 'limit' is th

Re: Question about textTight

2008-10-28 Thread Stephen Weiss
_ won't be dropped). You could set up another field for use with wildcard queries, or you could create separate query and index analyzers for textTight and set the index analyzer to use a WordDelimiterFilter that also indexes the original token. -Yonik On Tue, Oct 28, 2008 at 2:31 PM, Ste

Re: Question about textTight

2008-10-28 Thread Stephen Weiss
008, at 1:54 PM, Yonik Seeley wrote: These query parsing results don't match with the config you've posted. Double-check the type of the "name" field and that you have restarted Solr since changing the schema.xml -Yonik On Tue, Oct 28, 2008 at 11:25 AM, Stephen Weiss <[E

Re: Question about textTight

2008-10-28 Thread Stephen Weiss
che.org/solr/AnalyzersTokenizersTokenFilters?highlight=(t okenizer)#head-1c9b83870ca7890cd73b193cefed83c283339089 The admin analysis page and query debug will help a lot to see what's going on. -Todd -Original Message- From: Stephen Weiss [mailto:[EMAIL PROTECTED] Sent: Monday

Question about textTight

2008-10-27 Thread Stephen Weiss
Hi, So I've been using the textTight field to hold filenames, and I've run into a weird problem. Basically, people want to search by part of a filename (say, the filename is stm0810m_ws_001ftws and they want to find everything starting with stm0810m_ (stm0810m_*). I'm hoping someone mig

updating synonyms file

2008-09-24 Thread Stephen Weiss
Hi, I'm running Solr 1.2, we are not able to upgrade yet. We've started using synonyms to make the search results better but a few of the synonyms turn out to have unexpected results. We have modifications to our synonyms file that need to go in very quickly but I can't seem to figure o

Re: Rounding date fields

2008-08-28 Thread Stephen Weiss
I was just dealing with this issue the other day! My solution was to have PHP re-parse the dates to always be exactly at midnight of their respective day (so that the extra time info wouldn't affect sorting), and then when the dates are displayed my format string only includes date info...

Re: Sorting and also looking at stored fields

2008-08-26 Thread Stephen Weiss
e retrieved for further display. Here's a link on that... http://wiki.apache.org/solr/SchemaXml#head-af67aefdc51d18cd8556de164606030446f56554 If that's not it maybe seeing the schema would help. -- Steve On Aug 26, 2008, at 7:46 PM, jennyv wrote: On Tue, Aug 26, 2008 at 3:46 PM

Re: Sorting and also looking at stored fields

2008-08-26 Thread Stephen Weiss
I do this by specifying them in the query string: fl=field_name1,field_name2,etc See here: http://www.ibm.com/developerworks/java/library/j-solr1/#searching (table 2) -- Steve On Aug 26, 2008, at 6:36 PM, jennyv wrote: Ack. Reading the FAQ would help. :-) I've fixed the sorting issue by u

Re: How do I configure commit to run after updates

2008-08-07 Thread Stephen Weiss
I'm not exactly a long-time user of Solr yet so forgive me if I'm missing something really obvious here... but, if I'm not mistaken, you just add to the end of your update XML. This is how I've done all of my commits, I wasn't even aware there was a CLI method to this. I would worry abo

Re: Solr Logo thought

2008-08-05 Thread Stephen Weiss
My issue with the logos presented was they made solr look like a school project instead of the powerful tool that it is. The tricked out font or whatever just usually doesn't play well with the business types... they want serious-looking software. First impressions are everything. While

Re: Seeking suggestions - keyword related site promotion

2008-06-18 Thread Stephen Weiss
Is there a fixed set of keywords? If so, I suppose you could simply index these keywords into a field for each site (either through some kind of automatic parser or manually - from personal experience I would recommend manually unless you have tens of thousands of these things), and then s

Re: searching only within allowed documents

2008-06-10 Thread Stephen Weiss
tokenizer, but now I see there's PatternAnalyzer. I'll give it a shot. Regards, Steve On Jun 10, 2008, at 5:18 PM, Yonik Seeley wrote: On Mon, Jun 9, 2008 at 7:44 PM, Stephen Weiss <[EMAIL PROTECTED]> wrote: However, in the plain text search, the user automatically searc

searching only within allowed documents

2008-06-09 Thread Stephen Weiss
Hi, I'm new to Solr (and Lucene) and I'm trying to work out just how I could fit this technology into my app (I'm moving over from using MySQL fulltext indexes). Things are actually going really well - the facet functionality fits in just perfectly, and the basic full-text searching is w