Re: solr 5 leaving tomcat, will I be the only one fearing about this?

2016-10-08 Thread Shawn Heisey
On 10/7/2016 5:13 PM, Renee Sun wrote: > I just read through the following link Shawn shared in his reply: > https://wiki.apache.org/solr/WhyNoWar > > While the following statement is true: > > "Supporting a single set of binary bits is FAR easier than worrying > about what kind of customized e

Re: Real Time Search and External File Fields

2016-10-08 Thread Shawn Heisey
On 10/7/2016 6:19 PM, Mike Lissner wrote: > Soft commits seem to be exactly the thing for this, but whenever I open a > new searcher (which soft commits seem to do), the external file is > reloaded, and all queries are halted until it finishes loading. When I just > measured, this took about 30 sec

Re: Real Time Search and External File Fields

2016-10-08 Thread Mike Lissner
On Sat, Oct 8, 2016 at 8:46 AM Shawn Heisey wrote: > Most soft commit > > documentation talks about setting up soft commits with of > about a > > second. > > IMHO any documentation that recommends autoSoftCommit with a maxTime of > one second is bad documentation, and needs to be fixed. Where h

Re: Real Time Search and External File Fields

2016-10-08 Thread Mike Lissner
On Fri, Oct 7, 2016 at 8:18 PM Erick Erickson wrote: > What you haven't mentioned is how often you add new docs. Is it once a > day? Steadily > from 8:00 to 17:00? > Alas, it's a steady trickle during business hours. We're ingesting court documents as they're posted on court websites, then sendi

Re: Real Time Search and External File Fields

2016-10-08 Thread Walter Underwood
With time-oriented data, you can use an old trick (goes back to Infoseek in 1995). Make a “today” collection that is very fresh. Nightly, migrate new documents to the “not today” collection. The today collection will be small and can be updated quickly. The archive collection will be large and

Re: solr 5 leaving tomcat, will I be the only one fearing about this?

2016-10-08 Thread Aristedes Maniatis
On 9/10/16 2:09am, Shawn Heisey wrote: > One of the historical challenges on this mailing list is that we were > rarely aware of what steps the user had taken to install or start Solr, > and we had to support pretty much any scenario. Since 5.0, the number > of supported ways to deploy and start S

Re: solr 5 leaving tomcat, will I be the only one fearing about this?

2016-10-08 Thread Aristedes Maniatis
On 9/10/16 11:11am, Aristedes Maniatis wrote: > * deployment is also scattered: > - Solr platform specific package manager (pkg in FreeBSD in my case, which > I've had to write myself since it didn't exist) > - updating config files above > - writing custom scripts to push Zookeeper configurati

Re: Stream expressions: Break up multivalue field into usable tuples

2016-10-08 Thread Doug Turnbull
Joel -- thanks! Got this working and now feel in a better shape to grok what's happening Out of curiosity, is there any work being done to customize scoreNodes scoring? There's a bunch of other forms of similarity I wouldn't mind playing with as well. On Thu, Sep 22, 2016 at 6:06 PM Joel Bernstei

Re: Stream expressions: Break up multivalue field into usable tuples

2016-10-08 Thread Joel Bernstein
Great, I'm not sure if you noticed that SOLR-9537 has been committed and will be in 6.3. So now you can directly wrap a facet expression with the scoreNodes expression. Yeah, other scoring algorithms would be a great thing. We can adjust the ScoreNodesStream to make this more flexible. Feel free t

Re: Real Time Search and External File Fields

2016-10-08 Thread Erick Erickson
I chose 16 as a place to start. You usually reach diminishing returns pretty quickly, i feel it's a mistake to set your autowarm counts to, say 256 (and I've seen this in the thousands) unless you have some proof that it's useful to bump higher. But certainly if you set them to 16 and see spikes j