RE: [EXTERNAL] Re: Getting rid of Master/Slave nomenclature in Solr

2020-06-18 Thread Demian Katz
Regarding people having a problem with the word "master" -- GitHub is changing the default branch name away from "master," even in isolation from a "slave" pairing... so the terminology seems to be falling out of favor in all contexts. See: https://www.cnet.com/news/microsofts-github-is-removin

RE: Help with a DIH config file

2019-03-15 Thread Demian Katz
Jörn (and anyone else with more experience with this than I have), I've been working on Whitney with this issue. It is a PDF file, and it can be opened successfully in a PDF reader. Interestingly, if I try to extract data from it on the command line, Tika version 1.3 throws a lot of warnings but

Solr Cell, Tika and UpdateProcessorChains

2019-02-21 Thread Demian Katz
I'm posting this question on behalf of Whitney Clarke, who is a pending member of this list but is not able to post on her own yet. I've been working with her on some troubleshooting, but I'm not familiar with the components she's using and thought somebody here might be able to point her in the

RE: Installing Solr with Ivy

2016-08-03 Thread Demian Katz
uth is not quite good enough for me; but it is only my own fault I haven't contributed something more. Dan Davis, Systems/Applications Architect (Contractor), Office of Computer and Communications Systems, National Library of Medicine, NIH -Original Message- From: Demian

RE: Installing Solr with Ivy

2016-08-02 Thread Demian Katz
cations.Unfortunately, HTTP auth is not quite good enough for me; but it is only my own fault I haven't contributed something more. Dan Davis, Systems/Applications Architect (Contractor), Office of Computer and Communications Systems, National Library of Medicine, NIH -----Original

RE: Installing Solr with Ivy

2016-08-02 Thread Demian Katz
Thanks, Shawn, for confirming my suspicions. Regarding your question about how Solr differs from a database server, I agree with you in theory, but the problem is in the practice: there are very easy, familiar, well-established techniques for installing and maintaining database platforms, and t

Installing Solr with Ivy

2016-08-01 Thread Demian Katz
As a follow-up to last week's thread about loading Solr via dependency manager, I started experimenting with using Ivy to install Solr. Here's what I have (note that I'm trying to install Solr 5.5.0 as an arbitrary example, but that detail should not be important): ivy.xml:

RE: Installing Solr as a dependency

2016-08-01 Thread Demian Katz
down an official Docker build and adding your stuff to that? https://hub.docker.com/_/solr/ Regards, Alex. Newsletter and resources for Solr beginners and intermediates: http://www.solr-start.com/ On 30 July 2016 at 03:03, Demian Katz wrote: >> I wouldn't include Solr in my

RE: Installing Solr as a dependency

2016-07-29 Thread Demian Katz
hawn Heisey wrote: > On 7/28/2016 1:29 PM, Demian Katz wrote: > > I develop an open source project > > (https://github.com/vufind-org/vufind) that depends on Solr, and I'm > > trying to figure out if there is a better way to manage the Solr > > dependency. Presentl

RE: Installing Solr as a dependency

2016-07-29 Thread Demian Katz
> I wouldn't include Solr in my own project at all. I would probably > request that the user download the binary artifact and put it in a > predictable location, and configure my installation script to do the > download if the file is not there. I would strongly recommend taking > advantage of Ap

Installing Solr as a dependency

2016-07-28 Thread Demian Katz
Hello, I develop an open source project (https://github.com/vufind-org/vufind) that depends on Solr, and I'm trying to figure out if there is a better way to manage the Solr dependency. Presently, I simply bundle Solr with my software by committing the latest distribution to my Git repo. Over

qf boosts with MoreLikeThis query parser

2016-07-11 Thread Demian Katz
Hello, I am currently using field-specific boosts in the qf setting of the MoreLikeThis request handler: https://github.com/vufind-org/vufind/blob/master/solr/vufind/biblio/conf/solrconfig.xml#L410 I would like to accomplish the same effect using the MoreLikeThis query parser, so that I can ta

Pull request protocol question

2016-03-01 Thread Demian Katz
Hello, A few weeks ago, I submitted a pull request to Solr in association with a JIRA ticket, and it was eventually merged. More recently, I had an almost-trivial change I wanted to share, but on GitHub, my Solr fork appeared to have changed upstreams. Was the whole Solr repo moved and regener

SOLR-2798 (local params parsing issue) -- how can I help?

2015-12-02 Thread Demian Katz
Hello, I'd really love to see a resolution to SOLR-2798, since my application has a bug that cannot be addressed until this issue is fixed. It occurred to me that there's a good chance that the code involved in this issue is relatively isolated and testable, so I might be able to help with a s

Costs/benefits of DocValues

2015-11-09 Thread Demian Katz
Hello, I have a legacy Solr schema that I would like to update to take advantage of DocValues. I understand that by adding "docValues=true" to some of my fields, I can improve sorting/faceting performance. However, I have a couple of questions: 1.)Will Solr always take proper advantage of

ExternalFileField documentation problems?

2014-09-15 Thread Demian Katz
I've just been doing some experimentation with the ExternalFileField. I ran into obstacles due to some apparently incorrect documentation in the wiki: https://cwiki.apache.org/confluence/display/solr/Working+with+External+Files+and+Processes It seems that for some reason the and definitions ar

Preserving punctuation tokens with ICUTokenizerFactory

2012-04-10 Thread Demian Katz
It has been brought to my attention that ICUTokenizerFactory drops tokens like the ++ in "The C++ Programming Language." Is there any way to persuade it to preserve these types of tokens? thanks, Demian

RE: sun-java6 alternatives for Solr 3.5

2012-02-27 Thread Demian Katz
For what it's worth, I run Solr 3.5 on Ubuntu using the OpenJDK packages and I haven't run into any problems. I do realize that sometimes the Sun JDK has features that are missing from other Java implementations, but so far it hasn't affected my use of Solr. - Demian > -Original Message--

RE: SOLR - Just for search or whole site DB?

2012-02-21 Thread Demian Katz
I would strongly recommend using Solr just for search. Solr is designed for doing fast search lookups. It is really not designed for performing all the functions of a relational database system. You certainly COULD use Solr for everything, and the software is constantly being enhanced to make

RE: social/collaboration features on top of solr

2011-12-13 Thread Demian Katz
VuFind (http://vufind.org) uses Solr for library catalog (or similar) applications and features a MySQL database which it uses for storing user tags and comments outside of Solr itself. If there were a mechanism more closely tied to Solr for achieving this sort of effect, that would allow VuFin

Re: LocalParams, bq, and highlighting

2011-11-01 Thread Demian Katz
> This is definitely an interesting case that i don't think anyone ever > really considered before. It seems like a strong argument in favor of > adding an "hl.q" param that the HighlightingComponent would use as an > override for whatever the QueryComponent thinks the highlighting query > should

RE: DisMax and WordDelimiterFilterFactory (limitations of MultiPhraseQuery)

2011-10-27 Thread Demian Katz
ation. > > But I can't imagine a 100% solution here. You're effectively asking to > compensate for > any fat-fingered thing a user does. Impossible I think... > > Best > Erick > > On Tue, Oct 25, 2011 at 1:13 PM, Demian Katz > wrote: > > I've seen a co

RE: Dismax handler - whitespace and special character behaviour

2011-10-25 Thread Demian Katz
I just sent an email to the list about DisMax interacting with WordDelimiterFilterFactory, and I think our problems are at least partially related -- I think the reason you are seeing an OR where you expect an AND is that you have autoGeneratePhraseQueries set to false, which changes the way Di

DisMax and WordDelimiterFilterFactory

2011-10-25 Thread Demian Katz
I've seen a couple of threads related to this subject (for example, http://www.mail-archive.com/solr-user@lucene.apache.org/msg33400.html), but I haven't found an answer that addresses the aspect of the problem that concerns me... I have a field type set up like this:

LocalParams, bq, and highlighting

2011-09-21 Thread Demian Katz
I've run into another strange behavior related to LocalParams syntax in Solr 1.4.1. If I apply Dismax boosts using bq in LocalParams syntax, the contents of the boost queries get used by the highlighter. Obviously, when I use bq as a separate parameter, this is not an issue. To clarify, here

RE: Questions about LocalParams syntax

2011-09-20 Thread Demian Katz
It surprises me that this very powerful feature of Solr is so little-documented. - Demian > -Original Message- > From: Jonathan Rochkind [mailto:rochk...@jhu.edu] > Sent: Tuesday, September 20, 2011 10:32 AM > To: solr-user@lucene.apache.org > Cc: Demian Katz > Subject

Questions about LocalParams syntax

2011-09-20 Thread Demian Katz
I'm using the LocalParams syntax combined with the _query_ pseudo-field to build an advanced search screen (built on Solr 1.4.1's Dismax handler), but I'm running into some syntax questions that don't seem to be addressed by the wiki page here: http://wiki.apache.org/solr/LocalParams 1.)H

"String index out of range: -1" for hl.fl=* in Solr 1.4.1?

2011-09-09 Thread Demian Katz
I'm running into a strange problem with Solr 1.4.1 - this request: http://localhost:8080/solr/website/select/?q=*%3A*&rows=20&start=0&indent=yes&fl=score&facet=true&facet.mincount=1&facet.limit=30&facet.field=category&facet.field=linktype&facet.field=subject&facet.prefix=&facet.sort=&fq=category%3

RE: SpellCheckComponent performance

2011-06-07 Thread Demian Katz
x? > > It'd probably be best if you showed us your fieldtype and field > definition... > > Best > Erick > > On Mon, Jun 6, 2011 at 4:04 PM, Demian Katz > wrote: > > I'm continuing to work on tuning my Solr server, and now I'm noticing > that m

SpellCheckComponent performance

2011-06-06 Thread Demian Katz
I'm continuing to work on tuning my Solr server, and now I'm noticing that my biggest bottleneck is the SpellCheckComponent. This is eating multiple seconds on most first-time searches, and still taking around 500ms even on cached searches. Here is my configuration: basicSpell

RE: Solr performance tuning - disk i/o?

2011-06-06 Thread Demian Katz
rmal, I'm not quite sure what's > going on there but you may > want to look at firstsearcher, newsearcher and autowarm parameters in > config.xml.. > > Best > Erick > > On Mon, Jun 6, 2011 at 9:08 AM, Demian Katz > wrote: > > Thanks once again for th

RE: Solr performance tuning - disk i/o?

2011-06-06 Thread Demian Katz
ing interval for any slaves in relation to the warmup > times. > If your polling interval is shorter than the warmup times, you run a > risk of > "runaway warmups". > > As you've figured out, measuring responses to the first few queries > doesn't > always

RE: Solr performance tuning - disk i/o?

2011-06-03 Thread Demian Katz
g? > > > There's more, but that'll do for a start Let us know if you gather > more data > and it's still slow. > > Best > Erick > > On Fri, Jun 3, 2011 at 8:44 AM, Demian Katz > wrote: > > Hello, > > > > I'm trying to

Solr performance tuning - disk i/o?

2011-06-03 Thread Demian Katz
Hello, I'm trying to move a VuFind installation from an ailing physical server into a virtualized environment, and I'm running into performance problems. VuFind is a Solr 1.4.1-based application with fairly large and complex records (many stored fields, many words per record). My particular i

RE: Bug in solr.KeywordMarkerFilterFactory?

2011-04-20 Thread Demian Katz
lysis.jsp's "dontstems bees" to using > the query debug interface: > > "dontstems bees" > "dontstems bees" > PhraseQuery(text:"dontstems bee") > text:"dontstems bee" > > On Wed, Apr 20, 2011 at 2:43 PM, Yonik Seele

Bug in solr.KeywordMarkerFilterFactory?

2011-04-20 Thread Demian Katz
I've just started experimenting with the solr.KeywordMarkerFilterFactory in Solr 3.1, and I'm seeing some strange behavior. It seems that every word subsequent to a protected word is also treated as being protected. For testing purposes, I have put the word "spelling" in my protwords.txt. If I

RE: Solr 3.1 ICU filters (error loading class)

2011-04-18 Thread Demian Katz
gt; To: solr-user@lucene.apache.org > Cc: Demian Katz > Subject: Re: Solr 3.1 ICU filters (error loading class) > > I don't think you want to put them in solr_home, I think you want to > put > them in solr_home/lib/. Or did you mean that's where you put them? > >

RE: Solr 3.1 ICU filters (error loading class)

2011-04-18 Thread Demian Katz
Monday, April 18, 2011 1:46 PM > To: solr-user@lucene.apache.org > Subject: Re: Solr 3.1 ICU filters (error loading class) > > On Mon, Apr 18, 2011 at 1:31 PM, Demian Katz > wrote: > > Hello, > > > > I'm interested in trying out the new ICU features in Solr 3.

Solr 3.1 ICU filters (error loading class)

2011-04-18 Thread Demian Katz
Hello, I'm interested in trying out the new ICU features in Solr 3.1. However, when I attempt to set up a field type using solr.ICUTokenizerFactory and/or solr.ICUFoldingFilterFactory, Solr refuses to start up, issuing "Error loading class" exceptions. I did see the README.txt file that menti

RE: OAI on SOLR already done?

2011-02-02 Thread Demian Katz
I already replied to the original poster off-list, but it seems that it may be worth weighing in here as well... The next release of VuFind (http://vufind.org) is going to include OAI-PMH server support. As you say, there is really no way to plug OAI-PMH directly into Solr... but a tool like

RE: filter query from external list of Solr unique IDs

2010-10-15 Thread Demian Katz
The main problem I've encountered with the "lots of OR clauses" approach is that you eventually hit the limit on Boolean clauses and the whole query fails. You can keep raising the limit through the Solr configuration, but there's still a ceiling eventually. - Demian > -Original Message--

RE: solr.WordDelimiterFilterFactory problem with hyphenated terms?

2010-04-12 Thread Demian Katz
gt; if you want the second query (which has 2 gaps) to match, you need to > either > use slop, or disable these increments alltogether. > > On Fri, Apr 9, 2010 at 11:44 AM, Demian Katz > wrote: > > > I've given it a try, and it definitely seems to have improved t

RE: solr.WordDelimiterFilterFactory problem with hyphenated terms?

2010-04-09 Thread Demian Katz
bject: Re: solr.WordDelimiterFilterFactory problem with hyphenated > terms? > > I'm not all that familiar with the underlying issues, but of the two > I'd > pick moving the WordDelimiterFactory rather than setting increments = > "false". > > But that's

RE: solr.WordDelimiterFilterFactory problem with hyphenated terms?

2010-04-08 Thread Demian Katz
uot;false" in the "index" portion of the schema > also > causes things to work. > > Developer folks: > I didn't see anything in a quick look in SOLR or Lucene JIRAs, should I > refine this a bit (really, sleepy time is near) and add a JIRA? > > Best >

RE: solr.WordDelimiterFilterFactory problem with hyphenated terms?

2010-04-07 Thread Demian Katz
handled by the WordDelimiterFilterFactory, but I don't have a good understanding of how term positions are calculated or factored into searching. Can anyone recommend some good reading to familiarize myself with these concepts in better detail? thanks, Demian From: Demian Katz Sent: Tu

solr.WordDelimiterFilterFactory problem with hyphenated terms?

2010-03-16 Thread Demian Katz
This is my first post on this list -- apologies if this has been discussed before; I didn't come upon anything exactly equivalent in searching the archives via Google. I'm using Solr 1.4 as part of the VuFind application, and I just noticed that searches for hyphenated terms are failing in stra