Re: Odd interaction between {!tag..} and {!field}

2014-05-20 Thread Chris Hostetter
: when local params are "embedded" in a query being parsed by the : LuceneQParser, it applies them using the same scoping as other query : operators : : : fq: "{!tag=name_name}{!field f=name}United States" Think of that example in the context of this one -- the basics of when/what

Re: Issue paging when sorting on a Date field

2014-05-20 Thread Chris Hostetter
: So I think when I was paging through the results, if the query for page N : was handled by replica1 and page N+1 handled by replica2, and the page : boundary happened to be where the reversed rows were, this would produce : the behavior I was seeing where the last row from the previous page was

Re: Date truncation and time zone when searching

2014-05-21 Thread Chris Hostetter
: Try the TZ parameter on the query, as blah&TZ=GMT-4 Docs... https://cwiki.apache.org/confluence/display/solr/Working+with+Dates : There's a good discussion of why PDT is ambiguous here: : https://issues.apache.org/jira/browse/SOLR-2690. -Hoss http://www.lucidworks.com/

Solr GeoHash Field (Solr 4.5)

2014-05-29 Thread Chris Atkinson
Hi, I've been reading up a lot on what David has written about GeoHash fields and would like to use them. I'm trying to create a nice way to display cluster counts of geo points on a google map. It's naturally not going to be possible to send 40k marker information over the wire to cluster... so

Re: Regex with local params is not working

2014-06-05 Thread Chris Hostetter
: edismax was developed before the lucene query parser syntax was : changed to include regex, so maybe that's the issue. edismax has never been explicilty updated to support regex queries... https://issues.apache.org/jira/browse/SOLR-6009 -Hoss http://www.lucidworks.com/

Re: change in EnumField configuration - what do you think?

2014-06-05 Thread Chris Hostetter
: I have an external system that indexes the data. : One field has a closed set of values, the values are sent as integer but : should be represented as String. EnumField is perfect for it. The : problem is that the values are predefined and not order sequentially : (the values were "spaced"

RE: SolrCloud: facet range option f..facet.mincount=1 omits buckets on response

2014-06-06 Thread Chris Hostetter
Ronald: I'm having a little trouble understading the steps o reproduce that you are describing -- in particular Step "1 f ii" because i'm not really sure i understand what exactly you are putting in "mem2.xml" Also: Since you don't appera to be using implicit routing, i'm not clear on why yo

Re: how to apply multiplcative Boost in multivalued field

2014-06-06 Thread Chris Hostetter
You need to explain what you mean here -- you're using hte term "multiplicitive boost" but the context in which you are asking the question suggests that isn't really applicable. what does that expression mean to you? Or to step back and rephrase: What is your usecase? What does your data loo

Re: Solr Scale Toolkit Access Denied Error

2014-06-06 Thread Chris Hostetter
: My guess is that the customized toolkit AMI (ami-1e6b9d76) at AWS is not : accessible by my AWS credentials. Is this an AMI permissioning issue or is : it a problem with my particular account or how it is configured at AWS. I : did not experience this specific problem when working with the pre

Re: wildcard matches in EnumField - what do I need to change in code to enable wildcard matches?

2014-06-06 Thread Chris Hostetter
What you are asking for is comparable to saying: In my index I have a TrieIntField called severity when i search for severity:8765432 I get results, but when i search for severity:8* i get no results -- what do i need to change so that this type of query matches 8765432 and 8978 and 84356 etc.

Re: Any way to view lucene files

2014-06-06 Thread Chris Collins
Have you tried: https://code.google.com/p/luke/ Best Chris On Jun 6, 2014, at 11:24 PM, Aman Tandon wrote: > Hi, > > Is there any way so that i can view what information and which is there in > my _e.fnm, etc files. may be with the help of any application or any viewer >

Re: Any way to view lucene files

2014-06-06 Thread Chris Collins
gards > Aman Tandon > > > On Sat, Jun 7, 2014 at 12:01 PM, Chris Collins wrote: > >> Have you tried: >> >> https://code.google.com/p/luke/ >> >> Best >> >> Chris >> On Jun 6, 2014, at 11:24 PM, Aman Tandon wrote: >> >>

ANNOUNCE: ApacheCon deadlines: CFP June 25 / Travel Assistance Jul 25

2014-06-12 Thread Chris Hostetter
(NOTE: cross-posted announcement, please confine any replies to general@lucene) As you may be aware, ApacheCon will be held this year in Budapest, on November 17-23. (See http://apachecon.eu for more info.) ### ### 1 - Call For Papers - June 25 The CFP for the conference is still open, but w

Re: SOLR error on distributed query

2014-06-12 Thread Chris Hostetter
: > set with the latest solr version. (Now we are running version 4.8 - the : > current schema has a uniqueid field set, while it wasn't present in the : > earlier versions. This unique field is unsurprisingly called "id" but not : > all the documents have it.) this is going to be the source of a

Re: How to Query to Solr for comparing two dates in solr

2014-06-13 Thread Chris Hostetter
: I think you'd have to get creative with function queries. The trick is You don't have to get *very* creative... : > I want to retrieve all docs or records from solr where updateDate >= : > appliedDate OR appliedDate == null Pretty sure all you need is... fq={!frange l=0}ms(updateDat

Re: docFreq coming to be more than 1 for unique id field

2014-06-17 Thread Chris Hostetter
All index wide statistics (like the docFreq of each term) are over the entire index, which includes deleted docs -- because it's an *inverted* index, it's not feasible to update those statistics to account for deleted docs (that would basically kill all the performance advantages thatcome from

Re: docFreq coming to be more than 1 for unique id field

2014-06-17 Thread Chris Hostetter
: Currently we are not using SolrJ but are simply interacting with solr with : json over http, this will change in a couple of months but currently not : there. As of now we are putting all the logic in query building, using it : to query solr and then passing on the json returned by it to front e

Re: docFreq coming to be more than 1 for unique id field

2014-06-17 Thread Chris Hostetter
: text in it, query is of the type "keywords:(word1 OR word2 ... OR wordN)". : The client is relying on default relevancy based sort returned by solr. : Some documents can get penalised because of some other documents which were : deleted. Is this functionality correct? yes, because term stats ar

Re: Warning message logs on startup after upgrading to 4.8.1

2014-06-18 Thread Chris Hostetter
: WARN o.a.s.r.ManagedResource- No stored data found for : /schema/analysis/stopwords/english : WARN o.a.s.r.ManagedResource- No stored data found for : /schema/analysis/synonyms/english : : I fixed these by commenting out the managed_en field type in my : schema, see https://g

Re: Segment Count of my Index is greater than the Configured MergeFactor

2014-06-19 Thread Chris Hostetter
: > I want to understand why 13 segments are created in my index?? : > Could appreciate if i can get response ASAP : Imagine the following scenario. You start from a clean index and do FWIW: the TL;DR of Shawn's response can be seen in this animation of how Log based MergePolicy's work in the

Re: Cursor deep paging new behavior

2014-06-19 Thread Chris Hostetter
if by "old behavior" you mean incremending the start param then the the order of results when doing concurrent indexing was always dependent on what exactly your sort was. when using a cursor, the impacts of concurrent indexing are also dependent on what your sort clause looks like -- but in d

Re: Limit Porter stemmer to plural stemming only?

2014-06-19 Thread Chris Hostetter
: Can you please share the Java code for Plural Only Porter Stemmer for English if you don't mind? The Porter stemmer algorithm, by definition, does more then just stip plurals. If you are interested in a lighter weight stemmer for english, this is exactly what the EnglishMinimalStemFilterFac

Re: Getting stats on Date facet groups

2014-06-23 Thread Chris Hostetter
: record counts by faceted date. I also want to get the total of an integer : field by faceted date but am having difficulty. Unfortunately, what you are asking about isn't currently possible. FWIW: lately i've been thinking a lot lately about "stats" and accumulating "stats" over facets, and

Re: Evaluate function only on subset of documents

2014-06-23 Thread Chris Hostetter
: Now, if I want to make a query that also contains some OR, it is impossible : to do so with this approach. This is because fq with OR operator is not : supported (SOLR-1223). As an alternative I've tried these queries: : : county='New York' AND (location:Maylands OR location:Holliscort or : park

Re: Evaluate function only on subset of documents

2014-06-24 Thread Chris Hostetter
: Let's take this query sample: : XXX OR AAA AND {!frange ...} : : For my use case: : AAA returns a subset of 100k documents. : frange returns 5k documents, all part of these 100k documents. : : Therefore, frange skips the most documents. From what you are saying, : frange is going to be applied

Re: Evaluate function only on subset of documents

2014-06-24 Thread Chris Hostetter
: I don't believe however that the two queries (the one with the post filter : and the one without one) are equivalent. : : Suppose out of the whole document set: : XXX returns documents 1,2,3. : AAA returns documents 6,7,8. : {!frange}customfunction returns documents 7,8. : : Running this quer

Re: Trouble with TrieDateFields

2014-06-24 Thread Chris Hostetter
: I am upgrading an index from Solr 3.6 to 4.2.0. : Everything has been picked up except for the old DateFields. Just to be crystal clear: 1) 4.2 is alreayd over a year old. the current rleease of Solr is 4.8, and 4.9 will most likeley be available within a day or two 2) Even in 4.9, "solr.D

Re: Double cast exception with grouping and sort function

2014-06-24 Thread Chris Hostetter
: I recently tried upgrading our setup from 4.5.1 to 4.7+, and I'm : seeing an exception when I use (1) a function to sort and (2) result : grouping. The same query works fine with either (1) or (2) alone. : Example below. Did you modify your schema in any way when upgrading? Can you provide so

Re: Double cast exception with grouping and sort function

2014-06-25 Thread Chris Hostetter
: I repo'd using the example config (with sharding). I was missing one : necessary condition: the schema needs a "*" dynamic field. : It looks like serializeSearchGroup matches the sort expression as the : "*" field, thus marshalling the double as TextField. : : Should I enter a ticket with the

RC for 4.9 Solr Ref-Guide immenient, please help look for formatting mistakes

2014-06-25 Thread Chris Hostetter
FYI: The current plan is to call a vote for the 4.9 Solr Ref Guide sometime tomorrow (2014-06-26) morning (~11AM UTC-0500 maybe?) The main thing we are currently waiting on is that sarowe is working on a simple page to document using Solr with SSL -- but now would be a great time for folks to he

Re: Sorting date fields

2014-06-25 Thread Chris Hostetter
: I see that result is affected by sorting order (ASC/DESC change order) but : result is not precise. For example for query : : params={mm=2&pf=tags^10+title^5&sort=created+asc&q=query&qf=tags^10+title^5&wt=javabin&version=2&defType=edismax&rows=10} those results don't really make sense -- can

Re: Spellchecker causing 500 (ISE)

2014-06-26 Thread Chris Hostetter
: We are getting the results for the query but the spellchecker component is : returning 500. Please help us out. : : *query*: http://localhostt:8111/solr/srch/select?q=malerkotla&qt=search what version of solr? what does your solrconfig.xml show for /select & the spellcheck config? what does yo

Re: Search results not as expected.

2014-06-26 Thread Chris Hostetter
: *ab:(system entity) OR ab:authorization* : Number of results returned 2 : which is not expected. : It seems this query makes the previous terms as OR if the next term is : introduced by an OR. in general, that's they way the "boolean" operators like AND/OR work in all of the various parser var

Re: group.ngroups is set to an incorrect value - specific field types

2014-06-26 Thread Chris Hostetter
I think you are correct -- deinitely looks like a bug to me... https://issues.apache.org/jira/browse/LUCENE-5790 : Date: Fri, 13 Jun 2014 10:45:12 + : From: 海老澤 志信 : Reply-To: solr-user@lucene.apache.org : To: "solr-user@lucene.apache.org" : Subject: group.ngroups is set to an incorrect va

Re: Throwing Error "Missing Mandatory uniquekey field id"

2014-07-01 Thread Chris Hostetter
: I mentioned id as string in schema.xml and i copied the csv into example docs : folder. I used the below commaand to download the data " Java : -Dtype=application/csv -jar post.jar import.csv" : : it's throwing the below error.Please help in this regard. : : ERROR - 2014-07-01 19:57:43.902; o

Re: Disable all caches in Solr

2014-07-01 Thread Chris Hostetter
: I want to run some query benchmarks, so I want to disable all type of caches Just to be clear: disabling all internal caching because you want to run a benchmark means you're probably going to wind up running a useless benchmark. Solr's internal caching is a key component of it's perormance

Re: Streaming large updates with SolrJ

2014-07-02 Thread Chris Hostetter
: Now that I think about it, though, is there a way to use the Update Xml : messages with something akin to the cloud solr server? I only see examples : posting to actual Solr instances, but we really need to be able to take : advantage of the zookeepers to send our updates to the appropriate ser

Re: schema / config file names

2014-07-02 Thread Chris Hostetter
: Is it required for the schema.xml and solrconfig.xml to have those exact : filenames? It's an extremelely good idea ... but strictly speaking no... https://cwiki.apache.org/confluence/display/solr/CoreAdminHandler+Parameters+and+Usage#CoreAdminHandlerParametersandUsage-CREATE This smells lik

RE: Memory Leaks in solr 4.8.1

2014-07-02 Thread Chris Hostetter
This is a long standing issue in solr, that has some suggested fixes (see jira comments), but no one has been seriously afected by it enough for anyone to invest time in trying to improve it... https://issues.apache.org/jira/browse/SOLR-2357 In general, the fact that Solr is moving away from b

Re: Changing default behavior of solr for overwrite the whole document on uniquekey duplication

2014-07-08 Thread Chris Hostetter
I think you are missunderstanding what Himanshu is suggesting to you. You don't need to make lots of big changes ot the internals of solr's code to get what you want -- instead you can leverage the Atomic Updates & Optimistic Concurrency features of Solr to get the existing internal Solr to re

Re: Complement of {!join}

2014-07-09 Thread Chris Hostetter
: : Somebody (with more knowledge) should write up an in-depth article on : this issue and whether the parent parser has to be default (lucene) or : whatever. It's a feature of Solr's standard query parser... https://cwiki.apache.org/confluence/display/solr/Query+Syntax+and+Parsing https://cwiki

Re: [Schema API] EmbeddedSolrServer Test

2014-07-10 Thread Chris Hostetter
EmbeddedSolrServer is really just for dealing with embedded SolrCore objects. It doesn't have ay of the other HTTP related plumbing assocaited with it -- which is why it can't be used for replication and/or cloud features, let alone the RESTLet based endpoints. if you want to write a test tha

Re: Complement of {!join}

2014-07-11 Thread Chris Hostetter
: So, I think a lot of people may be missing that power. I suspect it's I've added some comments to the relevant pages as reminders to go back and add more detail later so we don't lose track of the doc feedback expressed in this thread ... when i get more time i'll try to go back and fill that

Re: delta import to detect deleted values

2014-07-11 Thread Chris Hostetter
: As Ahmet indicated, you must have a way to detect that deletions have : happened. Marking rows as deleted with an active/inactive field is one : way. Another way (the way that we use) is to have a delete trigger on : the table that creates an entry in a delete tracking table. If you have no c

Re: How to migrate content of a collection to a new collection

2014-07-23 Thread Chris Hostetter
: billions of documents (not enough memory). Please note that we are on 4.4, : which does not contain the new CURSOR-feature. Please also note that speed is : an important factor for us. for situations where you know you will be processing every doc and order doesn't matter you can use a "poor m

Re: How to migrate content of a collection to a new collection

2014-07-24 Thread Chris Hostetter
: I tried this "poor mans" cursor approach out ad-hoc, but I get OOM. Pretty : sure this is because you need all uniqueKey-values in FieldCache in order to : be able to sort on it. We do not have memory for that - and never will. Our : uniqueKey field is not DocValue. : Just out of curiosity : * W

Re: Understanding the Debug explanations for Query Result Scoring/Ranking

2014-07-25 Thread Chris Hostetter
: Thank you very much Erik. This is exactly what I was looking for. While at : the moment I have no clue about these numbers, they ruby formatting makes it : much more easier to understand. Just to be clear, regardless of *which* response writer you use (xml, ruby, json, etc...) the default beha

RE: Solr Wiki ContributorsGroup request

2014-07-29 Thread Chris Hostetter
: One of the specific pages that I've been looking at is the page on : compiling Solr: https://wiki.apache.org/solr/HowToCompileSolr . I go for it... : haven't seen a corresponding page in Confluence. Could you help me : understand the relationship between the wiki and the Confluence : doc

re: Implementing custom analyzer for multi-language stemming

2014-07-30 Thread Chris Morley
I know BasisTech.com has a plugin for elasticsearch that extends stemming/lemmatization to work across 40 natural languages. I'm not sure what they have for Solr, but I think something like that may exist as well. Cheers, -Chris. From: "Eug

re: Solr is working very slow after certain time

2014-07-31 Thread Chris Morley
A page Solr Performance Factors mentions 2 big tips that may help you, but you have to read the rest of the page to make sure you understand the caveats there. In general, adding many documents per update request is faster than one per update request. Reducing the frequency of automatic comm

Re: Content-Charset header in HttpSolrServer

2014-08-06 Thread Chris Hostetter
: I was reviewing the httpclient code in HttpSolrServer and noticed that : it sets a "Content-Charset" header. As far as I know this is not a real : header and is not necessary. Anyone know a reason for this to be there? : I'm guessing this was just a mistake when converting from httpclient3 to

Re: Character encoding problems

2014-08-07 Thread Chris Hostetter
It's not clear to me from any of the comments you've made in this thread wether you've ever confirmed *exactly* what you are getting back from solr, ignoring the PHP completley. (ie: you refer to "UTF-8 for all of the web pages" suggesting you are only looking at some web application which is

Re: Updates to index not available immediately as index scales, even with autoSoftCommit at 1 second

2014-08-12 Thread Chris Hostetter
You havne't given us a lot of information to go on (ie: full solrconfig.xml, log messages arround the tim of your update, etc...) but my best guess would be that you are seeing a delay between the time the new searcher is opened and the time the newSearcher is made available to requests due to

Re: Updates to index not available immediately as index scales, even with autoSoftCommit at 1 second

2014-08-12 Thread Chris Hostetter
: I'm not seeing any messages in the log with respect to cache warming at the : time, but I will investigate that possibility. Thank you. In case it is what logs *do* you see at the time you send the doc? w/o details, we can't help you. : helpful, I pasted the entire solrconfig.xml at http:/

Re: Syntax unavailable for parameter substitution Solr 3.5

2014-08-15 Thread Chris Hostetter
: but am stuck with syntax with does not allow any concatenation of params to : a prefix. there's no inherent support for anything like that in the local param parsing that would help you in building up facet params like this. you'd either need to do this in some layer betwen your client and so

Re: Find documents that are composed of % words

2013-10-14 Thread Chris Hostetter
: bq: but you cannot ask this to client. : : You _can_ ask this of a client. IMO you are obligated to. +1. >> When you are given a requirement/request from your client, >> always verify that you aren't dealing with an XY Problem: >> http://people.apache.org/~hossman/#xyproblem ... >> Don'

Re: Concurent indexing

2013-10-15 Thread Chris Geeringh
I can confirm I am seeing the same issue with Tomcat - cluster split over 4 nodes. Is this fix in a 4.6 snapshot? On 15 October 2013 08:28, maephisto wrote: > Thanks for the tip! > > I must mention that I am using Solr 4.4.0 and this problem only appears > when > i'm doing the indexing in the

Re: loading SolrInfoMBeanHandler is slow?

2013-10-15 Thread Chris Hostetter
: In my Mac OSX when starting Solr as OOTB, : I always have to wait 30 sec for completely loading. : It seems that loading SolrInfoMBeanHandler is slow in Solr 4.x (also 4.5). Are you certain the problem is coming from SolrInfoMBeanHandler and not one of the other admin handler rleated classes?

Re: Debugging update request

2013-10-16 Thread Chris Geeringh
I ran an import last night, and this morning my cloud wouldn't accept updates. I'm running the latest 4.6 snapshot. I was importing with latest solrj snapshot, and using java bin transport with CloudSolrServer. The cluster had indexed ~1.3 million docs before no further updates were accepted, quer

Re: Debugging update request

2013-10-16 Thread Chris Geeringh
Here is my jstack output... Lots of blocked threads. http://pastebin.com/1ktjBYbf On 16 October 2013 10:28, michael.boom wrote: > I got the trace from jstack. > I found references to "semaphore" but not sure if this is what you meant. > Here's the trace: > http://pastebin.com/15QKAz7U > > > >

Re: SolrCloud Query Balancing

2013-10-16 Thread Chris Geeringh
If your web application is using SolrJ/Java based - use a CloudSolrServer instance with the zkHosts. It will take care of load balancing when querying, indexing, and handle routing if a node goes down. On 16 October 2013 10:52, michael.boom wrote: > I have setup a SolrCloud system with: 3 shard

Re: Concurent indexing

2013-10-16 Thread Chris Geeringh
Hi Erick, here is a paste from other thread (debugging update request) with my input as I am seeing errors too: I ran an import last night, and this morning my cloud wouldn't accept updates. I'm running the latest 4.6 snapshot. I was importing with latest solrj snapshot, and using java bin transpo

Re: Concurent indexing

2013-10-16 Thread Chris Geeringh
Here's another jstack http://pastebin.com/8JiQc3rb On 16 October 2013 11:53, Chris Geeringh wrote: > Hi Erick, here is a paste from other thread (debugging update request) > with my input as I am seeing errors too: > > I ran an import last night, and this morning my clou

Re: Skipping caches on a /select

2013-10-17 Thread Chris Hostetter
: Does "cache=false" apply to all caches? The docs make it sound like it is for : filterCache only, but I could be misunderstanding. it's per *query* -- not per cache, or per request... /select?q={!cache=true}foo&fq={!cache=false}bar&fq={!cache=true}baz ...should cause 1 lookup/insert in the

Re: Concurent indexing

2013-10-18 Thread Chris Geeringh
Erick, yes. Using SolrJ and CloudSolrServer - both 4.6 snapshots from 13 Oct On 18 October 2013 12:17, Erick Erickson wrote: > Chris: > > OK, one of those stack traces does have the problem I referenced in the > other thread. Are you sending updates to the server with SolrJ?

Seeking New Moderators for solr-user@lucene

2013-10-18 Thread Chris Hostetter
It looks like it's time to inject some fresh blood into the solr-user@lucene moderation team. If you'd like to volunteer to be a moderator, please reply back to this thread and specify which email address you'd like to use as a moderator (if different from the one you use when sending the em

Re: Check if dynamic columns exists and query else ignore

2013-10-18 Thread Chris Hostetter
: I trying to do this: : : if (US_offers_i exists): :fq=US_offers_i:[1 TO *] : else: :fq=offers_count:[1 TO *] "if()" and "exist()" are functions, so you would have to explicitly use them in a function context (ie: {!func} parser, or {!frange} parser) and to use those nested queries i

Re: Questions developing custom functionquery

2013-10-18 Thread Chris Hostetter
: Field-Type: org.apache.solr.schema.TextField ... : DocTermsIndexDocValues. : Calling "getVal()" on a Do

RE: Facet performance

2013-10-18 Thread Chris Hostetter
: >> 1. q=word&facet.field=CONTENT&facet=true&facet.prefix=&facet.limit=10&facet.mincount=1&facet.method=enum&rows=0 : >> 2. q=word&facet.field=CONTENT&facet=true&facet.prefix=a&facet.limit=10&facet.mincount=1&facet.method=enum&rows=0 : > : >> The only difference is am empty facet.prefix in the

Re: Solr 4.6.0 latest build

2013-10-22 Thread Chris Geeringh
snapshots have not resolved the issue for me. Cheers, Chris On Tuesday, October 22, 2013, Rafał Kuć wrote: > Hello! > > The current development version of Solr can be found in the SVN > repository - https://lucene.apache.org/solr/versioncontrol.html > > You need to download th

Re: Solr 4.6.0 latest build

2013-10-22 Thread Chris Geeringh
che.org/job/Solr-Artifacts-4.x/lastSuccessfulBuild/artifact/solr/package/ > > > I have an issue with updates no longer being processed, and the snapshots > > have not resolved the issue for me. > > > Cheers, > > Chris > > > On Tuesday, October 22, 2013,

Re: SolrCloud frequently hanging

2013-10-22 Thread Chris Geeringh
icket open, please add any details to it https://issues.apache.org/jira/browse/SOLR-5364 Cheers, Chris On 22 October 2013 08:25, Prasi S wrote: > Hi all, > We are using solrcloud 4.4 (solrcloud with external zookeeper, 2 tomcats , > 2 solr- 1 in each tomcat) for indexing delimited files

Re: Seeking New Moderators for solr-user@lucene

2013-10-22 Thread Chris Hostetter
: It looks like it's time to inject some fresh blood into the solr-user@lucene : moderation team. Wow ... big thanks to everyone who replied. We've definitely never had that many volunteers before. Since Infra has previously suggested that "7" is "too high" a number moderators to have for a l

Re: Indexing logs files of thousands of GBs

2013-10-23 Thread Chris Geeringh
"walk" post up to the server with SolrJ. Cheers, Chris On 22 October 2013 18:58, keshari.prerna wrote: > Hello, > > I am tried to index log files (all text data) stored in file system. Data > can be as big as 1000 GBs or more. I am working on windows. > > A sample

Re: Solr facet field counts not correct

2013-10-23 Thread Chris Hostetter
: if I do group=false&group.facet=false the counts are what they should be for : the ungrouped counts... seems like group.facet isn't working correctly yeah ... thanks for digging int -- definitely seems like a problem with group.facet and Trie fields that use precisionStep. I've opened a Jira:

Re: Multiple facet fields in "defaults" section of a Request Handler

2013-10-24 Thread Chris Hostetter
: Now a client wants to use multi select faceting. He calls the following API: : http://localhost:8983/solr/collection1/search?q=*:*&facet.field={!ex=foo}category&fq={!tag=foo}category : :"cat" : Putting the facet definitions in "appends" cases it to facet category 2 : times. : : Is there a way

Re: measure result set quality

2013-10-24 Thread Chris Hostetter
: As a first approach I will evaluate (manually :( ) hits that are out of the : intersection set for every query in each system. Anyway I will keep FYI: LucidWorks has a "Relevancy Workbench" tool that serves as a simple UI designed explicitly for the purpose of comparing the result sets of fro

Re: Post filter cache question

2013-10-24 Thread Chris Hostetter
: Could it be a problem with my cache settings in solrconfig.xml (solr 3.1) : or is my query wrong? 3.1? ouch ... PostFilter wasn't even added until 3.4... https://wiki.apache.org/solr/CommonQueryParameters#Caching_of_filters ...so your spatial filter is definitely being applied to the entire in

Re: Reclaiming disk space from (large, optimized) segments

2013-10-24 Thread Chris Hostetter
I didn't dig into the details of your mail too much, but a few things jumped out at me... : - At some time in the past, a manual force merge / optimize with : maxSegments=2 was run to troubleshoot high disk i/o and remove "too many Have you tried a simple commit using expungeDeletes=true? It s

Re: Solr 4.5.1 and Illegal to have multiple roots (start tag in epilog?). (perhaps SOLR-4327 bug?)

2013-10-25 Thread Chris Geeringh
I'll close that ticket. Hope that helps. Cheers, Chris On 25 October 2013 03:48, Michael Tracey wrote: > Hey Solr-users, > > I've got a single solr 4.5.1 node with 96GB ram, a 65GB index (105 million > records) and a lot of daily churn of newly indexed files (auto soft

Re: SolrCloud frequently hanging

2013-10-25 Thread Chris Geeringh
Prasi, as per the ticket I linked to earlier, I was running into GC settings. May be worth investigating - and take a look at the GC settings I'm running with in the ticket. Cheers, Chris On 22 October 2013 10:25, Prasi S wrote: > bq: ...three different files each with a partial set &

Re: Queries with conditional field inclusions?

2013-10-25 Thread Chris Hostetter
: Search Department Name + Last Name every time : include First Name if user type in (faculty, staff) or if another token : matched last name. I haven't tested this, but i think conceptually what you want is... qq=richard frovarp q={!maxscore v=$maxof} maxof=({!dismax qf='lastname deptname' mm='

Re: Solr search in case the first keyword are not index

2013-10-25 Thread Chris Hostetter
: But when the first keyword is not index, and other keywords are index, solr : can not query it. : (example I search with: apacheee solr reference). run your queries in a browser and add "debug=query" to see how your query is being parsed. you most likely are seeing the results of the parser d

Re: DIH - URLDataSource import size

2013-10-25 Thread Chris Hostetter
: I have an issue that is only coming on live environment. The DIH : with URLDataSource is not working when the file size imported is large : (i.e. 100kb above - which is not so large). If its large, it returns : nothing (as seen in the Debug section of DataImport at Solr Admin). are you sure the

Re: Solr - Read sort data from external source

2013-10-25 Thread Chris Hostetter
: I am trying to write some code to read rank data from external db, I saw : some example done using database - : http://sujitpal.blogspot.com/2011/05/custom-sorting-in-solr-using-external.html, ... : But is there any way to pass parameter or choose different database during : FieldCompar

Re: Proposal for new feature, cold replicas, brainstorming

2013-10-25 Thread Chris Hostetter
: > I know that right now is not possible, but I think that it's a feature that : > can be implemented in a easy way creating a new status for shards. : : shardIsLastResort=true? On paper it seems like a simple addition, but I : am not at familiar enough with the SolrCloud-code to guess if it is

Re: Solr search in case the first keyword are not index

2013-10-25 Thread Chris Hostetter
: "debug": { : "rawquerystring": "display_add_level1: apache solr reference", : "querystring": "display_add_level1: apache solr reference", : "parsedquery": "display_add_level1:apache text:solr text:reference", : "parsedquery_toString": "display_add_level1:apache text:solr : text:

ANNOUNCE: Stump The Chump @ Lucene Revolution EU - Tommorrow

2013-11-05 Thread Chris Hostetter
(Note: cross posted announcement, please confine any replies to solr-user) Hey folks, On Wednesday, I'll be doing a "Stump The Chump" session at Lucene Revolution EU in Dublin Ireland. http://lucenerevolution.org/stump-the-chump If you aren't familiar with "Stump The Chump" it is a Q&A s

Re: unable to load core after cluster restart

2013-11-06 Thread Chris Geeringh
--- In the case of Java 7, 1.7.0_40 and 1.7.0_45 have known problems with Lucene/Solr. Shawn, this is interesting. What are the problems, where are the documented? On 6 November 2013 20:18, kaustubh147 wrote: > Hi, > > Here is my > solr.xml > > > > > > > ${host:} > 28081 > /sol

Range faceting or grouping on a String or count(field)

2013-11-08 Thread Chris Geeringh
fanta: 1 ... What are the suggestions to achieve something like this? Cheers, Chris

Re: queries including time zone

2013-11-14 Thread Chris Hostetter
: Can anybody provide any insight about using the tz param? The behavior : of this isn't affecting date math and /day rounding. What format does : the tz variables need to be in? Not finding any documentation on this. it's not "tz" it's "TZ" The input/output format is always in UTC, but TZ w

Re: Boosting documents by categorical preferences

2013-11-14 Thread Chris Hostetter
: I have a question around boosting. I wanted to use the &boost= to write a : nested query that will boost a document based on categorical preferences. You have no idea how stoked I am to see you working on this in a real world application. : Currently I have the weights set to the z-score equi

Re: queries including time zone

2013-11-14 Thread Chris Hostetter
I've beefed up the ref guide page on dates to include more info about all of this... https://cwiki.apache.org/confluence/display/solr/Working+with+Dates -Hoss

Re: Date range faceting with various gap sizes?

2013-11-14 Thread Chris Hostetter
: I'm experimenting with date range faceting, and would like to use : different gaps depending on how old the date is. But I am not sure on : how to do that. What you are trying to do is possible, but the SolrJ helper methods you are using predates the ability and doesn't currently work the w

Re: SolrCoreAware

2013-11-15 Thread Chris Hostetter
: So its something that can happen multiple times during the lifetime of : process, but i'm guessing something not occuring very often? it should be called only once during hte lifetime of a given plugin, usually not long after construction -- but it could be called many, many times in the life

Re: SolrCoreAware

2013-11-15 Thread Chris Hostetter
: So for a given instance of a handler it will only be called once during the : lifetime of that handler? correct (unless there is a bug somewhere) : Also, when the core is passed in as part of inform() is it guaranteed to be : ready to go? (ie I can start feeding content at this point?) Right,

Re: Parallelizing warmup queries

2013-11-18 Thread Chris Hostetter
: It looks like warmup queries execute sequentially. : : Considering servers have N CPU cores these days, would it make sense : to make them (optionally) run in parallel? This should help with : cases where warmup queries are CPU bound by letting Solr use more than : 1 thread and thus more than

Re: queries including time zone

2013-11-18 Thread Chris Hostetter
: We're still not seeing the proper result. I've included a gist of the : query and its debug result. This was run on a clean index running 4.4.0 : with just one document. That document has a date of 11/15/2013 yet the : date in the included TZ it is the 14th but I still get that document :

Re: Is this a reasonable way to boost?

2013-11-18 Thread Chris Hostetter
: > I'm trying to boost results slightly on a price (not currency) field that : > are closer to a certain value. I want results that are not too expensive : > or too inexpensive to be favored. Here is what we currently are trying: : > : > bf=sub(1,abs(sub(15,price)))^0.2 Hmm... using sub() for

Re: Boosting documents by categorical preferences

2013-11-19 Thread Chris Hostetter
: My approach was something like: : 1) Look at the categories that the user has preferred and compute the : z-score : 2) Pick the top 3 among those : 3) Use those to boost search results. I think that totaly makes sense ... the additional bit i was suggesting that you consider is that instead of

<    1   2   3   4   5   6   7   8   9   10   >