Strange relevance scoring

2014-04-08 Thread John Nielsen
in the text and the first word of the text is any other word. Is this normal behavior? Is special attention paid to the first word in a text field? I would think that the latter case would get the highest score. -- Med venlig hilsen / Best regards *John Nielsen* Programmer *MCB A/S*

Re: Strange relevance scoring

2014-04-08 Thread John Nielsen
o special attention paid to first word. You are probably hitting > length normalisation. > Lucene/Solr punishes long documents, favours short documents. > (5 times appearing one) longer? > > > > On Tuesday, April 8, 2014 12:03 PM, John Nielsen wrote: > Hi, > > We are seeing a

Re: Strange relevance scoring

2014-04-08 Thread John Nielsen
first word of the text is any other word. > > > > Is this normal behavior? Is special attention paid to the first word in a > > text field? I would think that the latter case would get the highest > score. > > > > > > -- > > Med venlig hilsen / Best

q and logical operators.

2014-09-11 Thread John Nielsen
erstand how to construct my query correctly, an RTFM pointer will be most welcome! -- Med venlig hilsen / Best regards *John Nielsen* Programmer *MCB A/S* Enghaven 15 DK-7500 Holstebro Kundeservice: +45 9610 2824 p...@mcb.dk www.mcb.dk

Re: q and logical operators.

2014-09-12 Thread John Nielsen
longer > returns anything. This is not an option for me > > Would slop help here? i.e. "vis dur dis"~3 or some such? > > Best > Erick > > On Thu, Sep 11, 2014 at 4:34 AM, John Nielsen wrote: > > q and logical operators. > > > > Hi all, > >

Re: q and logical operators.

2014-09-15 Thread John Nielsen
e slop is something of a balancing act > > Best, > Erick > > On Fri, Sep 12, 2014 at 2:10 AM, John Nielsen wrote: > > I didn't know about sloppy queries. This is great stuff! > > > > I solved it with a &qs=100. > > > > Thank you for the help. &

Re: Facets with 5000 facet fields

2013-03-21 Thread John Nielsen
ng method that may have some better nrt characteristics than fcs. > I have not played with it yet but hope to soon. > > - Mark > > -- Med venlig hilsen / Best regards *John Nielsen* Programmer *MCB A/S* Enghaven 15 DK-7500 Holstebro Kundeservice: +45 9610 2824 p...@mcb.dk www.mcb.dk

Re: SOLR - Documents with large number of fields ~ 450

2013-03-22 Thread John Nielsen
"with the on disk option". Could you elaborate on that? Den 22/03/2013 05.25 skrev "Mark Miller" : > You might try using docvalues with the on disk option and try and let the > OS manage all the memory needed for all the faceting/sorting. This would > require Solr 4.2. > > - Mark > > On Mar 21, 2

Solr using a ridiculous amount of memory

2013-03-24 Thread John Nielsen
field cache for anything would also (probably) work for me. I thought about killing off my other caches, but from the dumps, they just don't seem to use that much memory. I am at my wits end. Any help would be sorely appreciated. -- Med venlig hilsen / Best regards *John Nielsen* Pr

Re: Solr using a ridiculous amount of memory

2013-03-25 Thread John Nielsen
; On Sun, Mar 24, 2013 at 4:19 AM, John Nielsen wrote: > > > Schema with DocValues attempt at solving problem: > > http://pastebin.com/Ne23NnW4 > > Config: http://pastebin.com/x1qykyXW > > > > This schema isn't using docvalues, due to a typo in your config. >

Re: Solr using a ridiculous amount of memory

2013-04-15 Thread John Nielsen
that I will be able to solve this issue eventually. On Mon, Apr 15, 2013 at 9:00 AM, Toke Eskildsen wrote: > On Sun, 2013-03-24 at 09:19 +0100, John Nielsen wrote: > > Our memory requirements are running amok. We have less than a quarter of > > our customers running now and

Re: Solr using a ridiculous amount of memory

2013-04-15 Thread John Nielsen
. On Mon, Apr 15, 2013 at 1:38 PM, Toke Eskildsen wrote: > On Mon, 2013-04-15 at 10:25 +0200, John Nielsen wrote: > > > The FieldCache is the big culprit. We do a huge amount of faceting so > > it seems right. > > Yes, you wrote that earlier. The mystery is that the math

Re: Solr using a ridiculous amount of memory

2013-04-17 Thread John Nielsen
I must be calculating this wrong. On Mon, Apr 15, 2013 at 2:10 PM, John Nielsen wrote: > I did a search. I have no occurrence of "UnInverted" in the solr logs. > > > Another explanation for the large amount of memory presents itself if > > you use a single index: If

Re: Solr using a ridiculous amount of memory

2013-04-17 Thread John Nielsen
d9oIbGLCFQwl I would prefer to not turn off the indexer unless the numbers above suggests that I really should try this. Waiting for a full GC would take a long time. Unfortunately I don't know of a way to provoke a full GC on command. On Wed, Apr 17, 2013 at 11:48 AM, Toke Eskildsen wr

Re: Solr using a ridiculous amount of memory

2013-04-17 Thread John Nielsen
and unless I'm wrong, I don't think that it's possible. I am quite stomped on how to fix this. On Wed, Apr 17, 2013 at 3:06 PM, Toke Eskildsen wrote: > John Nielsen [j...@mcb.dk]: > > I never seriously looked at my fieldValueCache. It never seemed to get > used: >

Re: Solr using a ridiculous amount of memory

2013-04-18 Thread John Nielsen
the fq queries are for. So if i generate a core for each client, I would have a client specific fieldCache containing the data from that client. Wouldn't I just split up the same data into several cores? I'm afraid I don't understand how this would help. -- Med venlig hilsen / Best regards *John Nielsen* Programmer *MCB A/S* Enghaven 15 DK-7500 Holstebro Kundeservice: +45 9610 2824 p...@mcb.dk www.mcb.dk

Re: Solr using a ridiculous amount of memory

2013-04-18 Thread John Nielsen
dropped for me. I will definetely move to a multi-core setup. It will take some time and a lot of re-coding. As soon as I know the result, I will let you know! -- Med venlig hilsen / Best regards *John Nielsen* Programmer *MCB A/S* Enghaven 15 DK-7500 Holstebro Kundeservice: +45 9610 2

Re: SolrCloud loadbalancing, replication, and failover

2013-04-19 Thread John Nielsen
times. > Although Google's software probably works differently than Solr/Lucene, > when > it comes right down to it, all search engines do similar jobs and have > similar requirements. I would imagine that Google gets incredible response > time because they have incredib

Re: Solr using a ridiculous amount of memory

2013-06-14 Thread John Nielsen
to the list of wiki page contributors and help out!) > > Best > Erick > > On Thu, Apr 18, 2013 at 8:31 AM, John Nielsen wrote: > >> You are missing an essential part: Both the facet and the sort > >> structures needs to hold one reference for each document >

Searching for cache stats

2013-06-17 Thread John Nielsen
regards John Nielsen Programmer MCB A/S Enghaven 15 DK-7500 Holstebro Kundeservice: +45 9610 2824 p...@mcb.dk www.mcb.dk

Re: Informal poll on running Solr 4 on Java 7 with G1GC

2013-06-20 Thread John Nielsen
hilsen / Best regards John Nielsen Programmer MCB A/S Enghaven 15 DK-7500 Holstebro Kundeservice: +45 9610 2824 p...@mcb.dk www.mcb.dk On 20-06-2013 00:18, Timothy

SOLR4 cluster - strange CPU spike on slave

2012-11-23 Thread John Nielsen
spike did not disappear. The spike we are seeing is on varnish01 only. Please note that our use case requires us to continuously feed large amounts of data from our main system in the order of up to 1.000 registers every minute. Has anyone seen this phenomenon before? Best regards John Nielsen

Re: SOLR4 cluster - strange CPU spike on slave

2012-11-28 Thread John Nielsen
d 5 secs. Not the couple of minutes the spike lasts. I am really at a complete loss as to what this could be. Google has not given me any clues. Med venlig hilsen / Best regards *John Nielsen* Programmer *MCB A/S* Enghaven 15 DK-7500 Holstebro Kundeservice: +45 9610 2824 p...@mcb.dk www.mcb.d

Re: SOLR4 cluster - strange CPU spike on slave

2012-11-28 Thread John Nielsen
eliminate possibilities. It just occurred to me now; We tried switching off our feeder/index tool for 24 hours, and we didn't see any spikes during this period, so receiving replication data certainly has something to do with it. Med venlig hilsen / Best regards *John Nielsen* Programmer *MC

Re: SOLR4 cluster - strange CPU spike on slave

2012-12-04 Thread John Nielsen
e switched to using that from NRTCachingDirectory and am monitoring performance as well. Initially performance doesn't look stellar, but i suspect that we lack memory in the server to really make it shine. Med venlig hilsen / Best regards *John Nielsen* Programmer *MCB A/S* Enghaven 15 DK

Re: SOLR4 cluster - strange CPU spike on slave

2012-12-05 Thread John Nielsen
p on these? Med venlig hilsen / Best regards *John Nielsen* Programmer *MCB A/S* Enghaven 15 DK-7500 Holstebro Kundeservice: +45 9610 2824 p...@mcb.dk www.mcb.dk On Wed, Dec 5, 2012 at 1:11 AM, Mark Miller wrote: > > On Dec 4, 2012, at 2:25 AM, John Nielsen wrote: > > > The po

Re: SOLR4 cluster - strange CPU spike on slave

2012-12-05 Thread John Nielsen
109375MB) 53.347981109627995% used Med venlig hilsen / Best regards *John Nielsen* Programmer *MCB A/S* Enghaven 15 DK-7500 Holstebro Kundeservice: +45 9610 2824 p...@mcb.dk www.mcb.dk On Thu, Nov 29, 2012 at 4:08 AM, Otis Gospodnetic < otis.gospodne...@gmail.com> wrote: > If this i

Strange data-loss problem on one of our cores

2012-12-13 Thread John Nielsen
group_45879_combination:(*)&fq=is_searchable:(True)&querytype=Technical&mm=100%25&facet.missing=on&group.ngroups=true&facet.mincount=1&qf=%0a++text^0.5+name^1.2+searchable_text^0.8+typeahead_text^1.0+keywords^1.1+item_no^5.0%0a++++++ranking1_text^1.0+ranking

Re: Strange data-loss problem on one of our cores

2012-12-14 Thread John Nielsen
next week. Thanks for your help. -- Med venlig hilsen / Best regards *John Nielsen* Programmer *MCB A/S* Enghaven 15 DK-7500 Holstebro Kundeservice: +45 9610 2824 p...@mcb.dk www.mcb.dk On Thu, Dec 13, 2012 at 4:10 PM, Mark Miller wrote: > Couple things to start: > > B

Re: Strange data-loss problem on one of our cores

2012-12-14 Thread John Nielsen
How did you solve the problem? -- Med venlig hilsen / Best regards *John Nielsen* Programmer *MCB A/S* Enghaven 15 DK-7500 Holstebro Kundeservice: +45 9610 2824 p...@mcb.dk www.mcb.dk On Fri, Dec 14, 2012 at 12:04 PM, Markus Jelsma wrote: > FYI, we observe the same issue, after s

Re: Strange data-loss problem on one of our cores

2012-12-14 Thread John Nielsen
arameter I can add to the GET requests that will lock the request to a specific node in the cluster, treating the server receiving the request as a standalone server as opposed to a member of a cluster? I tried googeling it without luck. -- Med venlig hilsen / Best regards *John Nielsen*

Re: Strange data-loss problem on one of our cores

2012-12-14 Thread John Nielsen
Awesome! http://host:port/solr/admin/cores is exactly what i needed! -- Med venlig hilsen / Best regards *John Nielsen* Programmer *MCB A/S* Enghaven 15 DK-7500 Holstebro Kundeservice: +45 9610 2824 p...@mcb.dk www.mcb.dk On Fri, Dec 14, 2012 at 1:21 PM, Markus Jelsma wrote: >

Re: Strange data-loss problem on one of our cores

2012-12-18 Thread John Nielsen
I build a solr version from the solr-4x branch yesterday and so far am unable to replicate the problems i had before. I am cautiously optimistic that the problem has been resolved. If i run into any more problems, I'll let you all know. -- Med venlig hilsen / Best regards *John Ni

Re: Data from deleted from Solr (Solr cloud)

2012-12-20 Thread John Nielsen
st regards *John Nielsen* Programmer *MCB A/S* Enghaven 15 DK-7500 Holstebro Kundeservice: +45 9610 2824 p...@mcb.dk www.mcb.dk On Wed, Dec 19, 2012 at 5:08 PM, shreejay wrote: > Hi All, > > I have a solrlcoud instance with 3 shards. Each shard has 2 instance (2 > servers each runn