Highlighting bug with edgegrams

2014-04-09 Thread Software Dev
In 3.5.0 we have the following. If we searched for "c" with highlighting enabled we would get back results such as: cdat crocdile cool beans But in the latest Solr (4.7.1) we get the full words highlighted back.

Re: Question on highlighting edgegrams

2014-03-28 Thread Software Dev
, 2014 at 10:17 AM, Software Dev > wrote: >> Certainly I am not the only user experiencing this? >> >> On Wed, Mar 26, 2014 at 1:11 PM, Software Dev >> wrote: >>> Is this a known bug? >>> >>> On Tue, Mar 25, 2014 at 1:12 PM, Software Dev &g

Re: Question on highlighting edgegrams

2014-03-27 Thread Software Dev
Certainly I am not the only user experiencing this? On Wed, Mar 26, 2014 at 1:11 PM, Software Dev wrote: > Is this a known bug? > > On Tue, Mar 25, 2014 at 1:12 PM, Software Dev > wrote: >> Same problem here: >> http://lucene.472066.n3.nabble.com/Solr-4-x-Ed

What are my options?

2014-03-27 Thread Software Dev
We have a collection named "items". These are simply products that we sell. A large part of our scoring involves boosting on certain metrics for each product (amount sold, total GMS, ratings, etc). Some of these metrics are actually split across multiple tables. We are currently re-indexing the co

Re: Question on highlighting edgegrams

2014-03-26 Thread Software Dev
Is this a known bug? On Tue, Mar 25, 2014 at 1:12 PM, Software Dev wrote: > Same problem here: > http://lucene.472066.n3.nabble.com/Solr-4-x-EdgeNGramFilterFactory-and-highlighting-td4114748.html > > On Tue, Mar 25, 2014 at 9:39 AM, Software Dev > wrote: >> Bump >> &

What contributes to disk IO?

2014-03-25 Thread Software Dev
What are the main contributing factors for Solr Cloud generating a lot of disk IO? A lot of reads? Writes? Insufficient RAM? I would think if there was enough disk cache available for the whole index there would be little to no disk IO.

Re: Question on highlighting edgegrams

2014-03-25 Thread Software Dev
Same problem here: http://lucene.472066.n3.nabble.com/Solr-4-x-EdgeNGramFilterFactory-and-highlighting-td4114748.html On Tue, Mar 25, 2014 at 9:39 AM, Software Dev wrote: > Bump > > On Mon, Mar 24, 2014 at 3:00 PM, Software Dev > wrote: >> In 3.5.0 we

Re: Replication (Solr Cloud)

2014-03-25 Thread Software Dev
"In older versions it might have done them all at once, but I believe that newer versions only do one core at a time." It looks like it did it all at once and I'm on the latest (4.7) On Tue, Mar 25, 2014 at 11:27 AM, Software Dev wrote: > So its generally a bad idea to

Re: Replication (Solr Cloud)

2014-03-25 Thread Software Dev
So its generally a bad idea to optimize I gather? - In older versions it might have done them all at once, but I believe that newer versions only do one core at a time. On Tue, Mar 25, 2014 at 11:16 AM, Shawn Heisey wrote: > On 3/25/2014 11:59 AM, Software Dev wrote: >> >> Ehh.

Re: Replication (Solr Cloud)

2014-03-25 Thread Software Dev
Ehh.. found out the hard way. I optimized the collection on 1 machine and when it was completed it replicated to the others and took my cluster down. Shitty On Tue, Mar 25, 2014 at 10:46 AM, Software Dev wrote: > One other question. If I optimize a collection on one node, does this &g

Re: Replication (Solr Cloud)

2014-03-25 Thread Software Dev
One other question. If I optimize a collection on one node, does this get replicated to all others when finished? On Tue, Mar 25, 2014 at 10:13 AM, Software Dev wrote: > Thanks for the reply. Ill make sure NOT to disable it.

Re: Solr Cloud collection keep going down?

2014-03-25 Thread Software Dev
Can anyone else chime in? Thanks On Mon, Mar 24, 2014 at 10:10 AM, Software Dev wrote: > Shawn, > > Thanks for pointing me in the right direction. After consulting the > above document I *think* that the problem may be too large of a heap > and which may be affecting GC colle

Re: Replication (Solr Cloud)

2014-03-25 Thread Software Dev
Thanks for the reply. Ill make sure NOT to disable it.

Replication (Solr Cloud)

2014-03-25 Thread Software Dev
I see that by default in SolrCloud that my collections are replicating. Should this be disabled in SolrCloud as this is already handled by it? >From the documentation: "The Replication screen shows you the current replication state for the named core you have specified. In Solr, replication is fo

Re: Question on highlighting edgegrams

2014-03-25 Thread Software Dev
Bump On Mon, Mar 24, 2014 at 3:00 PM, Software Dev wrote: > In 3.5.0 we have the following. > > positionIncrementGap="100"> > > > > maxGramSize="30"/> > > > > &

Question on highlighting edgegrams

2014-03-24 Thread Software Dev
In 3.5.0 we have the following. If we searched for "c" with highlighting enabled we would get back results such as: cdat crocdile cool beans But in the latest Solr (4.7) we get the full words highlighted back. Di

Re: Solr Cloud collection keep going down?

2014-03-24 Thread Software Dev
reporting as well as some relevant portions of our SolrConfig.xml. Any thoughts/comments would be greatly appreciated. http://postimg.org/gallery/4t73sdks/1fc10f9c/ Thanks On Sat, Mar 22, 2014 at 2:26 PM, Shawn Heisey wrote: > On 3/22/2014 1:23 PM, Software Dev wrote: >> We have 2 collect

Re: Solr Cloud collection keep going down?

2014-03-22 Thread Software Dev
a:182) at org.eclipse.jetty.http.HttpGenerator.flushBuffer(HttpGenerator.java:838) ... 51 more ,code=500} On Sat, Mar 22, 2014 at 12:23 PM, Software Dev wrote: > We have 2 collections with 1 shard each replicated over 5 servers in the > cluster. We see a lot of flapping (down or recoverin

Solr Cloud collection keep going down?

2014-03-22 Thread Software Dev
We have 2 collections with 1 shard each replicated over 5 servers in the cluster. We see a lot of flapping (down or recovering) on one of the collections. When this happens the other collection hosted on the same machine is still marked as active. When this happens it takes a fairly long time (~30

Re: Help me understand these newrelic graphs

2014-03-17 Thread Software Dev
ring * Log Analytics * Search Analytics > Solr & Elasticsearch Support * http://sematext.com/ > > > On Fri, Mar 14, 2014 at 1:07 PM, Software Dev >wrote: > > > Here is a screenshot of the host information: > > http://postimg.org/image/vub5ihxix/ > > &

Re: Help me understand these newrelic graphs

2014-03-14 Thread Software Dev
Here is a screenshot of the host information: http://postimg.org/image/vub5ihxix/ As you can see we have 24 core CPU's and the load is only at 5-7.5. On Fri, Mar 14, 2014 at 10:02 AM, Software Dev wrote: > If that is the case, what would help? > > > On Thu, Mar 13, 2014

Re: Help me understand these newrelic graphs

2014-03-14 Thread Software Dev
high number of > concurrent queries than sharding may not be of any help at all. > > Otis > -- > Performance Monitoring * Log Analytics * Search Analytics > Solr & Elasticsearch Support * http://sematext.com/ > > > On Thu, Mar 13, 2014 at 7:42 PM, Software Dev >wrote:

Re: Help me understand these newrelic graphs

2014-03-13 Thread Software Dev
g the average response for an add > > operation, which generally returns very quickly and due to sheer number > are > > averaging out the response time of your queries. New Relic should break > > out requests based on which handler they're hitting but they don't s

Help me understand these newrelic graphs

2014-03-13 Thread Software Dev
Here are some screen shots of our Solr Cloud cluster via Newrelic http://postimg.org/gallery/2hyzyeyc/ We currently have a 5 node cluster and all indexing is done on separate machines and shipped over. Our machines are running on SSD's with 18G of ram (Index size is 8G). We only have 1 shard at t

How does Solr parse schema.xml?

2014-02-26 Thread Software Dev
Can anyone point me in the right direction. I'm trying to duplicate the functionality of the analysis request handler so we can wrap a service around it to return the terms given a string of text. We would like to read the same schema.xml file to configure the analyzer,tokenizer, etc but I can't se

Re: SolrCloudServer questions

2014-02-01 Thread Software Dev
Also, if we are seeing a huge cpu spike on the leader when doing a bulk index, would changing any of the options help? On Sat, Feb 1, 2014 at 2:59 PM, Software Dev wrote: > Out use case is we have 3 indexing machines pulling off a kafka queue and > they are all sending individual u

Re: SolrCloudServer questions

2014-02-01 Thread Software Dev
ds, or if > you need more fine grained responses, use the single add from multiple > threads (though bulk add can also be done via multiple threads if you > really want to try and push the max). > > - Mark > > http://about.me/markrmiller > > On Jan 31, 2014, at 3:50 PM, S

Re: SolrCloudServer questions

2014-01-31 Thread Software Dev
shards in > parallel rather than with a single thread. Can really increase update > speed. Still not as powerful as using CloudSolrServer from multiple > threads, but a nice improvement non the less. > > > - Mark > > http://about.me/markrmiller > > > > > I&#

Disabling Commit/Auto-Commit (SolrCloud)

2014-01-31 Thread Software Dev
Is there a way to disable commit/hard-commit at runtime? For example, we usually have our hard commit and soft-commit set really low but when we do bulk indexing we would like to disable this to increase performance. If there isn't a an easy way of doing this would simply pushing a new solrconfig t

SolrCloudServer questions

2014-01-31 Thread Software Dev
Can someone clarify what the following options are: - updatesToLeaders - shutdownLBHttpSolrServer - parallelUpdates Also, I remember in older version of Solr there was an efficient format that was used between SolrJ and Solr that is more compact. Does this sill exist in the latest version of Solr

Re: Solr Cloud Bulk Indexing Questions

2014-01-23 Thread Software Dev
/search-lucene.com/?q=maxWriteMBPerSec&fc_project=Solr > > Otis > -- > Performance Monitoring * Log Analytics * Search Analytics > Solr & Elasticsearch Support * http://sematext.com/ > > > On Mon, Jan 20, 2014 at 4:00 PM, Software Dev >wrote: > > >

Re: Solr Cloud Bulk Indexing Questions

2014-01-23 Thread Software Dev
Also, any suggestions on debugging? What should I look for and how? Thanks On Thu, Jan 23, 2014 at 10:01 AM, Software Dev wrote: > Thanks for suggestions. After reading that document I feel even more > confused though because I always thought that hard commits should be less > freq

Re: Solr Cloud Bulk Indexing Questions

2014-01-23 Thread Software Dev
t consequences: > > > http://searchhub.org/2013/08/23/understanding-transaction-logs-softcommit-and-commit-in-sorlcloud/ > > I suspect (but, of course, can't prove) that you're over-committing > and hitting segment > merges without meaning to... > > FWIW, > E

Re: Solr Cloud Bulk Indexing Questions

2014-01-22 Thread Software Dev
is a change. > How much system RAM ? JVM Heap ? Enough space in RAM for system disk cache > ? > What is the size of your documents ? A few KB, MB, ... ? > Ah, and what about network IO ? Could that be a limiting factor ? > > > André > > > On 2014-01-21 23:40, Software De

Re: Solr Cloud Bulk Indexing Questions

2014-01-21 Thread Software Dev
Any other suggestions? On Mon, Jan 20, 2014 at 2:49 PM, Software Dev wrote: > 4.6.0 > > > On Mon, Jan 20, 2014 at 2:47 PM, Mark Miller wrote: > >> What version are you running? >> >> - Mark >> >> On Jan 20, 2014, at 5:43 PM, Software Dev >> wr

Re: Removing a node from Solr Cloud

2014-01-21 Thread Software Dev
solr/CoreAdmin#UNLOAD. > > > On Tue, Jan 21, 2014 at 10:22 AM, Software Dev >wrote: > > > What is the process for completely removing a node from Solr Cloud? We > > recently removed one but t its still showing up as "Gone" in the Cloud > > adm

Setting leaderVoteWait for auto discovered cores

2014-01-21 Thread Software Dev
How is this accomplished? We currently have an empty solr.xml (auto-discovery) so I'm not sure where to put this value?

Removing a node from Solr Cloud

2014-01-21 Thread Software Dev
What is the process for completely removing a node from Solr Cloud? We recently removed one but t its still showing up as "Gone" in the Cloud admin. Thanks

Re: Solr Cloud Bulk Indexing Questions

2014-01-20 Thread Software Dev
4.6.0 On Mon, Jan 20, 2014 at 2:47 PM, Mark Miller wrote: > What version are you running? > > - Mark > > On Jan 20, 2014, at 5:43 PM, Software Dev > wrote: > > > We also noticed that disk IO shoots up to 100% on 1 of the nodes. Do all > > updates get

Re: Solr Cloud Bulk Indexing Questions

2014-01-20 Thread Software Dev
We also noticed that disk IO shoots up to 100% on 1 of the nodes. Do all updates get sent to one machine or something? On Mon, Jan 20, 2014 at 2:42 PM, Software Dev wrote: > We commit have a soft commit every 5 seconds and hard commit every 30. As > far as docs/second it would guess arou

Re: Solr Cloud Bulk Indexing Questions

2014-01-20 Thread Software Dev
e culprit. > > Best, > Erick > > On Mon, Jan 20, 2014 at 4:00 PM, Software Dev > wrote: > > We are testing our shiny new Solr Cloud architecture but we are > > experiencing some issues when doing bulk indexing. > > > > We have 5 solr cloud machines running

Solr Cloud Bulk Indexing Questions

2014-01-20 Thread Software Dev
We are testing our shiny new Solr Cloud architecture but we are experiencing some issues when doing bulk indexing. We have 5 solr cloud machines running and 3 indexing machines (separate from the cloud servers). The indexing machines pull off ids from a queue then they index and ship over a docume

Re: Sharding and replicas (Solr Cloud)

2013-11-07 Thread Software Dev
n Thu, Nov 7, 2013 at 3:15 PM, Shawn Heisey wrote: > On 11/7/2013 2:52 PM, Software Dev wrote: > >> Sorry about the confusion. I meant I created my config via the ZkCLI and >> then I wanted to create my core via the CollectionsAPI. I *think* I have >> it >> workin

Re: Sharding and replicas (Solr Cloud)

2013-11-07 Thread Software Dev
Sorry about the confusion. I meant I created my config via the ZkCLI and then I wanted to create my core via the CollectionsAPI. I *think* I have it working but was wondering why there are a crazy amount of core names under the admin "Core Selector"? When I create X amount of shards via the bootst