Hi there,
We are seeing a large number of commit index operations on solr4 master/slave
setup (150 to 200+ operations).
We don't initiate the commits manually as we are using auto commit . I believe
this results in search queries becoming slow/unresponsive over the course of a
few hours give
1
down vote
favorite
I need to perform an online search in solr i.e user need to find list of
user which are online with particular criteria.
how i am handling this:
we store the ids of user in a table and i send all online user id in solr
request like
&fq=-id:(id1 id2 id3 id5000)
pr
We have a 4 shard - 2 replica solr cloud setup, each with about 26GB of
index. A total of 24,000,000. We issued a rather large delete yesterday
morning to reduce that size by about half, this resulted in the loss of all
shards while the delete was taking place, but when it had apparently
finished
Alas I think CheckIndex can't do much here: there is no segments file,
so you'll have to reindex from scratch.
Just to check: did you ever called commit while building the index
before the machine crashed?
Mike McCandless
http://blog.mikemccandless.com
On Tue, Apr 30, 2013 at 8:17 PM, Otis Gos
I have a lot of large files that I've indexed into solr. Is there a way to
have solr return the snippets instead of the content? I'm only displaying
the snippets to my users, so transferring the content as well just wastes
bandwidth.
Thanks.
Fixed it. I just had to add "&fl=" with all of the fields except for
content that I wanted returned.
On Wed, May 1, 2013 at 7:38 AM, Bai Shen wrote:
> I have a lot of large files that I've indexed into solr. Is there a way
> to have solr return the snippets instead of the content? I'm only
>
Since you're doing this, you might want to make sure lazy field loading is
on as well. Unfortunately I can't link you to the wiki because it still
down (uh oh), but it's a setting in solrconfig.xml.
Michael Della Bitta
Appinions
18 East 41st Stree
Shawn thanks for the detailed answer. I have 5 shards and 1 leader - 1
replica for each. I mean I have 10 Solr nodes. When I look at admin gui of
one of the shards leader I see that its replica has less MB of index than
leader. I don't update the data, I don't index new ones. I think that after
a t
Sorry but what will I do? Will I copy everything under snapshot folder into
under index folder? If I don't run backup command and just copy index
folder anywhere else what is the difference between them (is it something
like consistency for if any writing operation for segment files did not
finishe
If you use Solr 4.x and SolrCloud there is no master-slave architecture
that has been before. You can change autoSoftCommit time, autoCommit time
at solrconfig.xml. Also you can consider using commitWithin, it is
explained here: http://wiki.apache.org/solr/UpdateXmlMessages
Beside that options if y
On 5/1/2013 2:28 AM, Mav Peri wrote:
> We are seeing a large number of commit index operations on solr4
> master/slave setup (150 to 200+ operations).
>
> We don't initiate the commits manually as we are using auto commit . I
> believe this results in search queries becoming slow/unresponsive ov
On 5/1/2013 3:39 AM, Annette Newton wrote:
> We have a 4 shard - 2 replica solr cloud setup, each with about 26GB of
> index. A total of 24,000,000. We issued a rather large delete yesterday
> morning to reduce that size by about half, this resulted in the loss of all
> shards while the delete wa
Was afraid of that and wondering if CheckIndex could regenerate the
segments file based on segments it finds in the index dir?
Otis
Solr & ElasticSearch Support
http://sematext.com/
On May 1, 2013 7:15 AM, "Michael McCandless"
wrote:
> Alas I think CheckIndex can't do much here: there is no s
Yeah, it's a consistency problem. Copying all those files takes time, and
without something with some knowledge of how Lucene works managing the
atomicity of the work, you might end up with a segments file that doesn't
match the segments you actually copied.
Michael Della Bitta
-
Hi Shawn
Thanks for the reply.
It was a single delete with a date range query. We have 8 machines each
with 35GB memory, 10GB is allocated to the JVM. Garbage collection has
always been a problem for us with the heap not clearing on Full garbage
collection. I don't know what is being held in m
I m trying to use the EmbeddedSolrServer and here is my sample code:
CoreContainer.Initializer initializer = new CoreContainer.Initializer();
CoreContainer coreContainer = initializer.initialize();
EmbeddedSolrServer server = new EmbeddedSolrServer(coreContainer, "");
Upon running I get the follo
Sorry but how do you use check index tool? Do you use Luke or does Solr has
built in functionality?
2013/5/1 Otis Gospodnetic
> Was afraid of that and wondering if CheckIndex could regenerate the
> segments file based on segments it finds in the index dir?
>
> Otis
> Solr & ElasticSearch Support
So does backup command is used for just to get more consistent index folder
compared to just using index folder for backup?
2013/5/1 Michael Della Bitta
> Yeah, it's a consistency problem. Copying all those files takes time, and
> without something with some knowledge of how Lucene works managi
You mean 6000 filter queries? Or do they really have 6000 faceted fields in
a single query?!
Even so, I wouldn't recommend that an average new Solr developer should have
either 6000 fields in a single document or 6000 query terms or even 6000
parameters. I mean, sure, you can try it and if it
Based on the fq ("-" in it) you posted are you trying to filter out all the
offline users?
Other option do you need the complete list in one request? did you try
splitting them in to batches of say 100 ids in one solr query.
--
View this message in context:
http://lucene.472066.n3.nabble.co
If I remember correctly, 3.6 DIH had bugs related to CachedSqlEntityProcessor
and some were fixed in 3.6.1, 3.6.2, but some were not fixed until 4.0. You
might want to use a 3.5 DIH jar with your 3.6 Solr. Or, post your
data-config.xml and maybe someone can figure something out.
James Dyer
In
An example:
https://gist.github.com/bretthoerner/2ffc362450bcd4c2487a
I'll note that all shards and replicas show as "Up" (green) in the Admin UI.
Does anyone know how this could happen? I can repeat this over and over
with the same terms. It was my understanding that something like a facet
query
Hi,
Can someone show me how to unsubscribe from JIRA?
Years ago, I subscribed to JIRA and since then I have been receiving emails
from JIRA for all kind of issues: when an issue is created, closed or commented
on. Yes, I looked around and could not figure out how to unsubscribe, but
maybe I
On 5/1/2013 8:42 AM, Annette Newton wrote:
It was a single delete with a date range query. We have 8 machines each
with 35GB memory, 10GB is allocated to the JVM. Garbage collection has
always been a problem for us with the heap not clearing on Full garbage
collection. I don't know what is bei
Hi MJ,
It looks like you're subscribed to the lucene dev list. Send an email to
dev-unsubscr...@lucene.apache.org to get yourself taken off the list.
Alan Woodward
www.flax.co.uk
On 1 May 2013, at 17:25, johnmu...@aol.com wrote:
> Hi,
>
>
> Can someone show me how to unsubscribe from JIRA?
Are you saying because I'm subscribed to dev, which I'm, is why I'm getting
JIRA mails too, and the only way I can stop JIRA mails is to unsubscribe from
dev? I don't think so. I'm subscribed to other projects, both dev and user,
and yet I do not receive JIRA mails.
--MJ
-Original Mes
I'll take a look. Thanks.
On Wed, May 1, 2013 at 8:27 AM, Michael Della Bitta <
michael.della.bi...@appinions.com> wrote:
> Since you're doing this, you might want to make sure lazy field loading is
> on as well. Unfortunately I can't link you to the wiki because it still
> down (uh oh), but it
i am sending list of online users and filters conditions as well.
--
View this message in context:
http://lucene.472066.n3.nabble.com/Handling-large-no-of-ids-in-solr-tp4060218p4060309.html
Sent from the Solr - User mailing list archive at Nabble.com.
On May 1, 2013, at 19:07 , johnmu...@aol.com wrote:
> Are you saying because I'm subscribed to dev, which I'm, is why I'm getting
> JIRA mails too, and the only way I can stop JIRA mails is to unsubscribe from
> dev? I don't think so. I'm subscribed to other projects, both dev and user,
> and
Dear Solr Developers,
I've been unable to find an answer to the question in the subject line
of this e-mail, except of a vague one.
We need to be able to index over 2bln+ documents. We were doing well
without sharding until the number of docs hit the limit ( 2bln+). The
performance was s
We are using Solr 3.6.2 with a single core setup on a glassfish server,
every 4-5 hours the server gradually gets into a some kind of a
inconsistent state and stops accepting any queries giving back cached
results. Even the core reload fails giving the following. Has anybody
experienced such behavi
On 5/1/2013 3:14 PM, Ravi Solr wrote:
We are using Solr 3.6.2 with a single core setup on a glassfish server,
every 4-5 hours the server gradually gets into a some kind of a
inconsistent state and stops accepting any queries giving back cached
results. Even the core reload fails giving the follow
Shawn,
I don't believe its the container because we use the same container
in another setup that has 6 cores which is serving almost 1.8 Million
requests a day without a hitch.
If you look at my email the container that is running SOLR got the request
params (http access logs provided in fir
Hi folks,
For facet seach, solr would create cache which is based on the whole docs. If I
import a new doc into index, the cache would out of time and need to create
again.
For real time seach, the docs would be import to index anytime. In this case,
the cache is nealy always need to create a
Hi
You can give soft-commit a try.
More details available here http://wiki.apache.org/solr/NearRealtimeSearch
-Original Message-
From: 李威 [mailto:li...@antvision.cn]
Sent: Thursday, 2 May 2013 12:02 PM
To: solr-user
Cc: 李景泽; 罗佳
Subject: How to deal with cache for facet search when inde
Hi Hoss,
I reviewed the code from other DataSouce classes as well, that's how I
learned it should work. And this is my actual code. I create this
DataSource for testing my ideas. I am blocked at the very beginning...sucks
:(
On Saturday, April 27, 2013, Chris Hostetter wrote:
> :
> : I am worki
Hi All,
WE have setup a 4.2 Solr cloud with 4 nodes and while the add/update/delete
operations are working we are not able to perform a rollback. Is there
something different for this operation vs the 3.x sole master/slave config?
Thanks,
Dipti
phone: 408.678.1595 | cell: 408.806.1970 | email:
Hi all,
I have recently migrated from solr 3.6 to solr 4.0. The documents in my core
are getting constantly updated and so I fire a code commit after every 10
thousand docs . However moving from 3.6 to 4.0 I have noticed that for the
same core size it takes about twice the time to commit in solr4.
38 matches
Mail list logo
