optimize status
Hi All, I have two indexers (Independent processes ) writing to a common solr core. If One indexer process issued an optimize on the core I want the second indexer to wait adding docs until the optimize has finished. Are there ways I can do this programmatically? pinging the core when the optimize is happening is returning OK because technically solr allows you to update when an optimize is happening. any suggestions ? thanks, Summer
Re: [MASSMAIL] Re: ConcurrentUpdateSolrClient ignoring the collection param in some methods
Hi Erick,
Yes CloudSolrClient is better suited for cloud environments, in this case I'm
dealing with a single server with multiple collections. In the code I provide
the ability to switch to CloudSolrClient instead of CUSC if a zookeper URL is
specified, so for now this must work for cloud and non-cloud deployments.
Thanks for the replies,
Regards,
- Original Message -
From: "Erick Erickson"
To: solr-user@lucene.apache.org
Sent: Saturday, June 27, 2015 11:33:53 AM
Subject: [MASSMAIL] Re: ConcurrentUpdateSolrClient ignoring the collection
param in some methods
Jorge:
Another thing to consider: CloudSolrClient is a much better choice in cloud
environments that CUSC. What you're seeing isn't right, no doubt but using
the other class may:
1> not require a work-around
2> be much more efficient.
In a sharded situation, CloudSolrClient will route all the documents
to the correct leader, leading to better performance. That class
scales nearly linearly in terms of indexing throughput with the
number of shards
FWIW,
Erick
On Sat, Jun 27, 2015 at 2:32 AM, Jorge Luis Betancourt González
wrote:
> Thanks for the prompt reply Shawn. I've created an issue about this [1] from
> the code the collection parameter is clearly ignored unless is a commit or
> the UpdateParams.WAIT_SEARCHER parameter is set in the params of the request,
> which is an indication of a commit/optimize.
>
> The queue only contain UpdateRequest objects which doesn't hold any data
> about the collection.
>
> Regards,
>
> - Original Message -
> From: "Shawn Heisey"
> To: solr-user@lucene.apache.org
> Sent: Friday, June 26, 2015 5:10:24 PM
> Subject: ConcurrentUpdateSolrClient ignoring the collection param in some
> methods
>
> On 6/26/2015 2:27 PM, Jorge Luis Betancourt González wrote:
>> I'm trying to use the ConcurrentUpdateSolrClient class, that has some
>> methods that accept and aditional parameter to indicate the collection, some
>> of this methods are add(String collection, SolrInputDocument doc),
>> request(SolrRequest, String collection). With HttpSolrClient this works
>> flawlessly, but with ConcurrentUpdateSolrClient the request gets made
>> against the URL specified in the constructor.
>>
>> In the case of the commit(String collection) method, works without any
>> trouble and the commit gets fired against the desired collection.
>
> This is definitely a bug. Can you open an issue in Jira?
>
> Looking into how an update request actually gets added to the background
> queue in ConcurrentUpdateSolrClient, it appears that the "collection"
> information is ignored before the request is added to the queue.
>
> As a workaround until the bug can be fixed, instead of this:
>
> client.add("status", doc);
>
> Use this (assuming that your update handler is indeed /update, which it
> normally will be):
>
> UpdateRequest req = new UpdateRequest("/status/update");
> req.add(doc);
> req.process(_client);
>
> Thanks,
> Shawn
Re: [MASSMAIL] Re: ConcurrentUpdateSolrClient ignoring the collection param in some methods
Ups, sorry I forgot the issue URL:
https://issues.apache.org/jira/browse/SOLR-7729
- Original Message -
From: "Erick Erickson"
To: solr-user@lucene.apache.org
Sent: Saturday, June 27, 2015 11:33:53 AM
Subject: [MASSMAIL] Re: ConcurrentUpdateSolrClient ignoring the collection
param in some methods
Jorge:
Another thing to consider: CloudSolrClient is a much better choice in cloud
environments that CUSC. What you're seeing isn't right, no doubt but using
the other class may:
1> not require a work-around
2> be much more efficient.
In a sharded situation, CloudSolrClient will route all the documents
to the correct leader, leading to better performance. That class
scales nearly linearly in terms of indexing throughput with the
number of shards
FWIW,
Erick
On Sat, Jun 27, 2015 at 2:32 AM, Jorge Luis Betancourt González
wrote:
> Thanks for the prompt reply Shawn. I've created an issue about this [1] from
> the code the collection parameter is clearly ignored unless is a commit or
> the UpdateParams.WAIT_SEARCHER parameter is set in the params of the request,
> which is an indication of a commit/optimize.
>
> The queue only contain UpdateRequest objects which doesn't hold any data
> about the collection.
>
> Regards,
>
> - Original Message -
> From: "Shawn Heisey"
> To: solr-user@lucene.apache.org
> Sent: Friday, June 26, 2015 5:10:24 PM
> Subject: ConcurrentUpdateSolrClient ignoring the collection param in some
> methods
>
> On 6/26/2015 2:27 PM, Jorge Luis Betancourt González wrote:
>> I'm trying to use the ConcurrentUpdateSolrClient class, that has some
>> methods that accept and aditional parameter to indicate the collection, some
>> of this methods are add(String collection, SolrInputDocument doc),
>> request(SolrRequest, String collection). With HttpSolrClient this works
>> flawlessly, but with ConcurrentUpdateSolrClient the request gets made
>> against the URL specified in the constructor.
>>
>> In the case of the commit(String collection) method, works without any
>> trouble and the commit gets fired against the desired collection.
>
> This is definitely a bug. Can you open an issue in Jira?
>
> Looking into how an update request actually gets added to the background
> queue in ConcurrentUpdateSolrClient, it appears that the "collection"
> information is ignored before the request is added to the queue.
>
> As a workaround until the bug can be fixed, instead of this:
>
> client.add("status", doc);
>
> Use this (assuming that your update handler is indeed /update, which it
> normally will be):
>
> UpdateRequest req = new UpdateRequest("/status/update");
> req.add(doc);
> req.process(_client);
>
> Thanks,
> Shawn
Re: ConcurrentUpdateSolrClient ignoring the collection param in some methods
On 6/28/2015 2:59 PM, Jorge Luis Betancourt González wrote: > Yes CloudSolrClient is better suited for cloud environments, in this case I'm > dealing with a single server with multiple collections. In the code I provide > the ability to switch to CloudSolrClient instead of CUSC if a zookeper URL is > specified, so for now this must work for cloud and non-cloud deployments. Please also be aware that when using CUSC, the server could be entirely down and your code would be unaware that your document adds did not succeed. I believe that CUSC does throw a proper exception when you do a commit and it fails, but adds always appear to succeed, even if nothing actually gets indexed. https://issues.apache.org/jira/browse/SOLR-3284 Thanks, Shawn
optimize status
Hi All, I have two indexers (Independent processes ) writing to a common solr core. If One indexer process issued an optimize on the core I want the second indexer to wait adding docs until the optimize has finished. Are there ways I can do this programmatically? pinging the core when the optimize is happening is returning OK because technically solr allows you to update when an optimize is happening. any suggestions ? thanks, Summer
Re: optimize status
Bigger question, why are you optimizing? Since 3.6 or so, it generally hasn't been requires, even, is a bad thing. Upayavira On Sun, Jun 28, 2015, at 09:37 PM, Summer Shire wrote: > Hi All, > > I have two indexers (Independent processes ) writing to a common solr > core. > If One indexer process issued an optimize on the core > I want the second indexer to wait adding docs until the optimize has > finished. > > Are there ways I can do this programmatically? > pinging the core when the optimize is happening is returning OK because > technically > solr allows you to update when an optimize is happening. > > any suggestions ? > > thanks, > Summer
