_
From: Matt Pearce
Sent: Wednesday, 13 February 2019 10:40 PM
To: solr-user@lucene.apache.org
Subject: Re: Delete by id
Hi Dwane,
The error suggests that Solr is trying to add a document, rather than
delete one, and is complaining that the DOC_ID is missing.
I tried each of your
/solr/guide/7_6/uploading-data-with-index-handlers.html
“The JSON update format allows for a simple delete-by-id. The value of a
delete can be an array which contains a list of zero or more specific document
id’s (not a range) to be deleted. For example, a single document”)
Attempt 1 – “shortha
My json format to delete the document (all looks correct according to
https://lucene.apache.org/solr/guide/7_6/uploading-data-with-index-handlers.html
“The JSON update format allows for a simple delete-by-id. The value of a
delete can be an array which contains a list of zero or more specific
Hi Emir,
We are deleting a larger subset of docs with a particular value which we
know based on the id and only updating a few of the deleted. Our document
is of the form
__, we need to delete all that has the same ,
that are no longer in DB and then update only a few that has been updated
in DB.
Hi Sujatha,
Did I get it right that you are deleting the same documents that will be
updated afterward? If that’s the case, then you can simply skip deleting, and
just send updated version of document. Solr (Lucene) does not have delete -
it’ll just flag document as deleted. Updating document (a
Thanks,Shawn.
Our use case is something like this in a batch load of several 1000's of
documents,we do a delete first followed by update.Example delete all 1000
docs and send an update request for 1000.
What we see is that there are many missing docs due to DBQ re-ordering of
the order of delet
On 6/21/2018 9:59 AM, sujatha sankaran wrote:
> Currently from our business perspective we find that we are left with no
> options for deleting docs in a batch load as :
>
> DBQ+ batch does not work well together
> DBI+ custom routing (batch load / normal)would not work as well.
I would expect
Thanks,Shawn.
Currently from our business perspective we find that we are left with no
options for deleting docs in a batch load as :
DBQ+ batch does not work well together
DBI+ custom routing (batch load / normal)would not work as well.
We are not sure how we can proceed unless we don't hav
On 6/20/2018 3:46 PM, sujatha sankaran wrote:
> Thanks,Shawn. Very useful information.
>
> Please find below the log details:-
Is your collection using the implicit router? You didn't say. If it
is, then I think you may not be able to use deleteById. This is indeed
a bug, one that has been re
Thanks,Shawn. Very useful information.
Please find below the log details:-
2018-06-20 17:19:06.661 ERROR
(updateExecutor-2-thread-8226-processing-crm_v2_01_shard3_replica1
x:crm_v2_01_shard3_replica2 r:core_node4 n:masked:8983_solr s:shard3
c:crm_v2_01) [c:crm_v2_01 s:shard3 r:core_node4
x:cr
On 6/15/2018 3:14 PM, sujatha sankaran wrote:
We were initially having an issue with DBQ and heavy batch updates which
used to result in many missing updates.
After reading many mails in mailing list which mentions that DBQ and batch
update do not work well together, we switched to DBI. But we
We were initially having an issue with DBQ and heavy batch updates which
used to result in many missing updates.
After reading many mails in mailing list which mentions that DBQ and batch
update do not work well together, we switched to DBI. But we are seeing
issue as mentioned in this jira is
Thanks everyone for replying to this issue. Just a final comment on this
issue which I was closely working on. We have fixed this issue. It was a
bug in our custom component which we wrote to convert delete by query to
delete by id. We were using BytesRef differently, we were not making a deep
t; > reference of same object, which was causing old deletes(LinkedHasmap)
> > adding more than 1K entries.
> >
> > But I think it is still not clearing those 1K entries. Eventually it will
> > throw OOM because UpdateLog is not singleton and when there will be many
>
ng old deletes(LinkedHasmap)
> adding more than 1K entries.
>
> But I think it is still not clearing those 1K entries. Eventually it will
> throw OOM because UpdateLog is not singleton and when there will be many
> delete by id and server is not re started for very long time then
> eve
Hi Tom
The moving alias design is interesting, will explore it.
Regarding themethod of creating the collection on a node for indexing
only and adding replicas of it to other nodes for queryinguponcompletion
of indexing.
Am I right to say this is used in conjunction with collection alias or
th
entries. Eventually it will
throw OOM because UpdateLog is not singleton and when there will be many
delete by id and server is not re started for very long time then
eventually throw OOM. I think we should clear this map when we are
committing. I am not a committer, it would be great if I get reply
On Thu, Mar 23, 2017 at 6:10 AM, Derek Poh wrote:
> Hi
>
> I have collections of products. I am doing indexing 3-4 times daily.
> Every day there are products that expired and I need to remove them from
> these collectionsdaily.
>
> Ican think of 2 ways to do this.
> 1. using collection aliasto sw
Erick
Generally the products have contracted date but they could be extended
and also get expired prematurely.
We will need additional processing to cater for these scenarios and
update the 'expiry date' fields accordingly.
Will go through thedocumentationagainand see if it can fitour use cas
Hi Emir
Thank you for pointing outdeleted docwill still existin the indextill it
is optimize and itwill skewed statistics. We dosort by score.
This new collectionsare partofa new business initiativeandwe do not know
as yet what will be their sizelike.
Willgo ponder on your inputs. Thank you
have you considered using TTL (Time To Live)?
You have to know at index time when the doc will expire.
If you do, Solr will delete the doc for you when its
life is over.
See: https://lucidworks.com/2014/05/07/document-expiration/
Also the Ref guide:
https://cwiki.apache.org/confluence/display/solr
Hi Derek,
There are both pros and cons for both approaches:
1. if you are doing full reindexing PRO is that you have clean index all
the time and even if something goes wrong, you don't have to switch
alias to updated index so your users will not notice issues. CON is that
you are doing full
Hi
I have collections of products. I am doing indexing 3-4 times daily.
Every day there are products that expired and I need to remove them from
these collectionsdaily.
Ican think of 2 ways to do this.
1. using collection aliasto switch between a main and temp collection.
- clear and index the
For commits we are relying on auto commits. We have define following in
configs:
1
3
false
15000
One thing which I would like to mention is that we are not calling directly
deleteById from client. We
: OK, The whole DBQ thing baffles the heck out of me so this may be
: totally off base. But would committing help here? Or at least be worth
: a test?
ths isn't DBQ -- the OP specifically said deleteById, and that the
oldDeletes map (only used for DBI) was the problem acording to the heap
dumps
Chris:
OK, The whole DBQ thing baffles the heck out of me so this may be
totally off base. But would committing help here? Or at least be worth
a test?
On Tue, Mar 21, 2017 at 4:28 PM, Chris Hostetter
wrote:
>
> : Thanks for replying. We are using Solr 6.1 version. Even I saw that it is
> : boun
: Thanks for replying. We are using Solr 6.1 version. Even I saw that it is
: bounded by 1K count, but after looking at heap dump I was amazed how can it
: keep more than 1K entries. But Yes I see around 7M entries according to
: heap dump and around 17G of memory occupied by BytesRef there.
what
job every night to delete old message documents. We are deleting
> : old documents by calling multiple delete by id query to solr. Document
> : count can be in millions which we are deleting using SolrJ client. We are
> : using delete by id because it is faster than delete by query. It works
: facing. We are storing messages in solr as documents. We are running a
: pruning job every night to delete old message documents. We are deleting
: old documents by calling multiple delete by id query to solr. Document
: count can be in millions which we are deleting using SolrJ client. We are
Hi All,
I am looking for some help to solve an out of memory issue which we are
facing. We are storing messages in solr as documents. We are running a
pruning job every night to delete old message documents. We are deleting
old documents by calling multiple delete by id query to solr. Document
half a minute,
version 2 will disappear, which means the update is not get write into the
disk.
I tried to delete by id with rsolr, and the doc just can't be removed.
Insert new doc into the index is fine though.
Anyone ran into this strange behavior before?
Thanks
Ming
quot;curl 'localhost:8995/solr/select?wt=json&indent=true&q=id:28583776' " ,
> the two versions above will appear randomly. And after half a minute,
> version 2 will disappear, which means the update is not get write into the
> disk.
>
> I tried to delete by i
version 2 will disappear, which means the update is not get write into the
disk.
I tried to delete by id with rsolr, and the doc just can't be removed.
Insert new doc into the index is fine though.
Anyone ran into this strange behavior before?
Thanks
Ming
deleting documents using Delete by ID
Thanks Bharat. This is a bug. I've opened LUCENE-5289.
https://issues.apache.org/jira/browse/LUCENE-5289
On Wed, Oct 16, 2013 at 9:35 PM, Akkinepalli, Bharat (ELS-CON) <
b.akkinepa...@elsevier.com> wrote:
> Hi Shalin,
> I am not sure why
red that the document is replicated successfully.
> > Issued a delete by ID.
> > Issued a commit on master
> > Replication did NOT happen.
>
> The logs are as follows:
> Master - http://pastebin.com/265CtCEp
> Slave - http://pastebin.com/Qx0xLwmK
>
> Regards,
>
iggered replication on slave
> Ensured that the document is replicated successfully.
> Issued a delete by ID.
> Issued a commit on master
> Replication did NOT happen.
The logs are as follows:
Master - http://pastebin.com/265CtCEp
Slave - http://pastebin.com/Qx0xLwmK
Regards,
Bha
October 11, 2013 2:16 PM
> To: solr-user@lucene.apache.org
> Subject: RE: Solr 4.4 - Master/Slave configuration - Replication Issue
> with Commits after deleting documents using Delete by ID
>
> Hi Otis,
> Thanks for the response. The log files can be found here.
>
>
@lucene.apache.org
Subject: RE: Solr 4.4 - Master/Slave configuration - Replication Issue with
Commits after deleting documents using Delete by ID
Hi Otis,
Thanks for the response. The log files can be found here.
MasterLog : http://pastebin.com/DPLKMPcF Slave Log:
http://pastebin.com
Hi Otis,
Thanks for the response. The log files can be found here.
MasterLog : http://pastebin.com/DPLKMPcF
Slave Log: http://pastebin.com/DX9sV6Jx
One more point worth mentioning here is that when we issue the commit with
expungeDeletes=true, then the delete by id replication is
; as commit &
> startup
>
> Steps to reproduce the problem:
> ==========
>
> 1. Delete a document in Solr (using delete by id). URL -
> http://localhost:8983/solr/annotation/update with body as
> change.me
>
> 2. Issue a commit in Master
>
Delete a document in Solr (using delete by id). URL -
http://localhost:8983/solr/annotation/update with body as
change.me
2. Issue a commit in Master
(http://localhost:8983/solr/annotation/update?commit=true).
3. The replication of the DELETE WILL NOT happen. The master
.
--
View this message in context:
http://www.nabble.com/solrj-delete-by-Id-problem-tp21433056p21704110.html
Sent from the Solr - User mailing list archive at Nabble.com.
earcher=false will not let you see the new results
immediately. If you do not want warming, you can disable it, however you
should test with your index and be aware of the performance impact of using
a cold searcher.
--
> View this message in context:
> http://www.nabble.com/solrj-delete-by
see
the search result immediately after delete.
--
View this message in context:
http://www.nabble.com/solrj-delete-by-Id-problem-tp21433056p21703654.html
Sent from the Solr - User mailing list archive at Nabble.com.
On Tue, Jan 27, 2009 at 8:51 PM, Parisa wrote:
>
> I found how the issue is created .when solr warm up the new searcher with
> cacheLists , if the queryResultCache is enable the issue is created.
>
> notice:as I mentioned before I commit with waitflush=false and
> waitsearcher=false
>
> so it has
rrSearcher);
} catch (Throwable e) {
SolrException.logOnce(log,null,e);
}
return null;
}
}
);
}
-
--
}
--
View this message in context:
http://www.nabble.com/solrj-de
On Mon, Jan 26, 2009 at 12:20 PM, Parisa wrote:
>
> Is there any solution for fixing this bug ?
> --
> View this message in context:
> http://www.nabble.com/solrj-delete-by-Id-problem-tp21433056p21661131.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
Is there any solution for fixing this bug ?
--
View this message in context:
http://www.nabble.com/solrj-delete-by-Id-problem-tp21433056p21661131.html
Sent from the Solr - User mailing list archive at Nabble.com.
It has the problem with the keywords that we have searched
>> before
>> deleting the docs and it has not problem with new key words.
>>
>> --
>> View this message in context:
>> http://www.nabble.com/solrj-delete-by-Id-problem-tp21433056p21433056.htm
Did you call commit after the delete?
On Tue, Jan 13, 2009 at 4:12 PM, Parisa wrote:
>
> I have a problem with solrj delete By Id . If I search a keyword and it has
> more than 1 result no (for example 7) then I delete on of the resulted doc
> with solrj (server.deleteById ) , I
I have a problem with solrj delete By Id . If I search a keyword and it has
more than 1 result no (for example 7) then I delete on of the resulted doc
with solrj (server.deleteById ) , I search this keyword again the result
no is zero . and it 's not correct because it should be 6 . It s
On Nov 19, 2007 12:18 AM, Norberto Meijome <[EMAIL PROTECTED]> wrote:
> I'm trying to issue, via curl to SOLR (testing at the moment), 3 deletes by
> id.
> I tried sending :
>
> 123
>
> and solr didn't like it at all.
This is relatively new syntax in Solr... older versions won't like it.
-Yonik
On Mon, 19 Nov 2007 16:53:17 +1100
climbingrose <[EMAIL PROTECTED]> wrote:
> The easiest solution I know is:
> id:1 OR id:2 OR ...
> If you know that all of these ids can be found by issuing a query, you
> can do delete by query:
> YOUR_DELETE_QUERY_HERE
thanks, so i'm not going nuts (at least no
The easiest solution I know is:
id:1 OR id:2 OR ...
If you know that all of these ids can be found by issuing a query, you
can do delete by query:
YOUR_DELETE_QUERY_HERE
Cheers
On Nov 19, 2007 4:18 PM, Norberto Meijome <[EMAIL PROTECTED]> wrote:
> Hi everyone,
>
> I'm trying to issue, via curl to
Hi everyone,
I'm trying to issue, via curl to SOLR (testing at the moment), 3 deletes by id.
I tried sending :
123
and solr didn't like it at all.
When I changed it to :
123
as in :
curl http://localhost:8983/vcs/update -H "Content-Type: text/xml" --data-binary
'816bc47fd52ffb9c6059e6975e
55 matches
Mail list logo
