I find it very useful to keep the used async IDs regardless of the status for some time. For example, If you have a workflow that involves multiple steps such as add/remove replicas, you can just retry/restart the workflow and be sure Solr will reject the request if the async ID already exists (and your code can then handle this accordingly, for example, checking the status of success/failed and act accordingly) as long as you use the async IDs consistently.
That said, async IDs do need to eventually be removed and AFAIK Solr doesn't do this automatically. This is a problem because of ever increasing objects in ZooKeeper. I think we should have some sort of task that cleans up async ID after some configurable amount of time. On Wed, May 10, 2023 at 1:01 AM Andras Salamon <andras.sala...@melda.info> wrote: > Hi, > > > > How can we be sure that the previous request status info has been already > processed? What about the following timeline: > > > > -Client1 sends an async request > > -Client1 reads status info, it's still running > > -Client1 reads status info, it's still running > > -Async request finishes > > -Right after that Client2 sends a new async request with the same ID, we > clear the async status because it's already finished > > -Client1 reads status info, but this time it will read info about the new > async request sent by Client2. > > > > Andras > > > > > > > > > ---- On Wed, 10 May 2023 05:15:40 +0200 David Smiley <dsmi...@apache.org> > wrote --- > > > > I noticed that async admin requests to Solr must have a unique asyncId or > else a request is rejected. Makes sense -- maybe the request is in > progress. But what if it isn't -- what if the previous request for the > same ID either succeeded or failed? Shouldn't we clear the previous > asyncId status and let the new request go through? > > I'm imagining leveraging this uniqueness constraint in order to be an > additional protection measure against requests that should be done > atomically, like a shard split. Yes there are already locks but this > additional measure will allow a fail-fast -- no enqueue of a doomed > message > to the Overseer that will ultimately never succeed any way. Thus the > sender of a shard split can use an async ID like > "SPLIT-collectionName-shardName". Maybe there are other parts of > SolrCloud > that could leverage this constraint to its advantage likewise. > > ~ David Smiley > Apache Lucene/Solr Search Developer > http://www.linkedin.com/in/davidwsmiley
