Re: Async collection requests and deduplication

2023-05-10 Thread Tomás Fernández Löbbe
> BTW these async status objects stored in ZK are in fact cleaned up when they reach 10k in number. See SizeLimitedDistributedMap. Yes, I now remember. The asyncIDs are actually on a regular DistributedMap, but "completed" and "failure" maps will delete asyncIDs when clearing their own elements.

Re: Async collection requests and deduplication

2023-05-10 Thread David Smiley
Good point Tomas; I hadn't considered that use-case. I suppose the behavior I suggest could be controlled with a boolean parameter flag like "asyncDeleteStatus" true/false. WDYT? I'm not married to it. BTW these async status objects stored in ZK are in fact cleaned up when they reach 10k in num

Re: Async collection requests and deduplication

2023-05-10 Thread Tomás Fernández Löbbe
I find it very useful to keep the used async IDs regardless of the status for some time. For example, If you have a workflow that involves multiple steps such as add/remove replicas, you can just retry/restart the workflow and be sure Solr will reject the request if the async ID already exists (and

Re: Async collection requests and deduplication

2023-05-10 Thread Andras Salamon
Hi, How can we be sure that the previous request status info has been already processed? What about the following timeline: -Client1 sends an async request -Client1 reads status info, it's still running -Client1 reads status info, it's still running -Async request finishes -Right after t