yixiutt opened a new pull request, #14231:
URL: https://github.com/apache/doris/pull/14231
In concurrent load, some publish timeout happens occasionally. This is cause
by meta lock hold by other thread so publish add increase rowset hang for
several seconds.
StorageEngine::start_delete_unused_rowset will hold gc_mutex and it cost a
lot of time, so that add_used_rowset wait lock, and compaction modify_rowset or
other tablet method will hold meta_lock and call add_unused_rowset which will
make meta_lock occupied for too long, finally makes publish timeout.
In this pr, I copy unused_rowsets in lock and delete these rowset without
lock, makes gc_mutex more lightweight so meta lock can be acquired immediately
in publish thread. My test shows that no publish timeout in concurrent stream
load.
# Proposed changes
Issue Number: close #xxx
## Problem summary
Describe your changes.
## Checklist(Required)
1. Does it affect the original behavior:
- [ ] Yes
- [ ] No
- [ ] I don't know
2. Has unit tests been added:
- [ ] Yes
- [ ] No
- [ ] No Need
3. Has document been added or modified:
- [ ] Yes
- [ ] No
- [ ] No Need
4. Does it need to update dependencies:
- [ ] Yes
- [ ] No
5. Are there any changes that cannot be rolled back:
- [ ] Yes (If Yes, please explain WHY)
- [ ] No
## Further comments
If this is a relatively large or complex change, kick off the discussion at
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you
chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
