kaka11chen opened a new pull request, #33959:
URL: https://github.com/apache/doris/pull/33959
## Proposed changes
## Issue:
```
F20240421 17:14:37.494115 184986 hdfs_util.h:65] Check failed: _ref_cnt == 0
*** Check failure stack trace: ***
F20240421 17:14:37.505879 185108 hdfs_util.h:65] Check failed: _ref_cnt == 0
*** Check failure stack trace: ***
@ 0x556f5236d316 google::LogMessageFatal::~LogMessageFatal()
@ 0x556f5236d316 google::LogMessageFatal::~LogMessageFatal()
@ 0x556f2830e200 doris::io::HdfsFileWriter::~HdfsFileWriter()
@ 0x556f2830e200 doris::io::HdfsFileWriter::~HdfsFileWriter()
@ 0x556f2830e21e doris::io::HdfsFileWriter::~HdfsFileWriter()
@ 0x556f2830e21e doris::io::HdfsFileWriter::~HdfsFileWriter()
@ 0x556f507893b0
doris::vectorized::VHivePartitionWriter::~VHivePartitionWriter()
@ 0x556f507893b0
doris::vectorized::VHivePartitionWriter::~VHivePartitionWriter()
@ 0x556f506c005e std::_Hashtable<>::clear()
@ 0x556f506c005e std::_Hashtable<>::clear()
@ 0x556f50780f4f doris::vectorized::VHiveTableWriter::close()
@ 0x556f50780f4f doris::vectorized::VHiveTableWriter::close()
@ 0x556f5072bc4f
doris::vectorized::AsyncResultWriter::process_block()
@ 0x556f5072bc4f
doris::vectorized::AsyncResultWriter::process_block()
@ 0x556f5072cd01 std::_Function_handler<>::_M_invoke()
@ 0x556f5072cd01 std::_Function_handler<>::_M_invoke()
@ 0x556f2b02b73d doris::ThreadPool::dispatch_thread()
@ 0x556f2b02b73d doris::ThreadPool::dispatch_thread()
@ 0x556f2b008d59 doris::Thread::supervise_thread()
@ 0x556f2b008d59 doris::Thread::supervise_thread()
@ 0x7f2c2bfb4609 start_thread
@ 0x7f2c2bfb4609 start_thread
@ 0x7f2c2c261133 clone
@ 0x7f2c2c261133 clone
@ (nil) (unknown)
*** Query id: ac4f457c003d4489-b04ac56ef05b12f0 ***
*** is nereids: 1 ***
*** tablet id: 0 ***
*** Aborted at 1713690877 (unix time) try "date -d @1713690877" if you are
using GNU date ***
*** Current BE git commitID: e6f4a2f ***
*** SIGABRT unknown detail explain (@0x38a6) received by PID 14502 (TID
184986 OR 0x7f21d0614700) from PID 14502; stack trace: ***
@ (nil) (unknown)
F20240421 17:14:37.505879 185108 hdfs_util.h:65] Check failed: _ref_cnt == 0
F20240421 17:14:37.887202 185110 hdfs_util.h:65] Check failed: _ref_cnt == 0
*** Check failure stack trace: ***
@ 0x556f5236d316 google::LogMessageFatal::~LogMessageFatal()
@ 0x556f2830e200 doris::io::HdfsFileWriter::~HdfsFileWriter()
@ 0x556f2830e21e doris::io::HdfsFileWriter::~HdfsFileWriter()
@ 0x556f507893b0
doris::vectorized::VHivePartitionWriter::~VHivePartitionWriter()
0# doris::signal::(anonymous namespace)::FailureSignalHandler(int,
siginfo_t*, void*) at /root/doris/be/src/common/signal_handler.h:421
```
### Solution
Fix hdfs file writer core with `check failed: _ref_cnt == 0` in dtor of
`HdfsFileWriter`.
- Change `fs_handler` ptr to `shared_ptr`.
- Remove ref count operations if not in cache.
## Further comments
If this is a relatively large or complex change, kick off the discussion at
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you
chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
