acelyc111 opened a new issue #3126: BE crash in Thrift frame when send RPC to a busy FE URL: https://github.com/apache/incubator-doris/issues/3126 **Describe the bug** BE crashed, and backtrace like: ``` (gdb) bt #0 0x00007fadd1fe31d7 in raise () from /lib64/libc.so.6 #1 0x00007fadd1fe48c8 in abort () from /lib64/libc.so.6 #2 0x0000000002ddf265 in __gnu_cxx::__verbose_terminate_handler () at ../../.././libstdc++-v3/libsupc++/vterminate.cc:95 #3 0x0000000002d4f696 in __cxxabiv1::__terminate (handler=<optimized out>) at ../../.././libstdc++-v3/libsupc++/eh_terminate.cc:47 #4 0x0000000002d4f6e1 in std::terminate () at ../../.././libstdc++-v3/libsupc++/eh_terminate.cc:57 #5 0x0000000002d455c3 in __cxxabiv1::__cxa_throw (obj=obj@entry=0x28382f1e10, tinfo=0x312b7f0 <typeinfo for apache::thrift::transport::TTransportException>, dest=dest@entry=0xfaf460 <apache::thrift::transport::TTransportException::~TTransportException()>) at ../../.././libstdc++-v3/libsupc++/eh_throw.cc:93 #6 0x00000000022de8a7 in apache::thrift::transport::TSocket::write (this=0x33555815e0, buf=0x129777ce00 "", len=509) at src/thrift/transport/TSocket.cpp:614 #7 0x00000000022e2636 in write (len=<optimized out>, buf=<optimized out>, this=<optimized out>) at ./src/thrift/transport/TTransport.h:153 #8 apache::thrift::transport::TBufferedTransport::flush (this=0x7a3a7860) at src/thrift/transport/TBufferTransports.cpp:128 #9 0x00000000022e32fa in apache::thrift::transport::TBufferedTransport::close (this=0x7a3a7860) at ./src/thrift/transport/TBufferTransports.h:230 #10 0x00000000010d3466 in doris::ThriftClientImpl::close (this=this@entry=0x3f560b8620) at /root/doris/doris-xiaomi/be/src/util/thrift_client.cpp:70 #11 0x0000000001037bb1 in doris::ClientCacheHelper::reopen_client(boost::function<doris::ThriftClientImpl* (doris::TNetworkAddress const&, void**)>, void**, int) (this=this@entry=0x3f9cb20 <doris::TaskWorkerPool::_master_service_client_cache>, factory_method=..., client_key=client_key@entry=0x7fad1247aef8, timeout_ms=timeout_ms@entry=5000) at /root/doris/doris-xiaomi/be/src/runtime/client_cache.cpp:87 #12 0x0000000000fd2338 in doris::ClientCache<doris::FrontendServiceClient>::reopen_client (this=0x3f9cb20 <doris::TaskWorkerPool::_master_service_client_cache>, client=client@entry=0x7fad1247aef8, timeout_ms=5000) at /root/doris/doris-xiaomi/be/src/runtime/client_cache.h:262 #13 0x00000000013b7458 in reopen (timeout_ms=<optimized out>, this=0x7fad1247aef0) at /root/doris/doris-xiaomi/be/src/runtime/client_cache.h:177 #14 doris::MasterServerClient::report (this=0x1465ac00, request=..., result=0x7fad1247d8a0) at /root/doris/doris-xiaomi/be/src/agent/utils.cpp:216 #15 0x00000000013b1ad2 in doris::TaskWorkerPool::_report_tablet_worker_thread_callback (arg_this=0x223825a0) at /root/doris/doris-xiaomi/be/src/agent/task_worker_pool.cpp:1371 #16 0x00007fadd1d99dc5 in start_thread () from /lib64/libpthread.so.0 #17 0x00007fadd20a573d in clone () from /lib64/libc.so.6 (gdb) ``` FE is very busy that time, BE side thrift client want to `reopen()` the socket but TBufferTransports throw an exception in `close()`. I checked Impala code, found this bug has been fixed(catch the exception), we can fix it in that way too.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
