[GitHub] [doris] tarepanda1024 commented on pull request #17578: [improvement](config)add task pool config

[via GitHub]

tarepanda1024 commented on PR #17578:
URL: https://github.com/apache/doris/pull/17578#issuecomment-1461167997

   > Why need this?
   
   When there are small querys：each query cost about 10ms ,create task thead 
may cost too long, and the gap  time not in audit.log .After thead pool warm 
up, the  99% latency become better.
   
   ```sql
   select count(1) from (select xx as user_id) as M left join dm_assets.xxx as 
Q_0 on M.user_id=Q_0.user_id and Q_0.user_id = xxx left join dm_assets.xxx  as 
Q_1 on M.user_id=Q_1.user_id and Q_1.user_id = xxx where 
(not((Q_0.order_confirm_time <= xx and Q_0.order_confirm_time >= xxx AND  
array_str_op(Q_0.item_id, 'in', 'xxx'))) AND ( Q_1.xx not in ('1') OR Q_1.xx IS 
NULL ) )
   ```
   
   Trace from mysql jdbc client.The gap about 200ms+，but in audit log，the real 
exec time is about 10ms.
   
   Trace from client:
   <img width="1835" alt="image" 
src="https://user-images.githubusercontent.com/23069203/223895397-20fe3d25-d642-43a7-b650-b24eece8978f.png";>
   
   The business log:
   <img width="1856" alt="image" 
src="https://user-images.githubusercontent.com/23069203/223895694-9f102acf-8578-4a47-9027-a444e261d9a9.png";>
   
   The audit log：
   <img width="1835" alt="image" 
src="https://user-images.githubusercontent.com/23069203/223895977-396f10c9-ced1-4032-b2b5-c21846ec41c1.png";>
   query result is ERR and cost 5ms, because query timeout and canceled.
   
   From client：215ms，but in doris，cost 5ms and canceled by timeout.the gap in 
task thread pool.
   I changed xnio IO thread num，affect little.And big flow,the task pool warm 
up and latency become better.
   
   Trace from doris fe(not match above query)：
   <img width="1564" alt="image" 
src="https://user-images.githubusercontent.com/23069203/223893550-f14a1791-df8b-481e-aa77-0cdd2420e8be.png";>
   <img width="1549" alt="image" 
src="https://user-images.githubusercontent.com/23069203/223893923-434ff7a9-a178-4fd0-985a-5f81b2ac953c.png";>
   <img width="920" alt="image" 
src="https://user-images.githubusercontent.com/23069203/223896258-2bcfb1c2-3f6a-4bd0-8522-a405403305a1.png";>
   <img width="898" alt="image" 
src="https://user-images.githubusercontent.com/23069203/223896328-43a077b7-d71c-409f-b5b6-f94288cfd65d.png";>
   the audit log not calc : readEvent to query dispatch cost.
   
   bench mark :
   
   <img width="977" alt="image" 
src="https://user-images.githubusercontent.com/23069203/223896967-fd671e60-6061-4fa5-9848-f60b57dd5668.png";>
   the blank is being benchmark
   
   <img width="1212" alt="image" 
src="https://user-images.githubusercontent.com/23069203/223897172-12611132-7648-4821-aa19-bdd3e412080e.png";>
   
   <img width="1173" alt="image" 
src="https://user-images.githubusercontent.com/23069203/223897367-10959bda-bce8-4f81-8414-c53baa1fc10c.png";>
   
   flow slow down, the latency up.
   
   
    


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org