Re: Re: Hive querying/importing instances

2023-03-14 Thread Nikita Sakharin
Hi, Mingyu Chen! Thank you for your advice! I did actually help, now I have about 95% of HDFS throughput using Hive. Kind regards, Nikita пт, 10 мар. 2023 г. в 18:37, Mingyu Chen : > Hi Nikita, > First of all, you don't have to set all those configs and session > variables. > Setting them does

Re:Re: Hive querying/importing instances

2023-03-10 Thread Mingyu Chen
Hi Nikita, First of all, you don't have to set all those configs and session variables. Setting them doesn't do anything for this problem. Let me make it clear, you problem is that: 1. Use `insert into table from hdfs()` can utilize most of your cluster's resource, and the speed is acceptable. 2

Re: Hive querying/importing instances

2023-03-10 Thread Nikita Sakharin
Hi, Yongqiang YANG, thank you for your response! I appended fe.conf with lines: async_load_task_pool_size = 320 async_loading_load_task_pool_size = 320 async_pending_load_task_pool_size = 320 320 is the total amount of cores in my configuration. I had restarted all the machines in the cluster (i

Re: Hive querying/importing instances

2023-03-10 Thread Yongqiang YANG
You can adjust async_pending_load_task_pool_size = 250 and async_loading_load_task_pool_size=250 in fe.conf and have a try. On Fri, Mar 10, 2023 at 8:49 PM Nikita Sakharin wrote: > Hi, Apache Doris Community! > > I'm a beginner user of Doris and I'm experiencing performance issues. > > I have *4

Hive querying/importing instances

2023-03-10 Thread Nikita Sakharin
Hi, Apache Doris Community! I'm a beginner user of Doris and I'm experiencing performance issues. I have *4 hosts for BE: Intel Xeon 80 cores, 768GB RAM each.* Frontend and broker were deployed on a different machine with the same configuration. All backends have "*mix*" as *Node Role*. Doris ver