villebro commented on PR #2288: URL: https://github.com/apache/datafusion-ballista/pull/2288#issuecomment-5296931717
> I haven't used this functionality but I would definitely recommend either making standalone proxy process or different thread pool for proxy @milenkovicm this was precisely what I was going to propose. Based on experience from other popular distributed query engines, they typically suffer from scheduler congestion (especially the ones that don't have HA scheduler support), due both to complex planing and result proxying. If Ballista can decouple scheduler from result proxy and both can scale horizontally, we would be able to start with minimal footprint and adjust automatically as traffic increases. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
