alamb commented on issue #2703: URL: https://github.com/apache/datafusion/issues/2703#issuecomment-2120780299
Hi @leoluan2009 In my opinion, I don't think DataFusion needs JIT to get good performance. In general, I find the paper ["Everything You Always Wanted to Know About Compiled and Vectorized Queries But Were Afraid to Ask"](https://www.vldb.org/pvldb/vol11/p2209-kersten.pdf) to explain the tradeoffs well DataFusion is a vectorized engine and we haven't found areas where JIt would be compelling compared to vectorized code. The only area I can really think of would be to implement type specialized comparisons for sorting (to avoid the RowFormat) but we would need to have a pretty compelling benchmark showing improvements to justify I think -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
