nmarasoiu commented on issue #13742: URL: https://github.com/apache/iceberg/issues/13742#issuecomment-3155566729
Important Performance Caveat - Java's Object Allocation Reality While replacing BigDecimal with a more efficient decimal library (like decimal4j's Decimal64) would eliminate the cascading heap allocations of BigDecimal → BigInteger → int[], we must acknowledge a fundamental limitation: Java will still require heap allocation of decimal objects when used within the generic type system. Unlike languages with true value types, Java cannot stack-allocate objects passed through generics like Literal<DecimalType>. Even with escape analysis optimizations, objects that flow through Iceberg's generic APIs will be heap-allocated. This means while we can achieve: ✅ Single object allocation vs BigDecimal's object graph ✅ Better cache locality with primitive-backed storage ✅ Reduced GC pressure from fewer total allocations We cannot achieve: ❌ True zero-allocation decimal operations in the generic API ❌ Stack allocation of decimal values ❌ Primitive-level performance while maintaining type safety For applications requiring maximum decimal performance, this architectural constraint may eventually necessitate either: Specialized non-generic code paths for hot paths Code generation for primitive-specialized implementations Waiting for Project Valhalla's value types Hybrid approaches mixing primitive operations with object-based APIs The proposed change should be viewed as a significant performance improvement within Java's current constraints, not a complete solution to the value type problem. This gives context to reviewers about the trade-offs while still advocating for the practical improvement. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
