Kontinuation commented on PR #737: URL: https://github.com/apache/sedona-db/pull/737#issuecomment-4148267435
> @Kontinuation if you have bandwidth, do you have an opinion on whether or not it is worth threading the `SpatialJoinProvider` through the spill reader machinery is worth it? (It's a bit ugly but can be done). (Also any critiques of the approach are very welcome!) > Details I think the performance drop for compressed spilling is tolerable considering the benefit of less dependency on other components. I also wonder will writing bounding boxes to spill files always be better when working with geography data, since the equibox of geography could be computationally intensive. I also wonder is it overly complex to support geography by implementing a new `SpatialJoinProvider`. We know the sedona_type of the geometries in the `EvaluatedGeometryArray`, so we can support different ways to evaluate bounding boxes within `EvaluatedGeometryArray` itself, and even serialize the spilled batches in its mostly optimized form (with or without rects) according to the value of sedona_type. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
