jiayuasu commented on issue #2779:
URL: https://github.com/apache/sedona/issues/2779#issuecomment-4140198013
Hi @nabilkazi27
We investigated this issue and here is what we found
## Investigation Summary
**Reproduced on:** AWS Glue 5.0, Spark 3.5.4-amzn-0, Sedona 1.8.1, Lake
Formation FGAC enabled
### Root Cause
AWS Glue's FGAC mode wraps Spark's function registry with
`SecureFunctionRegistry`, which serializes all function registrations via
gRPC to a separate "system driver" process. This fails for Sedona at two
levels:
1. **Registration** — Scala closures used as function builders are
"synthetic types" that FGAC's `TransformationEngine` serializer rejects
(`Cannot parse synthetic types`). Named Java/Scala classes are also
rejected by the FGAC class allowlist (`Disallowed class cannot be resolved
from the serialization`).
2. **Execution** — Even if registration succeeds, the system driver
validates all classes referenced in SQL plans.
`org.locationtech.jts.geom.Geometry` and all Sedona UDTs are rejected
(`Invalid class name Class[org.locationtech.jts.geom.Geometry]`).
### Recommendation
FGAC and Sedona are incompatible at the query execution level. Users should
either:
- Run Sedona workloads in a separate Glue job **without**
`--enable-lakeformation-fine-grained-access`
- Use FGAC only for non-spatial tables and queries
- Request AWS to add Sedona/JTS classes to the FGAC allowlist
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
