manuzhang commented on issue #3703:
URL: https://github.com/apache/iceberg/issues/3703#issuecomment-2054454593
@RussellSpitzer I wonder where we are now for this issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use t
RLashofRegas commented on issue #3703:
URL: https://github.com/apache/iceberg/issues/3703#issuecomment-1769511550
Ah. Thanks! setting `spark.sql.autoBroadcastJoinThreshold=-1` fixed it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to
RussellSpitzer commented on issue #3703:
URL: https://github.com/apache/iceberg/issues/3703#issuecomment-1769486606
That's the table metadata. If your JSON file is 8gb large you have other
problems :)
--
This is an automated message from the Apache Git Service.
To respond to the message,
RLashofRegas commented on issue #3703:
URL: https://github.com/apache/iceberg/issues/3703#issuecomment-1769378796
@RussellSpitzer Maybe I'm missing something but I don't think I can disable
the broadcast in this case. `spark.sql.autoBroadcastJoinThreshold=-1` will only
disable automatic bro
RussellSpitzer commented on issue #3703:
URL: https://github.com/apache/iceberg/issues/3703#issuecomment-1767362201
Not for broadcast, for that you just need to disable broadcast join in spark
--
This is an automated message from the Apache Git Service.
To respond to the message, please l
RLashofRegas commented on issue #3703:
URL: https://github.com/apache/iceberg/issues/3703#issuecomment-1767331436
@RussellSpitzer You mentioned you were working on a patch that might affect
this. Should I expect the issue I mentioned above to go away if we upgraded to
a more recent version
RLashofRegas commented on issue #3703:
URL: https://github.com/apache/iceberg/issues/3703#issuecomment-1765512670
@dchristle What was your solution that fixed the `Cannot broadcast the table
that is larger than 8GB` issue? I just ran into the same problem on expire
snapshots. I am using `ma
