HonahX commented on PR #283:
URL: https://github.com/apache/iceberg-python/pull/283#issuecomment-1902539269
Hi @Fokko.
> Maybe we should pass that in through an environment variable
Do you mean putting it in a `.env` file? Also, I am thinking if it is
possible to link the iceberg-spark-runtime 's version with our pyspark version
in `pyproject.toml`.
```python
spark_version =
".".join(importlib.metadata.version("pyspark").split(".")[:2])
os.environ["PYSPARK_SUBMIT_ARGS"] = (
f"--packages
org.apache.iceberg:iceberg-spark-runtime-{spark_version}_2.12:1.4.0,org.apache.iceberg:iceberg-aws-bundle:1.4.0
pyspark-shell"
)
```
So that next time the dependabot can automatically update the dependency or
fail if iceberg has not supported the version yet. Does this sound good to you?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For additional commands, e-mail: issues-h...@iceberg.apache.org
