MonkeyCanCode opened a new pull request, #2049: URL: https://github.com/apache/polaris/pull/2049
Currently we are using `<git_root_dir>/polaris` as the entrypoint to interact with polaris CLI. This is a okay for local usage but it is not ideal for others who doesn't want to downlaod the entire code base to just use the CLI as well as making polaris client distributable as part of release. To address this issue, we will need to public polaris as an CLI without depends on the bash script. This PR addressed this problem with following changes: 1. create main in `polaris_cli.py` as that will be the entrypoint for us to use 2. set scripts directly (which is point to item 1) and include both polaris and cli directories as part of packaging. Due to `client/python/polais` is auto generated from OpenAPI and `poetry.core.masonry.api` doe build backend (with `poetry.core.masonry.api` as build backend, it only includes files that are tracked by git. in this case, then auto generated files won't be track without including adding an include clause) 3. add couple more comments in the bash script `polaris` and re-organize the cleanup logic 4. Update the invoke command when calling python `polaris` within bash script `polaris` Here is a quick demo on the packaging: setup poetry then perform a poetry build (which we can use later to distribute the client code as well as push to [Python Package Index](https://pypi.org/)): ``` ➜ polaris git:(polaris_cli_package) python3 -m venv venv ➜ polaris git:(polaris_cli_package) source venv/bin/activate (venv) ➜ polaris git:(polaris_cli_package) pip install -r regtests/requirements.txt ... [notice] A new release of pip is available: 24.2 -> 25.1.1 [notice] To update, run: pip install --upgrade pip (venv) ➜ python git:(polaris_cli_package) poetry build Building polaris (1.0.0) Building sdist - Building sdist - Built polaris-1.0.0.tar.gz Building wheel - Building wheel - Built polaris-1.0.0-py3-none-any.whl ``` now assuming I am an user who doesn't want to download the code base but want to use the cli: ``` ➜ Desktop python3 -m venv venv_dist ➜ Desktop source venv_dist/bin/activate (venv_dist) ➜ Desktop pip freeze (venv_dist) ➜ Desktop (venv_dist) ➜ Desktop pip install GitHome/polaris/client/python/dist/polaris-1.0.0.tar.gz ... Building wheels for collected packages: polaris Building wheel for polaris (pyproject.toml) ... done Created wheel for polaris: filename=polaris-1.0.0-py3-none-any.whl size=520413 sha256=9c9a25c6edb2a0b642666ab07abf5bc52f0a51f939095b18f12404bb118ec2e5 Stored in directory: /Users/yong/Library/Caches/pip/wheels/b8/f1/20/1dd4b05f93820742954b32d0d44f211007e0d06e5742927628 Successfully built polaris Installing collected packages: urllib3, typing-extensions, six, jmespath, annotated-types, typing-inspection, python-dateutil, pydantic-core, pydantic, botocore, s3transfer, boto3, polaris Successfully installed annotated-types-0.7.0 boto3-1.38.36 botocore-1.38.46 jmespath-1.0.1 polaris-1.0.0 pydantic-2.11.7 pydantic-core-2.33.2 python-dateutil-2.9.0.post0 s3transfer-0.13.0 six-1.17.0 typing-extensions-4.14.1 typing-inspection-0.4.1 urllib3-2.5.0 [notice] A new release of pip is available: 24.2 -> 25.1.1 [notice] To update, run: pip install --upgrade pip (venv_dist) ➜ Desktop which polaris /Users/yong/Desktop/venv_dist/bin/polaris (venv_dist) ➜ Desktop cat /Users/yong/Desktop/venv_dist/bin/polaris #!/Users/yong/Desktop/venv_dist/bin/python3.13 # -*- coding: utf-8 -*- import re import sys from cli.polaris_cli import main if __name__ == '__main__': sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0]) sys.exit(main()) (venv_dist) ➜ Desktop SCRIPT_DIR="dummy" polaris --help input: polaris --help options: polaris --host hostname --port port --base-url complete base URL instead of hostname:port --client-id client ID for token-based authentication --client-secret client secret for token-based authentication --access-token access token for token-based authentication --profile profile for token-based authentication --proxy proxy URL ... ``` Here is a quick demo on the updated bash script: ``` ➜ polaris git:(polaris_cli_package) ./polaris --> Performing first-time setup for the Python client... --> Upgrading pip in virtual environment... Requirement already satisfied: pip in ./polaris-venv/lib/python3.13/site-packages (24.2) Collecting pip Using cached pip-25.1.1-py3-none-any.whl.metadata (3.6 kB) Using cached pip-25.1.1-py3-none-any.whl (1.8 MB) Installing collected packages: pip Attempting uninstall: pip Found existing installation: pip 24.2 Uninstalling pip-24.2: Successfully uninstalled pip-24.2 Successfully installed pip-25.1.1 --> Installing test requirements (including Poetry) into virtual environment... Collecting poetry==2.1.3 (from -r ./regtests/requirements.txt (line 20)) Using cached poetry-2.1.3-py3-none-any.whl.metadata (7.1 kB) Collecting build<2.0.0,>=1.2.1 (from poetry==2.1.3->-r ./regtests/requirements.txt (line 20)) Using cached build-1.2.2.post1-py3-none-any.whl.metadata (6.5 kB) Collecting cachecontrol<0.15.0,>=0.14.0 (from cachecontrol[filecache]<0.15.0,>=0.14.0->poetry==2.1.3->-r ./regtests/requirements.txt (line 20)) Using cached cachecontrol-0.14.3-py3-none-any.whl.metadata (3.1 kB) Collecting cleo<3.0.0,>=2.1.0 (from poetry==2.1.3->-r ./regtests/requirements.txt (line 20)) Using cached cleo-2.1.0-py3-none-any.whl.metadata (12 kB) Collecting dulwich<0.23.0,>=0.22.6 (from poetry==2.1.3->-r ./regtests/requirements.txt (line 20)) Using cached dulwich-0.22.8-cp313-cp313-macosx_11_0_arm64.whl.metadata (4.9 kB) Collecting fastjsonschema<3.0.0,>=2.18.0 (from poetry==2.1.3->-r ./regtests/requirements.txt (line 20)) Using cached fastjsonschema-2.21.1-py3-none-any.whl.metadata (2.2 kB) Collecting findpython<0.7.0,>=0.6.2 (from poetry==2.1.3->-r ./regtests/requirements.txt (line 20)) Using cached findpython-0.6.3-py3-none-any.whl.metadata (5.3 kB) Collecting installer<0.8.0,>=0.7.0 (from poetry==2.1.3->-r ./regtests/requirements.txt (line 20)) Using cached installer-0.7.0-py3-none-any.whl.metadata (936 bytes) Collecting keyring<26.0.0,>=25.1.0 (from poetry==2.1.3->-r ./regtests/requirements.txt (line 20)) Using cached keyring-25.6.0-py3-none-any.whl.metadata (20 kB) Collecting packaging>=24.0 (from poetry==2.1.3->-r ./regtests/requirements.txt (line 20)) Using cached packaging-25.0-py3-none-any.whl.metadata (3.3 kB) Collecting pbs-installer<2026.0.0,>=2025.1.6 (from pbs-installer[download,install]<2026.0.0,>=2025.1.6->poetry==2.1.3->-r ./regtests/requirements.txt (line 20)) Using cached pbs_installer-2025.7.12-py3-none-any.whl.metadata (991 bytes) Collecting pkginfo<2.0,>=1.12 (from poetry==2.1.3->-r ./regtests/requirements.txt (line 20)) Using cached pkginfo-1.12.1.2-py3-none-any.whl.metadata (13 kB) Collecting platformdirs<5,>=3.0.0 (from poetry==2.1.3->-r ./regtests/requirements.txt (line 20)) Using cached platformdirs-4.3.8-py3-none-any.whl.metadata (12 kB) Collecting poetry-core==2.1.3 (from poetry==2.1.3->-r ./regtests/requirements.txt (line 20)) Using cached poetry_core-2.1.3-py3-none-any.whl.metadata (3.5 kB) Collecting pyproject-hooks<2.0.0,>=1.0.0 (from poetry==2.1.3->-r ./regtests/requirements.txt (line 20)) Using cached pyproject_hooks-1.2.0-py3-none-any.whl.metadata (1.3 kB) Collecting requests<3.0,>=2.26 (from poetry==2.1.3->-r ./regtests/requirements.txt (line 20)) Using cached requests-2.32.4-py3-none-any.whl.metadata (4.9 kB) Collecting requests-toolbelt<2.0.0,>=1.0.0 (from poetry==2.1.3->-r ./regtests/requirements.txt (line 20)) Using cached requests_toolbelt-1.0.0-py2.py3-none-any.whl.metadata (14 kB) Collecting shellingham<2.0,>=1.5 (from poetry==2.1.3->-r ./regtests/requirements.txt (line 20)) Using cached shellingham-1.5.4-py2.py3-none-any.whl.metadata (3.5 kB) Collecting tomlkit<1.0.0,>=0.11.4 (from poetry==2.1.3->-r ./regtests/requirements.txt (line 20)) Using cached tomlkit-0.13.3-py3-none-any.whl.metadata (2.8 kB) Collecting trove-classifiers>=2022.5.19 (from poetry==2.1.3->-r ./regtests/requirements.txt (line 20)) Using cached trove_classifiers-2025.5.9.12-py3-none-any.whl.metadata (2.3 kB) Collecting virtualenv<21.0.0,>=20.26.6 (from poetry==2.1.3->-r ./regtests/requirements.txt (line 20)) Using cached virtualenv-20.31.2-py3-none-any.whl.metadata (4.5 kB) Collecting xattr<2.0.0,>=1.0.0 (from poetry==2.1.3->-r ./regtests/requirements.txt (line 20)) Using cached xattr-1.2.0-cp313-cp313-macosx_11_0_arm64.whl.metadata (3.8 kB) Collecting msgpack<2.0.0,>=0.5.2 (from cachecontrol<0.15.0,>=0.14.0->cachecontrol[filecache]<0.15.0,>=0.14.0->poetry==2.1.3->-r ./regtests/requirements.txt (line 20)) Using cached msgpack-1.1.1-cp313-cp313-macosx_11_0_arm64.whl.metadata (8.4 kB) Collecting filelock>=3.8.0 (from cachecontrol[filecache]<0.15.0,>=0.14.0->poetry==2.1.3->-r ./regtests/requirements.txt (line 20)) Using cached filelock-3.18.0-py3-none-any.whl.metadata (2.9 kB) Collecting crashtest<0.5.0,>=0.4.1 (from cleo<3.0.0,>=2.1.0->poetry==2.1.3->-r ./regtests/requirements.txt (line 20)) Using cached crashtest-0.4.1-py3-none-any.whl.metadata (1.1 kB) Collecting rapidfuzz<4.0.0,>=3.0.0 (from cleo<3.0.0,>=2.1.0->poetry==2.1.3->-r ./regtests/requirements.txt (line 20)) Using cached rapidfuzz-3.13.0-cp313-cp313-macosx_11_0_arm64.whl.metadata (12 kB) Collecting urllib3>=1.25 (from dulwich<0.23.0,>=0.22.6->poetry==2.1.3->-r ./regtests/requirements.txt (line 20)) Using cached urllib3-2.5.0-py3-none-any.whl.metadata (6.5 kB) Collecting jaraco.classes (from keyring<26.0.0,>=25.1.0->poetry==2.1.3->-r ./regtests/requirements.txt (line 20)) Using cached jaraco.classes-3.4.0-py3-none-any.whl.metadata (2.6 kB) Collecting jaraco.functools (from keyring<26.0.0,>=25.1.0->poetry==2.1.3->-r ./regtests/requirements.txt (line 20)) Using cached jaraco_functools-4.2.1-py3-none-any.whl.metadata (2.9 kB) Collecting jaraco.context (from keyring<26.0.0,>=25.1.0->poetry==2.1.3->-r ./regtests/requirements.txt (line 20)) Using cached jaraco.context-6.0.1-py3-none-any.whl.metadata (4.1 kB) Collecting httpx<1,>=0.27.0 (from pbs-installer[download,install]<2026.0.0,>=2025.1.6->poetry==2.1.3->-r ./regtests/requirements.txt (line 20)) Using cached httpx-0.28.1-py3-none-any.whl.metadata (7.1 kB) Collecting zstandard>=0.21.0 (from pbs-installer[download,install]<2026.0.0,>=2025.1.6->poetry==2.1.3->-r ./regtests/requirements.txt (line 20)) Using cached zstandard-0.23.0-cp313-cp313-macosx_11_0_arm64.whl.metadata (3.0 kB) Collecting anyio (from httpx<1,>=0.27.0->pbs-installer[download,install]<2026.0.0,>=2025.1.6->poetry==2.1.3->-r ./regtests/requirements.txt (line 20)) Using cached anyio-4.9.0-py3-none-any.whl.metadata (4.7 kB) Collecting certifi (from httpx<1,>=0.27.0->pbs-installer[download,install]<2026.0.0,>=2025.1.6->poetry==2.1.3->-r ./regtests/requirements.txt (line 20)) Using cached certifi-2025.7.14-py3-none-any.whl.metadata (2.4 kB) Collecting httpcore==1.* (from httpx<1,>=0.27.0->pbs-installer[download,install]<2026.0.0,>=2025.1.6->poetry==2.1.3->-r ./regtests/requirements.txt (line 20)) Using cached httpcore-1.0.9-py3-none-any.whl.metadata (21 kB) Collecting idna (from httpx<1,>=0.27.0->pbs-installer[download,install]<2026.0.0,>=2025.1.6->poetry==2.1.3->-r ./regtests/requirements.txt (line 20)) Using cached idna-3.10-py3-none-any.whl.metadata (10 kB) Collecting h11>=0.16 (from httpcore==1.*->httpx<1,>=0.27.0->pbs-installer[download,install]<2026.0.0,>=2025.1.6->poetry==2.1.3->-r ./regtests/requirements.txt (line 20)) Using cached h11-0.16.0-py3-none-any.whl.metadata (8.3 kB) Collecting charset_normalizer<4,>=2 (from requests<3.0,>=2.26->poetry==2.1.3->-r ./regtests/requirements.txt (line 20)) Using cached charset_normalizer-3.4.2-cp313-cp313-macosx_10_13_universal2.whl.metadata (35 kB) Collecting distlib<1,>=0.3.7 (from virtualenv<21.0.0,>=20.26.6->poetry==2.1.3->-r ./regtests/requirements.txt (line 20)) Using cached distlib-0.3.9-py2.py3-none-any.whl.metadata (5.2 kB) Collecting cffi>=1.16.0 (from xattr<2.0.0,>=1.0.0->poetry==2.1.3->-r ./regtests/requirements.txt (line 20)) Using cached cffi-1.17.1-cp313-cp313-macosx_11_0_arm64.whl.metadata (1.5 kB) Collecting pycparser (from cffi>=1.16.0->xattr<2.0.0,>=1.0.0->poetry==2.1.3->-r ./regtests/requirements.txt (line 20)) Using cached pycparser-2.22-py3-none-any.whl.metadata (943 bytes) Collecting sniffio>=1.1 (from anyio->httpx<1,>=0.27.0->pbs-installer[download,install]<2026.0.0,>=2025.1.6->poetry==2.1.3->-r ./regtests/requirements.txt (line 20)) Using cached sniffio-1.3.1-py3-none-any.whl.metadata (3.9 kB) Collecting more-itertools (from jaraco.classes->keyring<26.0.0,>=25.1.0->poetry==2.1.3->-r ./regtests/requirements.txt (line 20)) Using cached more_itertools-10.7.0-py3-none-any.whl.metadata (37 kB) Using cached poetry-2.1.3-py3-none-any.whl (278 kB) Using cached poetry_core-2.1.3-py3-none-any.whl (332 kB) Using cached build-1.2.2.post1-py3-none-any.whl (22 kB) Using cached cachecontrol-0.14.3-py3-none-any.whl (21 kB) Using cached cleo-2.1.0-py3-none-any.whl (78 kB) Using cached crashtest-0.4.1-py3-none-any.whl (7.6 kB) Using cached dulwich-0.22.8-cp313-cp313-macosx_11_0_arm64.whl (915 kB) Using cached fastjsonschema-2.21.1-py3-none-any.whl (23 kB) Using cached findpython-0.6.3-py3-none-any.whl (20 kB) Using cached installer-0.7.0-py3-none-any.whl (453 kB) Using cached keyring-25.6.0-py3-none-any.whl (39 kB) Using cached msgpack-1.1.1-cp313-cp313-macosx_11_0_arm64.whl (78 kB) Using cached pbs_installer-2025.7.12-py3-none-any.whl (59 kB) Using cached httpx-0.28.1-py3-none-any.whl (73 kB) Using cached httpcore-1.0.9-py3-none-any.whl (78 kB) Using cached pkginfo-1.12.1.2-py3-none-any.whl (32 kB) Using cached platformdirs-4.3.8-py3-none-any.whl (18 kB) Using cached pyproject_hooks-1.2.0-py3-none-any.whl (10 kB) Using cached rapidfuzz-3.13.0-cp313-cp313-macosx_11_0_arm64.whl (1.4 MB) Using cached requests-2.32.4-py3-none-any.whl (64 kB) Using cached charset_normalizer-3.4.2-cp313-cp313-macosx_10_13_universal2.whl (199 kB) Using cached idna-3.10-py3-none-any.whl (70 kB) Using cached requests_toolbelt-1.0.0-py2.py3-none-any.whl (54 kB) Using cached shellingham-1.5.4-py2.py3-none-any.whl (9.8 kB) Using cached tomlkit-0.13.3-py3-none-any.whl (38 kB) Using cached urllib3-2.5.0-py3-none-any.whl (129 kB) Using cached virtualenv-20.31.2-py3-none-any.whl (6.1 MB) Using cached distlib-0.3.9-py2.py3-none-any.whl (468 kB) Using cached filelock-3.18.0-py3-none-any.whl (16 kB) Using cached xattr-1.2.0-cp313-cp313-macosx_11_0_arm64.whl (19 kB) Using cached certifi-2025.7.14-py3-none-any.whl (162 kB) Using cached cffi-1.17.1-cp313-cp313-macosx_11_0_arm64.whl (178 kB) Using cached h11-0.16.0-py3-none-any.whl (37 kB) Using cached packaging-25.0-py3-none-any.whl (66 kB) Using cached trove_classifiers-2025.5.9.12-py3-none-any.whl (14 kB) Using cached zstandard-0.23.0-cp313-cp313-macosx_11_0_arm64.whl (633 kB) Using cached anyio-4.9.0-py3-none-any.whl (100 kB) Using cached sniffio-1.3.1-py3-none-any.whl (10 kB) Using cached jaraco.classes-3.4.0-py3-none-any.whl (6.8 kB) Using cached jaraco.context-6.0.1-py3-none-any.whl (6.8 kB) Using cached jaraco_functools-4.2.1-py3-none-any.whl (10 kB) Using cached more_itertools-10.7.0-py3-none-any.whl (65 kB) Using cached pycparser-2.22-py3-none-any.whl (117 kB) Installing collected packages: trove-classifiers, fastjsonschema, distlib, zstandard, urllib3, tomlkit, sniffio, shellingham, rapidfuzz, pyproject-hooks, pycparser, poetry-core, platformdirs, pkginfo, pbs-installer, packaging, msgpack, more-itertools, jaraco.context, installer, idna, h11, filelock, crashtest, charset_normalizer, certifi, virtualenv, requests, jaraco.functools, jaraco.classes, httpcore, findpython, dulwich, cleo, cffi, build, anyio, xattr, requests-toolbelt, keyring, httpx, cachecontrol, poetry Successfully installed anyio-4.9.0 build-1.2.2.post1 cachecontrol-0.14.3 certifi-2025.7.14 cffi-1.17.1 charset_normalizer-3.4.2 cleo-2.1.0 crashtest-0.4.1 distlib-0.3.9 dulwich-0.22.8 fastjsonschema-2.21.1 filelock-3.18.0 findpython-0.6.3 h11-0.16.0 httpcore-1.0.9 httpx-0.28.1 idna-3.10 installer-0.7.0 jaraco.classes-3.4.0 jaraco.context-6.0.1 jaraco.functools-4.2.1 keyring-25.6.0 more-itertools-10.7.0 msgpack-1.1.1 packaging-25.0 pbs-installer-2025.7.12 pkginfo-1.12.1.2 platformdirs-4.3.8 poetry-2.1.3 poetry-core-2.1.3 pycparser-2.22 pyproject-hooks-1.2.0 rapidfuzz-3.13.0 requests-2.32.4 requests-toolbelt-1.0.0 shellingham-1.5.4 sniffio-1.3.1 tomlkit-0.13.3 trove-classifiers-2025.5.9.12 urllib3-2.5.0 virtualenv-20.31.2 xattr-1.2.0 zstandard-0.23.0 --> Installing 'polaris' package using Poetry... Updating dependencies Resolving dependencies... (2.5s) Package operations: 46 installs, 0 updates, 0 removals - Installing six (1.17.0) - Installing mdurl (0.1.2) - Installing jmespath (1.0.1) - Installing python-dateutil (2.9.0.post0) - Installing typing-extensions (4.14.1) - Installing pluggy (1.6.0) - Installing iniconfig (2.1.0) - Installing botocore (1.38.46) - Installing markdown-it-py (3.0.0) - Installing annotated-types (0.7.0) - Installing pydantic-core (2.33.2) - Installing pygments (2.19.2) - Installing typing-inspection (0.4.1) - Installing cachetools (5.5.2) - Installing cfgv (3.4.0) - Installing chardet (5.2.0) - Installing click (8.2.1) - Installing colorama (0.4.6) - Installing coverage (7.9.2) - Installing fsspec (2025.5.1) - Installing identify (2.6.12) - Installing mccabe (0.7.0) - Installing mmh3 (5.1.0) - Installing mypy-extensions (1.1.0) - Installing nodeenv (1.9.1) - Installing pathspec (0.12.1) - Installing pycodestyle (2.14.0) - Installing pydantic (2.11.7) - Installing pyflakes (3.4.0) - Installing pyparsing (3.2.3) - Installing pytest (8.4.1) - Installing pyproject-api (1.9.1) - Installing rich (13.9.4) - Installing s3transfer (0.13.0) - Installing sortedcontainers (2.4.0) - Installing pyyaml (6.0.2) - Installing strictyaml (1.7.3) - Installing tenacity (9.1.2) - Installing boto3 (1.38.36) - Installing pytest-cov (6.2.1) - Installing tox (4.27.0) - Installing types-python-dateutil (2.9.0.20250708) - Installing flake8 (7.3.0) - Installing mypy (1.16.1) - Installing pre-commit (4.2.0) - Installing pyiceberg (0.9.1) Writing lock file Installing the current project: polaris (1.0.0) ✅ Dependencies repaired. --> Activating virtual environment for CLI execution: source ./polaris-venv/bin/activate --> SCRIPT_DIR is set to: . --> Running Polaris CLI... Traceback (most recent call last): File "/Users/yong/Desktop/GitHome/polaris/polaris-venv/bin/polaris", line 6, in <module> sys.exit(main()) ~~~~^^ File "/Users/yong/Desktop/GitHome/polaris/client/python/cli/polaris_cli.py", line 195, in main PolarisCli.execute() ~~~~~~~~~~~~~~~~~~^^ File "/Users/yong/Desktop/GitHome/polaris/client/python/cli/polaris_cli.py", line 68, in execute client_builder = PolarisCli._get_client_builder(options) File "/Users/yong/Desktop/GitHome/polaris/client/python/cli/polaris_cli.py", line 151, in _get_client_builder raise Exception( ...<5 lines>... ) Exception: Please provide credentials via either --client-id & --client-secret or --access-token. Alternatively, you may set the environment variables CLIENT_ID & CLIENT_SECRET. ``` By doing so, we really don't need bash script `polaris` (at least the name is confusing with this change...instead, it should be client_setup.sh or something similar) and end-users can do regular pip install to setup Polaris CLI. To avoid too much change, I will keep the bash script `polaris` for now (this script is still useful for initial setup as well as refresh dependencies etc. but with proper Python Package Index, we should just do pip upgrade instead). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
