The pleasant thing about this tooling is that 1) the packages are installable via pip and usable without configuring any sort of Rust environment for the user and 2) the core dependencies are on the Rust side, so we're not pulling in any particularly strange Python dependencies. The Python-only experience is shockingly straightforward as a result, just install them like you would pylint, flake8, isort, or mypy and use them from the command line within a virtual environment. The end result is a pretty lightweight change in terms of how you use the tools, just *much* faster (running pyrefly and ruff over the Beam Python codebase is so fast that I genuinely didn't think they ran at first.)
I'll update the doc with an explanation of how to install the tools later today, which should add some clarity. On Tue, Mar 31, 2026 at 5:15 PM Robert Bradshaw <[email protected]> wrote: > +1 to this investigation > > To me the primary question is what would adopting these tools look like > for a Python-only developer? If these tools are packaged up to look like > "ordinary" python packages without requiring extra (and, especially, not > arcane) dependencies, this seems like it could be a net win (but that > should probably be explored as another section of the doc). > > > On Tue, Mar 31, 2026 at 12:29 PM Jack McCluskey via dev < > [email protected]> wrote: > >> Hey everyone, >> >> I've put together a brief investigation on using Rust-backed alternatives >> to common Python tools for linting and type checking, using the Beam >> repository's Python Linting PreCommit as the testing ground: >> >> https://s.apache.org/beam-python-rust-tooling >> >> The main points: >> >> - Current linting and static type checking libraries (pylint, flake8, >> mypy) are replaceable through pyrefly and ruff (with the sole exception of >> isort, which has a large amount of extra configuration in the repository >> that ruff cannot replicate yet.) >> - Swapping to the Rust-based tools cuts our linting workflow runtimes >> by about half (saving ~36 days of runtime over 2025) >> - Anecdotally, local use is also much faster and removes a common >> bottleneck for developer velocity >> >> I recommend moving Beam Python to these tools; however, I am interested >> in what the broader Beam community thinks about the change. Please take a >> look and let me know your thoughts. >> >> Thanks, >> >> Jack McCluskey >> >> -- >> >> >> Jack McCluskey >> SWE - DataPLS PLAT/ Dataflow ML >> RDU >> [email protected] >> >> >>
