vogievetsky opened a new pull request, #20448: URL: https://github.com/apache/druid/pull/20448
### A note from the human So after looking at how well Claude Code running Opus 5.5 performed on https://github.com/apache/druid/pull/20421 I was sufficiently impressed to attempt an ambitious goal: update the console to all the latest dependencies as if the project was started today. I asked Claude to make me a "Command and Conquer" tech tree of updates that need to be made. <img width="458" height="326" alt="image" src="https://github.com/user-attachments/assets/95ccce29-c515-43c7-bb0d-de287e699242"; /> I then had Claude work through them one by one (see the order of the commits in this PR). I am extremely pleased with the result. The styling was all kept pixel perfect. Claude even fixed some bugs along the way. Maybe in a subsequent PR we can relax the style overrides and lean in more into the new Blueprint v6 style but for this PR I wanted pixel perfection as a goal for Claude. I decided to combine all these changes into one mega PR because a lot of these changes were huge on their own and this mega PR is not that much more complex in aggregate. I realize this will create a huge merge conflict if someone (like me!) has a fork or feature branch with web console changes in it, I have therefore **included a prompt** at the bottom of the description that you should use to agenticaly migrate your feature branch to the new stuff. If you use Opus 5.5 I am confident it will one-shot it. > Human out --- ### Description This PR moves the web console onto the current major versions of its framework and tooling: React 19, Blueprint 6, TypeScript 6, Ace 1.44, Prettier 3 and Node 24. It also replaces the libraries that have no React 19 support: React Router, react-table v6, react-shallow-renderer and react-diff-viewer. Blueprint 6 is restyled to look exactly like the Blueprint 5 console, so none of this should be visible to users, apart from the bug fixes listed at the end. The branch is a series of self-contained commits, one per migration, and each one builds and passes the tests on its own. Reviewing commit by commit is recommended. #### React 19 - `react` / `react-dom` / types 18.3 → 19.3; the `@types/react` npm override is gone. - `forwardRef` removed in favor of `ref` as a prop (`SqlInput`, `FlexibleQueryInput`, `FilterPane`, `DroppableContainer`); `useRef` calls get an explicit initial value; ref callbacks no longer return a value; `<Context value>` instead of `<Context.Provider value>`. - The React Compiler lint rules `react-hooks/refs` and `react-hooks/set-state-in-effect` (disabled with TODOs in the previous upgrade) are back on, and the hooks (`useConstant`, `useLastDefined`, `useLastDefinedDeep`, `useMemoWithPrevious`, `useQueryManager`) and components no longer read refs during render or set state in effects. - Every `react-hooks/exhaustive-deps` suppression and warning is replaced with `useEffectEvent` or with the correct dependencies. There are no lint suppressions left. #### Libraries replaced because they do not support React 19 - **react-table v6 → `ConsoleTable`** (`src/components/console-table`), built on TanStack Table v9. Its props are a drop-in replacement for how the console used react-table (`Header`, `accessor`, `Cell`, `Aggregated`, `show`, `pivotBy`, `SubComponent`, `manual`, `sorted`, `filtered`, `page`…). The rendered DOM keeps react-table's structure with `.ReactTable` / `rt-*` renamed to `.console-table` / `ct-*` (`rt-noData` → `ct-no-data`). Cell render props are now typed with the row type, expanded groups no longer collapse whenever the parent re-renders, and changes to `defaultPageSize` are now applied. `bootstrapReactTable` is gone. - **React Router 5 → a small hash router** (`src/utils/hash-routing.ts`, routing in `ConsoleApplication.renderView`). React Router 6+ has no "noslash" hash mode and decodes route params, so upgrading would have changed every console URL and broken table filters containing an encoded `&`, `%` or `/`. The existing semantics are kept: `#view/param` URLs, `decodeURI` of the path, case-insensitive views, `#/view` normalized to `#view`, the ingestion and query redirects, and capability-gated views. - **react-shallow-renderer → React Testing Library.** The 36 specs that used shallow rendering now do full renders. A DOM snapshot serializer (`src/test-utils/snapshot-serializer.ts`) and component stubs (`src/test-utils/stub-component.tsx`, including `consoleTableStub`) keep snapshots roughly as small as the shallow ones. `script/mkcomp` generates RTL specs. - **react-diff-viewer → react-diff-viewer-continued**, pinned to `~4.0.6`: 4.1+ pulls in `js-yaml`/`argparse` (Python-2.0 license, not in the allowlist), and 4.4 adds syntax highlighting, a web worker and lazy chunks (+109 KB). - **Popover → PopoverNext.** The legacy Blueprint `Popover` is built on react-popper, which does not support React 19. All 80 usages move to `PopoverNext` (`position` → `placement`, `minimal` → `animation="minimal" arrow={false}`). They keep `lazy` and `shouldReturnFocusOnClose={false}`, the legacy defaults the console relies on. #### Blueprint 6 - `@blueprintjs/core`, `icons`, `select` and `datetime` → 6. `datetime2` is removed (`DateRangePicker3` → `DateRangePicker`). - The CSS namespace changes from `bp5` to `bp6`. - `AppToaster.show` stays synchronous even though `OverlayToaster.create` is now async. - Blueprint 6's CSS design tokens are mapped onto the console palette and typography. The mapping is generated by `script/generate-blueprint-token-overrides.mjs`. - `src/blueprint-overrides/_blueprint5-look.scss` restores the Blueprint 5 metrics and dark theme the console was designed with. It was checked against the previous build with computed-style and screenshot diffs. - Deprecated props are migrated: - `minimal` → `variant="minimal"` - `small` / `large` → `size` - `rightIcon` → `endIcon` - `Alignment.LEFT` / `RIGHT` → `START` / `END` - `Keys` / `keyCode` → `key` - `@typescript-eslint/no-deprecated` is now on, so new usages of deprecated APIs fail the lint. #### Ace 1.5 → 1.44, react-ace 10 → 15 - Completers are now created once and read the latest props. Ace only uses the first completers it is given, so completions used to be stuck with the props of the first render. For example, the workbench offered no column completions after column metadata loaded. - The `dsql` and `hjson` modes are now classes built on Ace's `TextMode` and registered once. `initAceDsqlMode` resets the cached mode so later editors get the cluster's functions. - Functions with several signatures use the completion caption. Sub-query gutter markers work whatever element is under the mouse. The doc tooltip keeps its look. - Snapshots describe each Ace editor (mode, value, placeholder) instead of Ace's DOM. #### TypeScript, Sass, Prettier, Node - **TypeScript 5.5 → 6.0**: - `moduleResolution` is `bundler` instead of `node`. - `types` is `["jest"]`. - `target` is es2022. - `lib` is `dom` + `esnext`. - `experimentalDecorators` is removed. - `regenerator-runtime` is removed. - **Sass `@import` → `@use`:** - Components `@use` the `variables` module, which forwards the palette and the Blueprint variable overrides. - Blueprint's own sources still use `@import`, so they are loaded with `meta.load-css()` (`src/blueprint-overrides/_blueprint.scss`). - The compiled CSS is unchanged, and the build no longer prints Sass warnings. - **Prettier 2 → 3.9** with `@awesome-code-style/prettier-config` 5. The code is reformatted with the new defaults (for example, nested ternaries are now indented). - **Node 22.23.3 → 24.21.0 and npm 10.9.9 → 11.19.0** in `pom.xml` and `volta`, with `engines` set to `>=24`. - `license-checker-rseidelsohn` is replaced by its successor `@lizenz/checker`. `licenses.yaml` is unchanged by that switch. - An npm 11 `allowScripts` entry allows `playwright-chromium`, and denies the prebuilt-binary fallbacks of `@parcel/watcher` and `unrs-resolver`. #### Housekeeping - **Upgraded:** - memoize-one 6 - copy-to-clipboard 4 (clipboard writes go through a `copyToClipboard` helper that keeps the prompt fallback) - @flatten-js/interval-tree 2 - zustand 5 - uuid 14 - globals 17 - postcss-preset-env 11 - **Removed as unused:** - `date-fns` - `@types/enzyme*` - `ts-node` - `fs-extra` - **Gitignore:** top-level `*.jpeg` files (the e2e tests write their error screenshots there) are now ignored. #### Bug fixes that came out of the migrations - Workbench SQL completions now include columns once their metadata loads (the stale Ace completers described above). - Pie chart: a second click on a slice now closes its bubble. The click handler used to read a stale highlight. - Explore grouping table: it now re-runs when the module filter changes (`moduleWhere` was missing from the memo dependencies). - Tables: expanded groups no longer collapse on unrelated re-renders. - Execution stages: a missing React `key` in the sort stats cell is fixed. #### Not upgraded - ESLint 10 is blocked by `eslint-plugin-react`. - stylelint 17 is blocked by the Blueprint stylelint plugin. - TypeScript 7 is blocked by typescript-eslint and ts-jest. - react-diff-viewer-continued stays on 4.0.x for the license and bundle size reasons above. #### Testing - `npm test` passes: typecheck, ESLint (0 warnings), Prettier and stylelint, plus 162 suites with 806 tests and 248 snapshots. - The production build has no warnings, and `npm run check-licenses` passes. - E2E against a local Druid through the dev server: all 5 specs pass. `multi-stage-query` can fail when run alongside the other specs with a "not enough task slots" dialog, because of their concurrent ingestion tasks. It passes on its own and fails the same way against the master console. - Every view was compared against the master console with computed-style and screenshot diffs, in addition to manual testing of tables (sorting, filtering, pivoting, pagination, resizing), popovers and menus, the SQL editor and completions, the data loaders and the Explore charts. <hr> This PR has: - [x] been self-reviewed. - [x] added or updated version, license, or notice information in [licenses.yaml](https://github.com/apache/druid/blob/master/dev/license.md) - [x] added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader. - [x] added unit tests or modified existing tests to cover new code paths, ensuring the threshold for [code coverage](https://github.com/apache/druid/blob/master/dev/code-review/code-coverage.md) is met. - [x] been tested in a test Druid cluster. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --- ## Prompt for migrating a fork or feature branch > Copy everything below into a Claude Code session opened at the root of the fork's Druid checkout. You are migrating the Apache Druid web console code in this repository (`web-console/`) so it builds on top of an upstream change that moved the console to React 19, Blueprint 6, TypeScript 6, Ace 1.44, Prettier 3, Sass modules, a custom hash router, a custom `ConsoleTable` (instead of react-table v6), React Testing Library (instead of shallow rendering) and Node 24. The fork or feature branch adds its own code (for example new views, components, dialogs or Explore modules) that was written against the old stack. Your job is to bring that code up to the new stack with **no visual or behavioral changes**, and without changing the upstream code except where the feature genuinely needs a hook into it. **Getting set up** 1. Find the upstream merge commit of the migration (search `git log` on upstream master for "feat: Web console: upgrade to the latest and greatest" and the neighboring "Web console:" commits). Read the upstream `web-console/CLAUDE.md` and the PR description for that change. List the fork's own web-console changes with `git diff <merge-base> <fork-branch> -- web-console` so you know exactly which files are the fork's. 2. Merge or rebase onto the new upstream. For conflicts in files the fork only touched lightly, take upstream and re-apply the fork's intent on top; do not try to keep the old versions of upstream files. 3. Use Node 24 (`volta pin [email protected] [email protected]` inside `web-console/` if Volta is used; never `volta install`, which changes the user's global default). Run `npm ci`. If the fork added dependencies, check that they support React 19; if they need install scripts, npm 11 requires them to be listed in `allowScripts` in `package.json`. **Migration checklist (apply to every fork-owned file)** - **Tables:** `react-table` is gone. Replace `ReactTable` with `ConsoleTable` from `src/components` (`console-table/console-table.tsx`). The props deliberately mirror react-table v6 (`data`, `columns`, `Header`, `accessor`, `Cell`, `Aggregated`, `Filter`, `filterMethod`, `sortMethod`, `show`, `width`, `pivotBy`, `SubComponent`, `manual`, `pages`, `sorted`/`onSortedChange`, `filtered`/`onFilteredChange`, `page`/`onPageChange`, `pageSize`/`onPageSizeChange`, `defaultPageSize`, `noDataText`…), so most conversions are a rename plus types: use `ConsoleTableColumn<Row>` and `ConsoleTableCellInfo<Row>` instead of `Column`/`CellInfo`/`RowInfo`, and drop `any` in cell render props. Props that `ConsoleTable` does not support (for example `getTrProps`/`getTdProps`) need a different approach; read the component before adding anything to it. In SCSS and e2e selectors, `.ReactTable` → `.console-table`, `.rt-*` → `.ct-*` (`rt-tbody` → `ct-tbody`, `rt-tr-group` → `ct-tr-group`, `rt-noDa ta` → `ct-no-data`, …), while react-table's `-odd`, `-padRow`, `-sort-asc`, `-filters`, `-pagination`… modifier classes keep their names. Anything that used `bootstrapReactTable` or react-table defaults should rely on the defaults built into `ConsoleTable`. - **Routing:** `react-router` / `react-router-dom` are gone. A new top-level view is a `case '<view>':` in `ConsoleApplication.renderView()` (`src/console-application.tsx`), gated on capabilities like its neighbors, and it gets its URL param (if any) as a prop from `parseHashRoute(hashPath).param`. Do not add `RouteComponentProps`, `Link`, `useHistory`, `withRouter` or `useParams`; navigate by setting `location.hash` (for example `location.hash = 'my-view/' + param`) the way the existing views do. The URL format is unchanged (`#view/param`, no leading slash, case-insensitive view, `decodeURI` applied to the path). New redirects go in `REDIRECTS`. Add a header bar entry if the view needs one (`src/components/header-bar`). - **React 19:** - Replace `forwardRef` with `ref` as a regular prop. - Give every `useRef` an initial value (`useRef<T>(undefined)`). - Ref callbacks must not return a value, so use `el => { x = el; }`. - Use `<SomeContext value={…}>` instead of `<SomeContext.Provider value={…}>`. - Drop `defaultProps` on function components and remove string refs. - The React Compiler lint rules are on: - Do not read `ref.current` during render. - Do not call `setState` synchronously in an effect body. Derive the value during render, move the update to the event handler, or use the existing hooks (`useLastDefined`, `useMemoWithPrevious`, `useQueryManager`…). - Where an effect needs the latest props or state without re-running, wrap that logic in `useEffectEvent` and call it from the effect, or from a callback the effect registers (as the Explore chart modules do for ECharts handlers). - Fix `react-hooks/exhaustive-deps` warnings properly instead of suppressing them. Upstream has zero suppressions, so keep it that way. - **Blueprint 6:** - Class names are `bp6-*`. In SCSS use `#{$bp-ns}`, never a hard-coded `bp5`. - Replace `Popover` with `PopoverNext`, keeping the console defaults: `lazy` and `shouldReturnFocusOnClose={false}`. `position` becomes `placement`, and `minimal` becomes `animation="minimal" arrow={false}`. Copy the props from an existing upstream usage. - Button, tag and input props: - `minimal` → `variant="minimal"` - `small` / `large` → `size="small"` / `"large"` - `rightIcon` → `endIcon` - `Alignment.LEFT` / `RIGHT` → `START` / `END`. - `DateRangePicker3` from `@blueprintjs/datetime2` → `DateRangePicker` from `@blueprintjs/datetime`. - Use `event.key` instead of `Keys` / `keyCode`. - `@typescript-eslint/no-deprecated` is an error, so the linter will find the leftovers. - Toasts still go through `AppToaster.show(...)` synchronously. - Don't restyle Blueprint components locally to make them "look right". The Blueprint 5 look is restored globally in `src/blueprint-overrides/_blueprint5-look.scss`, so fix a mismatch there, and only if it's a genuine gap that affects upstream components too. - **Sass:** - `@import` is gone. Start each component stylesheet with `@use '<relative path>/variables' as *;` (see any upstream component `.scss`), and add `@use 'sass:math';` etc. for built-in modules (`math.div` instead of `/`). - Blueprint variables and mixins that used to leak in as globals come from the `variables` module (it forwards the colors, color aliases and Blueprint variable overrides). If one is missing, add it to `src/blueprint-overrides/common/` rather than redefining it locally. - The build must produce no Sass warnings. - **Ace:** - The editor is ace-builds 1.44 with react-ace 15. Remove `editorProps={{ $blockScrolling: … }}` and the `any` casts around Ace types. - Custom completers must be created once and read the current props through a ref or `useEffectEvent`, because Ace keeps only the first completers it is given. Reuse the upstream SQL completer setup instead of writing a parallel one. - Custom modes follow the class-based pattern in `src/ace-modes/`. - **Other upgraded libraries:** - `uuid` 14 is ESM only; import `{ v4 as uuidv4 }`. - `zustand` 5 uses named imports (upstream uses `createStore` plus `useStore(store, selector)`), and has no default export. - `memoize-one` 6 is a default import. - Clipboard writes go through the upstream `copyToClipboard` helper, not `copy-to-clipboard` directly. - `date-fns` is no longer a dependency. Use the existing `src/utils/date` helpers or `instanceof Date`. - For diffs, use `react-diff-viewer-continued` (keep it on `~4.0.6`). - Import `saveAs` from `file-saver` as a named import. - **Tests:** - Shallow rendering is gone. Specs use `render` from `@testing-library/react` and snapshot `container.firstChild`. Mock heavy children with `jest.mock(path, () => jest.requireActual('<rel>/test-utils/stub-component').consoleTableStub)` or `stubComponent('Name')`. - The global DOM snapshot serializer reduces icons to their names and collapses padding rows. - Ace modes are registered in `src/setup-tests.ts`. - Tests that depend on time must fake the clock, and tests that trigger network requests must mock them, so full renders stay quiet and deterministic. - Regenerate the fork's snapshots with `npm run update-snapshots`, then read the snapshot diffs to confirm only the expected structure changed. - In e2e (Playwright), use `fill` instead of `type` and the new `bp6-`/`ct-` selectors. - **TypeScript 6 and formatting:** - `moduleResolution: bundler` needs explicit file extensions only for non-TS assets; side-effect style imports are type-checked through the existing module declarations. - `target` is es2022, so class fields use define semantics. A subclass that re-declares a field set by its parent constructor must use `declare`. - Run `npm run autofix` for Prettier 3 formatting instead of fixing the format by hand. **Verification (do all of it, report honestly)** 1. `npm test` passes in `web-console/`, which runs the typecheck, ESLint (0 warnings), Prettier, stylelint and all Jest suites. `npm run build` (or `script/build`) must also compile with no warnings, and `npm run check-licenses` must pass. For any new dependency, update `licenses.yaml` at the repo root. 2. Start the dev server on a free port (`npx webpack serve --port <port> --no-open`) against a Druid cluster. Don't touch dev servers other people started, and only kill processes you started. 3. For every fork-owned view, dialog and component, compare the migrated build against the fork's pre-migration build, side by side. Use Playwright to check computed styles and take screenshots, and exercise the interactions by hand: tables (sort, filter, pivot, paginate, resize, expand), popovers and menus (open, close, focus), keyboard handling, editors and completions, and URL deep links, including params with encoded `&`, `%` and `/`. 4. Report any visual or behavioral difference you could not remove, along with its cause. Don't commit or push unless asked. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
