potiuk opened a new issue, #43167:
URL: https://github.com/apache/airflow/issues/43167
Currently when NPM packages are installed on CI they are installed from the
scratch by "compile www assets" pre-commit.
This causes fairly frequent, intermittent issues when there is a networking
issue or NPM server issue - for example:
```
Compile www assets
(manual)..............................................Failed
- hook id: compile-www-assets
- exit code: 1
yarn install v1.22.21
(node:490) [DEP0040] DeprecationWarning: The `punycode` module is
deprecated. Please use a userland alternative instead.
(Use `node --trace-deprecation ...` to show where the warning was created)
[1/4] Resolving packages...
[2/4] Fetching packages...
[] 0/1573[] 5/1573[] 11/1573[] 18/1573[] 26/1573[] 34/1573[] 41/1573[]
45/1573[] 53/1573[] 61/1573[] 68/1573[] 75/1573[] 84/1573[] 90/1573[] 97/1573[]
104/1573[] 111/1573[] 116/1573[] 122/1573[] 128/1573[] 135/1573[] 140/1573[]
147/1573[] 153/1573[] 160/1573[] 167/1573[] 176/1573[] 185/1573[] 192/1573[]
197/1573[] 198/1573[] 202/1573[] 206/1573[] 211/1573[] 216/1573[] 223/1573[]
229/1573[] 232/1573[] 234/1573[] 238/1573[] 241/1573[] 243/1573[] 246/1573[]
248/1573[] 250/1573[] 255/1573[] 260/1573[] 265/1573[] 269/1573[] 276/1573[]
284/1573[] 286/1573[] 287/1573[] 290/1573[] 292/1573[] 295/1573[] 298/1573[]
305/1573[] 311/1573[] 316/1573[] 325/1573[] 331/1573[] 336/1573[] 342/1573[]
346/1573[] 349/1573[] 353/1573[] 359/1573[] 364/1573[] 368/1573[] 372/1573[]
378/1573[] 385/1573[] 391/1573[] 396/1573[] 402/1573[] 409/1573[] 414/1573[]
420/1573[] 427/1573[] 432/1573[] 437/1573[] 441/1573[] 443/1573[] 444/1573[]
445/1573[] 448/1573[] 451/1573[] 454/1573[] 457/1573[] 462/1573[] 467/15
73[] 470/1573[] 473/1573[] 477/1573[] 481/1573[] 485/1573[] 490/1573[]
496/1573[] 503/1573[] 512/1573[] 519/1573[] 524/1573[] 526/1573[] 529/1573[]
531/1573[] 535/1573[] 539/1573[] 545/1573[] 548/1573[] 550/1573[] 553/1573[]
560/1573[] 567/1573[] 574/1573[] 580/1573[] 591/1573[] 599/1573[] 607/1573[]
616/1573[] 620/1573[] 624/1573[] 627/1573[] 630/1573[] 634/1573[] 641/1573[]
650/1573[] 658/1573[] 668/1573[] 676/1573[] 684/1573[] 692/1573error Error:
https://registry.yarnpkg.com/@babel/plugin-syntax-logical-assignment-operators/-/plugin-syntax-logical-assignment-operators-7.10.4.tgz:
Request failed "500 Internal Server Error"
at ResponseError.ExtendableBuiltin
(/opt/airflow/files/home/.cache/pre-commit/repocv9ljx22/node_env-22.2.0/lib/node_modules/yarn/lib/cli.js:696:66)
at new ResponseError
(/opt/airflow/files/home/.cache/pre-commit/repocv9ljx22/node_env-22.2.0/lib/node_modules/yarn/lib/cli.js:802:124)
at Request.<anonymous>
(/opt/airflow/files/home/.cache/pre-commit/repocv9ljx22/node_env-22.2.0/lib/node_modules/yarn/lib/cli.js:66218:16)
at Request.emit (node:events:520:28)
at module.exports.Request.onRequestResponse
(/opt/airflow/files/home/.cache/pre-commit/repocv9ljx22/node_env-22.2.0/lib/node_modules/yarn/lib/cli.js:141751:10)
at ClientRequest.emit (node:events:520:28)
at HTTPParser.parserOnIncomingClient (node:_http_client:700:27)
at HTTPParser.parserOnHeadersComplete (node:_http_common:119:17)
at TLSSocket.socketOnData (node:_http_client:542:22)
at TLSSocket.emit (node:events:520:28)
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about
this command.
Traceback (most recent call last):
File "/opt/airflow/./scripts/ci/pre_commit/compile_www_assets.py", line
71, in <module>
subprocess.check_call(["yarn", "install", "--frozen-lockfile"],
cwd=os.fspath(www_directory))
File "/usr/local/lib/python3.9/subprocess.py", line 373, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['yarn', 'install',
'--frozen-lockfile']' returned non-zero exit status 1.
Traceback (most recent call last):
File "/usr/local/lib/python3.9/runpy.py", line 197, in
_run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/local/lib/python3.9/runpy.py", line 87, in _run_code
exec(code, run_globals)
File
"/opt/airflow/files/home/.local/share/hatch/env/virtual/apache-airflow-build/lib/python3.9/site-packages/hatchling/__main__.py",
line 6, in <module>
sys.exit(hatchling())
File
"/opt/airflow/files/home/.local/share/hatch/env/virtual/apache-airflow-build/lib/python3.9/site-packages/hatchling/cli/__init__.py",
line 26, in hatchling
command(**kwargs)
File
"/opt/airflow/files/home/.local/share/hatch/env/virtual/apache-airflow-build/lib/python3.9/site-packages/hatchling/cli/build/__init__.py",
line 82, in build_impl
for artifact in builder.build(
File
"/opt/airflow/files/home/.local/share/hatch/env/virtual/apache-airflow-build/lib/python3.9/site-packages/hatchling/builders/plugin/interface.py",
line 155, in build
artifact = version_api[version](directory, **build_data)
File "/opt/airflow/hatch_build.py", line 613, in build_standard
run(cmd, cwd=work_dir.as_posix(), check=True, shell=True)
File "/usr/local/lib/python3.9/subprocess.py", line 528, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['pre-commit run --hook-stage
manual compile-www-assets --all-files']' returned non-zero exit status 1.
Error building Airflow packages
Error preparing Airflow package
```
We should - similarly to Python packages, implement caching strategy that
should speed up the installation of npm packages on a clean CI runner, as well
as local development instance - where previously installed packages could be
stored in the cache and reused. This should increase both - speed of CI jobs an
stability of them (and speed of local installation of npm packages).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
