[
https://issues.apache.org/jira/browse/HADOOP-19614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18008175#comment-18008175
]
ASF GitHub Bot commented on HADOOP-19614:
-----------------------------------------
Copilot commented on code in PR #7795:
URL: https://github.com/apache/hadoop/pull/7795#discussion_r2217013279
##########
.github/workflows/build-and-tag-hadoop-image.yaml:
##########
@@ -0,0 +1,138 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+name: build-and-tag-hadoop-image
+
+# This workflow builds the Hadoop docker image.
+# For non-PR runs, it also pushes the image to the registry, tagging it based
on the branch name.
+
+on:
+ pull_request:
+ types: [opened, synchronize]
+ branches:
+ - 'docker-hadoop-**'
+ - '!docker-hadoop-runner-**'
+ push:
+ branches:
+ - 'docker-hadoop-**'
+ - '!docker-hadoop-runner-**'
+
+permissions:
+ contents: read
+ packages: write
+
+jobs:
+ build:
+ runs-on: ubuntu-latest
+ steps:
+ - name: Generate image ID
+ id: meta
+ uses: docker/metadata-action@8e5442c4ef9f78752691e2d8f8d19755c6f78e81
+ with:
+ images: |
+ ghcr.io/${{ github.repository_owner }}/hadoop
+ tags: |
+ type=match,pattern=docker-hadoop-(.*),value={{branch}},group=1
+ flavor: |
+ latest=false
+
+ - name: Check if image exists
+ id: pull
+ run: |
+ success=false
+ if docker pull "$DOCKER_METADATA_OUTPUT_TAGS"; then
+ success=true
+ fi
+
+ echo "success=$success" >> $GITHUB_OUTPUT
+
+ - name: Set up QEMU
+ if: ${{ steps.pull.outputs.success == 'false' }}
+ uses: docker/setup-qemu-action@49b3bc8e6bdd4a60e6116a5414239cba5943d3cf
+
+ - name: Set up Docker Buildx
+ if: ${{ steps.pull.outputs.success == 'false' }}
+ uses:
docker/setup-buildx-action@c47758b77c9736f4b2ef4073d4d51994fabfe349
+
+ - name: Login to GitHub Container Registry
+ id: login
+ if: ${{ github.event_name != 'pull_request' &&
steps.pull.outputs.success == 'false' }}
+ uses: docker/login-action@9780b0c442fbb1117ed29e0efdff1e18412f7567
+ with:
+ registry: ghcr.io
+ username: ${{ github.repository_owner }}
+ password: ${{ secrets.GITHUB_TOKEN }}
+
+ - name: Build and push image
+ id: build
+ if: ${{ steps.pull.outputs.success == 'false' }}
+ uses: docker/build-push-action@4f58ea79222b3b9dc2c8bbdd6debcef730109a75
+ with:
+ platforms: linux/amd64,linux/arm64
+ push: ${{ github.event_name != 'pull_request' }}
+ tags: ${{ steps.meta.outputs.tags }}
+ labels: ${{ steps.meta.outputs.labels }}
+
+ tag:
+ needs: build
+ if: ${{ github.ref_type == 'tag' }}
+ runs-on: ubuntu-latest
+ env:
+ DOCKERHUB_USER: ${{ secrets.DOCKERHUB_USER }}
+ IMAGE_ID: ${{ needs.build.outputs.image-id }}
+ REGISTRIES: ghcr.io # docker.io is appended dynamically
+ steps:
+ - name: Generate tags
+ uses: docker/metadata-action@8e5442c4ef9f78752691e2d8f8d19755c6f78e81
+ with:
+ images: |
+ ${{ github.repository_owner }}/hadoop
Review Comment:
This image reference is missing the registry prefix. For DockerHub, it
should be `docker.io/${{ github.repository_owner }}/hadoop` or just
`apache/hadoop` if targeting the official apache organization.
```suggestion
docker.io/${{ github.repository_owner }}/hadoop
```
##########
.github/workflows/build-and-tag-hadoop-image.yaml:
##########
@@ -0,0 +1,138 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+name: build-and-tag-hadoop-image
+
+# This workflow builds the Hadoop docker image.
+# For non-PR runs, it also pushes the image to the registry, tagging it based
on the branch name.
+
+on:
+ pull_request:
+ types: [opened, synchronize]
+ branches:
+ - 'docker-hadoop-**'
+ - '!docker-hadoop-runner-**'
+ push:
+ branches:
+ - 'docker-hadoop-**'
+ - '!docker-hadoop-runner-**'
+
+permissions:
+ contents: read
+ packages: write
+
+jobs:
+ build:
+ runs-on: ubuntu-latest
+ steps:
+ - name: Generate image ID
+ id: meta
+ uses: docker/metadata-action@8e5442c4ef9f78752691e2d8f8d19755c6f78e81
+ with:
+ images: |
+ ghcr.io/${{ github.repository_owner }}/hadoop
+ tags: |
+ type=match,pattern=docker-hadoop-(.*),value={{branch}},group=1
+ flavor: |
+ latest=false
+
+ - name: Check if image exists
+ id: pull
+ run: |
+ success=false
+ if docker pull "$DOCKER_METADATA_OUTPUT_TAGS"; then
+ success=true
+ fi
+
+ echo "success=$success" >> $GITHUB_OUTPUT
+
+ - name: Set up QEMU
+ if: ${{ steps.pull.outputs.success == 'false' }}
+ uses: docker/setup-qemu-action@49b3bc8e6bdd4a60e6116a5414239cba5943d3cf
+
+ - name: Set up Docker Buildx
+ if: ${{ steps.pull.outputs.success == 'false' }}
+ uses:
docker/setup-buildx-action@c47758b77c9736f4b2ef4073d4d51994fabfe349
+
+ - name: Login to GitHub Container Registry
+ id: login
+ if: ${{ github.event_name != 'pull_request' &&
steps.pull.outputs.success == 'false' }}
+ uses: docker/login-action@9780b0c442fbb1117ed29e0efdff1e18412f7567
+ with:
+ registry: ghcr.io
+ username: ${{ github.repository_owner }}
+ password: ${{ secrets.GITHUB_TOKEN }}
+
+ - name: Build and push image
+ id: build
+ if: ${{ steps.pull.outputs.success == 'false' }}
+ uses: docker/build-push-action@4f58ea79222b3b9dc2c8bbdd6debcef730109a75
+ with:
+ platforms: linux/amd64,linux/arm64
+ push: ${{ github.event_name != 'pull_request' }}
+ tags: ${{ steps.meta.outputs.tags }}
+ labels: ${{ steps.meta.outputs.labels }}
+
+ tag:
+ needs: build
+ if: ${{ github.ref_type == 'tag' }}
+ runs-on: ubuntu-latest
+ env:
+ DOCKERHUB_USER: ${{ secrets.DOCKERHUB_USER }}
+ IMAGE_ID: ${{ needs.build.outputs.image-id }}
Review Comment:
The build job does not define any outputs, but the tag job is trying to
reference `needs.build.outputs.image-id`. This will result in an empty IMAGE_ID
variable and cause the tag job to fail.
##########
.github/workflows/build-and-tag-hadoop-image.yaml:
##########
@@ -0,0 +1,138 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+name: build-and-tag-hadoop-image
+
+# This workflow builds the Hadoop docker image.
+# For non-PR runs, it also pushes the image to the registry, tagging it based
on the branch name.
+
+on:
+ pull_request:
+ types: [opened, synchronize]
+ branches:
+ - 'docker-hadoop-**'
+ - '!docker-hadoop-runner-**'
+ push:
+ branches:
+ - 'docker-hadoop-**'
+ - '!docker-hadoop-runner-**'
+
+permissions:
+ contents: read
+ packages: write
+
+jobs:
+ build:
+ runs-on: ubuntu-latest
+ steps:
+ - name: Generate image ID
+ id: meta
+ uses: docker/metadata-action@8e5442c4ef9f78752691e2d8f8d19755c6f78e81
+ with:
+ images: |
+ ghcr.io/${{ github.repository_owner }}/hadoop
+ tags: |
+ type=match,pattern=docker-hadoop-(.*),value={{branch}},group=1
+ flavor: |
+ latest=false
+
+ - name: Check if image exists
+ id: pull
+ run: |
+ success=false
+ if docker pull "$DOCKER_METADATA_OUTPUT_TAGS"; then
+ success=true
+ fi
+
+ echo "success=$success" >> $GITHUB_OUTPUT
+
+ - name: Set up QEMU
+ if: ${{ steps.pull.outputs.success == 'false' }}
+ uses: docker/setup-qemu-action@49b3bc8e6bdd4a60e6116a5414239cba5943d3cf
+
+ - name: Set up Docker Buildx
+ if: ${{ steps.pull.outputs.success == 'false' }}
+ uses:
docker/setup-buildx-action@c47758b77c9736f4b2ef4073d4d51994fabfe349
+
+ - name: Login to GitHub Container Registry
+ id: login
+ if: ${{ github.event_name != 'pull_request' &&
steps.pull.outputs.success == 'false' }}
+ uses: docker/login-action@9780b0c442fbb1117ed29e0efdff1e18412f7567
+ with:
+ registry: ghcr.io
+ username: ${{ github.repository_owner }}
+ password: ${{ secrets.GITHUB_TOKEN }}
+
+ - name: Build and push image
+ id: build
+ if: ${{ steps.pull.outputs.success == 'false' }}
+ uses: docker/build-push-action@4f58ea79222b3b9dc2c8bbdd6debcef730109a75
+ with:
+ platforms: linux/amd64,linux/arm64
+ push: ${{ github.event_name != 'pull_request' }}
+ tags: ${{ steps.meta.outputs.tags }}
+ labels: ${{ steps.meta.outputs.labels }}
+
+ tag:
+ needs: build
+ if: ${{ github.ref_type == 'tag' }}
+ runs-on: ubuntu-latest
+ env:
+ DOCKERHUB_USER: ${{ secrets.DOCKERHUB_USER }}
+ IMAGE_ID: ${{ needs.build.outputs.image-id }}
+ REGISTRIES: ghcr.io # docker.io is appended dynamically
+ steps:
+ - name: Generate tags
+ uses: docker/metadata-action@8e5442c4ef9f78752691e2d8f8d19755c6f78e81
+ with:
+ images: |
+ ${{ github.repository_owner }}/hadoop
+ tags: |
+ type=match,pattern=docker-image-(.*),value={{ ref_name }},group=1
+ flavor: |
+ latest=false
+
+ - name: Add Docker Hub to targets
+ if: ${{ env.DOCKERHUB_USER }}
+ run: |
+ echo "REGISTRIES=${{ env.REGISTRIES }} docker.io" >> $GITHUB_ENV
+
+ - name: Pull image
+ run: |
+ docker pull "$IMAGE_ID"
+
+ - name: Login to GitHub Container Registry
+ uses: docker/login-action@9780b0c442fbb1117ed29e0efdff1e18412f7567
+ with:
+ registry: ghcr.io
+ username: ${{ github.repository_owner }}
+ password: ${{ secrets.GITHUB_TOKEN }}
+
+ - name: Login to Docker Hub
+ if: ${{ env.DOCKERHUB_USER }}
+ uses: docker/login-action@9780b0c442fbb1117ed29e0efdff1e18412f7567
+ with:
+ username: ${{ env.DOCKERHUB_USER }}
+ password: ${{ secrets.DOCKERHUB_TOKEN }}
+
+ - name: Apply tags to existing image
+ run: |
+ set -x
+ for registry in $REGISTRIES; do
+ opts="$(echo "$DOCKER_METADATA_OUTPUT_TAGS" | sed "s@^@--tag
$registry/@g" | xargs echo)"
Review Comment:
The sed command will incorrectly prepend the registry to already-qualified
tag names, potentially creating malformed tags like
`docker.io/apache/hadoop:3.4.1` becoming `--tag
docker.io/docker.io/apache/hadoop:3.4.1`.
```suggestion
opts="$(echo "$DOCKER_METADATA_OUTPUT_TAGS" | sed -E
"s@^[^/]+:[^:]+$@--tag $registry/&@g" | xargs echo)"
```
##########
.github/workflows/build-and-tag-hadoop-image.yaml:
##########
@@ -0,0 +1,138 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+name: build-and-tag-hadoop-image
+
+# This workflow builds the Hadoop docker image.
+# For non-PR runs, it also pushes the image to the registry, tagging it based
on the branch name.
+
+on:
+ pull_request:
+ types: [opened, synchronize]
+ branches:
+ - 'docker-hadoop-**'
+ - '!docker-hadoop-runner-**'
+ push:
+ branches:
+ - 'docker-hadoop-**'
+ - '!docker-hadoop-runner-**'
+
+permissions:
+ contents: read
+ packages: write
+
+jobs:
+ build:
+ runs-on: ubuntu-latest
+ steps:
+ - name: Generate image ID
+ id: meta
+ uses: docker/metadata-action@8e5442c4ef9f78752691e2d8f8d19755c6f78e81
+ with:
+ images: |
+ ghcr.io/${{ github.repository_owner }}/hadoop
+ tags: |
+ type=match,pattern=docker-hadoop-(.*),value={{branch}},group=1
+ flavor: |
+ latest=false
+
+ - name: Check if image exists
+ id: pull
+ run: |
+ success=false
+ if docker pull "$DOCKER_METADATA_OUTPUT_TAGS"; then
+ success=true
+ fi
+
+ echo "success=$success" >> $GITHUB_OUTPUT
+
+ - name: Set up QEMU
+ if: ${{ steps.pull.outputs.success == 'false' }}
+ uses: docker/setup-qemu-action@49b3bc8e6bdd4a60e6116a5414239cba5943d3cf
+
+ - name: Set up Docker Buildx
+ if: ${{ steps.pull.outputs.success == 'false' }}
+ uses:
docker/setup-buildx-action@c47758b77c9736f4b2ef4073d4d51994fabfe349
+
+ - name: Login to GitHub Container Registry
+ id: login
+ if: ${{ github.event_name != 'pull_request' &&
steps.pull.outputs.success == 'false' }}
+ uses: docker/login-action@9780b0c442fbb1117ed29e0efdff1e18412f7567
+ with:
+ registry: ghcr.io
+ username: ${{ github.repository_owner }}
+ password: ${{ secrets.GITHUB_TOKEN }}
+
+ - name: Build and push image
+ id: build
+ if: ${{ steps.pull.outputs.success == 'false' }}
+ uses: docker/build-push-action@4f58ea79222b3b9dc2c8bbdd6debcef730109a75
+ with:
+ platforms: linux/amd64,linux/arm64
+ push: ${{ github.event_name != 'pull_request' }}
+ tags: ${{ steps.meta.outputs.tags }}
+ labels: ${{ steps.meta.outputs.labels }}
+
+ tag:
+ needs: build
+ if: ${{ github.ref_type == 'tag' }}
+ runs-on: ubuntu-latest
+ env:
+ DOCKERHUB_USER: ${{ secrets.DOCKERHUB_USER }}
+ IMAGE_ID: ${{ needs.build.outputs.image-id }}
+ REGISTRIES: ghcr.io # docker.io is appended dynamically
+ steps:
+ - name: Generate tags
+ uses: docker/metadata-action@8e5442c4ef9f78752691e2d8f8d19755c6f78e81
+ with:
+ images: |
+ ${{ github.repository_owner }}/hadoop
+ tags: |
+ type=match,pattern=docker-image-(.*),value={{ ref_name }},group=1
Review Comment:
The syntax `{{ ref_name }}` is incorrect for GitHub Actions. It should be
`${{ github.ref_name }}` to properly reference the tag name.
```suggestion
type=match,pattern=docker-image-(.*),value=${{ github.ref_name
}},group=1
```
> Publish Hadoop Docker Image to DockerHub for amd64 and arm64
> -------------------------------------------------------------
>
> Key: HADOOP-19614
> URL: https://issues.apache.org/jira/browse/HADOOP-19614
> Project: Hadoop Common
> Issue Type: New Feature
> Reporter: Abhishek Kumar
> Assignee: Abhishek Kumar
> Priority: Major
> Labels: pull-request-available
>
> Update GitHub Actions workflow to publish the apache/hadoop Docker image to
> DockerHub for multiple arch's: amd64 and arm64.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
