Croway commented on code in PR #30191: URL: https://github.com/apache/camel/pull/30191#discussion_r4586831279
########## test-infra/camel-test-infra-google-bigquery/src/main/java/org/apache/camel/test/infra/google/bigquery/services/GoogleBigQueryLocalContainerInfraService.java: ########## @@ -0,0 +1,105 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.camel.test.infra.google.bigquery.services; + +import org.apache.camel.spi.annotations.InfraService; +import org.apache.camel.test.infra.common.LocalPropertyResolver; +import org.apache.camel.test.infra.common.services.ContainerEnvironmentUtil; +import org.apache.camel.test.infra.common.services.ContainerService; +import org.apache.camel.test.infra.google.bigquery.common.GoogleBigQueryProperties; +import org.apache.camel.test.infra.google.common.GoogleProperties; +import org.apache.camel.test.infra.google.common.services.FlociGcpContainer; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +/** + * A local BigQuery service on Floci GCP, which runs the queries on floci-duck, a SQL engine that it starts as a + * container of its own through the Docker socket. Floci GCP accepts any credentials, and gives an access token to a + * service account of any key at {@code <endpoint>/token}. + */ +@InfraService(service = GoogleBigQueryInfraService.class, + description = "Local Google BigQuery service with Floci GCP", + serviceAlias = "google", serviceImplementationAlias = "bigquery") +public class GoogleBigQueryLocalContainerInfraService + implements GoogleBigQueryInfraService, ContainerService<FlociGcpContainer> { + + private static final Logger LOG = LoggerFactory.getLogger(GoogleBigQueryLocalContainerInfraService.class); + private static final String DEFAULT_PROJECT_ID = "test-project"; + + private final FlociGcpContainer container; + + public GoogleBigQueryLocalContainerInfraService() { + this(LocalPropertyResolver.getProperty(FlociGcpContainer.class, GoogleProperties.GOOGLE_CONTAINER)); + } + + public GoogleBigQueryLocalContainerInfraService(String imageName) { + container = initContainer(imageName); + String name = ContainerEnvironmentUtil.containerName(this.getClass()); + if (name != null) { + container.withCreateContainerCmdModifier(cmd -> cmd.withName(name)); + } + } + + public GoogleBigQueryLocalContainerInfraService(FlociGcpContainer container) { + this.container = container; + } + + protected FlociGcpContainer initContainer(String imageName) { + return new FlociGcpContainer(imageName, ContainerEnvironmentUtil.isFixedPort(this.getClass())) + .withDockerSocket() + .withBigQueryEngine(); Review Comment: `withBigQueryEngine()` resolves the floci-duck image straight away (`RemoteDockerImage.get()`), and here it runs in the constructor. The ITs build the service in a static initializer (`BigQueryITSupport.service` → `GoogleBigQueryServiceFactory.SingletonServiceHolder`), so this talks to Docker before JUnit evaluates the `ContainerTestService` condition, which is what skips container-based tests when Docker is not available. Reproduced with `mvn verify -pl test-infra/camel-test-infra-google-common,test-infra/camel-test-infra-google-bigquery,components/camel-google/camel-google-bigquery`, run in a container with no Docker socket: - 49c9697e28: `Tests run: 7, Failures: 0, Errors: 7`, BUILD FAILURE. `AsyncIT` fails with `ExceptionInInitializerError` caused by `IllegalStateException: Could not find a valid Docker environment` (at `FlociGcpContainer.withBigQueryEngine` ← `GoogleBigQueryLocalContainerInfraService.initContainer` ← `BigQueryITSupport.<clinit>`). The other six fail with `NoClassDefFoundError: Could not initialize class BigQueryITSupport`, including `InsertIdIT`, which is disabled in local mode. - Base 21b8ed8e9613: 9 skipped, BUILD SUCCESS. Resolving the image eagerly is fine, only the call site needs to move. The Floci AZ services call `withBrokerImage()` in `initialize()`. The same here: ```suggestion .withDockerSocket(); ``` and in `initialize()`: ```java container.withBigQueryEngine(); container.start(); ``` With that change, without Docker: 9 skipped, BUILD SUCCESS. With Docker: 9 run, 2 skipped (`InsertIdIT`), all green. ########## test-infra/camel-test-infra-google-common/src/main/java/org/apache/camel/test/infra/google/common/services/FlociGcpContainer.java: ########## @@ -47,4 +76,86 @@ public FlociGcpContainer(String imageName, boolean fixedPort) { public String serviceAddress() { return String.format("http://%s:%d", getHost(), getMappedPort(SERVICE_PORT)); } + + /** + * Lets Floci GCP start containers of its own, such as the SQL engine of BigQuery: mounts the Docker socket of the + * host, and gives the containers that Floci GCP starts a resource namespace of this container, by which + * {@link #stop()} removes them. Each container has a namespace of its own: when it starts, Floci GCP removes the + * containers of its namespace that it finds, as left over. + */ + public FlociGcpContainer withDockerSocket() { + resourceNamespace = "camel" + UUID.randomUUID().toString().replace("-", "").substring(0, 12); + withEnv(RESOURCE_NAMESPACE_ENV, resourceNamespace); + return withCreateContainerCmdModifier(FlociGcpContainer::bindDockerSocket); + } + + /** + * Has Floci GCP run the queries of BigQuery on the floci-duck image of + * {@link GoogleProperties#BIGQUERY_ENGINE_CONTAINER}, which this method pulls as Testcontainers pulls its own + * images, so that image substitution and registry authentication apply: Floci GCP would pull it itself otherwise. + * Needs {@link #withDockerSocket()}. + */ + public FlociGcpContainer withBigQueryEngine() { + String image = LocalPropertyResolver.getProperty(FlociGcpContainer.class, + GoogleProperties.BIGQUERY_ENGINE_CONTAINER); + return withEnv(BIGQUERY_ENGINE_IMAGE_ENV, new RemoteDockerImage(DockerImageName.parse(image)).get()); + } + + @Override + public void start() { + if (resourceNamespace != null) { + // Testcontainers does not know the containers that Floci GCP starts: have them removed when the JVM exits, + // should this container not be stopped + ResourceReaper.instance().registerLabelsFilterForCleanup(Map.of(RESOURCE_NAMESPACE_LABEL, resourceNamespace)); + } + super.start(); + } + + @Override + public void stop() { + try { + super.stop(); + } finally { + if (resourceNamespace != null) { + removeStartedContainers(); + } + } + } + + // the containers that Floci GCP started are not removed with it + private void removeStartedContainers() { + DockerClient client = DockerClientFactory.instance().client(); + List<Container> containers = client.listContainersCmd() + .withShowAll(true) + .withLabelFilter(Map.of(RESOURCE_NAMESPACE_LABEL, resourceNamespace)) + .exec(); + for (Container container : containers) { + LOG.debug("Removing the container {} that Floci GCP started", Arrays.toString(container.getNames())); + try { + client.removeContainerCmd(container.getId()).withForce(true).withRemoveVolumes(true).exec(); Review Comment: [Low] `SingletonService.addToStore` registers a JVM shutdown hook for each test class that uses the service (7 here), and the JVM runs the hooks concurrently. Several threads then run `stop()` → `removeStartedContainers()` at the same time and list the same floci-duck container. Docker answers the later `removeContainerCmd` calls with 409. `ConflictException` is not caught here, so it ends the hook thread with an uncaught stack trace. Reproduced on 49c9697e28 with a small program that starts `GoogleBigQueryLocalContainerInfraService`, runs `SELECT 1` (so that Floci GCP starts floci-duck) and registers 7 hooks calling `shutdown()`. In 3 of 3 runs, 5 or 6 of the 7 hooks died with: ``` Exception in thread "hook-0" com.github.dockerjava.api.exception.ConflictException: Status 409: {"message":"removal of container 6a071232906d... is already in progress"} at org.apache.camel.test.infra.google.common.services.FlociGcpContainer.removeStartedContainers(FlociGcpContainer.java:135) at org.apache.camel.test.infra.google.common.services.FlociGcpContainer.stop(FlociGcpContainer.java:120) ``` With a single hook there is no exception. The container was removed in every run, so this is noise rather than a leak, and it did not show up in the failsafe output of my `mvn verify` run. Catching `ConflictException` next to `NotFoundException` is enough: Testcontainers' own `ResourceReaper.removeContainer` ignores any exception at this step. `FlociAzContainer` on main has the same code. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
