This is an automated email from the ASF dual-hosted git repository.
wusheng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/skywalking.git
The following commit(s) were added to refs/heads/master by this push:
new 6248d1b725 Collect BanyanDB data and access logs on e2e failure, bump
BanyanDB, update CLAUDE.md (#13765)
6248d1b725 is described below
commit 6248d1b7259dd24cc8218a8513d14cf018c519f0
Author: 吴晟 Wu Sheng <[email protected]>
AuthorDate: Sun Mar 29 14:40:37 2026 +0800
Collect BanyanDB data and access logs on e2e failure, bump BanyanDB, update
CLAUDE.md (#13765)
**Problem:** When BanyanDB-related e2e tests fail, only container logs are
uploaded. The actual
data files and access logs are lost with container cleanup.
**Changes:**
**1. Collect BanyanDB data and access logs on failure**
- Update `skywalking-infra-e2e` to `e26033e` (collect-on-failure with glob
pattern support)
- Add `cleanup.collect` to all 18 BanyanDB e2e YAMLs
- Collected directories: `/tmp/trace/`, `/tmp/stream/`, `/tmp/measure/`,
`/tmp/property/`, `/tmp/schema-property/`, `/tmp/accesslog/`
**2. Enable BanyanDB ingestion and query access logs**
- Compose: `--enable-ingestion-access-log --enable-query-access-log
--access-log-root-path /tmp`
- Kind: `BYDB_ENABLE_INGESTION_ACCESS_LOG`, `BYDB_ENABLE_QUERY_ACCESS_LOG`,
`BYDB_ACCESS_LOG_ROOT_PATH` via Helm `--set` with escaped quotes (`\"true\"`)
to force string type
- BanyanDB auto-creates `/tmp/accesslog/` subdirectory
- 8 access log files per node:
`{measure,stream,trace,property}-{ingest,query}-<timestamp>`
**3. Remove custom root-path overrides**
- Removed `--stream-root-path /tmp/stream-data --measure-root-path
/tmp/measure-data` from base compose
- All modules use default `/tmp` root path consistently
- Updated stages data-generate workflow and volume mounts
**4. Bump BanyanDB to `d0f41f9a`**
- Adds auto-creation of accesslog directory (no mkdir workaround needed)
**5. Update CLAUDE.md**
- Added "Analysis and Design Principles" section: never guess, read source
code, verify locally, ask developer
- Docker image registry guidance (ghcr.io vs Docker Hub)
**Verified results:**
Docker Compose (`test-logs-BanyanDB`):
```
banyandb-data/banyandb/tmp/
trace: 2.8M (818 files) ✅
stream: 404K (113 files) ✅
measure: 476K (109 files) ✅
property: 1.1M (6 files) ✅
schema-property: 14M (8 files) ✅
accesslog: 2.8M (8 files) ✅
```
Kind / K8s (`test-logs-eBPF Profiling On CPU BanyanDB`):
```
banyandb-data/istio-system/skywalking-banyandb-0/tmp/
trace: 0B (2 files) ✅
stream: 252K (75 files) ✅
measure: 20K (8 files) ✅
property: 1.1M (8 files) ✅
schema-property: 14M (8 files) ✅
accesslog: 1.1M (8 files) ✅
```
---
.github/workflows/skywalking.yaml | 14 +++++-----
CLAUDE.md | 26 ++++++++++++++++++
test/e2e-v2/cases/alarm/banyandb/e2e.yaml | 15 ++++++++++
test/e2e-v2/cases/banyandb/e2e.yaml | 15 ++++++++++
test/e2e-v2/cases/baseline/banyandb/e2e.yaml | 15 ++++++++++
test/e2e-v2/cases/cluster/zk/banyandb/e2e.yaml | 15 ++++++++++
test/e2e-v2/cases/event/banyandb/e2e.yaml | 15 ++++++++++
test/e2e-v2/cases/log/banyandb/e2e.yaml | 16 ++++++++++-
test/e2e-v2/cases/menu/banyandb/e2e.yaml | 16 ++++++++++-
.../profiling/async-profiler/banyandb/e2e.yaml | 15 ++++++++++
.../profiling/ebpf/access_log/banyandb/e2e.yaml | 22 +++++++++++++++
.../profiling/ebpf/continuous/banyandb/e2e.yaml | 22 +++++++++++++++
.../cases/profiling/ebpf/network/banyandb/e2e.yaml | 23 +++++++++++++++-
.../cases/profiling/ebpf/oncpu/banyandb/e2e.yaml | 22 +++++++++++++++
.../e2e-v2/cases/profiling/pprof/banyandb/e2e.yaml | 15 ++++++++++
.../e2e-v2/cases/profiling/trace/banyandb/e2e.yaml | 15 ++++++++++
.../banyandb/data-generate/docker-compose.yml | 2 +-
.../cases/storage/banyandb/docker-compose.yml | 2 +-
test/e2e-v2/cases/storage/banyandb/e2e.yaml | 16 ++++++++++-
.../storage/banyandb/stages/docker-compose.yml | 4 +--
test/e2e-v2/cases/storage/banyandb/stages/e2e.yaml | 32 +++++++++++++++++++++-
.../cases/storage/banyandb/tls/docker-compose.yml | 2 +-
test/e2e-v2/cases/storage/banyandb/tls/e2e.yaml | 15 ++++++++++
test/e2e-v2/cases/zipkin/banyandb/e2e.yaml | 15 ++++++++++
test/e2e-v2/script/docker-compose/base-compose.yml | 2 +-
test/e2e-v2/script/env | 2 +-
26 files changed, 354 insertions(+), 19 deletions(-)
diff --git a/.github/workflows/skywalking.yaml
b/.github/workflows/skywalking.yaml
index 3f4340ba33..fcc2a11b7b 100644
--- a/.github/workflows/skywalking.yaml
+++ b/.github/workflows/skywalking.yaml
@@ -758,7 +758,7 @@ jobs:
if: matrix.test.docker != null
run: docker build -t ${{ matrix.test.docker.name }} -f ${{
matrix.test.docker.base }}/${{ matrix.test.docker.file }} ${{
matrix.test.docker.base }}
- name: ${{ matrix.test.name }}
- uses:
apache/skywalking-infra-e2e@8c21e43e241a32a54bdf8eeceb9099eb27e5e9b4
+ uses:
apache/skywalking-infra-e2e@e26033e1faaf865899c486ffe17dabdf17b90aae
with:
e2e-file: $GITHUB_WORKSPACE/${{ matrix.test.config }}
- if: ${{ failure() }}
@@ -831,7 +831,7 @@ jobs:
username: ${{ github.repository_owner }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: ${{ matrix.test.name }}
- uses:
apache/skywalking-infra-e2e@8c21e43e241a32a54bdf8eeceb9099eb27e5e9b4
+ uses:
apache/skywalking-infra-e2e@e26033e1faaf865899c486ffe17dabdf17b90aae
env:
ISTIO_VERSION: ${{ matrix.versions.istio }}
KUBERNETES_VERSION: ${{ matrix.versions.kubernetes }}
@@ -903,7 +903,7 @@ jobs:
username: ${{ github.repository_owner }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: ${{ matrix.test.name }}
- uses:
apache/skywalking-infra-e2e@8c21e43e241a32a54bdf8eeceb9099eb27e5e9b4
+ uses:
apache/skywalking-infra-e2e@e26033e1faaf865899c486ffe17dabdf17b90aae
env:
ISTIO_VERSION: ${{ matrix.versions.istio }}
KUBERNETES_VERSION: ${{ matrix.versions.kubernetes }}
@@ -967,7 +967,7 @@ jobs:
shell: bash
run: ./mvnw -B -q -f test/e2e-v2/java-test-service/pom.xml clean
package
- name: Java version ${{ matrix.java-version }}
- uses:
apache/skywalking-infra-e2e@8c21e43e241a32a54bdf8eeceb9099eb27e5e9b4
+ uses:
apache/skywalking-infra-e2e@e26033e1faaf865899c486ffe17dabdf17b90aae
env:
SW_AGENT_JDK_VERSION: ${{ matrix.java-version }}
with:
@@ -1042,12 +1042,12 @@ jobs:
found=false
for i in {1..60}; do
# check if segment files exist
- if docker exec $CONTAINER_ID sh -c '[ -n "$(ls
/tmp/measure-data/measure/data/sw_metricsDay/seg* 2>/dev/null)" ]'; then
+ if docker exec $CONTAINER_ID sh -c '[ -n "$(ls
/tmp/measure/data/sw_metricsDay/seg* 2>/dev/null)" ]'; then
echo "✅ found segment files"
sleep 180
# create and copy files
docker cp $CONTAINER_ID:/tmp ${BANYANDB_DATA_GENERATE_ROOT}
- docker cp
$CONTAINER_ID:/tmp/measure-data/measure/data/sw_metadata
${BANYANDB_DATA_GENERATE_ROOT}
+ docker cp $CONTAINER_ID:/tmp/measure/data/sw_metadata
${BANYANDB_DATA_GENERATE_ROOT}
found=true
break
else
@@ -1063,7 +1063,7 @@ jobs:
fi
docker compose -f ${BANYANDB_DATA_GENERATE_ROOT}/docker-compose.yml
down -v
- name: ${{ matrix.test.name }}
- uses:
apache/skywalking-infra-e2e@8c21e43e241a32a54bdf8eeceb9099eb27e5e9b4
+ uses:
apache/skywalking-infra-e2e@e26033e1faaf865899c486ffe17dabdf17b90aae
with:
e2e-file: $GITHUB_WORKSPACE/${{ matrix.test.config }}
- if: ${{ failure() }}
diff --git a/CLAUDE.md b/CLAUDE.md
index 5cda8aa29b..e5be190464 100644
--- a/CLAUDE.md
+++ b/CLAUDE.md
@@ -253,3 +253,29 @@ Actions owned by `actions/*` (GitHub), `github/*`, and
`apache/*` are always all
8. **Test both unit and integration**: Different test patterns for different
scopes
9. **Documentation is rendered via markdown**: When reviewing docs, consider
how they will be rendered by a markdown engine
10. **Relative paths in docs are valid**: Relative file paths (e.g.,
`../../../oap-server/...`) in documentation work both in the repo and on the
documentation website, supported by website build tooling
+
+## Analysis and Design Principles
+
+**Never guess or speculate.** All analysis must be grounded in source code,
documentation, or verified behavior.
+
+### Before making claims
+- **Read the source code** — don't assume how a feature works based on naming
or convention. Check the actual implementation.
+- **Read the documentation** — check `docs/en/`, CLAUDE.md files in
submodules, and README files before proposing designs.
+- **Check configuration and flags** — verify what flags/env vars exist, their
default values, and how they are parsed (e.g., BanyanDB uses viper with `BYDB_`
prefix to auto-bind flags to env vars).
+- **Check dependent projects** — SkyWalking depends on BanyanDB, infra-e2e,
Helm charts, etc. Read their source code and docs before assuming capabilities
(e.g., check Helm chart `values.yaml` for supported fields, check infra-e2e for
supported config options). For `skywalking-*` projects, ask the developer if
they have the source code locally — searching a local clone is much faster than
fetching files via GitHub API.
+
+### Before proposing changes
+- **Verify locally first** — run the code, start the container, execute the
test before pushing to CI. Don't use CI as a trial-and-error environment.
+- **Validate file paths and directory structures** — check where data actually
goes (e.g., BanyanDB `--stream-root-path /tmp` creates `/tmp/stream/`,
`--access-log-root-path /tmp` creates `/tmp/accesslog/`). Don't assume
directory names.
+- **Validate YAML syntax** — after editing YAML files (especially with
sed/awk), validate with a YAML parser before committing. Corrupted YAML causes
silent failures in CI.
+- **Check the actual Docker image** — verify what's available in the container
(binaries, shell, directories) before writing commands that depend on them.
+
+### When uncertain
+- **Say "I don't know" and investigate** — reading the code is always better
than guessing. Use grep, find, and read tools to locate the answer.
+- **Ask the developer first** — if you can't find the source code, don't know
how to run something, or the code doesn't make the answer clear, ask the
developer where to find it rather than speculate.
+- **Test with real data** — when investigating runtime behavior (e.g., what
model names an API returns, what directory structure BanyanDB creates), set up
a local test and observe the actual output.
+
+### Docker images
+- **Apache SkyWalking projects** — images are on `ghcr.io/apache/` (e.g.,
`ghcr.io/apache/skywalking-banyandb:${COMMIT_SHA}`). Tags are full commit SHAs,
not short SHAs or version tags.
+- **Official and 3rd-party images** — on Docker Hub (e.g., `ollama/ollama`,
`otel/opentelemetry-collector`, `envoyproxy/gateway`).
+- **Always verify the image exists** — `docker pull` before writing CI or e2e
configs. Image tags depend on CI publish workflows completing successfully.
diff --git a/test/e2e-v2/cases/alarm/banyandb/e2e.yaml
b/test/e2e-v2/cases/alarm/banyandb/e2e.yaml
index 338976fb3f..6800dc46d3 100644
--- a/test/e2e-v2/cases/alarm/banyandb/e2e.yaml
+++ b/test/e2e-v2/cases/alarm/banyandb/e2e.yaml
@@ -45,3 +45,18 @@ verify:
cases:
- includes:
- ../alarm-cases.yaml
+
+cleanup:
+ on: always
+ collect:
+ on: failure
+ output-dir: $SW_INFRA_E2E_LOG_DIR/banyandb-data
+ items:
+ - service: banyandb
+ paths:
+ - /tmp/trace/
+ - /tmp/stream/
+ - /tmp/measure/
+ - /tmp/property/
+ - /tmp/schema-property/
+ - /tmp/accesslog/
diff --git a/test/e2e-v2/cases/banyandb/e2e.yaml
b/test/e2e-v2/cases/banyandb/e2e.yaml
index ce742441ce..965f8da653 100644
--- a/test/e2e-v2/cases/banyandb/e2e.yaml
+++ b/test/e2e-v2/cases/banyandb/e2e.yaml
@@ -37,3 +37,18 @@ verify:
- ./banyandb-cases.yaml
+
+cleanup:
+ on: always
+ collect:
+ on: failure
+ output-dir: $SW_INFRA_E2E_LOG_DIR/banyandb-data
+ items:
+ - service: banyandb
+ paths:
+ - /tmp/trace/
+ - /tmp/stream/
+ - /tmp/measure/
+ - /tmp/property/
+ - /tmp/schema-property/
+ - /tmp/accesslog/
diff --git a/test/e2e-v2/cases/baseline/banyandb/e2e.yaml
b/test/e2e-v2/cases/baseline/banyandb/e2e.yaml
index da9bbc39e2..fd8ba6b8f6 100644
--- a/test/e2e-v2/cases/baseline/banyandb/e2e.yaml
+++ b/test/e2e-v2/cases/baseline/banyandb/e2e.yaml
@@ -45,3 +45,18 @@ verify:
cases:
- includes:
- ../alarm-cases.yaml
+
+cleanup:
+ on: always
+ collect:
+ on: failure
+ output-dir: $SW_INFRA_E2E_LOG_DIR/banyandb-data
+ items:
+ - service: banyandb
+ paths:
+ - /tmp/trace/
+ - /tmp/stream/
+ - /tmp/measure/
+ - /tmp/property/
+ - /tmp/schema-property/
+ - /tmp/accesslog/
diff --git a/test/e2e-v2/cases/cluster/zk/banyandb/e2e.yaml
b/test/e2e-v2/cases/cluster/zk/banyandb/e2e.yaml
index 4ac22598c7..7db97cd61c 100644
--- a/test/e2e-v2/cases/cluster/zk/banyandb/e2e.yaml
+++ b/test/e2e-v2/cases/cluster/zk/banyandb/e2e.yaml
@@ -38,3 +38,18 @@ verify:
cases:
- includes:
- ../../cluster-cases.yaml
+
+cleanup:
+ on: always
+ collect:
+ on: failure
+ output-dir: $SW_INFRA_E2E_LOG_DIR/banyandb-data
+ items:
+ - service: banyandb
+ paths:
+ - /tmp/trace/
+ - /tmp/stream/
+ - /tmp/measure/
+ - /tmp/property/
+ - /tmp/schema-property/
+ - /tmp/accesslog/
diff --git a/test/e2e-v2/cases/event/banyandb/e2e.yaml
b/test/e2e-v2/cases/event/banyandb/e2e.yaml
index c957da2316..7642e4ccf7 100644
--- a/test/e2e-v2/cases/event/banyandb/e2e.yaml
+++ b/test/e2e-v2/cases/event/banyandb/e2e.yaml
@@ -35,3 +35,18 @@ verify:
cases:
- includes:
- ../event-cases.yaml
+
+cleanup:
+ on: always
+ collect:
+ on: failure
+ output-dir: $SW_INFRA_E2E_LOG_DIR/banyandb-data
+ items:
+ - service: banyandb
+ paths:
+ - /tmp/trace/
+ - /tmp/stream/
+ - /tmp/measure/
+ - /tmp/property/
+ - /tmp/schema-property/
+ - /tmp/accesslog/
diff --git a/test/e2e-v2/cases/log/banyandb/e2e.yaml
b/test/e2e-v2/cases/log/banyandb/e2e.yaml
index bc8437238c..b5db06da63 100644
--- a/test/e2e-v2/cases/log/banyandb/e2e.yaml
+++ b/test/e2e-v2/cases/log/banyandb/e2e.yaml
@@ -56,4 +56,18 @@ verify:
swctl --display yaml
--base-url=http://${oap_host}:${oap_12800}/graphql logs list
--service-name=e2e-service-provider --tags level=INFO --trace-id=$( \
swctl --display yaml
--base-url=http://${oap_host}:${oap_12800}/graphql tv2 ls | yq e '.traces |
map(select(.spans[0].endpointname == "GET:/logs/trigger")) |
.[0].spans[0].traceid' -
)
- expected: ../expected/logs.yml
\ No newline at end of file
+ expected: ../expected/logs.yml
+cleanup:
+ on: always
+ collect:
+ on: failure
+ output-dir: $SW_INFRA_E2E_LOG_DIR/banyandb-data
+ items:
+ - service: banyandb
+ paths:
+ - /tmp/trace/
+ - /tmp/stream/
+ - /tmp/measure/
+ - /tmp/property/
+ - /tmp/schema-property/
+ - /tmp/accesslog/
diff --git a/test/e2e-v2/cases/menu/banyandb/e2e.yaml
b/test/e2e-v2/cases/menu/banyandb/e2e.yaml
index 74b8b82d44..b39eeac856 100644
--- a/test/e2e-v2/cases/menu/banyandb/e2e.yaml
+++ b/test/e2e-v2/cases/menu/banyandb/e2e.yaml
@@ -44,4 +44,18 @@ verify:
interval: 10s
cases:
- includes:
- - ../menu-cases.yaml
\ No newline at end of file
+ - ../menu-cases.yaml
+cleanup:
+ on: always
+ collect:
+ on: failure
+ output-dir: $SW_INFRA_E2E_LOG_DIR/banyandb-data
+ items:
+ - service: banyandb
+ paths:
+ - /tmp/trace/
+ - /tmp/stream/
+ - /tmp/measure/
+ - /tmp/property/
+ - /tmp/schema-property/
+ - /tmp/accesslog/
diff --git a/test/e2e-v2/cases/profiling/async-profiler/banyandb/e2e.yaml
b/test/e2e-v2/cases/profiling/async-profiler/banyandb/e2e.yaml
index 956e87c282..73849b9701 100644
--- a/test/e2e-v2/cases/profiling/async-profiler/banyandb/e2e.yaml
+++ b/test/e2e-v2/cases/profiling/async-profiler/banyandb/e2e.yaml
@@ -35,3 +35,18 @@ verify:
cases:
- includes:
- ../profiling-cases.yaml
+
+cleanup:
+ on: always
+ collect:
+ on: failure
+ output-dir: $SW_INFRA_E2E_LOG_DIR/banyandb-data
+ items:
+ - service: banyandb
+ paths:
+ - /tmp/trace/
+ - /tmp/stream/
+ - /tmp/measure/
+ - /tmp/property/
+ - /tmp/schema-property/
+ - /tmp/accesslog/
diff --git a/test/e2e-v2/cases/profiling/ebpf/access_log/banyandb/e2e.yaml
b/test/e2e-v2/cases/profiling/ebpf/access_log/banyandb/e2e.yaml
index a4d7e8dee9..269367aa58 100644
--- a/test/e2e-v2/cases/profiling/ebpf/access_log/banyandb/e2e.yaml
+++ b/test/e2e-v2/cases/profiling/ebpf/access_log/banyandb/e2e.yaml
@@ -58,6 +58,12 @@ setup:
--set oap.image.repository=skywalking/oap \
--set oap.storageType=banyandb \
--set oap.env.SW_HEALTH_CHECKER=default \
+ --set
"banyandb.standalone.env[0].name=BYDB_ENABLE_INGESTION_ACCESS_LOG" \
+ --set "banyandb.standalone.env[0].value=\"true\"" \
+ --set "banyandb.standalone.env[1].name=BYDB_ENABLE_QUERY_ACCESS_LOG"
\
+ --set "banyandb.standalone.env[1].value=\"true\"" \
+ --set "banyandb.standalone.env[2].name=BYDB_ACCESS_LOG_ROOT_PATH" \
+ --set "banyandb.standalone.env[2].value=/tmp" \
-f test/e2e-v2/cases/profiling/ebpf/kubernetes-values.yaml
wait:
- namespace: istio-system
@@ -86,3 +92,19 @@ verify:
cases:
- includes:
- ../accesslog-cases.yaml
+
+cleanup:
+ on: always
+ collect:
+ on: failure
+ output-dir: $SW_INFRA_E2E_LOG_DIR/banyandb-data
+ items:
+ - namespace: istio-system
+ label-selector: app.kubernetes.io/name=banyandb
+ paths:
+ - /tmp/trace/
+ - /tmp/stream/
+ - /tmp/measure/
+ - /tmp/property/
+ - /tmp/schema-property/
+ - /tmp/accesslog/
diff --git a/test/e2e-v2/cases/profiling/ebpf/continuous/banyandb/e2e.yaml
b/test/e2e-v2/cases/profiling/ebpf/continuous/banyandb/e2e.yaml
index 8742c915e4..691f6e830e 100644
--- a/test/e2e-v2/cases/profiling/ebpf/continuous/banyandb/e2e.yaml
+++ b/test/e2e-v2/cases/profiling/ebpf/continuous/banyandb/e2e.yaml
@@ -59,6 +59,12 @@ setup:
--set oap.image.repository=skywalking/oap \
--set oap.storageType=banyandb \
--set oap.env.SW_HEALTH_CHECKER=default \
+ --set
"banyandb.standalone.env[0].name=BYDB_ENABLE_INGESTION_ACCESS_LOG" \
+ --set "banyandb.standalone.env[0].value=\"true\"" \
+ --set "banyandb.standalone.env[1].name=BYDB_ENABLE_QUERY_ACCESS_LOG"
\
+ --set "banyandb.standalone.env[1].value=\"true\"" \
+ --set "banyandb.standalone.env[2].name=BYDB_ACCESS_LOG_ROOT_PATH" \
+ --set "banyandb.standalone.env[2].value=/tmp" \
-f test/e2e-v2/cases/profiling/ebpf/kubernetes-values.yaml
wait:
- namespace: istio-system
@@ -84,3 +90,19 @@ verify:
cases:
- includes:
- ../profiling-cases.yaml
+
+cleanup:
+ on: always
+ collect:
+ on: failure
+ output-dir: $SW_INFRA_E2E_LOG_DIR/banyandb-data
+ items:
+ - namespace: istio-system
+ label-selector: app.kubernetes.io/name=banyandb
+ paths:
+ - /tmp/trace/
+ - /tmp/stream/
+ - /tmp/measure/
+ - /tmp/property/
+ - /tmp/schema-property/
+ - /tmp/accesslog/
diff --git a/test/e2e-v2/cases/profiling/ebpf/network/banyandb/e2e.yaml
b/test/e2e-v2/cases/profiling/ebpf/network/banyandb/e2e.yaml
index 9f9befaa8e..e6bdb0aea1 100644
--- a/test/e2e-v2/cases/profiling/ebpf/network/banyandb/e2e.yaml
+++ b/test/e2e-v2/cases/profiling/ebpf/network/banyandb/e2e.yaml
@@ -62,6 +62,12 @@ setup:
--set oap.image.repository=skywalking/oap \
--set oap.storageType=banyandb \
--set oap.env.SW_HEALTH_CHECKER=default \
+ --set
"banyandb.standalone.env[0].name=BYDB_ENABLE_INGESTION_ACCESS_LOG" \
+ --set "banyandb.standalone.env[0].value=\"true\"" \
+ --set "banyandb.standalone.env[1].name=BYDB_ENABLE_QUERY_ACCESS_LOG"
\
+ --set "banyandb.standalone.env[1].value=\"true\"" \
+ --set "banyandb.standalone.env[2].name=BYDB_ACCESS_LOG_ROOT_PATH" \
+ --set "banyandb.standalone.env[2].value=/tmp" \
-f test/e2e-v2/cases/profiling/ebpf/network/kubernetes-values.yaml
wait:
- namespace: istio-system
@@ -93,4 +99,19 @@ verify:
interval: 10s
cases:
- includes:
- - ../network-cases.yaml
\ No newline at end of file
+ - ../network-cases.yaml
+cleanup:
+ on: always
+ collect:
+ on: failure
+ output-dir: $SW_INFRA_E2E_LOG_DIR/banyandb-data
+ items:
+ - namespace: istio-system
+ label-selector: app.kubernetes.io/name=banyandb
+ paths:
+ - /tmp/trace/
+ - /tmp/stream/
+ - /tmp/measure/
+ - /tmp/property/
+ - /tmp/schema-property/
+ - /tmp/accesslog/
diff --git a/test/e2e-v2/cases/profiling/ebpf/oncpu/banyandb/e2e.yaml
b/test/e2e-v2/cases/profiling/ebpf/oncpu/banyandb/e2e.yaml
index 5ab6c5d9f0..184d80df80 100644
--- a/test/e2e-v2/cases/profiling/ebpf/oncpu/banyandb/e2e.yaml
+++ b/test/e2e-v2/cases/profiling/ebpf/oncpu/banyandb/e2e.yaml
@@ -59,6 +59,12 @@ setup:
--set oap.image.repository=skywalking/oap \
--set oap.storageType=banyandb \
--set oap.env.SW_HEALTH_CHECKER=default \
+ --set
"banyandb.standalone.env[0].name=BYDB_ENABLE_INGESTION_ACCESS_LOG" \
+ --set "banyandb.standalone.env[0].value=\"true\"" \
+ --set "banyandb.standalone.env[1].name=BYDB_ENABLE_QUERY_ACCESS_LOG"
\
+ --set "banyandb.standalone.env[1].value=\"true\"" \
+ --set "banyandb.standalone.env[2].name=BYDB_ACCESS_LOG_ROOT_PATH" \
+ --set "banyandb.standalone.env[2].value=/tmp" \
-f test/e2e-v2/cases/profiling/ebpf/kubernetes-values.yaml
wait:
- namespace: istio-system
@@ -84,3 +90,19 @@ verify:
cases:
- includes:
- ../profiling-cases.yaml
+
+cleanup:
+ on: always
+ collect:
+ on: failure
+ output-dir: $SW_INFRA_E2E_LOG_DIR/banyandb-data
+ items:
+ - namespace: istio-system
+ label-selector: app.kubernetes.io/name=banyandb
+ paths:
+ - /tmp/trace/
+ - /tmp/stream/
+ - /tmp/measure/
+ - /tmp/property/
+ - /tmp/schema-property/
+ - /tmp/accesslog/
diff --git a/test/e2e-v2/cases/profiling/pprof/banyandb/e2e.yaml
b/test/e2e-v2/cases/profiling/pprof/banyandb/e2e.yaml
index 956e87c282..73849b9701 100644
--- a/test/e2e-v2/cases/profiling/pprof/banyandb/e2e.yaml
+++ b/test/e2e-v2/cases/profiling/pprof/banyandb/e2e.yaml
@@ -35,3 +35,18 @@ verify:
cases:
- includes:
- ../profiling-cases.yaml
+
+cleanup:
+ on: always
+ collect:
+ on: failure
+ output-dir: $SW_INFRA_E2E_LOG_DIR/banyandb-data
+ items:
+ - service: banyandb
+ paths:
+ - /tmp/trace/
+ - /tmp/stream/
+ - /tmp/measure/
+ - /tmp/property/
+ - /tmp/schema-property/
+ - /tmp/accesslog/
diff --git a/test/e2e-v2/cases/profiling/trace/banyandb/e2e.yaml
b/test/e2e-v2/cases/profiling/trace/banyandb/e2e.yaml
index 76ded1735d..97e14b6ba6 100644
--- a/test/e2e-v2/cases/profiling/trace/banyandb/e2e.yaml
+++ b/test/e2e-v2/cases/profiling/trace/banyandb/e2e.yaml
@@ -36,3 +36,18 @@ verify:
- includes:
- ../profiling-cases-trace-v2.yaml
- ../profiling-cases.yaml
+
+cleanup:
+ on: always
+ collect:
+ on: failure
+ output-dir: $SW_INFRA_E2E_LOG_DIR/banyandb-data
+ items:
+ - service: banyandb
+ paths:
+ - /tmp/trace/
+ - /tmp/stream/
+ - /tmp/measure/
+ - /tmp/property/
+ - /tmp/schema-property/
+ - /tmp/accesslog/
diff --git
a/test/e2e-v2/cases/storage/banyandb/data-generate/docker-compose.yml
b/test/e2e-v2/cases/storage/banyandb/data-generate/docker-compose.yml
index 8a54ef9275..10c048ffaf 100644
--- a/test/e2e-v2/cases/storage/banyandb/data-generate/docker-compose.yml
+++ b/test/e2e-v2/cases/storage/banyandb/data-generate/docker-compose.yml
@@ -21,7 +21,7 @@ services:
ports:
- 17912:17912
- 17913:17913
- command: standalone --stream-root-path /tmp/stream-data
--measure-root-path /tmp/measure-data --measure-metadata-cache-wait-duration 1m
--stream-metadata-cache-wait-duration 1m
+ command: standalone --measure-metadata-cache-wait-duration 1m
--stream-metadata-cache-wait-duration 1m --enable-ingestion-access-log
--enable-query-access-log --access-log-root-path /tmp
networks:
- e2e
healthcheck:
diff --git a/test/e2e-v2/cases/storage/banyandb/docker-compose.yml
b/test/e2e-v2/cases/storage/banyandb/docker-compose.yml
index 6c65818547..32b44ec207 100644
--- a/test/e2e-v2/cases/storage/banyandb/docker-compose.yml
+++ b/test/e2e-v2/cases/storage/banyandb/docker-compose.yml
@@ -25,7 +25,7 @@ services:
entrypoint: /bin/sh -c
command: |
"chmod 600 /auth-config.yaml &&
- /banyand standalone --stream-root-path /tmp/stream-data
--measure-root-path /tmp/measure-data --measure-metadata-cache-wait-duration 1m
--stream-metadata-cache-wait-duration 1m --auth-config-file=/auth-config.yaml"
+ /banyand standalone --measure-metadata-cache-wait-duration 1m
--stream-metadata-cache-wait-duration 1m --enable-ingestion-access-log
--enable-query-access-log --access-log-root-path /tmp
--auth-config-file=/auth-config.yaml"
networks:
- e2e
diff --git a/test/e2e-v2/cases/storage/banyandb/e2e.yaml
b/test/e2e-v2/cases/storage/banyandb/e2e.yaml
index a6d4d8489b..2c2cb051a6 100644
--- a/test/e2e-v2/cases/storage/banyandb/e2e.yaml
+++ b/test/e2e-v2/cases/storage/banyandb/e2e.yaml
@@ -62,4 +62,18 @@ verify:
- query: swctl --display yaml
--base-url=http://${oap_host}:${oap_12800}/graphql tv2 ls --tags
http.method=POST
expected: ../expected/traces-v2-list.yml
- query: swctl --display yaml
--base-url=http://${oap_host}:${oap_12800}/graphql tv2 ls --tags
http.method=POST,http.status_code=201
- expected: ../expected/empty-traces-v2-list.yml
\ No newline at end of file
+ expected: ../expected/empty-traces-v2-list.yml
+cleanup:
+ on: always
+ collect:
+ on: failure
+ output-dir: $SW_INFRA_E2E_LOG_DIR/banyandb-data
+ items:
+ - service: banyandb
+ paths:
+ - /tmp/trace/
+ - /tmp/stream/
+ - /tmp/measure/
+ - /tmp/property/
+ - /tmp/schema-property/
+ - /tmp/accesslog/
diff --git a/test/e2e-v2/cases/storage/banyandb/stages/docker-compose.yml
b/test/e2e-v2/cases/storage/banyandb/stages/docker-compose.yml
index 12469e3b4e..f44df42d26 100644
--- a/test/e2e-v2/cases/storage/banyandb/stages/docker-compose.yml
+++ b/test/e2e-v2/cases/storage/banyandb/stages/docker-compose.yml
@@ -46,8 +46,8 @@ services:
hostname: data-cold1
command: data --node-discovery-mode=file
--node-discovery-file-path=/etc/banyandb/nodes.yaml --node-labels type=cold
volumes:
- - ../data-generate/tmp/measure-data/measure:/tmp/measure
- - ../data-generate/tmp/stream-data/stream:/tmp/stream
+ - ../data-generate/tmp/measure:/tmp/measure
+ - ../data-generate/tmp/stream:/tmp/stream
- ../data-generate/tmp/property:/tmp/property
- ../data-generate/tmp/trace:/tmp/trace
- ./nodes.yaml:/etc/banyandb/nodes.yaml
diff --git a/test/e2e-v2/cases/storage/banyandb/stages/e2e.yaml
b/test/e2e-v2/cases/storage/banyandb/stages/e2e.yaml
index 22d2548b01..f31019a7b3 100644
--- a/test/e2e-v2/cases/storage/banyandb/stages/e2e.yaml
+++ b/test/e2e-v2/cases/storage/banyandb/stages/e2e.yaml
@@ -67,4 +67,34 @@ verify:
- query: swctl --display yaml
--base-url=http://${oap_host}:${oap_12800}/graphql metrics exec
--expression="top_n(service_resp_time,3,des,attr0='GENERAL')" --start="-96h"
--end="-48h" --cold=true --step=DAY
expected: ../../expected/cold/topN-OP-service.yml
- query: swctl --display yaml
--base-url=http://${oap_host}:${oap_12800}/graphql metrics exec
--expression="top_n(endpoint_resp_time,3,des)" --start="-96h" --end="-48h"
--cold=true --step=DAY
- expected: ../../expected/cold/topN-OP-endpoint.yml
\ No newline at end of file
+ expected: ../../expected/cold/topN-OP-endpoint.yml
+cleanup:
+ on: always
+ collect:
+ on: failure
+ output-dir: $SW_INFRA_E2E_LOG_DIR/banyandb-data
+ items:
+ - service: data-hot1
+ paths:
+ - /tmp/trace/
+ - /tmp/stream/
+ - /tmp/measure/
+ - /tmp/property/
+ - /tmp/schema-property/
+ - /tmp/accesslog/
+ - service: data-warm1
+ paths:
+ - /tmp/trace/
+ - /tmp/stream/
+ - /tmp/measure/
+ - /tmp/property/
+ - /tmp/schema-property/
+ - /tmp/accesslog/
+ - service: data-cold1
+ paths:
+ - /tmp/trace/
+ - /tmp/stream/
+ - /tmp/measure/
+ - /tmp/property/
+ - /tmp/schema-property/
+ - /tmp/accesslog/
diff --git a/test/e2e-v2/cases/storage/banyandb/tls/docker-compose.yml
b/test/e2e-v2/cases/storage/banyandb/tls/docker-compose.yml
index 75f4c6fcc5..fe842021b6 100644
--- a/test/e2e-v2/cases/storage/banyandb/tls/docker-compose.yml
+++ b/test/e2e-v2/cases/storage/banyandb/tls/docker-compose.yml
@@ -22,7 +22,7 @@ services:
service: banyandb
volumes:
- ./tls:/tls
- command: standalone --stream-root-path /tmp/stream-data
--measure-root-path /tmp/measure-data --tls=true --key-file=/tls/cert.key
--cert-file=/tls/cert.crt --http-grpc-cert-file=/tls/ca.crt
+ command: standalone --tls=true --key-file=/tls/cert.key
--cert-file=/tls/cert.crt --http-grpc-cert-file=/tls/ca.crt
--enable-ingestion-access-log --enable-query-access-log --access-log-root-path
/tmp
networks:
- e2e
diff --git a/test/e2e-v2/cases/storage/banyandb/tls/e2e.yaml
b/test/e2e-v2/cases/storage/banyandb/tls/e2e.yaml
index c93bbedceb..507cf9ef90 100644
--- a/test/e2e-v2/cases/storage/banyandb/tls/e2e.yaml
+++ b/test/e2e-v2/cases/storage/banyandb/tls/e2e.yaml
@@ -48,3 +48,18 @@ verify:
cases:
- includes:
- ../../storage-cases.yaml
+
+cleanup:
+ on: always
+ collect:
+ on: failure
+ output-dir: $SW_INFRA_E2E_LOG_DIR/banyandb-data
+ items:
+ - service: banyandb
+ paths:
+ - /tmp/trace/
+ - /tmp/stream/
+ - /tmp/measure/
+ - /tmp/property/
+ - /tmp/schema-property/
+ - /tmp/accesslog/
diff --git a/test/e2e-v2/cases/zipkin/banyandb/e2e.yaml
b/test/e2e-v2/cases/zipkin/banyandb/e2e.yaml
index b77ab08e18..c8b724aa9e 100644
--- a/test/e2e-v2/cases/zipkin/banyandb/e2e.yaml
+++ b/test/e2e-v2/cases/zipkin/banyandb/e2e.yaml
@@ -45,3 +45,18 @@ verify:
cases:
- includes:
- ../zipkin-cases.yaml
+
+cleanup:
+ on: always
+ collect:
+ on: failure
+ output-dir: $SW_INFRA_E2E_LOG_DIR/banyandb-data
+ items:
+ - service: banyandb
+ paths:
+ - /tmp/trace/
+ - /tmp/stream/
+ - /tmp/measure/
+ - /tmp/property/
+ - /tmp/schema-property/
+ - /tmp/accesslog/
diff --git a/test/e2e-v2/script/docker-compose/base-compose.yml
b/test/e2e-v2/script/docker-compose/base-compose.yml
index ceb62bae95..d421ea6d28 100644
--- a/test/e2e-v2/script/docker-compose/base-compose.yml
+++ b/test/e2e-v2/script/docker-compose/base-compose.yml
@@ -60,7 +60,7 @@ services:
ports:
- 17912:17912
- 17913:17913
- command: standalone --stream-root-path /tmp/stream-data
--measure-root-path /tmp/measure-data --measure-metadata-cache-wait-duration 1m
--stream-metadata-cache-wait-duration 1m
+ command: standalone --measure-metadata-cache-wait-duration 1m
--stream-metadata-cache-wait-duration 1m --enable-ingestion-access-log
--enable-query-access-log --access-log-root-path /tmp
healthcheck:
test: [ "CMD", "sh", "-c", "nc -nz 127.0.0.1 17912" ]
interval: 5s
diff --git a/test/e2e-v2/script/env b/test/e2e-v2/script/env
index e8b227794e..ae6486451f 100644
--- a/test/e2e-v2/script/env
+++ b/test/e2e-v2/script/env
@@ -23,7 +23,7 @@
SW_AGENT_CLIENT_JS_COMMIT=f08776d909eb1d9bc79c600e493030651b97e491
SW_AGENT_CLIENT_JS_TEST_COMMIT=4f1eb1dcdbde3ec4a38534bf01dded4ab5d2f016
SW_KUBERNETES_COMMIT_SHA=2850db1502283a2d8516146c57cc2b49f1da934b
SW_ROVER_COMMIT=79292fe07f17f98f486e0c4471213e1961fb2d1d
-SW_BANYANDB_COMMIT=3c0e3a78d112445c293d9b0f0a5599cd9083334f
+SW_BANYANDB_COMMIT=d0f41f9ad139c917c1398c2e62a9f7034214495f
SW_AGENT_PHP_COMMIT=d1114e7be5d89881eec76e5b56e69ff844691e35
SW_PREDICTOR_COMMIT=54a0197654a3781a6f73ce35146c712af297c994
