shangeyao opened a new issue, #4477:
URL: https://github.com/apache/streampark/issues/4477
## Background
Current E2E (`streampark-e2e`) depends on:
- Full Docker image build (`build.sh` + `docker/Dockerfile`)
- Testcontainers + docker-compose
- Selenium Chrome (UI automation against legacy webapp
`streampark-console-webapp`)
This causes:
1. **High flakiness** — Selenium timeouts, element not interactable, CI
`cancel-in-progress` cascading cancellations
2. **Long CI runtime** — E2E-Build ~7 min + matrix jobs ~5–7 min each (often
30–60 min total)
3. **Tight coupling to UI/DTO wire format** — backend API behavior can be
correct while E2E fails due to wire format or `@Valid` strictness (e.g. Alarm
list JSON string vs object, `resourcePath` null copy)
4. **High maintenance** — page objects break on Ant Design / i18n / layout
changes
**Goal:** Replace UI-driven E2E with **API integration tests** in
`streampark-console-service`, covering the same business flows via REST without
a browser or frontend.
## Scope
**In scope (migrate from E2E matrix):**
| Current E2E class | API integration test target |
|-------------------|----------------------------|
| `EnvironmentTest` | Flink/Spark env CRUD + default |
| `AlarmTest` | `/flink/alert/*` CRUD + exists + send |
| `UserManagementTest` | `/user/*` CRUD + password |
| `RoleManagementTest` | `/role/*` |
| `TeamManagementTest` | `/team/*` |
| `MemberManagementTest` | `/member/*` |
| `ExternalLinkTest` | `/flink/externalLink/*` |
| `YarnQueueTest` | `/yarn/queue/*` + check |
| `TokenManagementTest` | `/token/*` |
| `UploadManagementTest` | `/resource/*` + upload |
| `ProjectsManagementTest` | `/project/*` |
| `VariableManagementTest` | `/variable/*` |
**Out of scope / separate track:**
- `Flink120OnRemoteClusterDeployTest` — real Flink cluster deploy; keep as
optional heavy E2E or dedicated deploy test job
**Not changing (this issue):**
- Legacy webapp (`streampark-console-webapp`)
- webapp-v2
## Proposed architecture
```
streampark-console/streampark-console-service/src/test/java/
org/apache/streampark/console/
integration/ # NEW package
base/
ApiIntegrationTestBase.java # @SpringBootTest(RANDOM_PORT) +
auth helper
ApiClient.java # TestRestTemplate wrapper,
JWT/Shiro session
cases/
AlarmApiIT.java
UserApiIT.java
...
```
**Stack:**
- `@SpringBootTest(webEnvironment =
SpringBootTest.WebEnvironment.RANDOM_PORT)`
- `TestRestTemplate` or `@AutoConfigureMockMvc` + `MockMvc`
- Reuse H2 in-memory DB + existing test SQL (`SpringUnitTestBase` patterns)
- Shiro/JWT: login via `POST /passport/signin`, reuse token for
`@Permission` endpoints
- Assert `RestResponseBody` envelope: `code`, `data`, `message`, legacy
`extra` fields
**Request paths to cover explicitly:**
- `@FormOrJson` form-urlencoded path (legacy webapp default)
- `@RequestBody` JSON path (Alert, OpenAPI)
- Query/form params for list/page endpoints
## Implementation phases
### Phase 0 — Infrastructure (1 PR)
- [ ] `ApiIntegrationTestBase` with admin login, team context, helper methods
- [ ] `ApiClient.postForm()`, `postJson()`, `deleteForm()`, multipart upload
- [ ] Sample `PassportApiIT` or `HealthApiIT` as template
- [ ] Maven: ensure integration tests run in `./mvnw test -pl
streampark-console-service`
- [ ] Document conventions in test class Javadoc
### Phase 1 — Settings module (1 PR)
Migrate: Alarm, YarnQueue, ExternalLink, Variable, Environment (Flink/Spark
env)
- Wire-format regression tests (e.g. alert list returns JSON **strings** for
`*Params`)
- Check/exists endpoints use slim DTOs, not full create DTO + `@Valid`
### Phase 2 — System module (1 PR)
Migrate: User, Role, Team, Member, Token
- RBAC: test with admin vs non-admin where applicable
- Token create legacy `extra("code", 0)` behavior
### Phase 3 — Resource & Project (1 PR)
Migrate: Upload/Resource, Projects
- Multipart upload via MockMvc
- Project build/build_log polling (assert `offset` / `readFinished` extras)
### Phase 4 — CI & deprecation (1 PR)
- [ ] New workflow `.github/workflows/integration-test.yml`:
- JDK 11, `./mvnw test -pl streampark-console/streampark-console-service`
- No Docker image build, no Selenium
- Target: < 10 min total
- [ ] Mark E2E matrix jobs as `continue-on-error` or move to nightly
- [ ] Update `AGENTS.md` testing section
- [ ] Eventually remove or archive `streampark-e2e` (separate issue/PR after
soak period)
## Test design principles
1. **Test behavior, not UI** — assert HTTP status + `RestResponseBody` + DB
state
2. **One endpoint contract per assertion** — avoid mega-tests
3. **Legacy wire compatibility** — form field names match webapp; document
in test names
4. **No `@Valid` on progressive check endpoints** — match service-layer
status codes
5. **Idempotent setup** — `@Transactional` or `@Sql` per class; unique names
with UUID suffix
6. **Do not duplicate unit tests** — integration = full stack Controller →
Service → DB
## Unit vs integration vs E2E boundary
| Layer | What to test | Example |
|-------|--------------|---------|
| Unit | Service/Assembler logic | `UserServiceTest` |
| Integration | Controller → DB full path + HTTP envelope | `AlarmApiIT` |
| E2E (keep few) | Real Flink/K8s deploy |
`Flink120OnRemoteClusterDeployTest` |
## Acceptance criteria
- [ ] All 12 E2E cases above have API IT equivalents with ≥ same scenario
coverage
- [ ] CI integration-test job green on PR to `dev`
- [ ] E2E workflow no longer blocks PR merge (nightly or removed)
- [ ] No dependency on Selenium/Testcontainers for console CRUD flows
- [ ] Documented migration mapping E2E class → IT class in issue or `docs/`
## Risks & mitigations
| Risk | Mitigation |
|------|------------|
| Shiro permission hard to simulate | Base class admin login; `@MockBean`
only for PermissionAspect unit tests |
| FormOrJson dual path missed | At least 1 form + 1 json case per write
endpoint (Alert etc.) |
| H2 dialect differences | Reuse existing test SQL; MySQL-specific syntax
via unit + manual validation |
| Team used to Selenium debug | Log HTTP request/response body on IT failure
|
## References
- Current E2E workflow: `.github/workflows/e2e.yml`
- DTO migration PR: #4475
- Recent E2E failures: DTO wire format, `@Valid` strictness, Selenium
flakiness
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
