ton/ASG_Framework
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 4 Wiki Activity

update

Browse Source
This commit is contained in:
narawat
2026-09-04 09:45:40 +07:00
parent 9c8b2e6453
commit f560951379
1 changed files with 38 additions and 38 deletions
Download Patch File Download Diff File Expand all files Collapse all files
ASG_Framework.md
+38 -38
View File
@@ -207,44 +207,36 @@ The walkthrough is organized as one document with a dedicated section per system
**Purpose**: The *enforcer* — defines tests **before implementation begins** to ensure spec compliance. Blocks drift and human error. **Purpose**: The *enforcer* — defines tests **before implementation begins** to ensure spec compliance. Blocks drift and human error.
**Why It Matters**: Because implementation must satisfy specifications, Validation derives all test definitions from the preceding 5 documents:
- Ensures tests are written before implementation, so code can be tested as it's built
- Prevents breaking changes from reaching production
- Catches specification violations early in the CI pipeline
- Maintains data integrity and API consistency
- Reduces manual QA effort through automation
- Provides immediate feedback to developers during implementation
- Validates performance NFRs (latency, throughput) against measurable KPIs before deployment
- Validates security requirements through adversarial testing before production
**Content Guidelines**: | Source Document | What Validation Tests |
- **Unit tests** — atomic tests for each specification rule, data schema field, and UI component state (cites specification.md sections, ui-specification.md component specs) |-----------------|----------------------|
- **Integration tests** — end-to-end flow validation covering workflows defined in 3-walkthrough.md (cites specification.md endpoint chains, 3-walkthrough.md participant sections) | **1. Requirements** | NFR thresholds (latency, throughput), security constraints, observability checks |
- **Performance tests** — load tests, stress tests, and benchmarking against NFR thresholds (cites requirements.md NFRs, solution-design.md performance decisions) | **2. Solution Design** | Architectural decisions, performance choices, risk mitigations |
- **Penetration tests** — adversarial testing of security requirements and auth flows (cites requirements.md security NFRs, specification.md auth endpoints) | **3. Walkthrough** | End-to-end flows for each participant section, cross-participant interactions |
- Contract testing scripts (mapped to specification.md sections) | **4. Specification** | API contracts, data schemas, error codes, validation rules |
- Linting rules and configurations (enforcing style guides from ui-specification.md) | **5. UI Specification** | Component states, visual patterns, accessibility compliance |
- Schema validation jobs (validating data shapes from specification.md)
- Visual regression tests (verifying UI matches ui-specification.md)
- Security scanning and audit jobs (SAST, dependency vulnerability checks)
- CI/CD pipeline configurations for automated test execution
- Each validation test must include explicit references to:
- **specification.md** (spec IDs or paths — API contracts, data schemas, e.g., `/api/v1/endpoint` or `#/components/schemas/Request`)
- **ui-specification.md** (components, interactions, visual states, e.g., `## Wine Table` or `### Sort States`)
- **requirements.md** (e.g., FR-001, NFR-101, NFR-201) — business intent and NFR thresholds being validated
- **solution-design.md** (e.g., SD-2.1, SD-3.3) — architectural decisions being tested
**Best Practices**: Every specification rule, walkthrough journey, and NFR threshold must have a corresponding test. Tests are written before code so implementation can be validated as it's built. If a spec rule, walkthrough journey, or NFR has no corresponding test, the Validation doc is incomplete.
- Define tests **before** writing implementation code
- Fail CI on specification violations
- Run validation jobs on every commit and PR
- Use automated code review tools
- Maintain validation job health dashboard
- Document validation failure remediation steps
--- **Test Types**
#### Specification-to-Test Execution Trace Each validation test must include explicit references to its source document:
| Test Type | Source | Example Reference |
|-----------|--------|-------------------|
| **Unit tests** | specification.md, ui-specification.md | `specification.md:3.2.1` rejects camelCase headers; `ui-specification.md:## Wine Table` sort states |
| **Integration tests** | 3-walkthrough.md, specification.md | Participant A section step 4.2 → endpoint chain in specification.md |
| **Performance tests** | requirements.md NFRs, solution-design.md | `NFR-101` latency threshold; `SD-002` WebSocket performance decision |
| **Penetration tests** | requirements.md security NFRs, specification.md auth endpoints | `NFR-301` authentication; `specification.md:auth/login` endpoint |
| **Contract tests** | specification.md sections | Mapped to endpoint definitions and schema constraints |
| **Linting rules** | ui-specification.md style guides | Enforcing design tokens from ui-specification.md |
| **Schema validation** | specification.md data schemas | Validating Arrow stream headers (snake_case), JSON payloads |
| **Visual regression** | ui-specification.md component states | Verifying hover, disabled, error states match spec |
| **Security scanning** | requirements.md, specification.md | SAST, dependency vulnerability checks, OWASP rules |
| **CI/CD pipeline** | All test types above | Automated execution on every commit and PR |
**Specification-to-Test Execution Trace**
Every specification rule must have a runnable, verifiable test. This trace maps each spec item to its corresponding test file, test function, and CI gate — proving that every requirement was tested, not just claimed. Every specification rule must have a runnable, verifiable test. This trace maps each spec item to its corresponding test file, test function, and CI gate — proving that every requirement was tested, not just claimed.
@@ -252,12 +244,12 @@ Every specification rule must have a runnable, verifiable test. This trace maps
| Requirement ID | Source Path | Unit Test File | Unit Test Function | Additional Test File(s) | CI Gate | Status | | Requirement ID | Source Path | Unit Test File | Unit Test Function | Additional Test File(s) | CI Gate | Status |
|----------------|-------------|----------------|-------------------|------------------------|---------|--------| |----------------|-------------|----------------|-------------------|------------------------|---------|--------|
| FR-001 | `specification.md:3.2.1` | `tests/unit/schema-validation.test.ts` | `test_rejects_camelCase_headers` | — | `ci/contract-validation.yaml` | Pass | | FR-001 | `specification.md#/components/schemas/Request` | `tests/unit/schema-validation.test.ts` | `test_rejects_camelCase_headers` | — | `ci/contract-validation.yaml` | Pass |
| FR-002 | `specification.md:4.1` | `tests/unit/api-routes.test.ts` | `test_post_wine_returns_201` | `tests/integration/wine-flow.test.ts` | `ci/contract-validation.yaml` | Pass | | FR-002 | `specification.md#/paths/~1api~1v1~1endpoint` | `tests/unit/api-routes.test.ts` | `test_post_wine_returns_201` | `tests/integration/wine-flow.test.ts` | `ci/contract-validation.yaml` | Pass |
| FR-003 | `specification.md:3.2.3` | — | — | `tests/integration/error-handling.test.ts` | `ci/contract-validation.yaml` | Pass | | FR-003 | `specification.md#/paths/~1api~1v1~1endpoint` | — | — | `tests/integration/error-handling.test.ts` | `ci/contract-validation.yaml` | Pass |
| NFR-101 | `requirements.md:4.1` | — | — | `tests/performance/load.test.ts` | `ci/performance.yaml` | Pass | | NFR-101 | `requirements.md:4.1` | — | — | `tests/performance/load.test.ts` | `ci/performance.yaml` | Pass |
| SD-001 | `solution-design.md:5.1` | — | — | `tests/integration/nats-async.test.ts` | `ci/integration.yaml` | Pass | | SD-001 | `solution-design.md:5.1` | — | — | `tests/integration/nats-async.test.ts` | `ci/integration.yaml` | Pass |
| UI-15.3.2 | `ui-specification.md:15.3.2` | `tests/unit/wine-table.test.tsx` | `test_sort_by_name` | `tests/visual/wine-table.spec.js` | `ci/visual-regression.yaml` | Pass | | UI-WineTable | `ui-specification.md## Wine Table` | `tests/unit/wine-table.test.tsx` | `test_sort_by_name` | `tests/visual/wine-table.spec.js` | `ci/visual-regression.yaml` | Pass |
**Additional Test File(s)** column covers: integration tests (participant section flows), performance tests (NFR benchmarks), penetration tests (security NFRs), and visual regression tests (UI component states). **Additional Test File(s)** column covers: integration tests (participant section flows), performance tests (NFR benchmarks), penetration tests (security NFRs), and visual regression tests (UI component states).
@@ -270,6 +262,14 @@ Every specification rule must have a runnable, verifiable test. This trace maps
- A test may cover multiple spec items, but each spec item must be covered by **at least one** test - A test may cover multiple spec items, but each spec item must be covered by **at least one** test
- Status is tracked via CI gate results (Pass/Fail/Skipped) — not manual checks - Status is tracked via CI gate results (Pass/Fail/Skipped) — not manual checks
**Best Practices**:
- Define tests **before** writing implementation code
- Fail CI on specification violations
- Run validation jobs on every commit and PR
- Use automated code review tools
- Maintain validation job health dashboard
- Document validation failure remediation steps
**How to Maintain**: **How to Maintain**:
1. When writing a new spec rule, add a trace row **before** implementation 1. When writing a new spec rule, add a trace row **before** implementation
2. When writing the test, fill in the test file and function 2. When writing the test, fill in the test file and function