update
This commit is contained in:
279
SDD_FRAMEWORK.md
Normal file
279
SDD_FRAMEWORK.md
Normal file
@@ -0,0 +1,279 @@
|
|||||||
|
# SDD + GitOps Documentation Stack
|
||||||
|
|
||||||
|
A comprehensive documentation strategy for modern software development that aligns different types of documentation with their specific purposes, audiences, and tooling.
|
||||||
|
|
||||||
|
## The Big Picture
|
||||||
|
|
||||||
|
This framework ensures that every piece of documentation serves a clear purpose and reaches the right audience. It emphasizes:
|
||||||
|
|
||||||
|
- **Machine-readable truths** as the foundation for automation
|
||||||
|
- **Separation of concerns** between human-facing docs and machine-consumable contracts
|
||||||
|
- **GitOps integration** where deployment and configuration are version-controlled
|
||||||
|
- **Multi-role audience targeting** from stakeholders to DevOps
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Documentation Matrix
|
||||||
|
|
||||||
|
| Document | Purpose ("The Why") | Primary Audience | Format / Tooling | Example (SaaS Context) |
|
||||||
|
|----------|---------------------|------------------|------------------|------------------------|
|
||||||
|
| **Requirements** | Define business goals & user needs | Stakeholders, PM, Lead Dev | GitHub Issues, Notion | "System must support 5-member teams with real-time sync." |
|
||||||
|
| **The Spec** | The Contract. Machine-readable truth. | Developers, QA, Machines | OpenAPI, Protobuf, YAML | A `.yaml` file defining `user_id` as a UUID in snake_case. |
|
||||||
|
| **Architecture** | High-level structural blueprint | Senior Devs, DevOps | Mermaid.js, IcePanel | Diagram of SvelteKit ↔ NATS ↔ Julia 6-node cluster. |
|
||||||
|
| **Walkthrough** | The Intuition. The "Big Picture" narrative. | New Devs, The Team | Recorded Video, TOUR.md | "Why we use a Claim-Check pattern for large Arrow data." |
|
||||||
|
| **Implementation** | The actual logic & generated code | Developers | SvelteKit, Julia, Node.js | Auto-generated TypeScript types from the OpenAPI spec. |
|
||||||
|
| **Validation** | Automated "Contract" enforcement | CI/CD Pipelines, QA | GitHub Actions, Prism | A test that fails if the Julia API returns camelCase keys. |
|
||||||
|
| **Runbook** | Deployment, Scaling, & Recovery | DevOps, SRE | K8s Manifests, Flux | `git push` to update the replica count from 3 to 6. |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Detailed Explanations
|
||||||
|
|
||||||
|
### 1. Requirements
|
||||||
|
|
||||||
|
**Purpose**: Define business goals & user needs.
|
||||||
|
|
||||||
|
**Why it matters**: Before writing code, we need to understand *why* we're building something. Requirements capture the business context, user pain points, and success criteria.
|
||||||
|
|
||||||
|
**Primary Audience**:
|
||||||
|
- **Stakeholders**: Business owners who need to approve the direction
|
||||||
|
- **Product Managers**: Translate requirements into features
|
||||||
|
- **Lead Developers**: Understand scope and technical constraints
|
||||||
|
|
||||||
|
**Format / Tooling**:
|
||||||
|
- **GitHub Issues**: Simple, version-controlled, integrated with code
|
||||||
|
- **Notion**: Rich text, collaborative, good for initial brainstorming
|
||||||
|
|
||||||
|
**Best Practices**:
|
||||||
|
- Write in user story format: "As a [role], I want [feature] so that [benefit]"
|
||||||
|
- Include acceptance criteria as checklist items
|
||||||
|
- Link to related specs and architecture decisions
|
||||||
|
|
||||||
|
**Example**: "System must support 5-member teams with real-time sync."
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 2. The Spec (The Contract)
|
||||||
|
|
||||||
|
**Purpose**: Machine-readable truth that defines the API contract.
|
||||||
|
|
||||||
|
**Why it matters**: The spec is the single source of truth for how systems communicate. It enables code generation, automated testing, and ensures consistency across services.
|
||||||
|
|
||||||
|
**Primary Audience**:
|
||||||
|
- **Developers**: Implement the API according to the spec
|
||||||
|
- **QA Engineers**: Create test cases based on the spec
|
||||||
|
- **Machines**: Used for code generation, validation, and documentation
|
||||||
|
|
||||||
|
**Format / Tooling**:
|
||||||
|
- **OpenAPI (Swagger)**: REST API specifications
|
||||||
|
- **Protobuf**: gRPC service definitions
|
||||||
|
- **YAML/JSON**: Configuration and data schema definitions
|
||||||
|
|
||||||
|
**Best Practices**:
|
||||||
|
- Use snake_case for consistency
|
||||||
|
- Define all fields with types and constraints
|
||||||
|
- Include examples for complex data structures
|
||||||
|
- Keep specs versioned alongside code
|
||||||
|
|
||||||
|
**Example**: A `.yaml` file defining `user_id` as a UUID in snake_case.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 3. Architecture
|
||||||
|
|
||||||
|
**Purpose**: High-level structural blueprint showing how components interact.
|
||||||
|
|
||||||
|
**Why it matters**: Architecture diagrams help everyone understand the system's structure without drowning in implementation details. They're crucial for onboarding, design reviews, and long-term maintainability.
|
||||||
|
|
||||||
|
**Primary Audience**:
|
||||||
|
- **Senior Developers**: Design decisions and component responsibilities
|
||||||
|
- **DevOps**: Understand deployment topology and service dependencies
|
||||||
|
- **Technical Leads**: Evaluate trade-offs and scalability concerns
|
||||||
|
|
||||||
|
**Format / Tooling**:
|
||||||
|
- **Mermaid.js**: Code-based diagrams that are version-controlled
|
||||||
|
- **IcePanel**: Interactive, automated architecture visualization
|
||||||
|
- **C4 Model**: Standardized approach to architectural diagrams
|
||||||
|
|
||||||
|
**Best Practices**:
|
||||||
|
- Focus on *relationships* between components, not implementation details
|
||||||
|
- Include technology choices (e.g., NATS vs WebSocket)
|
||||||
|
- Show data flow direction with arrows
|
||||||
|
- Update diagrams when architecture changes
|
||||||
|
|
||||||
|
**Example**: Diagram of SvelteKit ↔ NATS ↔ Julia 6-node cluster.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 4. Walkthrough
|
||||||
|
|
||||||
|
**Purpose**: The intuition and "Big Picture" narrative.
|
||||||
|
|
||||||
|
**Why it matters**: Code alone doesn't explain *why* decisions were made. Walkthroughs provide context, historical decisions, and architectural intuition that helps new developers become productive quickly.
|
||||||
|
|
||||||
|
**Primary Audience**:
|
||||||
|
- **New Developers**: Understand the system's philosophy and patterns
|
||||||
|
- **The Team**: Share context and reasoning behind design choices
|
||||||
|
- **Code Reviewers**: Evaluate design decisions alongside implementation
|
||||||
|
|
||||||
|
**Format / Tooling**:
|
||||||
|
- **Recorded Video**: Personal, engaging, good for complex explanations
|
||||||
|
- **TOUR.md**: Markdown file with narrative walk-through of the codebase
|
||||||
|
- **Architecture Decision Records (ADRs)**: Formal documentation of key decisions
|
||||||
|
|
||||||
|
**Best Practices**:
|
||||||
|
- Explain *why* more than *how*
|
||||||
|
- Include anti-patterns to avoid
|
||||||
|
- Link to related documentation
|
||||||
|
- Keep walkthroughs updated with architecture changes
|
||||||
|
|
||||||
|
**Example**: "Why we use a Claim-Check pattern for large Arrow data."
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 5. Implementation
|
||||||
|
|
||||||
|
**Purpose**: The actual logic and generated code.
|
||||||
|
|
||||||
|
**Why it matters**: This is the executable truth of the system. Well-structured implementation code should be clear, maintainable, and follow established patterns.
|
||||||
|
|
||||||
|
**Primary Audience**:
|
||||||
|
- **Developers**: Read, modify, and extend the code
|
||||||
|
- **Reviewers**: Verify correctness and adherence to standards
|
||||||
|
- **CI/CD**: Run tests and builds
|
||||||
|
|
||||||
|
**Format / Tooling**:
|
||||||
|
- **SvelteKit**: Frontend framework with server-side rendering
|
||||||
|
- **Julia**: High-performance numerical computing
|
||||||
|
- **Node.js**: Backend services and tooling
|
||||||
|
|
||||||
|
**Best Practices**:
|
||||||
|
- Generate code from specs to ensure consistency
|
||||||
|
- Use consistent naming conventions (snake_case, camelCase appropriately)
|
||||||
|
- Include unit tests alongside implementation
|
||||||
|
- Document complex algorithms with inline comments
|
||||||
|
|
||||||
|
**Example**: Auto-generated TypeScript types from the OpenAPI spec.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 6. Validation
|
||||||
|
|
||||||
|
**Purpose**: Automated "Contract" enforcement.
|
||||||
|
|
||||||
|
**Why it matters**: Automated tests ensure that the system behaves as specified and prevent regressions. Validation in CI/CD pipelines catches issues before they reach production.
|
||||||
|
|
||||||
|
**Primary Audience**:
|
||||||
|
- **CI/CD Pipelines**: Run tests automatically on every commit
|
||||||
|
- **QA Engineers**: Verify system behavior against requirements
|
||||||
|
- **Developers**: Get immediate feedback on changes
|
||||||
|
|
||||||
|
**Format / Tooling**:
|
||||||
|
- **GitHub Actions**: Automated testing and validation workflows
|
||||||
|
- **Prism (ReadMe)**: OpenAPI spec validation in CI
|
||||||
|
- **Jest/Vitest**: JavaScript testing framework
|
||||||
|
- **Pytest**: Python testing framework
|
||||||
|
|
||||||
|
**Best Practices**:
|
||||||
|
- Test the contract (spec) not just implementation details
|
||||||
|
- Use contract testing (PACT) for service-to-service validation
|
||||||
|
- Fail fast: tests should run quickly and provide clear error messages
|
||||||
|
- Include negative test cases (invalid inputs, edge cases)
|
||||||
|
|
||||||
|
**Example**: A test that fails if the Julia API returns camelCase keys.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 7. Runbook
|
||||||
|
|
||||||
|
**Purpose**: Deployment, scaling, and recovery procedures.
|
||||||
|
|
||||||
|
**Why it matters**: Runbooks ensure that deployments are consistent, repeatable, and recoverable. In GitOps, the runbook *is* the configuration, version-controlled alongside the code.
|
||||||
|
|
||||||
|
**Primary Audience**:
|
||||||
|
- **DevOps Engineers**: Execute deployments and scaling operations
|
||||||
|
- **SREs**: Manage system reliability and incident response
|
||||||
|
- **Developers**: Deploy feature branches for testing
|
||||||
|
|
||||||
|
**Format / Tooling**:
|
||||||
|
- **Kubernetes Manifests**: Declarative deployment configurations
|
||||||
|
- **Flux**: GitOps operator for Kubernetes
|
||||||
|
- **Helm Charts**: Package management for Kubernetes
|
||||||
|
- **Docker Compose**: Local development environments
|
||||||
|
|
||||||
|
**Best Practices**:
|
||||||
|
- Use Git as the source of truth (GitOps)
|
||||||
|
- Make deployments idempotent (running twice has same effect)
|
||||||
|
- Include rollback procedures
|
||||||
|
- Document scaling procedures for different load levels
|
||||||
|
|
||||||
|
**Example**: `git push` to update the replica count from 3 to 6.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## How the Stack Fits Together
|
||||||
|
|
||||||
|
```
|
||||||
|
┌─────────────────────────────────────────────────────────────┐
|
||||||
|
│ Requirements │
|
||||||
|
│ (Business goals, user needs) │
|
||||||
|
└───────────────────┬─────────────────────────────────────────┘
|
||||||
|
│
|
||||||
|
▼
|
||||||
|
┌─────────────────────────────────────────────────────────────┐
|
||||||
|
│ The Spec │
|
||||||
|
│ (Machine-readable contract: OpenAPI, Protobuf) │
|
||||||
|
└───────────────────┬─────────────────────────────────────────┘
|
||||||
|
│
|
||||||
|
▼
|
||||||
|
┌─────────────────────────────────────────────────────────────┐
|
||||||
|
│ Architecture │
|
||||||
|
│ (Structural blueprint: Mermaid, IcePanel) │
|
||||||
|
└───────────────────┬─────────────────────────────────────────┘
|
||||||
|
│
|
||||||
|
▼
|
||||||
|
┌─────────────────────────────────────────────────────────────┐
|
||||||
|
│ Walkthrough │
|
||||||
|
│ (Intuition, big picture narrative) │
|
||||||
|
└───────────────────┬─────────────────────────────────────────┘
|
||||||
|
│
|
||||||
|
▼
|
||||||
|
┌─────────────────────────────────────────────────────────────┐
|
||||||
|
│ Implementation │
|
||||||
|
│ (Actual code: SvelteKit, Julia, Node.js) │
|
||||||
|
└───────────────────┬─────────────────────────────────────────┘
|
||||||
|
│
|
||||||
|
▼
|
||||||
|
┌─────────────────────────────────────────────────────────────┐
|
||||||
|
│ Validation │
|
||||||
|
│ (Automated tests: GitHub Actions, Prism) │
|
||||||
|
└───────────────────┬─────────────────────────────────────────┘
|
||||||
|
│
|
||||||
|
▼
|
||||||
|
┌─────────────────────────────────────────────────────────────┐
|
||||||
|
│ Runbook │
|
||||||
|
│ (Deployment, scaling: K8s, Flux) │
|
||||||
|
└─────────────────────────────────────────────────────────────┘
|
||||||
|
```
|
||||||
|
|
||||||
|
## Key Principles
|
||||||
|
|
||||||
|
1. **Machine-Readable Truth**: Specs and configurations should be machine-readable to enable automation
|
||||||
|
2. **Separation of Concerns**: Different audiences need different types of information
|
||||||
|
3. **Version Control**: All documentation should be in Git, just like code
|
||||||
|
4. **Automation-First**: Validation should be automated and integrated into CI/CD
|
||||||
|
5. **Living Documentation**: Documentation should evolve with the codebase
|
||||||
|
|
||||||
|
## Getting Started
|
||||||
|
|
||||||
|
To adopt this stack in your project:
|
||||||
|
|
||||||
|
1. Start with requirements in GitHub Issues or Notion
|
||||||
|
2. Create a spec file (OpenAPI/Protobuf) as the contract
|
||||||
|
3. Add architecture diagrams using Mermaid.js
|
||||||
|
4. Write a walkthrough explaining the "why" behind decisions
|
||||||
|
5. Implement code following the spec
|
||||||
|
6. Add automated tests that validate the spec
|
||||||
|
7. Create runbooks for deployment and scaling
|
||||||
|
|
||||||
|
This framework ensures that every piece of documentation serves a clear purpose and reaches the right audience.
|
||||||
@@ -1,152 +0,0 @@
|
|||||||
# Software Design Document (SDD) Framework
|
|
||||||
|
|
||||||
A structured taxonomy of artifacts for comprehensive software documentation and design.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Overview
|
|
||||||
|
|
||||||
This framework categorizes Software Design Document artifacts by their purpose in the software development lifecycle. Each artifact type serves a specific role in defining **what** we build, **how** it works, and **how** we verify and document it.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Artifact Taxonomy
|
|
||||||
|
|
||||||
| Document Type | "Artifact" Type | Purpose in SDD | Tooling Examples |
|
|
||||||
|---------------|-----------------|----------------|------------------|
|
|
||||||
| Requirements | User Stories / ADRs | Defines the "Business Contract" — why are we building this? | GitHub Issues, Jira, Architecture Decision Records (ADRs) |
|
|
||||||
| The Spec | Schema Definition | The Source of Truth — defines data structures, types, and endpoints | OpenAPI (YAML), Protobuf, Apache Arrow Schemas |
|
|
||||||
| Architecture | System Blueprint | Defines how the specs connect (e.g., Service A talks to B via NATS) | Mermaid.js (Diagrams-as-code), IcePanel, Structurizr |
|
|
||||||
| Implementation | Generated Code | The code "fills in" the logic defined by the Spec | OpenAPI Generator, TypeSafe clients (Zod, TypeScript) |
|
|
||||||
| Validation | Contract Tests | Automatically checks if the code matches the Spec | Prism (Mocking), Dredd, Schemathesis |
|
|
||||||
| Tutorial | Interactive Sandbox | Allows others to "play" with the spec without writing code | Swagger UI, Redoc, Postman Collections |
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Purpose by Artifact Type
|
|
||||||
|
|
||||||
### 1. Requirements (User Stories / ADRs)
|
|
||||||
|
|
||||||
**Purpose:** Defines the "Business Contract" — why are we building this?
|
|
||||||
|
|
||||||
**Key Questions Answered:**
|
|
||||||
- What problem are we solving?
|
|
||||||
- Who are the stakeholders?
|
|
||||||
- What are the success criteria?
|
|
||||||
|
|
||||||
**Examples:**
|
|
||||||
- GitHub Issues
|
|
||||||
- Jira tickets
|
|
||||||
- Architecture Decision Records (ADRs)
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### 2. The Spec (Schema Definition)
|
|
||||||
|
|
||||||
**Purpose:** The Source of Truth — defines data structures, types, and endpoints.
|
|
||||||
|
|
||||||
**Key Questions Answered:**
|
|
||||||
- What data structures do we use?
|
|
||||||
- What are the API endpoints?
|
|
||||||
- What are the message formats?
|
|
||||||
|
|
||||||
**Examples:**
|
|
||||||
- OpenAPI (YAML) specifications
|
|
||||||
- Protocol Buffers (Protobuf)
|
|
||||||
- Apache Arrow Schemas
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### 3. Architecture (System Blueprint)
|
|
||||||
|
|
||||||
**Purpose:** Defines how the specs connect — the high-level structure and communication patterns.
|
|
||||||
|
|
||||||
**Key Questions Answered:**
|
|
||||||
- How do services communicate?
|
|
||||||
- What is the deployment topology?
|
|
||||||
- What are the system boundaries?
|
|
||||||
|
|
||||||
**Examples:**
|
|
||||||
- Mermaid.js diagrams (Diagrams-as-code)
|
|
||||||
- IcePanel
|
|
||||||
- Structurizr
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### 4. Implementation (Generated Code)
|
|
||||||
|
|
||||||
**Purpose:** The code "fills in" the logic defined by the Spec.
|
|
||||||
|
|
||||||
**Key Questions Answered:**
|
|
||||||
- How do we implement the spec?
|
|
||||||
- What libraries/frameworks do we use?
|
|
||||||
- How do we ensure type safety?
|
|
||||||
|
|
||||||
**Examples:**
|
|
||||||
- OpenAPI Generator
|
|
||||||
- TypeSafe clients (Zod, TypeScript)
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### 5. Validation (Contract Tests)
|
|
||||||
|
|
||||||
**Purpose:** Automatically checks if the code matches the Spec.
|
|
||||||
|
|
||||||
**Key Questions Answered:**
|
|
||||||
- Does the implementation match the spec?
|
|
||||||
- Are there breaking changes?
|
|
||||||
- Is the contract upheld?
|
|
||||||
|
|
||||||
**Examples:**
|
|
||||||
- Prism (Mocking)
|
|
||||||
- Dredd
|
|
||||||
- Schemathesis
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### 6. Tutorial (Interactive Sandbox)
|
|
||||||
|
|
||||||
**Purpose:** Allows others to "play" with the spec without writing code.
|
|
||||||
|
|
||||||
**Key Questions Answered:**
|
|
||||||
- How do I use this API?
|
|
||||||
- What are the expected inputs/outputs?
|
|
||||||
- How can I experiment safely?
|
|
||||||
|
|
||||||
**Examples:**
|
|
||||||
- Swagger UI
|
|
||||||
- Redoc
|
|
||||||
- Postman Collections
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Usage Guidelines
|
|
||||||
|
|
||||||
1. **Start with Requirements** — Define the business context and decision records
|
|
||||||
2. **Define The Spec** — Create the source of truth for data and endpoints
|
|
||||||
3. **Design Architecture** — Visualize how components connect
|
|
||||||
4. **Implement** — Generate or write code based on the spec
|
|
||||||
5. **Validate** — Run contract tests to ensure alignment
|
|
||||||
6. **Tutorial** — Provide interactive documentation for users
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Relationship to NATSBridge
|
|
||||||
|
|
||||||
This NATSBridge project implements a NATS-based messaging bridge that can be used with all artifact types in this framework:
|
|
||||||
|
|
||||||
- **Architecture:** NATS as the communication backbone
|
|
||||||
- **Spec:** Message schemas for bridge operations
|
|
||||||
- **Implementation:** Generated code for NATS message handling
|
|
||||||
- **Validation:** Contract tests for message format compliance
|
|
||||||
- **Tutorial:** Interactive examples for bridge configuration
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## References
|
|
||||||
|
|
||||||
- Architecture Decision Records (ADRs): [http://adr.cascadely.com/](http://adr.cascadely.com/)
|
|
||||||
- OpenAPI Specification: [https://spec.openapis.org/](https://spec.openapis.org/)
|
|
||||||
- Protocol Buffers: [https://protobuf.dev/](https://protobuf.dev/)
|
|
||||||
- Mermaid.js: [https://mermaid.js.org/](https://mermaid.js.org/)
|
|
||||||
- Schemathesis: [https://schemathesis.io/](https://schemathesis.io/)
|
|
||||||
Reference in New Issue
Block a user