update
This commit is contained in:
279
docs/SDD_FRAMEWORK.md
Normal file
279
docs/SDD_FRAMEWORK.md
Normal file
@@ -0,0 +1,279 @@
|
||||
# SDD + GitOps Documentation Stack
|
||||
|
||||
A comprehensive documentation strategy for modern software development that aligns different types of documentation with their specific purposes, audiences, and tooling.
|
||||
|
||||
## The Big Picture
|
||||
|
||||
This framework ensures that every piece of documentation serves a clear purpose and reaches the right audience. It emphasizes:
|
||||
|
||||
- **Machine-readable truths** as the foundation for automation
|
||||
- **Separation of concerns** between human-facing docs and machine-consumable contracts
|
||||
- **GitOps integration** where deployment and configuration are version-controlled
|
||||
- **Multi-role audience targeting** from stakeholders to DevOps
|
||||
|
||||
---
|
||||
|
||||
## Documentation Matrix
|
||||
|
||||
| Document | Purpose ("The Why") | Primary Audience | Format / Tooling | Example (SaaS Context) |
|
||||
|----------|---------------------|------------------|------------------|------------------------|
|
||||
| **Requirements** | Define business goals & user needs | Stakeholders, PM, Lead Dev | GitHub Issues, Notion | "System must support 5-member teams with real-time sync." |
|
||||
| **The Spec** | The Contract. Machine-readable truth. | Developers, QA, Machines | OpenAPI, Protobuf, YAML | A `.yaml` file defining `user_id` as a UUID in snake_case. |
|
||||
| **Architecture** | High-level structural blueprint | Senior Devs, DevOps | Mermaid.js, IcePanel | Diagram of SvelteKit ↔ NATS ↔ Julia 6-node cluster. |
|
||||
| **Walkthrough** | The Intuition. The "Big Picture" narrative. | New Devs, The Team | Recorded Video, TOUR.md | "Why we use a Claim-Check pattern for large Arrow data." |
|
||||
| **Implementation** | The actual logic & generated code | Developers | SvelteKit, Julia, Node.js | Auto-generated TypeScript types from the OpenAPI spec. |
|
||||
| **Validation** | Automated "Contract" enforcement | CI/CD Pipelines, QA | GitHub Actions, Prism | A test that fails if the Julia API returns camelCase keys. |
|
||||
| **Runbook** | Deployment, Scaling, & Recovery | DevOps, SRE | K8s Manifests, Flux | `git push` to update the replica count from 3 to 6. |
|
||||
|
||||
---
|
||||
|
||||
## Detailed Explanations
|
||||
|
||||
### 1. Requirements
|
||||
|
||||
**Purpose**: Define business goals & user needs.
|
||||
|
||||
**Why it matters**: Before writing code, we need to understand *why* we're building something. Requirements capture the business context, user pain points, and success criteria.
|
||||
|
||||
**Primary Audience**:
|
||||
- **Stakeholders**: Business owners who need to approve the direction
|
||||
- **Product Managers**: Translate requirements into features
|
||||
- **Lead Developers**: Understand scope and technical constraints
|
||||
|
||||
**Format / Tooling**:
|
||||
- **GitHub Issues**: Simple, version-controlled, integrated with code
|
||||
- **Notion**: Rich text, collaborative, good for initial brainstorming
|
||||
|
||||
**Best Practices**:
|
||||
- Write in user story format: "As a [role], I want [feature] so that [benefit]"
|
||||
- Include acceptance criteria as checklist items
|
||||
- Link to related specs and architecture decisions
|
||||
|
||||
**Example**: "System must support 5-member teams with real-time sync."
|
||||
|
||||
---
|
||||
|
||||
### 2. The Spec (The Contract)
|
||||
|
||||
**Purpose**: Machine-readable truth that defines the API contract.
|
||||
|
||||
**Why it matters**: The spec is the single source of truth for how systems communicate. It enables code generation, automated testing, and ensures consistency across services.
|
||||
|
||||
**Primary Audience**:
|
||||
- **Developers**: Implement the API according to the spec
|
||||
- **QA Engineers**: Create test cases based on the spec
|
||||
- **Machines**: Used for code generation, validation, and documentation
|
||||
|
||||
**Format / Tooling**:
|
||||
- **OpenAPI (Swagger)**: REST API specifications
|
||||
- **Protobuf**: gRPC service definitions
|
||||
- **YAML/JSON**: Configuration and data schema definitions
|
||||
|
||||
**Best Practices**:
|
||||
- Use snake_case for consistency
|
||||
- Define all fields with types and constraints
|
||||
- Include examples for complex data structures
|
||||
- Keep specs versioned alongside code
|
||||
|
||||
**Example**: A `.yaml` file defining `user_id` as a UUID in snake_case.
|
||||
|
||||
---
|
||||
|
||||
### 3. Architecture
|
||||
|
||||
**Purpose**: High-level structural blueprint showing how components interact.
|
||||
|
||||
**Why it matters**: Architecture diagrams help everyone understand the system's structure without drowning in implementation details. They're crucial for onboarding, design reviews, and long-term maintainability.
|
||||
|
||||
**Primary Audience**:
|
||||
- **Senior Developers**: Design decisions and component responsibilities
|
||||
- **DevOps**: Understand deployment topology and service dependencies
|
||||
- **Technical Leads**: Evaluate trade-offs and scalability concerns
|
||||
|
||||
**Format / Tooling**:
|
||||
- **Mermaid.js**: Code-based diagrams that are version-controlled
|
||||
- **IcePanel**: Interactive, automated architecture visualization
|
||||
- **C4 Model**: Standardized approach to architectural diagrams
|
||||
|
||||
**Best Practices**:
|
||||
- Focus on *relationships* between components, not implementation details
|
||||
- Include technology choices (e.g., NATS vs WebSocket)
|
||||
- Show data flow direction with arrows
|
||||
- Update diagrams when architecture changes
|
||||
|
||||
**Example**: Diagram of SvelteKit ↔ NATS ↔ Julia 6-node cluster.
|
||||
|
||||
---
|
||||
|
||||
### 4. Walkthrough
|
||||
|
||||
**Purpose**: The intuition and "Big Picture" narrative.
|
||||
|
||||
**Why it matters**: Code alone doesn't explain *why* decisions were made. Walkthroughs provide context, historical decisions, and architectural intuition that helps new developers become productive quickly.
|
||||
|
||||
**Primary Audience**:
|
||||
- **New Developers**: Understand the system's philosophy and patterns
|
||||
- **The Team**: Share context and reasoning behind design choices
|
||||
- **Code Reviewers**: Evaluate design decisions alongside implementation
|
||||
|
||||
**Format / Tooling**:
|
||||
- **Recorded Video**: Personal, engaging, good for complex explanations
|
||||
- **TOUR.md**: Markdown file with narrative walk-through of the codebase
|
||||
- **Architecture Decision Records (ADRs)**: Formal documentation of key decisions
|
||||
|
||||
**Best Practices**:
|
||||
- Explain *why* more than *how*
|
||||
- Include anti-patterns to avoid
|
||||
- Link to related documentation
|
||||
- Keep walkthroughs updated with architecture changes
|
||||
|
||||
**Example**: "Why we use a Claim-Check pattern for large Arrow data."
|
||||
|
||||
---
|
||||
|
||||
### 5. Implementation
|
||||
|
||||
**Purpose**: The actual logic and generated code.
|
||||
|
||||
**Why it matters**: This is the executable truth of the system. Well-structured implementation code should be clear, maintainable, and follow established patterns.
|
||||
|
||||
**Primary Audience**:
|
||||
- **Developers**: Read, modify, and extend the code
|
||||
- **Reviewers**: Verify correctness and adherence to standards
|
||||
- **CI/CD**: Run tests and builds
|
||||
|
||||
**Format / Tooling**:
|
||||
- **SvelteKit**: Frontend framework with server-side rendering
|
||||
- **Julia**: High-performance numerical computing
|
||||
- **Node.js**: Backend services and tooling
|
||||
|
||||
**Best Practices**:
|
||||
- Generate code from specs to ensure consistency
|
||||
- Use consistent naming conventions (snake_case, camelCase appropriately)
|
||||
- Include unit tests alongside implementation
|
||||
- Document complex algorithms with inline comments
|
||||
|
||||
**Example**: Auto-generated TypeScript types from the OpenAPI spec.
|
||||
|
||||
---
|
||||
|
||||
### 6. Validation
|
||||
|
||||
**Purpose**: Automated "Contract" enforcement.
|
||||
|
||||
**Why it matters**: Automated tests ensure that the system behaves as specified and prevent regressions. Validation in CI/CD pipelines catches issues before they reach production.
|
||||
|
||||
**Primary Audience**:
|
||||
- **CI/CD Pipelines**: Run tests automatically on every commit
|
||||
- **QA Engineers**: Verify system behavior against requirements
|
||||
- **Developers**: Get immediate feedback on changes
|
||||
|
||||
**Format / Tooling**:
|
||||
- **GitHub Actions**: Automated testing and validation workflows
|
||||
- **Prism (ReadMe)**: OpenAPI spec validation in CI
|
||||
- **Jest/Vitest**: JavaScript testing framework
|
||||
- **Pytest**: Python testing framework
|
||||
|
||||
**Best Practices**:
|
||||
- Test the contract (spec) not just implementation details
|
||||
- Use contract testing (PACT) for service-to-service validation
|
||||
- Fail fast: tests should run quickly and provide clear error messages
|
||||
- Include negative test cases (invalid inputs, edge cases)
|
||||
|
||||
**Example**: A test that fails if the Julia API returns camelCase keys.
|
||||
|
||||
---
|
||||
|
||||
### 7. Runbook
|
||||
|
||||
**Purpose**: Deployment, scaling, and recovery procedures.
|
||||
|
||||
**Why it matters**: Runbooks ensure that deployments are consistent, repeatable, and recoverable. In GitOps, the runbook *is* the configuration, version-controlled alongside the code.
|
||||
|
||||
**Primary Audience**:
|
||||
- **DevOps Engineers**: Execute deployments and scaling operations
|
||||
- **SREs**: Manage system reliability and incident response
|
||||
- **Developers**: Deploy feature branches for testing
|
||||
|
||||
**Format / Tooling**:
|
||||
- **Kubernetes Manifests**: Declarative deployment configurations
|
||||
- **Flux**: GitOps operator for Kubernetes
|
||||
- **Helm Charts**: Package management for Kubernetes
|
||||
- **Docker Compose**: Local development environments
|
||||
|
||||
**Best Practices**:
|
||||
- Use Git as the source of truth (GitOps)
|
||||
- Make deployments idempotent (running twice has same effect)
|
||||
- Include rollback procedures
|
||||
- Document scaling procedures for different load levels
|
||||
|
||||
**Example**: `git push` to update the replica count from 3 to 6.
|
||||
|
||||
---
|
||||
|
||||
## How the Stack Fits Together
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ Requirements │
|
||||
│ (Business goals, user needs) │
|
||||
└───────────────────┬─────────────────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ The Spec │
|
||||
│ (Machine-readable contract: OpenAPI, Protobuf) │
|
||||
└───────────────────┬─────────────────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ Architecture │
|
||||
│ (Structural blueprint: Mermaid, IcePanel) │
|
||||
└───────────────────┬─────────────────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ Walkthrough │
|
||||
│ (Intuition, big picture narrative) │
|
||||
└───────────────────┬─────────────────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ Implementation │
|
||||
│ (Actual code: SvelteKit, Julia, Node.js) │
|
||||
└───────────────────┬─────────────────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ Validation │
|
||||
│ (Automated tests: GitHub Actions, Prism) │
|
||||
└───────────────────┬─────────────────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ Runbook │
|
||||
│ (Deployment, scaling: K8s, Flux) │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## Key Principles
|
||||
|
||||
1. **Machine-Readable Truth**: Specs and configurations should be machine-readable to enable automation
|
||||
2. **Separation of Concerns**: Different audiences need different types of information
|
||||
3. **Version Control**: All documentation should be in Git, just like code
|
||||
4. **Automation-First**: Validation should be automated and integrated into CI/CD
|
||||
5. **Living Documentation**: Documentation should evolve with the codebase
|
||||
|
||||
## Getting Started
|
||||
|
||||
To adopt this stack in your project:
|
||||
|
||||
1. Start with requirements in GitHub Issues or Notion
|
||||
2. Create a spec file (OpenAPI/Protobuf) as the contract
|
||||
3. Add architecture diagrams using Mermaid.js
|
||||
4. Write a walkthrough explaining the "why" behind decisions
|
||||
5. Implement code following the spec
|
||||
6. Add automated tests that validate the spec
|
||||
7. Create runbooks for deployment and scaling
|
||||
|
||||
This framework ensures that every piece of documentation serves a clear purpose and reaches the right audience.
|
||||
Reference in New Issue
Block a user