279 lines
14 KiB
Markdown
279 lines
14 KiB
Markdown
# SDD + GitOps Documentation Stack
|
|
|
|
A comprehensive documentation strategy for modern software development that aligns different types of documentation with their specific purposes, audiences, and tooling.
|
|
|
|
## The Big Picture
|
|
|
|
This framework ensures that every piece of documentation serves a clear purpose and reaches the right audience. It emphasizes:
|
|
|
|
- **Machine-readable truths** as the foundation for automation
|
|
- **Separation of concerns** between human-facing docs and machine-consumable contracts
|
|
- **GitOps integration** where deployment and configuration are version-controlled
|
|
- **Multi-role audience targeting** from stakeholders to DevOps
|
|
|
|
---
|
|
|
|
## Documentation Matrix
|
|
|
|
| Document | Purpose ("The Why") | Primary Audience | Format / Tooling | Example (SaaS Context) |
|
|
|----------|---------------------|------------------|------------------|------------------------|
|
|
| **Requirements** | Define business goals & user needs | Stakeholders, PM, Lead Dev | GitHub Issues, Notion | "System must support 5-member teams with real-time sync." |
|
|
| **The Spec** | The Contract. Machine-readable truth. | Developers, QA, Machines | OpenAPI, Protobuf, YAML | A `.yaml` file defining `user_id` as a UUID in snake_case. |
|
|
| **Architecture** | High-level structural blueprint | Senior Devs, DevOps | Mermaid.js, IcePanel | Diagram of SvelteKit ↔ NATS ↔ Julia 6-node cluster. |
|
|
| **Walkthrough** | The Intuition. The "Big Picture" narrative. | New Devs, The Team | Recorded Video, TOUR.md | "Why we use a Claim-Check pattern for large Arrow data." |
|
|
| **Implementation** | The actual logic & generated code | Developers | SvelteKit, Julia, Node.js | Auto-generated TypeScript types from the OpenAPI spec. |
|
|
| **Validation** | Automated "Contract" enforcement | CI/CD Pipelines, QA | GitHub Actions, Prism | A test that fails if the Julia API returns camelCase keys. |
|
|
| **Runbook** | Deployment, Scaling, & Recovery | DevOps, SRE | K8s Manifests, Flux | `git push` to update the replica count from 3 to 6. |
|
|
|
|
---
|
|
|
|
## Detailed Explanations
|
|
|
|
### 1. Requirements
|
|
|
|
**Purpose**: Define business goals & user needs.
|
|
|
|
**Why it matters**: Before writing code, we need to understand *why* we're building something. Requirements capture the business context, user pain points, and success criteria.
|
|
|
|
**Primary Audience**:
|
|
- **Stakeholders**: Business owners who need to approve the direction
|
|
- **Product Managers**: Translate requirements into features
|
|
- **Lead Developers**: Understand scope and technical constraints
|
|
|
|
**Format / Tooling**:
|
|
- **GitHub Issues**: Simple, version-controlled, integrated with code
|
|
- **Notion**: Rich text, collaborative, good for initial brainstorming
|
|
|
|
**Best Practices**:
|
|
- Write in user story format: "As a [role], I want [feature] so that [benefit]"
|
|
- Include acceptance criteria as checklist items
|
|
- Link to related specs and architecture decisions
|
|
|
|
**Example**: "System must support 5-member teams with real-time sync."
|
|
|
|
---
|
|
|
|
### 2. The Spec (The Contract)
|
|
|
|
**Purpose**: Machine-readable truth that defines the API contract.
|
|
|
|
**Why it matters**: The spec is the single source of truth for how systems communicate. It enables code generation, automated testing, and ensures consistency across services.
|
|
|
|
**Primary Audience**:
|
|
- **Developers**: Implement the API according to the spec
|
|
- **QA Engineers**: Create test cases based on the spec
|
|
- **Machines**: Used for code generation, validation, and documentation
|
|
|
|
**Format / Tooling**:
|
|
- **OpenAPI (Swagger)**: REST API specifications
|
|
- **Protobuf**: gRPC service definitions
|
|
- **YAML/JSON**: Configuration and data schema definitions
|
|
|
|
**Best Practices**:
|
|
- Use snake_case for consistency
|
|
- Define all fields with types and constraints
|
|
- Include examples for complex data structures
|
|
- Keep specs versioned alongside code
|
|
|
|
**Example**: A `.yaml` file defining `user_id` as a UUID in snake_case.
|
|
|
|
---
|
|
|
|
### 3. Architecture
|
|
|
|
**Purpose**: High-level structural blueprint showing how components interact.
|
|
|
|
**Why it matters**: Architecture diagrams help everyone understand the system's structure without drowning in implementation details. They're crucial for onboarding, design reviews, and long-term maintainability.
|
|
|
|
**Primary Audience**:
|
|
- **Senior Developers**: Design decisions and component responsibilities
|
|
- **DevOps**: Understand deployment topology and service dependencies
|
|
- **Technical Leads**: Evaluate trade-offs and scalability concerns
|
|
|
|
**Format / Tooling**:
|
|
- **Mermaid.js**: Code-based diagrams that are version-controlled
|
|
- **IcePanel**: Interactive, automated architecture visualization
|
|
- **C4 Model**: Standardized approach to architectural diagrams
|
|
|
|
**Best Practices**:
|
|
- Focus on *relationships* between components, not implementation details
|
|
- Include technology choices (e.g., NATS vs WebSocket)
|
|
- Show data flow direction with arrows
|
|
- Update diagrams when architecture changes
|
|
|
|
**Example**: Diagram of SvelteKit ↔ NATS ↔ Julia 6-node cluster.
|
|
|
|
---
|
|
|
|
### 4. Walkthrough
|
|
|
|
**Purpose**: The intuition and "Big Picture" narrative.
|
|
|
|
**Why it matters**: Code alone doesn't explain *why* decisions were made. Walkthroughs provide context, historical decisions, and architectural intuition that helps new developers become productive quickly.
|
|
|
|
**Primary Audience**:
|
|
- **New Developers**: Understand the system's philosophy and patterns
|
|
- **The Team**: Share context and reasoning behind design choices
|
|
- **Code Reviewers**: Evaluate design decisions alongside implementation
|
|
|
|
**Format / Tooling**:
|
|
- **Recorded Video**: Personal, engaging, good for complex explanations
|
|
- **TOUR.md**: Markdown file with narrative walk-through of the codebase
|
|
- **Architecture Decision Records (ADRs)**: Formal documentation of key decisions
|
|
|
|
**Best Practices**:
|
|
- Explain *why* more than *how*
|
|
- Include anti-patterns to avoid
|
|
- Link to related documentation
|
|
- Keep walkthroughs updated with architecture changes
|
|
|
|
**Example**: "Why we use a Claim-Check pattern for large Arrow data."
|
|
|
|
---
|
|
|
|
### 5. Implementation
|
|
|
|
**Purpose**: The actual logic and generated code.
|
|
|
|
**Why it matters**: This is the executable truth of the system. Well-structured implementation code should be clear, maintainable, and follow established patterns.
|
|
|
|
**Primary Audience**:
|
|
- **Developers**: Read, modify, and extend the code
|
|
- **Reviewers**: Verify correctness and adherence to standards
|
|
- **CI/CD**: Run tests and builds
|
|
|
|
**Format / Tooling**:
|
|
- **SvelteKit**: Frontend framework with server-side rendering
|
|
- **Julia**: High-performance numerical computing
|
|
- **Node.js**: Backend services and tooling
|
|
|
|
**Best Practices**:
|
|
- Generate code from specs to ensure consistency
|
|
- Use consistent naming conventions (snake_case, camelCase appropriately)
|
|
- Include unit tests alongside implementation
|
|
- Document complex algorithms with inline comments
|
|
|
|
**Example**: Auto-generated TypeScript types from the OpenAPI spec.
|
|
|
|
---
|
|
|
|
### 6. Validation
|
|
|
|
**Purpose**: Automated "Contract" enforcement.
|
|
|
|
**Why it matters**: Automated tests ensure that the system behaves as specified and prevent regressions. Validation in CI/CD pipelines catches issues before they reach production.
|
|
|
|
**Primary Audience**:
|
|
- **CI/CD Pipelines**: Run tests automatically on every commit
|
|
- **QA Engineers**: Verify system behavior against requirements
|
|
- **Developers**: Get immediate feedback on changes
|
|
|
|
**Format / Tooling**:
|
|
- **GitHub Actions**: Automated testing and validation workflows
|
|
- **Prism (ReadMe)**: OpenAPI spec validation in CI
|
|
- **Jest/Vitest**: JavaScript testing framework
|
|
- **Pytest**: Python testing framework
|
|
|
|
**Best Practices**:
|
|
- Test the contract (spec) not just implementation details
|
|
- Use contract testing (PACT) for service-to-service validation
|
|
- Fail fast: tests should run quickly and provide clear error messages
|
|
- Include negative test cases (invalid inputs, edge cases)
|
|
|
|
**Example**: A test that fails if the Julia API returns camelCase keys.
|
|
|
|
---
|
|
|
|
### 7. Runbook
|
|
|
|
**Purpose**: Deployment, scaling, and recovery procedures.
|
|
|
|
**Why it matters**: Runbooks ensure that deployments are consistent, repeatable, and recoverable. In GitOps, the runbook *is* the configuration, version-controlled alongside the code.
|
|
|
|
**Primary Audience**:
|
|
- **DevOps Engineers**: Execute deployments and scaling operations
|
|
- **SREs**: Manage system reliability and incident response
|
|
- **Developers**: Deploy feature branches for testing
|
|
|
|
**Format / Tooling**:
|
|
- **Kubernetes Manifests**: Declarative deployment configurations
|
|
- **Flux**: GitOps operator for Kubernetes
|
|
- **Helm Charts**: Package management for Kubernetes
|
|
- **Docker Compose**: Local development environments
|
|
|
|
**Best Practices**:
|
|
- Use Git as the source of truth (GitOps)
|
|
- Make deployments idempotent (running twice has same effect)
|
|
- Include rollback procedures
|
|
- Document scaling procedures for different load levels
|
|
|
|
**Example**: `git push` to update the replica count from 3 to 6.
|
|
|
|
---
|
|
|
|
## How the Stack Fits Together
|
|
|
|
```
|
|
┌─────────────────────────────────────────────────────────────┐
|
|
│ Requirements │
|
|
│ (Business goals, user needs) │
|
|
└───────────────────┬─────────────────────────────────────────┘
|
|
│
|
|
▼
|
|
┌─────────────────────────────────────────────────────────────┐
|
|
│ The Spec │
|
|
│ (Machine-readable contract: OpenAPI, Protobuf) │
|
|
└───────────────────┬─────────────────────────────────────────┘
|
|
│
|
|
▼
|
|
┌─────────────────────────────────────────────────────────────┐
|
|
│ Architecture │
|
|
│ (Structural blueprint: Mermaid, IcePanel) │
|
|
└───────────────────┬─────────────────────────────────────────┘
|
|
│
|
|
▼
|
|
┌─────────────────────────────────────────────────────────────┐
|
|
│ Walkthrough │
|
|
│ (Intuition, big picture narrative) │
|
|
└───────────────────┬─────────────────────────────────────────┘
|
|
│
|
|
▼
|
|
┌─────────────────────────────────────────────────────────────┐
|
|
│ Implementation │
|
|
│ (Actual code: SvelteKit, Julia, Node.js) │
|
|
└───────────────────┬─────────────────────────────────────────┘
|
|
│
|
|
▼
|
|
┌─────────────────────────────────────────────────────────────┐
|
|
│ Validation │
|
|
│ (Automated tests: GitHub Actions, Prism) │
|
|
└───────────────────┬─────────────────────────────────────────┘
|
|
│
|
|
▼
|
|
┌─────────────────────────────────────────────────────────────┐
|
|
│ Runbook │
|
|
│ (Deployment, scaling: K8s, Flux) │
|
|
└─────────────────────────────────────────────────────────────┘
|
|
```
|
|
|
|
## Key Principles
|
|
|
|
1. **Machine-Readable Truth**: Specs and configurations should be machine-readable to enable automation
|
|
2. **Separation of Concerns**: Different audiences need different types of information
|
|
3. **Version Control**: All documentation should be in Git, just like code
|
|
4. **Automation-First**: Validation should be automated and integrated into CI/CD
|
|
5. **Living Documentation**: Documentation should evolve with the codebase
|
|
|
|
## Getting Started
|
|
|
|
To adopt this stack in your project:
|
|
|
|
1. Start with requirements in GitHub Issues or Notion
|
|
2. Create a spec file (OpenAPI/Protobuf) as the contract
|
|
3. Add architecture diagrams using Mermaid.js
|
|
4. Write a walkthrough explaining the "why" behind decisions
|
|
5. Implement code following the spec
|
|
6. Add automated tests that validate the spec
|
|
7. Create runbooks for deployment and scaling
|
|
|
|
This framework ensures that every piece of documentation serves a clear purpose and reaches the right audience. |