diff --git a/docs/solution-design.md b/docs/solution-design.md index cc57b35..fb0ffd7 100644 --- a/docs/solution-design.md +++ b/docs/solution-design.md @@ -4,6 +4,7 @@ **Date**: 2026-05-22 **Status**: Active **Ground Truth**: [`src/msghandler.jl`](../src/msghandler.jl) +**ASG Framework Alignment**: v8 pillars - Requirements → Solution Design → Specification → Walkthrough → Implementation Plan → Validation → Runbook --- @@ -11,16 +12,16 @@ msghandler addresses the challenge of cross-platform data exchange between **Julia**, **JavaScript**, **Python**, **Dart**, **Rust**, and **MicroPython** applications using message brokers as transport layers. -### Problem Statement +### User Problems -Developers working across multiple programming languages face significant obstacles when trying to share data: - -| Problem | Description | User Impact | -|---------|-------------|-------------| -| **P-001**: Cross-platform data serialization | Different languages have incompatible data types and serialization formats | Developers must write platform-specific conversion code | -| **P-002**: Large payload handling | Message brokers have size limits, but large files need to be transferred | Large files either fail or require complex workarounds | -| **P-003**: Transport abstraction | Each platform has different message broker libraries and APIs | No unified interface across platforms | -| **P-004**: Request-response patterns | Bi-directional communication requires complex correlation tracking | Developers must implement custom message routing | +| Problem | Description | User Impact | Requirement ID | +|---------|-------------|-------------|----------------| +| **P-001**: Cross-platform data serialization | Different languages have incompatible data types and serialization formats | Developers must write platform-specific conversion code | FR-001, FR-002 | +| **P-002**: Large payload handling | Message brokers have size limits, but large files need to be transferred | Large files either fail or require complex workarounds | FR-003 | +| **P-003**: Transport abstraction | Each platform has different message broker libraries and APIs | No unified interface across platforms | FR-013, FR-014 | +| **P-004**: Request-response patterns | Bi-directional communication requires complex correlation tracking | Developers must implement custom message routing | FR-011 | +| **P-005**: File server reliability | File server may be temporarily unavailable during downloads | Failed downloads without retry mechanism | FR-010 | +| **P-006**: Payload type preservation | Different platforms have different type systems | Data corruption or misinterpretation on receiving end | FR-006, FR-007 | ### Solution Boundaries @@ -30,6 +31,7 @@ Developers working across multiple programming languages face significant obstac - File server integration using Claim-Check pattern - Multi-payload support with mixed types in single message - Exponential backoff for reliable file downloads +- Correlation ID propagation for message tracing **Out of Scope**: - Message compression (adds complexity without clear benefit) @@ -39,14 +41,16 @@ Developers working across multiple programming languages face significant obstac ### Decision IDs -| Decision ID | Decision | Description | -|-------------|----------|-------------| -| SD-001 | Claim-Check Pattern | Large payloads uploaded to HTTP server, small payloads sent directly | -| SD-002 | Automatic Transport Selection | <0.5MB = direct, ≥0.5MB = link based on size threshold | -| SD-003 | Handler Function Abstraction | Pluggable file server implementations via handler functions | -| SD-004 | Unified Tuple Format | Same `(dataname, data, type)` format across all platforms | -| SD-005 | Base64 Encoding | JSON-compatible binary data transport | -| SD-006 | Transport Abstraction | Support multiple broker protocols (NATS/MQTT/WebSocket) transparently | +| Decision ID | Decision | Description | Requirement IDs | NFR IDs | +|-------------|----------|-------------|-----------------|---------| +| SD-001 | Claim-Check Pattern | Large payloads uploaded to HTTP server, small payloads sent directly | FR-003, FR-004 | NFR-104, NFR-105 | +| SD-002 | Automatic Transport Selection | <0.5MB = direct, ≥0.5MB = link based on size threshold | FR-003, FR-004 | NFR-104, NFR-105 | +| SD-003 | Handler Function Abstraction | Pluggable file server implementations via handler functions | FR-008, FR-009 | NFR-202 | +| SD-004 | Unified Tuple Format | Same `(dataname, data, type)` format across all platforms | FR-006, FR-007 | - | +| SD-005 | Base64 Encoding | JSON-compatible binary data transport | FR-012 | - | +| SD-006 | Transport Abstraction | Support multiple broker protocols (NATS/MQTT/WebSocket) transparently | FR-013, FR-014 | NFR-201 | +| SD-007 | Exponential Backoff | Retry failed file downloads with exponential backoff | FR-010 | NFR-202 | +| SD-008 | Correlation ID Propagation | Propagate correlation IDs through all message processing steps | FR-011 | NFR-401, NFR-403 | --- @@ -77,6 +81,8 @@ Sender (smartpack) Transport Layer Receiver (smartunpa | **SD-004** | Unified Tuple Format | Same input/output format across all platforms | Platform-native formats - no interoperability; Protocol buffers - too heavy | | **SD-005** | Base64 Encoding | JSON-compatible binary data transport | Raw bytes - not JSON-compatible; Hex encoding - 2x size overhead | | **SD-006** | Transport Abstraction | Support multiple broker protocols (NATS/MQTT/WebSocket) transparently | Platform-specific libraries - no interoperability | +| **SD-007** | Exponential Backoff | Retry failed file downloads with exponential backoff | Simple retry - too aggressive; No retry - poor reliability | +| **SD-008** | Correlation ID Propagation | Propagate correlation IDs through all message processing steps | Manual correlation - error-prone; No tracing - debug impossible | ### Architecture Components @@ -110,7 +116,7 @@ flowchart TB DOWNLOAD -.->|Fetch URL| API end - style CLIENT fill:#e1f5fe,stroke:#0288d1,stroke-width:2px + style Client fill:#e1f5fe,stroke:#0288d1,stroke-width:2px style Transport fill:#ffe0b2,stroke:#f57c00,stroke-width:2px style FileServer fill:#c8e6c9,stroke:#43a047,stroke-width:2px ``` @@ -190,8 +196,8 @@ flowchart TD | Component | Responsibilities | Decision IDs | Requirements Addressed | |-----------|-----------------|--------------|----------------------| | **Serialization Layer** | Convert data types to transport format (Base64/URL) | SD-005 | FR-001, FR-002, FR-012 | -| **Envelope Builder** | Create standardized message envelope with metadata | SD-001 | FR-011, FR-013, FR-014 | -| **Handler Functions** | Abstract file server operations for pluggability | SD-003 | FR-008, FR-009 | +| **Envelope Builder** | Create standardized message envelope with metadata | SD-001, SD-008 | FR-011, FR-013, FR-014 | +| **Handler Functions** | Abstract file server operations for pluggability | SD-003, SD-007 | FR-008, FR-009, FR-010 | | **Transport Adapter** | Support multiple broker protocols transparently | SD-006 | FR-013, FR-014 | | **Payload Manager** | Track payload types, sizes, and encoding | SD-004 | FR-006, FR-007 | @@ -201,7 +207,8 @@ flowchart TD ### SD-001: Why Claim-Check Pattern? -**Requirement**: FR-003 - Large file handling, FR-004 - Direct transport for small payloads +**Requirement**: FR-003 (Large file handling), FR-004 (Direct transport for small payloads) +**NFRs**: NFR-104 (File upload latency <1s), NFR-105 (File download latency <1s) **Rationale**: - Transport layers (NATS, MQTT) have message size limits (typically 1MB) @@ -211,7 +218,8 @@ flowchart TD ### SD-002: Why Handler Functions for File Server? -**Requirement**: FR-008 - Plik integration, FR-009 - Custom file server support +**Requirement**: FR-008 (Plik integration), FR-009 (Custom file server support) +**NFR**: NFR-202 (File server availability <5% failure rate) **Rationale**: - Plik is common open-source solution for file server @@ -221,7 +229,7 @@ flowchart TD ### SD-003: Why Tuple Format for Payloads? -**Requirement**: FR-006 - Multi-payload messages, FR-007 - Payload type preservation +**Requirement**: FR-006 (Multi-payload messages), FR-007 (Payload type preservation) **Rationale**: - `(dataname, data, type)` tuple is language-agnostic @@ -231,7 +239,7 @@ flowchart TD ### SD-004: Why Base64 Encoding? -**Requirement**: FR-012 - Message serialization, FR-001 - Cross-platform text messaging +**Requirement**: FR-012 (Message serialization), FR-001 (Cross-platform text messaging) **Rationale**: - JSON is universal - works on all platforms @@ -241,7 +249,8 @@ flowchart TD ### SD-005: Why Automatic Transport Selection? -**Requirement**: FR-003, FR-004, NFR-104, NFR-105 +**Requirement**: FR-003 (Large file handling), FR-004 (Direct transport for small payloads) +**NFRs**: NFR-104 (File upload latency <1s), NFR-105 (File download latency <1s) **Rationale**: - <0.5MB payloads use direct transport (<1s latency, FR-004 KPI) @@ -250,7 +259,8 @@ flowchart TD ### SD-006: Why Transport Abstraction? -**Requirement**: FR-013, FR-014, NFR-201 +**Requirement**: FR-013 (Transport publishing), FR-014 (Transport subscription) +**NFR**: NFR-201 (Message delivery at-least-once) **Rationale**: - Support multiple broker protocols (NATS, MQTT, WebSocket) transparently @@ -258,17 +268,40 @@ flowchart TD - Unified API across all platforms - At-least-once delivery semantics via transport layer +### SD-007: Why Exponential Backoff? + +**Requirement**: FR-010 (Exponential backoff retry) +**NFR**: NFR-202 (File server availability <5% failure rate) + +**Rationale**: +- File server may be temporarily unavailable +- Exponential backoff prevents overwhelming server during outages +- Default: 5 retries, 100ms base delay, 5000ms max delay +- 95% successful downloads within retry limit (FR-010 KPI) + +### SD-008: Why Correlation ID Propagation? + +**Requirement**: FR-011 (Correlation ID propagation) +**NFRs**: NFR-401 (Required logs), NFR-403 (Tracing) + +**Rationale**: +- Trace messages across distributed systems +- Correlation ID logged with every message (NFR-401) +- Propagated through all message processing steps (NFR-403) +- Enables debugging and performance analysis in production + --- ## 6. Risk Assessment -| Risk | Impact | Probability | Mitigation | -|------|--------|-------------|------------| -| **Performance degradation with >500KB payloads** | High | Medium | Size threshold detection; Link transport fallback | -| **File server availability issues** | Medium | Low | Exponential backoff retry; Graceful degradation | -| **Platform-specific bugs** | Medium | Low | Comprehensive test suite per platform; CI validation | -| **Encoding mismatches between platforms** | High | Low | Strict specification; Test contracts; Validation rules | -| **Transport layer incompatibility** | Medium | Low | Transport-agnostic design; Handler abstraction | +| Risk | Impact | Probability | Mitigation | Requirement IDs | NFR IDs | +|------|--------|-------------|------------|-----------------|---------| +| **Performance degradation with >500KB payloads** | High | Medium | Size threshold detection; Link transport fallback | FR-003, FR-004 | NFR-104, NFR-105 | +| **File server availability issues** | Medium | Low | Exponential backoff retry; Graceful degradation | FR-010 | NFR-202 | +| **Platform-specific bugs** | Medium | Low | Comprehensive test suite per platform; CI validation | FR-001, FR-002, FR-006, FR-007 | - | +| **Encoding mismatches between platforms** | High | Low | Strict specification; Test contracts; Validation rules | FR-012 | NFR-301 | +| **Transport layer incompatibility** | Medium | Low | Transport-agnostic design; Handler abstraction | FR-013, FR-014 | NFR-201 | +| **Correlation ID loss in processing** | Medium | Low | Centralized trace context management | FR-011 | NFR-401, NFR-403 | --- @@ -276,13 +309,13 @@ flowchart TD | Solution Component | Decision ID | Requirement ID | Description | |-------------------|-------------|----------------|-------------| -| **smartpack() function** | SD-001, SD-002, SD-004, SD-005, SD-006 | FR-001, FR-002, FR-003, FR-004, FR-005, FR-006, FR-007, FR-008, FR-009, FR-010, FR-011, FR-012, FR-013, FR-014 | Unified API for sending messages across all platforms | -| **smartunpack() function** | SD-001, SD-002, SD-004, SD-005, SD-006 | FR-001, FR-002, FR-003, FR-004, FR-005, FR-006, FR-007, FR-008, FR-009, FR-010, FR-011, FR-012, FR-013, FR-014 | Unified API for receiving messages across all platforms | -| **Direct transport** | SD-002 | FR-004, NFR-101, NFR-102, NFR-104, NFR-105 | Send payloads < threshold directly via transport | -| **Link transport** | SD-001, SD-002 | FR-003, NFR-104, NFR-105 | Upload payloads ≥ threshold to file server | -| **File server handler** | SD-003 | FR-008, FR-009, FR-010 | Pluggable upload/download handlers with retry logic | +| **smartpack() function** | SD-001, SD-002, SD-004, SD-005, SD-006, SD-008 | FR-001, FR-002, FR-003, FR-004, FR-005, FR-006, FR-007, FR-008, FR-009, FR-010, FR-011, FR-012, FR-013, FR-014 | Unified API for sending messages across all platforms | +| **smartunpack() function** | SD-001, SD-002, SD-004, SD-005, SD-006, SD-007, SD-008 | FR-001, FR-002, FR-003, FR-004, FR-005, FR-006, FR-007, FR-008, FR-009, FR-010, FR-011, FR-012, FR-013, FR-014 | Unified API for receiving messages across all platforms | +| **Direct transport** | SD-002 | FR-004 | Send payloads < threshold directly via transport | +| **Link transport** | SD-001, SD-002 | FR-003 | Upload payloads ≥ threshold to file server | +| **File server handler** | SD-003, SD-007 | FR-008, FR-009, FR-010 | Pluggable upload/download handlers with retry logic | | **Payload type preservation** | SD-004 | FR-006, FR-007 | Support text, dictionary, arrowtable, jsontable, image, audio, video, binary | -| **Correlation ID** | SD-001 | FR-011, NFR-401, NFR-403 | Message tracing across distributed systems | +| **Correlation ID propagation** | SD-008 | FR-011 | Message tracing across distributed systems | | **Multi-payload support** | SD-004 | FR-006, FR-007 | List of (dataname, data, type) tuples | ### Non-Functional Requirements Traceability @@ -295,11 +328,15 @@ flowchart TD | **Concurrent connections** | SD-006 | NFR-106 | Support 100+ simultaneous connections | | **Message throughput** | SD-005, SD-006 | NFR-107 | Handle 1000+ messages/second per instance | | **At-least-once delivery** | SD-006 | NFR-201 | Transport layer semantics | -| **Graceful degradation** | SD-003 | NFR-202 | File server unavailability handling | +| **Graceful degradation** | SD-003, SD-007 | NFR-202 | File server unavailability handling | | **Auto-reconnect** | SD-006 | NFR-203 | Transport connection failure recovery | -| **Required logs** | SD-001 | NFR-401 | Correlation ID, msg_id, timestamp, etc. | +| **Payload integrity** | SD-005 | NFR-301 | 100% SHA-256 checksum validation | +| **Transport security** | SD-006 | NFR-302 | 100% TLS connections in production | +| **File server security** | SD-003 | NFR-303 | 100% authenticated file uploads | +| **Required logs** | SD-001, SD-008 | NFR-401 | Correlation ID, msg_id, timestamp, etc. | | **Critical metrics** | SD-001, SD-005 | NFR-402 | messages_sent_total, file upload/download duration | -| **Tracing** | SD-001 | NFR-403 | Correlation ID propagation | +| **Tracing** | SD-001, SD-008 | NFR-403 | Correlation ID propagation | +| **Alerting** | SD-007 | NFR-404 | <5min alert latency for `download_retry_exceeded` | --- @@ -307,24 +344,33 @@ flowchart TD | Stage Transition | Gap-Check Question | Status | |------------------|-------------------|--------| -| **Requirements → Solution Design** | Does the Solution Design clearly explain how the system solves the user problem, not just what it does? | ✅ Verified - All user stories mapped to solution components with requirement ID and decision ID references | +| **Requirements → Solution Design** | Does the Solution Design clearly explain how the system solves the user problem, not just what it does? | ✅ Verified - All user problems mapped to solution components with requirement ID and decision ID references | | **Solution Design → Specification** | Does the Specification define all technical details that the solution approach requires? | ⏳ Pending - Specification needs review for completeness | | **Solution Design → Walkthrough** | Does the Walkthrough reflect the complete flow including error states and timing? | ⏳ Pending - Walkthrough needs validation against design | ### Solution Design Validation -**Problem**: Users need to send mixed payload types (text + image + large file) between Julia, JavaScript, Python, and MicroPython applications. +**User Problems** (from requirements.md): +- **P-001**: Cross-platform data serialization (FR-001, FR-002) +- **P-002**: Large payload handling (FR-003) +- **P-003**: Transport abstraction (FR-013, FR-014) +- **P-004**: Request-response patterns (FR-011) +- **P-005**: File server reliability (FR-010) +- **P-006**: Payload type preservation (FR-006, FR-007) **Solution Components**: -1. **SD-001** - `smartpack()` - Unified API for all platforms -2. **SD-002** - Tuple format - `(dataname, data, type)` - platform-agnostic -3. **SD-003** - Automatic transport selection - <0.5MB = direct, ≥0.5MB = link -4. **SD-004** - File server handler abstraction - Plik/AWS S3/custom support -5. **SD-005** - Exponential backoff - Reliable file downloads -6. **SD-006** - Correlation ID - Message tracing +1. **SD-001** - `smartpack()` / `smartunpack()` - Unified API for all platforms +2. **SD-002** - Claim-Check pattern - Automatic transport selection based on size threshold +3. **SD-003** - Handler function abstraction - Plik/AWS S3/custom file server support +4. **SD-004** - Tuple format - `(dataname, data, type)` - platform-agnostic +5. **SD-005** - Base64 encoding - JSON-compatible binary data transport +6. **SD-006** - Transport abstraction - Support multiple broker protocols transparently +7. **SD-007** - Exponential backoff - Reliable file downloads with retry logic +8. **SD-008** - Correlation ID propagation - Message tracing across distributed systems **Requirement Mapping**: -- FR-001, FR-002, FR-003, FR-004, FR-005, FR-006, FR-007, FR-008, FR-009, FR-010, FR-011, FR-012, FR-013, FR-014 ✅ +- **Functional Requirements**: FR-001 through FR-014 ✅ +- **Non-Functional Requirements**: NFR-101 through NFR-405 ✅ **Gap Check**: Does this solution explain *how* users will actually use the system? @@ -333,7 +379,13 @@ flowchart TD 2. `smartpack()` automatically selects transport based on size (SD-002) 3. Large file (≥0.5MB) → link transport → file server upload (SD-001) 4. Small payload (<0.5MB) → direct transport → base64 encoding (SD-005) -5. Receiver calls `smartunpack()` → receives same tuple format +5. Receiver calls `smartunpack()` → receives same tuple format with preserved types + +**NFR Traceability**: +- **Performance**: NFR-101 (serialization <50ms), NFR-102 (deserialization <50ms), NFR-103 (connection <100ms) ✅ +- **Reliability**: NFR-201 (at-least-once delivery), NFR-202 (file server <5% failure), NFR-203 (auto-reconnect <30s) ✅ +- **Security**: NFR-301 (SHA-256 checksum), NFR-302 (TLS 100%), NFR-303 (authenticated uploads) ✅ +- **Observability**: NFR-401 (required logs), NFR-402 (metrics), NFR-403 (tracing), NFR-404 (alerting <5min) ✅ --- diff --git a/docs/specification.md b/docs/specification.md index df773f6..101e698 100644 --- a/docs/specification.md +++ b/docs/specification.md @@ -1,10 +1,11 @@ # Specification: msghandler -**Version**: 1.2.0 -**Date**: 2026-05-13 +**Version**: 1.3.0 +**Date**: 2026-05-22 **Status**: Active **Ground Truth**: [`src/msghandler.jl`](../src/msghandler.jl) -**Specification Format**: JSON Schema + AsyncAPI +**Specification Format**: JSON Schema + AsyncAPI +**ASG Framework Alignment**: v8 pillars - Requirements → Solution Design → Specification → Walkthrough → Implementation Plan → Validation → Runbook --- @@ -20,26 +21,26 @@ This specification serves as the single source of truth for: ### 1.1 Requirements Traceability -| Specification Section | Requirement ID(s) | Description | -|----------------------|-------------------|-------------| -| Section 2 (Message Envelope) | FR-012, FR-013, NFR-101, NFR-102 | Message envelope structure and validation | -| Section 3 (Payload Schema) | FR-001, FR-002, FR-003, FR-004, NFR-101, NFR-102 | Payload structure and field definitions | -| Section 4 (Payload Format) | FR-006, FR-007 | Tuple format for smartpack() | -| Section 5 (Enumerations) | FR-003, FR-004, FR-006, NFR-101 | Enumerations for transport and encoding | -| Section 6 (Transport Protocols) | FR-003, FR-004, NFR-104, NFR-105 | Direct and link transport protocols | -| Section 7 (Size Thresholds) | FR-004, FR-005, NFR-104, NFR-105 | Size thresholds for transport selection | -| Section 8 (Topic Convention) | FR-013, FR-014 | Topic/subject naming patterns | -| Section 9 (Error Handling) | FR-010, FR-011, NFR-201, NFR-202, NFR-203 | Error codes and exception handling | -| Section 10 (Serialization Rules) | FR-001, FR-002, FR-003, FR-012, NFR-101, NFR-102 | Serialization and encoding rules | -| Section 11 (API Contract) | FR-001, FR-002, FR-003, FR-004, FR-005, FR-006, FR-007, FR-008, FR-009, FR-010, FR-011, FR-012, FR-013, FR-014 | Function signatures for all platforms | -| Section 12 (File Server Interface) | FR-008, FR-009, FR-010 | Upload and download handler contracts | -| Section 13 (Platform-Specific Constraints) | FR-005, FR-006, NFR-106, NFR-107 | Platform-specific feature support | -| Section 14 (Implementation Files) | FR-001, FR-002, FR-003, FR-004, FR-005, FR-006, FR-007 | Implementation file mapping | -| Section 15 (Message Flow) | FR-001, FR-002, FR-003, FR-004, FR-005, FR-006, FR-007, FR-008, FR-009, FR-010, FR-011, FR-012, FR-013, FR-014 | Mermaid diagrams for send/receive flows | -| Section 16 (Validation Rules) | FR-001, FR-002, FR-003, FR-004, FR-005, FR-006, FR-007, FR-008, FR-009, FR-010, FR-011, FR-012, FR-013, FR-014 | Envelope and payload validation rules | -| Section 17 (Test Contracts) | FR-001, FR-002, FR-003, FR-004, FR-005, FR-006, FR-007, FR-008, FR-009, FR-010, FR-011, FR-012, FR-013, FR-014 | Unit and integration test scenarios | -| Section 18 (Dependencies) | FR-001, FR-002, FR-003, FR-004, FR-005, FR-006, FR-007, FR-008, FR-009, FR-010, FR-011, FR-012, FR-013, FR-014 | Platform-specific dependencies | -| Section 19 (Change Log) | N/A | Version history and changes | +| Specification Section | Requirement ID(s) | Solution Design Ref(s) | Description | +|----------------------|-------------------|------------------------|-------------| +| Section 2 (Message Envelope) | FR-012, FR-013, NFR-101, NFR-102 | SD-008 | Message envelope structure and validation | +| Section 3 (Payload Schema) | FR-001, FR-002, FR-003, FR-004, NFR-101, NFR-102 | SD-001, SD-005 | Payload structure and field definitions | +| Section 4 (Payload Format) | FR-006, FR-007 | SD-004 | Tuple format for smartpack() | +| Section 5 (Enumerations) | FR-003, FR-004, FR-006, NFR-101 | SD-001, SD-002, SD-005 | Enumerations for transport and encoding | +| Section 6 (Transport Protocols) | FR-003, FR-004, NFR-104, NFR-105 | SD-001, SD-002 | Direct and link transport protocols | +| Section 7 (Size Thresholds) | FR-004, FR-005, NFR-104, NFR-105 | SD-002 | Size thresholds for transport selection | +| Section 8 (Topic Convention) | FR-013, FR-014 | SD-006 | Topic/subject naming patterns | +| Section 9 (Error Handling) | FR-010, FR-011, NFR-201, NFR-202, NFR-203 | SD-003, SD-007 | Error codes and exception handling | +| Section 10 (Serialization Rules) | FR-001, FR-002, FR-003, FR-012, NFR-101, NFR-102 | SD-005 | Serialization and encoding rules | +| Section 11 (API Contract) | FR-001 through FR-014, NFR-101 through NFR-107 | SD-001 through SD-008 | Function signatures for all platforms | +| Section 12 (File Server Interface) | FR-008, FR-009, FR-010 | SD-003, SD-007 | Upload and download handler contracts | +| Section 13 (Platform-Specific Constraints) | FR-005, FR-006, NFR-106, NFR-107 | SD-004, SD-006 | Platform-specific feature support | +| Section 14 (Implementation Files) | FR-001 through FR-007 | - | Implementation file mapping | +| Section 15 (Message Flow) | FR-001 through FR-014, NFR-101 through NFR-107 | SD-001 through SD-008 | Mermaid diagrams for send/receive flows | +| Section 16 (Validation Rules) | FR-001 through FR-014, NFR-101 through NFR-405 | SD-001 through SD-008 | Envelope and payload validation rules | +| Section 17 (Test Contracts) | FR-001 through FR-014, NFR-101 through NFR-107 | SD-001 through SD-008 | Unit and integration test scenarios | +| Section 18 (Dependencies) | FR-001 through FR-014, NFR-101 through NFR-107 | SD-001 through SD-008 | Platform-specific dependencies | +| Section 19 (Change Log) | - | - | Version history and changes | --- @@ -90,22 +91,22 @@ This specification serves as the single source of truth for: ### Field Definitions -| Field | Type | Required | Validation | Description | Requirement ID | -|-------|------|----------|------------|-------------|----------------| -| `correlation_id` | `string` | Yes | UUID v4 format | Track message flow across distributed systems | FR-011, NFR-401 | -| `msg_id` | `string` | Yes | UUID v4 format | Unique identifier for this specific message | FR-012, NFR-401 | -| `timestamp` | `string` | Yes | ISO 8601 UTC | Message publication timestamp | FR-012, NFR-401 | -| `send_to` | `string` | Yes | Non-empty string | Topic/subject to publish the message to | FR-013 | -| `msg_purpose` | `string` | Yes | Enum | Purpose of the message | FR-013 | -| `sender_name` | `string` | Yes | Non-empty string | Name of the sender application | FR-013 | -| `sender_id` | `string` | Yes | UUID v4 format | Unique identifier for the sender | FR-013 | -| `receiver_name` | `string` | Yes | Any string | Name of the receiver (empty = broadcast) | FR-013 | -| `receiver_id` | `string` | Yes | Any string | UUID of the receiver (empty = broadcast) | FR-013 | -| `reply_to` | `string` | Yes | Any string | Topic where receiver should reply | FR-013 | -| `reply_to_msg_id` | `string` | Yes | Any string | Message ID this message is replying to | FR-013 | -| `broker_url` | `string` | Yes | Valid URL | Broker URL for the transport layer | FR-013 | -| `metadata` | `object` | No | Any JSON object | Message-level metadata | NFR-401 | -| `payloads` | `array` | Yes | Non-empty array | List of payload objects | FR-012, FR-013 | +| Field | Type | Required | Validation | Description | Requirement ID | Solution Design Ref | +|-------|------|----------|------------|-------------|----------------|-------------------| +| `correlation_id` | `string` | Yes | UUID v4 format | Track message flow across distributed systems | FR-011, NFR-401 | SD-008 | +| `msg_id` | `string` | Yes | UUID v4 format | Unique identifier for this specific message | FR-012, NFR-401 | SD-001 | +| `timestamp` | `string` | Yes | ISO 8601 UTC | Message publication timestamp | FR-012, NFR-401 | SD-001 | +| `send_to` | `string` | Yes | Non-empty string | Topic/subject to publish the message to | FR-013 | SD-006 | +| `msg_purpose` | `string` | Yes | Enum | Purpose of the message | FR-013 | SD-006 | +| `sender_name` | `string` | Yes | Non-empty string | Name of the sender application | FR-013 | SD-006 | +| `sender_id` | `string` | Yes | UUID v4 format | Unique identifier for the sender | FR-013 | SD-006 | +| `receiver_name` | `string` | Yes | Any string | Name of the receiver (empty = broadcast) | FR-013 | SD-006 | +| `receiver_id` | `string` | Yes | Any string | UUID of the receiver (empty = broadcast) | FR-013 | SD-006 | +| `reply_to` | `string` | Yes | Any string | Topic where receiver should reply | FR-013 | SD-006 | +| `reply_to_msg_id` | `string` | Yes | Any string | Message ID this message is replying to | FR-013 | SD-006 | +| `broker_url` | `string` | Yes | Valid URL | Broker URL for the transport layer | FR-013 | SD-006 | +| `metadata` | `object` | No | Any JSON object | Message-level metadata | NFR-401 | SD-001, SD-008 | +| `payloads` | `array` | Yes | Non-empty array | List of payload objects | FR-012, FR-013 | SD-001, SD-005 | --- @@ -128,16 +129,16 @@ This specification serves as the single source of truth for: ### Payload Field Definitions -| Field | Type | Required | Validation | Description | Requirement ID | -|-------|------|----------|------------|-------------|----------------| -| `id` | `string` | Yes | UUID v4 format | Unique identifier for this payload | FR-012 | -| `dataname` | `string` | Yes | Non-empty string | Name of the payload (e.g., `login_image`, `user_data`) | FR-001, FR-002, FR-003 | -| `payload_type` | `string` | Yes | Enum | Type of payload (see `payload_type` enum) | FR-001, FR-002, FR-003, FR-006 | -| `transport` | `string` | Yes | Enum | Transport method: `direct` or `link` | FR-003, FR-004, NFR-104, NFR-105 | -| `encoding` | `string` | Yes | Enum | Encoding method (see `encoding` enum) | FR-001, FR-002, FR-003, FR-012 | -| `size` | `integer` | Yes | Positive integer | Size of the payload in bytes | FR-003, FR-004, NFR-104, NFR-105 | -| `data` | `string` or `URL` | Yes | Base64 string or URL | Payload data (base64 for direct, URL for link) | FR-003, FR-004, FR-008, FR-009, FR-010 | -| `metadata` | `object` | No | Any JSON object | Payload-level metadata | NFR-401 | +| Field | Type | Required | Validation | Description | Requirement ID | Solution Design Ref | +|-------|------|----------|------------|-------------|----------------|-------------------| +| `id` | `string` | Yes | UUID v4 format | Unique identifier for this payload | FR-012 | SD-001 | +| `dataname` | `string` | Yes | Non-empty string | Name of the payload (e.g., `login_image`, `user_data`) | FR-001, FR-002, FR-003 | SD-004 | +| `payload_type` | `string` | Yes | Enum | Type of payload (see `payload_type` enum) | FR-001, FR-002, FR-003, FR-006 | SD-004, SD-005 | +| `transport` | `string` | Yes | Enum | Transport method: `direct` or `link` | FR-003, FR-004, NFR-104, NFR-105 | SD-001, SD-002 | +| `encoding` | `string` | Yes | Enum | Encoding method (see `encoding` enum) | FR-001, FR-002, FR-003, FR-012 | SD-005 | +| `size` | `integer` | Yes | Positive integer | Size of the payload in bytes | FR-003, FR-004, NFR-104, NFR-105 | SD-002 | +| `data` | `string` or `URL` | Yes | Base64 string or URL | Payload data (base64 for direct, URL for link) | FR-003, FR-004, FR-008, FR-009, FR-010 | SD-001, SD-003 | +| `metadata` | `object` | No | Any JSON object | Payload-level metadata | NFR-401 | SD-008 | --- @@ -227,30 +228,30 @@ await smartpack("/agent/v1/process", data) | Value | Description | Supported Platforms | Encoding Options | |-------|-------------|---------------------|------------------| -| `text` | Plain text string | All | `base64` | -| `dictionary` | JSON object/dictionary | All | `base64`, `json` | -| `arrowtable` | Apache Arrow IPC table | Desktop (Julia/Python/Node.js/Dart) | `base64`, `arrow-ipc` | -| `jsontable` | JSON array of objects | All (including Browser/Dart Web) | `base64`, `json` | -| `image` | Binary image data | All | `base64` | -| `audio` | Binary audio data | All | `base64` | -| `video` | Binary video data | All | `base64` | -| `binary` | Generic binary data | All | `base64` | +| `text` | Plain text string | All | `base64` | FR-001, FR-012 | SD-004, SD-005 | +| `dictionary` | JSON object/dictionary | All | `base64`, `json` | FR-002, FR-012 | SD-004, SD-005 | +| `arrowtable` | Apache Arrow IPC table | Desktop (Julia/Python/Node.js/Dart) | `base64`, `arrow-ipc` | FR-002 | SD-004, SD-005 | +| `jsontable` | JSON array of objects | All (including Browser/Dart Web) | `base64`, `json` | FR-002, FR-006 | SD-004, SD-005 | +| `image` | Binary image data | All | `base64` | FR-001, FR-006 | SD-004, SD-005 | +| `audio` | Binary audio data | All | `base64` | FR-001, FR-006 | SD-004, SD-005 | +| `video` | Binary video data | All | `base64` | FR-001, FR-006 | SD-004, SD-005 | +| `binary` | Generic binary data | All | `base64` | FR-001, FR-006 | SD-004, SD-005 | ### `transport` Enum | Value | Description | Data Format | Use Case | |-------|-------------|-------------|----------| -| `direct` | Payload sent directly via the transport layer | Base64-encoded string | Payloads < size_threshold | -| `link` | Payload uploaded to file server | HTTP URL | Payloads ≥ size_threshold | +| `direct` | Payload sent directly via the transport layer | Base64-encoded string | Payloads < size_threshold | FR-003, FR-004, NFR-104, NFR-105 | SD-001, SD-002 | +| `link` | Payload uploaded to file server | HTTP URL | Payloads ≥ size_threshold | FR-003, FR-004, NFR-104, NFR-105 | SD-001, SD-002 | ### `encoding` Enum | Value | Description | Payload Types | |-------|-------------|---------------| -| `none` | No additional encoding | Link transport URLs | -| `base64` | Base64 encoding | Text, binary, image, audio, video | -| `json` | JSON encoding | Dictionary, jsontable | -| `arrow-ipc` | Apache Arrow IPC format | Arrowtable | +| `none` | No additional encoding | Link transport URLs | FR-003, FR-004 | SD-001, SD-002 | +| `base64` | Base64 encoding | Text, binary, image, audio, video | FR-001, FR-002, FR-003, FR-012 | SD-005 | +| `json` | JSON encoding | Dictionary, jsontable | FR-002, FR-006 | SD-004, SD-005 | +| `arrow-ipc` | Apache Arrow IPC format | Arrowtable | FR-002 | SD-004, SD-005 | --- @@ -300,15 +301,15 @@ When `transport = "link"`, the `data` field contains a URL pointing to the uploa ### Desktop Platforms (Julia/JS/Python) -| Platform | Size Threshold | Notes | -|----------|----------------|-------| -| Desktop | 500,000 bytes (0.5MB) | Default threshold | +| Platform | Size Threshold | Notes | Requirement ID | Solution Design Ref | +|----------|----------------|-------|----------------|-------------------| +| Desktop | 500,000 bytes (0.5MB) | Default threshold | FR-004, FR-005, NFR-104, NFR-105 | SD-002 | ### MicroPython Platform -| Platform | Size Threshold | Maximum Payload | Notes | -|----------|----------------|-----------------|-------| -| MicroPython | 100,000 bytes (100KB) | 50,000 bytes | Hard limit due to memory constraints | +| Platform | Size Threshold | Maximum Payload | Notes | Requirement ID | Solution Design Ref | +|----------|----------------|-----------------|-------|----------------|-------------------| +| MicroPython | 100,000 bytes (100KB) | 50,000 bytes | Hard limit due to memory constraints | FR-005, NFR-106 | SD-002 | --- @@ -321,16 +322,16 @@ When `transport = "link"`, the `data` field contains a URL pointing to the uploa ``` **Examples**: -- `/agent/wine/api/v1/prompt` - AI agent prompt endpoint -- `/chat/user/v1/message` - User chat message -- `/system/worker/v1/status` - Worker status update +- `/agent/wine/api/v1/prompt` - AI agent prompt endpoint | FR-013 | SD-006 +- `/chat/user/v1/message` - User chat message | FR-013 | SD-006 +- `/system/worker/v1/status` - Worker status update | FR-013 | SD-006 ### Subject Wildcards | Wildcard | Description | Example | |----------|-------------|---------| -| `*` | Single-level wildcard | `/chat/user/v1/*` matches `/chat/user/v1/message` | -| `>` | Multi-level wildcard | `/chat/user/v1/>` matches all `/chat/user/v1/*` subjects | +| `*` | Single-level wildcard | `/chat/user/v1/*` matches `/chat/user/v1/message` | FR-013 | SD-006 | +| `>` | Multi-level wildcard | `/chat/user/v1/>` matches all `/chat/user/v1/*` subjects | FR-013 | SD-006 | --- @@ -352,23 +353,23 @@ When `transport = "link"`, the `data` field contains a URL pointing to the uploa | Code | HTTP Status | Description | Recovery | Requirement ID | |------|-------------|-------------|----------|----------------| -| `INVALID_ENVELOPE` | 400 | Message envelope validation failed | Fix envelope structure | FR-012, FR-013, FR-014 | -| `INVALID_PAYLOAD_TYPE` | 400 | Unsupported payload type | Use supported payload_type | FR-001, FR-002, FR-003, FR-006 | -| `INVALID_TRANSPORT` | 400 | Unsupported transport type | Use `direct` or `link` | FR-003, FR-004, FR-006 | -| `UPLOAD_FAILED` | 500 | File server upload failed | Retry or use direct transport | FR-008, FR-009 | -| `DOWNLOAD_FAILED` | 503 | File server download failed | Retry with exponential backoff | FR-010, FR-011, NFR-201, NFR-202 | -| `TRANSPORT_CONNECTION_FAILED` | 503 | Transport connection failed | Check broker/server availability | FR-013, FR-014, NFR-201, NFR-203 | -| `DESERIALIZATION_ERROR` | 500 | Payload deserialization failed | Check payload_type matches data | FR-001, FR-002, FR-003, FR-012 | -| `SIZE_EXCEEDED` | 413 | Payload exceeds maximum size | Split payload or use link transport | FR-003, FR-004, FR-005, NFR-104, NFR-105 | +| `INVALID_ENVELOPE` | 400 | Message envelope validation failed | Fix envelope structure | FR-012, FR-013, FR-014 | SD-001, SD-006 | +| `INVALID_PAYLOAD_TYPE` | 400 | Unsupported payload type | Use supported payload_type | FR-001, FR-002, FR-003, FR-006 | SD-004, SD-005 | +| `INVALID_TRANSPORT` | 400 | Unsupported transport type | Use `direct` or `link` | FR-003, FR-004, FR-006 | SD-001, SD-002 | +| `UPLOAD_FAILED` | 500 | File server upload failed | Retry or use direct transport | FR-008, FR-009 | SD-003 | +| `DOWNLOAD_FAILED` | 503 | File server download failed | Retry with exponential backoff | FR-010, FR-011, NFR-201, NFR-202 | SD-003, SD-007 | +| `TRANSPORT_CONNECTION_FAILED` | 503 | Transport connection failed | Check broker/server availability | FR-013, FR-014, NFR-201, NFR-203 | SD-006 | +| `DESERIALIZATION_ERROR` | 500 | Payload deserialization failed | Check payload_type matches data | FR-001, FR-002, FR-003, FR-012 | SD-004, SD-005 | +| `SIZE_EXCEEDED` | 413 | Payload exceeds maximum size | Split payload or use link transport | FR-003, FR-004, FR-005, NFR-104, NFR-105 | SD-001, SD-002 | ### Exception Handling | Scenario | Handler | Retry Policy | Requirement ID | |----------|---------|--------------|----------------| -| File server unavailable | Retry up to 5 times | Exponential backoff (100ms → 5000ms) | FR-010, NFR-202 | -| Transport publish failure | Handled by caller | Caller's retry logic | FR-013, FR-014, NFR-201, NFR-203 | -| Deserialization error | Log correlation ID and throw | No retry (data corruption) | FR-001, FR-002, FR-003, FR-012, NFR-401 | -| Memory overflow (MicroPython) | Reject payloads >50KB | No retry (client-side check) | FR-005, NFR-106 | +| File server unavailable | Retry up to 5 times | Exponential backoff (100ms → 5000ms) | FR-010, NFR-202 | SD-003, SD-007 | +| Transport publish failure | Handled by caller | Caller's retry logic | FR-013, FR-014, NFR-201, NFR-203 | SD-006 | +| Deserialization error | Log correlation ID and throw | No retry (data corruption) | FR-001, FR-002, FR-003, FR-012, NFR-401 | SD-004, SD-005, SD-008 | +| Memory overflow (MicroPython) | Reject payloads >50KB | No retry (client-side check) | FR-005, NFR-106 | SD-004 | --- @@ -411,6 +412,11 @@ When `transport = "link"`, the `data` field contains a URL pointing to the uploa ## API Contract +| Platform | Function | Requirements | Solution Design Ref | +|----------|----------|--------------|-------------------| +| All | `smartpack()` | FR-001 through FR-014, NFR-101 through NFR-107 | SD-001 through SD-008 | +| All | `smartunpack()` | FR-001 through FR-014, NFR-101 through NFR-107 | SD-001 through SD-008 | + ### `smartpack` Function Signature #### Julia @@ -638,6 +644,10 @@ pub struct MsgEnvelopeV1 { ### `smartunpack` Function Signature +**Note**: Input is the JSON string payload from the transport subscription. Returns `(envelope::msg_envelope_v1, payloads::Array{Tuple{String, Any, String}, 1})`. + +**Traceability**: FR-001 through FR-014, NFR-101 through NFR-107 | SD-001 through SD-008 + #### Julia ```julia @@ -757,6 +767,8 @@ pub struct smartunpackOptions { ## File Server Interface +**Traceability**: FR-008, FR-009, FR-010, NFR-202 | SD-003, SD-007 + ### Upload Handler Contract **Function Signature**: @@ -785,15 +797,19 @@ function fileserver_upload_handler( ``` **Required Keys**: -| Key | Type | Description | -|-----|------|-------------| -| `status` | `integer` | HTTP response status code | -| `uploadid` | `string` | Upload session identifier | -| `fileid` | `string` | File identifier within session | -| `url` | `string` | Full download URL | +| Key | Type | Description | Requirement ID | Solution Design Ref | +|-----|------|-------------|----------------|-------------------| +| `status` | `integer` | HTTP response status code | FR-008, FR-009 | SD-003 | +| `uploadid` | `string` | Upload session identifier | FR-008 | SD-003 | +| `fileid` | `string` | File identifier within session | FR-008 | SD-003 | +| `url` | `string` | Full download URL | FR-008, FR-009, FR-010 | SD-003, SD-007 | ### Download Handler Contract +**Function Signature**: + +**Traceability**: FR-010, NFR-202 | SD-003, SD-007 + **Function Signature**: ```julia function fileserver_download_handler( @@ -815,102 +831,108 @@ function fileserver_download_handler( ## Platform-Specific Constraints +**Traceability**: FR-005, FR-006, NFR-106, NFR-107 | SD-002, SD-004 + ### Desktop (Julia/Python/Node.js/Dart) -| Feature | Status | Notes | -|---------|--------|-------| -| Arrow IPC | ✅ Supported | Requires Arrow.jl/pyarrow/dart-arrow | -| JSON table | ✅ Supported | Human-readable format | -| File server upload | ✅ Supported | HTTP/HTTPS | -| File server download | ✅ Supported | HTTP/HTTPS | -| Size threshold | 500KB | Configurable | +| Feature | Status | Notes | Requirement ID | Solution Design Ref | +|---------|--------|-------|----------------|-------------------| +| Arrow IPC | ✅ Supported | Requires Arrow.jl/pyarrow/dart-arrow | FR-002, FR-012 | SD-004, SD-005 | +| JSON table | ✅ Supported | Human-readable format | FR-002, FR-006 | SD-004, SD-005 | +| File server upload | ✅ Supported | HTTP/HTTPS | FR-008, FR-009 | SD-003 | +| File server download | ✅ Supported | HTTP/HTTPS | FR-010, NFR-202 | SD-003, SD-007 | +| Size threshold | 500KB | Configurable | FR-004, FR-005, NFR-104, NFR-105 | SD-002 | ### Browser (JavaScript) -| Feature | Status | Notes | -|---------|--------|-------| -| Arrow IPC | ❌ Not supported | Apache Arrow not browser-compatible | -| JSON table | ✅ Supported | Only table type available in browser | -| File server upload | ✅ Supported | HTTP/HTTPS | -| File server download | ✅ Supported | HTTP/HTTPS | -| Size threshold | 500KB | Configurable | +| Feature | Status | Notes | Requirement ID | Solution Design Ref | +|---------|--------|-------|----------------|-------------------| +| Arrow IPC | ❌ Not supported | Apache Arrow not browser-compatible | FR-002 | SD-004, SD-005 | +| JSON table | ✅ Supported | Only table type available in browser | FR-002, FR-006 | SD-004, SD-005 | +| File server upload | ✅ Supported | HTTP/HTTPS | FR-008, FR-009 | SD-003 | +| File server download | ✅ Supported | HTTP/HTTPS | FR-010, NFR-202 | SD-003, SD-007 | +| Size threshold | 500KB | Configurable | FR-004, FR-005, NFR-104, NFR-105 | SD-002 | ### Dart Desktop (Dart SDK) -| Feature | Status | Notes | -|---------|--------|-------| -| Arrow IPC | ✅ Supported | Requires dart-arrow package | -| JSON table | ✅ Supported | Human-readable format | -| File server upload | ✅ Supported | HTTP/HTTPS | -| File server download | ✅ Supported | HTTP/HTTPS | -| Size threshold | 500KB | Configurable | +| Feature | Status | Notes | Requirement ID | Solution Design Ref | +|---------|--------|-------|----------------|-------------------| +| Arrow IPC | ✅ Supported | Requires dart-arrow package | FR-002, FR-012 | SD-004, SD-005 | +| JSON table | ✅ Supported | Human-readable format | FR-002, FR-006 | SD-004, SD-005 | +| File server upload | ✅ Supported | HTTP/HTTPS | FR-008, FR-009 | SD-003 | +| File server download | ✅ Supported | HTTP/HTTPS | FR-010, NFR-202 | SD-003, SD-007 | +| Size threshold | 500KB | Configurable | FR-004, FR-005, NFR-104, NFR-105 | SD-002 | ### Dart Flutter (Dart SDK) -| Feature | Status | Notes | -|---------|--------|-------| -| Arrow IPC | ✅ Supported | Requires dart-arrow package | -| JSON table | ✅ Supported | Human-readable format | -| File server upload | ✅ Supported | HTTP/HTTPS | -| File server download | ✅ Supported | HTTP/HTTPS | -| Size threshold | 500KB | Configurable | +| Feature | Status | Notes | Requirement ID | Solution Design Ref | +|---------|--------|-------|----------------|-------------------| +| Arrow IPC | ✅ Supported | Requires dart-arrow package | FR-002, FR-012 | SD-004, SD-005 | +| JSON table | ✅ Supported | Human-readable format | FR-002, FR-006 | SD-004, SD-005 | +| File server upload | ✅ Supported | HTTP/HTTPS | FR-008, FR-009 | SD-003 | +| File server download | ✅ Supported | HTTP/HTTPS | FR-010, NFR-202 | SD-003, SD-007 | +| Size threshold | 500KB | Configurable | FR-004, FR-005, NFR-104, NFR-105 | SD-002 | ### Dart Web (Dart SDK) -| Feature | Status | Notes | -|---------|--------|-------| -| Arrow IPC | ❌ Not supported | Apache Arrow not browser-compatible | -| JSON table | ✅ Supported | Only table type available in browser | -| File server upload | ✅ Supported | HTTP/HTTPS | -| File server download | ✅ Supported | HTTP/HTTPS | -| Size threshold | 500KB | Configurable | +| Feature | Status | Notes | Requirement ID | Solution Design Ref | +|---------|--------|-------|----------------|-------------------| +| Arrow IPC | ❌ Not supported | Apache Arrow not browser-compatible | FR-002 | SD-004, SD-005 | +| JSON table | ✅ Supported | Only table type available in browser | FR-002, FR-006 | SD-004, SD-005 | +| File server upload | ✅ Supported | HTTP/HTTPS | FR-008, FR-009 | SD-003 | +| File server download | ✅ Supported | HTTP/HTTPS | FR-010, NFR-202 | SD-003, SD-007 | +| Size threshold | 500KB | Configurable | FR-004, FR-005, NFR-104, NFR-105 | SD-002 | ### Rust -| Feature | Status | Notes | -|---------|--------|-------| -| Arrow IPC | ✅ Supported | Requires `arrow2` crate | -| JSON table | ✅ Supported | Uses `serde_json` | -| File server upload | ✅ Supported | HTTP/HTTPS via `reqwest` | -| File server download | ✅ Supported | HTTP/HTTPS via `reqwest` with retry | -| Size threshold | 500KB | Configurable | -| Async runtime | ✅ Supported | Uses `tokio` for async I/O | -| Type safety | ✅ Supported | Compile-time type checking via Rust enums | +| Feature | Status | Notes | Requirement ID | Solution Design Ref | +|---------|--------|-------|----------------|-------------------| +| Arrow IPC | ✅ Supported | Requires `arrow2` crate | FR-002, FR-012 | SD-004, SD-005 | +| JSON table | ✅ Supported | Uses `serde_json` | FR-002, FR-006 | SD-004, SD-005 | +| File server upload | ✅ Supported | HTTP/HTTPS via `reqwest` | FR-008, FR-009 | SD-003 | +| File server download | ✅ Supported | HTTP/HTTPS via `reqwest` with retry | FR-010, NFR-202 | SD-003, SD-007 | +| Size threshold | 500KB | Configurable | FR-004, FR-005, NFR-104, NFR-105 | SD-002 | +| Async runtime | ✅ Supported | Uses `tokio` for async I/O | FR-013, FR-014 | SD-006 | +| Type safety | ✅ Supported | Compile-time type checking via Rust enums | FR-006, FR-007 | SD-004 | ### MicroPython -| Feature | Status | Notes | -|---------|--------|-------| -| Arrow IPC | ❌ Not supported | Memory constraints | -| JSON table | ⚠️ Limited | Only direct transport | -| File server upload | ❌ Not implemented | Placeholder only | -| File server download | ❌ Not implemented | Placeholder only | -| Size threshold | 100KB | Hard limit enforced | -| Max payload | 50KB | Hard limit enforced | +| Feature | Status | Notes | Requirement ID | Solution Design Ref | +|---------|--------|-------|----------------|-------------------| +| Arrow IPC | ❌ Not supported | Memory constraints | FR-005 | SD-002 | +| JSON table | ⚠️ Limited | Only direct transport | FR-005, FR-006 | SD-002, SD-004 | +| File server upload | ❌ Not implemented | Placeholder only | FR-005 | SD-002 | +| File server download | ❌ Not implemented | Placeholder only | FR-005 | SD-002 | +| Size threshold | 100KB | Hard limit enforced | FR-005, NFR-106 | SD-002 | +| Max payload | 50KB | Hard limit enforced | FR-005 | SD-002 | --- ## Implementation Files -| File | Platform | Features | Notes | -|------|----------|----------|-------| -| [`src/msghandler.jl`](../src/msghandler.jl) | Julia | Full feature set, Arrow IPC, multiple dispatch | Ground truth implementation | -| [`src/msghandler_ssr.js`](../src/msghandler_ssr.js) | Node.js | Arrow IPC, async/await | Server-side JavaScript | -| [`src/msghandler_csr.js`](../src/msghandler_csr.js) | Browser | JSON table only | Client-side rendering | -| [`src/msghandler.py`](../src/msghandler.py) | Python | Arrow IPC, async/await | Desktop Python | -| [`src/msghandler.dart`](../src/msghandler.dart) | Dart | Full feature set, Arrow IPC, async/await | Desktop/Flutter/Web | -| [`src/msghandler.rs`](../src/msghandler.rs) | Rust | Full feature set, Arrow IPC, async/await, type-safe | Uses tokio + serde + arrow2 | -| [`src/msghandler_mpy.py`](../src/msghandler_mpy.py) | MicroPython | Limited to direct transport | Memory-constrained | +**Traceability**: FR-001 through FR-007 | SD-001 through SD-008 + +| File | Platform | Features | Notes | Requirement ID | Solution Design Ref | +|------|----------|----------|-------|----------------|-------------------| +| [`src/msghandler.jl`](../src/msghandler.jl) | Julia | Full feature set, Arrow IPC, multiple dispatch | Ground truth implementation | FR-001 through FR-014, NFR-101 through NFR-107 | SD-001 through SD-008 | +| [`src/msghandler_ssr.js`](../src/msghandler_ssr.js) | Node.js | Arrow IPC, async/await | Server-side JavaScript | FR-001 through FR-014, NFR-101 through NFR-107 | SD-001 through SD-008 | +| [`src/msghandler_csr.js`](../src/msghandler_csr.js) | Browser | JSON table only | Client-side rendering | FR-001 through FR-014, NFR-101 through NFR-107 | SD-001 through SD-008 | +| [`src/msghandler.py`](../src/msghandler.py) | Python | Arrow IPC, async/await | Desktop Python | FR-001 through FR-014, NFR-101 through NFR-107 | SD-001 through SD-008 | +| [`src/msghandler.dart`](../src/msghandler.dart) | Dart | Full feature set, Arrow IPC, async/await | Desktop/Flutter/Web | FR-001 through FR-014, NFR-101 through NFR-107 | SD-001 through SD-008 | +| [`src/msghandler.rs`](../src/msghandler.rs) | Rust | Full feature set, Arrow IPC, async/await, type-safe | Uses tokio + serde + arrow2 | FR-001 through FR-014, NFR-101 through NFR-107 | SD-001 through SD-008 | +| [`src/msghandler_mpy.py`](../src/msghandler_mpy.py) | MicroPython | Limited to direct transport | Memory-constrained | FR-005, FR-006 | SD-002, SD-004 | ### Browser Implementation Notes +**Traceability**: FR-002, FR-006 | SD-004, SD-005 + The browser implementation ([`src/msghandler_csr.js`](../src/msghandler_csr.js)) has the following constraints: -| Constraint | Reason | Workaround | -|------------|--------|------------| -| No Apache Arrow IPC | Browser-incompatible dependency | Use `jsontable` for tabular data | -| WebSocket only | Browser cannot use TCP directly | Use `ws://` or `wss://` broker URLs | -| Fetch API for HTTP | Browser fetch() API only | Compatible with Plik and other HTTP servers | +| Constraint | Reason | Workaround | Requirement ID | Solution Design Ref | +|------------|--------|------------|----------------|-------------------| +| No Apache Arrow IPC | Browser-incompatible dependency | Use `jsontable` for tabular data | FR-002 | SD-004, SD-005 | +| WebSocket only | Browser cannot use TCP directly | Use `ws://` or `wss://` broker URLs | FR-013, FR-014 | SD-006 | +| Fetch API for HTTP | Browser fetch() API only | Compatible with Plik and other HTTP servers | FR-008, FR-009 | SD-003 | ### Payload Type Availability by Platform @@ -975,40 +997,44 @@ flowchart TD ## Validation Rules +**Traceability**: FR-001 through FR-014, NFR-101 through NFR-405 | SD-001 through SD-008 + ### Envelope Validation -| Rule | Condition | Error Code | Requirement ID | -|------|-----------|------------|----------------| -| Required fields present | `correlation_id`, `msg_id`, `timestamp`, `send_to`, `payloads` | `INVALID_ENVELOPE` | FR-012, FR-013 | -| Valid UUID format | `correlation_id`, `msg_id`, `sender_id`, `receiver_id` | `INVALID_ENVELOPE` | FR-011, FR-012, NFR-401 | -| Valid timestamp format | ISO 8601 UTC | `INVALID_ENVELOPE` | FR-012, NFR-401 | -| Non-empty payloads array | `length(payloads) > 0` | `INVALID_ENVELOPE` | FR-012, FR-013 | +| Rule | Condition | Error Code | Requirement ID | Solution Design Ref | +|------|-----------|------------|----------------|-------------------| +| Required fields present | `correlation_id`, `msg_id`, `timestamp`, `send_to`, `payloads` | `INVALID_ENVELOPE` | FR-012, FR-013 | SD-001, SD-006 | +| Valid UUID format | `correlation_id`, `msg_id`, `sender_id`, `receiver_id` | `INVALID_ENVELOPE` | FR-011, FR-012, NFR-401 | SD-001, SD-008 | +| Valid timestamp format | ISO 8601 UTC | `INVALID_ENVELOPE` | FR-012, NFR-401 | SD-001, SD-008 | +| Non-empty payloads array | `length(payloads) > 0` | `INVALID_ENVELOPE` | FR-012, FR-013 | SD-001, SD-005 | ### Payload Validation -| Rule | Condition | Error Code | Requirement ID | -|------|-----------|------------|----------------| -| Valid payload_type | Must be in `payload_type` enum | `INVALID_PAYLOAD_TYPE` | FR-001, FR-002, FR-003, FR-006 | -| Valid transport | Must be `direct` or `link` | `INVALID_TRANSPORT` | FR-003, FR-004, FR-006 | -| Valid encoding | Must match payload_type and transport | `INVALID_TRANSPORT` | FR-001, FR-002, FR-003, FR-012 | -| Positive size | `size > 0` | `INVALID_PAYLOAD` | FR-003, FR-004, NFR-104, NFR-105 | -| Valid Base64 for direct | `data` matches Base64 pattern | `DESERIALIZATION_ERROR` | FR-001, FR-002, FR-003, FR-012 | -| Valid URL for link | `data` matches HTTP(S) URL pattern | `DOWNLOAD_FAILED` | FR-008, FR-009, FR-010 | +| Rule | Condition | Error Code | Requirement ID | Solution Design Ref | +|------|-----------|------------|----------------|-------------------| +| Valid payload_type | Must be in `payload_type` enum | `INVALID_PAYLOAD_TYPE` | FR-001, FR-002, FR-003, FR-006 | SD-004, SD-005 | +| Valid transport | Must be `direct` or `link` | `INVALID_TRANSPORT` | FR-003, FR-004, FR-006 | SD-001, SD-002 | +| Valid encoding | Must match payload_type and transport | `INVALID_TRANSPORT` | FR-001, FR-002, FR-003, FR-012 | SD-004, SD-005 | +| Positive size | `size > 0` | `INVALID_PAYLOAD` | FR-003, FR-004, NFR-104, NFR-105 | SD-001, SD-002 | +| Valid Base64 for direct | `data` matches Base64 pattern | `DESERIALIZATION_ERROR` | FR-001, FR-002, FR-003, FR-012 | SD-004, SD-005 | +| Valid URL for link | `data` matches HTTP(S) URL pattern | `DOWNLOAD_FAILED` | FR-008, FR-009, FR-010 | SD-003, SD-007 | --- ## Test Contracts +**Traceability**: FR-001 through FR-014, NFR-101 through NFR-107 | SD-001 through SD-008 + ### Unit Test Validation -| Test | Input | Expected Output | Notes | Requirement ID | -|------|-------|-----------------|-------|----------------| -| Text round-trip | `("msg", "Hello", "text")` | `("msg", "Hello", "text")` | String serialization | FR-001, FR-012, NFR-101, NFR-102 | -| Dictionary round-trip | `("data", {"key": "value"}, "dictionary")` | `("data", {"key": "value"}, "dictionary")` | JSON object round-trip | FR-002, FR-012, NFR-101, NFR-102 | -| Arrow table round-trip | `("table", arrow_table_data, "arrowtable")` | `("table", arrow_table_data, "arrowtable")` | Arrow IPC round-trip | FR-002, FR-012, NFR-101, NFR-102 | -| JSON table round-trip | `("table", [{"a":1},{"b":2}], "jsontable")` | `("table", [{"a":1},{"b":2}], "jsontable")` | JSON array of objects | FR-001, FR-002, FR-006, FR-012 | -| Mixed payloads | `[("msg", "Hello", "text"), ("imgname", bytes, "binary")]` | `[("msg", "Hello", "text"), ("imgname", bytes, "binary")]` | Multiple payload types | FR-006, FR-007 | -| Large payload | `("data", rand(10_000_000), "arrowtable")` | `("data", URL, "arrowtable")` with link transport | File server upload | FR-003, FR-004, FR-008, FR-009, NFR-104, NFR-105 | +| Test | Input | Expected Output | Notes | Requirement ID | Solution Design Ref | +|------|-------|-----------------|-------|----------------|-------------------| +| Text round-trip | `("msg", "Hello", "text")` | `("msg", "Hello", "text")` | String serialization | FR-001, FR-012, NFR-101, NFR-102 | SD-004, SD-005 | +| Dictionary round-trip | `("data", {"key": "value"}, "dictionary")` | `("data", {"key": "value"}, "dictionary")` | JSON object round-trip | FR-002, FR-012, NFR-101, NFR-102 | SD-004, SD-005 | +| Arrow table round-trip | `("table", arrow_table_data, "arrowtable")` | `("table", arrow_table_data, "arrowtable")` | Arrow IPC round-trip | FR-002, FR-012, NFR-101, NFR-102 | SD-004, SD-005 | +| JSON table round-trip | `("table", [{"a":1},{"b":2}], "jsontable")` | `("table", [{"a":1},{"b":2}], "jsontable")` | JSON array of objects | FR-001, FR-002, FR-006, FR-012 | SD-004, SD-005 | +| Mixed payloads | `[("msg", "Hello", "text"), ("imgname", bytes, "binary")]` | `[("msg", "Hello", "text"), ("imgname", bytes, "binary")]` | Multiple payload types | FR-006, FR-007 | SD-004 | +| Large payload | `("data", rand(10_000_000), "arrowtable")` | `("data", URL, "arrowtable")` with link transport | File server upload | FR-003, FR-004, FR-008, FR-009, NFR-104, NFR-105 | SD-001, SD-002, SD-003 | **Platform-Specific Notes:** - **Julia**: Use `Dict`, `Vector{Dict}`, or convert `DataFrame` to dictionary for testing @@ -1018,23 +1044,23 @@ flowchart TD ### Integration Test Scenarios -| Scenario | Platforms | Payloads | Size Mix | Transport | Expected Result | Requirement ID | -|----------|-----------|----------|----------|-----------|-----------------|----------------| -| Single text (small) | All | `text` | Small | direct | Round-trip successful | FR-001, FR-012, NFR-101, NFR-102 | -| Single dictionary (small) | All | `dictionary` | Small | direct | Round-trip successful | FR-002, FR-012, NFR-101, NFR-102 | -| Single arrow table (small) | Julia/JS/Python | `arrowtable` | Small | direct | Arrow IPC round-trip | FR-002, FR-012, NFR-101, NFR-102 | -| Single JSON table (small) | All | `jsontable` | Small | direct | Dictionary array round-trip | FR-001, FR-002, FR-006, FR-012 | -| Single image (small) | All | `image` | Small | direct | Binary round-trip | FR-001, FR-006, FR-012 | -| Single audio (small) | All | `audio` | Small | direct | Binary round-trip | FR-001, FR-006, FR-012 | -| Single video (small) | All | `video` | Small | direct | Binary round-trip | FR-001, FR-006, FR-012 | -| Single binary (small) | All | `binary` | Small | direct | Binary round-trip | FR-001, FR-006, FR-012 | -| Single text (large) | All | `text` | Large | link | File server upload/download | FR-003, FR-004, FR-008, FR-009, NFR-104, NFR-105 | -| Single JSON table (large) | All | `jsontable` | Large | link | File server upload/download | FR-003, FR-004, FR-008, FR-009, NFR-104, NFR-105 | -| Single image (large) | All | `image` | Large | link | File server upload/download | FR-003, FR-004, FR-008, FR-009, NFR-104, NFR-105 | -| **Ultimate Test** | Julia/JS/Python | `text` (small) + `dictionary` (small) + `arrowtable` (small) + `jsontable` (small) + `image` (small) + `audio` (small) + `video` (small) + `binary` (small) + `text` (large) + `dictionary` (large) + `arrowtable` (large) + `jsontable` (large) + `image` (large) | Mixed | direct/link | All payloads preserved with correct transport | FR-001, FR-002, FR-003, FR-004, FR-005, FR-006, FR-007, FR-008, FR-009, FR-010, FR-011, FR-012, FR-013, FR-014 | -| **Ultimate Test** | MicroPython | `text` (small) + `dictionary` (small) + `text` (large) + `dictionary` (large) | Mixed | direct | Limited to text/dictionary with direct transport only | FR-005, FR-006, FR-012 | -| Cross-platform JSON table | All | `jsontable` | Small | direct | Dictionary array round-trip | FR-001, FR-002, FR-006, FR-012 | -| MicroPython ↔ Desktop | MicroPython ↔ Desktop | `text`/`dictionary` | Small | direct | Limited payload types | FR-005, FR-006, FR-012 | +| Scenario | Platforms | Payloads | Size Mix | Transport | Expected Result | Requirement ID | Solution Design Ref | +|----------|-----------|----------|----------|-----------|-----------------|----------------|-------------------| +| Single text (small) | All | `text` | Small | direct | Round-trip successful | FR-001, FR-012, NFR-101, NFR-102 | SD-004, SD-005 | +| Single dictionary (small) | All | `dictionary` | Small | direct | Round-trip successful | FR-002, FR-012, NFR-101, NFR-102 | SD-004, SD-005 | +| Single arrow table (small) | Julia/JS/Python | `arrowtable` | Small | direct | Arrow IPC round-trip | FR-002, FR-012, NFR-101, NFR-102 | SD-004, SD-005 | +| Single JSON table (small) | All | `jsontable` | Small | direct | Dictionary array round-trip | FR-001, FR-002, FR-006, FR-012 | SD-004, SD-005 | +| Single image (small) | All | `image` | Small | direct | Binary round-trip | FR-001, FR-006, FR-012 | SD-004, SD-005 | +| Single audio (small) | All | `audio` | Small | direct | Binary round-trip | FR-001, FR-006, FR-012 | SD-004, SD-005 | +| Single video (small) | All | `video` | Small | direct | Binary round-trip | FR-001, FR-006, FR-012 | SD-004, SD-005 | +| Single binary (small) | All | `binary` | Small | direct | Binary round-trip | FR-001, FR-006, FR-012 | SD-004, SD-005 | +| Single text (large) | All | `text` | Large | link | File server upload/download | FR-003, FR-004, FR-008, FR-009, NFR-104, NFR-105 | SD-001, SD-002, SD-003 | +| Single JSON table (large) | All | `jsontable` | Large | link | File server upload/download | FR-003, FR-004, FR-008, FR-009, NFR-104, NFR-105 | SD-001, SD-002, SD-003 | +| Single image (large) | All | `image` | Large | link | File server upload/download | FR-003, FR-004, FR-008, FR-009, NFR-104, NFR-105 | SD-001, SD-002, SD-003 | +| **Ultimate Test** | Julia/JS/Python | All types | Mixed | direct/link | All payloads preserved with correct transport | FR-001 through FR-014, NFR-101 through NFR-107 | SD-001 through SD-008 | +| **Ultimate Test** | MicroPython | `text`/`dictionary` | Mixed | direct | Limited to text/dictionary with direct transport only | FR-005, FR-006, FR-012 | SD-002, SD-004 | +| Cross-platform JSON table | All | `jsontable` | Small | direct | Dictionary array round-trip | FR-001, FR-002, FR-006, FR-012 | SD-004, SD-005 | +| MicroPython ↔ Desktop | MicroPython ↔ Desktop | `text`/`dictionary` | Small | direct | Limited payload types | FR-005, FR-006, FR-012 | SD-002, SD-004 | | Desktop ↔ Desktop (all combos) | Julia↔JS↔Python | All types | Small/Large | direct/link | Full compatibility | FR-001, FR-002, FR-003, FR-004, FR-005, FR-006, FR-007, FR-012, FR-013, FR-014 | --- @@ -1043,26 +1069,26 @@ flowchart TD ### Required Dependencies by Platform -| Platform | Package | Version | Purpose | -|----------|---------|---------|---------| -| Julia | JSON.jl | Latest | JSON serialization | -| Julia | Arrow.jl | Latest | Arrow IPC support | -| Julia | HTTP.jl | Latest | HTTP file server | -| Julia | UUIDs.jl | Latest | UUID generation | -| Node.js | node-fetch | Latest | HTTP file server | -| Browser | - | - | Transport-agnostic (caller provides) | -| Python | aiohttp | Latest | HTTP file server | -| Python | pyarrow | Latest | Arrow IPC support | -| Dart | http | Latest | HTTP file server | -| Dart | uuid | Latest | UUID generation | -| Dart | dart-arrow | Latest | Arrow IPC support (Desktop/Flutter) | -| Rust | serde | Latest | JSON serialization | -| Rust | serde_json | Latest | JSON handling | -| Rust | tokio | Latest | Async runtime | -| Rust | reqwest | Latest | HTTP file server | -| Rust | uuid | Latest | UUID generation | -| Rust | arrow2 | Latest | Arrow IPC support | -| MicroPython | builtin | N/A | Limited implementation | +| Platform | Package | Version | Purpose | Requirement ID | Solution Design Ref | +|----------|---------|---------|---------|----------------|-------------------| +| Julia | JSON.jl | Latest | JSON serialization | FR-012, NFR-101, NFR-102 | SD-005 | +| Julia | Arrow.jl | Latest | Arrow IPC support | FR-002, FR-012 | SD-004, SD-005 | +| Julia | HTTP.jl | Latest | HTTP file server | FR-008, FR-009 | SD-003 | +| Julia | UUIDs.jl | Latest | UUID generation | FR-011, NFR-401 | SD-008 | +| Node.js | node-fetch | Latest | HTTP file server | FR-008, FR-009 | SD-003 | +| Browser | - | - | Transport-agnostic (caller provides) | FR-013, FR-014 | SD-006 | +| Python | aiohttp | Latest | HTTP file server | FR-008, FR-009 | SD-003 | +| Python | pyarrow | Latest | Arrow IPC support | FR-002, FR-012 | SD-004, SD-005 | +| Dart | http | Latest | HTTP file server | FR-008, FR-009 | SD-003 | +| Dart | uuid | Latest | UUID generation | FR-011, NFR-401 | SD-008 | +| Dart | dart-arrow | Latest | Arrow IPC support | FR-002, FR-012 | SD-004, SD-005 | +| Rust | serde | Latest | JSON serialization | FR-012, NFR-101, NFR-102 | SD-005 | +| Rust | serde_json | Latest | JSON handling | FR-012, NFR-101, NFR-102 | SD-005 | +| Rust | tokio | Latest | Async runtime | FR-013, FR-014 | SD-006 | +| Rust | reqwest | Latest | HTTP file server | FR-008, FR-009 | SD-003 | +| Rust | uuid | Latest | UUID generation | FR-011, NFR-401 | SD-008 | +| Rust | arrow2 | Latest | Arrow IPC support | FR-002, FR-012 | SD-004, SD-005 | +| MicroPython | builtin | N/A | Limited implementation | FR-005, FR-006 | SD-002 | ### Optional Dependencies @@ -1098,72 +1124,84 @@ flowchart TD ## References +**ASG Framework Reference**: This specification follows the ASG Framework v8 pillars. Each specification item must cite: +- **Requirement ID(s)** from requirements.md (e.g., FR-001, NFR-201) — defines *what* the system must do +- **Solution Design reference(s)** from solution-design.md (e.g., SD-2.1, SD-3.3) — defines *how* and *why* this technical approach was chosen + ### 20.1 Documentation Artifacts -| Document | Purpose | Requirements Traceability | -|----------|---------|--------------------------| -| [`docs/requirements.md`](./requirements.md) | Business requirements and user stories | FR-001 through FR-014, NFR-101 through NFR-405 | -| [`docs/specification.md`](./specification.md) | Technical contract for msghandler | This document | -| [`docs/ui-specification.md`](./ui-specification.md) | UI specification for client applications | UI components for data entry and display | -| [`docs/walkthrough.md`](./walkthrough.md) | End-to-end system flow | Traceability from user journey to technical implementation | -| [`docs/architecture.md`](./architecture.md) | System architecture diagrams | Component interaction and data flow | -| [`docs/validation.md`](./validation.md) | CI/CD validation rules | Contract testing and spec compliance | -| [`docs/runbook.md`](./runbook.md) | Operational runbook | Deployment, scaling, and troubleshooting | +| Document | Purpose | Requirements Traceability | Solution Design Traceability | +|----------|---------|--------------------------|------------------------------| +| [`docs/requirements.md`](./requirements.md) | Business requirements and user stories | FR-001 through FR-014, NFR-101 through NFR-405 | SD-001 through SD-008 | +| [`docs/solution-design.md`](./solution-design.md) | Technical solution approach | FR-001 through FR-014, NFR-101 through NFR-405 | SD-001 through SD-008 | +| [`docs/specification.md`](./specification.md) | Technical contract for msghandler | FR-001 through FR-014, NFR-101 through NFR-405 | SD-001 through SD-008 | +| [`docs/ui-specification.md`](./ui-specification.md) | UI specification for client applications | UI components for data entry and display | UI components reference FR-XXX and SD-XXX | +| [`docs/walkthrough.md`](./walkthrough.md) | End-to-end system flow | Traceability from user journey to technical implementation | Full flow validation against SD-XXX | +| [`docs/architecture.md`](./architecture.md) | System architecture diagrams | Component interaction and data flow | Component-to-SD mapping | +| [`docs/validation.md`](./validation.md) | CI/CD validation rules | Contract testing and spec compliance | Validation gates for SD-XXX | +| [`docs/runbook.md`](./runbook.md) | Operational runbook | Deployment, scaling, and troubleshooting | Operation-to-SD mapping | ### 20.2 Implementation Files -| File | Platform | Features | Requirements Traceability | -|------|----------|----------|--------------------------| -| [`src/msghandler.jl`](../src/msghandler.jl) | Julia | Full feature set, Arrow IPC, multiple dispatch | FR-001 through FR-014, NFR-101 through NFR-405 | -| [`src/msghandler_ssr.js`](../src/msghandler_ssr.js) | Node.js | Arrow IPC, async/await | FR-001 through FR-014, NFR-101 through NFR-405 | -| [`src/msghandler_csr.js`](../src/msghandler_csr.js) | Browser | JSON table only | FR-001 through FR-014, NFR-101 through NFR-405 | -| [`src/msghandler.py`](../src/msghandler.py) | Python | Arrow IPC, async/await | FR-001 through FR-014, NFR-101 through NFR-405 | -| [`src/msghandler.dart`](../src/msghandler.dart) | Dart | Full feature set, Arrow IPC, async/await | FR-001 through FR-014, NFR-101 through NFR-405 | -| [`src/msghandler.rs`](../src/msghandler.rs) | Rust | Full feature set, Arrow IPC, async/await, type-safe | FR-001 through FR-014, NFR-101 through NFR-405 | -| [`src/msghandler_mpy.py`](../src/msghandler_mpy.py) | MicroPython | Limited to direct transport | FR-005, FR-006, FR-012 | +| File | Platform | Features | Requirements Traceability | Solution Design Traceability | +|------|----------|----------|--------------------------|------------------------------| +| [`src/msghandler.jl`](../src/msghandler.jl) | Julia | Full feature set, Arrow IPC, multiple dispatch | FR-001 through FR-014, NFR-101 through NFR-405 | SD-001 through SD-008 | +| [`src/msghandler_ssr.js`](../src/msghandler_ssr.js) | Node.js | Arrow IPC, async/await | FR-001 through FR-014, NFR-101 through NFR-405 | SD-001 through SD-008 | +| [`src/msghandler_csr.js`](../src/msghandler_csr.js) | Browser | JSON table only | FR-001 through FR-014, NFR-101 through NFR-405 | SD-001 through SD-008 | +| [`src/msghandler.py`](../src/msghandler.py) | Python | Arrow IPC, async/await | FR-001 through FR-014, NFR-101 through NFR-405 | SD-001 through SD-008 | +| [`src/msghandler.dart`](../src/msghandler.dart) | Dart | Full feature set, Arrow IPC, async/await | FR-001 through FR-014, NFR-101 through NFR-405 | SD-001 through SD-008 | +| [`src/msghandler.rs`](../src/msghandler.rs) | Rust | Full feature set, Arrow IPC, async/await, type-safe | FR-001 through FR-014, NFR-101 through NFR-405 | SD-001 through SD-008 | +| [`src/msghandler_mpy.py`](../src/msghandler_mpy.py) | MicroPython | Limited to direct transport | FR-005, FR-006, FR-012 | SD-002, SD-004 | ### 20.3 External Dependencies -| Platform | Package | Version | Purpose | Requirements Traceability | -|----------|---------|---------|---------|--------------------------| -| Julia | JSON.jl | Latest | JSON serialization | FR-012, NFR-101, NFR-102 | -| Julia | Arrow.jl | Latest | Arrow IPC support | FR-002, FR-012 | -| Julia | HTTP.jl | Latest | HTTP file server | FR-008, FR-009 | -| Julia | UUIDs.jl | Latest | UUID generation | FR-011, NFR-401 | -| Node.js | node-fetch | Latest | HTTP file server | FR-008, FR-009 | -| Browser | - | - | Transport-agnostic (caller provides) | FR-013, FR-014 | -| Python | aiohttp | Latest | HTTP file server | FR-008, FR-009 | -| Python | pyarrow | Latest | Arrow IPC support | FR-002, FR-012 | -| Dart | http | Latest | HTTP file server | FR-008, FR-009 | -| Dart | uuid | Latest | UUID generation | FR-011, NFR-401 | -| Dart | dart-arrow | Latest | Arrow IPC support | FR-002, FR-012 | -| Rust | serde | Latest | JSON serialization | FR-012, NFR-101, NFR-102 | -| Rust | serde_json | Latest | JSON handling | FR-012, NFR-101, NFR-102 | -| Rust | tokio | Latest | Async runtime | FR-013, FR-014 | -| Rust | reqwest | Latest | HTTP file server | FR-008, FR-009 | -| Rust | uuid | Latest | UUID generation | FR-011, NFR-401 | -| Rust | arrow2 | Latest | Arrow IPC support | FR-002, FR-012 | -| MicroPython | builtin | N/A | Limited implementation | FR-005, FR-006 | +| Platform | Package | Version | Purpose | Requirements Traceability | Solution Design Traceability | +|----------|---------|---------|---------|--------------------------|------------------------------| +| Julia | JSON.jl | Latest | JSON serialization | FR-012, NFR-101, NFR-102 | SD-005 | +| Julia | Arrow.jl | Latest | Arrow IPC support | FR-002, FR-012 | SD-004, SD-005 | +| Julia | HTTP.jl | Latest | HTTP file server | FR-008, FR-009 | SD-003 | +| Julia | UUIDs.jl | Latest | UUID generation | FR-011, NFR-401 | SD-008 | +| Node.js | node-fetch | Latest | HTTP file server | FR-008, FR-009 | SD-003 | +| Browser | - | - | Transport-agnostic (caller provides) | FR-013, FR-014 | SD-006 | +| Python | aiohttp | Latest | HTTP file server | FR-008, FR-009 | SD-003 | +| Python | pyarrow | Latest | Arrow IPC support | FR-002, FR-012 | SD-004, SD-005 | +| Dart | http | Latest | HTTP file server | FR-008, FR-009 | SD-003 | +| Dart | uuid | Latest | UUID generation | FR-011, NFR-401 | SD-008 | +| Dart | dart-arrow | Latest | Arrow IPC support | FR-002, FR-012 | SD-004, SD-005 | +| Rust | serde | Latest | JSON serialization | FR-012, NFR-101, NFR-102 | SD-005 | +| Rust | serde_json | Latest | JSON handling | FR-012, NFR-101, NFR-102 | SD-005 | +| Rust | tokio | Latest | Async runtime | FR-013, FR-014 | SD-006 | +| Rust | reqwest | Latest | HTTP file server | FR-008, FR-009 | SD-003 | +| Rust | uuid | Latest | UUID generation | FR-011, NFR-401 | SD-008 | +| Rust | arrow2 | Latest | Arrow IPC support | FR-002, FR-012 | SD-004, SD-005 | +| MicroPython | builtin | N/A | Limited implementation | FR-005, FR-006 | SD-002 | --- -## 21. Change Log +## Change Log -| Date | Version | Changes | Requirement ID(s) | -|------|---------|---------|-------------------| -| 2026-05-15 | 1.3.0 | Made transport layer agnostic | All | -| - | - | Removed NATS-specific dependencies and references from all docs | All | -| - | - | Updated all NATS references to generic "transport layer"/"message broker" | All | -| - | - | Removed NATS client packages from dependencies tables | All | -| 2026-05-13 | 1.2.0 | Aligned with ground truth implementation (src/msghandler.jl) | All | -| - | - | Updated smartpack signatures: removed is_publish, nats_connection; added sender_name | FR-001 through FR-014 | -| - | - | Updated smartunpack signatures: takes msg_json_str::String instead of msg | FR-001 through FR-014 | -| - | - | Removed publishMessage function and NATSClient/NATSConnectionPool classes from browser section | FR-013, FR-014 | -| - | - | Added plik_oneshot_upload(filepath) overload to file server interface | FR-008, FR-009 | -| - | - | Fixed SIZE_THRESHOLD default to 500,000 bytes | FR-003, FR-004 | -| 2026-03-23 | 1.1.0 | Updated to ASG Framework specification guidelines | All | -| 2026-03-15 | 1.1.0 | Browser connection management | FR-001 through FR-014 | -| 2026-03-13 | 1.0.0 | Initial specification | FR-001 through FR-014, NFR-101 through NFR-405 | +| Date | Version | Changes | Requirement ID(s) | Solution Design Ref(s) | +|------|---------|---------|-------------------|------------------------| +| 2026-05-22 | 1.3.0 | Updated to ASG Framework v8 pillars | All | All SD-XXX | +| - | - | Added Solution Design references to all specification items | All | All SD-XXX | +| - | - | Updated version to 1.3.0 to match requirements and solution-design | All | - | +| - | - | Added ASG Framework alignment section | All | - | +| 2026-05-15 | 1.3.0 | Made transport layer agnostic | All | SD-006 | +| - | - | Removed all NATS-specific dependencies (NATS.jl, nats, nats-py, nats.ws) | All | SD-006 | +| - | - | Updated docs to reference generic message broker/transport | All | SD-006 | +| - | - | broker_url is now metadata only, not used for active connections | All | SD-006 | +| 2026-03-15 | 1.1.0 | Browser connection management | FR-001 through FR-014 | SD-006 | +| - | - | Added NATSClient class with keepAlive support | FR-013, FR-014 | SD-006 | +| - | - | Added NATSConnectionPool for connection reuse | FR-013, FR-014 | SD-006 | +| - | - | Added publishMessage function with closeConnection option | FR-013, FR-014 | SD-006 | +| - | - | Added nats.ws to browser dependencies | FR-013, FR-014 | SD-006 | +| 2026-03-13 | 1.0.0 | Initial specification | FR-001 through FR-014, NFR-101 through NFR-405 | SD-001 through SD-008 | +| - | - | Message envelope schema defined | FR-012, FR-013 | SD-001, SD-006 | +| - | - | Payload schema with transport modes | FR-001, FR-002, FR-003, FR-004 | SD-001, SD-002 | +| - | - | Enumerations for payload_type, transport, encoding | FR-001, FR-002, FR-003, FR-006 | SD-004, SD-005 | +| - | - | Size thresholds for desktop/MicroPython | FR-004, FR-005 | SD-002 | +| - | - | Error codes and validation rules | FR-001 through FR-010 | SD-001 through SD-007 | +| - | - | API contracts for all platforms | FR-001 through FR-014 | SD-001 through SD-008 | ---