update
This commit is contained in:
@@ -1,10 +1,11 @@
|
||||
# Specification: msghandler
|
||||
|
||||
**Version**: 1.2.0
|
||||
**Date**: 2026-05-13
|
||||
**Version**: 1.3.0
|
||||
**Date**: 2026-05-22
|
||||
**Status**: Active
|
||||
**Ground Truth**: [`src/msghandler.jl`](../src/msghandler.jl)
|
||||
**Specification Format**: JSON Schema + AsyncAPI
|
||||
**Specification Format**: JSON Schema + AsyncAPI
|
||||
**ASG Framework Alignment**: v8 pillars - Requirements → Solution Design → Specification → Walkthrough → Implementation Plan → Validation → Runbook
|
||||
|
||||
---
|
||||
|
||||
@@ -20,26 +21,26 @@ This specification serves as the single source of truth for:
|
||||
|
||||
### 1.1 Requirements Traceability
|
||||
|
||||
| Specification Section | Requirement ID(s) | Description |
|
||||
|----------------------|-------------------|-------------|
|
||||
| Section 2 (Message Envelope) | FR-012, FR-013, NFR-101, NFR-102 | Message envelope structure and validation |
|
||||
| Section 3 (Payload Schema) | FR-001, FR-002, FR-003, FR-004, NFR-101, NFR-102 | Payload structure and field definitions |
|
||||
| Section 4 (Payload Format) | FR-006, FR-007 | Tuple format for smartpack() |
|
||||
| Section 5 (Enumerations) | FR-003, FR-004, FR-006, NFR-101 | Enumerations for transport and encoding |
|
||||
| Section 6 (Transport Protocols) | FR-003, FR-004, NFR-104, NFR-105 | Direct and link transport protocols |
|
||||
| Section 7 (Size Thresholds) | FR-004, FR-005, NFR-104, NFR-105 | Size thresholds for transport selection |
|
||||
| Section 8 (Topic Convention) | FR-013, FR-014 | Topic/subject naming patterns |
|
||||
| Section 9 (Error Handling) | FR-010, FR-011, NFR-201, NFR-202, NFR-203 | Error codes and exception handling |
|
||||
| Section 10 (Serialization Rules) | FR-001, FR-002, FR-003, FR-012, NFR-101, NFR-102 | Serialization and encoding rules |
|
||||
| Section 11 (API Contract) | FR-001, FR-002, FR-003, FR-004, FR-005, FR-006, FR-007, FR-008, FR-009, FR-010, FR-011, FR-012, FR-013, FR-014 | Function signatures for all platforms |
|
||||
| Section 12 (File Server Interface) | FR-008, FR-009, FR-010 | Upload and download handler contracts |
|
||||
| Section 13 (Platform-Specific Constraints) | FR-005, FR-006, NFR-106, NFR-107 | Platform-specific feature support |
|
||||
| Section 14 (Implementation Files) | FR-001, FR-002, FR-003, FR-004, FR-005, FR-006, FR-007 | Implementation file mapping |
|
||||
| Section 15 (Message Flow) | FR-001, FR-002, FR-003, FR-004, FR-005, FR-006, FR-007, FR-008, FR-009, FR-010, FR-011, FR-012, FR-013, FR-014 | Mermaid diagrams for send/receive flows |
|
||||
| Section 16 (Validation Rules) | FR-001, FR-002, FR-003, FR-004, FR-005, FR-006, FR-007, FR-008, FR-009, FR-010, FR-011, FR-012, FR-013, FR-014 | Envelope and payload validation rules |
|
||||
| Section 17 (Test Contracts) | FR-001, FR-002, FR-003, FR-004, FR-005, FR-006, FR-007, FR-008, FR-009, FR-010, FR-011, FR-012, FR-013, FR-014 | Unit and integration test scenarios |
|
||||
| Section 18 (Dependencies) | FR-001, FR-002, FR-003, FR-004, FR-005, FR-006, FR-007, FR-008, FR-009, FR-010, FR-011, FR-012, FR-013, FR-014 | Platform-specific dependencies |
|
||||
| Section 19 (Change Log) | N/A | Version history and changes |
|
||||
| Specification Section | Requirement ID(s) | Solution Design Ref(s) | Description |
|
||||
|----------------------|-------------------|------------------------|-------------|
|
||||
| Section 2 (Message Envelope) | FR-012, FR-013, NFR-101, NFR-102 | SD-008 | Message envelope structure and validation |
|
||||
| Section 3 (Payload Schema) | FR-001, FR-002, FR-003, FR-004, NFR-101, NFR-102 | SD-001, SD-005 | Payload structure and field definitions |
|
||||
| Section 4 (Payload Format) | FR-006, FR-007 | SD-004 | Tuple format for smartpack() |
|
||||
| Section 5 (Enumerations) | FR-003, FR-004, FR-006, NFR-101 | SD-001, SD-002, SD-005 | Enumerations for transport and encoding |
|
||||
| Section 6 (Transport Protocols) | FR-003, FR-004, NFR-104, NFR-105 | SD-001, SD-002 | Direct and link transport protocols |
|
||||
| Section 7 (Size Thresholds) | FR-004, FR-005, NFR-104, NFR-105 | SD-002 | Size thresholds for transport selection |
|
||||
| Section 8 (Topic Convention) | FR-013, FR-014 | SD-006 | Topic/subject naming patterns |
|
||||
| Section 9 (Error Handling) | FR-010, FR-011, NFR-201, NFR-202, NFR-203 | SD-003, SD-007 | Error codes and exception handling |
|
||||
| Section 10 (Serialization Rules) | FR-001, FR-002, FR-003, FR-012, NFR-101, NFR-102 | SD-005 | Serialization and encoding rules |
|
||||
| Section 11 (API Contract) | FR-001 through FR-014, NFR-101 through NFR-107 | SD-001 through SD-008 | Function signatures for all platforms |
|
||||
| Section 12 (File Server Interface) | FR-008, FR-009, FR-010 | SD-003, SD-007 | Upload and download handler contracts |
|
||||
| Section 13 (Platform-Specific Constraints) | FR-005, FR-006, NFR-106, NFR-107 | SD-004, SD-006 | Platform-specific feature support |
|
||||
| Section 14 (Implementation Files) | FR-001 through FR-007 | - | Implementation file mapping |
|
||||
| Section 15 (Message Flow) | FR-001 through FR-014, NFR-101 through NFR-107 | SD-001 through SD-008 | Mermaid diagrams for send/receive flows |
|
||||
| Section 16 (Validation Rules) | FR-001 through FR-014, NFR-101 through NFR-405 | SD-001 through SD-008 | Envelope and payload validation rules |
|
||||
| Section 17 (Test Contracts) | FR-001 through FR-014, NFR-101 through NFR-107 | SD-001 through SD-008 | Unit and integration test scenarios |
|
||||
| Section 18 (Dependencies) | FR-001 through FR-014, NFR-101 through NFR-107 | SD-001 through SD-008 | Platform-specific dependencies |
|
||||
| Section 19 (Change Log) | - | - | Version history and changes |
|
||||
|
||||
---
|
||||
|
||||
@@ -90,22 +91,22 @@ This specification serves as the single source of truth for:
|
||||
|
||||
### Field Definitions
|
||||
|
||||
| Field | Type | Required | Validation | Description | Requirement ID |
|
||||
|-------|------|----------|------------|-------------|----------------|
|
||||
| `correlation_id` | `string` | Yes | UUID v4 format | Track message flow across distributed systems | FR-011, NFR-401 |
|
||||
| `msg_id` | `string` | Yes | UUID v4 format | Unique identifier for this specific message | FR-012, NFR-401 |
|
||||
| `timestamp` | `string` | Yes | ISO 8601 UTC | Message publication timestamp | FR-012, NFR-401 |
|
||||
| `send_to` | `string` | Yes | Non-empty string | Topic/subject to publish the message to | FR-013 |
|
||||
| `msg_purpose` | `string` | Yes | Enum | Purpose of the message | FR-013 |
|
||||
| `sender_name` | `string` | Yes | Non-empty string | Name of the sender application | FR-013 |
|
||||
| `sender_id` | `string` | Yes | UUID v4 format | Unique identifier for the sender | FR-013 |
|
||||
| `receiver_name` | `string` | Yes | Any string | Name of the receiver (empty = broadcast) | FR-013 |
|
||||
| `receiver_id` | `string` | Yes | Any string | UUID of the receiver (empty = broadcast) | FR-013 |
|
||||
| `reply_to` | `string` | Yes | Any string | Topic where receiver should reply | FR-013 |
|
||||
| `reply_to_msg_id` | `string` | Yes | Any string | Message ID this message is replying to | FR-013 |
|
||||
| `broker_url` | `string` | Yes | Valid URL | Broker URL for the transport layer | FR-013 |
|
||||
| `metadata` | `object` | No | Any JSON object | Message-level metadata | NFR-401 |
|
||||
| `payloads` | `array` | Yes | Non-empty array | List of payload objects | FR-012, FR-013 |
|
||||
| Field | Type | Required | Validation | Description | Requirement ID | Solution Design Ref |
|
||||
|-------|------|----------|------------|-------------|----------------|-------------------|
|
||||
| `correlation_id` | `string` | Yes | UUID v4 format | Track message flow across distributed systems | FR-011, NFR-401 | SD-008 |
|
||||
| `msg_id` | `string` | Yes | UUID v4 format | Unique identifier for this specific message | FR-012, NFR-401 | SD-001 |
|
||||
| `timestamp` | `string` | Yes | ISO 8601 UTC | Message publication timestamp | FR-012, NFR-401 | SD-001 |
|
||||
| `send_to` | `string` | Yes | Non-empty string | Topic/subject to publish the message to | FR-013 | SD-006 |
|
||||
| `msg_purpose` | `string` | Yes | Enum | Purpose of the message | FR-013 | SD-006 |
|
||||
| `sender_name` | `string` | Yes | Non-empty string | Name of the sender application | FR-013 | SD-006 |
|
||||
| `sender_id` | `string` | Yes | UUID v4 format | Unique identifier for the sender | FR-013 | SD-006 |
|
||||
| `receiver_name` | `string` | Yes | Any string | Name of the receiver (empty = broadcast) | FR-013 | SD-006 |
|
||||
| `receiver_id` | `string` | Yes | Any string | UUID of the receiver (empty = broadcast) | FR-013 | SD-006 |
|
||||
| `reply_to` | `string` | Yes | Any string | Topic where receiver should reply | FR-013 | SD-006 |
|
||||
| `reply_to_msg_id` | `string` | Yes | Any string | Message ID this message is replying to | FR-013 | SD-006 |
|
||||
| `broker_url` | `string` | Yes | Valid URL | Broker URL for the transport layer | FR-013 | SD-006 |
|
||||
| `metadata` | `object` | No | Any JSON object | Message-level metadata | NFR-401 | SD-001, SD-008 |
|
||||
| `payloads` | `array` | Yes | Non-empty array | List of payload objects | FR-012, FR-013 | SD-001, SD-005 |
|
||||
|
||||
---
|
||||
|
||||
@@ -128,16 +129,16 @@ This specification serves as the single source of truth for:
|
||||
|
||||
### Payload Field Definitions
|
||||
|
||||
| Field | Type | Required | Validation | Description | Requirement ID |
|
||||
|-------|------|----------|------------|-------------|----------------|
|
||||
| `id` | `string` | Yes | UUID v4 format | Unique identifier for this payload | FR-012 |
|
||||
| `dataname` | `string` | Yes | Non-empty string | Name of the payload (e.g., `login_image`, `user_data`) | FR-001, FR-002, FR-003 |
|
||||
| `payload_type` | `string` | Yes | Enum | Type of payload (see `payload_type` enum) | FR-001, FR-002, FR-003, FR-006 |
|
||||
| `transport` | `string` | Yes | Enum | Transport method: `direct` or `link` | FR-003, FR-004, NFR-104, NFR-105 |
|
||||
| `encoding` | `string` | Yes | Enum | Encoding method (see `encoding` enum) | FR-001, FR-002, FR-003, FR-012 |
|
||||
| `size` | `integer` | Yes | Positive integer | Size of the payload in bytes | FR-003, FR-004, NFR-104, NFR-105 |
|
||||
| `data` | `string` or `URL` | Yes | Base64 string or URL | Payload data (base64 for direct, URL for link) | FR-003, FR-004, FR-008, FR-009, FR-010 |
|
||||
| `metadata` | `object` | No | Any JSON object | Payload-level metadata | NFR-401 |
|
||||
| Field | Type | Required | Validation | Description | Requirement ID | Solution Design Ref |
|
||||
|-------|------|----------|------------|-------------|----------------|-------------------|
|
||||
| `id` | `string` | Yes | UUID v4 format | Unique identifier for this payload | FR-012 | SD-001 |
|
||||
| `dataname` | `string` | Yes | Non-empty string | Name of the payload (e.g., `login_image`, `user_data`) | FR-001, FR-002, FR-003 | SD-004 |
|
||||
| `payload_type` | `string` | Yes | Enum | Type of payload (see `payload_type` enum) | FR-001, FR-002, FR-003, FR-006 | SD-004, SD-005 |
|
||||
| `transport` | `string` | Yes | Enum | Transport method: `direct` or `link` | FR-003, FR-004, NFR-104, NFR-105 | SD-001, SD-002 |
|
||||
| `encoding` | `string` | Yes | Enum | Encoding method (see `encoding` enum) | FR-001, FR-002, FR-003, FR-012 | SD-005 |
|
||||
| `size` | `integer` | Yes | Positive integer | Size of the payload in bytes | FR-003, FR-004, NFR-104, NFR-105 | SD-002 |
|
||||
| `data` | `string` or `URL` | Yes | Base64 string or URL | Payload data (base64 for direct, URL for link) | FR-003, FR-004, FR-008, FR-009, FR-010 | SD-001, SD-003 |
|
||||
| `metadata` | `object` | No | Any JSON object | Payload-level metadata | NFR-401 | SD-008 |
|
||||
|
||||
---
|
||||
|
||||
@@ -227,30 +228,30 @@ await smartpack("/agent/v1/process", data)
|
||||
|
||||
| Value | Description | Supported Platforms | Encoding Options |
|
||||
|-------|-------------|---------------------|------------------|
|
||||
| `text` | Plain text string | All | `base64` |
|
||||
| `dictionary` | JSON object/dictionary | All | `base64`, `json` |
|
||||
| `arrowtable` | Apache Arrow IPC table | Desktop (Julia/Python/Node.js/Dart) | `base64`, `arrow-ipc` |
|
||||
| `jsontable` | JSON array of objects | All (including Browser/Dart Web) | `base64`, `json` |
|
||||
| `image` | Binary image data | All | `base64` |
|
||||
| `audio` | Binary audio data | All | `base64` |
|
||||
| `video` | Binary video data | All | `base64` |
|
||||
| `binary` | Generic binary data | All | `base64` |
|
||||
| `text` | Plain text string | All | `base64` | FR-001, FR-012 | SD-004, SD-005 |
|
||||
| `dictionary` | JSON object/dictionary | All | `base64`, `json` | FR-002, FR-012 | SD-004, SD-005 |
|
||||
| `arrowtable` | Apache Arrow IPC table | Desktop (Julia/Python/Node.js/Dart) | `base64`, `arrow-ipc` | FR-002 | SD-004, SD-005 |
|
||||
| `jsontable` | JSON array of objects | All (including Browser/Dart Web) | `base64`, `json` | FR-002, FR-006 | SD-004, SD-005 |
|
||||
| `image` | Binary image data | All | `base64` | FR-001, FR-006 | SD-004, SD-005 |
|
||||
| `audio` | Binary audio data | All | `base64` | FR-001, FR-006 | SD-004, SD-005 |
|
||||
| `video` | Binary video data | All | `base64` | FR-001, FR-006 | SD-004, SD-005 |
|
||||
| `binary` | Generic binary data | All | `base64` | FR-001, FR-006 | SD-004, SD-005 |
|
||||
|
||||
### `transport` Enum
|
||||
|
||||
| Value | Description | Data Format | Use Case |
|
||||
|-------|-------------|-------------|----------|
|
||||
| `direct` | Payload sent directly via the transport layer | Base64-encoded string | Payloads < size_threshold |
|
||||
| `link` | Payload uploaded to file server | HTTP URL | Payloads ≥ size_threshold |
|
||||
| `direct` | Payload sent directly via the transport layer | Base64-encoded string | Payloads < size_threshold | FR-003, FR-004, NFR-104, NFR-105 | SD-001, SD-002 |
|
||||
| `link` | Payload uploaded to file server | HTTP URL | Payloads ≥ size_threshold | FR-003, FR-004, NFR-104, NFR-105 | SD-001, SD-002 |
|
||||
|
||||
### `encoding` Enum
|
||||
|
||||
| Value | Description | Payload Types |
|
||||
|-------|-------------|---------------|
|
||||
| `none` | No additional encoding | Link transport URLs |
|
||||
| `base64` | Base64 encoding | Text, binary, image, audio, video |
|
||||
| `json` | JSON encoding | Dictionary, jsontable |
|
||||
| `arrow-ipc` | Apache Arrow IPC format | Arrowtable |
|
||||
| `none` | No additional encoding | Link transport URLs | FR-003, FR-004 | SD-001, SD-002 |
|
||||
| `base64` | Base64 encoding | Text, binary, image, audio, video | FR-001, FR-002, FR-003, FR-012 | SD-005 |
|
||||
| `json` | JSON encoding | Dictionary, jsontable | FR-002, FR-006 | SD-004, SD-005 |
|
||||
| `arrow-ipc` | Apache Arrow IPC format | Arrowtable | FR-002 | SD-004, SD-005 |
|
||||
|
||||
---
|
||||
|
||||
@@ -300,15 +301,15 @@ When `transport = "link"`, the `data` field contains a URL pointing to the uploa
|
||||
|
||||
### Desktop Platforms (Julia/JS/Python)
|
||||
|
||||
| Platform | Size Threshold | Notes |
|
||||
|----------|----------------|-------|
|
||||
| Desktop | 500,000 bytes (0.5MB) | Default threshold |
|
||||
| Platform | Size Threshold | Notes | Requirement ID | Solution Design Ref |
|
||||
|----------|----------------|-------|----------------|-------------------|
|
||||
| Desktop | 500,000 bytes (0.5MB) | Default threshold | FR-004, FR-005, NFR-104, NFR-105 | SD-002 |
|
||||
|
||||
### MicroPython Platform
|
||||
|
||||
| Platform | Size Threshold | Maximum Payload | Notes |
|
||||
|----------|----------------|-----------------|-------|
|
||||
| MicroPython | 100,000 bytes (100KB) | 50,000 bytes | Hard limit due to memory constraints |
|
||||
| Platform | Size Threshold | Maximum Payload | Notes | Requirement ID | Solution Design Ref |
|
||||
|----------|----------------|-----------------|-------|----------------|-------------------|
|
||||
| MicroPython | 100,000 bytes (100KB) | 50,000 bytes | Hard limit due to memory constraints | FR-005, NFR-106 | SD-002 |
|
||||
|
||||
---
|
||||
|
||||
@@ -321,16 +322,16 @@ When `transport = "link"`, the `data` field contains a URL pointing to the uploa
|
||||
```
|
||||
|
||||
**Examples**:
|
||||
- `/agent/wine/api/v1/prompt` - AI agent prompt endpoint
|
||||
- `/chat/user/v1/message` - User chat message
|
||||
- `/system/worker/v1/status` - Worker status update
|
||||
- `/agent/wine/api/v1/prompt` - AI agent prompt endpoint | FR-013 | SD-006
|
||||
- `/chat/user/v1/message` - User chat message | FR-013 | SD-006
|
||||
- `/system/worker/v1/status` - Worker status update | FR-013 | SD-006
|
||||
|
||||
### Subject Wildcards
|
||||
|
||||
| Wildcard | Description | Example |
|
||||
|----------|-------------|---------|
|
||||
| `*` | Single-level wildcard | `/chat/user/v1/*` matches `/chat/user/v1/message` |
|
||||
| `>` | Multi-level wildcard | `/chat/user/v1/>` matches all `/chat/user/v1/*` subjects |
|
||||
| `*` | Single-level wildcard | `/chat/user/v1/*` matches `/chat/user/v1/message` | FR-013 | SD-006 |
|
||||
| `>` | Multi-level wildcard | `/chat/user/v1/>` matches all `/chat/user/v1/*` subjects | FR-013 | SD-006 |
|
||||
|
||||
---
|
||||
|
||||
@@ -352,23 +353,23 @@ When `transport = "link"`, the `data` field contains a URL pointing to the uploa
|
||||
|
||||
| Code | HTTP Status | Description | Recovery | Requirement ID |
|
||||
|------|-------------|-------------|----------|----------------|
|
||||
| `INVALID_ENVELOPE` | 400 | Message envelope validation failed | Fix envelope structure | FR-012, FR-013, FR-014 |
|
||||
| `INVALID_PAYLOAD_TYPE` | 400 | Unsupported payload type | Use supported payload_type | FR-001, FR-002, FR-003, FR-006 |
|
||||
| `INVALID_TRANSPORT` | 400 | Unsupported transport type | Use `direct` or `link` | FR-003, FR-004, FR-006 |
|
||||
| `UPLOAD_FAILED` | 500 | File server upload failed | Retry or use direct transport | FR-008, FR-009 |
|
||||
| `DOWNLOAD_FAILED` | 503 | File server download failed | Retry with exponential backoff | FR-010, FR-011, NFR-201, NFR-202 |
|
||||
| `TRANSPORT_CONNECTION_FAILED` | 503 | Transport connection failed | Check broker/server availability | FR-013, FR-014, NFR-201, NFR-203 |
|
||||
| `DESERIALIZATION_ERROR` | 500 | Payload deserialization failed | Check payload_type matches data | FR-001, FR-002, FR-003, FR-012 |
|
||||
| `SIZE_EXCEEDED` | 413 | Payload exceeds maximum size | Split payload or use link transport | FR-003, FR-004, FR-005, NFR-104, NFR-105 |
|
||||
| `INVALID_ENVELOPE` | 400 | Message envelope validation failed | Fix envelope structure | FR-012, FR-013, FR-014 | SD-001, SD-006 |
|
||||
| `INVALID_PAYLOAD_TYPE` | 400 | Unsupported payload type | Use supported payload_type | FR-001, FR-002, FR-003, FR-006 | SD-004, SD-005 |
|
||||
| `INVALID_TRANSPORT` | 400 | Unsupported transport type | Use `direct` or `link` | FR-003, FR-004, FR-006 | SD-001, SD-002 |
|
||||
| `UPLOAD_FAILED` | 500 | File server upload failed | Retry or use direct transport | FR-008, FR-009 | SD-003 |
|
||||
| `DOWNLOAD_FAILED` | 503 | File server download failed | Retry with exponential backoff | FR-010, FR-011, NFR-201, NFR-202 | SD-003, SD-007 |
|
||||
| `TRANSPORT_CONNECTION_FAILED` | 503 | Transport connection failed | Check broker/server availability | FR-013, FR-014, NFR-201, NFR-203 | SD-006 |
|
||||
| `DESERIALIZATION_ERROR` | 500 | Payload deserialization failed | Check payload_type matches data | FR-001, FR-002, FR-003, FR-012 | SD-004, SD-005 |
|
||||
| `SIZE_EXCEEDED` | 413 | Payload exceeds maximum size | Split payload or use link transport | FR-003, FR-004, FR-005, NFR-104, NFR-105 | SD-001, SD-002 |
|
||||
|
||||
### Exception Handling
|
||||
|
||||
| Scenario | Handler | Retry Policy | Requirement ID |
|
||||
|----------|---------|--------------|----------------|
|
||||
| File server unavailable | Retry up to 5 times | Exponential backoff (100ms → 5000ms) | FR-010, NFR-202 |
|
||||
| Transport publish failure | Handled by caller | Caller's retry logic | FR-013, FR-014, NFR-201, NFR-203 |
|
||||
| Deserialization error | Log correlation ID and throw | No retry (data corruption) | FR-001, FR-002, FR-003, FR-012, NFR-401 |
|
||||
| Memory overflow (MicroPython) | Reject payloads >50KB | No retry (client-side check) | FR-005, NFR-106 |
|
||||
| File server unavailable | Retry up to 5 times | Exponential backoff (100ms → 5000ms) | FR-010, NFR-202 | SD-003, SD-007 |
|
||||
| Transport publish failure | Handled by caller | Caller's retry logic | FR-013, FR-014, NFR-201, NFR-203 | SD-006 |
|
||||
| Deserialization error | Log correlation ID and throw | No retry (data corruption) | FR-001, FR-002, FR-003, FR-012, NFR-401 | SD-004, SD-005, SD-008 |
|
||||
| Memory overflow (MicroPython) | Reject payloads >50KB | No retry (client-side check) | FR-005, NFR-106 | SD-004 |
|
||||
|
||||
---
|
||||
|
||||
@@ -411,6 +412,11 @@ When `transport = "link"`, the `data` field contains a URL pointing to the uploa
|
||||
|
||||
## API Contract
|
||||
|
||||
| Platform | Function | Requirements | Solution Design Ref |
|
||||
|----------|----------|--------------|-------------------|
|
||||
| All | `smartpack()` | FR-001 through FR-014, NFR-101 through NFR-107 | SD-001 through SD-008 |
|
||||
| All | `smartunpack()` | FR-001 through FR-014, NFR-101 through NFR-107 | SD-001 through SD-008 |
|
||||
|
||||
### `smartpack` Function Signature
|
||||
|
||||
#### Julia
|
||||
@@ -638,6 +644,10 @@ pub struct MsgEnvelopeV1 {
|
||||
|
||||
### `smartunpack` Function Signature
|
||||
|
||||
**Note**: Input is the JSON string payload from the transport subscription. Returns `(envelope::msg_envelope_v1, payloads::Array{Tuple{String, Any, String}, 1})`.
|
||||
|
||||
**Traceability**: FR-001 through FR-014, NFR-101 through NFR-107 | SD-001 through SD-008
|
||||
|
||||
#### Julia
|
||||
|
||||
```julia
|
||||
@@ -757,6 +767,8 @@ pub struct smartunpackOptions {
|
||||
|
||||
## File Server Interface
|
||||
|
||||
**Traceability**: FR-008, FR-009, FR-010, NFR-202 | SD-003, SD-007
|
||||
|
||||
### Upload Handler Contract
|
||||
|
||||
**Function Signature**:
|
||||
@@ -785,15 +797,19 @@ function fileserver_upload_handler(
|
||||
```
|
||||
|
||||
**Required Keys**:
|
||||
| Key | Type | Description |
|
||||
|-----|------|-------------|
|
||||
| `status` | `integer` | HTTP response status code |
|
||||
| `uploadid` | `string` | Upload session identifier |
|
||||
| `fileid` | `string` | File identifier within session |
|
||||
| `url` | `string` | Full download URL |
|
||||
| Key | Type | Description | Requirement ID | Solution Design Ref |
|
||||
|-----|------|-------------|----------------|-------------------|
|
||||
| `status` | `integer` | HTTP response status code | FR-008, FR-009 | SD-003 |
|
||||
| `uploadid` | `string` | Upload session identifier | FR-008 | SD-003 |
|
||||
| `fileid` | `string` | File identifier within session | FR-008 | SD-003 |
|
||||
| `url` | `string` | Full download URL | FR-008, FR-009, FR-010 | SD-003, SD-007 |
|
||||
|
||||
### Download Handler Contract
|
||||
|
||||
**Function Signature**:
|
||||
|
||||
**Traceability**: FR-010, NFR-202 | SD-003, SD-007
|
||||
|
||||
**Function Signature**:
|
||||
```julia
|
||||
function fileserver_download_handler(
|
||||
@@ -815,102 +831,108 @@ function fileserver_download_handler(
|
||||
|
||||
## Platform-Specific Constraints
|
||||
|
||||
**Traceability**: FR-005, FR-006, NFR-106, NFR-107 | SD-002, SD-004
|
||||
|
||||
### Desktop (Julia/Python/Node.js/Dart)
|
||||
|
||||
| Feature | Status | Notes |
|
||||
|---------|--------|-------|
|
||||
| Arrow IPC | ✅ Supported | Requires Arrow.jl/pyarrow/dart-arrow |
|
||||
| JSON table | ✅ Supported | Human-readable format |
|
||||
| File server upload | ✅ Supported | HTTP/HTTPS |
|
||||
| File server download | ✅ Supported | HTTP/HTTPS |
|
||||
| Size threshold | 500KB | Configurable |
|
||||
| Feature | Status | Notes | Requirement ID | Solution Design Ref |
|
||||
|---------|--------|-------|----------------|-------------------|
|
||||
| Arrow IPC | ✅ Supported | Requires Arrow.jl/pyarrow/dart-arrow | FR-002, FR-012 | SD-004, SD-005 |
|
||||
| JSON table | ✅ Supported | Human-readable format | FR-002, FR-006 | SD-004, SD-005 |
|
||||
| File server upload | ✅ Supported | HTTP/HTTPS | FR-008, FR-009 | SD-003 |
|
||||
| File server download | ✅ Supported | HTTP/HTTPS | FR-010, NFR-202 | SD-003, SD-007 |
|
||||
| Size threshold | 500KB | Configurable | FR-004, FR-005, NFR-104, NFR-105 | SD-002 |
|
||||
|
||||
### Browser (JavaScript)
|
||||
|
||||
| Feature | Status | Notes |
|
||||
|---------|--------|-------|
|
||||
| Arrow IPC | ❌ Not supported | Apache Arrow not browser-compatible |
|
||||
| JSON table | ✅ Supported | Only table type available in browser |
|
||||
| File server upload | ✅ Supported | HTTP/HTTPS |
|
||||
| File server download | ✅ Supported | HTTP/HTTPS |
|
||||
| Size threshold | 500KB | Configurable |
|
||||
| Feature | Status | Notes | Requirement ID | Solution Design Ref |
|
||||
|---------|--------|-------|----------------|-------------------|
|
||||
| Arrow IPC | ❌ Not supported | Apache Arrow not browser-compatible | FR-002 | SD-004, SD-005 |
|
||||
| JSON table | ✅ Supported | Only table type available in browser | FR-002, FR-006 | SD-004, SD-005 |
|
||||
| File server upload | ✅ Supported | HTTP/HTTPS | FR-008, FR-009 | SD-003 |
|
||||
| File server download | ✅ Supported | HTTP/HTTPS | FR-010, NFR-202 | SD-003, SD-007 |
|
||||
| Size threshold | 500KB | Configurable | FR-004, FR-005, NFR-104, NFR-105 | SD-002 |
|
||||
|
||||
### Dart Desktop (Dart SDK)
|
||||
|
||||
| Feature | Status | Notes |
|
||||
|---------|--------|-------|
|
||||
| Arrow IPC | ✅ Supported | Requires dart-arrow package |
|
||||
| JSON table | ✅ Supported | Human-readable format |
|
||||
| File server upload | ✅ Supported | HTTP/HTTPS |
|
||||
| File server download | ✅ Supported | HTTP/HTTPS |
|
||||
| Size threshold | 500KB | Configurable |
|
||||
| Feature | Status | Notes | Requirement ID | Solution Design Ref |
|
||||
|---------|--------|-------|----------------|-------------------|
|
||||
| Arrow IPC | ✅ Supported | Requires dart-arrow package | FR-002, FR-012 | SD-004, SD-005 |
|
||||
| JSON table | ✅ Supported | Human-readable format | FR-002, FR-006 | SD-004, SD-005 |
|
||||
| File server upload | ✅ Supported | HTTP/HTTPS | FR-008, FR-009 | SD-003 |
|
||||
| File server download | ✅ Supported | HTTP/HTTPS | FR-010, NFR-202 | SD-003, SD-007 |
|
||||
| Size threshold | 500KB | Configurable | FR-004, FR-005, NFR-104, NFR-105 | SD-002 |
|
||||
|
||||
### Dart Flutter (Dart SDK)
|
||||
|
||||
| Feature | Status | Notes |
|
||||
|---------|--------|-------|
|
||||
| Arrow IPC | ✅ Supported | Requires dart-arrow package |
|
||||
| JSON table | ✅ Supported | Human-readable format |
|
||||
| File server upload | ✅ Supported | HTTP/HTTPS |
|
||||
| File server download | ✅ Supported | HTTP/HTTPS |
|
||||
| Size threshold | 500KB | Configurable |
|
||||
| Feature | Status | Notes | Requirement ID | Solution Design Ref |
|
||||
|---------|--------|-------|----------------|-------------------|
|
||||
| Arrow IPC | ✅ Supported | Requires dart-arrow package | FR-002, FR-012 | SD-004, SD-005 |
|
||||
| JSON table | ✅ Supported | Human-readable format | FR-002, FR-006 | SD-004, SD-005 |
|
||||
| File server upload | ✅ Supported | HTTP/HTTPS | FR-008, FR-009 | SD-003 |
|
||||
| File server download | ✅ Supported | HTTP/HTTPS | FR-010, NFR-202 | SD-003, SD-007 |
|
||||
| Size threshold | 500KB | Configurable | FR-004, FR-005, NFR-104, NFR-105 | SD-002 |
|
||||
|
||||
### Dart Web (Dart SDK)
|
||||
|
||||
| Feature | Status | Notes |
|
||||
|---------|--------|-------|
|
||||
| Arrow IPC | ❌ Not supported | Apache Arrow not browser-compatible |
|
||||
| JSON table | ✅ Supported | Only table type available in browser |
|
||||
| File server upload | ✅ Supported | HTTP/HTTPS |
|
||||
| File server download | ✅ Supported | HTTP/HTTPS |
|
||||
| Size threshold | 500KB | Configurable |
|
||||
| Feature | Status | Notes | Requirement ID | Solution Design Ref |
|
||||
|---------|--------|-------|----------------|-------------------|
|
||||
| Arrow IPC | ❌ Not supported | Apache Arrow not browser-compatible | FR-002 | SD-004, SD-005 |
|
||||
| JSON table | ✅ Supported | Only table type available in browser | FR-002, FR-006 | SD-004, SD-005 |
|
||||
| File server upload | ✅ Supported | HTTP/HTTPS | FR-008, FR-009 | SD-003 |
|
||||
| File server download | ✅ Supported | HTTP/HTTPS | FR-010, NFR-202 | SD-003, SD-007 |
|
||||
| Size threshold | 500KB | Configurable | FR-004, FR-005, NFR-104, NFR-105 | SD-002 |
|
||||
|
||||
### Rust
|
||||
|
||||
| Feature | Status | Notes |
|
||||
|---------|--------|-------|
|
||||
| Arrow IPC | ✅ Supported | Requires `arrow2` crate |
|
||||
| JSON table | ✅ Supported | Uses `serde_json` |
|
||||
| File server upload | ✅ Supported | HTTP/HTTPS via `reqwest` |
|
||||
| File server download | ✅ Supported | HTTP/HTTPS via `reqwest` with retry |
|
||||
| Size threshold | 500KB | Configurable |
|
||||
| Async runtime | ✅ Supported | Uses `tokio` for async I/O |
|
||||
| Type safety | ✅ Supported | Compile-time type checking via Rust enums |
|
||||
| Feature | Status | Notes | Requirement ID | Solution Design Ref |
|
||||
|---------|--------|-------|----------------|-------------------|
|
||||
| Arrow IPC | ✅ Supported | Requires `arrow2` crate | FR-002, FR-012 | SD-004, SD-005 |
|
||||
| JSON table | ✅ Supported | Uses `serde_json` | FR-002, FR-006 | SD-004, SD-005 |
|
||||
| File server upload | ✅ Supported | HTTP/HTTPS via `reqwest` | FR-008, FR-009 | SD-003 |
|
||||
| File server download | ✅ Supported | HTTP/HTTPS via `reqwest` with retry | FR-010, NFR-202 | SD-003, SD-007 |
|
||||
| Size threshold | 500KB | Configurable | FR-004, FR-005, NFR-104, NFR-105 | SD-002 |
|
||||
| Async runtime | ✅ Supported | Uses `tokio` for async I/O | FR-013, FR-014 | SD-006 |
|
||||
| Type safety | ✅ Supported | Compile-time type checking via Rust enums | FR-006, FR-007 | SD-004 |
|
||||
|
||||
### MicroPython
|
||||
|
||||
| Feature | Status | Notes |
|
||||
|---------|--------|-------|
|
||||
| Arrow IPC | ❌ Not supported | Memory constraints |
|
||||
| JSON table | ⚠️ Limited | Only direct transport |
|
||||
| File server upload | ❌ Not implemented | Placeholder only |
|
||||
| File server download | ❌ Not implemented | Placeholder only |
|
||||
| Size threshold | 100KB | Hard limit enforced |
|
||||
| Max payload | 50KB | Hard limit enforced |
|
||||
| Feature | Status | Notes | Requirement ID | Solution Design Ref |
|
||||
|---------|--------|-------|----------------|-------------------|
|
||||
| Arrow IPC | ❌ Not supported | Memory constraints | FR-005 | SD-002 |
|
||||
| JSON table | ⚠️ Limited | Only direct transport | FR-005, FR-006 | SD-002, SD-004 |
|
||||
| File server upload | ❌ Not implemented | Placeholder only | FR-005 | SD-002 |
|
||||
| File server download | ❌ Not implemented | Placeholder only | FR-005 | SD-002 |
|
||||
| Size threshold | 100KB | Hard limit enforced | FR-005, NFR-106 | SD-002 |
|
||||
| Max payload | 50KB | Hard limit enforced | FR-005 | SD-002 |
|
||||
|
||||
---
|
||||
|
||||
## Implementation Files
|
||||
|
||||
| File | Platform | Features | Notes |
|
||||
|------|----------|----------|-------|
|
||||
| [`src/msghandler.jl`](../src/msghandler.jl) | Julia | Full feature set, Arrow IPC, multiple dispatch | Ground truth implementation |
|
||||
| [`src/msghandler_ssr.js`](../src/msghandler_ssr.js) | Node.js | Arrow IPC, async/await | Server-side JavaScript |
|
||||
| [`src/msghandler_csr.js`](../src/msghandler_csr.js) | Browser | JSON table only | Client-side rendering |
|
||||
| [`src/msghandler.py`](../src/msghandler.py) | Python | Arrow IPC, async/await | Desktop Python |
|
||||
| [`src/msghandler.dart`](../src/msghandler.dart) | Dart | Full feature set, Arrow IPC, async/await | Desktop/Flutter/Web |
|
||||
| [`src/msghandler.rs`](../src/msghandler.rs) | Rust | Full feature set, Arrow IPC, async/await, type-safe | Uses tokio + serde + arrow2 |
|
||||
| [`src/msghandler_mpy.py`](../src/msghandler_mpy.py) | MicroPython | Limited to direct transport | Memory-constrained |
|
||||
**Traceability**: FR-001 through FR-007 | SD-001 through SD-008
|
||||
|
||||
| File | Platform | Features | Notes | Requirement ID | Solution Design Ref |
|
||||
|------|----------|----------|-------|----------------|-------------------|
|
||||
| [`src/msghandler.jl`](../src/msghandler.jl) | Julia | Full feature set, Arrow IPC, multiple dispatch | Ground truth implementation | FR-001 through FR-014, NFR-101 through NFR-107 | SD-001 through SD-008 |
|
||||
| [`src/msghandler_ssr.js`](../src/msghandler_ssr.js) | Node.js | Arrow IPC, async/await | Server-side JavaScript | FR-001 through FR-014, NFR-101 through NFR-107 | SD-001 through SD-008 |
|
||||
| [`src/msghandler_csr.js`](../src/msghandler_csr.js) | Browser | JSON table only | Client-side rendering | FR-001 through FR-014, NFR-101 through NFR-107 | SD-001 through SD-008 |
|
||||
| [`src/msghandler.py`](../src/msghandler.py) | Python | Arrow IPC, async/await | Desktop Python | FR-001 through FR-014, NFR-101 through NFR-107 | SD-001 through SD-008 |
|
||||
| [`src/msghandler.dart`](../src/msghandler.dart) | Dart | Full feature set, Arrow IPC, async/await | Desktop/Flutter/Web | FR-001 through FR-014, NFR-101 through NFR-107 | SD-001 through SD-008 |
|
||||
| [`src/msghandler.rs`](../src/msghandler.rs) | Rust | Full feature set, Arrow IPC, async/await, type-safe | Uses tokio + serde + arrow2 | FR-001 through FR-014, NFR-101 through NFR-107 | SD-001 through SD-008 |
|
||||
| [`src/msghandler_mpy.py`](../src/msghandler_mpy.py) | MicroPython | Limited to direct transport | Memory-constrained | FR-005, FR-006 | SD-002, SD-004 |
|
||||
|
||||
### Browser Implementation Notes
|
||||
|
||||
**Traceability**: FR-002, FR-006 | SD-004, SD-005
|
||||
|
||||
The browser implementation ([`src/msghandler_csr.js`](../src/msghandler_csr.js)) has the following constraints:
|
||||
|
||||
| Constraint | Reason | Workaround |
|
||||
|------------|--------|------------|
|
||||
| No Apache Arrow IPC | Browser-incompatible dependency | Use `jsontable` for tabular data |
|
||||
| WebSocket only | Browser cannot use TCP directly | Use `ws://` or `wss://` broker URLs |
|
||||
| Fetch API for HTTP | Browser fetch() API only | Compatible with Plik and other HTTP servers |
|
||||
| Constraint | Reason | Workaround | Requirement ID | Solution Design Ref |
|
||||
|------------|--------|------------|----------------|-------------------|
|
||||
| No Apache Arrow IPC | Browser-incompatible dependency | Use `jsontable` for tabular data | FR-002 | SD-004, SD-005 |
|
||||
| WebSocket only | Browser cannot use TCP directly | Use `ws://` or `wss://` broker URLs | FR-013, FR-014 | SD-006 |
|
||||
| Fetch API for HTTP | Browser fetch() API only | Compatible with Plik and other HTTP servers | FR-008, FR-009 | SD-003 |
|
||||
|
||||
### Payload Type Availability by Platform
|
||||
|
||||
@@ -975,40 +997,44 @@ flowchart TD
|
||||
|
||||
## Validation Rules
|
||||
|
||||
**Traceability**: FR-001 through FR-014, NFR-101 through NFR-405 | SD-001 through SD-008
|
||||
|
||||
### Envelope Validation
|
||||
|
||||
| Rule | Condition | Error Code | Requirement ID |
|
||||
|------|-----------|------------|----------------|
|
||||
| Required fields present | `correlation_id`, `msg_id`, `timestamp`, `send_to`, `payloads` | `INVALID_ENVELOPE` | FR-012, FR-013 |
|
||||
| Valid UUID format | `correlation_id`, `msg_id`, `sender_id`, `receiver_id` | `INVALID_ENVELOPE` | FR-011, FR-012, NFR-401 |
|
||||
| Valid timestamp format | ISO 8601 UTC | `INVALID_ENVELOPE` | FR-012, NFR-401 |
|
||||
| Non-empty payloads array | `length(payloads) > 0` | `INVALID_ENVELOPE` | FR-012, FR-013 |
|
||||
| Rule | Condition | Error Code | Requirement ID | Solution Design Ref |
|
||||
|------|-----------|------------|----------------|-------------------|
|
||||
| Required fields present | `correlation_id`, `msg_id`, `timestamp`, `send_to`, `payloads` | `INVALID_ENVELOPE` | FR-012, FR-013 | SD-001, SD-006 |
|
||||
| Valid UUID format | `correlation_id`, `msg_id`, `sender_id`, `receiver_id` | `INVALID_ENVELOPE` | FR-011, FR-012, NFR-401 | SD-001, SD-008 |
|
||||
| Valid timestamp format | ISO 8601 UTC | `INVALID_ENVELOPE` | FR-012, NFR-401 | SD-001, SD-008 |
|
||||
| Non-empty payloads array | `length(payloads) > 0` | `INVALID_ENVELOPE` | FR-012, FR-013 | SD-001, SD-005 |
|
||||
|
||||
### Payload Validation
|
||||
|
||||
| Rule | Condition | Error Code | Requirement ID |
|
||||
|------|-----------|------------|----------------|
|
||||
| Valid payload_type | Must be in `payload_type` enum | `INVALID_PAYLOAD_TYPE` | FR-001, FR-002, FR-003, FR-006 |
|
||||
| Valid transport | Must be `direct` or `link` | `INVALID_TRANSPORT` | FR-003, FR-004, FR-006 |
|
||||
| Valid encoding | Must match payload_type and transport | `INVALID_TRANSPORT` | FR-001, FR-002, FR-003, FR-012 |
|
||||
| Positive size | `size > 0` | `INVALID_PAYLOAD` | FR-003, FR-004, NFR-104, NFR-105 |
|
||||
| Valid Base64 for direct | `data` matches Base64 pattern | `DESERIALIZATION_ERROR` | FR-001, FR-002, FR-003, FR-012 |
|
||||
| Valid URL for link | `data` matches HTTP(S) URL pattern | `DOWNLOAD_FAILED` | FR-008, FR-009, FR-010 |
|
||||
| Rule | Condition | Error Code | Requirement ID | Solution Design Ref |
|
||||
|------|-----------|------------|----------------|-------------------|
|
||||
| Valid payload_type | Must be in `payload_type` enum | `INVALID_PAYLOAD_TYPE` | FR-001, FR-002, FR-003, FR-006 | SD-004, SD-005 |
|
||||
| Valid transport | Must be `direct` or `link` | `INVALID_TRANSPORT` | FR-003, FR-004, FR-006 | SD-001, SD-002 |
|
||||
| Valid encoding | Must match payload_type and transport | `INVALID_TRANSPORT` | FR-001, FR-002, FR-003, FR-012 | SD-004, SD-005 |
|
||||
| Positive size | `size > 0` | `INVALID_PAYLOAD` | FR-003, FR-004, NFR-104, NFR-105 | SD-001, SD-002 |
|
||||
| Valid Base64 for direct | `data` matches Base64 pattern | `DESERIALIZATION_ERROR` | FR-001, FR-002, FR-003, FR-012 | SD-004, SD-005 |
|
||||
| Valid URL for link | `data` matches HTTP(S) URL pattern | `DOWNLOAD_FAILED` | FR-008, FR-009, FR-010 | SD-003, SD-007 |
|
||||
|
||||
---
|
||||
|
||||
## Test Contracts
|
||||
|
||||
**Traceability**: FR-001 through FR-014, NFR-101 through NFR-107 | SD-001 through SD-008
|
||||
|
||||
### Unit Test Validation
|
||||
|
||||
| Test | Input | Expected Output | Notes | Requirement ID |
|
||||
|------|-------|-----------------|-------|----------------|
|
||||
| Text round-trip | `("msg", "Hello", "text")` | `("msg", "Hello", "text")` | String serialization | FR-001, FR-012, NFR-101, NFR-102 |
|
||||
| Dictionary round-trip | `("data", {"key": "value"}, "dictionary")` | `("data", {"key": "value"}, "dictionary")` | JSON object round-trip | FR-002, FR-012, NFR-101, NFR-102 |
|
||||
| Arrow table round-trip | `("table", arrow_table_data, "arrowtable")` | `("table", arrow_table_data, "arrowtable")` | Arrow IPC round-trip | FR-002, FR-012, NFR-101, NFR-102 |
|
||||
| JSON table round-trip | `("table", [{"a":1},{"b":2}], "jsontable")` | `("table", [{"a":1},{"b":2}], "jsontable")` | JSON array of objects | FR-001, FR-002, FR-006, FR-012 |
|
||||
| Mixed payloads | `[("msg", "Hello", "text"), ("imgname", bytes, "binary")]` | `[("msg", "Hello", "text"), ("imgname", bytes, "binary")]` | Multiple payload types | FR-006, FR-007 |
|
||||
| Large payload | `("data", rand(10_000_000), "arrowtable")` | `("data", URL, "arrowtable")` with link transport | File server upload | FR-003, FR-004, FR-008, FR-009, NFR-104, NFR-105 |
|
||||
| Test | Input | Expected Output | Notes | Requirement ID | Solution Design Ref |
|
||||
|------|-------|-----------------|-------|----------------|-------------------|
|
||||
| Text round-trip | `("msg", "Hello", "text")` | `("msg", "Hello", "text")` | String serialization | FR-001, FR-012, NFR-101, NFR-102 | SD-004, SD-005 |
|
||||
| Dictionary round-trip | `("data", {"key": "value"}, "dictionary")` | `("data", {"key": "value"}, "dictionary")` | JSON object round-trip | FR-002, FR-012, NFR-101, NFR-102 | SD-004, SD-005 |
|
||||
| Arrow table round-trip | `("table", arrow_table_data, "arrowtable")` | `("table", arrow_table_data, "arrowtable")` | Arrow IPC round-trip | FR-002, FR-012, NFR-101, NFR-102 | SD-004, SD-005 |
|
||||
| JSON table round-trip | `("table", [{"a":1},{"b":2}], "jsontable")` | `("table", [{"a":1},{"b":2}], "jsontable")` | JSON array of objects | FR-001, FR-002, FR-006, FR-012 | SD-004, SD-005 |
|
||||
| Mixed payloads | `[("msg", "Hello", "text"), ("imgname", bytes, "binary")]` | `[("msg", "Hello", "text"), ("imgname", bytes, "binary")]` | Multiple payload types | FR-006, FR-007 | SD-004 |
|
||||
| Large payload | `("data", rand(10_000_000), "arrowtable")` | `("data", URL, "arrowtable")` with link transport | File server upload | FR-003, FR-004, FR-008, FR-009, NFR-104, NFR-105 | SD-001, SD-002, SD-003 |
|
||||
|
||||
**Platform-Specific Notes:**
|
||||
- **Julia**: Use `Dict`, `Vector{Dict}`, or convert `DataFrame` to dictionary for testing
|
||||
@@ -1018,23 +1044,23 @@ flowchart TD
|
||||
|
||||
### Integration Test Scenarios
|
||||
|
||||
| Scenario | Platforms | Payloads | Size Mix | Transport | Expected Result | Requirement ID |
|
||||
|----------|-----------|----------|----------|-----------|-----------------|----------------|
|
||||
| Single text (small) | All | `text` | Small | direct | Round-trip successful | FR-001, FR-012, NFR-101, NFR-102 |
|
||||
| Single dictionary (small) | All | `dictionary` | Small | direct | Round-trip successful | FR-002, FR-012, NFR-101, NFR-102 |
|
||||
| Single arrow table (small) | Julia/JS/Python | `arrowtable` | Small | direct | Arrow IPC round-trip | FR-002, FR-012, NFR-101, NFR-102 |
|
||||
| Single JSON table (small) | All | `jsontable` | Small | direct | Dictionary array round-trip | FR-001, FR-002, FR-006, FR-012 |
|
||||
| Single image (small) | All | `image` | Small | direct | Binary round-trip | FR-001, FR-006, FR-012 |
|
||||
| Single audio (small) | All | `audio` | Small | direct | Binary round-trip | FR-001, FR-006, FR-012 |
|
||||
| Single video (small) | All | `video` | Small | direct | Binary round-trip | FR-001, FR-006, FR-012 |
|
||||
| Single binary (small) | All | `binary` | Small | direct | Binary round-trip | FR-001, FR-006, FR-012 |
|
||||
| Single text (large) | All | `text` | Large | link | File server upload/download | FR-003, FR-004, FR-008, FR-009, NFR-104, NFR-105 |
|
||||
| Single JSON table (large) | All | `jsontable` | Large | link | File server upload/download | FR-003, FR-004, FR-008, FR-009, NFR-104, NFR-105 |
|
||||
| Single image (large) | All | `image` | Large | link | File server upload/download | FR-003, FR-004, FR-008, FR-009, NFR-104, NFR-105 |
|
||||
| **Ultimate Test** | Julia/JS/Python | `text` (small) + `dictionary` (small) + `arrowtable` (small) + `jsontable` (small) + `image` (small) + `audio` (small) + `video` (small) + `binary` (small) + `text` (large) + `dictionary` (large) + `arrowtable` (large) + `jsontable` (large) + `image` (large) | Mixed | direct/link | All payloads preserved with correct transport | FR-001, FR-002, FR-003, FR-004, FR-005, FR-006, FR-007, FR-008, FR-009, FR-010, FR-011, FR-012, FR-013, FR-014 |
|
||||
| **Ultimate Test** | MicroPython | `text` (small) + `dictionary` (small) + `text` (large) + `dictionary` (large) | Mixed | direct | Limited to text/dictionary with direct transport only | FR-005, FR-006, FR-012 |
|
||||
| Cross-platform JSON table | All | `jsontable` | Small | direct | Dictionary array round-trip | FR-001, FR-002, FR-006, FR-012 |
|
||||
| MicroPython ↔ Desktop | MicroPython ↔ Desktop | `text`/`dictionary` | Small | direct | Limited payload types | FR-005, FR-006, FR-012 |
|
||||
| Scenario | Platforms | Payloads | Size Mix | Transport | Expected Result | Requirement ID | Solution Design Ref |
|
||||
|----------|-----------|----------|----------|-----------|-----------------|----------------|-------------------|
|
||||
| Single text (small) | All | `text` | Small | direct | Round-trip successful | FR-001, FR-012, NFR-101, NFR-102 | SD-004, SD-005 |
|
||||
| Single dictionary (small) | All | `dictionary` | Small | direct | Round-trip successful | FR-002, FR-012, NFR-101, NFR-102 | SD-004, SD-005 |
|
||||
| Single arrow table (small) | Julia/JS/Python | `arrowtable` | Small | direct | Arrow IPC round-trip | FR-002, FR-012, NFR-101, NFR-102 | SD-004, SD-005 |
|
||||
| Single JSON table (small) | All | `jsontable` | Small | direct | Dictionary array round-trip | FR-001, FR-002, FR-006, FR-012 | SD-004, SD-005 |
|
||||
| Single image (small) | All | `image` | Small | direct | Binary round-trip | FR-001, FR-006, FR-012 | SD-004, SD-005 |
|
||||
| Single audio (small) | All | `audio` | Small | direct | Binary round-trip | FR-001, FR-006, FR-012 | SD-004, SD-005 |
|
||||
| Single video (small) | All | `video` | Small | direct | Binary round-trip | FR-001, FR-006, FR-012 | SD-004, SD-005 |
|
||||
| Single binary (small) | All | `binary` | Small | direct | Binary round-trip | FR-001, FR-006, FR-012 | SD-004, SD-005 |
|
||||
| Single text (large) | All | `text` | Large | link | File server upload/download | FR-003, FR-004, FR-008, FR-009, NFR-104, NFR-105 | SD-001, SD-002, SD-003 |
|
||||
| Single JSON table (large) | All | `jsontable` | Large | link | File server upload/download | FR-003, FR-004, FR-008, FR-009, NFR-104, NFR-105 | SD-001, SD-002, SD-003 |
|
||||
| Single image (large) | All | `image` | Large | link | File server upload/download | FR-003, FR-004, FR-008, FR-009, NFR-104, NFR-105 | SD-001, SD-002, SD-003 |
|
||||
| **Ultimate Test** | Julia/JS/Python | All types | Mixed | direct/link | All payloads preserved with correct transport | FR-001 through FR-014, NFR-101 through NFR-107 | SD-001 through SD-008 |
|
||||
| **Ultimate Test** | MicroPython | `text`/`dictionary` | Mixed | direct | Limited to text/dictionary with direct transport only | FR-005, FR-006, FR-012 | SD-002, SD-004 |
|
||||
| Cross-platform JSON table | All | `jsontable` | Small | direct | Dictionary array round-trip | FR-001, FR-002, FR-006, FR-012 | SD-004, SD-005 |
|
||||
| MicroPython ↔ Desktop | MicroPython ↔ Desktop | `text`/`dictionary` | Small | direct | Limited payload types | FR-005, FR-006, FR-012 | SD-002, SD-004 |
|
||||
| Desktop ↔ Desktop (all combos) | Julia↔JS↔Python | All types | Small/Large | direct/link | Full compatibility | FR-001, FR-002, FR-003, FR-004, FR-005, FR-006, FR-007, FR-012, FR-013, FR-014 |
|
||||
|
||||
---
|
||||
@@ -1043,26 +1069,26 @@ flowchart TD
|
||||
|
||||
### Required Dependencies by Platform
|
||||
|
||||
| Platform | Package | Version | Purpose |
|
||||
|----------|---------|---------|---------|
|
||||
| Julia | JSON.jl | Latest | JSON serialization |
|
||||
| Julia | Arrow.jl | Latest | Arrow IPC support |
|
||||
| Julia | HTTP.jl | Latest | HTTP file server |
|
||||
| Julia | UUIDs.jl | Latest | UUID generation |
|
||||
| Node.js | node-fetch | Latest | HTTP file server |
|
||||
| Browser | - | - | Transport-agnostic (caller provides) |
|
||||
| Python | aiohttp | Latest | HTTP file server |
|
||||
| Python | pyarrow | Latest | Arrow IPC support |
|
||||
| Dart | http | Latest | HTTP file server |
|
||||
| Dart | uuid | Latest | UUID generation |
|
||||
| Dart | dart-arrow | Latest | Arrow IPC support (Desktop/Flutter) |
|
||||
| Rust | serde | Latest | JSON serialization |
|
||||
| Rust | serde_json | Latest | JSON handling |
|
||||
| Rust | tokio | Latest | Async runtime |
|
||||
| Rust | reqwest | Latest | HTTP file server |
|
||||
| Rust | uuid | Latest | UUID generation |
|
||||
| Rust | arrow2 | Latest | Arrow IPC support |
|
||||
| MicroPython | builtin | N/A | Limited implementation |
|
||||
| Platform | Package | Version | Purpose | Requirement ID | Solution Design Ref |
|
||||
|----------|---------|---------|---------|----------------|-------------------|
|
||||
| Julia | JSON.jl | Latest | JSON serialization | FR-012, NFR-101, NFR-102 | SD-005 |
|
||||
| Julia | Arrow.jl | Latest | Arrow IPC support | FR-002, FR-012 | SD-004, SD-005 |
|
||||
| Julia | HTTP.jl | Latest | HTTP file server | FR-008, FR-009 | SD-003 |
|
||||
| Julia | UUIDs.jl | Latest | UUID generation | FR-011, NFR-401 | SD-008 |
|
||||
| Node.js | node-fetch | Latest | HTTP file server | FR-008, FR-009 | SD-003 |
|
||||
| Browser | - | - | Transport-agnostic (caller provides) | FR-013, FR-014 | SD-006 |
|
||||
| Python | aiohttp | Latest | HTTP file server | FR-008, FR-009 | SD-003 |
|
||||
| Python | pyarrow | Latest | Arrow IPC support | FR-002, FR-012 | SD-004, SD-005 |
|
||||
| Dart | http | Latest | HTTP file server | FR-008, FR-009 | SD-003 |
|
||||
| Dart | uuid | Latest | UUID generation | FR-011, NFR-401 | SD-008 |
|
||||
| Dart | dart-arrow | Latest | Arrow IPC support | FR-002, FR-012 | SD-004, SD-005 |
|
||||
| Rust | serde | Latest | JSON serialization | FR-012, NFR-101, NFR-102 | SD-005 |
|
||||
| Rust | serde_json | Latest | JSON handling | FR-012, NFR-101, NFR-102 | SD-005 |
|
||||
| Rust | tokio | Latest | Async runtime | FR-013, FR-014 | SD-006 |
|
||||
| Rust | reqwest | Latest | HTTP file server | FR-008, FR-009 | SD-003 |
|
||||
| Rust | uuid | Latest | UUID generation | FR-011, NFR-401 | SD-008 |
|
||||
| Rust | arrow2 | Latest | Arrow IPC support | FR-002, FR-012 | SD-004, SD-005 |
|
||||
| MicroPython | builtin | N/A | Limited implementation | FR-005, FR-006 | SD-002 |
|
||||
|
||||
### Optional Dependencies
|
||||
|
||||
@@ -1098,72 +1124,84 @@ flowchart TD
|
||||
|
||||
## References
|
||||
|
||||
**ASG Framework Reference**: This specification follows the ASG Framework v8 pillars. Each specification item must cite:
|
||||
- **Requirement ID(s)** from requirements.md (e.g., FR-001, NFR-201) — defines *what* the system must do
|
||||
- **Solution Design reference(s)** from solution-design.md (e.g., SD-2.1, SD-3.3) — defines *how* and *why* this technical approach was chosen
|
||||
|
||||
### 20.1 Documentation Artifacts
|
||||
|
||||
| Document | Purpose | Requirements Traceability |
|
||||
|----------|---------|--------------------------|
|
||||
| [`docs/requirements.md`](./requirements.md) | Business requirements and user stories | FR-001 through FR-014, NFR-101 through NFR-405 |
|
||||
| [`docs/specification.md`](./specification.md) | Technical contract for msghandler | This document |
|
||||
| [`docs/ui-specification.md`](./ui-specification.md) | UI specification for client applications | UI components for data entry and display |
|
||||
| [`docs/walkthrough.md`](./walkthrough.md) | End-to-end system flow | Traceability from user journey to technical implementation |
|
||||
| [`docs/architecture.md`](./architecture.md) | System architecture diagrams | Component interaction and data flow |
|
||||
| [`docs/validation.md`](./validation.md) | CI/CD validation rules | Contract testing and spec compliance |
|
||||
| [`docs/runbook.md`](./runbook.md) | Operational runbook | Deployment, scaling, and troubleshooting |
|
||||
| Document | Purpose | Requirements Traceability | Solution Design Traceability |
|
||||
|----------|---------|--------------------------|------------------------------|
|
||||
| [`docs/requirements.md`](./requirements.md) | Business requirements and user stories | FR-001 through FR-014, NFR-101 through NFR-405 | SD-001 through SD-008 |
|
||||
| [`docs/solution-design.md`](./solution-design.md) | Technical solution approach | FR-001 through FR-014, NFR-101 through NFR-405 | SD-001 through SD-008 |
|
||||
| [`docs/specification.md`](./specification.md) | Technical contract for msghandler | FR-001 through FR-014, NFR-101 through NFR-405 | SD-001 through SD-008 |
|
||||
| [`docs/ui-specification.md`](./ui-specification.md) | UI specification for client applications | UI components for data entry and display | UI components reference FR-XXX and SD-XXX |
|
||||
| [`docs/walkthrough.md`](./walkthrough.md) | End-to-end system flow | Traceability from user journey to technical implementation | Full flow validation against SD-XXX |
|
||||
| [`docs/architecture.md`](./architecture.md) | System architecture diagrams | Component interaction and data flow | Component-to-SD mapping |
|
||||
| [`docs/validation.md`](./validation.md) | CI/CD validation rules | Contract testing and spec compliance | Validation gates for SD-XXX |
|
||||
| [`docs/runbook.md`](./runbook.md) | Operational runbook | Deployment, scaling, and troubleshooting | Operation-to-SD mapping |
|
||||
|
||||
### 20.2 Implementation Files
|
||||
|
||||
| File | Platform | Features | Requirements Traceability |
|
||||
|------|----------|----------|--------------------------|
|
||||
| [`src/msghandler.jl`](../src/msghandler.jl) | Julia | Full feature set, Arrow IPC, multiple dispatch | FR-001 through FR-014, NFR-101 through NFR-405 |
|
||||
| [`src/msghandler_ssr.js`](../src/msghandler_ssr.js) | Node.js | Arrow IPC, async/await | FR-001 through FR-014, NFR-101 through NFR-405 |
|
||||
| [`src/msghandler_csr.js`](../src/msghandler_csr.js) | Browser | JSON table only | FR-001 through FR-014, NFR-101 through NFR-405 |
|
||||
| [`src/msghandler.py`](../src/msghandler.py) | Python | Arrow IPC, async/await | FR-001 through FR-014, NFR-101 through NFR-405 |
|
||||
| [`src/msghandler.dart`](../src/msghandler.dart) | Dart | Full feature set, Arrow IPC, async/await | FR-001 through FR-014, NFR-101 through NFR-405 |
|
||||
| [`src/msghandler.rs`](../src/msghandler.rs) | Rust | Full feature set, Arrow IPC, async/await, type-safe | FR-001 through FR-014, NFR-101 through NFR-405 |
|
||||
| [`src/msghandler_mpy.py`](../src/msghandler_mpy.py) | MicroPython | Limited to direct transport | FR-005, FR-006, FR-012 |
|
||||
| File | Platform | Features | Requirements Traceability | Solution Design Traceability |
|
||||
|------|----------|----------|--------------------------|------------------------------|
|
||||
| [`src/msghandler.jl`](../src/msghandler.jl) | Julia | Full feature set, Arrow IPC, multiple dispatch | FR-001 through FR-014, NFR-101 through NFR-405 | SD-001 through SD-008 |
|
||||
| [`src/msghandler_ssr.js`](../src/msghandler_ssr.js) | Node.js | Arrow IPC, async/await | FR-001 through FR-014, NFR-101 through NFR-405 | SD-001 through SD-008 |
|
||||
| [`src/msghandler_csr.js`](../src/msghandler_csr.js) | Browser | JSON table only | FR-001 through FR-014, NFR-101 through NFR-405 | SD-001 through SD-008 |
|
||||
| [`src/msghandler.py`](../src/msghandler.py) | Python | Arrow IPC, async/await | FR-001 through FR-014, NFR-101 through NFR-405 | SD-001 through SD-008 |
|
||||
| [`src/msghandler.dart`](../src/msghandler.dart) | Dart | Full feature set, Arrow IPC, async/await | FR-001 through FR-014, NFR-101 through NFR-405 | SD-001 through SD-008 |
|
||||
| [`src/msghandler.rs`](../src/msghandler.rs) | Rust | Full feature set, Arrow IPC, async/await, type-safe | FR-001 through FR-014, NFR-101 through NFR-405 | SD-001 through SD-008 |
|
||||
| [`src/msghandler_mpy.py`](../src/msghandler_mpy.py) | MicroPython | Limited to direct transport | FR-005, FR-006, FR-012 | SD-002, SD-004 |
|
||||
|
||||
### 20.3 External Dependencies
|
||||
|
||||
| Platform | Package | Version | Purpose | Requirements Traceability |
|
||||
|----------|---------|---------|---------|--------------------------|
|
||||
| Julia | JSON.jl | Latest | JSON serialization | FR-012, NFR-101, NFR-102 |
|
||||
| Julia | Arrow.jl | Latest | Arrow IPC support | FR-002, FR-012 |
|
||||
| Julia | HTTP.jl | Latest | HTTP file server | FR-008, FR-009 |
|
||||
| Julia | UUIDs.jl | Latest | UUID generation | FR-011, NFR-401 |
|
||||
| Node.js | node-fetch | Latest | HTTP file server | FR-008, FR-009 |
|
||||
| Browser | - | - | Transport-agnostic (caller provides) | FR-013, FR-014 |
|
||||
| Python | aiohttp | Latest | HTTP file server | FR-008, FR-009 |
|
||||
| Python | pyarrow | Latest | Arrow IPC support | FR-002, FR-012 |
|
||||
| Dart | http | Latest | HTTP file server | FR-008, FR-009 |
|
||||
| Dart | uuid | Latest | UUID generation | FR-011, NFR-401 |
|
||||
| Dart | dart-arrow | Latest | Arrow IPC support | FR-002, FR-012 |
|
||||
| Rust | serde | Latest | JSON serialization | FR-012, NFR-101, NFR-102 |
|
||||
| Rust | serde_json | Latest | JSON handling | FR-012, NFR-101, NFR-102 |
|
||||
| Rust | tokio | Latest | Async runtime | FR-013, FR-014 |
|
||||
| Rust | reqwest | Latest | HTTP file server | FR-008, FR-009 |
|
||||
| Rust | uuid | Latest | UUID generation | FR-011, NFR-401 |
|
||||
| Rust | arrow2 | Latest | Arrow IPC support | FR-002, FR-012 |
|
||||
| MicroPython | builtin | N/A | Limited implementation | FR-005, FR-006 |
|
||||
| Platform | Package | Version | Purpose | Requirements Traceability | Solution Design Traceability |
|
||||
|----------|---------|---------|---------|--------------------------|------------------------------|
|
||||
| Julia | JSON.jl | Latest | JSON serialization | FR-012, NFR-101, NFR-102 | SD-005 |
|
||||
| Julia | Arrow.jl | Latest | Arrow IPC support | FR-002, FR-012 | SD-004, SD-005 |
|
||||
| Julia | HTTP.jl | Latest | HTTP file server | FR-008, FR-009 | SD-003 |
|
||||
| Julia | UUIDs.jl | Latest | UUID generation | FR-011, NFR-401 | SD-008 |
|
||||
| Node.js | node-fetch | Latest | HTTP file server | FR-008, FR-009 | SD-003 |
|
||||
| Browser | - | - | Transport-agnostic (caller provides) | FR-013, FR-014 | SD-006 |
|
||||
| Python | aiohttp | Latest | HTTP file server | FR-008, FR-009 | SD-003 |
|
||||
| Python | pyarrow | Latest | Arrow IPC support | FR-002, FR-012 | SD-004, SD-005 |
|
||||
| Dart | http | Latest | HTTP file server | FR-008, FR-009 | SD-003 |
|
||||
| Dart | uuid | Latest | UUID generation | FR-011, NFR-401 | SD-008 |
|
||||
| Dart | dart-arrow | Latest | Arrow IPC support | FR-002, FR-012 | SD-004, SD-005 |
|
||||
| Rust | serde | Latest | JSON serialization | FR-012, NFR-101, NFR-102 | SD-005 |
|
||||
| Rust | serde_json | Latest | JSON handling | FR-012, NFR-101, NFR-102 | SD-005 |
|
||||
| Rust | tokio | Latest | Async runtime | FR-013, FR-014 | SD-006 |
|
||||
| Rust | reqwest | Latest | HTTP file server | FR-008, FR-009 | SD-003 |
|
||||
| Rust | uuid | Latest | UUID generation | FR-011, NFR-401 | SD-008 |
|
||||
| Rust | arrow2 | Latest | Arrow IPC support | FR-002, FR-012 | SD-004, SD-005 |
|
||||
| MicroPython | builtin | N/A | Limited implementation | FR-005, FR-006 | SD-002 |
|
||||
|
||||
---
|
||||
|
||||
## 21. Change Log
|
||||
## Change Log
|
||||
|
||||
| Date | Version | Changes | Requirement ID(s) |
|
||||
|------|---------|---------|-------------------|
|
||||
| 2026-05-15 | 1.3.0 | Made transport layer agnostic | All |
|
||||
| - | - | Removed NATS-specific dependencies and references from all docs | All |
|
||||
| - | - | Updated all NATS references to generic "transport layer"/"message broker" | All |
|
||||
| - | - | Removed NATS client packages from dependencies tables | All |
|
||||
| 2026-05-13 | 1.2.0 | Aligned with ground truth implementation (src/msghandler.jl) | All |
|
||||
| - | - | Updated smartpack signatures: removed is_publish, nats_connection; added sender_name | FR-001 through FR-014 |
|
||||
| - | - | Updated smartunpack signatures: takes msg_json_str::String instead of msg | FR-001 through FR-014 |
|
||||
| - | - | Removed publishMessage function and NATSClient/NATSConnectionPool classes from browser section | FR-013, FR-014 |
|
||||
| - | - | Added plik_oneshot_upload(filepath) overload to file server interface | FR-008, FR-009 |
|
||||
| - | - | Fixed SIZE_THRESHOLD default to 500,000 bytes | FR-003, FR-004 |
|
||||
| 2026-03-23 | 1.1.0 | Updated to ASG Framework specification guidelines | All |
|
||||
| 2026-03-15 | 1.1.0 | Browser connection management | FR-001 through FR-014 |
|
||||
| 2026-03-13 | 1.0.0 | Initial specification | FR-001 through FR-014, NFR-101 through NFR-405 |
|
||||
| Date | Version | Changes | Requirement ID(s) | Solution Design Ref(s) |
|
||||
|------|---------|---------|-------------------|------------------------|
|
||||
| 2026-05-22 | 1.3.0 | Updated to ASG Framework v8 pillars | All | All SD-XXX |
|
||||
| - | - | Added Solution Design references to all specification items | All | All SD-XXX |
|
||||
| - | - | Updated version to 1.3.0 to match requirements and solution-design | All | - |
|
||||
| - | - | Added ASG Framework alignment section | All | - |
|
||||
| 2026-05-15 | 1.3.0 | Made transport layer agnostic | All | SD-006 |
|
||||
| - | - | Removed all NATS-specific dependencies (NATS.jl, nats, nats-py, nats.ws) | All | SD-006 |
|
||||
| - | - | Updated docs to reference generic message broker/transport | All | SD-006 |
|
||||
| - | - | broker_url is now metadata only, not used for active connections | All | SD-006 |
|
||||
| 2026-03-15 | 1.1.0 | Browser connection management | FR-001 through FR-014 | SD-006 |
|
||||
| - | - | Added NATSClient class with keepAlive support | FR-013, FR-014 | SD-006 |
|
||||
| - | - | Added NATSConnectionPool for connection reuse | FR-013, FR-014 | SD-006 |
|
||||
| - | - | Added publishMessage function with closeConnection option | FR-013, FR-014 | SD-006 |
|
||||
| - | - | Added nats.ws to browser dependencies | FR-013, FR-014 | SD-006 |
|
||||
| 2026-03-13 | 1.0.0 | Initial specification | FR-001 through FR-014, NFR-101 through NFR-405 | SD-001 through SD-008 |
|
||||
| - | - | Message envelope schema defined | FR-012, FR-013 | SD-001, SD-006 |
|
||||
| - | - | Payload schema with transport modes | FR-001, FR-002, FR-003, FR-004 | SD-001, SD-002 |
|
||||
| - | - | Enumerations for payload_type, transport, encoding | FR-001, FR-002, FR-003, FR-006 | SD-004, SD-005 |
|
||||
| - | - | Size thresholds for desktop/MicroPython | FR-004, FR-005 | SD-002 |
|
||||
| - | - | Error codes and validation rules | FR-001 through FR-010 | SD-001 through SD-007 |
|
||||
| - | - | API contracts for all platforms | FR-001 through FR-014 | SD-001 through SD-008 |
|
||||
|
||||
---
|
||||
|
||||
|
||||
Reference in New Issue
Block a user