1398 lines
61 KiB
Markdown
1398 lines
61 KiB
Markdown
# Specification: msghandler
|
|
|
|
**Version**: 1.4.0
|
|
**Date**: 2026-09-16
|
|
**Status**: Active
|
|
**Ground Truth**: [`src/msghandler.jl`](../src/msghandler.jl)
|
|
**Specification Format**: JSON Schema + AsyncAPI
|
|
**ASG Framework Alignment**: Eight Pillars - Requirements → Solution Design → Walkthrough → Specification → UI Specification → Validation → Implementation Plan → Runbook
|
|
**Specification Item IDs**: `SP-XXX` (see [Specification Traceability Matrix](#11-requirements-traceability)); every spec item cites its Requirement ID(s) (`FR-XXX`/`NFR-XXX`) and Solution Design reference(s) (`SD-XXX`).
|
|
|
|
---
|
|
|
|
## 1. Technical Contract Overview
|
|
|
|
This document defines the **technical contract** for msghandler - the cross-platform bi-directional data bridge that enables seamless communication between **Julia**, **JavaScript**, **Python**, **Rust**, and **MicroPython** applications using a message broker as the transport layer.
|
|
|
|
This specification serves as the single source of truth for:
|
|
- **Inputs**: What data structures are accepted by `smartpack()`
|
|
- **Outputs**: What data structures are returned by `smartunpack()`
|
|
- **Data Shapes**: Exact field names, types, and constraints
|
|
- **Error Codes**: Standardized error responses for failure scenarios
|
|
|
|
### 1.1 Requirements Traceability
|
|
|
|
Per the ASG Framework, every technical contract below carries a unique `SP-XXX` Specification Item ID and cites its Requirement ID(s) (the *what*) and Solution Design reference(s) (the *how/why*). The canonical cross-reference is the [Specification Traceability Matrix](#specification-traceability-matrix).
|
|
|
|
| Specification Section | Spec ID (SP) | Requirement ID(s) | Solution Design Ref(s) | Description |
|
|
|----------------------|-------------------|------------------------|-------------|
|
|
| Section 2 (Message Envelope) | SP-001 | FR-012, FR-013, NFR-101, NFR-102 | SD-008 | Message envelope structure and validation |
|
|
| Section 3 (Payload Schema) | SP-002 | FR-001, FR-002, FR-003, FR-004, NFR-101, NFR-102 | SD-001, SD-005 | Payload structure and field definitions |
|
|
| Section 4 (Payload Format) | SP-003 | FR-006, FR-007 | SD-004 | Tuple format for smartpack() |
|
|
| Section 5 (Enumerations) | SP-004..SP-007 | FR-003, FR-004, FR-006, NFR-101 | SD-001, SD-002, SD-005 | Enumerations for transport and encoding |
|
|
| Section 6 (Transport Protocols) | SP-008, SP-009 | FR-003, FR-004, NFR-104, NFR-105 | SD-001, SD-002 | Direct and link transport protocols |
|
|
| Section 7 (Size Thresholds) | SP-010 | FR-004, FR-005, NFR-104, NFR-105 | SD-002 | Size thresholds for transport selection |
|
|
| Section 8 (Topic Convention) | SP-011 | FR-013, FR-014 | SD-006 | Topic/subject naming patterns |
|
|
| Section 9 (Error Handling) | SP-012..SP-014 | FR-010, FR-011, NFR-201, NFR-202, NFR-203 | SD-003, SD-007 | Error codes and exception handling |
|
|
| Section 10 (Serialization Rules) | SP-015 | FR-001, FR-002, FR-003, FR-012, NFR-101, NFR-102 | SD-005 | Serialization and encoding rules |
|
|
| Section 11 (API Contract) | SP-016, SP-017 | FR-001 through FR-014, NFR-101 through NFR-107 | SD-001 through SD-008 | Function signatures for all platforms |
|
|
| Section 12 (File Server Interface) | SP-018, SP-019 | FR-008, FR-009, FR-010 | SD-003, SD-007 | Upload and download handler contracts |
|
|
| Section 13 (Platform-Specific Constraints) | SP-020 | FR-005, FR-006, NFR-106, NFR-107 | SD-004, SD-006 | Platform-specific feature support |
|
|
| Section 14 (Implementation Files) | SP-021 | FR-001 through FR-007 | - | Implementation file mapping |
|
|
| Section 15 (Message Flow) | SP-022 | FR-001 through FR-014, NFR-101 through NFR-107 | SD-001 through SD-008 | Mermaid diagrams for send/receive flows |
|
|
| Section 16 (Validation Rules) | SP-023 | FR-001 through FR-014, NFR-101 through NFR-405 | SD-001 through SD-008 | Envelope and payload validation rules |
|
|
| Section 17 (Test Contracts) | SP-024 | FR-001 through FR-014, NFR-101 through NFR-107 | SD-001 through SD-008 | Unit and integration test scenarios |
|
|
| Section 18 (Dependencies) | SP-025 | FR-001 through FR-014, NFR-101 through NFR-107 | SD-001 through SD-008 | Platform-specific dependencies |
|
|
| Section 19 (Change Log) | — | — | — | Version history and changes |
|
|
|
|
### Specification Traceability Matrix
|
|
|
|
Canonical mapping of each `SP-XXX` Specification Item ID to its Requirement IDs and Solution Design references, per the ASG Framework. Every `SP-XXX` below is implemented by at least one file under [`src/`](../src/) (ground truth: [`src/msghandler.jl`](../src/msghandler.jl)).
|
|
|
|
| SP ID | Specification Item | Requirement ID(s) | Solution Design Ref | Implementation File |
|
|
|-------|--------------------|-------------------|---------------------|---------------------|
|
|
| SP-001 | Message Envelope Schema (`envelope_to_json`) | FR-012, FR-013, NFR-101, NFR-102 | SD-008 | msghandler.jl |
|
|
| SP-002 | Payload Schema | FR-001, FR-002, FR-003, FR-004, NFR-101, NFR-102 | SD-001, SD-005 | msghandler.jl |
|
|
| SP-003 | Payload Tuple Format `("name", data, "type")` | FR-006, FR-007 | SD-004 | msghandler.jl |
|
|
| SP-004 | `msg_purpose` Enum (ACK/NACK/updateStatus/shutdown/chat/command/event) | FR-003, FR-004, FR-006, NFR-101 | SD-001 | msghandler.jl |
|
|
| SP-005 | `payload_type` Enum | FR-003, FR-004, FR-006, NFR-101 | SD-002 | msghandler.jl |
|
|
| SP-006 | `transport` Enum (direct/link) | FR-003, FR-004, NFR-104, NFR-105 | SD-001, SD-002 | msghandler.jl |
|
|
| SP-007 | `encoding` Enum (none/base64/json/arrow-ipc) | FR-001, FR-002, FR-003, FR-012 | SD-005 | msghandler.jl |
|
|
| SP-008 | Direct Transport Protocol | FR-003, FR-004, NFR-104, NFR-105 | SD-001, SD-002 | msghandler.jl |
|
|
| SP-009 | Link Transport Protocol (Claim-Check / file server) | FR-003, FR-004, NFR-104, NFR-105 | SD-001, SD-002 | msghandler.jl |
|
|
| SP-010 | Size Thresholds (10,000 bytes desktop; 100,000 MicroPython) | FR-004, FR-005, NFR-104, NFR-105 | SD-002 | msghandler.jl |
|
|
| SP-011 | Topic/Subject Convention `<scope>/<service>/<version>/<operation>` | FR-013, FR-014 | SD-006 | msghandler.jl |
|
|
| SP-012 | Error Response Format | FR-010, FR-011, NFR-201, NFR-202, NFR-203 | SD-003, SD-007 | msghandler.jl |
|
|
| SP-013 | Error Codes | FR-010, FR-011, NFR-201, NFR-202, NFR-203 | SD-003, SD-007 | msghandler.jl |
|
|
| SP-014 | Exception Handling / Retry Policy | FR-010, FR-011, NFR-201, NFR-202, NFR-203 | SD-003, SD-007 | msghandler.jl |
|
|
| SP-015 | Serialization Rules | FR-001, FR-002, FR-003, FR-012, NFR-101, NFR-102 | SD-005 | msghandler.jl |
|
|
| SP-016 | `smartpack()` API Contract | FR-001..FR-014, NFR-101..NFR-107 | SD-001..SD-008 | msghandler.jl |
|
|
| SP-017 | `smartunpack()` API Contract | FR-001..FR-014, NFR-101..NFR-107 | SD-001..SD-008 | msghandler.jl |
|
|
| SP-018 | Upload Handler Contract (`plik_oneshot_upload`) | FR-008, FR-009, FR-010 | SD-003, SD-007 | msghandler.jl |
|
|
| SP-019 | Download Handler Contract (`_fetch_with_backoff`) | FR-010, FR-011, NFR-202 | SD-003, SD-007 | msghandler.jl |
|
|
| SP-020 | Platform-Specific Constraints | FR-005, FR-006, NFR-106, NFR-107 | SD-004, SD-006 | msghandler.jl, msghandler-csr.js, msghandler.py, msghandler.rs, msghandler_mpy.py |
|
|
| SP-021 | Implementation Files | FR-001..FR-007 | — | msghandler.jl, msghandler-csr.js, msghandler.py, msghandler.rs, msghandler_mpy.py |
|
|
| SP-022 | Message Flow (send/receive) | FR-001..FR-014, NFR-101..NFR-107 | SD-001..SD-008 | msghandler.jl |
|
|
| SP-023 | Validation Rules | FR-001..FR-014, NFR-101..NFR-405 | SD-001..SD-008 | msghandler.jl |
|
|
| SP-024 | Test Contracts | FR-001..FR-014, NFR-101..NFR-107 | SD-001..SD-008 | msghandler.jl |
|
|
| SP-025 | Dependencies | FR-001..FR-014, NFR-101..NFR-107 | SD-001..SD-008 | msghandler.jl |
|
|
|
|
### Walkthrough → Specification Trace
|
|
|
|
Maps each `WT-XXX` walkthrough step (from `3-walkthrough.md`) to the `SP-XXX` specification item(s) it exercises, closing the walkthrough↔specification gap-check. Derived from the walkthrough's `WT-XXX → SD-XXX → FR-XXX` trace combined with the `SD-XXX → SP-XXX` mapping above.
|
|
|
|
| WT ID | Participant | Walkthrough Step | SP ID(s) |
|
|
|-------|-------------|------------------|----------|
|
|
| WT-001 | JavaScript | Session start / transport connect | SP-011, SP-016 |
|
|
| WT-002 | JavaScript | Send mixed payloads (text + image) | SP-003, SP-015, SP-016 |
|
|
| WT-002.1 | JavaScript | Per-payload transport selection | SP-010, SP-008, SP-016 |
|
|
| WT-003 | JavaScript | Publish JSON via transport | SP-011, SP-016 |
|
|
| WT-004 | JavaScript | Envelope shape crosses the wire | SP-001 |
|
|
| WT-005 | JavaScript | Receive reply | SP-017 |
|
|
| WT-006 | JavaScript | Large file upload fails (link transport) | SP-009, SP-018, SP-013 |
|
|
| WT-007 | JavaScript | Transport connection lost (caller retry) | SP-011, SP-016 |
|
|
| WT-010 | Julia | Session start / subscribe | SP-011, SP-017 |
|
|
| WT-011 | Julia | Receive & unpack chat | SP-017, SP-015, SP-002 |
|
|
| WT-011.1 | Julia | Direct path base64 decode / deserialize | SP-015, SP-008, SP-002 |
|
|
| WT-012 | Julia | Send reply | SP-016, SP-001 |
|
|
| WT-013 | Julia | Receive large file (link transport) | SP-009, SP-019, SP-010 |
|
|
| WT-014 | Julia | Download fails after max_retries | SP-014, SP-019, SP-013 |
|
|
| WT-015 | Julia | Unknown payload type throws | SP-005, SP-013 |
|
|
| WT-020 | Python | Session start / connect | SP-011, SP-017 |
|
|
| WT-021 | Python | Send arrowtable | SP-002, SP-015, SP-016 |
|
|
| WT-021.1 | Python | Arrow IPC serialization (arrow-ipc) | SP-015, SP-010, SP-008 |
|
|
| WT-022 | Python | Receive results | SP-017, SP-015 |
|
|
| WT-023 | Python | Large arrowtable upload fails | SP-009, SP-018, SP-013 |
|
|
| WT-024 | Python | pyarrow not available | SP-005, SP-013, SP-020 |
|
|
| WT-030 | Rust | Session start / subscribe | SP-011, SP-016 |
|
|
| WT-031 | Rust | Receive & dispatch by payload_type | SP-017, SP-002, SP-005 |
|
|
| WT-032 | Rust | Send type-safe results | SP-016, SP-002 |
|
|
| WT-033 | Rust | Large file via link transport | SP-009, SP-018, SP-010 |
|
|
| WT-034 | Rust | Download exceeds retries | SP-014, SP-019, SP-013 |
|
|
| WT-040 | MicroPython | Session start / publish sensor reading | SP-011, SP-016 |
|
|
| WT-041 | MicroPython | Send dictionary (direct transport) | SP-002, SP-015, SP-008, SP-016 |
|
|
| WT-041.1 | MicroPython | JSON serialization, direct transport | SP-015, SP-010, SP-008 |
|
|
| WT-042 | MicroPython | Receive ACK | SP-017 |
|
|
| WT-043 | MicroPython | Threshold exceeded (direct-only) | SP-010, SP-009, SP-020 |
|
|
| WT-044 | MicroPython | Non-serializable dictionary | SP-005, SP-013, SP-020 |
|
|
| WT-050 | — | Cross-participant chat broadcast | SP-011, SP-022, SP-016 |
|
|
|
|
---
|
|
|
|
## 2. Message Envelope Schema
|
|
|
|
## Specification Versioning
|
|
|
|
| Component | Version | Notes |
|
|
|-----------|---------|-------|
|
|
| Specification | 1.0.0 | Initial release |
|
|
| Protocol | v1 | Message envelope protocol version |
|
|
|
|
---
|
|
|
|
## Message Envelope Schema
|
|
|
|
### Envelope Structure (JSON)
|
|
|
|
```json
|
|
{
|
|
"correlation_id": "string (UUID)",
|
|
"msg_id": "string (UUID)",
|
|
"timestamp": "string (ISO 8601 UTC)",
|
|
"send_to": "string",
|
|
"msg_purpose": "string",
|
|
"sender_name": "string",
|
|
"sender_id": "string (UUID)",
|
|
"receiver_name": "string",
|
|
"receiver_id": "string (UUID)",
|
|
"reply_to": "string",
|
|
"reply_to_msg_id": "string",
|
|
"broker_url": "string",
|
|
"metadata": "object",
|
|
"payloads": [
|
|
{
|
|
"id": "string (UUID)",
|
|
"dataname": "string",
|
|
"payload_type": "string",
|
|
"transport": "string",
|
|
"encoding": "string",
|
|
"size": "integer",
|
|
"data": "string or URL",
|
|
"metadata": "object"
|
|
}
|
|
]
|
|
}
|
|
```
|
|
|
|
### Field Definitions (SP-001)
|
|
|
|
| SP ID | Field | Type | Required | Validation | Description | Requirement ID | Solution Design Ref |
|
|
|-------|------|------|----------|------------|-------------|----------------|-------------------|
|
|
| SP-001 | `correlation_id` | `string` | Yes | UUID v4 format | Track message flow across distributed systems | FR-011, NFR-401 | SD-008 |
|
|
| SP-001 | `msg_id` | `string` | Yes | UUID v4 format | Unique identifier for this specific message | FR-012, NFR-401 | SD-001 |
|
|
| SP-001 | `timestamp` | `string` | Yes | ISO 8601 UTC | Message publication timestamp | FR-012, NFR-401 | SD-001 |
|
|
| SP-001 | `send_to` | `string` | Yes | Non-empty string | Topic/subject to publish the message to | FR-013 | SD-006 |
|
|
| SP-001 | `msg_purpose` | `string` | Yes | Enum | Purpose of the message | FR-013 | SD-006 |
|
|
| SP-001 | `sender_name` | `string` | Yes | Non-empty string | Name of the sender application | FR-013 | SD-006 |
|
|
| SP-001 | `receiver_name` | `string` | Yes | Any string | Name of the receiver (empty = broadcast) | FR-013 | SD-006 |
|
|
| SP-001 | `receiver_id` | `string` | Yes | Any string | UUID of the receiver (empty = broadcast) | FR-013 | SD-006 |
|
|
| SP-001 | `reply_to` | `string` | Yes | Any string | Topic where receiver should reply | FR-013 | SD-006 |
|
|
| SP-001 | `reply_to_msg_id` | `string` | Yes | Any string | Message ID this message is replying to | FR-013 | SD-006 |
|
|
| SP-001 | `broker_url` | `string` | Yes | Valid URL | Broker URL for the transport layer | FR-013 | SD-006 |
|
|
| SP-001 | `metadata` | `object` | No | Any JSON object | Message-level metadata | NFR-401 | SD-001, SD-008 |
|
|
| SP-001 | `payloads` | `array` | Yes | Non-empty array | List of payload objects | FR-012, FR-013 | SD-001, SD-005 |
|
|
|
|
---
|
|
|
|
## Payload Schema
|
|
|
|
### Payload Structure (JSON)
|
|
|
|
```json
|
|
{
|
|
"id": "string (UUID)",
|
|
"dataname": "string",
|
|
"payload_type": "string",
|
|
"transport": "string",
|
|
"encoding": "string",
|
|
"size": "integer",
|
|
"data": "string or URL",
|
|
"metadata": "object"
|
|
}
|
|
```
|
|
|
|
### Payload Field Definitions (SP-002)
|
|
|
|
| SP ID | Field | Type | Required | Validation | Description | Requirement ID | Solution Design Ref |
|
|
|-------|------|------|----------|------------|-------------|----------------|-------------------|
|
|
| SP-002 | `id` | `string` | Yes | UUID v4 format | Unique identifier for this payload | FR-012 | SD-001 |
|
|
| SP-002 | `dataname` | `string` | Yes | Non-empty string | Name of the payload (e.g., `login_image`, `user_data`) | FR-001, FR-002, FR-003 | SD-004 |
|
|
| SP-002 | `payload_type` | `string` | Yes | Enum | Type of payload (see `payload_type` enum) | FR-001, FR-002, FR-003, FR-006 | SD-004, SD-005 |
|
|
| SP-002 | `transport` | `string` | Yes | Enum | Transport method: `direct` or `link` | FR-003, FR-004, NFR-104, NFR-105 | SD-001, SD-002 |
|
|
| SP-002 | `encoding` | `string` | Yes | Enum | Encoding method (see `encoding` enum) | FR-001, FR-002, FR-003, FR-012 | SD-005 |
|
|
| SP-002 | `size` | `integer` | Yes | Positive integer | Size of the payload in bytes | FR-003, FR-004, NFR-104, NFR-105 | SD-002 |
|
|
| SP-002 | `data` | `string` or `URL` | Yes | Base64 string or URL | Payload data (base64 for direct, URL for link) | FR-003, FR-004, FR-008, FR-009, FR-010 | SD-001, SD-003 |
|
|
| SP-002 | `metadata` | `object` | No | Any JSON object | Payload-level metadata | NFR-401 | SD-008 |
|
|
|
|
---
|
|
|
|
## Payload Format (SP-003)
|
|
|
|
### Tuple Format for `smartpack()`
|
|
|
|
The `smartpack()` function accepts data as an array of tuples with the format:
|
|
|
|
```
|
|
("data_name", data, "data_type")
|
|
```
|
|
|
|
| Position | Type | Description | Example |
|
|
|----------|------|-------------|---------|
|
|
| 1 | `string` | Data name - identifier for the payload | `"msg"`, `"login_image"`, `"user_data"` |
|
|
| 2 | `any` | Actual data - content to be serialized | `"Hello"`, `{"key": "value"}`, `DataFrame(...)` |
|
|
| 3 | `string` | Data type - must be in `payload_type` enum | `"text"`, `"dictionary"`, `"arrowtable"` |
|
|
|
|
### Single Payload Example
|
|
|
|
```julia
|
|
# Julia
|
|
smartpack("/chat/user/v1/message", [("msg", "Hello World", "text")])
|
|
```
|
|
|
|
```python
|
|
# Python
|
|
await smartpack("/chat/user/v1/message", [("msg", "Hello World", "text")])
|
|
```
|
|
|
|
```typescript
|
|
// JavaScript
|
|
await smartpack("/chat/user/v1/message", [["msg", "Hello World", "text"]]);
|
|
```
|
|
|
|
### Multiple Payloads Example
|
|
|
|
```julia
|
|
# Julia - Mixed text and binary data
|
|
data = [
|
|
("msg", "Hello", "text"),
|
|
("img", binary_data, "image")
|
|
]
|
|
smartpack("/agent/v1/process", data)
|
|
```
|
|
|
|
```python
|
|
# Python - Mixed types
|
|
data = [
|
|
("msg", "Hello", "text"),
|
|
("img", binary_data, "image")
|
|
]
|
|
await smartpack("/agent/v1/process", data)
|
|
```
|
|
|
|
### Data Type Mapping
|
|
|
|
| Platform | Input Type | Data Type String |
|
|
|----------|------------|------------------|
|
|
| All | `String` | `"text"` |
|
|
| All | `Dict`/`Object` | `"dictionary"` |
|
|
| Desktop | `DataFrame` | `"arrowtable"` or `"jsontable"` |
|
|
| Browser | `Array` of objects | `"jsontable"` (only table type) |
|
|
| All | `Array` of objects | `"jsontable"` |
|
|
| All | `Uint8Array`/`Buffer`/`bytes` | `"binary"` |
|
|
| Desktop | `Arrow.Table` | `"arrowtable"` |
|
|
| All | Image/Audio/Video binary | `"image"`, `"audio"`, `"video"` |
|
|
|
|
---
|
|
|
|
## Enumerations (SP-004..SP-007)
|
|
|
|
### `msg_purpose` Enum
|
|
|
|
| Value | Description |
|
|
|-------|-------------|
|
|
| `ACK` | Acknowledgment of successful message processing |
|
|
| `NACK` | Negative acknowledgment of message processing failure |
|
|
| `updateStatus` | Status update message |
|
|
| `shutdown` | Graceful shutdown request |
|
|
| `chat` | Chat/message payload |
|
|
| `command` | Command payload |
|
|
| `event` | Event payload |
|
|
|
|
### `payload_type` Enum
|
|
|
|
| Value | Description | Supported Platforms | Encoding Options |
|
|
|-------|-------------|---------------------|------------------|
|
|
| `text` | Plain text string | All | `base64` | FR-001, FR-012 | SD-004, SD-005 |
|
|
| `dictionary` | JSON object/dictionary | All | `base64`, `json` | FR-002, FR-012 | SD-004, SD-005 |
|
|
| `arrowtable` | Apache Arrow IPC table | Desktop (Julia/Python) | `base64`, `arrow-ipc` | FR-002 | SD-004, SD-005 |
|
|
| `jsontable` | JSON array of objects | All (including Browser) | `base64`, `json` | FR-002, FR-006 | SD-004, SD-005 |
|
|
| `image` | Binary image data | All | `base64` | FR-001, FR-006 | SD-004, SD-005 |
|
|
| `audio` | Binary audio data | All | `base64` | FR-001, FR-006 | SD-004, SD-005 |
|
|
| `video` | Binary video data | All | `base64` | FR-001, FR-006 | SD-004, SD-005 |
|
|
| `binary` | Generic binary data | All | `base64` | FR-001, FR-006 | SD-004, SD-005 |
|
|
|
|
### `transport` Enum
|
|
|
|
| Value | Description | Data Format | Use Case |
|
|
|-------|-------------|-------------|----------|
|
|
| `direct` | Payload sent directly via the transport layer | Base64-encoded string | Payloads < size_threshold | FR-003, FR-004, NFR-104, NFR-105 | SD-001, SD-002 |
|
|
| `link` | Payload uploaded to file server | HTTP URL | Payloads ≥ size_threshold | FR-003, FR-004, NFR-104, NFR-105 | SD-001, SD-002 |
|
|
|
|
### `encoding` Enum
|
|
|
|
| Value | Description | Payload Types |
|
|
|-------|-------------|---------------|
|
|
| `none` | No additional encoding | Link transport URLs | FR-003, FR-004 | SD-001, SD-002 |
|
|
| `base64` | Base64 encoding | Text, binary, image, audio, video | FR-001, FR-002, FR-003, FR-012 | SD-005 |
|
|
| `json` | JSON encoding | Dictionary, jsontable | FR-002, FR-006 | SD-004, SD-005 |
|
|
| `arrow-ipc` | Apache Arrow IPC format | Arrowtable | FR-002 | SD-004, SD-005 |
|
|
|
|
---
|
|
|
|
## Transport Protocols (SP-008, SP-009)
|
|
|
|
### Direct Transport Protocol (SP-008)
|
|
|
|
When `transport = "direct"`, the `data` field contains a Base64-encoded string of the serialized payload.
|
|
|
|
**Flow**:
|
|
1. Serialize payload according to `payload_type`
|
|
2. Encode serialized bytes as Base64
|
|
3. Include Base64 string in `data` field
|
|
|
|
**Example**:
|
|
```json
|
|
{
|
|
"transport": "direct",
|
|
"encoding": "base64",
|
|
"size": 11,
|
|
"data": "SGVsbG8gV29ybGQ="
|
|
}
|
|
```
|
|
|
|
### Link Transport Protocol (SP-009)
|
|
|
|
When `transport = "link"`, the `data` field contains a URL pointing to the uploaded payload.
|
|
|
|
**Flow**:
|
|
1. Serialize payload according to `payload_type`
|
|
2. Upload to HTTP file server (e.g., Plik)
|
|
3. Include returned URL in `data` field
|
|
|
|
**Example**:
|
|
```json
|
|
{
|
|
"transport": "link",
|
|
"encoding": "none",
|
|
"size": 1000000,
|
|
"data": "http://localhost:8080/file/3F62E/4AgGT/data.zip"
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## Size Thresholds (SP-010)
|
|
|
|
### Desktop Platforms (Julia/JS/Python)
|
|
|
|
| Platform | Size Threshold | Notes | Requirement ID | Solution Design Ref |
|
|
|----------|----------------|-------|----------------|-------------------|
|
|
| Desktop | 10,000 bytes (10KB) | Default threshold | FR-004, FR-005, NFR-104, NFR-105 | SD-002 |
|
|
|
|
### MicroPython Platform
|
|
|
|
| Platform | Size Threshold | Maximum Payload | Notes | Requirement ID | Solution Design Ref |
|
|
|----------|----------------|-----------------|-------|----------------|-------------------|
|
|
| MicroPython | 100,000 bytes (100KB) | 50,000 bytes | Hard limit due to memory constraints | FR-005, NFR-106 | SD-002 |
|
|
|
|
---
|
|
|
|
### Rate Limiting & Quotas (SP-010.1)
|
|
|
|
Per the ASG Framework content guidelines, this section defines rate-limit and quota behavior. msghandler is a **transport-agnostic bridge** and does **not** implement rate limiting or quotas itself; those are governed by the underlying transport layer (broker) and the file server.
|
|
|
|
| Level | Rate Limit / Quota | Owner | Notes | SP ID |
|
|
|-------|--------------------|-------|-------|-------|
|
|
| Transport (broker) | Publish/subscribe rate limits defined by the broker | Message broker (NATS / MQTT / WebSocket) | Enforced by the broker, not by msghandler | SP-010.1 |
|
|
| File server | Per-file and per-session upload/download size limits | File server (Plik or custom upload/download handler) | Enforced by the upload/download handler | SP-010.1 |
|
|
| Payload | 10,000-byte default threshold (direct vs link) | msghandler (`size_threshold`, configurable) | Applied per payload in `smartpack()` | SP-010 |
|
|
| MicroPython | 50,000-byte hard per-payload limit | msghandler | Hard limit due to memory constraints | SP-010 |
|
|
|
|
## Topic Convention (SP-011)
|
|
|
|
### Subject Naming Pattern
|
|
|
|
```
|
|
<scope>/<service>/<version>/<operation>
|
|
```
|
|
|
|
**Examples**:
|
|
- `/agent/wine/api/v1/prompt` - AI agent prompt endpoint | FR-013 | SD-006
|
|
- `/chat/user/v1/message` - User chat message | FR-013 | SD-006
|
|
- `/system/worker/v1/status` - Worker status update | FR-013 | SD-006
|
|
|
|
### Subject Wildcards
|
|
|
|
| Wildcard | Description | Example |
|
|
|----------|-------------|---------|
|
|
| `*` | Single-level wildcard | `/chat/user/v1/*` matches `/chat/user/v1/message` | FR-013 | SD-006 |
|
|
| `>` | Multi-level wildcard | `/chat/user/v1/>` matches all `/chat/user/v1/*` subjects | FR-013 | SD-006 |
|
|
|
|
---
|
|
|
|
## Error Handling (SP-012..SP-014)
|
|
|
|
### Error Response Format (SP-012)
|
|
|
|
```json
|
|
{
|
|
"error": {
|
|
"code": "string",
|
|
"message": "string",
|
|
"details": "object"
|
|
}
|
|
}
|
|
```
|
|
|
|
### Error Codes (SP-013)
|
|
|
|
| Code | HTTP Status | Description | Recovery | Requirement ID |
|
|
|------|-------------|-------------|----------|----------------|
|
|
| `INVALID_ENVELOPE` | 400 | Message envelope validation failed | Fix envelope structure | FR-012, FR-013, FR-014 | SD-001, SD-006 |
|
|
| `INVALID_PAYLOAD_TYPE` | 400 | Unsupported payload type | Use supported payload_type | FR-001, FR-002, FR-003, FR-006 | SD-004, SD-005 |
|
|
| `INVALID_TRANSPORT` | 400 | Unsupported transport type | Use `direct` or `link` | FR-003, FR-004, FR-006 | SD-001, SD-002 |
|
|
| `UPLOAD_FAILED` | 500 | File server upload failed | Retry or use direct transport | FR-008, FR-009 | SD-003 |
|
|
| `DOWNLOAD_FAILED` | 503 | File server download failed | Retry with exponential backoff | FR-010, FR-011, NFR-201, NFR-202 | SD-003, SD-007 |
|
|
| `TRANSPORT_CONNECTION_FAILED` | 503 | Transport connection failed | Check broker/server availability | FR-013, FR-014, NFR-201, NFR-203 | SD-006 |
|
|
| `DESERIALIZATION_ERROR` | 500 | Payload deserialization failed | Check payload_type matches data | FR-001, FR-002, FR-003, FR-012 | SD-004, SD-005 |
|
|
| `SIZE_EXCEEDED` | 413 | Payload exceeds maximum size | Split payload or use link transport | FR-003, FR-004, FR-005, NFR-104, NFR-105 | SD-001, SD-002 |
|
|
|
|
### Exception Handling (SP-014)
|
|
|
|
| Scenario | Handler | Retry Policy | Requirement ID |
|
|
|----------|---------|--------------|----------------|
|
|
| File server unavailable | Retry up to 5 times | Exponential backoff (100ms → 5000ms) | FR-010, NFR-202 | SD-003, SD-007 |
|
|
| Transport publish failure | Handled by caller | Caller's retry logic | FR-013, FR-014, NFR-201, NFR-203 | SD-006 |
|
|
| Deserialization error | Log correlation ID and throw | No retry (data corruption) | FR-001, FR-002, FR-003, FR-012, NFR-401 | SD-004, SD-005, SD-008 |
|
|
| Memory overflow (MicroPython) | Reject payloads >50KB | No retry (client-side check) | FR-005, NFR-106 | SD-004 |
|
|
|
|
---
|
|
|
|
## Serialization Rules (SP-015)
|
|
|
|
### Text Serialization
|
|
|
|
| Platform | Input Type | Serialization | Encoding |
|
|
|----------|------------|---------------|----------|
|
|
| All | `String` | UTF-8 bytes | Base64 |
|
|
|
|
### Dictionary Serialization
|
|
|
|
| Platform | Input Type | Serialization | Encoding |
|
|
|----------|------------|---------------|----------|
|
|
| All | `Object`/`Dict` | JSON string | Base64 or direct JSON |
|
|
|
|
### Arrow Table Serialization
|
|
|
|
| Platform | Input Type | Serialization | Encoding |
|
|
|----------|------------|---------------|----------|
|
|
| Desktop | `DataFrame` | Arrow IPC stream | Base64 or arrow-ipc |
|
|
| Desktop | `Arrow.Table` | Arrow IPC stream | Base64 or arrow-ipc |
|
|
| MicroPython | ❌ | Not supported | N/A |
|
|
|
|
### JSON Table Serialization
|
|
|
|
| Platform | Input Type | Serialization | Encoding |
|
|
|----------|------------|---------------|----------|
|
|
| All | `Vector{Dict}`/`Array<Object>` | JSON array | Base64 or direct JSON |
|
|
| Desktop | `pandas.DataFrame` | JSON array | Base64 or direct JSON |
|
|
|
|
### Binary Serialization
|
|
|
|
| Platform | Input Type | Serialization | Encoding |
|
|
|----------|------------|---------------|----------|
|
|
| All | `Uint8Array`/`Buffer`/`bytes` | Raw bytes | Base64 |
|
|
|
|
---
|
|
|
|
## API Contract (SP-016, SP-017)
|
|
|
|
| Platform | Function | Requirements | Solution Design Ref |
|
|
|----------|----------|--------------|-------------------|
|
|
| All | `smartpack()` | FR-001 through FR-014, NFR-101 through NFR-107 | SD-001 through SD-008 |
|
|
| All | `smartunpack()` | FR-001 through FR-014, NFR-101 through NFR-107 | SD-001 through SD-008 |
|
|
|
|
### `smartpack` Function Signature (SP-016)
|
|
|
|
#### Julia
|
|
|
|
```julia
|
|
function smartpack(
|
|
subject::String,
|
|
data::AbstractArray{Tuple{String, T1, String}, 1};
|
|
broker_url::String = DEFAULT_BROKER_URL,
|
|
fileserver_url::String = DEFAULT_FILESERVER_URL,
|
|
fileserver_upload_handler::Function = plik_oneshot_upload,
|
|
size_threshold::Int = DEFAULT_SIZE_THRESHOLD,
|
|
correlation_id::String = string(uuid4()),
|
|
msg_purpose::String = "chat",
|
|
sender_name::String = "msghandler",
|
|
receiver_name::String = "",
|
|
receiver_id::String = "",
|
|
reply_to::String = "",
|
|
reply_to_msg_id::String = "",
|
|
msg_id::String = string(uuid4()),
|
|
sender_id::String = string(uuid4())
|
|
)::Tuple{msg_envelope_v1, String} where {T1<:Any}
|
|
```
|
|
|
|
**Note**: Publishing via the transport layer is the caller's responsibility. Returns `(env::msg_envelope_v1, env_json_str::String)`.
|
|
|
|
#### Python
|
|
|
|
```python
|
|
async def smartpack(
|
|
subject: str,
|
|
data: List[Tuple[str, Any, str]],
|
|
broker_url: str = DEFAULT_BROKER_URL,
|
|
fileserver_url: str = "http://localhost:8080",
|
|
fileserver_upload_handler: Callable = plik_oneshot_upload,
|
|
size_threshold: int = 10_000,
|
|
correlation_id: str = None,
|
|
msg_purpose: str = "chat",
|
|
sender_name: str = "msghandler",
|
|
receiver_name: str = "",
|
|
receiver_id: str = "",
|
|
reply_to: str = "",
|
|
reply_to_msg_id: str = "",
|
|
msg_id: str = None,
|
|
sender_id: str = None
|
|
) -> Tuple[Dict, str]:
|
|
```
|
|
|
|
**Note**: Publishing via the transport layer is the caller's responsibility.
|
|
|
|
|
|
|
|
#### JavaScript (Browser)
|
|
|
|
```typescript
|
|
async function smartpack(
|
|
subject: string,
|
|
data: Array<[string, any, string]>,
|
|
options?: {
|
|
broker_url?: string;
|
|
fileserver_url?: string;
|
|
fileserver_upload_handler?: Function;
|
|
size_threshold?: number;
|
|
correlation_id?: string;
|
|
msg_purpose?: string;
|
|
sender_name?: string;
|
|
receiver_name?: string;
|
|
receiver_id?: string;
|
|
reply_to?: string;
|
|
reply_to_msg_id?: string;
|
|
msg_id?: string;
|
|
sender_id?: string;
|
|
}
|
|
): Promise<[Object, string]>;
|
|
```
|
|
|
|
**Note**: Publishing via the transport layer is the caller's responsibility.
|
|
|
|
#### MicroPython
|
|
|
|
```python
|
|
def smartpack(
|
|
subject: str,
|
|
data: List[Tuple[str, Any, str]],
|
|
size_threshold: int = 100_000, # Lower threshold for memory constraints
|
|
**kwargs
|
|
) -> Tuple[Dict, str]:
|
|
```
|
|
|
|
**Note**: Publishing via the transport layer is the caller's responsibility.
|
|
|
|
#### Rust
|
|
|
|
```rust
|
|
pub async fn smartpack(
|
|
subject: &str,
|
|
data: &[(String, Payload, String)],
|
|
options: &smartpackOptions,
|
|
) -> Result<(MsgEnvelopeV1, String), msghandlerError>
|
|
|
|
// smartpackOptions struct
|
|
pub struct smartpackOptions {
|
|
pub broker_url: String,
|
|
pub fileserver_url: String,
|
|
pub fileserver_upload_handler: Option<UploadHandler>,
|
|
pub size_threshold: usize,
|
|
pub correlation_id: String,
|
|
pub msg_purpose: String,
|
|
pub sender_name: String,
|
|
pub receiver_name: String,
|
|
pub receiver_id: String,
|
|
pub reply_to: String,
|
|
pub reply_to_msg_id: String,
|
|
pub msg_id: String,
|
|
pub sender_id: String,
|
|
}
|
|
|
|
// Payload enum for type-safe data handling
|
|
#[derive(Serialize, Deserialize, Clone)]
|
|
pub enum Payload {
|
|
Text(String),
|
|
Dictionary(serde_json::Value),
|
|
ArrowTable(Vec<u8>),
|
|
JsonTable(serde_json::Value),
|
|
Image(Vec<u8>),
|
|
Audio(Vec<u8>),
|
|
Video(Vec<u8>),
|
|
Binary(Vec<u8>),
|
|
}
|
|
|
|
// MsgEnvelopeV1 struct (serde-serializable)
|
|
#[derive(Serialize, Deserialize, Clone)]
|
|
pub struct MsgEnvelopeV1 {
|
|
pub correlation_id: String,
|
|
pub msg_id: String,
|
|
pub timestamp: String,
|
|
pub send_to: String,
|
|
pub msg_purpose: String,
|
|
pub sender_name: String,
|
|
pub sender_id: String,
|
|
pub receiver_name: String,
|
|
pub receiver_id: String,
|
|
pub reply_to: String,
|
|
pub reply_to_msg_id: String,
|
|
pub broker_url: String,
|
|
pub metadata: serde_json::Value,
|
|
pub payloads: Vec<MsgPayloadV1>,
|
|
}
|
|
```
|
|
|
|
**Note**: Publishing via the transport layer is the caller's responsibility. Returns `Result<(MsgEnvelopeV1, String), msghandlerError>`. Uses `serde` for JSON serialization.
|
|
|
|
### `smartunpack` Function Signature (SP-017)
|
|
|
|
**Note**: Input is the JSON string payload from the transport subscription. Returns `(envelope::msg_envelope_v1, payloads::Array{Tuple{String, Any, String}, 1})`.
|
|
|
|
**Traceability**: FR-001 through FR-014, NFR-101 through NFR-107 | SD-001 through SD-008
|
|
|
|
#### Julia
|
|
|
|
```julia
|
|
function smartunpack(
|
|
msg_json_str::String; # Pass payload from transport subscription
|
|
fileserver_download_handler::Function = _fetch_with_backoff,
|
|
max_retries::Int = 5,
|
|
base_delay::Int = 100,
|
|
max_delay::Int = 5000
|
|
)::JSON.Object{String, Any}
|
|
```
|
|
|
|
**Note**: Input is the JSON string payload from the transport subscription, not the transport message object directly.
|
|
|
|
#### Python
|
|
|
|
```python
|
|
async def smartunpack(
|
|
msg_json_str: str, # JSON string from transport message payload
|
|
fileserver_download_handler: Callable = fetch_with_backoff,
|
|
max_retries: int = 5,
|
|
base_delay: int = 100,
|
|
max_delay: int = 5000
|
|
) -> Dict[str, Any]:
|
|
```
|
|
|
|
**Note**: Input is the JSON string payload from the transport message.
|
|
|
|
#### JavaScript (Browser)
|
|
|
|
```typescript
|
|
async function smartunpack(
|
|
msg_json_str: string, // JSON string from transport message payload
|
|
options?: {
|
|
fileserver_download_handler?: Function;
|
|
max_retries?: number;
|
|
base_delay?: number;
|
|
max_delay?: number;
|
|
}
|
|
): Promise<Object>;
|
|
```
|
|
|
|
**Note**: Input is the JSON string payload from the transport message.
|
|
|
|
#### MicroPython
|
|
|
|
```python
|
|
def smartunpack(msg_json_str: str, **kwargs) -> Dict[str, Any]:
|
|
```
|
|
|
|
**Note**: Input is the JSON string payload from the transport message.
|
|
|
|
#### Rust
|
|
|
|
```rust
|
|
pub async fn smartunpack(
|
|
msg_json_str: &str, // JSON string from transport message payload
|
|
options: &smartunpackOptions,
|
|
) -> Result<MsgEnvelopeV1, msghandlerError>
|
|
|
|
// smartunpackOptions struct
|
|
pub struct smartunpackOptions {
|
|
pub fileserver_download_handler: Option<DownloadHandler>,
|
|
pub max_retries: u32,
|
|
pub base_delay: u64,
|
|
pub max_delay: u64,
|
|
}
|
|
```
|
|
|
|
**Note**: Input is the JSON string payload from the transport message. Returns `Result<MsgEnvelopeV1, msghandlerError>`.
|
|
|
|
---
|
|
|
|
## File Server Interface (SP-018, SP-019)
|
|
|
|
**Traceability**: FR-008, FR-009, FR-010, NFR-202 | SD-003, SD-007
|
|
|
|
### Upload Handler Contract (SP-018)
|
|
|
|
**Function Signature**:
|
|
```julia
|
|
function fileserver_upload_handler(
|
|
file_server_url::String,
|
|
dataname::String,
|
|
data::Vector{UInt8}
|
|
)::Dict{String, Any}
|
|
|
|
# Overload: Upload file from disk
|
|
function fileserver_upload_handler(
|
|
file_server_url::String,
|
|
filepath::String
|
|
)::Dict{String, Any}
|
|
```
|
|
|
|
**Return Format**:
|
|
```json
|
|
{
|
|
"status": 200,
|
|
"uploadid": "string",
|
|
"fileid": "string",
|
|
"url": "string"
|
|
}
|
|
```
|
|
|
|
**Required Keys**:
|
|
| Key | Type | Description | Requirement ID | Solution Design Ref |
|
|
|-----|------|-------------|----------------|-------------------|
|
|
| `status` | `integer` | HTTP response status code | FR-008, FR-009 | SD-003 |
|
|
| `uploadid` | `string` | Upload session identifier | FR-008 | SD-003 |
|
|
| `fileid` | `string` | File identifier within session | FR-008 | SD-003 |
|
|
| `url` | `string` | Full download URL | FR-008, FR-009, FR-010 | SD-003, SD-007 |
|
|
|
|
### Download Handler Contract (SP-019)
|
|
|
|
**Function Signature**:
|
|
|
|
**Traceability**: FR-010, NFR-202 | SD-003, SD-007
|
|
|
|
**Function Signature**:
|
|
```julia
|
|
function fileserver_download_handler(
|
|
url::String,
|
|
max_retries::Int,
|
|
base_delay::Int,
|
|
max_delay::Int,
|
|
correlation_id::String
|
|
)::Vector{UInt8}
|
|
```
|
|
|
|
**Retry Policy**:
|
|
- Initial delay: `base_delay` milliseconds
|
|
- Maximum delay: `max_delay` milliseconds
|
|
- Multiplier: 2x per retry
|
|
- Maximum retries: `max_retries`
|
|
|
|
---
|
|
|
|
## Platform-Specific Constraints (SP-020)
|
|
|
|
**Traceability**: FR-005, FR-006, NFR-106, NFR-107 | SD-002, SD-004
|
|
|
|
### Desktop (Julia/Python)
|
|
|
|
| Feature | Status | Notes | Requirement ID | Solution Design Ref |
|
|
|---------|--------|-------|----------------|-------------------|
|
|
| Arrow IPC | ✅ Supported | Requires Arrow.jl/pyarrow | FR-002, FR-012 | SD-004, SD-005 |
|
|
| JSON table | ✅ Supported | Human-readable format | FR-002, FR-006 | SD-004, SD-005 |
|
|
| File server upload | ✅ Supported | HTTP/HTTPS | FR-008, FR-009 | SD-003 |
|
|
| File server download | ✅ Supported | HTTP/HTTPS | FR-010, NFR-202 | SD-003, SD-007 |
|
|
| Size threshold | 10,000 bytes (10KB) | Configurable | FR-004, FR-005, NFR-104, NFR-105 | SD-002 |
|
|
|
|
### Browser (JavaScript)
|
|
|
|
| Feature | Status | Notes | Requirement ID | Solution Design Ref |
|
|
|---------|--------|-------|----------------|-------------------|
|
|
| Arrow IPC | ❌ Not supported | Apache Arrow not browser-compatible | FR-002 | SD-004, SD-005 |
|
|
| JSON table | ✅ Supported | Only table type available in browser | FR-002, FR-006 | SD-004, SD-005 |
|
|
| File server upload | ✅ Supported | HTTP/HTTPS | FR-008, FR-009 | SD-003 |
|
|
| File server download | ✅ Supported | HTTP/HTTPS | FR-010, NFR-202 | SD-003, SD-007 |
|
|
| Size threshold | 10,000 bytes (10KB) | Configurable | FR-004, FR-005, NFR-104, NFR-105 | SD-002 |
|
|
|
|
### Rust
|
|
|
|
| Feature | Status | Notes | Requirement ID | Solution Design Ref |
|
|
|---------|--------|-------|----------------|-------------------|
|
|
| Arrow IPC | ✅ Supported | Requires `arrow2` crate | FR-002, FR-012 | SD-004, SD-005 |
|
|
| JSON table | ✅ Supported | Uses `serde_json` | FR-002, FR-006 | SD-004, SD-005 |
|
|
| File server upload | ✅ Supported | HTTP/HTTPS via `reqwest` | FR-008, FR-009 | SD-003 |
|
|
| File server download | ✅ Supported | HTTP/HTTPS via `reqwest` with retry | FR-010, NFR-202 | SD-003, SD-007 |
|
|
| Size threshold | 10,000 bytes (10KB) | Configurable | FR-004, FR-005, NFR-104, NFR-105 | SD-002 |
|
|
| Async runtime | ✅ Supported | Uses `tokio` for async I/O | FR-013, FR-014 | SD-006 |
|
|
| Type safety | ✅ Supported | Compile-time type checking via Rust enums | FR-006, FR-007 | SD-004 |
|
|
|
|
### MicroPython
|
|
|
|
| Feature | Status | Notes | Requirement ID | Solution Design Ref |
|
|
|---------|--------|-------|----------------|-------------------|
|
|
| Arrow IPC | ❌ Not supported | Memory constraints | FR-005 | SD-002 |
|
|
| JSON table | ⚠️ Limited | Only direct transport | FR-005, FR-006 | SD-002, SD-004 |
|
|
| File server upload | ❌ Not implemented | Placeholder only | FR-005 | SD-002 |
|
|
| File server download | ❌ Not implemented | Placeholder only | FR-005 | SD-002 |
|
|
| Size threshold | 100KB | Hard limit enforced | FR-005, NFR-106 | SD-002 |
|
|
| Max payload | 50KB | Hard limit enforced | FR-005 | SD-002 |
|
|
|
|
---
|
|
|
|
## Implementation Files (SP-021)
|
|
|
|
**Traceability**: FR-001 through FR-007 | SD-001 through SD-008
|
|
|
|
| File | Platform | Features | Notes | Requirement ID | Solution Design Ref |
|
|
|------|----------|----------|-------|----------------|-------------------|
|
|
| [`src/msghandler.jl`](../src/msghandler.jl) | Julia | Full feature set, Arrow IPC, multiple dispatch | Ground truth implementation | FR-001 through FR-014, NFR-101 through NFR-107 | SD-001 through SD-008 |
|
|
| [`src/msghandler_csr.js`](../src/msghandler_csr.js) | Browser | JSON table only | Client-side rendering | FR-001 through FR-014, NFR-101 through NFR-107 | SD-001 through SD-008 |
|
|
| [`src/msghandler.py`](../src/msghandler.py) | Python | Arrow IPC, async/await | Desktop Python | FR-001 through FR-014, NFR-101 through NFR-107 | SD-001 through SD-008 |
|
|
| [`src/msghandler.rs`](../src/msghandler.rs) | Rust | Full feature set, Arrow IPC, async/await, type-safe | Uses tokio + serde + arrow2 | FR-001 through FR-014, NFR-101 through NFR-107 | SD-001 through SD-008 |
|
|
| [`src/msghandler_mpy.py`](../src/msghandler_mpy.py) | MicroPython | Limited to direct transport | Memory-constrained | FR-005, FR-006 | SD-002, SD-004 |
|
|
|
|
### Browser Implementation Notes
|
|
|
|
**Traceability**: FR-002, FR-006 | SD-004, SD-005
|
|
|
|
The browser implementation ([`src/msghandler_csr.js`](../src/msghandler_csr.js)) has the following constraints:
|
|
|
|
| Constraint | Reason | Workaround | Requirement ID | Solution Design Ref |
|
|
|------------|--------|------------|----------------|-------------------|
|
|
| No Apache Arrow IPC | Browser-incompatible dependency | Use `jsontable` for tabular data | FR-002 | SD-004, SD-005 |
|
|
| WebSocket only | Browser cannot use TCP directly | Use `ws://` or `wss://` broker URLs | FR-013, FR-014 | SD-006 |
|
|
| Fetch API for HTTP | Browser fetch() API only | Compatible with Plik and other HTTP servers | FR-008, FR-009 | SD-003 |
|
|
|
|
### Payload Type Availability by Platform
|
|
|
|
| Payload Type | Julia | Browser | Python | Rust | MicroPython |
|
|
|--------------|-------|---------|--------|------|-------------|
|
|
| `text` | ✅ | ✅ | ✅ | ✅ | ✅ |
|
|
| `dictionary` | ✅ | ✅ | ✅ | ✅ | ✅ |
|
|
| `arrowtable` | ✅ | ❌ | ✅ | ✅ | ❌ |
|
|
| `jsontable` | ✅ | ✅ | ✅ | ✅ | ⚠️ |
|
|
| `image` | ✅ | ✅ | ✅ | ✅ | ✅ |
|
|
| `audio` | ✅ | ✅ | ✅ | ✅ | ✅ |
|
|
| `video` | ✅ | ✅ | ✅ | ✅ | ✅ |
|
|
| `binary` | ✅ | ✅ | ✅ | ✅ | ✅ |
|
|
|
|
---
|
|
|
|
## Message Flow (SP-022)
|
|
|
|
### Sending Flow
|
|
|
|
```mermaid
|
|
flowchart TD
|
|
A[User calls smartpack subject data] --> B[Serialize payload according to payload_type]
|
|
B --> C{Calculate serialized size}
|
|
C -->|Size < Threshold| D[Direct Transport: Encode as Base64]
|
|
C -->|Size >= Threshold| E[Link Transport: Upload to file server]
|
|
D --> F[Build envelope with metadata]
|
|
E --> F
|
|
F --> G[Convert envelope to JSON string]
|
|
G --> H[Publish to topic via transport]
|
|
H --> I[Return envelope and JSON string to caller]
|
|
|
|
style A fill:#f9f9f9,stroke:#333
|
|
style I fill:#e0e7ff,stroke:#3b82f6
|
|
style D fill:#d1fae5,stroke:#10b981
|
|
style E fill:#fef3c7,stroke:#f59e0b
|
|
```
|
|
|
|
### Receiving Flow
|
|
|
|
```mermaid
|
|
flowchart TD
|
|
A[Transport message arrives] --> B[Parse JSON envelope]
|
|
B --> C[For each payload: Check transport type]
|
|
C -->|transport == direct| D[Direct Transport: Extract Base64]
|
|
C -->|transport == link| E[Link Transport: Fetch from URL]
|
|
D --> F[Decode Base64]
|
|
E --> G[Fetch with exponential backoff]
|
|
F --> H[Deserialize based on payload_type]
|
|
G --> H
|
|
H --> I[Build payloads array]
|
|
I --> J[Replace payloads array with deserialized tuples]
|
|
J --> K[Return envelope with processed payloads]
|
|
|
|
style A fill:#f9f9f9,stroke:#333
|
|
style K fill:#e0e7ff,stroke:#3b82f6
|
|
style D fill:#d1fae5,stroke:#10b981
|
|
style E fill:#fef3c7,stroke:#f59e0b
|
|
```
|
|
|
|
---
|
|
|
|
## Validation Rules (SP-023)
|
|
|
|
**Traceability**: FR-001 through FR-014, NFR-101 through NFR-405 | SD-001 through SD-008
|
|
|
|
### Envelope Validation
|
|
|
|
| Rule | Condition | Error Code | Requirement ID | Solution Design Ref |
|
|
|------|-----------|------------|----------------|-------------------|
|
|
| Required fields present | `correlation_id`, `msg_id`, `timestamp`, `send_to`, `payloads` | `INVALID_ENVELOPE` | FR-012, FR-013 | SD-001, SD-006 |
|
|
| Valid UUID format | `correlation_id`, `msg_id`, `sender_id`, `receiver_id` | `INVALID_ENVELOPE` | FR-011, FR-012, NFR-401 | SD-001, SD-008 |
|
|
| Valid timestamp format | ISO 8601 UTC | `INVALID_ENVELOPE` | FR-012, NFR-401 | SD-001, SD-008 |
|
|
| Non-empty payloads array | `length(payloads) > 0` | `INVALID_ENVELOPE` | FR-012, FR-013 | SD-001, SD-005 |
|
|
|
|
### Payload Validation
|
|
|
|
| Rule | Condition | Error Code | Requirement ID | Solution Design Ref |
|
|
|------|-----------|------------|----------------|-------------------|
|
|
| Valid payload_type | Must be in `payload_type` enum | `INVALID_PAYLOAD_TYPE` | FR-001, FR-002, FR-003, FR-006 | SD-004, SD-005 |
|
|
| Valid transport | Must be `direct` or `link` | `INVALID_TRANSPORT` | FR-003, FR-004, FR-006 | SD-001, SD-002 |
|
|
| Valid encoding | Must match payload_type and transport | `INVALID_TRANSPORT` | FR-001, FR-002, FR-003, FR-012 | SD-004, SD-005 |
|
|
| Positive size | `size > 0` | `INVALID_PAYLOAD` | FR-003, FR-004, NFR-104, NFR-105 | SD-001, SD-002 |
|
|
| Valid Base64 for direct | `data` matches Base64 pattern | `DESERIALIZATION_ERROR` | FR-001, FR-002, FR-003, FR-012 | SD-004, SD-005 |
|
|
| Valid URL for link | `data` matches HTTP(S) URL pattern | `DOWNLOAD_FAILED` | FR-008, FR-009, FR-010 | SD-003, SD-007 |
|
|
|
|
---
|
|
|
|
## Test Contracts (SP-024)
|
|
|
|
**Traceability**: FR-001 through FR-014, NFR-101 through NFR-107 | SD-001 through SD-008
|
|
|
|
### Unit Test Validation
|
|
|
|
| Test | Input | Expected Output | Notes | Requirement ID | Solution Design Ref |
|
|
|------|-------|-----------------|-------|----------------|-------------------|
|
|
| Text round-trip | `("msg", "Hello", "text")` | `("msg", "Hello", "text")` | String serialization | FR-001, FR-012, NFR-101, NFR-102 | SD-004, SD-005 |
|
|
| Dictionary round-trip | `("data", {"key": "value"}, "dictionary")` | `("data", {"key": "value"}, "dictionary")` | JSON object round-trip | FR-002, FR-012, NFR-101, NFR-102 | SD-004, SD-005 |
|
|
| Arrow table round-trip | `("table", arrow_table_data, "arrowtable")` | `("table", arrow_table_data, "arrowtable")` | Arrow IPC round-trip | FR-002, FR-012, NFR-101, NFR-102 | SD-004, SD-005 |
|
|
| JSON table round-trip | `("table", [{"a":1},{"b":2}], "jsontable")` | `("table", [{"a":1},{"b":2}], "jsontable")` | JSON array of objects | FR-001, FR-002, FR-006, FR-012 | SD-004, SD-005 |
|
|
| Mixed payloads | `[("msg", "Hello", "text"), ("imgname", bytes, "binary")]` | `[("msg", "Hello", "text"), ("imgname", bytes, "binary")]` | Multiple payload types | FR-006, FR-007 | SD-004 |
|
|
| Large payload | `("data", rand(10_000_000), "arrowtable")` | `("data", URL, "arrowtable")` with link transport | File server upload | FR-003, FR-004, FR-008, FR-009, NFR-104, NFR-105 | SD-001, SD-002, SD-003 |
|
|
|
|
**Platform-Specific Notes:**
|
|
- **Julia**: Use `Dict`, `Vector{Dict}`, or convert `DataFrame` to dictionary for testing
|
|
- **Python**: Use `dict`, `list[dict]`, or convert `pandas.DataFrame` to dictionary for testing
|
|
- **JavaScript**: Use plain objects `{}` and arrays `[]`
|
|
- **MicroPython**: Use plain `dict` and `list` (limited to JSON table and text types)
|
|
|
|
### Integration Test Scenarios
|
|
|
|
| Scenario | Platforms | Payloads | Size Mix | Transport | Expected Result | Requirement ID | Solution Design Ref |
|
|
|----------|-----------|----------|----------|-----------|-----------------|----------------|-------------------|
|
|
| Single text (small) | All | `text` | Small | direct | Round-trip successful | FR-001, FR-012, NFR-101, NFR-102 | SD-004, SD-005 |
|
|
| Single dictionary (small) | All | `dictionary` | Small | direct | Round-trip successful | FR-002, FR-012, NFR-101, NFR-102 | SD-004, SD-005 |
|
|
| Single arrow table (small) | Julia/JS/Python | `arrowtable` | Small | direct | Arrow IPC round-trip | FR-002, FR-012, NFR-101, NFR-102 | SD-004, SD-005 |
|
|
| Single JSON table (small) | All | `jsontable` | Small | direct | Dictionary array round-trip | FR-001, FR-002, FR-006, FR-012 | SD-004, SD-005 |
|
|
| Single image (small) | All | `image` | Small | direct | Binary round-trip | FR-001, FR-006, FR-012 | SD-004, SD-005 |
|
|
| Single audio (small) | All | `audio` | Small | direct | Binary round-trip | FR-001, FR-006, FR-012 | SD-004, SD-005 |
|
|
| Single video (small) | All | `video` | Small | direct | Binary round-trip | FR-001, FR-006, FR-012 | SD-004, SD-005 |
|
|
| Single binary (small) | All | `binary` | Small | direct | Binary round-trip | FR-001, FR-006, FR-012 | SD-004, SD-005 |
|
|
| Single text (large) | All | `text` | Large | link | File server upload/download | FR-003, FR-004, FR-008, FR-009, NFR-104, NFR-105 | SD-001, SD-002, SD-003 |
|
|
| Single JSON table (large) | All | `jsontable` | Large | link | File server upload/download | FR-003, FR-004, FR-008, FR-009, NFR-104, NFR-105 | SD-001, SD-002, SD-003 |
|
|
| Single image (large) | All | `image` | Large | link | File server upload/download | FR-003, FR-004, FR-008, FR-009, NFR-104, NFR-105 | SD-001, SD-002, SD-003 |
|
|
| **Ultimate Test** | Julia/JS/Python | All types | Mixed | direct/link | All payloads preserved with correct transport | FR-001 through FR-014, NFR-101 through NFR-107 | SD-001 through SD-008 |
|
|
| **Ultimate Test** | MicroPython | `text`/`dictionary` | Mixed | direct | Limited to text/dictionary with direct transport only | FR-005, FR-006, FR-012 | SD-002, SD-004 |
|
|
| Cross-platform JSON table | All | `jsontable` | Small | direct | Dictionary array round-trip | FR-001, FR-002, FR-006, FR-012 | SD-004, SD-005 |
|
|
| MicroPython ↔ Desktop | MicroPython ↔ Desktop | `text`/`dictionary` | Small | direct | Limited payload types | FR-005, FR-006, FR-012 | SD-002, SD-004 |
|
|
| Desktop ↔ Desktop (all combos) | Julia↔JS↔Python | All types | Small/Large | direct/link | Full compatibility | FR-001, FR-002, FR-003, FR-004, FR-005, FR-006, FR-007, FR-012, FR-013, FR-014 |
|
|
|
|
---
|
|
|
|
## Dependencies (SP-025)
|
|
|
|
### Required Dependencies by Platform
|
|
|
|
| Platform | Package | Version | Purpose | Requirement ID | Solution Design Ref |
|
|
|----------|---------|---------|---------|----------------|-------------------|
|
|
| Julia | JSON.jl | Latest | JSON serialization | FR-012, NFR-101, NFR-102 | SD-005 |
|
|
| Julia | Arrow.jl | Latest | Arrow IPC support | FR-002, FR-012 | SD-004, SD-005 |
|
|
| Julia | HTTP.jl | Latest | HTTP file server | FR-008, FR-009 | SD-003 |
|
|
| Julia | UUIDs.jl | Latest | UUID generation | FR-011, NFR-401 | SD-008 |
|
|
| Browser | - | - | Transport-agnostic (caller provides) | FR-013, FR-014 | SD-006 |
|
|
| Python | aiohttp | Latest | HTTP file server | FR-008, FR-009 | SD-003 |
|
|
| Python | pyarrow | Latest | Arrow IPC support | FR-002, FR-012 | SD-004, SD-005 |
|
|
| Rust | serde | Latest | JSON serialization | FR-012, NFR-101, NFR-102 | SD-005 |
|
|
| Rust | serde_json | Latest | JSON handling | FR-012, NFR-101, NFR-102 | SD-005 |
|
|
| Rust | tokio | Latest | Async runtime | FR-013, FR-014 | SD-006 |
|
|
| Rust | reqwest | Latest | HTTP file server | FR-008, FR-009 | SD-003 |
|
|
| Rust | uuid | Latest | UUID generation | FR-011, NFR-401 | SD-008 |
|
|
| Rust | arrow2 | Latest | Arrow IPC support | FR-002, FR-012 | SD-004, SD-005 |
|
|
| MicroPython | builtin | N/A | Limited implementation | FR-005, FR-006 | SD-002 |
|
|
|
|
### Optional Dependencies
|
|
|
|
| Platform | Package | Purpose |
|
|
|----------|---------|---------|
|
|
| Julia | DataFrames.jl | DataFrame support |
|
|
| Python | pandas | DataFrame support |
|
|
|
|
---
|
|
|
|
## Change Log
|
|
|
|
| Date | Version | Changes |
|
|
|------|---------|---------|
|
|
| 2026-09-16 | 1.4.0 | Aligned to ASG Framework eight pillars; added SP-XXX Specification Item IDs, Specification Traceability Matrix, section-level SP references, and Rate Limiting & Quotas (SP-010.1) |
|
|
| 2026-05-15 | 1.3.0 | Made transport layer agnostic |
|
|
| - | - | Removed all NATS-specific dependencies (NATS.jl, nats, nats-py, nats.ws) |
|
|
| - | - | Updated docs to reference generic message broker/transport |
|
|
| - | - | broker_url is now metadata only, not used for active connections |
|
|
| 2026-03-15 | 1.1.0 | Browser connection management |
|
|
| - | - | Added NATSClient class with keepAlive support |
|
|
| - | - | Added NATSConnectionPool for connection reuse |
|
|
| - | - | Added publishMessage function with closeConnection option |
|
|
| - | - | Added nats.ws to browser dependencies |
|
|
| 2026-03-13 | 1.0.0 | Initial specification |
|
|
| - | - | Message envelope schema defined |
|
|
| - | - | Payload schema with transport modes |
|
|
| - | - | Enumerations for payload_type, transport, encoding |
|
|
| - | - | Size thresholds for desktop/MicroPython |
|
|
| - | - | Error codes and validation rules |
|
|
| - | - | API contracts for all platforms |
|
|
|
|
---
|
|
|
|
## References
|
|
|
|
**ASG Framework Reference**: This specification follows the ASG Framework v8 pillars. Each specification item must cite:
|
|
- **Requirement ID(s)** from requirements.md (e.g., FR-001, NFR-201) — defines *what* the system must do
|
|
- **Solution Design reference(s)** from solution-design.md (e.g., SD-2.1, SD-3.3) — defines *how* and *why* this technical approach was chosen
|
|
|
|
### 20.1 Documentation Artifacts
|
|
|
|
| Document | Purpose | Requirements Traceability | Solution Design Traceability |
|
|
|----------|---------|--------------------------|------------------------------|
|
|
| [`docs/requirements.md`](./requirements.md) | Business requirements and user stories | FR-001 through FR-014, NFR-101 through NFR-405 | SD-001 through SD-008 |
|
|
| [`docs/solution-design.md`](./solution-design.md) | Technical solution approach | FR-001 through FR-014, NFR-101 through NFR-405 | SD-001 through SD-008 |
|
|
| [`docs/specification.md`](./specification.md) | Technical contract for msghandler | FR-001 through FR-014, NFR-101 through NFR-405 | SD-001 through SD-008 |
|
|
| [`docs/ui-specification.md`](./ui-specification.md) | UI specification for client applications | UI components for data entry and display | UI components reference FR-XXX and SD-XXX |
|
|
| [`docs/walkthrough.md`](./walkthrough.md) | End-to-end system flow | Traceability from user journey to technical implementation | Full flow validation against SD-XXX |
|
|
| [`docs/architecture.md`](./architecture.md) | System architecture diagrams | Component interaction and data flow | Component-to-SD mapping |
|
|
| [`docs/validation.md`](./validation.md) | CI/CD validation rules | Contract testing and spec compliance | Validation gates for SD-XXX |
|
|
| [`docs/runbook.md`](./runbook.md) | Operational runbook | Deployment, scaling, and troubleshooting | Operation-to-SD mapping |
|
|
|
|
### 20.2 Implementation Files
|
|
|
|
| File | Platform | Features | Requirements Traceability | Solution Design Traceability |
|
|
|------|----------|----------|--------------------------|------------------------------|
|
|
| [`src/msghandler.jl`](../src/msghandler.jl) | Julia | Full feature set, Arrow IPC, multiple dispatch | FR-001 through FR-014, NFR-101 through NFR-405 | SD-001 through SD-008 |
|
|
| [`src/msghandler_csr.js`](../src/msghandler_csr.js) | Browser | JSON table only | FR-001 through FR-014, NFR-101 through NFR-405 | SD-001 through SD-008 |
|
|
| [`src/msghandler.py`](../src/msghandler.py) | Python | Arrow IPC, async/await | FR-001 through FR-014, NFR-101 through NFR-405 | SD-001 through SD-008 |
|
|
| [`src/msghandler.rs`](../src/msghandler.rs) | Rust | Full feature set, Arrow IPC, async/await, type-safe | FR-001 through FR-014, NFR-101 through NFR-405 | SD-001 through SD-008 |
|
|
| [`src/msghandler_mpy.py`](../src/msghandler_mpy.py) | MicroPython | Limited to direct transport | FR-005, FR-006, FR-012 | SD-002, SD-004 |
|
|
|
|
### 20.3 External Dependencies
|
|
|
|
| Platform | Package | Version | Purpose | Requirements Traceability | Solution Design Traceability |
|
|
|----------|---------|---------|---------|--------------------------|------------------------------|
|
|
| Julia | JSON.jl | Latest | JSON serialization | FR-012, NFR-101, NFR-102 | SD-005 |
|
|
| Julia | Arrow.jl | Latest | Arrow IPC support | FR-002, FR-012 | SD-004, SD-005 |
|
|
| Julia | HTTP.jl | Latest | HTTP file server | FR-008, FR-009 | SD-003 |
|
|
| Julia | UUIDs.jl | Latest | UUID generation | FR-011, NFR-401 | SD-008 |
|
|
| Browser | - | - | Transport-agnostic (caller provides) | FR-013, FR-014 | SD-006 |
|
|
| Python | aiohttp | Latest | HTTP file server | FR-008, FR-009 | SD-003 |
|
|
| Python | pyarrow | Latest | Arrow IPC support | FR-002, FR-012 | SD-004, SD-005 |
|
|
| Rust | serde | Latest | JSON serialization | FR-012, NFR-101, NFR-102 | SD-005 |
|
|
| Rust | serde_json | Latest | JSON handling | FR-012, NFR-101, NFR-102 | SD-005 |
|
|
| Rust | tokio | Latest | Async runtime | FR-013, FR-014 | SD-006 |
|
|
| Rust | reqwest | Latest | HTTP file server | FR-008, FR-009 | SD-003 |
|
|
| Rust | uuid | Latest | UUID generation | FR-011, NFR-401 | SD-008 |
|
|
| Rust | arrow2 | Latest | Arrow IPC support | FR-002, FR-012 | SD-004, SD-005 |
|
|
| MicroPython | builtin | N/A | Limited implementation | FR-005, FR-006 | SD-002 |
|
|
|
|
---
|
|
|
|
## Change Log
|
|
|
|
| Date | Version | Changes | Requirement ID(s) | Solution Design Ref(s) |
|
|
|------|---------|---------|-------------------|------------------------|
|
|
| 2026-09-16 | 1.4.0 | Aligned to ASG Framework eight pillars; assigned SP-XXX IDs to all spec items; added Specification Traceability Matrix, Rate Limiting & Quotas (SP-010.1); fixed desktop size threshold to 10,000 bytes per ground truth | All | All SD-XXX |
|
|
| 2026-05-22 | 1.3.0 | Updated to ASG Framework v8 pillars | All | All SD-XXX |
|
|
| - | - | Added Solution Design references to all specification items | All | All SD-XXX |
|
|
| - | - | Updated version to 1.3.0 to match requirements and solution-design | All | - |
|
|
| - | - | Added ASG Framework alignment section | All | - |
|
|
| 2026-05-15 | 1.3.0 | Made transport layer agnostic | All | SD-006 |
|
|
| - | - | Removed all NATS-specific dependencies (NATS.jl, nats, nats-py, nats.ws) | All | SD-006 |
|
|
| - | - | Updated docs to reference generic message broker/transport | All | SD-006 |
|
|
| - | - | broker_url is now metadata only, not used for active connections | All | SD-006 |
|
|
| 2026-03-15 | 1.1.0 | Browser connection management | FR-001 through FR-014 | SD-006 |
|
|
| - | - | Added NATSClient class with keepAlive support | FR-013, FR-014 | SD-006 |
|
|
| - | - | Added NATSConnectionPool for connection reuse | FR-013, FR-014 | SD-006 |
|
|
| - | - | Added publishMessage function with closeConnection option | FR-013, FR-014 | SD-006 |
|
|
| - | - | Added nats.ws to browser dependencies | FR-013, FR-014 | SD-006 |
|
|
| 2026-03-13 | 1.0.0 | Initial specification | FR-001 through FR-014, NFR-101 through NFR-405 | SD-001 through SD-008 |
|
|
| - | - | Message envelope schema defined | FR-012, FR-013 | SD-001, SD-006 |
|
|
| - | - | Payload schema with transport modes | FR-001, FR-002, FR-003, FR-004 | SD-001, SD-002 |
|
|
| - | - | Enumerations for payload_type, transport, encoding | FR-001, FR-002, FR-003, FR-006 | SD-004, SD-005 |
|
|
| - | - | Size thresholds for desktop/MicroPython | FR-004, FR-005 | SD-002 |
|
|
| - | - | Error codes and validation rules | FR-001 through FR-010 | SD-001 through SD-007 |
|
|
| - | - | API contracts for all platforms | FR-001 through FR-014 | SD-001 through SD-008 |
|
|
|
|
---
|
|
|
|
## Appendix
|
|
|
|
### A. Complete JSON Schema
|
|
|
|
```json
|
|
{
|
|
"$schema": "http://json-schema.org/draft-07/schema#",
|
|
"title": "msghandler Envelope",
|
|
"type": "object",
|
|
"properties": {
|
|
"correlation_id": {
|
|
"type": "string",
|
|
"pattern": "^[0-9A-F]{8}-[0-9A-F]{4}-[0-9A-F]{4}-[0-9A-F]{4}-[0-9A-F]{12}$",
|
|
"description": "UUID v4 format for tracking message flow"
|
|
},
|
|
"msg_id": {
|
|
"type": "string",
|
|
"pattern": "^[0-9A-F]{8}-[0-9A-F]{4}-[0-9A-F]{4}-[0-9A-F]{4}-[0-9A-F]{12}$",
|
|
"description": "Unique message identifier"
|
|
},
|
|
"timestamp": {
|
|
"type": "string",
|
|
"pattern": "^\\d{4}-\\d{2}-\\d{2}T\\d{2}:\\d{2}:\\d{2}\\.\\d{3}Z$",
|
|
"description": "ISO 8601 UTC timestamp"
|
|
},
|
|
"send_to": {
|
|
"type": "string",
|
|
"minLength": 1,
|
|
"description": "Topic/subject to publish to"
|
|
},
|
|
"msg_purpose": {
|
|
"type": "string",
|
|
"enum": ["ACK", "NACK", "updateStatus", "shutdown", "chat", "command", "event"],
|
|
"description": "Purpose of the message"
|
|
},
|
|
"sender_name": {
|
|
"type": "string",
|
|
"minLength": 1,
|
|
"description": "Sender application name"
|
|
},
|
|
"sender_id": {
|
|
"type": "string",
|
|
"pattern": "^[0-9A-F]{8}-[0-9A-F]{4}-[0-9A-F]{4}-[0-9A-F]{4}-[0-9A-F]{12}$",
|
|
"description": "Sender UUID"
|
|
},
|
|
"receiver_name": {
|
|
"type": "string",
|
|
"description": "Receiver name (empty = broadcast)"
|
|
},
|
|
"receiver_id": {
|
|
"type": "string",
|
|
"pattern": "^[0-9A-F]{8}-[0-9A-F]{4}-[0-9A-F]{4}-[0-9A-F]{4}-[0-9A-F]{12}$|^$",
|
|
"description": "Receiver UUID (empty = broadcast)"
|
|
},
|
|
"reply_to": {
|
|
"type": "string",
|
|
"description": "Topic for reply messages"
|
|
},
|
|
"reply_to_msg_id": {
|
|
"type": "string",
|
|
"description": "Message ID being replied to"
|
|
},
|
|
"broker_url": {
|
|
"type": "string",
|
|
"description": "Broker URL for the transport layer"
|
|
},
|
|
"metadata": {
|
|
"type": "object",
|
|
"description": "Message-level metadata"
|
|
},
|
|
"payloads": {
|
|
"type": "array",
|
|
"minItems": 1,
|
|
"items": {
|
|
"$ref": "#/definitions/Payload"
|
|
}
|
|
}
|
|
},
|
|
"required": ["correlation_id", "msg_id", "timestamp", "send_to", "msg_purpose", "sender_name", "sender_id", "receiver_name", "receiver_id", "reply_to", "reply_to_msg_id", "broker_url", "payloads"],
|
|
"definitions": {
|
|
"Payload": {
|
|
"type": "object",
|
|
"properties": {
|
|
"id": {
|
|
"type": "string",
|
|
"pattern": "^[0-9A-F]{8}-[0-9A-F]{4}-[0-9A-F]{4}-[0-9A-F]{4}-[0-9A-F]{12}$"
|
|
},
|
|
"dataname": {
|
|
"type": "string",
|
|
"minLength": 1
|
|
},
|
|
"payload_type": {
|
|
"type": "string",
|
|
"enum": ["text", "dictionary", "arrowtable", "jsontable", "image", "audio", "video", "binary"]
|
|
},
|
|
"transport": {
|
|
"type": "string",
|
|
"enum": ["direct", "link"]
|
|
},
|
|
"encoding": {
|
|
"type": "string",
|
|
"enum": ["none", "base64", "json", "arrow-ipc"]
|
|
},
|
|
"size": {
|
|
"type": "integer",
|
|
"minimum": 1
|
|
},
|
|
"data": {
|
|
"anyOf": [
|
|
{
|
|
"type": "string",
|
|
"pattern": "^(https?://[^\\s]+)$"
|
|
},
|
|
{
|
|
"type": "string",
|
|
"pattern": "^[A-Za-z0-9+/]+=*$"
|
|
}
|
|
]
|
|
},
|
|
"metadata": {
|
|
"type": "object"
|
|
}
|
|
},
|
|
"required": ["id", "dataname", "payload_type", "transport", "encoding", "size", "data"]
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
### B. AsyncAPI Specification
|
|
|
|
```yaml
|
|
asyncapi: '2.6.0'
|
|
info:
|
|
title: msghandler API
|
|
version: '1.0.0'
|
|
description: Cross-platform bi-directional data bridge using a message broker
|
|
contact:
|
|
name: msghandler Team
|
|
url: https://github.com/your-org/msghandler
|
|
license:
|
|
name: MIT
|
|
url: https://opensource.org/licenses/MIT
|
|
channels:
|
|
/agent/{service}/api/v{version}/{operation}:
|
|
address: /agent/{service}/api/v{version}/{operation}
|
|
parameters:
|
|
service:
|
|
schema:
|
|
type: string
|
|
version:
|
|
schema:
|
|
type: string
|
|
enum: ['v1']
|
|
operation:
|
|
schema:
|
|
type: string
|
|
publish:
|
|
summary: Publish message to transport
|
|
operationId: publishMessage
|
|
message:
|
|
$ref: '#/components/message'
|
|
subscribe:
|
|
summary: Subscribe to messages from transport
|
|
operationId: subscribeMessage
|
|
message:
|
|
$ref: '#/components/message'
|
|
components:
|
|
message:
|
|
payload:
|
|
$ref: '#/components/schemas/Envelope'
|
|
schemas:
|
|
Envelope:
|
|
type: object
|
|
properties:
|
|
correlation_id:
|
|
type: string
|
|
format: uuid
|
|
msg_id:
|
|
type: string
|
|
format: uuid
|
|
timestamp:
|
|
type: string
|
|
format: date-time
|
|
send_to:
|
|
type: string
|
|
msg_purpose:
|
|
type: string
|
|
enum: [ACK, NACK, updateStatus, shutdown, chat, command, event]
|
|
sender_name:
|
|
type: string
|
|
sender_id:
|
|
type: string
|
|
format: uuid
|
|
receiver_name:
|
|
type: string
|
|
receiver_id:
|
|
type: string
|
|
format: uuid
|
|
reply_to:
|
|
type: string
|
|
reply_to_msg_id:
|
|
type: string
|
|
broker_url:
|
|
type: string
|
|
metadata:
|
|
type: object
|
|
payloads:
|
|
type: array
|
|
items:
|
|
$ref: '#/components/schemas/Payload'
|
|
required:
|
|
- correlation_id
|
|
- msg_id
|
|
- timestamp
|
|
- send_to
|
|
- msg_purpose
|
|
- sender_name
|
|
- sender_id
|
|
- receiver_name
|
|
- receiver_id
|
|
- reply_to
|
|
- reply_to_msg_id
|
|
- broker_url
|
|
- payloads
|
|
Payload:
|
|
type: object
|
|
properties:
|
|
id:
|
|
type: string
|
|
format: uuid
|
|
dataname:
|
|
type: string
|
|
payload_type:
|
|
type: string
|
|
enum: [text, dictionary, arrowtable, jsontable, image, audio, video, binary]
|
|
transport:
|
|
type: string
|
|
enum: [direct, link]
|
|
encoding:
|
|
type: string
|
|
enum: [none, base64, json, arrow-ipc]
|
|
size:
|
|
type: integer
|
|
minimum: 1
|
|
data:
|
|
type: string
|
|
metadata:
|
|
type: object
|
|
required:
|
|
- id
|
|
- dataname
|
|
- payload_type
|
|
- transport
|
|
- encoding
|
|
- size
|
|
- data
|
|
```
|
|
|
|
---
|
|
|
|
*This specification is versioned and maintained in git alongside the codebase. All implementations must adhere to this specification.*
|