Files

narawat 6171923c05 update

2026-05-22 08:51:47 +07:00

58 KiB

Raw Blame History

Specification: msghandler

Version: 1.3.0
Date: 2026-05-22
Status: Active
Ground Truth: src/msghandler.jl
Specification Format: JSON Schema + AsyncAPI
ASG Framework Alignment: v8 pillars - Requirements → Solution Design → Specification → Walkthrough → Implementation Plan → Validation → Runbook

1. Technical Contract Overview

This document defines the technical contract for msghandler - the cross-platform bi-directional data bridge that enables seamless communication between Julia, JavaScript, Python, Dart, Rust, and MicroPython applications using a message broker as the transport layer.

This specification serves as the single source of truth for:

Inputs: What data structures are accepted by smartpack()
Outputs: What data structures are returned by smartunpack()
Data Shapes: Exact field names, types, and constraints
Error Codes: Standardized error responses for failure scenarios

1.1 Requirements Traceability

Specification Section	Requirement ID(s)	Solution Design Ref(s)	Description
Section 2 (Message Envelope)	FR-012, FR-013, NFR-101, NFR-102	SD-008	Message envelope structure and validation
Section 3 (Payload Schema)	FR-001, FR-002, FR-003, FR-004, NFR-101, NFR-102	SD-001, SD-005	Payload structure and field definitions
Section 4 (Payload Format)	FR-006, FR-007	SD-004	Tuple format for smartpack()
Section 5 (Enumerations)	FR-003, FR-004, FR-006, NFR-101	SD-001, SD-002, SD-005	Enumerations for transport and encoding
Section 6 (Transport Protocols)	FR-003, FR-004, NFR-104, NFR-105	SD-001, SD-002	Direct and link transport protocols
Section 7 (Size Thresholds)	FR-004, FR-005, NFR-104, NFR-105	SD-002	Size thresholds for transport selection
Section 8 (Topic Convention)	FR-013, FR-014	SD-006	Topic/subject naming patterns
Section 9 (Error Handling)	FR-010, FR-011, NFR-201, NFR-202, NFR-203	SD-003, SD-007	Error codes and exception handling
Section 10 (Serialization Rules)	FR-001, FR-002, FR-003, FR-012, NFR-101, NFR-102	SD-005	Serialization and encoding rules
Section 11 (API Contract)	FR-001 through FR-014, NFR-101 through NFR-107	SD-001 through SD-008	Function signatures for all platforms
Section 12 (File Server Interface)	FR-008, FR-009, FR-010	SD-003, SD-007	Upload and download handler contracts
Section 13 (Platform-Specific Constraints)	FR-005, FR-006, NFR-106, NFR-107	SD-004, SD-006	Platform-specific feature support
Section 14 (Implementation Files)	FR-001 through FR-007	-	Implementation file mapping
Section 15 (Message Flow)	FR-001 through FR-014, NFR-101 through NFR-107	SD-001 through SD-008	Mermaid diagrams for send/receive flows
Section 16 (Validation Rules)	FR-001 through FR-014, NFR-101 through NFR-405	SD-001 through SD-008	Envelope and payload validation rules
Section 17 (Test Contracts)	FR-001 through FR-014, NFR-101 through NFR-107	SD-001 through SD-008	Unit and integration test scenarios
Section 18 (Dependencies)	FR-001 through FR-014, NFR-101 through NFR-107	SD-001 through SD-008	Platform-specific dependencies
Section 19 (Change Log)	-	-	Version history and changes

2. Message Envelope Schema

Specification Versioning

Component	Version	Notes
Specification	1.0.0	Initial release
Protocol	v1	Message envelope protocol version

Message Envelope Schema

Envelope Structure (JSON)

{
  "correlation_id": "string (UUID)",
  "msg_id": "string (UUID)",
  "timestamp": "string (ISO 8601 UTC)",
  "send_to": "string",
  "msg_purpose": "string",
  "sender_name": "string",
  "sender_id": "string (UUID)",
  "receiver_name": "string",
  "receiver_id": "string (UUID)",
  "reply_to": "string",
  "reply_to_msg_id": "string",
  "broker_url": "string",
  "metadata": "object",
  "payloads": [
    {
      "id": "string (UUID)",
      "dataname": "string",
      "payload_type": "string",
      "transport": "string",
      "encoding": "string",
      "size": "integer",
      "data": "string or URL",
      "metadata": "object"
    }
  ]
}

Field Definitions

Field	Type	Required	Validation	Description	Requirement ID	Solution Design Ref
`correlation_id`	`string`	Yes	UUID v4 format	Track message flow across distributed systems	FR-011, NFR-401	SD-008
`msg_id`	`string`	Yes	UUID v4 format	Unique identifier for this specific message	FR-012, NFR-401	SD-001
`timestamp`	`string`	Yes	ISO 8601 UTC	Message publication timestamp	FR-012, NFR-401	SD-001
`send_to`	`string`	Yes	Non-empty string	Topic/subject to publish the message to	FR-013	SD-006
`msg_purpose`	`string`	Yes	Enum	Purpose of the message	FR-013	SD-006
`sender_name`	`string`	Yes	Non-empty string	Name of the sender application	FR-013	SD-006
`sender_id`	`string`	Yes	UUID v4 format	Unique identifier for the sender	FR-013	SD-006
`receiver_name`	`string`	Yes	Any string	Name of the receiver (empty = broadcast)	FR-013	SD-006
`receiver_id`	`string`	Yes	Any string	UUID of the receiver (empty = broadcast)	FR-013	SD-006
`reply_to`	`string`	Yes	Any string	Topic where receiver should reply	FR-013	SD-006
`reply_to_msg_id`	`string`	Yes	Any string	Message ID this message is replying to	FR-013	SD-006
`broker_url`	`string`	Yes	Valid URL	Broker URL for the transport layer	FR-013	SD-006
`metadata`	`object`	No	Any JSON object	Message-level metadata	NFR-401	SD-001, SD-008
`payloads`	`array`	Yes	Non-empty array	List of payload objects	FR-012, FR-013	SD-001, SD-005

Payload Schema

Payload Structure (JSON)

{
  "id": "string (UUID)",
  "dataname": "string",
  "payload_type": "string",
  "transport": "string",
  "encoding": "string",
  "size": "integer",
  "data": "string or URL",
  "metadata": "object"
}

Payload Field Definitions

Field	Type	Required	Validation	Description	Requirement ID	Solution Design Ref
`id`	`string`	Yes	UUID v4 format	Unique identifier for this payload	FR-012	SD-001
`dataname`	`string`	Yes	Non-empty string	Name of the payload (e.g., `login_image`, `user_data`)	FR-001, FR-002, FR-003	SD-004
`payload_type`	`string`	Yes	Enum	Type of payload (see `payload_type` enum)	FR-001, FR-002, FR-003, FR-006	SD-004, SD-005
`transport`	`string`	Yes	Enum	Transport method: `direct` or `link`	FR-003, FR-004, NFR-104, NFR-105	SD-001, SD-002
`encoding`	`string`	Yes	Enum	Encoding method (see `encoding` enum)	FR-001, FR-002, FR-003, FR-012	SD-005
`size`	`integer`	Yes	Positive integer	Size of the payload in bytes	FR-003, FR-004, NFR-104, NFR-105	SD-002
`data`	`string` or `URL`	Yes	Base64 string or URL	Payload data (base64 for direct, URL for link)	FR-003, FR-004, FR-008, FR-009, FR-010	SD-001, SD-003
`metadata`	`object`	No	Any JSON object	Payload-level metadata	NFR-401	SD-008

Payload Format

Tuple Format for `smartpack()`

The smartpack() function accepts data as an array of tuples with the format:

("data_name", data, "data_type")

Position	Type	Description	Example
1	`string`	Data name - identifier for the payload	`"msg"`, `"login_image"`, `"user_data"`
2	`any`	Actual data - content to be serialized	`"Hello"`, `{"key": "value"}`, `DataFrame(...)`
3	`string`	Data type - must be in `payload_type` enum	`"text"`, `"dictionary"`, `"arrowtable"`

Single Payload Example

# Julia
smartpack("/chat/user/v1/message", [("msg", "Hello World", "text")])

# Python
await smartpack("/chat/user/v1/message", [("msg", "Hello World", "text")])

// JavaScript
await smartpack("/chat/user/v1/message", [["msg", "Hello World", "text"]]);

Multiple Payloads Example

# Julia - Mixed text and binary data
data = [
    ("msg", "Hello", "text"),
    ("img", binary_data, "image")
]
smartpack("/agent/v1/process", data)

# Python - Mixed types
data = [
    ("msg", "Hello", "text"),
    ("img", binary_data, "image")
]
await smartpack("/agent/v1/process", data)

Data Type Mapping

Platform	Input Type	Data Type String
All	`String`	`"text"`
All	`Dict`/`Object`	`"dictionary"`
Desktop	`DataFrame`	`"arrowtable"` or `"jsontable"`
Browser	`Array` of objects	`"jsontable"` (only table type)
All	`Array` of objects	`"jsontable"`
All	`Uint8Array`/`Buffer`/`bytes`	`"binary"`
Desktop	`Arrow.Table`	`"arrowtable"`
All	Image/Audio/Video binary	`"image"`, `"audio"`, `"video"`

Enumerations

`msg_purpose` Enum

Value	Description
`ACK`	Acknowledgment of successful message processing
`NACK`	Negative acknowledgment of message processing failure
`updateStatus`	Status update message
`shutdown`	Graceful shutdown request
`chat`	Chat/message payload
`command`	Command payload
`event`	Event payload

`payload_type` Enum

Value	Description	Supported Platforms	Encoding Options
`text`	Plain text string	All	`base64`
`dictionary`	JSON object/dictionary	All	`base64`, `json`
`arrowtable`	Apache Arrow IPC table	Desktop (Julia/Python/Node.js/Dart)	`base64`, `arrow-ipc`
`jsontable`	JSON array of objects	All (including Browser/Dart Web)	`base64`, `json`
`image`	Binary image data	All	`base64`
`audio`	Binary audio data	All	`base64`
`video`	Binary video data	All	`base64`
`binary`	Generic binary data	All	`base64`

`transport` Enum

Value	Description	Data Format	Use Case
`direct`	Payload sent directly via the transport layer	Base64-encoded string	Payloads < size_threshold
`link`	Payload uploaded to file server	HTTP URL	Payloads ≥ size_threshold

`encoding` Enum

Value	Description	Payload Types
`none`	No additional encoding	Link transport URLs
`base64`	Base64 encoding	Text, binary, image, audio, video
`json`	JSON encoding	Dictionary, jsontable
`arrow-ipc`	Apache Arrow IPC format	Arrowtable

Transport Protocols

Direct Transport Protocol

When transport = "direct", the data field contains a Base64-encoded string of the serialized payload.

Flow:

Serialize payload according to payload_type
Encode serialized bytes as Base64
Include Base64 string in data field

Example:

{
  "transport": "direct",
  "encoding": "base64",
  "size": 11,
  "data": "SGVsbG8gV29ybGQ="
}

Link Transport Protocol

When transport = "link", the data field contains a URL pointing to the uploaded payload.

Flow:

Serialize payload according to payload_type
Upload to HTTP file server (e.g., Plik)
Include returned URL in data field

Example:

{
  "transport": "link",
  "encoding": "none",
  "size": 1000000,
  "data": "http://localhost:8080/file/3F62E/4AgGT/data.zip"
}

Size Thresholds

Desktop Platforms (Julia/JS/Python)

Platform	Size Threshold	Notes	Requirement ID	Solution Design Ref
Desktop	500,000 bytes (0.5MB)	Default threshold	FR-004, FR-005, NFR-104, NFR-105	SD-002

MicroPython Platform

Platform	Size Threshold	Maximum Payload	Notes	Requirement ID	Solution Design Ref
MicroPython	100,000 bytes (100KB)	50,000 bytes	Hard limit due to memory constraints	FR-005, NFR-106	SD-002

Topic Convention

Subject Naming Pattern

<scope>/<service>/<version>/<operation>

Examples:

/agent/wine/api/v1/prompt - AI agent prompt endpoint | FR-013 | SD-006
/chat/user/v1/message - User chat message | FR-013 | SD-006
/system/worker/v1/status - Worker status update | FR-013 | SD-006

Subject Wildcards

Wildcard	Description	Example
`*`	Single-level wildcard	`/chat/user/v1/*` matches `/chat/user/v1/message`
`>`	Multi-level wildcard	`/chat/user/v1/>` matches all `/chat/user/v1/*` subjects

Error Handling

Error Response Format

{
  "error": {
    "code": "string",
    "message": "string",
    "details": "object"
  }
}

Error Codes

Code	HTTP Status	Description	Recovery	Requirement ID
`INVALID_ENVELOPE`	400	Message envelope validation failed	Fix envelope structure	FR-012, FR-013, FR-014
`INVALID_PAYLOAD_TYPE`	400	Unsupported payload type	Use supported payload_type	FR-001, FR-002, FR-003, FR-006
`INVALID_TRANSPORT`	400	Unsupported transport type	Use `direct` or `link`	FR-003, FR-004, FR-006
`UPLOAD_FAILED`	500	File server upload failed	Retry or use direct transport	FR-008, FR-009
`DOWNLOAD_FAILED`	503	File server download failed	Retry with exponential backoff	FR-010, FR-011, NFR-201, NFR-202
`TRANSPORT_CONNECTION_FAILED`	503	Transport connection failed	Check broker/server availability	FR-013, FR-014, NFR-201, NFR-203
`DESERIALIZATION_ERROR`	500	Payload deserialization failed	Check payload_type matches data	FR-001, FR-002, FR-003, FR-012
`SIZE_EXCEEDED`	413	Payload exceeds maximum size	Split payload or use link transport	FR-003, FR-004, FR-005, NFR-104, NFR-105

Exception Handling

Scenario	Handler	Retry Policy	Requirement ID
File server unavailable	Retry up to 5 times	Exponential backoff (100ms → 5000ms)	FR-010, NFR-202
Transport publish failure	Handled by caller	Caller's retry logic	FR-013, FR-014, NFR-201, NFR-203
Deserialization error	Log correlation ID and throw	No retry (data corruption)	FR-001, FR-002, FR-003, FR-012, NFR-401
Memory overflow (MicroPython)	Reject payloads >50KB	No retry (client-side check)	FR-005, NFR-106

Serialization Rules

Text Serialization

Platform	Input Type	Serialization	Encoding
All	`String`	UTF-8 bytes	Base64

Dictionary Serialization

Platform	Input Type	Serialization	Encoding
All	`Object`/`Dict`	JSON string	Base64 or direct JSON

Arrow Table Serialization

Platform	Input Type	Serialization	Encoding
Desktop	`DataFrame`	Arrow IPC stream	Base64 or arrow-ipc
Desktop	`Arrow.Table`	Arrow IPC stream	Base64 or arrow-ipc
MicroPython	❌	Not supported	N/A

JSON Table Serialization

Platform	Input Type	Serialization	Encoding
All	`Vector{Dict}`/`Array<Object>`	JSON array	Base64 or direct JSON
Desktop	`pandas.DataFrame`	JSON array	Base64 or direct JSON

Binary Serialization

Platform	Input Type	Serialization	Encoding
All	`Uint8Array`/`Buffer`/`bytes`	Raw bytes	Base64

API Contract

Platform	Function	Requirements	Solution Design Ref
All	`smartpack()`	FR-001 through FR-014, NFR-101 through NFR-107	SD-001 through SD-008
All	`smartunpack()`	FR-001 through FR-014, NFR-101 through NFR-107	SD-001 through SD-008

`smartpack` Function Signature

Julia

function smartpack(
    subject::String,
    data::AbstractArray{Tuple{String, T1, String}, 1};
    broker_url::String = DEFAULT_BROKER_URL,
    fileserver_url::String = DEFAULT_FILESERVER_URL,
    fileserver_upload_handler::Function = plik_oneshot_upload,
    size_threshold::Int = DEFAULT_SIZE_THRESHOLD,
    correlation_id::String = string(uuid4()),
    msg_purpose::String = "chat",
    sender_name::String = "msghandler",
    receiver_name::String = "",
    receiver_id::String = "",
    reply_to::String = "",
    reply_to_msg_id::String = "",
    msg_id::String = string(uuid4()),
    sender_id::String = string(uuid4())
)::Tuple{msg_envelope_v1, String} where {T1<:Any}

Note: Publishing via the transport layer is the caller's responsibility. Returns (env::msg_envelope_v1, env_json_str::String).

Python

async def smartpack(
    subject: str,
    data: List[Tuple[str, Any, str]],
    broker_url: str = DEFAULT_BROKER_URL,
    fileserver_url: str = "http://localhost:8080",
    fileserver_upload_handler: Callable = plik_oneshot_upload,
    size_threshold: int = 500_000,
    correlation_id: str = None,
    msg_purpose: str = "chat",
    sender_name: str = "msghandler",
    receiver_name: str = "",
    receiver_id: str = "",
    reply_to: str = "",
    reply_to_msg_id: str = "",
    msg_id: str = None,
    sender_id: str = None
) -> Tuple[Dict, str]:

Note: Publishing via the transport layer is the caller's responsibility.

JavaScript (Node.js)

async function smartpack(
    subject: string,
    data: Array<[string, any, string]>,
    options?: {
        broker_url?: string;
        fileserver_url?: string;
        fileserver_upload_handler?: Function;
        size_threshold?: number;
        correlation_id?: string;
        msg_purpose?: string;
        sender_name?: string;
        receiver_name?: string;
        receiver_id?: string;
        reply_to?: string;
        reply_to_msg_id?: string;
        msg_id?: string;
        sender_id?: string;
    }
): Promise<[Object, string]>;

Note: Publishing via the transport layer is the caller's responsibility.

JavaScript (Browser)

async function smartpack(
    subject: string,
    data: Array<[string, any, string]>,
    options?: {
        broker_url?: string;
        fileserver_url?: string;
        fileserver_upload_handler?: Function;
        size_threshold?: number;
        correlation_id?: string;
        msg_purpose?: string;
        sender_name?: string;
        receiver_name?: string;
        receiver_id?: string;
        reply_to?: string;
        reply_to_msg_id?: string;
        msg_id?: string;
        sender_id?: string;
    }
): Promise<[Object, string]>;

Note: Publishing via the transport layer is the caller's responsibility.

MicroPython

def smartpack(
    subject: str,
    data: List[Tuple[str, Any, str]],
    size_threshold: int = 100_000,  # Lower threshold for memory constraints
    **kwargs
) -> Tuple[Dict, str]:

Note: Publishing via the transport layer is the caller's responsibility.

Dart (Desktop/Flutter)

Future<[Map<String, dynamic>, String]> smartpack(
  String subject,
  List<List<dynamic>> data, {
 String brokerUrl = DEFAULT_BROKER_URL,
  String fileserverUrl = DEFAULT_FILESERVER_URL,
  Function? fileserverUploadHandler,
  int sizeThreshold = DEFAULT_SIZE_THRESHOLD,
  String? correlationId,
  String msgPurpose = 'chat',
  String senderName = 'msghandler',
  String receiverName = '',
  String receiverId = '',
  String replyTo = '',
  String replyToMsgId = '',
  String? msgId,
  String? senderId,
}) async {
  // Returns [envelope, jsonString]
  // Publishing via transport layer is caller's responsibility
}

#### Dart Web

```dart
Future<[Map<String, dynamic>, String]> smartpack(
  String subject,
  List<List<dynamic>> data, {
  String brokerUrl = DEFAULT_BROKER_URL,
  String fileserverUrl = 'http://localhost:8080',
  Function? fileserverUploadHandler,
  int sizeThreshold = 500000,
  String? correlationId,
  String msgPurpose = 'chat',
  String senderName = 'msghandler',
  String receiverName = '',
  String receiverId = '',
  String replyTo = '',
  String replyToMsgId = '',
  String? msgId,
  String? senderId,
}) async {
  // Returns [envelope, jsonString]
  // Publishing via transport layer is caller's responsibility
}

Rust

pub async fn smartpack(
    subject: &str,
    data: &[(String, Payload, String)],
    options: &smartpackOptions,
) -> Result<(MsgEnvelopeV1, String), msghandlerError>

// smartpackOptions struct
pub struct smartpackOptions {
    pub broker_url: String,
    pub fileserver_url: String,
    pub fileserver_upload_handler: Option<UploadHandler>,
    pub size_threshold: usize,
    pub correlation_id: String,
    pub msg_purpose: String,
    pub sender_name: String,
    pub receiver_name: String,
    pub receiver_id: String,
    pub reply_to: String,
    pub reply_to_msg_id: String,
    pub msg_id: String,
    pub sender_id: String,
}

// Payload enum for type-safe data handling
#[derive(Serialize, Deserialize, Clone)]
pub enum Payload {
    Text(String),
    Dictionary(serde_json::Value),
    ArrowTable(Vec<u8>),
    JsonTable(serde_json::Value),
    Image(Vec<u8>),
    Audio(Vec<u8>),
    Video(Vec<u8>),
    Binary(Vec<u8>),
}

// MsgEnvelopeV1 struct (serde-serializable)
#[derive(Serialize, Deserialize, Clone)]
pub struct MsgEnvelopeV1 {
    pub correlation_id: String,
    pub msg_id: String,
    pub timestamp: String,
    pub send_to: String,
    pub msg_purpose: String,
    pub sender_name: String,
    pub sender_id: String,
    pub receiver_name: String,
    pub receiver_id: String,
    pub reply_to: String,
    pub reply_to_msg_id: String,
    pub broker_url: String,
    pub metadata: serde_json::Value,
    pub payloads: Vec<MsgPayloadV1>,
}

Note: Publishing via the transport layer is the caller's responsibility. Returns Result<(MsgEnvelopeV1, String), msghandlerError>. Uses serde for JSON serialization.

`smartunpack` Function Signature

Note: Input is the JSON string payload from the transport subscription. Returns (envelope::msg_envelope_v1, payloads::Array{Tuple{String, Any, String}, 1}).

Traceability: FR-001 through FR-014, NFR-101 through NFR-107 | SD-001 through SD-008

Julia

function smartunpack(
    msg_json_str::String;  # Pass payload from transport subscription
    fileserver_download_handler::Function = _fetch_with_backoff,
    max_retries::Int = 5,
    base_delay::Int = 100,
    max_delay::Int = 5000
)::JSON.Object{String, Any}

Note: Input is the JSON string payload from the transport subscription, not the transport message object directly.

Python

async def smartunpack(
    msg_json_str: str,  # JSON string from transport message payload
    fileserver_download_handler: Callable = fetch_with_backoff,
    max_retries: int = 5,
    base_delay: int = 100,
    max_delay: int = 5000
) -> Dict[str, Any]:

Note: Input is the JSON string payload from the transport message.

JavaScript (Node.js)

async function smartunpack(
    msg_json_str: string,  // JSON string from transport message payload
    options?: {
        fileserver_download_handler?: Function;
        max_retries?: number;
        base_delay?: number;
        max_delay?: number;
    }
): Promise<Object>;

JavaScript (Browser)

async function smartunpack(
    msg_json_str: string,  // JSON string from transport message payload
    options?: {
        fileserver_download_handler?: Function;
        max_retries?: number;
        base_delay?: number;
        max_delay?: number;
    }
): Promise<Object>;

Note: Input is the JSON string payload from the transport message.

MicroPython

def smartunpack(msg_json_str: str, **kwargs) -> Dict[str, Any]:

Note: Input is the JSON string payload from the transport message.

Dart (Desktop/Flutter)

Future<Map<String, dynamic>> smartunpack(
  Map<String, dynamic> msg_json_str,  // JSON object from transport message payload
  {
  Function? fileserverDownloadHandler,
  int maxRetries = 5,
  int baseDelay = 100,
  int maxDelay = 5000,
}) async {
  // Returns envelope with processed payloads
}

Dart Web

Future<Map<String, dynamic>> smartunpack(
  Map<String, dynamic> msg_json_str,  // JSON object from transport message payload
  {
  Function? fileserverDownloadHandler,
  int maxRetries = 5,
  int baseDelay = 100,
  int maxDelay = 5000,
}) async {
  // Returns envelope with processed payloads
}

Rust

pub async fn smartunpack(
    msg_json_str: &str,  // JSON string from transport message payload
    options: &smartunpackOptions,
) -> Result<MsgEnvelopeV1, msghandlerError>

// smartunpackOptions struct
pub struct smartunpackOptions {
    pub fileserver_download_handler: Option<DownloadHandler>,
    pub max_retries: u32,
    pub base_delay: u64,
    pub max_delay: u64,
}

Note: Input is the JSON string payload from the transport message. Returns Result<MsgEnvelopeV1, msghandlerError>.

File Server Interface

Traceability: FR-008, FR-009, FR-010, NFR-202 | SD-003, SD-007

Upload Handler Contract

Function Signature:

function fileserver_upload_handler(
    file_server_url::String,
    dataname::String,
    data::Vector{UInt8}
)::Dict{String, Any}

# Overload: Upload file from disk
function fileserver_upload_handler(
    file_server_url::String,
    filepath::String
)::Dict{String, Any}

Return Format:

{
  "status": 200,
  "uploadid": "string",
  "fileid": "string",
  "url": "string"
}

Required Keys:

Key	Type	Description	Requirement ID	Solution Design Ref
`status`	`integer`	HTTP response status code	FR-008, FR-009	SD-003
`uploadid`	`string`	Upload session identifier	FR-008	SD-003
`fileid`	`string`	File identifier within session	FR-008	SD-003
`url`	`string`	Full download URL	FR-008, FR-009, FR-010	SD-003, SD-007

Download Handler Contract

Function Signature:

Traceability: FR-010, NFR-202 | SD-003, SD-007

Function Signature:

function fileserver_download_handler(
    url::String,
    max_retries::Int,
    base_delay::Int,
    max_delay::Int,
    correlation_id::String
)::Vector{UInt8}

Retry Policy:

Initial delay: base_delay milliseconds
Maximum delay: max_delay milliseconds
Multiplier: 2x per retry
Maximum retries: max_retries

Platform-Specific Constraints

Traceability: FR-005, FR-006, NFR-106, NFR-107 | SD-002, SD-004

Desktop (Julia/Python/Node.js/Dart)

Feature	Status	Notes	Requirement ID	Solution Design Ref
Arrow IPC	✅ Supported	Requires Arrow.jl/pyarrow/dart-arrow	FR-002, FR-012	SD-004, SD-005
JSON table	✅ Supported	Human-readable format	FR-002, FR-006	SD-004, SD-005
File server upload	✅ Supported	HTTP/HTTPS	FR-008, FR-009	SD-003
File server download	✅ Supported	HTTP/HTTPS	FR-010, NFR-202	SD-003, SD-007
Size threshold	500KB	Configurable	FR-004, FR-005, NFR-104, NFR-105	SD-002

Browser (JavaScript)

Feature	Status	Notes	Requirement ID	Solution Design Ref
Arrow IPC	❌ Not supported	Apache Arrow not browser-compatible	FR-002	SD-004, SD-005
JSON table	✅ Supported	Only table type available in browser	FR-002, FR-006	SD-004, SD-005
File server upload	✅ Supported	HTTP/HTTPS	FR-008, FR-009	SD-003
File server download	✅ Supported	HTTP/HTTPS	FR-010, NFR-202	SD-003, SD-007
Size threshold	500KB	Configurable	FR-004, FR-005, NFR-104, NFR-105	SD-002

Dart Desktop (Dart SDK)

Feature	Status	Notes	Requirement ID	Solution Design Ref
Arrow IPC	✅ Supported	Requires dart-arrow package	FR-002, FR-012	SD-004, SD-005
JSON table	✅ Supported	Human-readable format	FR-002, FR-006	SD-004, SD-005
File server upload	✅ Supported	HTTP/HTTPS	FR-008, FR-009	SD-003
File server download	✅ Supported	HTTP/HTTPS	FR-010, NFR-202	SD-003, SD-007
Size threshold	500KB	Configurable	FR-004, FR-005, NFR-104, NFR-105	SD-002

Dart Flutter (Dart SDK)

Feature	Status	Notes	Requirement ID	Solution Design Ref
Arrow IPC	✅ Supported	Requires dart-arrow package	FR-002, FR-012	SD-004, SD-005
JSON table	✅ Supported	Human-readable format	FR-002, FR-006	SD-004, SD-005
File server upload	✅ Supported	HTTP/HTTPS	FR-008, FR-009	SD-003
File server download	✅ Supported	HTTP/HTTPS	FR-010, NFR-202	SD-003, SD-007
Size threshold	500KB	Configurable	FR-004, FR-005, NFR-104, NFR-105	SD-002

Dart Web (Dart SDK)

Feature	Status	Notes	Requirement ID	Solution Design Ref
Arrow IPC	❌ Not supported	Apache Arrow not browser-compatible	FR-002	SD-004, SD-005
JSON table	✅ Supported	Only table type available in browser	FR-002, FR-006	SD-004, SD-005
File server upload	✅ Supported	HTTP/HTTPS	FR-008, FR-009	SD-003
File server download	✅ Supported	HTTP/HTTPS	FR-010, NFR-202	SD-003, SD-007
Size threshold	500KB	Configurable	FR-004, FR-005, NFR-104, NFR-105	SD-002

Rust

Feature	Status	Notes	Requirement ID	Solution Design Ref
Arrow IPC	✅ Supported	Requires `arrow2` crate	FR-002, FR-012	SD-004, SD-005
JSON table	✅ Supported	Uses `serde_json`	FR-002, FR-006	SD-004, SD-005
File server upload	✅ Supported	HTTP/HTTPS via `reqwest`	FR-008, FR-009	SD-003
File server download	✅ Supported	HTTP/HTTPS via `reqwest` with retry	FR-010, NFR-202	SD-003, SD-007
Size threshold	500KB	Configurable	FR-004, FR-005, NFR-104, NFR-105	SD-002
Async runtime	✅ Supported	Uses `tokio` for async I/O	FR-013, FR-014	SD-006
Type safety	✅ Supported	Compile-time type checking via Rust enums	FR-006, FR-007	SD-004

MicroPython

Feature	Status	Notes	Requirement ID	Solution Design Ref
Arrow IPC	❌ Not supported	Memory constraints	FR-005	SD-002
JSON table	⚠️ Limited	Only direct transport	FR-005, FR-006	SD-002, SD-004
File server upload	❌ Not implemented	Placeholder only	FR-005	SD-002
File server download	❌ Not implemented	Placeholder only	FR-005	SD-002
Size threshold	100KB	Hard limit enforced	FR-005, NFR-106	SD-002
Max payload	50KB	Hard limit enforced	FR-005	SD-002

Implementation Files

Traceability: FR-001 through FR-007 | SD-001 through SD-008

File	Platform	Features	Notes	Requirement ID	Solution Design Ref
`src/msghandler.jl`	Julia	Full feature set, Arrow IPC, multiple dispatch	Ground truth implementation	FR-001 through FR-014, NFR-101 through NFR-107	SD-001 through SD-008
`src/msghandler_ssr.js`	Node.js	Arrow IPC, async/await	Server-side JavaScript	FR-001 through FR-014, NFR-101 through NFR-107	SD-001 through SD-008
`src/msghandler_csr.js`	Browser	JSON table only	Client-side rendering	FR-001 through FR-014, NFR-101 through NFR-107	SD-001 through SD-008
`src/msghandler.py`	Python	Arrow IPC, async/await	Desktop Python	FR-001 through FR-014, NFR-101 through NFR-107	SD-001 through SD-008
`src/msghandler.dart`	Dart	Full feature set, Arrow IPC, async/await	Desktop/Flutter/Web	FR-001 through FR-014, NFR-101 through NFR-107	SD-001 through SD-008
`src/msghandler.rs`	Rust	Full feature set, Arrow IPC, async/await, type-safe	Uses tokio + serde + arrow2	FR-001 through FR-014, NFR-101 through NFR-107	SD-001 through SD-008
`src/msghandler_mpy.py`	MicroPython	Limited to direct transport	Memory-constrained	FR-005, FR-006	SD-002, SD-004

Browser Implementation Notes

Traceability: FR-002, FR-006 | SD-004, SD-005

The browser implementation (src/msghandler_csr.js) has the following constraints:

Constraint	Reason	Workaround	Requirement ID	Solution Design Ref
No Apache Arrow IPC	Browser-incompatible dependency	Use `jsontable` for tabular data	FR-002	SD-004, SD-005
WebSocket only	Browser cannot use TCP directly	Use `ws://` or `wss://` broker URLs	FR-013, FR-014	SD-006
Fetch API for HTTP	Browser fetch() API only	Compatible with Plik and other HTTP servers	FR-008, FR-009	SD-003

Payload Type Availability by Platform

Payload Type	Julia	Node.js	Browser	Python	Dart	Rust	MicroPython
`text`	✅	✅	✅	✅	✅	✅	✅
`dictionary`	✅	✅	✅	✅	✅	✅	✅
`arrowtable`	✅	✅	❌	✅	✅	✅	❌
`jsontable`	✅	✅	✅	✅	✅	✅	⚠️
`image`	✅	✅	✅	✅	✅	✅	✅
`audio`	✅	✅	✅	✅	✅	✅	✅
`video`	✅	✅	✅	✅	✅	✅	✅
`binary`	✅	✅	✅	✅	✅	✅	✅

Message Flow

Sending Flow

flowchart TD
    A[User calls smartpack subject data] --> B[Serialize payload according to payload_type]
    B --> C{Calculate serialized size}
    C -->|Size < Threshold| D[Direct Transport: Encode as Base64]
    C -->|Size >= Threshold| E[Link Transport: Upload to file server]
    D --> F[Build envelope with metadata]
    E --> F
    F --> G[Convert envelope to JSON string]
    G --> H[Publish to topic via transport]
    H --> I[Return envelope and JSON string to caller]
    
    style A fill:#f9f9f9,stroke:#333
    style I fill:#e0e7ff,stroke:#3b82f6
    style D fill:#d1fae5,stroke:#10b981
    style E fill:#fef3c7,stroke:#f59e0b

Receiving Flow

flowchart TD
  A[Transport message arrives] --> B[Parse JSON envelope]
    B --> C[For each payload: Check transport type]
    C -->|transport == direct| D[Direct Transport: Extract Base64]
    C -->|transport == link| E[Link Transport: Fetch from URL]
    D --> F[Decode Base64]
    E --> G[Fetch with exponential backoff]
    F --> H[Deserialize based on payload_type]
    G --> H
    H --> I[Build payloads array]
    I --> J[Replace payloads array with deserialized tuples]
    J --> K[Return envelope with processed payloads]
    
    style A fill:#f9f9f9,stroke:#333
    style K fill:#e0e7ff,stroke:#3b82f6
    style D fill:#d1fae5,stroke:#10b981
    style E fill:#fef3c7,stroke:#f59e0b

Validation Rules

Traceability: FR-001 through FR-014, NFR-101 through NFR-405 | SD-001 through SD-008

Envelope Validation

Rule	Condition	Error Code	Requirement ID	Solution Design Ref
Required fields present	`correlation_id`, `msg_id`, `timestamp`, `send_to`, `payloads`	`INVALID_ENVELOPE`	FR-012, FR-013	SD-001, SD-006
Valid UUID format	`correlation_id`, `msg_id`, `sender_id`, `receiver_id`	`INVALID_ENVELOPE`	FR-011, FR-012, NFR-401	SD-001, SD-008
Valid timestamp format	ISO 8601 UTC	`INVALID_ENVELOPE`	FR-012, NFR-401	SD-001, SD-008
Non-empty payloads array	`length(payloads) > 0`	`INVALID_ENVELOPE`	FR-012, FR-013	SD-001, SD-005

Payload Validation

Rule	Condition	Error Code	Requirement ID	Solution Design Ref
Valid payload_type	Must be in `payload_type` enum	`INVALID_PAYLOAD_TYPE`	FR-001, FR-002, FR-003, FR-006	SD-004, SD-005
Valid transport	Must be `direct` or `link`	`INVALID_TRANSPORT`	FR-003, FR-004, FR-006	SD-001, SD-002
Valid encoding	Must match payload_type and transport	`INVALID_TRANSPORT`	FR-001, FR-002, FR-003, FR-012	SD-004, SD-005
Positive size	`size > 0`	`INVALID_PAYLOAD`	FR-003, FR-004, NFR-104, NFR-105	SD-001, SD-002
Valid Base64 for direct	`data` matches Base64 pattern	`DESERIALIZATION_ERROR`	FR-001, FR-002, FR-003, FR-012	SD-004, SD-005
Valid URL for link	`data` matches HTTP(S) URL pattern	`DOWNLOAD_FAILED`	FR-008, FR-009, FR-010	SD-003, SD-007

Test Contracts

Traceability: FR-001 through FR-014, NFR-101 through NFR-107 | SD-001 through SD-008

Unit Test Validation

Test	Input	Expected Output	Notes	Requirement ID	Solution Design Ref
Text round-trip	`("msg", "Hello", "text")`	`("msg", "Hello", "text")`	String serialization	FR-001, FR-012, NFR-101, NFR-102	SD-004, SD-005
Dictionary round-trip	`("data", {"key": "value"}, "dictionary")`	`("data", {"key": "value"}, "dictionary")`	JSON object round-trip	FR-002, FR-012, NFR-101, NFR-102	SD-004, SD-005
Arrow table round-trip	`("table", arrow_table_data, "arrowtable")`	`("table", arrow_table_data, "arrowtable")`	Arrow IPC round-trip	FR-002, FR-012, NFR-101, NFR-102	SD-004, SD-005
JSON table round-trip	`("table", [{"a":1},{"b":2}], "jsontable")`	`("table", [{"a":1},{"b":2}], "jsontable")`	JSON array of objects	FR-001, FR-002, FR-006, FR-012	SD-004, SD-005
Mixed payloads	`[("msg", "Hello", "text"), ("imgname", bytes, "binary")]`	`[("msg", "Hello", "text"), ("imgname", bytes, "binary")]`	Multiple payload types	FR-006, FR-007	SD-004
Large payload	`("data", rand(10_000_000), "arrowtable")`	`("data", URL, "arrowtable")` with link transport	File server upload	FR-003, FR-004, FR-008, FR-009, NFR-104, NFR-105	SD-001, SD-002, SD-003

Platform-Specific Notes:

Julia: Use Dict, Vector{Dict}, or convert DataFrame to dictionary for testing
Python: Use dict, list[dict], or convert pandas.DataFrame to dictionary for testing
JavaScript: Use plain objects {} and arrays []
MicroPython: Use plain dict and list (limited to JSON table and text types)

Integration Test Scenarios

Scenario	Platforms	Payloads	Size Mix	Transport	Expected Result	Requirement ID	Solution Design Ref
Single text (small)	All	`text`	Small	direct	Round-trip successful	FR-001, FR-012, NFR-101, NFR-102	SD-004, SD-005
Single dictionary (small)	All	`dictionary`	Small	direct	Round-trip successful	FR-002, FR-012, NFR-101, NFR-102	SD-004, SD-005
Single arrow table (small)	Julia/JS/Python	`arrowtable`	Small	direct	Arrow IPC round-trip	FR-002, FR-012, NFR-101, NFR-102	SD-004, SD-005
Single JSON table (small)	All	`jsontable`	Small	direct	Dictionary array round-trip	FR-001, FR-002, FR-006, FR-012	SD-004, SD-005
Single image (small)	All	`image`	Small	direct	Binary round-trip	FR-001, FR-006, FR-012	SD-004, SD-005
Single audio (small)	All	`audio`	Small	direct	Binary round-trip	FR-001, FR-006, FR-012	SD-004, SD-005
Single video (small)	All	`video`	Small	direct	Binary round-trip	FR-001, FR-006, FR-012	SD-004, SD-005
Single binary (small)	All	`binary`	Small	direct	Binary round-trip	FR-001, FR-006, FR-012	SD-004, SD-005
Single text (large)	All	`text`	Large	link	File server upload/download	FR-003, FR-004, FR-008, FR-009, NFR-104, NFR-105	SD-001, SD-002, SD-003
Single JSON table (large)	All	`jsontable`	Large	link	File server upload/download	FR-003, FR-004, FR-008, FR-009, NFR-104, NFR-105	SD-001, SD-002, SD-003
Single image (large)	All	`image`	Large	link	File server upload/download	FR-003, FR-004, FR-008, FR-009, NFR-104, NFR-105	SD-001, SD-002, SD-003
Ultimate Test	Julia/JS/Python	All types	Mixed	direct/link	All payloads preserved with correct transport	FR-001 through FR-014, NFR-101 through NFR-107	SD-001 through SD-008
Ultimate Test	MicroPython	`text`/`dictionary`	Mixed	direct	Limited to text/dictionary with direct transport only	FR-005, FR-006, FR-012	SD-002, SD-004
Cross-platform JSON table	All	`jsontable`	Small	direct	Dictionary array round-trip	FR-001, FR-002, FR-006, FR-012	SD-004, SD-005
MicroPython ↔ Desktop	MicroPython ↔ Desktop	`text`/`dictionary`	Small	direct	Limited payload types	FR-005, FR-006, FR-012	SD-002, SD-004
Desktop ↔ Desktop (all combos)	Julia↔JS↔Python	All types	Small/Large	direct/link	Full compatibility	FR-001, FR-002, FR-003, FR-004, FR-005, FR-006, FR-007, FR-012, FR-013, FR-014

Dependencies

Required Dependencies by Platform

Platform	Package	Version	Purpose	Requirement ID	Solution Design Ref
Julia	JSON.jl	Latest	JSON serialization	FR-012, NFR-101, NFR-102	SD-005
Julia	Arrow.jl	Latest	Arrow IPC support	FR-002, FR-012	SD-004, SD-005
Julia	HTTP.jl	Latest	HTTP file server	FR-008, FR-009	SD-003
Julia	UUIDs.jl	Latest	UUID generation	FR-011, NFR-401	SD-008
Node.js	node-fetch	Latest	HTTP file server	FR-008, FR-009	SD-003
Browser	-	-	Transport-agnostic (caller provides)	FR-013, FR-014	SD-006
Python	aiohttp	Latest	HTTP file server	FR-008, FR-009	SD-003
Python	pyarrow	Latest	Arrow IPC support	FR-002, FR-012	SD-004, SD-005
Dart	http	Latest	HTTP file server	FR-008, FR-009	SD-003
Dart	uuid	Latest	UUID generation	FR-011, NFR-401	SD-008
Dart	dart-arrow	Latest	Arrow IPC support	FR-002, FR-012	SD-004, SD-005
Rust	serde	Latest	JSON serialization	FR-012, NFR-101, NFR-102	SD-005
Rust	serde_json	Latest	JSON handling	FR-012, NFR-101, NFR-102	SD-005
Rust	tokio	Latest	Async runtime	FR-013, FR-014	SD-006
Rust	reqwest	Latest	HTTP file server	FR-008, FR-009	SD-003
Rust	uuid	Latest	UUID generation	FR-011, NFR-401	SD-008
Rust	arrow2	Latest	Arrow IPC support	FR-002, FR-012	SD-004, SD-005
MicroPython	builtin	N/A	Limited implementation	FR-005, FR-006	SD-002

Optional Dependencies

Platform	Package	Purpose
Julia	DataFrames.jl	DataFrame support
Python	pandas	DataFrame support

Change Log

Date	Version	Changes
2026-05-15	1.3.0	Made transport layer agnostic
-	-	Removed all NATS-specific dependencies (NATS.jl, nats, nats-py, nats.ws)
-	-	Updated docs to reference generic message broker/transport
-	-	broker_url is now metadata only, not used for active connections
2026-03-15	1.1.0	Browser connection management
-	-	Added NATSClient class with keepAlive support
-	-	Added NATSConnectionPool for connection reuse
-	-	Added publishMessage function with closeConnection option
-	-	Added nats.ws to browser dependencies
2026-03-13	1.0.0	Initial specification
-	-	Message envelope schema defined
-	-	Payload schema with transport modes
-	-	Enumerations for payload_type, transport, encoding
-	-	Size thresholds for desktop/MicroPython
-	-	Error codes and validation rules
-	-	API contracts for all platforms

References

ASG Framework Reference: This specification follows the ASG Framework v8 pillars. Each specification item must cite:

Requirement ID(s) from requirements.md (e.g., FR-001, NFR-201) — defines what the system must do
Solution Design reference(s) from solution-design.md (e.g., SD-2.1, SD-3.3) — defines how and why this technical approach was chosen

20.1 Documentation Artifacts

Document	Purpose	Requirements Traceability	Solution Design Traceability
`docs/requirements.md`	Business requirements and user stories	FR-001 through FR-014, NFR-101 through NFR-405	SD-001 through SD-008
`docs/solution-design.md`	Technical solution approach	FR-001 through FR-014, NFR-101 through NFR-405	SD-001 through SD-008
`docs/specification.md`	Technical contract for msghandler	FR-001 through FR-014, NFR-101 through NFR-405	SD-001 through SD-008
`docs/ui-specification.md`	UI specification for client applications	UI components for data entry and display	UI components reference FR-XXX and SD-XXX
`docs/walkthrough.md`	End-to-end system flow	Traceability from user journey to technical implementation	Full flow validation against SD-XXX
`docs/architecture.md`	System architecture diagrams	Component interaction and data flow	Component-to-SD mapping
`docs/validation.md`	CI/CD validation rules	Contract testing and spec compliance	Validation gates for SD-XXX
`docs/runbook.md`	Operational runbook	Deployment, scaling, and troubleshooting	Operation-to-SD mapping

20.2 Implementation Files

File	Platform	Features	Requirements Traceability	Solution Design Traceability
`src/msghandler.jl`	Julia	Full feature set, Arrow IPC, multiple dispatch	FR-001 through FR-014, NFR-101 through NFR-405	SD-001 through SD-008
`src/msghandler_ssr.js`	Node.js	Arrow IPC, async/await	FR-001 through FR-014, NFR-101 through NFR-405	SD-001 through SD-008
`src/msghandler_csr.js`	Browser	JSON table only	FR-001 through FR-014, NFR-101 through NFR-405	SD-001 through SD-008
`src/msghandler.py`	Python	Arrow IPC, async/await	FR-001 through FR-014, NFR-101 through NFR-405	SD-001 through SD-008
`src/msghandler.dart`	Dart	Full feature set, Arrow IPC, async/await	FR-001 through FR-014, NFR-101 through NFR-405	SD-001 through SD-008
`src/msghandler.rs`	Rust	Full feature set, Arrow IPC, async/await, type-safe	FR-001 through FR-014, NFR-101 through NFR-405	SD-001 through SD-008
`src/msghandler_mpy.py`	MicroPython	Limited to direct transport	FR-005, FR-006, FR-012	SD-002, SD-004

20.3 External Dependencies

Platform	Package	Version	Purpose	Requirements Traceability	Solution Design Traceability
Julia	JSON.jl	Latest	JSON serialization	FR-012, NFR-101, NFR-102	SD-005
Julia	Arrow.jl	Latest	Arrow IPC support	FR-002, FR-012	SD-004, SD-005
Julia	HTTP.jl	Latest	HTTP file server	FR-008, FR-009	SD-003
Julia	UUIDs.jl	Latest	UUID generation	FR-011, NFR-401	SD-008
Node.js	node-fetch	Latest	HTTP file server	FR-008, FR-009	SD-003
Browser	-	-	Transport-agnostic (caller provides)	FR-013, FR-014	SD-006
Python	aiohttp	Latest	HTTP file server	FR-008, FR-009	SD-003
Python	pyarrow	Latest	Arrow IPC support	FR-002, FR-012	SD-004, SD-005
Dart	http	Latest	HTTP file server	FR-008, FR-009	SD-003
Dart	uuid	Latest	UUID generation	FR-011, NFR-401	SD-008
Dart	dart-arrow	Latest	Arrow IPC support	FR-002, FR-012	SD-004, SD-005
Rust	serde	Latest	JSON serialization	FR-012, NFR-101, NFR-102	SD-005
Rust	serde_json	Latest	JSON handling	FR-012, NFR-101, NFR-102	SD-005
Rust	tokio	Latest	Async runtime	FR-013, FR-014	SD-006
Rust	reqwest	Latest	HTTP file server	FR-008, FR-009	SD-003
Rust	uuid	Latest	UUID generation	FR-011, NFR-401	SD-008
Rust	arrow2	Latest	Arrow IPC support	FR-002, FR-012	SD-004, SD-005
MicroPython	builtin	N/A	Limited implementation	FR-005, FR-006	SD-002

Change Log

Date	Version	Changes	Requirement ID(s)	Solution Design Ref(s)
2026-05-22	1.3.0	Updated to ASG Framework v8 pillars	All	All SD-XXX
-	-	Added Solution Design references to all specification items	All	All SD-XXX
-	-	Updated version to 1.3.0 to match requirements and solution-design	All	-
-	-	Added ASG Framework alignment section	All	-
2026-05-15	1.3.0	Made transport layer agnostic	All	SD-006
-	-	Removed all NATS-specific dependencies (NATS.jl, nats, nats-py, nats.ws)	All	SD-006
-	-	Updated docs to reference generic message broker/transport	All	SD-006
-	-	broker_url is now metadata only, not used for active connections	All	SD-006
2026-03-15	1.1.0	Browser connection management	FR-001 through FR-014	SD-006
-	-	Added NATSClient class with keepAlive support	FR-013, FR-014	SD-006
-	-	Added NATSConnectionPool for connection reuse	FR-013, FR-014	SD-006
-	-	Added publishMessage function with closeConnection option	FR-013, FR-014	SD-006
-	-	Added nats.ws to browser dependencies	FR-013, FR-014	SD-006
2026-03-13	1.0.0	Initial specification	FR-001 through FR-014, NFR-101 through NFR-405	SD-001 through SD-008
-	-	Message envelope schema defined	FR-012, FR-013	SD-001, SD-006
-	-	Payload schema with transport modes	FR-001, FR-002, FR-003, FR-004	SD-001, SD-002
-	-	Enumerations for payload_type, transport, encoding	FR-001, FR-002, FR-003, FR-006	SD-004, SD-005
-	-	Size thresholds for desktop/MicroPython	FR-004, FR-005	SD-002
-	-	Error codes and validation rules	FR-001 through FR-010	SD-001 through SD-007
-	-	API contracts for all platforms	FR-001 through FR-014	SD-001 through SD-008

Appendix

A. Complete JSON Schema

{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "title": "msghandler Envelope",
  "type": "object",
  "properties": {
    "correlation_id": {
      "type": "string",
      "pattern": "^[0-9A-F]{8}-[0-9A-F]{4}-[0-9A-F]{4}-[0-9A-F]{4}-[0-9A-F]{12}$",
      "description": "UUID v4 format for tracking message flow"
    },
    "msg_id": {
      "type": "string",
      "pattern": "^[0-9A-F]{8}-[0-9A-F]{4}-[0-9A-F]{4}-[0-9A-F]{4}-[0-9A-F]{12}$",
      "description": "Unique message identifier"
    },
    "timestamp": {
      "type": "string",
      "pattern": "^\\d{4}-\\d{2}-\\d{2}T\\d{2}:\\d{2}:\\d{2}\\.\\d{3}Z$",
      "description": "ISO 8601 UTC timestamp"
    },
    "send_to": {
      "type": "string",
      "minLength": 1,
      "description": "Topic/subject to publish to"
    },
    "msg_purpose": {
      "type": "string",
      "enum": ["ACK", "NACK", "updateStatus", "shutdown", "chat", "command", "event"],
      "description": "Purpose of the message"
    },
    "sender_name": {
      "type": "string",
      "minLength": 1,
      "description": "Sender application name"
    },
    "sender_id": {
      "type": "string",
      "pattern": "^[0-9A-F]{8}-[0-9A-F]{4}-[0-9A-F]{4}-[0-9A-F]{4}-[0-9A-F]{12}$",
      "description": "Sender UUID"
    },
    "receiver_name": {
      "type": "string",
      "description": "Receiver name (empty = broadcast)"
    },
    "receiver_id": {
      "type": "string",
      "pattern": "^[0-9A-F]{8}-[0-9A-F]{4}-[0-9A-F]{4}-[0-9A-F]{4}-[0-9A-F]{12}$|^$",
      "description": "Receiver UUID (empty = broadcast)"
    },
    "reply_to": {
      "type": "string",
      "description": "Topic for reply messages"
    },
    "reply_to_msg_id": {
      "type": "string",
      "description": "Message ID being replied to"
    },
    "broker_url": {
      "type": "string",
      "description": "Broker URL for the transport layer"
    },
    "metadata": {
      "type": "object",
      "description": "Message-level metadata"
    },
    "payloads": {
      "type": "array",
      "minItems": 1,
      "items": {
        "$ref": "#/definitions/Payload"
      }
    }
  },
  "required": ["correlation_id", "msg_id", "timestamp", "send_to", "msg_purpose", "sender_name", "sender_id", "receiver_name", "receiver_id", "reply_to", "reply_to_msg_id", "broker_url", "payloads"],
  "definitions": {
    "Payload": {
      "type": "object",
      "properties": {
        "id": {
          "type": "string",
          "pattern": "^[0-9A-F]{8}-[0-9A-F]{4}-[0-9A-F]{4}-[0-9A-F]{4}-[0-9A-F]{12}$"
        },
        "dataname": {
          "type": "string",
          "minLength": 1
        },
        "payload_type": {
          "type": "string",
          "enum": ["text", "dictionary", "arrowtable", "jsontable", "image", "audio", "video", "binary"]
        },
        "transport": {
          "type": "string",
          "enum": ["direct", "link"]
        },
        "encoding": {
          "type": "string",
          "enum": ["none", "base64", "json", "arrow-ipc"]
        },
        "size": {
          "type": "integer",
          "minimum": 1
        },
        "data": {
          "anyOf": [
            {
              "type": "string",
              "pattern": "^(https?://[^\\s]+)$"
            },
            {
              "type": "string",
              "pattern": "^[A-Za-z0-9+/]+=*$"
            }
          ]
        },
        "metadata": {
          "type": "object"
        }
      },
      "required": ["id", "dataname", "payload_type", "transport", "encoding", "size", "data"]
    }
  }
}

B. AsyncAPI Specification

asyncapi: '2.6.0'
info:
  title: msghandler API
  version: '1.0.0'
  description: Cross-platform bi-directional data bridge using a message broker
  contact:
    name: msghandler Team
    url: https://github.com/your-org/msghandler
  license:
    name: MIT
    url: https://opensource.org/licenses/MIT
channels:
  /agent/{service}/api/v{version}/{operation}:
    address: /agent/{service}/api/v{version}/{operation}
    parameters:
      service:
        schema:
          type: string
      version:
        schema:
          type: string
          enum: ['v1']
      operation:
        schema:
          type: string
    publish:
      summary: Publish message to transport
      operationId: publishMessage
      message:
        $ref: '#/components/message'
    subscribe:
      summary: Subscribe to messages from transport
      operationId: subscribeMessage
      message:
        $ref: '#/components/message'
components:
  message:
    payload:
      $ref: '#/components/schemas/Envelope'
  schemas:
    Envelope:
      type: object
      properties:
        correlation_id:
          type: string
          format: uuid
        msg_id:
          type: string
          format: uuid
        timestamp:
          type: string
          format: date-time
        send_to:
          type: string
        msg_purpose:
          type: string
          enum: [ACK, NACK, updateStatus, shutdown, chat, command, event]
        sender_name:
          type: string
        sender_id:
          type: string
          format: uuid
        receiver_name:
          type: string
        receiver_id:
          type: string
          format: uuid
        reply_to:
          type: string
        reply_to_msg_id:
          type: string
        broker_url:
          type: string
        metadata:
          type: object
        payloads:
          type: array
          items:
            $ref: '#/components/schemas/Payload'
      required:
        - correlation_id
        - msg_id
        - timestamp
        - send_to
        - msg_purpose
        - sender_name
        - sender_id
        - receiver_name
        - receiver_id
        - reply_to
        - reply_to_msg_id
        - broker_url
        - payloads
    Payload:
      type: object
      properties:
        id:
          type: string
          format: uuid
        dataname:
          type: string
        payload_type:
          type: string
          enum: [text, dictionary, arrowtable, jsontable, image, audio, video, binary]
        transport:
          type: string
          enum: [direct, link]
        encoding:
          type: string
          enum: [none, base64, json, arrow-ipc]
        size:
          type: integer
          minimum: 1
        data:
          type: string
        metadata:
          type: object
      required:
        - id
        - dataname
        - payload_type
        - transport
        - encoding
        - size
        - data

This specification is versioned and maintained in git alongside the codebase. All implementations must adhere to this specification.

58 KiB Raw Blame History

Specification: msghandler

1. Technical Contract Overview

1.1 Requirements Traceability

2. Message Envelope Schema

Specification Versioning

Message Envelope Schema

Envelope Structure (JSON)

Field Definitions

Payload Schema

Payload Structure (JSON)

Payload Field Definitions

Payload Format

Tuple Format for smartpack()

Single Payload Example

Multiple Payloads Example

Data Type Mapping

Enumerations

msg_purpose Enum

payload_type Enum

transport Enum

encoding Enum

Transport Protocols

Direct Transport Protocol

Link Transport Protocol

Size Thresholds

Desktop Platforms (Julia/JS/Python)

MicroPython Platform

Topic Convention

Subject Naming Pattern

Subject Wildcards

Error Handling

Error Response Format

Error Codes

Exception Handling

Serialization Rules

Text Serialization

Dictionary Serialization

Arrow Table Serialization

JSON Table Serialization

Binary Serialization

API Contract

smartpack Function Signature

Julia

Python

JavaScript (Node.js)

JavaScript (Browser)

MicroPython

Dart (Desktop/Flutter)

Rust

smartunpack Function Signature

Julia

Python

JavaScript (Node.js)

JavaScript (Browser)

MicroPython

Dart (Desktop/Flutter)

Dart Web

Rust

File Server Interface

Upload Handler Contract

Download Handler Contract

Platform-Specific Constraints

Desktop (Julia/Python/Node.js/Dart)

Browser (JavaScript)

Dart Desktop (Dart SDK)

Dart Flutter (Dart SDK)

Dart Web (Dart SDK)

Rust

MicroPython

Implementation Files

Browser Implementation Notes

Payload Type Availability by Platform

Message Flow

Sending Flow

Receiving Flow

Validation Rules

Envelope Validation

Payload Validation

Test Contracts

58 KiB

Raw Blame History

Tuple Format for `smartpack()`

`msg_purpose` Enum

`payload_type` Enum

`transport` Enum

`encoding` Enum

`smartpack` Function Signature

`smartunpack` Function Signature