msghandler - Cross-Platform Bi-Directional Data Bridge

A high-performance, bi-directional data bridge for Julia, JavaScript, Python, and MicroPython applications using NATS (Core & JetStream), implementing the Claim-Check pattern for large payloads.

---

Table of Contents

• Overview
• Cross-Platform Support
• Features
• Quick Start
• API Reference
• Payload Types
• Cross-Platform Examples
• Testing
• Documentation
• License

---

Overview

msghandler enables seamless communication across multiple platforms through NATS, with intelligent transport selection based on payload size:

Transport	Payload Size	Method
Direct	< 500KB	Sent directly via NATS (Base64 encoded)
Link	≥ 500KB	Uploaded to HTTP file server, URL sent via NATS

Use Cases

• Chat Applications: Text, images, audio, video in a single message
• File Transfer: Efficient transfer of large files using claim-check pattern
• IoT/Embedded: Sensor data, telemetry, and analytics pipelines (MicroPython)
• Cross-Platform Communication: Interoperability between Julia, JavaScript, Python, and MicroPython systems

---

Cross-Platform Support

Platform	Implementation	Features
Julia	src/msghandler.jl	Full feature set, Arrow IPC, multiple dispatch
JavaScript (Node.js)	src/msghandler_ssr.js	Node.js, async/await, Arrow IPC
JavaScript (Browser)	src/msghandler_csr.js	Browser, WebSocket NATS, async/await, JSON table only
Python	src/msghandler.py	Desktop Python, asyncio, type hints, Arrow IPC
MicroPython	src/msghandler_mpy.py	Memory-constrained, synchronous API

Platform Comparison

Feature	Julia	JavaScript	JavaScript (Browser)	Python	MicroPython
Multiple Dispatch	✅ Native	❌	❌	❌	❌
Async/Await	❌	✅ Native	✅ Native	✅ Native	⚠️ (uasyncio)
Type Safety	✅ Strong	⚠️ (TypeScript)	⚠️ (TypeScript)	✅ (Type hints)	❌
Arrow IPC	✅ Native	✅ Native	❌ (Browser incompatible)	✅ Native	❌
JSON Table	✅	✅	✅ (Only table type)	✅	⚠️ (Limited)
Direct Transport	✅	✅	✅	✅	✅
Link Transport	✅	✅	✅	✅	⚠️ (Limited)
Handler Functions	✅	✅	✅	✅	✅
Cross-Platform API	✅	✅	✅	✅	✅
WebSocket NATS	❌	❌	✅	❌	❌

---

Features

• ✅ Cross-platform messaging for Julia, JavaScript, Python, and MicroPython applications
• ✅ Bi-directional messaging with request-reply patterns
• ✅ Multi-payload support - send multiple payloads with different types in one message
• ✅ Automatic transport selection - direct vs link based on payload size
• ✅ Claim-Check pattern for payloads ≥ 500KB
• ✅ Apache Arrow IPC support for tabular data (Desktop: Julia/Python/Node.js)
• ✅ JSON Table support for tabular data (All platforms including Browser)
• ✅ Exponential backoff for reliable file server downloads
• ✅ Correlation ID tracking for message tracing
• ✅ Reply-to support for request-response patterns
• ✅ Handler function abstraction - pluggable file server implementations (Plik, AWS S3, custom)

---

Quick Start

Prerequisites

1. NATS Server - Install and run a NATS server:

   docker run -p 4222:4222 nats:latest
   

2. HTTP File Server (optional, for large payloads) - Install and run a file server:

   # Using Plik
   docker run -p 8080:8080 -v /tmp/fileserver:/var/lib/plik -e PLIK_ADMIN_PASSWORD=admin plik/plik
   
   # OR using simple Python HTTP server
   mkdir -p /tmp/fileserver
   python3 -m http.server 8080 --directory /tmp/fileserver
   

Send Your First Message

Julia

using msghandler, NATS

data = [("test_message", "Hello World", "text")]  # payloads to be send format is [(dataname, data, type), ...] tuples.
envelope, envelope_json_str = smartpack("test.topic", data; broker_url="nats.yiem.cc")
conn = NATS.connect(broker_url)  # Create NATS connection
NATS.publish(conn, "test.topic", envelope_json_str)  # Publish message to NATS
NATS.drain(conn)

JavaScript (Node.js)

import msghandler from './src/msghandler_ssr.js';

const data = [["message", "Hello World", "text"]];
const [env, env_json_str] = await msghandler.smartpack(
    "/chat/room1",
    data,
    { broker_url: "nats://localhost:4222" }
);
console.log("Message sent!");

JavaScript (Browser)

import msghandler from './src/msghandler_csr.js';

const data = [["message", "Hello World", "text"]];
const [env, env_json_str] = await msghandler.smartpack(
    "/chat/room1",
    data,
    { broker_url: "ws://localhost:4222" }
);
console.log("Message sent!");

Python

from msghandler import smartpack

data = [("message", "Hello World", "text")]
env, env_json_str = await smartpack(
    "/chat/room1",
    data,
    broker_url="nats://localhost:4222"
)
print("Message sent!")

MicroPython

from msghandler import smartpack

data = [("message", "Hello World", "text")]
env, env_json_str = smartpack(
    "/chat/room1",
    data,
    broker_url="nats://localhost:4222",
    size_threshold=100000  # 100KB for MicroPython
)
print("Message sent!")

Receive Your First Message

Julia

using msghandler, NATS

conn = NATS.connect("nats.yiem.cc")
NATS.subscribe(conn, "test.topic") do msg
  log_trace("Received message on $(msg.subject)")
  
  # Use NATSBridge.smartreceive to handle the data
  # API: smartreceive(msg, download_handler; max_retries, base_delay, max_delay)
  envelope = msghandler.smartunpack(
    msg;
    max_retries = 5,
    base_delay = 100,
    max_delay = 5000
  )
end

---

API Reference

Unified API Standard

All platforms use the same input/output format for payloads:

Input format for smartpack:

[(dataname1, data1, type1), (dataname2, data2, type2), ...]

Output format for smartunpack:

{
  "correlation_id": "...",
  "msg_id": "...",
  "timestamp": "...",
  "send_to": "...",
  "msg_purpose": "...",
  "sender_name": "...",
  "sender_id": "...",
  "receiver_name": "...",
  "receiver_id": "...",
  "reply_to": "...",
  "reply_to_msg_id": "...",
  "broker_url": "...",
  "metadata": {...},
  "payloads": [(dataname1, data1, type1), (dataname2, data2, type2), ...]
}

smartpack

Sends data either directly via NATS or via a fileserver URL, depending on payload size.

Julia

using msghandler

env, env_json_str = msghandler.smartpack(
    subject::String,
    data::AbstractArray{Tuple{String, Any, String}};
    broker_url::String = "nats://localhost:4222",
    fileserver_url = "http://localhost:8080",
    fileserver_upload_handler::Function = plik_oneshot_upload,
    size_threshold::Int = 500_000,
    correlation_id::String = string(uuid4()),
    msg_purpose::String = "chat",
    sender_name::String = "msghandler",
    receiver_name::String = "",
    receiver_id::String = "",
    reply_to::String = "",
    reply_to_msg_id::String = "",
    is_publish::Bool = true,
    NATS_connection::Union{NATS.Connection, Nothing} = nothing,
    msg_id::String = string(uuid4()),
    sender_id::String = string(uuid4())
)
# Returns: ::Tuple{msg_envelope_v1, String}

JavaScript (Node.js)

import msghandler from './src/msghandler_ssr.js';

const [env, env_json_str] = await msghandler.smartpack(
    subject,
    data,  // Array of [dataname, data, type] tuples
    {
        broker_url: 'nats://localhost:4222',
        fileserver_url: 'http://localhost:8080',
        fileserver_upload_handler: msghandler.plikOneshotUpload,
        size_threshold: 500_000,
        correlation_id: uuidv4(),
        msg_purpose: 'chat',
        sender_name: 'msghandler',
        receiver_name: '',
        receiver_id: '',
        reply_to: '',
        reply_to_msg_id: '',
        is_publish: true,
        nats_connection: null,
        msg_id: uuidv4(),
        sender_id: uuidv4()
    }
);
// Returns: Promise<[env, env_json_str]>

JavaScript (Browser)

import msghandler from './src/msghandler_csr.js';

const [env, env_json_str] = await msghandler.smartpack(
    subject,
    data,
    {
        broker_url: 'ws://localhost:4222',
        fileserver_url: 'http://localhost:8080',
        fileserver_upload_handler: msghandler.plikOneshotUpload,
        size_threshold: 500_000,
        correlation_id: uuidv4(),
        msg_purpose: 'chat',
        sender_name: 'msghandler',
        receiver_name: '',
        receiver_id: '',
        reply_to: '',
        reply_to_msg_id: '',
        is_publish: true,
        nats_connection: null,
        msg_id: uuidv4(),
        sender_id: uuidv4()
    }
);
// Returns: Promise<[env, env_json_str]>

Python

from msghandler import msghandler

env, env_json_str = await msghandler.smartpack(
    subject: str,
    data: List[Tuple[str, Any, str]],
    broker_url: str = "nats://localhost:4222",
    fileserver_url: str = "http://localhost:8080",
    fileserver_upload_handler: Callable = plik_oneshot_upload,
    size_threshold: int = 500_000,
    correlation_id: str = None,
    msg_purpose: str = "chat",
    sender_name: str = "msghandler",
    receiver_name: str = "",
    receiver_id: str = "",
    reply_to: str = "",
    reply_to_msg_id: str = "",
    is_publish: bool = True,
    nats_connection: Any = None,
    msg_id: str = None,
    sender_id: str = None
)
# Returns: Tuple[Dict, str]

MicroPython

from msghandler import msghandler

# Limited to direct transport (< 100KB threshold)
env, env_json_str = msghandler.smartpack(
    subject,
    data,  # List of (dataname, data, type) tuples
    broker_url="nats://localhost:4222",
    size_threshold=100000  # Lower threshold for memory constraints
)
# Returns: Tuple[Dict, str]

smartunpack

Receives and processes messages from NATS, handling both direct and link transport.

Julia

using msghandler

env = msghandler.smartunpack(
    msg::NATS.Msg;
    fileserver_download_handler::Function = _fetch_with_backoff,
    max_retries::Int = 5,
    base_delay::Int = 100,
    max_delay::Int = 5000
)
# Returns: ::JSON.Object{String, Any}

JavaScript (Node.js)

import msghandler from './src/msghandler_ssr.js';

const env = await msghandler.smartunpack(
    msg,
    {
        fileserver_download_handler: msghandler.fetchWithBackoff,
        max_retries: 5,
        base_delay: 100,
        max_delay: 5000
    }
);
// Returns: Promise<env_object>

JavaScript (Browser)

import msghandler from './src/msghandler_csr.js';

const env = await msghandler.smartunpack(
    msg,
    {
        fileserver_download_handler: msghandler.fetchWithBackoff,
        max_retries: 5,
        base_delay: 100,
        max_delay: 5000
    }
);
// Returns: Promise<env_object>

Python

from msghandler import msghandler

env = await msghandler.smartunpack(
    msg,
    fileserver_download_handler=fetch_with_backoff,
    max_retries=5,
    base_delay=100,
    max_delay=5000
)
# Returns: Dict with "payloads" key

MicroPython

from msghandler import msghandler

env = msghandler.smartunpack(
    msg,
    fileserver_download_handler=_sync_fileserver_download,
    max_retries=3,
    base_delay=100,
    max_delay=1000
)
# Returns: Dict with "payloads" key

---

Payload Types

Type	Julia	JavaScript	Python	MicroPython	Description
text	String	string	str	str	Plain text strings
dictionary	Dict, NamedTuple	Object, Array	dict, list	dict	JSON-serializable dictionaries
arrowtable	DataFrame, Arrow.Table	❌ (Browser), ✅ (Node.js)	pandas.DataFrame	❌	Tabular data (Arrow IPC)
jsontable	DataFrame, Vector{NamedTuple}	Array<Object>	list[dict]	⚠️	Tabular data (JSON) - Only table type in Browser
image	Vector{UInt8}	Uint8Array, Buffer	bytes	bytearray	Image data (PNG, JPG)
audio	Vector{UInt8}	Uint8Array, Buffer	bytes	bytearray	Audio data (WAV, MP3)
video	Vector{UInt8}	Uint8Array, Buffer	bytes	bytearray	Video data (MP4, AVI)
binary	Vector{UInt8}, IOBuffer	Uint8Array, Buffer	bytes, bytearray	bytearray	Generic binary data

---

Cross-Platform Examples

Example 1: Chat with Mixed Content

Send text, image, and large file in one message.

Julia

using msghandler

data = [
    ("message_text", "Hello!", "text"),
    ("user_avatar", image_data, "image"),
    ("large_document", large_file_data, "binary")
]

env, env_json_str = smartpack("/chat/room1", data; fileserver_url="http://localhost:8080")

JavaScript (Node.js)

import msghandler from './src/msghandler_ssr.js';

const data = [
    ["message_text", "Hello!", "text"],
    ["user_avatar", imageData, "image"],
    ["large_document", largeFileData, "binary"]
];

const [env, env_json_str] = await msghandler.smartpack(
    "/chat/room1",
    data,
    { fileserver_url: 'http://localhost:8080' }
);

JavaScript (Browser)

import msghandler from './src/msghandler_csr.js';

const data = [
    ["message_text", "Hello!", "text"],
    ["user_avatar", imageData, "image"],
    ["large_document", largeFileData, "binary"]
];

const [env, env_json_str] = await msghandler.smartpack(
    "/chat/room1",
    data,
    { broker_url: 'ws://localhost:4222', fileserver_url: 'http://localhost:8080' }
);

Python

from msghandler import msghandler

data = [
    ("message_text", "Hello!", "text"),
    ("user_avatar", image_data, "image"),
    ("large_document", large_file_data, "binary")
]

env, env_json_str = await msghandler.smartpack(
    "/chat/room1",
    data,
    fileserver_url="http://localhost:8080"
)

Example 2: Dictionary Exchange

Send configuration data between platforms.

Julia

using msghandler

config = Dict(
    "wifi_ssid" => "MyNetwork",
    "wifi_password" => "password123",
    "update_interval" => 60
)

data = [("config", config, "dictionary")]
env, env_json_str = smartpack("/device/config", data)

JavaScript (Node.js)

import msghandler from './src/msghandler_ssr.js';

const config = {
    wifi_ssid: "MyNetwork",
    wifi_password: "password123",
    update_interval: 60
};

const [env, env_json_str] = await msghandler.smartpack(
    "/device/config",
    [["config", config, "dictionary"]]
);

Python

from msghandler import msghandler

config = {
    "wifi_ssid": "MyNetwork",
    "wifi_password": "password123",
    "update_interval": 60
}

data = [("config", config, "dictionary")]
env, env_json_str = await msghandler.smartpack("/device/config", data)

Example 3: Table Data (Arrow IPC)

Send tabular data using Apache Arrow IPC format.

Julia

using msghandler
using DataFrames

df = DataFrame(
    id = [1, 2, 3],
    name = ["Alice", "Bob", "Charlie"],
    score = [95, 88, 92]
)

data = [("students", df, "arrowtable")]
env, env_json_str = smartpack("/data/analysis", data)

JavaScript (Node.js)

import msghandler from './src/msghandler_ssr.js';

const df = [
    { id: 1, name: "Alice", score: 95 },
    { id: 2, name: "Bob", score: 88 },
    { id: 3, name: "Charlie", score: 92 }
];

const [env, env_json_str] = await msghandler.smartpack(
    "/data/analysis",
    [["students", df, "arrowtable"]]
);

Python

from msghandler import msghandler
import pandas as pd

df = pd.DataFrame({
    "id": [1, 2, 3],
    "name": ["Alice", "Bob", "Charlie"],
    "score": [95, 88, 92]
})

data = [("students", df, "arrowtable")]
env, env_json_str = await msghandler.smartpack("/data/analysis", data)

JavaScript (Browser)

import msghandler from './src/msghandler_csr.js';

// Browser uses jsontable (JSON array of objects) instead of arrowtable
// Apache Arrow is not compatible with browsers
const df = [
    { id: 1, name: "Alice", score: 95 },
    { id: 2, name: "Bob", score: 88 },
    { id: 3, name: "Charlie", score: 92 }
];

const [env, env_json_str] = await msghandler.smartpack(
    "/data/analysis",
    [["students", df, "jsontable"]],  // Use jsontable for browser
    { broker_url: 'ws://localhost:4222' }
);

Example 4: Request-Response Pattern

Bi-directional communication with reply-to support.

Julia

using msghandler

# Requester
env, env_json_str = smartpack(
    "/device/command",
    [("command", Dict("action" => "read_sensor"), "dictionary")];
    broker_url="nats://localhost:4222",
    reply_to="/device/response"
)

# Receiver (in separate application)
msg = NATS.subscription.next()
env = smartunpack(msg)
# Process request and send response
response_env, response_json = smartpack(
    "/device/response",
    [("result", Dict("value" => 42), "dictionary")],
    reply_to="/device/command",
    reply_to_msg_id=env["msg_id"]
)

JavaScript (Node.js)

import msghandler from './src/msghandler_ssr.js';

// Requester
const [env, env_json_str] = await msghandler.smartpack(
    "/device/command",
    [["command", { action: "read_sensor" }, "dictionary"]],
    { broker_url: 'nats://localhost:4222', reply_to: '/device/response' }
);

// Receiver (in separate application)
// const msg = await natsConsumer.next();
// const env = await msghandler.smartunpack(msg);
// Process request and send response
// const response_env, response_json = await msghandler.smartpack(
//     "/device/response",
//     [["result", { value: 42 }, "dictionary"]],
//     { reply_to: '/device/command', reply_to_msg_id: env.msg_id }
// );

Python

from msghandler import msghandler

# Requester
env, env_json_str = await msghandler.smartpack(
    "/device/command",
    [("command", {"action": "read_sensor"}, "dictionary")],
    broker_url="nats://localhost:4222",
    reply_to="/device/response"
)

# Receiver (in separate application)
# msg = await nats_consumer.next()
# env = await msghandler.smartunpack(msg)
# Process request and send response
# response_env, response_json = await msghandler.smartpack(
#     "/device/response",
#     [("result", {"value": 42}, "dictionary")],
#     reply_to="/device/command",
#     reply_to_msg_id=env["msg_id"]
# )

---

Testing

Test File Organization

Platform	Sender Tests	Receiver Tests
Julia	test/test_julia_*_sender.jl	test/test_julia_*_receiver.jl
JavaScript	test/test_js_*_sender.js	test/test_js_*_receiver.js
Python	test/test_py_*_sender.py	test/test_py_*_receiver.py

Run Tests

Julia

# Text message exchange
julia test/test_julia_text_sender.jl
julia test/test_julia_text_receiver.jl

# Dictionary exchange
julia test/test_julia_dict_sender.jl
julia test/test_julia_dict_receiver.jl

# File transfer
julia test/test_julia_file_sender.jl
julia test/test_julia_file_receiver.jl

# Mixed payload types
julia test/test_julia_mix_payloads_sender.jl
julia test/test_julia_mix_payloads_receiver.jl

# Table exchange
julia test/test_julia_table_sender.jl
julia test/test_julia_table_receiver.jl

JavaScript (Node.js)

# Text message exchange
node test/test_js_text_sender.js
node test/test_js_text_receiver.js

# Dictionary exchange
node test/test_js_dictionary_sender.js
node test/test_js_dictionary_receiver.js

# Binary transfer
node test/test_js_binary_sender.js
node test/test_js_binary_receiver.js

# Table exchange
node test/test_js_table_sender.js
node test/test_js_table_receiver.js

Python

# Text message exchange
python3 test/test_py_text_sender.py
python3 test/test_py_text_receiver.py

# Dictionary exchange
python3 test/test_py_dictionary_sender.py
python3 test/test_py_dictionary_receiver.py

# Binary transfer
python3 test/test_py_binary_sender.py
python3 test/test_py_binary_receiver.py

# Table exchange
python3 test/test_py_table_sender.py
python3 test/test_py_table_receiver.py

---

Browser Deployment

Using with Node.js Build Tools

The browser implementation (src/msghandler_csr.js) can be bundled for production deployment using modern JavaScript build tools.

Prerequisites

# Install the browser-compatible NATS client
npm install nats.ws

Vite (Recommended)

npm create vite@latest my-app -- --template vanilla
cd my-app
npm install nats.ws

In vite.config.js:

import { defineConfig } from 'vite';
export default defineConfig({
  resolve: {
    alias: {
      'nats.ws': 'nats.ws/dist/esm/browser.js'
    }
  }
});

Build command:

npm run build  # Outputs to dist/ folder

Webpack

npm install webpack webpack-cli --save-dev
npm install nats.ws

In webpack.config.js:

module.exports = {
  entry: './src/index.js',
  output: {
    filename: 'bundle.js',
    path: __dirname + '/dist'
  },
  resolve: {
    alias: {
      'nats.ws': 'nats.ws/dist/esm/browser.js'
    }
  }
};

Build command:

npx webpack

esbuild (Simple & Fast)

npm install esbuild nats.ws --save-dev

Create build.js:

import esbuild from 'esbuild';

esbuild.buildSync({
  entryPoints: ['src/msghandler_csr.js'],
  bundle: true,
  outfile: 'dist/msghandler-csr-bundle.js',
  format: 'esm',
  platform: 'browser',
  target: 'es2020'
});

Build command:

node build.js

Using in Your HTML

<!DOCTYPE html>
<html>
<head>
  <title>My App</title>
</head>
<body>
  <script type="module" src="dist/msghandler-csr-bundle.js"></script>
  <script type="module">
    import msghandlerCSR from './dist/msghandler-csr-bundle.js';
    
    // Use the library
    const [env, envJson] = await msghandlerCSR.smartpack(
      "/chat/user/v1/message",
      [["msg", "Hello", "text"]],
      { broker_url: "wss://nats.example.com" }
    );
  </script>
</body>
</html>

---

Documentation

For detailed architecture and implementation information, see:

• docs/architecture.md - Cross-platform architecture, API parity, platform-specific patterns
• docs/requirements.md - Business requirements and user stories
• docs/spec.md - Technical specification and contracts
• docs/walkthrough.md - Real-world application building guides

---

License

MIT License

Copyright (c) 2026 msghandler Contributors

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.