diff --git a/README.md b/README.md index b7f25ec..31a093b 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ # NATSBridge - Cross-Platform Bi-Directional Data Bridge -A high-performance, bi-directional data bridge for **Julia, JavaScript, Python, and MicroPython** applications using NATS (Core & JetStream), implementing the Claim-Check pattern for large payloads. +A high-performance, bi-directional data bridge for **Julia**, **JavaScript**, **Python**, and **MicroPython** applications using NATS (Core & JetStream), implementing the Claim-Check pattern for large payloads. [![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](https://opensource.org/licenses/MIT) [![NATS](https://img.shields.io/badge/NATS-Enabled-green.svg)](https://nats.io) @@ -28,8 +28,8 @@ NATSBridge enables seamless communication across multiple platforms through NATS | Transport | Payload Size | Method | |-----------|--------------|--------| -| **Direct** | < 1MB | Sent directly via NATS (Base64 encoded) | -| **Link** | >= 1MB | Uploaded to HTTP file server, URL sent via NATS | +| **Direct** | < 500KB | Sent directly via NATS (Base64 encoded) | +| **Link** | ≥ 500KB | Uploaded to HTTP file server, URL sent via NATS | ### Use Cases @@ -45,7 +45,7 @@ NATSBridge enables seamless communication across multiple platforms through NATS | Platform | Implementation | Features | |----------|----------------|----------| | **Julia** | [`src/NATSBridge.jl`](src/NATSBridge.jl) | Full feature set, Arrow IPC, multiple dispatch | -| **JavaScript** | [`src/natsbridge.js`](src/natsbridge.js) | Node.js, async/await | +| **JavaScript (Node.js)** | [`src/natsbridge.js`](src/natsbridge_ssr.js) | Node.js, async/await | | **JavaScript (Browser)** | [`src/natsbridge_csr.js`](src/natsbridge_csr.js) | Browser, WebSocket NATS, async/await | | **Python** | [`src/natsbridge.py`](src/natsbridge.py) | Desktop Python, asyncio, type hints | | **MicroPython** | [`src/natsbridge_mpy.py`](src/natsbridge_mpy.py) | Memory-constrained, synchronous API | @@ -72,7 +72,7 @@ NATSBridge enables seamless communication across multiple platforms through NATS - ✅ **Bi-directional messaging** with request-reply patterns - ✅ **Multi-payload support** - send multiple payloads with different types in one message - ✅ **Automatic transport selection** - direct vs link based on payload size -- ✅ **Claim-Check pattern** for payloads > 1MB +- ✅ **Claim-Check pattern** for payloads ≥ 500KB - ✅ **Apache Arrow IPC** support for tabular data (zero-copy reading) - ✅ **Exponential backoff** for reliable file server downloads - ✅ **Correlation ID tracking** for message tracing @@ -83,23 +83,24 @@ NATSBridge enables seamless communication across multiple platforms through NATS ## Quick Start -### Step 1: Start NATS Server +### Prerequisites -```bash -docker run -p 4222:4222 nats:latest -``` +1. **NATS Server** - Install and run a NATS server: + ```bash + docker run -p 4222:4222 nats:latest + ``` -### Step 2: Start HTTP File Server (Optional) +2. **HTTP File Server** (optional, for large payloads) - Install and run a file server: + ```bash + # Using Plik + docker run -p 8080:8080 -v /tmp/fileserver:/var/lib/plik -e PLIK_ADMIN_PASSWORD=admin plik/plik + + # OR using simple Python HTTP server + mkdir -p /tmp/fileserver + python3 -m http.server 8080 --directory /tmp/fileserver + ``` -```bash -# Create a directory for file uploads -mkdir -p /tmp/fileserver - -# Start HTTP file server -python3 -m http.server 8080 --directory /tmp/fileserver -``` - -### Step 3: Send Your First Message +### Send Your First Message #### Julia @@ -107,14 +108,14 @@ python3 -m http.server 8080 --directory /tmp/fileserver using NATSBridge data = [("message", "Hello World", "text")] -env, env_json_str = smartsend("/chat/room1", data, broker_url="nats://localhost:4222") +env, env_json_str = smartsend("/chat/room1", data; broker_url="nats://localhost:4222") println("Message sent!") ``` -#### JavaScript +#### JavaScript (Node.js) ```javascript -const NATSBridge = require('./src/natsbridge.js'); +import NATSBridge from './src/natsbridge_ssr.js'; const data = [["message", "Hello World", "text"]]; const [env, env_json_str] = await NATSBridge.smartsend( @@ -125,6 +126,20 @@ const [env, env_json_str] = await NATSBridge.smartsend( console.log("Message sent!"); ``` +#### JavaScript (Browser) + +```javascript +import NATSBridge from './src/natsbridge_csr.js'; + +const data = [["message", "Hello World", "text"]]; +const [env, env_json_str] = await NATSBridge.smartsend( + "/chat/room1", + data, + { broker_url: "ws://localhost:4222" } +); +console.log("Message sent!"); +``` + #### Python ```python @@ -139,6 +154,21 @@ env, env_json_str = await smartsend( print("Message sent!") ``` +#### MicroPython + +```python +from natsbridge import smartsend + +data = [("message", "Hello World", "text")] +env, env_json_str = smartsend( + "/chat/room1", + data, + broker_url="nats://localhost:4222", + size_threshold=100000 # 100KB for MicroPython +) +print("Message sent!") +``` + --- ## API Reference @@ -147,13 +177,13 @@ print("Message sent!") All platforms use the same input/output format for payloads: -**Input format for smartsend:** +**Input format for `smartsend`:** ``` [(dataname1, data1, type1), (dataname2, data2, type2), ...] ``` -**Output format for smartreceive:** -``` +**Output format for `smartreceive`:** +```json { "correlation_id": "...", "msg_id": "...", @@ -187,7 +217,7 @@ env, env_json_str = NATSBridge.smartsend( broker_url::String = "nats://localhost:4222", fileserver_url = "http://localhost:8080", fileserver_upload_handler::Function = plik_oneshot_upload, - size_threshold::Int = 1_000_000, + size_threshold::Int = 500_000, correlation_id::String = string(uuid4()), msg_purpose::String = "chat", sender_name::String = "NATSBridge", @@ -203,10 +233,10 @@ env, env_json_str = NATSBridge.smartsend( # Returns: ::Tuple{msg_envelope_v1, String} ``` -#### JavaScript +#### JavaScript (Node.js) ```javascript -const NATSBridge = require('natsbridge'); +import NATSBridge from './src/natsbridge_ssr.js'; const [env, env_json_str] = await NATSBridge.smartsend( subject, @@ -215,7 +245,36 @@ const [env, env_json_str] = await NATSBridge.smartsend( broker_url: 'nats://localhost:4222', fileserver_url: 'http://localhost:8080', fileserver_upload_handler: NATSBridge.plikOneshotUpload, - size_threshold: 1_000_000, + size_threshold: 500_000, + correlation_id: uuidv4(), + msg_purpose: 'chat', + sender_name: 'NATSBridge', + receiver_name: '', + receiver_id: '', + reply_to: '', + reply_to_msg_id: '', + is_publish: true, + nats_connection: null, + msg_id: uuidv4(), + sender_id: uuidv4() + } +); +// Returns: Promise<[env, env_json_str]> +``` + +#### JavaScript (Browser) + +```javascript +import NATSBridge from './src/natsbridge_csr.js'; + +const [env, env_json_str] = await NATSBridge.smartsend( + subject, + data, + { + broker_url: 'ws://localhost:4222', + fileserver_url: 'http://localhost:8080', + fileserver_upload_handler: NATSBridge.plikOneshotUpload, + size_threshold: 500_000, correlation_id: uuidv4(), msg_purpose: 'chat', sender_name: 'NATSBridge', @@ -243,7 +302,7 @@ env, env_json_str = await NATSBridge.smartsend( broker_url: str = "nats://localhost:4222", fileserver_url: str = "http://localhost:8080", fileserver_upload_handler: Callable = plik_oneshot_upload, - size_threshold: int = 1_000_000, + size_threshold: int = 500_000, correlation_id: str = None, msg_purpose: str = "chat", sender_name: str = "NATSBridge", @@ -293,9 +352,28 @@ env = NATSBridge.smartreceive( # Returns: ::JSON.Object{String, Any} ``` -#### JavaScript +#### JavaScript (Node.js) ```javascript +import NATSBridge from './src/natsbridge_ssr.js'; + +const env = await NATSBridge.smartreceive( + msg, + { + fileserver_download_handler: NATSBridge.fetchWithBackoff, + max_retries: 5, + base_delay: 100, + max_delay: 5000 + } +); +// Returns: Promise +``` + +#### JavaScript (Browser) + +```javascript +import NATSBridge from './src/natsbridge_csr.js'; + const env = await NATSBridge.smartreceive( msg, { @@ -311,6 +389,8 @@ const env = await NATSBridge.smartreceive( #### Python ```python +from natsbridge import NATSBridge + env = await NATSBridge.smartreceive( msg, fileserver_download_handler=fetch_with_backoff, @@ -324,6 +404,8 @@ env = await NATSBridge.smartreceive( #### MicroPython ```python +from natsbridge import NATSBridge + env = NATSBridge.smartreceive( msg, fileserver_download_handler=_sync_fileserver_download, @@ -343,7 +425,7 @@ env = NATSBridge.smartreceive( | `text` | `String` | `string` | `str` | `str` | Plain text strings | | `dictionary` | `Dict`, `NamedTuple` | `Object`, `Array` | `dict`, `list` | `dict` | JSON-serializable dictionaries | | `arrowtable` | `DataFrame`, `Arrow.Table` | `Array` | `pandas.DataFrame` | ❌ | Tabular data (Arrow IPC) | -| `jsontable` | `Vector{NamedTuple}` | `Array` | `list[dict]` | ❌ | Tabular data (JSON) | +| `jsontable` | `Vector{NamedTuple}` | `Array` | `list[dict]` | ⚠️ | Tabular data (JSON) | | `image` | `Vector{UInt8}` | `Uint8Array`, `Buffer` | `bytes` | `bytearray` | Image data (PNG, JPG) | | `audio` | `Vector{UInt8}` | `Uint8Array`, `Buffer` | `bytes` | `bytearray` | Audio data (WAV, MP3) | | `video` | `Vector{UInt8}` | `Uint8Array`, `Buffer` | `bytes` | `bytearray` | Video data (MP4, AVI) | @@ -368,13 +450,13 @@ data = [ ("large_document", large_file_data, "binary") ] -env, env_json_str = NATSBridge.smartsend("/chat/room1", data; fileserver_url="http://localhost:8080") +env, env_json_str = smartsend("/chat/room1", data; fileserver_url="http://localhost:8080") ``` -#### JavaScript +#### JavaScript (Node.js) ```javascript -const NATSBridge = require('natsbridge'); +import NATSBridge from './src/natsbridge_ssr.js'; const data = [ ["message_text", "Hello!", "text"], @@ -389,6 +471,24 @@ const [env, env_json_str] = await NATSBridge.smartsend( ); ``` +#### JavaScript (Browser) + +```javascript +import NATSBridge from './src/natsbridge_csr.js'; + +const data = [ + ["message_text", "Hello!", "text"], + ["user_avatar", imageData, "image"], + ["large_document", largeFileData, "binary"] +]; + +const [env, env_json_str] = await NATSBridge.smartsend( + "/chat/room1", + data, + { broker_url: 'ws://localhost:4222', fileserver_url: 'http://localhost:8080' } +); +``` + #### Python ```python @@ -423,13 +523,13 @@ config = Dict( ) data = [("config", config, "dictionary")] -env, env_json_str = NATSBridge.smartsend("/device/config", data) +env, env_json_str = smartsend("/device/config", data) ``` -#### JavaScript +#### JavaScript (Node.js) ```javascript -const NATSBridge = require('natsbridge'); +import NATSBridge from './src/natsbridge_ssr.js'; const config = { wifi_ssid: "MyNetwork", @@ -475,13 +575,13 @@ df = DataFrame( ) data = [("students", df, "arrowtable")] -env, env_json_str = NATSBridge.smartsend("/data/analysis", data) +env, env_json_str = smartsend("/data/analysis", data) ``` -#### JavaScript +#### JavaScript (Node.js) ```javascript -const NATSBridge = require('natsbridge'); +import NATSBridge from './src/natsbridge_ssr.js'; const df = [ { id: 1, name: "Alice", score: 95 }, @@ -521,18 +621,29 @@ Bi-directional communication with reply-to support. using NATSBridge # Requester -env, env_json_str = NATSBridge.smartsend( +env, env_json_str = smartsend( "/device/command", [("command", Dict("action" => "read_sensor"), "dictionary")]; broker_url="nats://localhost:4222", reply_to="/device/response" ) + +# Receiver (in separate application) +msg = NATS.subscription.next() +env = smartreceive(msg) +# Process request and send response +response_env, response_json = smartsend( + "/device/response", + [("result", Dict("value" => 42), "dictionary")], + reply_to="/device/command", + reply_to_msg_id=env["msg_id"] +) ``` -#### JavaScript +#### JavaScript (Node.js) ```javascript -const NATSBridge = require('natsbridge'); +import NATSBridge from './src/natsbridge_ssr.js'; // Requester const [env, env_json_str] = await NATSBridge.smartsend( @@ -540,6 +651,16 @@ const [env, env_json_str] = await NATSBridge.smartsend( [["command", { action: "read_sensor" }, "dictionary"]], { broker_url: 'nats://localhost:4222', reply_to: '/device/response' } ); + +// Receiver (in separate application) +// const msg = await natsConsumer.next(); +// const env = await NATSBridge.smartreceive(msg); +// Process request and send response +// const response_env, response_json = await NATSBridge.smartsend( +// "/device/response", +// [["result", { value: 42 }, "dictionary"]], +// { reply_to: '/device/command', reply_to_msg_id: env.msg_id } +// ); ``` #### Python @@ -554,6 +675,17 @@ env, env_json_str = await NATSBridge.smartsend( broker_url="nats://localhost:4222", reply_to="/device/response" ) + +# Receiver (in separate application) +# msg = await nats_consumer.next() +# env = await NATSBridge.smartreceive(msg) +# Process request and send response +# response_env, response_json = await NATSBridge.smartsend( +# "/device/response", +# [("result", {"value": 42}, "dictionary")], +# reply_to="/device/command", +# reply_to_msg_id=env["msg_id"] +# ) ``` --- @@ -640,10 +772,10 @@ python3 test/test_py_table_receiver.py For detailed architecture and implementation information, see: -- [Architecture Documentation](docs/architecture_updated.md) - Cross-platform architecture, API parity, platform-specific patterns -- [Implementation Guide](docs/implementation_updated.md) - Detailed implementation for each platform, handler functions, testing -- [Tutorial](docs/tutorial_updated.md) - Step-by-step getting started guide -- [Walkthrough](docs/walkthrough_updated.md) - Real-world application building guides +- [`docs/architecture.md`](docs/architecture.md) - Cross-platform architecture, API parity, platform-specific patterns +- [`docs/requirements.md`](docs/requirements.md) - Business requirements and user stories +- [`docs/spec.md`](docs/spec.md) - Technical specification and contracts +- [`docs/walkthrough.md`](docs/walkthrough.md) - Real-world application building guides --- diff --git a/docs/tutorial.md b/docs/tutorial.md deleted file mode 100644 index 562e082..0000000 --- a/docs/tutorial.md +++ /dev/null @@ -1,741 +0,0 @@ -# Cross-Platform NATSBridge Tutorial - -A step-by-step guide to get started with NATSBridge across **Julia**, **JavaScript**, and **Python/MicroPython**. - -## Table of Contents - -1. [Overview](#overview) -2. [Prerequisites](#prerequisites) -3. [Installation](#installation) -4. [Quick Start](#quick-start) -5. [Basic Examples](#basic-examples) -6. [Advanced Usage](#advanced-usage) - ---- - -## Overview - -NATSBridge enables seamless communication across platforms through NATS, with automatic transport selection based on payload size: - -- **Direct Transport**: Payloads < 1MB are sent directly via NATS (Base64 encoded) -- **Link Transport**: Payloads >= 1MB are uploaded to an HTTP file server and referenced via URL - -### Cross-Platform API Parity - -All three platforms use the same high-level API: - -``` -# Input format -smartsend(subject, [(dataname, data, type), ...], options) - -# Output format -(env, env_json_str) = smartsend(...) -env = smartreceive(msg, options) -``` - -**Important Platform Differences:** - -1. **Encoding field:** Julia and JavaScript preserve the original serialization format in the encoding field (`"base64"`, `"json"`, or `"arrow-ipc"`), while Python and MicroPython always use `"base64"` for all direct transport payloads. - -2. **Async vs Sync:** JavaScript and Python desktop use async/await, while MicroPython uses synchronous API. - -### Supported Payload Types - -| Type | Julia | JavaScript | Python | MicroPython | -|------|-------|------------|--------|-------------| -| `text` | `String` | `string` | `str` | `str` | -| `dictionary` | `Dict` | `Object` | `dict` | `dict` | -| `arrowtable` | `DataFrame` | `Array` | `pandas.DataFrame` | ❌ | -| `jsontable` | `Vector{NamedTuple}` | `Array` | `list[dict]` | ❌ | -| `table` | ❌ | ❌ | `pandas.DataFrame` | ❌ | -| `image` | `Vector{UInt8}` | `Uint8Array` | `bytes` | `bytearray` | -| `audio` | `Vector{UInt8}` | `Uint8Array` | `bytes` | `bytearray` | -| `video` | `Vector{UInt8}` | `Uint8Array` | `bytes` | `bytearray` | -| `binary` | `Vector{UInt8}` | `Uint8Array` | `bytes` | `bytearray` | - -**Note on MicroPython:** MicroPython does not support table types (`arrowtable`, `jsontable`, or `table`) due to memory constraints. Use `dictionary` or `binary` instead. - ---- - -## Prerequisites - -Before you begin, ensure you have: - -1. **NATS Server** running (or accessible) -2. **HTTP File Server** (optional, for large payloads > 1MB) -3. **Platform-specific packages** installed - ---- - -## Installation - -### Julia - -```julia -using Pkg -Pkg.add("NATS") -Pkg.add("Arrow") -Pkg.add("JSON3") -Pkg.add("HTTP") -Pkg.add("UUIDs") -Pkg.add("Dates") -``` - -### JavaScript (Node.js) - -```bash -npm install nats uuid apache-arrow node-fetch -``` - -### JavaScript (Browser) - -```html - - -``` - -### Python (Desktop) - -```bash -pip install nats-py aiohttp pyarrow pandas -``` - -### MicroPython - -Uses built-in modules: `network`, `socket`, `time`, `json`, `base64` - ---- - -## Quick Start - -### Step 1: Start NATS Server - -```bash -docker run -p 4222:4222 nats:latest -``` - -### Step 2: Start HTTP File Server (Optional) - -```bash -mkdir -p /tmp/fileserver -python3 -m http.server 8080 --directory /tmp/fileserver -``` - -### Step 3: Send Your First Message - -#### Julia - -```julia -using NATSBridge - -# Send a text message -data = [("message", "Hello World", "text")] -env, env_json_str = smartsend("/chat/room1", data, broker_url="nats://localhost:4222") -# env: msg_envelope_v1 struct with all metadata and payloads -# env_json_str: JSON string representation of the envelope for publishing -println("Message sent!") - -# Or use is_publish=false to get envelope and JSON without publishing -env, env_json_str = smartsend("/chat/room1", data, broker_url="nats://localhost:4222", is_publish=false) -# env: msg_envelope_v1 struct -# env_json_str: JSON string for publishing to NATS -``` - -#### JavaScript - -```javascript -const NATSBridge = require('./src/natsbridge.js'); - -// Send a text message -const data = [["message", "Hello World", "text"]]; -const [env, env_json_str] = await NATSBridge.smartsend( - "/chat/room1", - data, - { broker_url: "nats://localhost:4222" } -); -// env: Object with all metadata and payloads -// env_json_str: JSON string for publishing -console.log("Message sent!"); - -// Or use is_publish=false -const [env, env_json_str] = await NATSBridge.smartsend( - "/chat/room1", - data, - { broker_url: "nats://localhost:4222", is_publish: false } -); -``` - -#### Python - -```python -from natsbridge import smartsend - -# Send a text message -data = [("message", "Hello World", "text")] -env, env_json_str = await smartsend( - "/chat/room1", - data, - broker_url="nats://localhost:4222" -) -# env: Dict with all metadata and payloads -# env_json_str: JSON string for publishing -print("Message sent!") - -# Or use is_publish=False -env, env_json_str = await smartsend( - "/chat/room1", - data, - broker_url="nats://localhost:4222", - is_publish=False -) -# env: Dict with all metadata and payloads -# env_json_str: JSON string for publishing to NATS -``` - -#### MicroPython - -```python -from natsbridge_mpy import NATSBridge - -bridge = NATSBridge() - -# Send a text message (limited to small payloads) -data = [("message", "Hello World", "text")] -env, env_json_str = bridge.smartsend( - "/chat/room1", - data, - size_threshold=100000 # Lower threshold for MicroPython -) -print("Message sent!") -``` - -### Step 4: Receive Messages - -#### Julia - -```julia -using NATSBridge - -# Receive and process message -env = smartreceive(msg; fileserver_download_handler=_fetch_with_backoff) -# Returns: ::JSON.Object{String, Any} with "payloads" field containing Vector{Tuple{String, Any, String}} -# Access payloads: for (dataname, data, type) in env["payloads"] -for (dataname, data, type) in env["payloads"] - println("Received $dataname: $data") -end -``` - -#### JavaScript - -```javascript -const NATSBridge = require('./src/natsbridge.js'); - -// Receive and process message -const env = await NATSBridge.smartreceive(msg, { - fileserver_download_handler: NATSBridge.fetchWithBackoff -}); -// env.payloads = [[dataname, data, type], ...] -for (const [dataname, data, type] of env.payloads) { - console.log(`Received ${dataname}:`, data); -} -``` - -#### Python - -```python -from natsbridge import smartreceive, fetch_with_backoff - -# Receive and process message -env = await smartreceive( - msg, - fileserver_download_handler=fetch_with_backoff -) -# env["payloads"] = [(dataname, data, type), ...] -for dataname, data, type_ in env["payloads"]: - print(f"Received {dataname}: {data}") -``` - ---- - -## Basic Examples - -### Example 1: Sending a Dictionary - -#### Julia - -```julia -using NATSBridge - -config = Dict( - "wifi_ssid" => "MyNetwork", - "wifi_password" => "password123", - "update_interval" => 60 -) - -data = [("config", config, "dictionary")] -env, env_json_str = smartsend("/device/config", data, broker_url="nats://localhost:4222") -``` - -#### JavaScript - -```javascript -const NATSBridge = require('./src/natsbridge.js'); - -const config = { - wifi_ssid: "MyNetwork", - wifi_password: "password123", - update_interval: 60 -}; - -const data = [["config", config, "dictionary"]]; -const [env, env_json_str] = await NATSBridge.smartsend( - "/device/config", - data, - { broker_url: "nats://localhost:4222" } -); -``` - -#### Python - -```python -from natsbridge import smartsend - -config = { - "wifi_ssid": "MyNetwork", - "wifi_password": "password123", - "update_interval": 60 -} - -data = [("config", config, "dictionary")] -env, env_json_str = await smartsend( - "/device/config", - data, - broker_url="nats://localhost:4222" -) -``` - -#### MicroPython - -```python -from natsbridge_mpy import NATSBridge - -bridge = NATSBridge() - -config = { - "wifi_ssid": "MyNetwork", - "wifi_password": "password123", - "update_interval": 60 -} - -data = [("config", config, "dictionary")] -env, env_json_str = bridge.smartsend( - "/device/config", - data, - size_threshold=100000 -) -``` - -### Example 2: Sending Binary Data (Image) - -#### Julia - -```julia -using NATSBridge - -# Read image file -image_data = read("image.png") - -data = [("user_image", image_data, "binary")] -env, env_json_str = smartsend("/chat/image", data, broker_url="nats://localhost:4222") -``` - -#### JavaScript - -```javascript -const NATSBridge = require('./src/natsbridge.js'); -const fs = require('fs'); - -// Read image file -const image_data = fs.readFileSync('image.png'); - -const data = [["user_image", image_data, "binary"]]; -const [env, env_json_str] = await NATSBridge.smartsend( - "/chat/image", - data, - { broker_url: "nats://localhost:4222" } -); -``` - -#### Python - -```python -from natsbridge import smartsend - -# Read image file -with open("image.png", "rb") as f: - image_data = f.read() - -data = [("user_image", image_data, "binary")] -env, env_json_str = await smartsend( - "/chat/image", - data, - broker_url="nats://localhost:4222" -) -``` - -#### MicroPython - -```python -from natsbridge_mpy import NATSBridge - -bridge = NATSBridge() - -# Read image file -with open("image.png", "rb") as f: - image_data = f.read() - -data = [("user_image", image_data, "binary")] -env, env_json_str = bridge.smartsend( - "/chat/image", - data, - size_threshold=100000 -) -``` - -### Example 3: Request-Response Pattern - -#### Julia (Requester) - -```julia -using NATSBridge - -# Send command with reply-to -data = [("command", Dict("action" => "read_sensor"), "dictionary")] -env, env_json_str = smartsend( - "/device/command", - data, - broker_url="nats://localhost:4222", - reply_to="/device/response", - reply_to_msg_id="cmd-001" -) -``` - -#### JavaScript (Requester) - -```javascript -const NATSBridge = require('./src/natsbridge.js'); - -// Send command with reply-to -const data = [["command", { action: "read_sensor" }, "dictionary"]]; -const [env, env_json_str] = await NATSBridge.smartsend( - "/device/command", - data, - { - broker_url: "nats://localhost:4222", - reply_to: "/device/response", - reply_to_msg_id: "cmd-001" - } -); -``` - -#### Python (Requester) - -```python -from natsbridge import smartsend - -# Send command with reply-to -data = [("command", {"action": "read_sensor"}, "dictionary")] -env, env_json_str = await smartsend( - "/device/command", - data, - broker_url="nats://localhost:4222", - reply_to="/device/response", - reply_to_msg_id="cmd-001" -) -``` - -#### Julia (Responder) - -```julia -using NATSBridge, NATS - -const SUBJECT = "/device/command" -const NATS_URL = "nats://localhost:4222" - -function test_responder() - conn = NATS.connect(NATS_URL) - NATS.subscribe(conn, SUBJECT) do msg - env = smartreceive(msg, fileserver_download_handler=_fetch_with_backoff) - - reply_to = env["reply_to"] - - for (dataname, data, type) in env["payloads"] - if dataname == "command" && data["action"] == "read_sensor" - response = Dict("sensor_id" => "sensor-001", "value" => 42.5) - if !isempty(reply_to) - smartsend(reply_to, [("data", response, "dictionary")]) - end - end - end - end - - sleep(120) - NATS.drain(conn) -end - -test_responder() -``` - ---- - -## Advanced Usage - -### Example 4: Large Payloads (File Server) - -For payloads larger than 1MB, NATSBridge automatically uses the file server: - -#### Julia - -```julia -using NATSBridge - -# Create large data (> 1MB) -large_data = rand(UInt8, 2_000_000) - -env, env_json_str = smartsend( - "/data/large", - [("large_file", large_data, "binary")], - broker_url="nats://localhost:4222", - fileserver_url="http://localhost:8080" -) - -println("File uploaded to: $(env.payloads[1].data)") -# Note: For link transport, data field contains the URL string -``` - -#### JavaScript - -```javascript -const NATSBridge = require('./src/natsbridge.js'); - -// Create large data (> 1MB) -const large_data = Buffer.alloc(2_000_000); -for (let i = 0; i < large_data.length; i++) { - large_data[i] = Math.floor(Math.random() * 256); -} - -const [env, env_json_str] = await NATSBridge.smartsend( - "/data/large", - [["large_file", large_data, "binary"]], - { - broker_url: "nats://localhost:4222", - fileserver_url: "http://localhost:8080" - } -); - -console.log("File uploaded to:", env.payloads[0].data); -// Note: For link transport, data field contains the URL string -``` - -#### Python - -```python -from natsbridge import smartsend - -# Create large data (> 1MB) -import os -large_data = os.urandom(2_000_000) - -env, env_json_str = await smartsend( - "/data/large", - [("large_file", large_data, "binary")], - broker_url="nats://localhost:4222", - fileserver_url="http://localhost:8080" -) - -print(f"File uploaded to: {env['payloads'][0]['data']}") -# Note: For link transport, data field contains the URL string -``` - -#### MicroPython - -MicroPython enforces a hard limit of 50KB per payload: - -```python -from natsbridge_mpy import NATSBridge - -bridge = NATSBridge() - -# MicroPython has a hard limit of 50KB per payload -# Use streaming or chunking for larger data -small_data = bytes(1000) # 1KB - -data = [("small_file", small_data, "binary")] -env, env_json_str = bridge.smartsend( - "/data/small", - data, - size_threshold=100000 # Enforced max: 50000 bytes -) -``` - -### Example 5: Mixed Content (Chat with Text + Image) - -NATSBridge supports sending multiple payloads with different types in a single message: - -#### Julia - -```julia -using NATSBridge - -image_data = read("avatar.png") - -data = [ - ("message_text", "Hello with image!", "text"), - ("user_avatar", image_data, "image") -] - -env, env_json_str = smartsend("/chat/mixed", data, broker_url="nats://localhost:4222") -``` - -#### JavaScript - -```javascript -const NATSBridge = require('./src/natsbridge.js'); -const fs = require('fs'); - -const image_data = fs.readFileSync('avatar.png'); - -const data = [ - ["message_text", "Hello with image!", "text"], - ["user_avatar", image_data, "image"] -]; - -const [env, env_json_str] = await NATSBridge.smartsend( - "/chat/mixed", - data, - { broker_url: "nats://localhost:4222" } -); -``` - -#### Python - -```python -from natsbridge import smartsend - -with open("avatar.png", "rb") as f: - image_data = f.read() - -data = [ - ("message_text", "Hello with image!", "text"), - ("user_avatar", image_data, "image") -] - -env, env_json_str = await smartsend( - "/chat/mixed", - data, - broker_url="nats://localhost:4222" -) -# env: Dict with all metadata and payloads -``` - -### Example 6: Table Data (Arrow IPC) - -For tabular data, NATSBridge uses Apache Arrow IPC format: - -#### Julia - -```julia -using NATSBridge -using DataFrames - -# Create DataFrame -df = DataFrame( - id = [1, 2, 3], - name = ["Alice", "Bob", "Charlie"], - score = [95, 88, 92] -) - -data = [("students", df, "arrowtable")] -env, env_json_str = smartsend("/data/students", data, broker_url="nats://localhost:4222") -``` - -#### JavaScript - -```javascript -const NATSBridge = require('./src/natsbridge.js'); - -// Create table data (array of objects) -const table_data = [ - { id: 1, name: "Alice", score: 95 }, - { id: 2, name: "Bob", score: 88 }, - { id: 3, name: "Charlie", score: 92 } -]; - -const data = [["students", table_data, "arrowtable"]]; -const [env, env_json_str] = await NATSBridge.smartsend( - "/data/students", - data, - { broker_url: "nats://localhost:4222" } -); -``` - -#### Python - -```python -from natsbridge import smartsend -import pandas as pd - -# Create DataFrame -df = pd.DataFrame({ - 'id': [1, 2, 3], - 'name': ['Alice', 'Bob', 'Charlie'], - 'score': [95, 88, 92] -}) - -data = [("students", df, "table")] -env, env_json_str = await smartsend( - "/data/students", - data, - broker_url="nats://localhost:4222" -) -``` - -#### MicroPython - -MicroPython does not support table type due to memory constraints. Use dictionary or binary instead. - ---- - -## Next Steps - -1. **Explore the test directory** for more examples -2. **Check the documentation** for advanced configuration options -3. **Read the walkthrough** for building real-world applications - ---- - -## Troubleshooting - -### Connection Issues - -- Ensure NATS server is running: `docker ps | grep nats` -- Check firewall settings -- Verify NATS URL configuration - -### File Server Issues - -- Ensure file server is running and accessible -- Check upload permissions -- Verify file server URL configuration - -### Serialization Errors - -- Verify data type matches the specified type -- Check that binary data is in the correct format -- MicroPython: Ensure payload size < 50KB - ---- - -## License - -MIT