724 lines
15 KiB
Markdown
724 lines
15 KiB
Markdown
# Cross-Platform NATSBridge Tutorial
|
|
|
|
A step-by-step guide to get started with NATSBridge across **Julia**, **JavaScript**, and **Python/MicroPython**.
|
|
|
|
## Table of Contents
|
|
|
|
1. [Overview](#overview)
|
|
2. [Prerequisites](#prerequisites)
|
|
3. [Installation](#installation)
|
|
4. [Quick Start](#quick-start)
|
|
5. [Basic Examples](#basic-examples)
|
|
6. [Advanced Usage](#advanced-usage)
|
|
|
|
---
|
|
|
|
## Overview
|
|
|
|
NATSBridge enables seamless communication across platforms through NATS, with automatic transport selection based on payload size:
|
|
|
|
- **Direct Transport**: Payloads < 1MB are sent directly via NATS (Base64 encoded)
|
|
- **Link Transport**: Payloads >= 1MB are uploaded to an HTTP file server and referenced via URL
|
|
|
|
### Cross-Platform API Parity
|
|
|
|
All three platforms use the same high-level API:
|
|
|
|
```
|
|
# Input format
|
|
smartsend(subject, [(dataname, data, type), ...], options)
|
|
|
|
# Output format
|
|
(env, env_json_str) = smartsend(...)
|
|
env = smartreceive(msg, options)
|
|
```
|
|
|
|
### Supported Payload Types
|
|
|
|
| Type | Julia | JavaScript | Python | MicroPython |
|
|
|------|-------|------------|--------|-------------|
|
|
| `text` | `String` | `string` | `str` | `str` |
|
|
| `dictionary` | `Dict` | `Object` | `dict` | `dict` |
|
|
| `table` | `DataFrame` | `Array<Object>` | `DataFrame` | ❌ |
|
|
| `image` | `Vector{UInt8}` | `Uint8Array` | `bytes` | `bytearray` |
|
|
| `audio` | `Vector{UInt8}` | `Uint8Array` | `bytes` | `bytearray` |
|
|
| `video` | `Vector{UInt8}` | `Uint8Array` | `bytes` | `bytearray` |
|
|
| `binary` | `Vector{UInt8}` | `Uint8Array` | `bytes` | `bytearray` |
|
|
|
|
---
|
|
|
|
## Prerequisites
|
|
|
|
Before you begin, ensure you have:
|
|
|
|
1. **NATS Server** running (or accessible)
|
|
2. **HTTP File Server** (optional, for large payloads > 1MB)
|
|
3. **Platform-specific packages** installed
|
|
|
|
---
|
|
|
|
## Installation
|
|
|
|
### Julia
|
|
|
|
```julia
|
|
using Pkg
|
|
Pkg.add("NATS")
|
|
Pkg.add("Arrow")
|
|
Pkg.add("JSON3")
|
|
Pkg.add("HTTP")
|
|
Pkg.add("UUIDs")
|
|
Pkg.add("Dates")
|
|
```
|
|
|
|
### JavaScript (Node.js)
|
|
|
|
```bash
|
|
npm install nats uuid apache-arrow node-fetch
|
|
```
|
|
|
|
### JavaScript (Browser)
|
|
|
|
```html
|
|
<script src="https://unpkg.com/nats-js/dist/bundle/nats.min.js"></script>
|
|
<script src="https://unpkg.com/apache-arrow/arrow.min.js"></script>
|
|
```
|
|
|
|
### Python (Desktop)
|
|
|
|
```bash
|
|
pip install nats-py aiohttp pyarrow pandas
|
|
```
|
|
|
|
### MicroPython
|
|
|
|
Uses built-in modules: `network`, `socket`, `time`, `json`, `base64`
|
|
|
|
---
|
|
|
|
## Quick Start
|
|
|
|
### Step 1: Start NATS Server
|
|
|
|
```bash
|
|
docker run -p 4222:4222 nats:latest
|
|
```
|
|
|
|
### Step 2: Start HTTP File Server (Optional)
|
|
|
|
```bash
|
|
mkdir -p /tmp/fileserver
|
|
python3 -m http.server 8080 --directory /tmp/fileserver
|
|
```
|
|
|
|
### Step 3: Send Your First Message
|
|
|
|
#### Julia
|
|
|
|
```julia
|
|
using NATSBridge
|
|
|
|
# Send a text message
|
|
data = [("message", "Hello World", "text")]
|
|
env, env_json_str = smartsend("/chat/room1", data, broker_url="nats://localhost:4222")
|
|
# env: msg_envelope_v1 object with all metadata and payloads
|
|
# env_json_str: JSON string representation of the envelope for publishing
|
|
println("Message sent!")
|
|
|
|
# Or use is_publish=false to get envelope and JSON without publishing
|
|
env, env_json_str = smartsend("/chat/room1", data, broker_url="nats://localhost:4222", is_publish=false)
|
|
# env: msg_envelope_v1 object
|
|
# env_json_str: JSON string for publishing to NATS
|
|
```
|
|
|
|
#### JavaScript
|
|
|
|
```javascript
|
|
const NATSBridge = require('./src/natbridge.js');
|
|
|
|
// Send a text message
|
|
const data = [["message", "Hello World", "text"]];
|
|
const [env, env_json_str] = await NATSBridge.smartsend(
|
|
"/chat/room1",
|
|
data,
|
|
{ broker_url: "nats://localhost:4222" }
|
|
);
|
|
// env: Object with all metadata and payloads
|
|
// env_json_str: JSON string for publishing
|
|
console.log("Message sent!");
|
|
|
|
// Or use is_publish=false
|
|
const [env, env_json_str] = await NATSBridge.smartsend(
|
|
"/chat/room1",
|
|
data,
|
|
{ broker_url: "nats://localhost:4222", is_publish: false }
|
|
);
|
|
```
|
|
|
|
#### Python
|
|
|
|
```python
|
|
from natbridge import smartsend
|
|
|
|
# Send a text message
|
|
data = [("message", "Hello World", "text")]
|
|
env, env_json_str = await smartsend(
|
|
"/chat/room1",
|
|
data,
|
|
broker_url="nats://localhost:4222"
|
|
)
|
|
# env: Dict with all metadata and payloads
|
|
# env_json_str: JSON string for publishing
|
|
print("Message sent!")
|
|
|
|
# Or use is_publish=False
|
|
env, env_json_str = await smartsend(
|
|
"/chat/room1",
|
|
data,
|
|
broker_url="nats://localhost:4222",
|
|
is_publish=False
|
|
)
|
|
```
|
|
|
|
#### MicroPython
|
|
|
|
```python
|
|
from natbridge_mpy import NATSBridge
|
|
|
|
bridge = NATSBridge()
|
|
|
|
# Send a text message (limited to small payloads)
|
|
data = [("message", "Hello World", "text")]
|
|
env, env_json_str = bridge.smartsend(
|
|
"/chat/room1",
|
|
data,
|
|
size_threshold=100000 # Lower threshold for MicroPython
|
|
)
|
|
print("Message sent!")
|
|
```
|
|
|
|
### Step 4: Receive Messages
|
|
|
|
#### Julia
|
|
|
|
```julia
|
|
using NATSBridge
|
|
|
|
# Receive and process message
|
|
env = smartreceive(msg; fileserver_download_handler=_fetch_with_backoff)
|
|
# Returns: ::JSON.Object{String, Any} - key-value structure resemble msg_envelope_v1
|
|
for (dataname, data, type) in env["payloads"]
|
|
println("Received $dataname: $data")
|
|
end
|
|
```
|
|
|
|
#### JavaScript
|
|
|
|
```javascript
|
|
const NATSBridge = require('./src/natbridge.js');
|
|
|
|
// Receive and process message
|
|
const env = await NATSBridge.smartreceive(msg, {
|
|
fileserver_download_handler: NATSBridge.fetchWithBackoff
|
|
});
|
|
// env.payloads = [[dataname, data, type], ...]
|
|
for (const [dataname, data, type] of env.payloads) {
|
|
console.log(`Received ${dataname}:`, data);
|
|
}
|
|
```
|
|
|
|
#### Python
|
|
|
|
```python
|
|
from natbridge import smartreceive
|
|
|
|
# Receive and process message
|
|
env = await smartreceive(
|
|
msg,
|
|
fileserver_download_handler=fetch_with_backoff
|
|
)
|
|
# env["payloads"] = [(dataname, data, type), ...]
|
|
for dataname, data, type in env["payloads"]:
|
|
print(f"Received {dataname}: {data}")
|
|
```
|
|
|
|
---
|
|
|
|
## Basic Examples
|
|
|
|
### Example 1: Sending a Dictionary
|
|
|
|
#### Julia
|
|
|
|
```julia
|
|
using NATSBridge
|
|
|
|
config = Dict(
|
|
"wifi_ssid" => "MyNetwork",
|
|
"wifi_password" => "password123",
|
|
"update_interval" => 60
|
|
)
|
|
|
|
data = [("config", config, "dictionary")]
|
|
env, env_json_str = smartsend("/device/config", data, broker_url="nats://localhost:4222")
|
|
```
|
|
|
|
#### JavaScript
|
|
|
|
```javascript
|
|
const NATSBridge = require('./src/natbridge.js');
|
|
|
|
const config = {
|
|
wifi_ssid: "MyNetwork",
|
|
wifi_password: "password123",
|
|
update_interval: 60
|
|
};
|
|
|
|
const data = [["config", config, "dictionary"]];
|
|
const [env, env_json_str] = await NATSBridge.smartsend(
|
|
"/device/config",
|
|
data,
|
|
{ broker_url: "nats://localhost:4222" }
|
|
);
|
|
```
|
|
|
|
#### Python
|
|
|
|
```python
|
|
from natbridge import smartsend
|
|
|
|
config = {
|
|
"wifi_ssid": "MyNetwork",
|
|
"wifi_password": "password123",
|
|
"update_interval": 60
|
|
}
|
|
|
|
data = [("config", config, "dictionary")]
|
|
env, env_json_str = await smartsend(
|
|
"/device/config",
|
|
data,
|
|
broker_url="nats://localhost:4222"
|
|
)
|
|
```
|
|
|
|
#### MicroPython
|
|
|
|
```python
|
|
from natbridge_mpy import NATSBridge
|
|
|
|
bridge = NATSBridge()
|
|
|
|
config = {
|
|
"wifi_ssid": "MyNetwork",
|
|
"wifi_password": "password123",
|
|
"update_interval": 60
|
|
}
|
|
|
|
data = [("config", config, "dictionary")]
|
|
env, env_json_str = bridge.smartsend(
|
|
"/device/config",
|
|
data,
|
|
size_threshold=100000
|
|
)
|
|
```
|
|
|
|
### Example 2: Sending Binary Data (Image)
|
|
|
|
#### Julia
|
|
|
|
```julia
|
|
using NATSBridge
|
|
|
|
# Read image file
|
|
image_data = read("image.png")
|
|
|
|
data = [("user_image", image_data, "binary")]
|
|
env, env_json_str = smartsend("/chat/image", data, broker_url="nats://localhost:4222")
|
|
```
|
|
|
|
#### JavaScript
|
|
|
|
```javascript
|
|
const NATSBridge = require('./src/natbridge.js');
|
|
const fs = require('fs');
|
|
|
|
// Read image file
|
|
const image_data = fs.readFileSync('image.png');
|
|
|
|
const data = [["user_image", image_data, "binary"]];
|
|
const [env, env_json_str] = await NATSBridge.smartsend(
|
|
"/chat/image",
|
|
data,
|
|
{ broker_url: "nats://localhost:4222" }
|
|
);
|
|
```
|
|
|
|
#### Python
|
|
|
|
```python
|
|
from natbridge import smartsend
|
|
|
|
# Read image file
|
|
with open("image.png", "rb") as f:
|
|
image_data = f.read()
|
|
|
|
data = [("user_image", image_data, "binary")]
|
|
env, env_json_str = await smartsend(
|
|
"/chat/image",
|
|
data,
|
|
broker_url="nats://localhost:4222"
|
|
)
|
|
```
|
|
|
|
#### MicroPython
|
|
|
|
```python
|
|
from natbridge_mpy import NATSBridge
|
|
|
|
bridge = NATSBridge()
|
|
|
|
# Read image file
|
|
with open("image.png", "rb") as f:
|
|
image_data = f.read()
|
|
|
|
data = [("user_image", image_data, "binary")]
|
|
env, env_json_str = bridge.smartsend(
|
|
"/chat/image",
|
|
data,
|
|
size_threshold=100000
|
|
)
|
|
```
|
|
|
|
### Example 3: Request-Response Pattern
|
|
|
|
#### Julia (Requester)
|
|
|
|
```julia
|
|
using NATSBridge
|
|
|
|
# Send command with reply-to
|
|
data = [("command", Dict("action" => "read_sensor"), "dictionary")]
|
|
env, env_json_str = smartsend(
|
|
"/device/command",
|
|
data,
|
|
broker_url="nats://localhost:4222",
|
|
reply_to="/device/response",
|
|
reply_to_msg_id="cmd-001"
|
|
)
|
|
```
|
|
|
|
#### JavaScript (Requester)
|
|
|
|
```javascript
|
|
const NATSBridge = require('./src/natbridge.js');
|
|
|
|
// Send command with reply-to
|
|
const data = [["command", { action: "read_sensor" }, "dictionary"]];
|
|
const [env, env_json_str] = await NATSBridge.smartsend(
|
|
"/device/command",
|
|
data,
|
|
{
|
|
broker_url: "nats://localhost:4222",
|
|
reply_to: "/device/response",
|
|
reply_to_msg_id: "cmd-001"
|
|
}
|
|
);
|
|
```
|
|
|
|
#### Python (Requester)
|
|
|
|
```python
|
|
from natbridge import smartsend
|
|
|
|
# Send command with reply-to
|
|
data = [("command", {"action": "read_sensor"}, "dictionary")]
|
|
env, env_json_str = await smartsend(
|
|
"/device/command",
|
|
data,
|
|
broker_url="nats://localhost:4222",
|
|
reply_to="/device/response",
|
|
reply_to_msg_id="cmd-001"
|
|
)
|
|
```
|
|
|
|
#### Julia (Responder)
|
|
|
|
```julia
|
|
using NATSBridge, NATS
|
|
|
|
const SUBJECT = "/device/command"
|
|
const NATS_URL = "nats://localhost:4222"
|
|
|
|
function test_responder()
|
|
conn = NATS.connect(NATS_URL)
|
|
NATS.subscribe(conn, SUBJECT) do msg
|
|
env = smartreceive(msg, fileserver_download_handler=_fetch_with_backoff)
|
|
|
|
reply_to = env["reply_to"]
|
|
|
|
for (dataname, data, type) in env["payloads"]
|
|
if dataname == "command" && data["action"] == "read_sensor"
|
|
response = Dict("sensor_id" => "sensor-001", "value" => 42.5)
|
|
if !isempty(reply_to)
|
|
smartsend(reply_to, [("data", response, "dictionary")])
|
|
end
|
|
end
|
|
end
|
|
end
|
|
|
|
sleep(120)
|
|
NATS.drain(conn)
|
|
end
|
|
|
|
test_responder()
|
|
```
|
|
|
|
---
|
|
|
|
## Advanced Usage
|
|
|
|
### Example 4: Large Payloads (File Server)
|
|
|
|
For payloads larger than 1MB, NATSBridge automatically uses the file server:
|
|
|
|
#### Julia
|
|
|
|
```julia
|
|
using NATSBridge
|
|
|
|
# Create large data (> 1MB)
|
|
large_data = rand(UInt8, 2_000_000)
|
|
|
|
env, env_json_str = smartsend(
|
|
"/data/large",
|
|
[("large_file", large_data, "binary")],
|
|
broker_url="nats://localhost:4222",
|
|
fileserver_url="http://localhost:8080"
|
|
)
|
|
|
|
println("File uploaded to: $(env.payloads[1].data)")
|
|
```
|
|
|
|
#### JavaScript
|
|
|
|
```javascript
|
|
const NATSBridge = require('./src/natbridge.js');
|
|
|
|
// Create large data (> 1MB)
|
|
const large_data = Buffer.alloc(2_000_000);
|
|
for (let i = 0; i < large_data.length; i++) {
|
|
large_data[i] = Math.floor(Math.random() * 256);
|
|
}
|
|
|
|
const [env, env_json_str] = await NATSBridge.smartsend(
|
|
"/data/large",
|
|
[["large_file", large_data, "binary"]],
|
|
{
|
|
broker_url: "nats://localhost:4222",
|
|
fileserver_url: "http://localhost:8080"
|
|
}
|
|
);
|
|
|
|
console.log("File uploaded to:", env.payloads[0].data);
|
|
```
|
|
|
|
#### Python
|
|
|
|
```python
|
|
from natbridge import smartsend
|
|
|
|
# Create large data (> 1MB)
|
|
import os
|
|
large_data = os.urandom(2_000_000)
|
|
|
|
env, env_json_str = await smartsend(
|
|
"/data/large",
|
|
[("large_file", large_data, "binary")],
|
|
broker_url="nats://localhost:4222",
|
|
fileserver_url="http://localhost:8080"
|
|
)
|
|
|
|
print(f"File uploaded to: {env['payloads'][0]['data']}")
|
|
```
|
|
|
|
#### MicroPython
|
|
|
|
MicroPython enforces a hard limit of 50KB per payload:
|
|
|
|
```python
|
|
from natbridge_mpy import NATSBridge
|
|
|
|
bridge = NATSBridge()
|
|
|
|
# MicroPython has a hard limit of 50KB per payload
|
|
# Use streaming or chunking for larger data
|
|
small_data = bytes(1000) # 1KB
|
|
|
|
data = [("small_file", small_data, "binary")]
|
|
env, env_json_str = bridge.smartsend(
|
|
"/data/small",
|
|
data,
|
|
size_threshold=100000 # Enforced max: 50000 bytes
|
|
)
|
|
```
|
|
|
|
### Example 5: Mixed Content (Chat with Text + Image)
|
|
|
|
NATSBridge supports sending multiple payloads with different types in a single message:
|
|
|
|
#### Julia
|
|
|
|
```julia
|
|
using NATSBridge
|
|
|
|
image_data = read("avatar.png")
|
|
|
|
data = [
|
|
("message_text", "Hello with image!", "text"),
|
|
("user_avatar", image_data, "image")
|
|
]
|
|
|
|
env, env_json_str = smartsend("/chat/mixed", data, broker_url="nats://localhost:4222")
|
|
```
|
|
|
|
#### JavaScript
|
|
|
|
```javascript
|
|
const NATSBridge = require('./src/natbridge.js');
|
|
const fs = require('fs');
|
|
|
|
const image_data = fs.readFileSync('avatar.png');
|
|
|
|
const data = [
|
|
["message_text", "Hello with image!", "text"],
|
|
["user_avatar", image_data, "image"]
|
|
];
|
|
|
|
const [env, env_json_str] = await NATSBridge.smartsend(
|
|
"/chat/mixed",
|
|
data,
|
|
{ broker_url: "nats://localhost:4222" }
|
|
);
|
|
```
|
|
|
|
#### Python
|
|
|
|
```python
|
|
from natbridge import smartsend
|
|
|
|
with open("avatar.png", "rb") as f:
|
|
image_data = f.read()
|
|
|
|
data = [
|
|
("message_text", "Hello with image!", "text"),
|
|
("user_avatar", image_data, "image")
|
|
]
|
|
|
|
env, env_json_str = await smartsend(
|
|
"/chat/mixed",
|
|
data,
|
|
broker_url="nats://localhost:4222"
|
|
)
|
|
```
|
|
|
|
### Example 6: Table Data (Arrow IPC)
|
|
|
|
For tabular data, NATSBridge uses Apache Arrow IPC format:
|
|
|
|
#### Julia
|
|
|
|
```julia
|
|
using NATSBridge
|
|
using DataFrames
|
|
|
|
# Create DataFrame
|
|
df = DataFrame(
|
|
id = [1, 2, 3],
|
|
name = ["Alice", "Bob", "Charlie"],
|
|
score = [95, 88, 92]
|
|
)
|
|
|
|
data = [("students", df, "table")]
|
|
env, env_json_str = smartsend("/data/students", data, broker_url="nats://localhost:4222")
|
|
```
|
|
|
|
#### JavaScript
|
|
|
|
```javascript
|
|
const NATSBridge = require('./src/natbridge.js');
|
|
|
|
// Create table data (array of objects)
|
|
const table_data = [
|
|
{ id: 1, name: "Alice", score: 95 },
|
|
{ id: 2, name: "Bob", score: 88 },
|
|
{ id: 3, name: "Charlie", score: 92 }
|
|
];
|
|
|
|
const data = [["students", table_data, "table"]];
|
|
const [env, env_json_str] = await NATSBridge.smartsend(
|
|
"/data/students",
|
|
data,
|
|
{ broker_url: "nats://localhost:4222" }
|
|
);
|
|
```
|
|
|
|
#### Python
|
|
|
|
```python
|
|
from natbridge import smartsend
|
|
import pandas as pd
|
|
|
|
# Create DataFrame
|
|
df = pd.DataFrame({
|
|
'id': [1, 2, 3],
|
|
'name': ['Alice', 'Bob', 'Charlie'],
|
|
'score': [95, 88, 92]
|
|
})
|
|
|
|
data = [("students", df, "table")]
|
|
env, env_json_str = await smartsend(
|
|
"/data/students",
|
|
data,
|
|
broker_url="nats://localhost:4222"
|
|
)
|
|
```
|
|
|
|
#### MicroPython
|
|
|
|
MicroPython does not support table type due to memory constraints. Use dictionary or binary instead.
|
|
|
|
---
|
|
|
|
## Next Steps
|
|
|
|
1. **Explore the test directory** for more examples
|
|
2. **Check the documentation** for advanced configuration options
|
|
3. **Read the walkthrough** for building real-world applications
|
|
|
|
---
|
|
|
|
## Troubleshooting
|
|
|
|
### Connection Issues
|
|
|
|
- Ensure NATS server is running: `docker ps | grep nats`
|
|
- Check firewall settings
|
|
- Verify NATS URL configuration
|
|
|
|
### File Server Issues
|
|
|
|
- Ensure file server is running and accessible
|
|
- Check upload permissions
|
|
- Verify file server URL configuration
|
|
|
|
### Serialization Errors
|
|
|
|
- Verify data type matches the specified type
|
|
- Check that binary data is in the correct format
|
|
- MicroPython: Ensure payload size < 50KB
|
|
|
|
---
|
|
|
|
## License
|
|
|
|
MIT |