604 lines
12 KiB
Markdown
604 lines
12 KiB
Markdown
# NATSBridge Tutorial
|
|
|
|
A step-by-step guide to get started with NATSBridge - a high-performance, bi-directional data bridge for **Julia**, **JavaScript**, and **Python/Micropython**.
|
|
|
|
## Table of Contents
|
|
|
|
1. [Overview](#overview)
|
|
2. [Prerequisites](#prerequisites)
|
|
3. [Installation](#installation)
|
|
4. [Quick Start](#quick-start)
|
|
5. [Basic Examples](#basic-examples)
|
|
6. [Advanced Usage](#advanced-usage)
|
|
7. [Cross-Platform Communication](#cross-platform-communication)
|
|
|
|
---
|
|
|
|
## Overview
|
|
|
|
NATSBridge enables seamless communication between Julia, JavaScript, and Python/Micropython applications through NATS, with automatic transport selection based on payload size:
|
|
|
|
- **Direct Transport**: Payloads < 1MB are sent directly via NATS (Base64 encoded)
|
|
- **Link Transport**: Payloads >= 1MB are uploaded to an HTTP file server and referenced via URL
|
|
|
|
### Supported Payload Types
|
|
|
|
| Type | Description |
|
|
|------|-------------|
|
|
| `text` | Plain text strings |
|
|
| `dictionary` | JSON-serializable dictionaries |
|
|
| `table` | Tabular data (Arrow IPC format) |
|
|
| `image` | Image data (PNG, JPG bytes) |
|
|
| `audio` | Audio data (WAV, MP3 bytes) |
|
|
| `video` | Video data (MP4, AVI bytes) |
|
|
| `binary` | Generic binary data |
|
|
|
|
---
|
|
|
|
## Prerequisites
|
|
|
|
Before you begin, ensure you have:
|
|
|
|
1. **NATS Server** running (or accessible)
|
|
2. **HTTP File Server** (optional, for large payloads > 1MB)
|
|
3. **One of the supported platforms**: Julia, JavaScript (Node.js), or Python/Micropython
|
|
|
|
---
|
|
|
|
## Installation
|
|
|
|
### Julia
|
|
|
|
```julia
|
|
using Pkg
|
|
Pkg.add("NATS")
|
|
Pkg.add("Arrow")
|
|
Pkg.add("JSON3")
|
|
Pkg.add("HTTP")
|
|
Pkg.add("UUIDs")
|
|
Pkg.add("Dates")
|
|
```
|
|
|
|
### JavaScript
|
|
|
|
```bash
|
|
npm install nats.js apache-arrow uuid base64-url
|
|
```
|
|
|
|
### Python/Micropython
|
|
|
|
1. Copy `src/nats_bridge.py` to your device
|
|
2. Install dependencies:
|
|
|
|
**For Python (desktop):**
|
|
```bash
|
|
pip install nats-py
|
|
```
|
|
|
|
**For Micropython:**
|
|
- `urequests` for HTTP requests
|
|
- `base64` for base64 encoding (built-in)
|
|
- `json` for JSON handling (built-in)
|
|
|
|
---
|
|
|
|
## Quick Start
|
|
|
|
### Step 1: Start NATS Server
|
|
|
|
```bash
|
|
docker run -p 4222:4222 nats:latest
|
|
```
|
|
|
|
### Step 2: Start HTTP File Server (Optional)
|
|
|
|
```bash
|
|
# Create a directory for file uploads
|
|
mkdir -p /tmp/fileserver
|
|
|
|
# Use Python's built-in server
|
|
python3 -m http.server 8080 --directory /tmp/fileserver
|
|
```
|
|
|
|
### Step 3: Send Your First Message
|
|
|
|
#### Python/Micropython
|
|
|
|
```python
|
|
from nats_bridge import smartsend
|
|
|
|
# Send a text message
|
|
data = [("message", "Hello World", "text")]
|
|
env = smartsend("/chat/room1", data, nats_url="nats://localhost:4222")
|
|
print("Message sent!")
|
|
```
|
|
|
|
#### JavaScript
|
|
|
|
```javascript
|
|
const { smartsend } = require('./src/NATSBridge');
|
|
|
|
// Send a text message
|
|
await smartsend("/chat/room1", [
|
|
{ dataname: "message", data: "Hello World", type: "text" }
|
|
], { natsUrl: "nats://localhost:4222" });
|
|
|
|
console.log("Message sent!");
|
|
```
|
|
|
|
#### Julia
|
|
|
|
```julia
|
|
using NATSBridge
|
|
|
|
# Send a text message
|
|
data = [("message", "Hello World", "text")]
|
|
env = smartsend("/chat/room1", data, nats_url="nats://localhost:4222")
|
|
println("Message sent!")
|
|
```
|
|
|
|
### Step 4: Receive Messages
|
|
|
|
#### Python/Micropython
|
|
|
|
```python
|
|
from nats_bridge import smartreceive
|
|
|
|
# Receive and process message
|
|
envelope = smartreceive(msg)
|
|
for dataname, data, type in envelope["payloads"]:
|
|
print(f"Received {dataname}: {data}")
|
|
```
|
|
|
|
#### JavaScript
|
|
|
|
```javascript
|
|
const { smartreceive } = require('./src/NATSBridge');
|
|
|
|
// Receive and process message
|
|
const envelope = await smartreceive(msg);
|
|
for (const payload of envelope.payloads) {
|
|
console.log(`Received ${payload.dataname}: ${payload.data}`);
|
|
}
|
|
```
|
|
|
|
#### Julia
|
|
|
|
```julia
|
|
using NATSBridge
|
|
|
|
# Receive and process message
|
|
envelope = smartreceive(msg, fileserverDownloadHandler)
|
|
for (dataname, data, type) in envelope["payloads"]
|
|
println("Received $dataname: $data")
|
|
end
|
|
```
|
|
|
|
---
|
|
|
|
## Basic Examples
|
|
|
|
### Example 1: Sending a Dictionary
|
|
|
|
#### Python/Micropython
|
|
|
|
```python
|
|
from nats_bridge import smartsend
|
|
|
|
# Create configuration dictionary
|
|
config = {
|
|
"wifi_ssid": "MyNetwork",
|
|
"wifi_password": "password123",
|
|
"update_interval": 60
|
|
}
|
|
|
|
# Send as dictionary type
|
|
data = [("config", config, "dictionary")]
|
|
env = smartsend("/device/config", data, nats_url="nats://localhost:4222")
|
|
```
|
|
|
|
#### JavaScript
|
|
|
|
```javascript
|
|
const { smartsend } = require('./src/NATSBridge');
|
|
|
|
const config = {
|
|
wifi_ssid: "MyNetwork",
|
|
wifi_password: "password123",
|
|
update_interval: 60
|
|
};
|
|
|
|
await smartsend("/device/config", [
|
|
{ dataname: "config", data: config, type: "dictionary" }
|
|
]);
|
|
```
|
|
|
|
#### Julia
|
|
|
|
```julia
|
|
using NATSBridge
|
|
|
|
config = Dict(
|
|
"wifi_ssid" => "MyNetwork",
|
|
"wifi_password" => "password123",
|
|
"update_interval" => 60
|
|
)
|
|
|
|
data = [("config", config, "dictionary")]
|
|
smartsend("/device/config", data)
|
|
```
|
|
|
|
### Example 2: Sending Binary Data (Image)
|
|
|
|
#### Python/Micropython
|
|
|
|
```python
|
|
from nats_bridge import smartsend
|
|
|
|
# Read image file
|
|
with open("image.png", "rb") as f:
|
|
image_data = f.read()
|
|
|
|
# Send as binary type
|
|
data = [("user_image", image_data, "binary")]
|
|
env = smartsend("/chat/image", data, nats_url="nats://localhost:4222")
|
|
```
|
|
|
|
#### JavaScript
|
|
|
|
```javascript
|
|
const { smartsend } = require('./src/NATSBridge');
|
|
|
|
// Read image file (Node.js)
|
|
const fs = require('fs');
|
|
const image_data = fs.readFileSync('image.png');
|
|
|
|
await smartsend("/chat/image", [
|
|
{ dataname: "user_image", data: image_data, type: "binary" }
|
|
]);
|
|
```
|
|
|
|
#### Julia
|
|
|
|
```julia
|
|
using NATSBridge
|
|
|
|
# Read image file
|
|
image_data = read("image.png")
|
|
|
|
data = [("user_image", image_data, "binary")]
|
|
smartsend("/chat/image", data)
|
|
```
|
|
|
|
### Example 3: Request-Response Pattern
|
|
|
|
#### Python/Micropython (Requester)
|
|
|
|
```python
|
|
from nats_bridge import smartsend
|
|
|
|
# Send command with reply-to
|
|
data = [("command", {"action": "read_sensor"}, "dictionary")]
|
|
env = smartsend(
|
|
"/device/command",
|
|
data,
|
|
nats_url="nats://localhost:4222",
|
|
reply_to="/device/response",
|
|
reply_to_msg_id="cmd-001"
|
|
)
|
|
```
|
|
|
|
#### JavaScript (Responder)
|
|
|
|
```javascript
|
|
const { smartreceive, smartsend } = require('./src/NATSBridge');
|
|
|
|
// Subscribe to command topic
|
|
const sub = nc.subscribe("/device/command");
|
|
|
|
for await (const msg of sub) {
|
|
const envelope = await smartreceive(msg);
|
|
|
|
// Process command
|
|
for (const payload of envelope.payloads) {
|
|
if (payload.dataname === "command") {
|
|
const command = payload.data;
|
|
|
|
if (command.action === "read_sensor") {
|
|
// Read sensor and send response
|
|
const response = {
|
|
sensor_id: "sensor-001",
|
|
value: 42.5,
|
|
timestamp: new Date().toISOString()
|
|
};
|
|
|
|
await smartsend("/device/response", [
|
|
{ dataname: "sensor_data", data: response, type: "dictionary" }
|
|
], {
|
|
reply_to: envelope.replyTo,
|
|
reply_to_msg_id: envelope.msgId
|
|
});
|
|
}
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## Advanced Usage
|
|
|
|
### Example 4: Large Payloads (File Server)
|
|
|
|
For payloads larger than 1MB, NATSBridge automatically uses the file server:
|
|
|
|
#### Python/Micropython
|
|
|
|
```python
|
|
from nats_bridge import smartsend
|
|
import os
|
|
|
|
# Create large data (> 1MB)
|
|
large_data = os.urandom(2_000_000) # 2MB of random data
|
|
|
|
# Send with file server URL
|
|
env = smartsend(
|
|
"/data/large",
|
|
[("large_file", large_data, "binary")],
|
|
nats_url="nats://localhost:4222",
|
|
fileserver_url="http://localhost:8080",
|
|
size_threshold=1_000_000
|
|
)
|
|
|
|
# The envelope will contain the download URL
|
|
print(f"File uploaded to: {env.payloads[0].data}")
|
|
```
|
|
|
|
#### JavaScript
|
|
|
|
```javascript
|
|
const { smartsend } = require('./src/NATSBridge');
|
|
|
|
// Create large data (> 1MB)
|
|
const largeData = new ArrayBuffer(2_000_000);
|
|
const view = new Uint8Array(largeData);
|
|
view.fill(42); // Fill with some data
|
|
|
|
await smartsend("/data/large", [
|
|
{ dataname: "large_file", data: largeData, type: "binary" }
|
|
], {
|
|
fileserverUrl: "http://localhost:8080",
|
|
sizeThreshold: 1_000_000
|
|
});
|
|
```
|
|
|
|
#### Julia
|
|
|
|
```julia
|
|
using NATSBridge
|
|
|
|
# Create large data (> 1MB)
|
|
large_data = rand(UInt8, 2_000_000)
|
|
|
|
env = smartsend(
|
|
"/data/large",
|
|
[("large_file", large_data, "binary")],
|
|
fileserver_url="http://localhost:8080"
|
|
)
|
|
|
|
# The envelope will contain the download URL
|
|
println("File uploaded to: $(env.payloads[1].data)")
|
|
```
|
|
|
|
### Example 5: Mixed Content (Chat with Text + Image)
|
|
|
|
NATSBridge supports sending multiple payloads with different types in a single message:
|
|
|
|
#### Python/Micropython
|
|
|
|
```python
|
|
from nats_bridge import smartsend
|
|
|
|
# Read image file
|
|
with open("avatar.png", "rb") as f:
|
|
image_data = f.read()
|
|
|
|
# Send mixed content
|
|
data = [
|
|
("message_text", "Hello with image!", "text"),
|
|
("user_avatar", image_data, "image")
|
|
]
|
|
|
|
env = smartsend("/chat/mixed", data, nats_url="nats://localhost:4222")
|
|
```
|
|
|
|
#### JavaScript
|
|
|
|
```javascript
|
|
const { smartsend } = require('./src/NATSBridge');
|
|
|
|
const fs = require('fs');
|
|
|
|
await smartsend("/chat/mixed", [
|
|
{
|
|
dataname: "message_text",
|
|
data: "Hello with image!",
|
|
type: "text"
|
|
},
|
|
{
|
|
dataname: "user_avatar",
|
|
data: fs.readFileSync("avatar.png"),
|
|
type: "image"
|
|
}
|
|
]);
|
|
```
|
|
|
|
#### Julia
|
|
|
|
```julia
|
|
using NATSBridge
|
|
|
|
image_data = read("avatar.png")
|
|
|
|
data = [
|
|
("message_text", "Hello with image!", "text"),
|
|
("user_avatar", image_data, "image")
|
|
]
|
|
|
|
smartsend("/chat/mixed", data)
|
|
```
|
|
|
|
### Example 6: Table Data (Arrow IPC)
|
|
|
|
For tabular data, NATSBridge uses Apache Arrow IPC format:
|
|
|
|
#### Python/Micropython
|
|
|
|
```python
|
|
from nats_bridge import smartsend
|
|
import pandas as pd
|
|
|
|
# Create DataFrame
|
|
df = pd.DataFrame({
|
|
"id": [1, 2, 3],
|
|
"name": ["Alice", "Bob", "Charlie"],
|
|
"score": [95, 88, 92]
|
|
})
|
|
|
|
# Send as table type
|
|
data = [("students", df, "table")]
|
|
env = smartsend("/data/students", data, nats_url="nats://localhost:4222")
|
|
```
|
|
|
|
#### Julia
|
|
|
|
```julia
|
|
using NATSBridge
|
|
using DataFrames
|
|
|
|
# Create DataFrame
|
|
df = DataFrame(
|
|
id = [1, 2, 3],
|
|
name = ["Alice", "Bob", "Charlie"],
|
|
score = [95, 88, 92]
|
|
)
|
|
|
|
data = [("students", df, "table")]
|
|
smartsend("/data/students", data)
|
|
```
|
|
|
|
---
|
|
|
|
## Cross-Platform Communication
|
|
|
|
NATSBridge enables seamless communication between different platforms:
|
|
|
|
### Julia ↔ JavaScript
|
|
|
|
#### Julia Sender
|
|
|
|
```julia
|
|
using NATSBridge
|
|
|
|
# Send dictionary from Julia to JavaScript
|
|
config = Dict("step_size" => 0.01, "iterations" => 1000)
|
|
data = [("config", config, "dictionary")]
|
|
smartsend("/analysis/config", data, nats_url="nats://localhost:4222")
|
|
```
|
|
|
|
#### JavaScript Receiver
|
|
|
|
```javascript
|
|
const { smartreceive } = require('./src/NATSBridge');
|
|
|
|
// Receive dictionary from Julia
|
|
const envelope = await smartreceive(msg);
|
|
for (const payload of envelope.payloads) {
|
|
if (payload.type === "dictionary") {
|
|
console.log("Received config:", payload.data);
|
|
// payload.data = { step_size: 0.01, iterations: 1000 }
|
|
}
|
|
}
|
|
```
|
|
|
|
### JavaScript ↔ Python
|
|
|
|
#### JavaScript Sender
|
|
|
|
```javascript
|
|
const { smartsend } = require('./src/NATSBridge');
|
|
|
|
await smartsend("/data/transfer", [
|
|
{ dataname: "message", data: "Hello from JS!", type: "text" }
|
|
]);
|
|
```
|
|
|
|
#### Python Receiver
|
|
|
|
```python
|
|
from nats_bridge import smartreceive
|
|
|
|
envelope = smartreceive(msg)
|
|
for dataname, data, type in envelope["payloads"]:
|
|
if type == "text":
|
|
print(f"Received from JS: {data}")
|
|
```
|
|
|
|
### Python ↔ Julia
|
|
|
|
#### Python Sender
|
|
|
|
```python
|
|
from nats_bridge import smartsend
|
|
|
|
data = [("message", "Hello from Python!", "text")]
|
|
smartsend("/chat/python", data)
|
|
```
|
|
|
|
#### Julia Receiver
|
|
|
|
```julia
|
|
using NATSBridge
|
|
|
|
envelope = smartreceive(msg, fileserverDownloadHandler)
|
|
for (dataname, data, type) in envelope["payloads"]
|
|
if type == "text"
|
|
println("Received from Python: $data")
|
|
end
|
|
end
|
|
```
|
|
|
|
---
|
|
|
|
## Next Steps
|
|
|
|
1. **Explore the test directory** for more examples
|
|
2. **Check the documentation** for advanced configuration options
|
|
3. **Join the community** to share your use cases
|
|
|
|
---
|
|
|
|
## Troubleshooting
|
|
|
|
### Connection Issues
|
|
|
|
- Ensure NATS server is running: `docker ps | grep nats`
|
|
- Check firewall settings
|
|
- Verify NATS URL configuration
|
|
|
|
### File Server Issues
|
|
|
|
- Ensure file server is running and accessible
|
|
- Check upload permissions
|
|
- Verify file server URL configuration
|
|
|
|
### Serialization Errors
|
|
|
|
- Verify data type matches the specified type
|
|
- Check that binary data is in the correct format (bytes/Vector{UInt8})
|
|
|
|
---
|
|
|
|
## License
|
|
|
|
MIT |