update readme
This commit is contained in:
228
README.md
228
README.md
@@ -1,6 +1,6 @@
|
||||
# NATSBridge - Cross-Platform Bi-Directional Data Bridge
|
||||
|
||||
A high-performance, bi-directional data bridge for **Julia, JavaScript, Python, and MicroPython** applications using NATS (Core & JetStream), implementing the Claim-Check pattern for large payloads.
|
||||
A high-performance, bi-directional data bridge for **Julia**, **JavaScript**, **Python**, and **MicroPython** applications using NATS (Core & JetStream), implementing the Claim-Check pattern for large payloads.
|
||||
|
||||
[](https://opensource.org/licenses/MIT)
|
||||
[](https://nats.io)
|
||||
@@ -28,8 +28,8 @@ NATSBridge enables seamless communication across multiple platforms through NATS
|
||||
|
||||
| Transport | Payload Size | Method |
|
||||
|-----------|--------------|--------|
|
||||
| **Direct** | < 1MB | Sent directly via NATS (Base64 encoded) |
|
||||
| **Link** | >= 1MB | Uploaded to HTTP file server, URL sent via NATS |
|
||||
| **Direct** | < 500KB | Sent directly via NATS (Base64 encoded) |
|
||||
| **Link** | ≥ 500KB | Uploaded to HTTP file server, URL sent via NATS |
|
||||
|
||||
### Use Cases
|
||||
|
||||
@@ -45,7 +45,7 @@ NATSBridge enables seamless communication across multiple platforms through NATS
|
||||
| Platform | Implementation | Features |
|
||||
|----------|----------------|----------|
|
||||
| **Julia** | [`src/NATSBridge.jl`](src/NATSBridge.jl) | Full feature set, Arrow IPC, multiple dispatch |
|
||||
| **JavaScript** | [`src/natsbridge.js`](src/natsbridge.js) | Node.js, async/await |
|
||||
| **JavaScript (Node.js)** | [`src/natsbridge.js`](src/natsbridge_ssr.js) | Node.js, async/await |
|
||||
| **JavaScript (Browser)** | [`src/natsbridge_csr.js`](src/natsbridge_csr.js) | Browser, WebSocket NATS, async/await |
|
||||
| **Python** | [`src/natsbridge.py`](src/natsbridge.py) | Desktop Python, asyncio, type hints |
|
||||
| **MicroPython** | [`src/natsbridge_mpy.py`](src/natsbridge_mpy.py) | Memory-constrained, synchronous API |
|
||||
@@ -72,7 +72,7 @@ NATSBridge enables seamless communication across multiple platforms through NATS
|
||||
- ✅ **Bi-directional messaging** with request-reply patterns
|
||||
- ✅ **Multi-payload support** - send multiple payloads with different types in one message
|
||||
- ✅ **Automatic transport selection** - direct vs link based on payload size
|
||||
- ✅ **Claim-Check pattern** for payloads > 1MB
|
||||
- ✅ **Claim-Check pattern** for payloads ≥ 500KB
|
||||
- ✅ **Apache Arrow IPC** support for tabular data (zero-copy reading)
|
||||
- ✅ **Exponential backoff** for reliable file server downloads
|
||||
- ✅ **Correlation ID tracking** for message tracing
|
||||
@@ -83,23 +83,24 @@ NATSBridge enables seamless communication across multiple platforms through NATS
|
||||
|
||||
## Quick Start
|
||||
|
||||
### Step 1: Start NATS Server
|
||||
### Prerequisites
|
||||
|
||||
```bash
|
||||
docker run -p 4222:4222 nats:latest
|
||||
```
|
||||
1. **NATS Server** - Install and run a NATS server:
|
||||
```bash
|
||||
docker run -p 4222:4222 nats:latest
|
||||
```
|
||||
|
||||
### Step 2: Start HTTP File Server (Optional)
|
||||
2. **HTTP File Server** (optional, for large payloads) - Install and run a file server:
|
||||
```bash
|
||||
# Using Plik
|
||||
docker run -p 8080:8080 -v /tmp/fileserver:/var/lib/plik -e PLIK_ADMIN_PASSWORD=admin plik/plik
|
||||
|
||||
# OR using simple Python HTTP server
|
||||
mkdir -p /tmp/fileserver
|
||||
python3 -m http.server 8080 --directory /tmp/fileserver
|
||||
```
|
||||
|
||||
```bash
|
||||
# Create a directory for file uploads
|
||||
mkdir -p /tmp/fileserver
|
||||
|
||||
# Start HTTP file server
|
||||
python3 -m http.server 8080 --directory /tmp/fileserver
|
||||
```
|
||||
|
||||
### Step 3: Send Your First Message
|
||||
### Send Your First Message
|
||||
|
||||
#### Julia
|
||||
|
||||
@@ -107,14 +108,14 @@ python3 -m http.server 8080 --directory /tmp/fileserver
|
||||
using NATSBridge
|
||||
|
||||
data = [("message", "Hello World", "text")]
|
||||
env, env_json_str = smartsend("/chat/room1", data, broker_url="nats://localhost:4222")
|
||||
env, env_json_str = smartsend("/chat/room1", data; broker_url="nats://localhost:4222")
|
||||
println("Message sent!")
|
||||
```
|
||||
|
||||
#### JavaScript
|
||||
#### JavaScript (Node.js)
|
||||
|
||||
```javascript
|
||||
const NATSBridge = require('./src/natsbridge.js');
|
||||
import NATSBridge from './src/natsbridge_ssr.js';
|
||||
|
||||
const data = [["message", "Hello World", "text"]];
|
||||
const [env, env_json_str] = await NATSBridge.smartsend(
|
||||
@@ -125,6 +126,20 @@ const [env, env_json_str] = await NATSBridge.smartsend(
|
||||
console.log("Message sent!");
|
||||
```
|
||||
|
||||
#### JavaScript (Browser)
|
||||
|
||||
```javascript
|
||||
import NATSBridge from './src/natsbridge_csr.js';
|
||||
|
||||
const data = [["message", "Hello World", "text"]];
|
||||
const [env, env_json_str] = await NATSBridge.smartsend(
|
||||
"/chat/room1",
|
||||
data,
|
||||
{ broker_url: "ws://localhost:4222" }
|
||||
);
|
||||
console.log("Message sent!");
|
||||
```
|
||||
|
||||
#### Python
|
||||
|
||||
```python
|
||||
@@ -139,6 +154,21 @@ env, env_json_str = await smartsend(
|
||||
print("Message sent!")
|
||||
```
|
||||
|
||||
#### MicroPython
|
||||
|
||||
```python
|
||||
from natsbridge import smartsend
|
||||
|
||||
data = [("message", "Hello World", "text")]
|
||||
env, env_json_str = smartsend(
|
||||
"/chat/room1",
|
||||
data,
|
||||
broker_url="nats://localhost:4222",
|
||||
size_threshold=100000 # 100KB for MicroPython
|
||||
)
|
||||
print("Message sent!")
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## API Reference
|
||||
@@ -147,13 +177,13 @@ print("Message sent!")
|
||||
|
||||
All platforms use the same input/output format for payloads:
|
||||
|
||||
**Input format for smartsend:**
|
||||
**Input format for `smartsend`:**
|
||||
```
|
||||
[(dataname1, data1, type1), (dataname2, data2, type2), ...]
|
||||
```
|
||||
|
||||
**Output format for smartreceive:**
|
||||
```
|
||||
**Output format for `smartreceive`:**
|
||||
```json
|
||||
{
|
||||
"correlation_id": "...",
|
||||
"msg_id": "...",
|
||||
@@ -187,7 +217,7 @@ env, env_json_str = NATSBridge.smartsend(
|
||||
broker_url::String = "nats://localhost:4222",
|
||||
fileserver_url = "http://localhost:8080",
|
||||
fileserver_upload_handler::Function = plik_oneshot_upload,
|
||||
size_threshold::Int = 1_000_000,
|
||||
size_threshold::Int = 500_000,
|
||||
correlation_id::String = string(uuid4()),
|
||||
msg_purpose::String = "chat",
|
||||
sender_name::String = "NATSBridge",
|
||||
@@ -203,10 +233,10 @@ env, env_json_str = NATSBridge.smartsend(
|
||||
# Returns: ::Tuple{msg_envelope_v1, String}
|
||||
```
|
||||
|
||||
#### JavaScript
|
||||
#### JavaScript (Node.js)
|
||||
|
||||
```javascript
|
||||
const NATSBridge = require('natsbridge');
|
||||
import NATSBridge from './src/natsbridge_ssr.js';
|
||||
|
||||
const [env, env_json_str] = await NATSBridge.smartsend(
|
||||
subject,
|
||||
@@ -215,7 +245,36 @@ const [env, env_json_str] = await NATSBridge.smartsend(
|
||||
broker_url: 'nats://localhost:4222',
|
||||
fileserver_url: 'http://localhost:8080',
|
||||
fileserver_upload_handler: NATSBridge.plikOneshotUpload,
|
||||
size_threshold: 1_000_000,
|
||||
size_threshold: 500_000,
|
||||
correlation_id: uuidv4(),
|
||||
msg_purpose: 'chat',
|
||||
sender_name: 'NATSBridge',
|
||||
receiver_name: '',
|
||||
receiver_id: '',
|
||||
reply_to: '',
|
||||
reply_to_msg_id: '',
|
||||
is_publish: true,
|
||||
nats_connection: null,
|
||||
msg_id: uuidv4(),
|
||||
sender_id: uuidv4()
|
||||
}
|
||||
);
|
||||
// Returns: Promise<[env, env_json_str]>
|
||||
```
|
||||
|
||||
#### JavaScript (Browser)
|
||||
|
||||
```javascript
|
||||
import NATSBridge from './src/natsbridge_csr.js';
|
||||
|
||||
const [env, env_json_str] = await NATSBridge.smartsend(
|
||||
subject,
|
||||
data,
|
||||
{
|
||||
broker_url: 'ws://localhost:4222',
|
||||
fileserver_url: 'http://localhost:8080',
|
||||
fileserver_upload_handler: NATSBridge.plikOneshotUpload,
|
||||
size_threshold: 500_000,
|
||||
correlation_id: uuidv4(),
|
||||
msg_purpose: 'chat',
|
||||
sender_name: 'NATSBridge',
|
||||
@@ -243,7 +302,7 @@ env, env_json_str = await NATSBridge.smartsend(
|
||||
broker_url: str = "nats://localhost:4222",
|
||||
fileserver_url: str = "http://localhost:8080",
|
||||
fileserver_upload_handler: Callable = plik_oneshot_upload,
|
||||
size_threshold: int = 1_000_000,
|
||||
size_threshold: int = 500_000,
|
||||
correlation_id: str = None,
|
||||
msg_purpose: str = "chat",
|
||||
sender_name: str = "NATSBridge",
|
||||
@@ -293,9 +352,28 @@ env = NATSBridge.smartreceive(
|
||||
# Returns: ::JSON.Object{String, Any}
|
||||
```
|
||||
|
||||
#### JavaScript
|
||||
#### JavaScript (Node.js)
|
||||
|
||||
```javascript
|
||||
import NATSBridge from './src/natsbridge_ssr.js';
|
||||
|
||||
const env = await NATSBridge.smartreceive(
|
||||
msg,
|
||||
{
|
||||
fileserver_download_handler: NATSBridge.fetchWithBackoff,
|
||||
max_retries: 5,
|
||||
base_delay: 100,
|
||||
max_delay: 5000
|
||||
}
|
||||
);
|
||||
// Returns: Promise<env_object>
|
||||
```
|
||||
|
||||
#### JavaScript (Browser)
|
||||
|
||||
```javascript
|
||||
import NATSBridge from './src/natsbridge_csr.js';
|
||||
|
||||
const env = await NATSBridge.smartreceive(
|
||||
msg,
|
||||
{
|
||||
@@ -311,6 +389,8 @@ const env = await NATSBridge.smartreceive(
|
||||
#### Python
|
||||
|
||||
```python
|
||||
from natsbridge import NATSBridge
|
||||
|
||||
env = await NATSBridge.smartreceive(
|
||||
msg,
|
||||
fileserver_download_handler=fetch_with_backoff,
|
||||
@@ -324,6 +404,8 @@ env = await NATSBridge.smartreceive(
|
||||
#### MicroPython
|
||||
|
||||
```python
|
||||
from natsbridge import NATSBridge
|
||||
|
||||
env = NATSBridge.smartreceive(
|
||||
msg,
|
||||
fileserver_download_handler=_sync_fileserver_download,
|
||||
@@ -343,7 +425,7 @@ env = NATSBridge.smartreceive(
|
||||
| `text` | `String` | `string` | `str` | `str` | Plain text strings |
|
||||
| `dictionary` | `Dict`, `NamedTuple` | `Object`, `Array` | `dict`, `list` | `dict` | JSON-serializable dictionaries |
|
||||
| `arrowtable` | `DataFrame`, `Arrow.Table` | `Array<Object>` | `pandas.DataFrame` | ❌ | Tabular data (Arrow IPC) |
|
||||
| `jsontable` | `Vector{NamedTuple}` | `Array<Object>` | `list[dict]` | ❌ | Tabular data (JSON) |
|
||||
| `jsontable` | `Vector{NamedTuple}` | `Array<Object>` | `list[dict]` | ⚠️ | Tabular data (JSON) |
|
||||
| `image` | `Vector{UInt8}` | `Uint8Array`, `Buffer` | `bytes` | `bytearray` | Image data (PNG, JPG) |
|
||||
| `audio` | `Vector{UInt8}` | `Uint8Array`, `Buffer` | `bytes` | `bytearray` | Audio data (WAV, MP3) |
|
||||
| `video` | `Vector{UInt8}` | `Uint8Array`, `Buffer` | `bytes` | `bytearray` | Video data (MP4, AVI) |
|
||||
@@ -368,13 +450,13 @@ data = [
|
||||
("large_document", large_file_data, "binary")
|
||||
]
|
||||
|
||||
env, env_json_str = NATSBridge.smartsend("/chat/room1", data; fileserver_url="http://localhost:8080")
|
||||
env, env_json_str = smartsend("/chat/room1", data; fileserver_url="http://localhost:8080")
|
||||
```
|
||||
|
||||
#### JavaScript
|
||||
#### JavaScript (Node.js)
|
||||
|
||||
```javascript
|
||||
const NATSBridge = require('natsbridge');
|
||||
import NATSBridge from './src/natsbridge_ssr.js';
|
||||
|
||||
const data = [
|
||||
["message_text", "Hello!", "text"],
|
||||
@@ -389,6 +471,24 @@ const [env, env_json_str] = await NATSBridge.smartsend(
|
||||
);
|
||||
```
|
||||
|
||||
#### JavaScript (Browser)
|
||||
|
||||
```javascript
|
||||
import NATSBridge from './src/natsbridge_csr.js';
|
||||
|
||||
const data = [
|
||||
["message_text", "Hello!", "text"],
|
||||
["user_avatar", imageData, "image"],
|
||||
["large_document", largeFileData, "binary"]
|
||||
];
|
||||
|
||||
const [env, env_json_str] = await NATSBridge.smartsend(
|
||||
"/chat/room1",
|
||||
data,
|
||||
{ broker_url: 'ws://localhost:4222', fileserver_url: 'http://localhost:8080' }
|
||||
);
|
||||
```
|
||||
|
||||
#### Python
|
||||
|
||||
```python
|
||||
@@ -423,13 +523,13 @@ config = Dict(
|
||||
)
|
||||
|
||||
data = [("config", config, "dictionary")]
|
||||
env, env_json_str = NATSBridge.smartsend("/device/config", data)
|
||||
env, env_json_str = smartsend("/device/config", data)
|
||||
```
|
||||
|
||||
#### JavaScript
|
||||
#### JavaScript (Node.js)
|
||||
|
||||
```javascript
|
||||
const NATSBridge = require('natsbridge');
|
||||
import NATSBridge from './src/natsbridge_ssr.js';
|
||||
|
||||
const config = {
|
||||
wifi_ssid: "MyNetwork",
|
||||
@@ -475,13 +575,13 @@ df = DataFrame(
|
||||
)
|
||||
|
||||
data = [("students", df, "arrowtable")]
|
||||
env, env_json_str = NATSBridge.smartsend("/data/analysis", data)
|
||||
env, env_json_str = smartsend("/data/analysis", data)
|
||||
```
|
||||
|
||||
#### JavaScript
|
||||
#### JavaScript (Node.js)
|
||||
|
||||
```javascript
|
||||
const NATSBridge = require('natsbridge');
|
||||
import NATSBridge from './src/natsbridge_ssr.js';
|
||||
|
||||
const df = [
|
||||
{ id: 1, name: "Alice", score: 95 },
|
||||
@@ -521,18 +621,29 @@ Bi-directional communication with reply-to support.
|
||||
using NATSBridge
|
||||
|
||||
# Requester
|
||||
env, env_json_str = NATSBridge.smartsend(
|
||||
env, env_json_str = smartsend(
|
||||
"/device/command",
|
||||
[("command", Dict("action" => "read_sensor"), "dictionary")];
|
||||
broker_url="nats://localhost:4222",
|
||||
reply_to="/device/response"
|
||||
)
|
||||
|
||||
# Receiver (in separate application)
|
||||
msg = NATS.subscription.next()
|
||||
env = smartreceive(msg)
|
||||
# Process request and send response
|
||||
response_env, response_json = smartsend(
|
||||
"/device/response",
|
||||
[("result", Dict("value" => 42), "dictionary")],
|
||||
reply_to="/device/command",
|
||||
reply_to_msg_id=env["msg_id"]
|
||||
)
|
||||
```
|
||||
|
||||
#### JavaScript
|
||||
#### JavaScript (Node.js)
|
||||
|
||||
```javascript
|
||||
const NATSBridge = require('natsbridge');
|
||||
import NATSBridge from './src/natsbridge_ssr.js';
|
||||
|
||||
// Requester
|
||||
const [env, env_json_str] = await NATSBridge.smartsend(
|
||||
@@ -540,6 +651,16 @@ const [env, env_json_str] = await NATSBridge.smartsend(
|
||||
[["command", { action: "read_sensor" }, "dictionary"]],
|
||||
{ broker_url: 'nats://localhost:4222', reply_to: '/device/response' }
|
||||
);
|
||||
|
||||
// Receiver (in separate application)
|
||||
// const msg = await natsConsumer.next();
|
||||
// const env = await NATSBridge.smartreceive(msg);
|
||||
// Process request and send response
|
||||
// const response_env, response_json = await NATSBridge.smartsend(
|
||||
// "/device/response",
|
||||
// [["result", { value: 42 }, "dictionary"]],
|
||||
// { reply_to: '/device/command', reply_to_msg_id: env.msg_id }
|
||||
// );
|
||||
```
|
||||
|
||||
#### Python
|
||||
@@ -554,6 +675,17 @@ env, env_json_str = await NATSBridge.smartsend(
|
||||
broker_url="nats://localhost:4222",
|
||||
reply_to="/device/response"
|
||||
)
|
||||
|
||||
# Receiver (in separate application)
|
||||
# msg = await nats_consumer.next()
|
||||
# env = await NATSBridge.smartreceive(msg)
|
||||
# Process request and send response
|
||||
# response_env, response_json = await NATSBridge.smartsend(
|
||||
# "/device/response",
|
||||
# [("result", {"value": 42}, "dictionary")],
|
||||
# reply_to="/device/command",
|
||||
# reply_to_msg_id=env["msg_id"]
|
||||
# )
|
||||
```
|
||||
|
||||
---
|
||||
@@ -640,10 +772,10 @@ python3 test/test_py_table_receiver.py
|
||||
|
||||
For detailed architecture and implementation information, see:
|
||||
|
||||
- [Architecture Documentation](docs/architecture_updated.md) - Cross-platform architecture, API parity, platform-specific patterns
|
||||
- [Implementation Guide](docs/implementation_updated.md) - Detailed implementation for each platform, handler functions, testing
|
||||
- [Tutorial](docs/tutorial_updated.md) - Step-by-step getting started guide
|
||||
- [Walkthrough](docs/walkthrough_updated.md) - Real-world application building guides
|
||||
- [`docs/architecture.md`](docs/architecture.md) - Cross-platform architecture, API parity, platform-specific patterns
|
||||
- [`docs/requirements.md`](docs/requirements.md) - Business requirements and user stories
|
||||
- [`docs/spec.md`](docs/spec.md) - Technical specification and contracts
|
||||
- [`docs/walkthrough.md`](docs/walkthrough.md) - Real-world application building guides
|
||||
|
||||
---
|
||||
|
||||
|
||||
741
docs/tutorial.md
741
docs/tutorial.md
@@ -1,741 +0,0 @@
|
||||
# Cross-Platform NATSBridge Tutorial
|
||||
|
||||
A step-by-step guide to get started with NATSBridge across **Julia**, **JavaScript**, and **Python/MicroPython**.
|
||||
|
||||
## Table of Contents
|
||||
|
||||
1. [Overview](#overview)
|
||||
2. [Prerequisites](#prerequisites)
|
||||
3. [Installation](#installation)
|
||||
4. [Quick Start](#quick-start)
|
||||
5. [Basic Examples](#basic-examples)
|
||||
6. [Advanced Usage](#advanced-usage)
|
||||
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
NATSBridge enables seamless communication across platforms through NATS, with automatic transport selection based on payload size:
|
||||
|
||||
- **Direct Transport**: Payloads < 1MB are sent directly via NATS (Base64 encoded)
|
||||
- **Link Transport**: Payloads >= 1MB are uploaded to an HTTP file server and referenced via URL
|
||||
|
||||
### Cross-Platform API Parity
|
||||
|
||||
All three platforms use the same high-level API:
|
||||
|
||||
```
|
||||
# Input format
|
||||
smartsend(subject, [(dataname, data, type), ...], options)
|
||||
|
||||
# Output format
|
||||
(env, env_json_str) = smartsend(...)
|
||||
env = smartreceive(msg, options)
|
||||
```
|
||||
|
||||
**Important Platform Differences:**
|
||||
|
||||
1. **Encoding field:** Julia and JavaScript preserve the original serialization format in the encoding field (`"base64"`, `"json"`, or `"arrow-ipc"`), while Python and MicroPython always use `"base64"` for all direct transport payloads.
|
||||
|
||||
2. **Async vs Sync:** JavaScript and Python desktop use async/await, while MicroPython uses synchronous API.
|
||||
|
||||
### Supported Payload Types
|
||||
|
||||
| Type | Julia | JavaScript | Python | MicroPython |
|
||||
|------|-------|------------|--------|-------------|
|
||||
| `text` | `String` | `string` | `str` | `str` |
|
||||
| `dictionary` | `Dict` | `Object` | `dict` | `dict` |
|
||||
| `arrowtable` | `DataFrame` | `Array<Object>` | `pandas.DataFrame` | ❌ |
|
||||
| `jsontable` | `Vector{NamedTuple}` | `Array<Object>` | `list[dict]` | ❌ |
|
||||
| `table` | ❌ | ❌ | `pandas.DataFrame` | ❌ |
|
||||
| `image` | `Vector{UInt8}` | `Uint8Array` | `bytes` | `bytearray` |
|
||||
| `audio` | `Vector{UInt8}` | `Uint8Array` | `bytes` | `bytearray` |
|
||||
| `video` | `Vector{UInt8}` | `Uint8Array` | `bytes` | `bytearray` |
|
||||
| `binary` | `Vector{UInt8}` | `Uint8Array` | `bytes` | `bytearray` |
|
||||
|
||||
**Note on MicroPython:** MicroPython does not support table types (`arrowtable`, `jsontable`, or `table`) due to memory constraints. Use `dictionary` or `binary` instead.
|
||||
|
||||
---
|
||||
|
||||
## Prerequisites
|
||||
|
||||
Before you begin, ensure you have:
|
||||
|
||||
1. **NATS Server** running (or accessible)
|
||||
2. **HTTP File Server** (optional, for large payloads > 1MB)
|
||||
3. **Platform-specific packages** installed
|
||||
|
||||
---
|
||||
|
||||
## Installation
|
||||
|
||||
### Julia
|
||||
|
||||
```julia
|
||||
using Pkg
|
||||
Pkg.add("NATS")
|
||||
Pkg.add("Arrow")
|
||||
Pkg.add("JSON3")
|
||||
Pkg.add("HTTP")
|
||||
Pkg.add("UUIDs")
|
||||
Pkg.add("Dates")
|
||||
```
|
||||
|
||||
### JavaScript (Node.js)
|
||||
|
||||
```bash
|
||||
npm install nats uuid apache-arrow node-fetch
|
||||
```
|
||||
|
||||
### JavaScript (Browser)
|
||||
|
||||
```html
|
||||
<script src="https://unpkg.com/nats-js/dist/bundle/nats.min.js"></script>
|
||||
<script src="https://unpkg.com/apache-arrow/arrow.min.js"></script>
|
||||
```
|
||||
|
||||
### Python (Desktop)
|
||||
|
||||
```bash
|
||||
pip install nats-py aiohttp pyarrow pandas
|
||||
```
|
||||
|
||||
### MicroPython
|
||||
|
||||
Uses built-in modules: `network`, `socket`, `time`, `json`, `base64`
|
||||
|
||||
---
|
||||
|
||||
## Quick Start
|
||||
|
||||
### Step 1: Start NATS Server
|
||||
|
||||
```bash
|
||||
docker run -p 4222:4222 nats:latest
|
||||
```
|
||||
|
||||
### Step 2: Start HTTP File Server (Optional)
|
||||
|
||||
```bash
|
||||
mkdir -p /tmp/fileserver
|
||||
python3 -m http.server 8080 --directory /tmp/fileserver
|
||||
```
|
||||
|
||||
### Step 3: Send Your First Message
|
||||
|
||||
#### Julia
|
||||
|
||||
```julia
|
||||
using NATSBridge
|
||||
|
||||
# Send a text message
|
||||
data = [("message", "Hello World", "text")]
|
||||
env, env_json_str = smartsend("/chat/room1", data, broker_url="nats://localhost:4222")
|
||||
# env: msg_envelope_v1 struct with all metadata and payloads
|
||||
# env_json_str: JSON string representation of the envelope for publishing
|
||||
println("Message sent!")
|
||||
|
||||
# Or use is_publish=false to get envelope and JSON without publishing
|
||||
env, env_json_str = smartsend("/chat/room1", data, broker_url="nats://localhost:4222", is_publish=false)
|
||||
# env: msg_envelope_v1 struct
|
||||
# env_json_str: JSON string for publishing to NATS
|
||||
```
|
||||
|
||||
#### JavaScript
|
||||
|
||||
```javascript
|
||||
const NATSBridge = require('./src/natsbridge.js');
|
||||
|
||||
// Send a text message
|
||||
const data = [["message", "Hello World", "text"]];
|
||||
const [env, env_json_str] = await NATSBridge.smartsend(
|
||||
"/chat/room1",
|
||||
data,
|
||||
{ broker_url: "nats://localhost:4222" }
|
||||
);
|
||||
// env: Object with all metadata and payloads
|
||||
// env_json_str: JSON string for publishing
|
||||
console.log("Message sent!");
|
||||
|
||||
// Or use is_publish=false
|
||||
const [env, env_json_str] = await NATSBridge.smartsend(
|
||||
"/chat/room1",
|
||||
data,
|
||||
{ broker_url: "nats://localhost:4222", is_publish: false }
|
||||
);
|
||||
```
|
||||
|
||||
#### Python
|
||||
|
||||
```python
|
||||
from natsbridge import smartsend
|
||||
|
||||
# Send a text message
|
||||
data = [("message", "Hello World", "text")]
|
||||
env, env_json_str = await smartsend(
|
||||
"/chat/room1",
|
||||
data,
|
||||
broker_url="nats://localhost:4222"
|
||||
)
|
||||
# env: Dict with all metadata and payloads
|
||||
# env_json_str: JSON string for publishing
|
||||
print("Message sent!")
|
||||
|
||||
# Or use is_publish=False
|
||||
env, env_json_str = await smartsend(
|
||||
"/chat/room1",
|
||||
data,
|
||||
broker_url="nats://localhost:4222",
|
||||
is_publish=False
|
||||
)
|
||||
# env: Dict with all metadata and payloads
|
||||
# env_json_str: JSON string for publishing to NATS
|
||||
```
|
||||
|
||||
#### MicroPython
|
||||
|
||||
```python
|
||||
from natsbridge_mpy import NATSBridge
|
||||
|
||||
bridge = NATSBridge()
|
||||
|
||||
# Send a text message (limited to small payloads)
|
||||
data = [("message", "Hello World", "text")]
|
||||
env, env_json_str = bridge.smartsend(
|
||||
"/chat/room1",
|
||||
data,
|
||||
size_threshold=100000 # Lower threshold for MicroPython
|
||||
)
|
||||
print("Message sent!")
|
||||
```
|
||||
|
||||
### Step 4: Receive Messages
|
||||
|
||||
#### Julia
|
||||
|
||||
```julia
|
||||
using NATSBridge
|
||||
|
||||
# Receive and process message
|
||||
env = smartreceive(msg; fileserver_download_handler=_fetch_with_backoff)
|
||||
# Returns: ::JSON.Object{String, Any} with "payloads" field containing Vector{Tuple{String, Any, String}}
|
||||
# Access payloads: for (dataname, data, type) in env["payloads"]
|
||||
for (dataname, data, type) in env["payloads"]
|
||||
println("Received $dataname: $data")
|
||||
end
|
||||
```
|
||||
|
||||
#### JavaScript
|
||||
|
||||
```javascript
|
||||
const NATSBridge = require('./src/natsbridge.js');
|
||||
|
||||
// Receive and process message
|
||||
const env = await NATSBridge.smartreceive(msg, {
|
||||
fileserver_download_handler: NATSBridge.fetchWithBackoff
|
||||
});
|
||||
// env.payloads = [[dataname, data, type], ...]
|
||||
for (const [dataname, data, type] of env.payloads) {
|
||||
console.log(`Received ${dataname}:`, data);
|
||||
}
|
||||
```
|
||||
|
||||
#### Python
|
||||
|
||||
```python
|
||||
from natsbridge import smartreceive, fetch_with_backoff
|
||||
|
||||
# Receive and process message
|
||||
env = await smartreceive(
|
||||
msg,
|
||||
fileserver_download_handler=fetch_with_backoff
|
||||
)
|
||||
# env["payloads"] = [(dataname, data, type), ...]
|
||||
for dataname, data, type_ in env["payloads"]:
|
||||
print(f"Received {dataname}: {data}")
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Basic Examples
|
||||
|
||||
### Example 1: Sending a Dictionary
|
||||
|
||||
#### Julia
|
||||
|
||||
```julia
|
||||
using NATSBridge
|
||||
|
||||
config = Dict(
|
||||
"wifi_ssid" => "MyNetwork",
|
||||
"wifi_password" => "password123",
|
||||
"update_interval" => 60
|
||||
)
|
||||
|
||||
data = [("config", config, "dictionary")]
|
||||
env, env_json_str = smartsend("/device/config", data, broker_url="nats://localhost:4222")
|
||||
```
|
||||
|
||||
#### JavaScript
|
||||
|
||||
```javascript
|
||||
const NATSBridge = require('./src/natsbridge.js');
|
||||
|
||||
const config = {
|
||||
wifi_ssid: "MyNetwork",
|
||||
wifi_password: "password123",
|
||||
update_interval: 60
|
||||
};
|
||||
|
||||
const data = [["config", config, "dictionary"]];
|
||||
const [env, env_json_str] = await NATSBridge.smartsend(
|
||||
"/device/config",
|
||||
data,
|
||||
{ broker_url: "nats://localhost:4222" }
|
||||
);
|
||||
```
|
||||
|
||||
#### Python
|
||||
|
||||
```python
|
||||
from natsbridge import smartsend
|
||||
|
||||
config = {
|
||||
"wifi_ssid": "MyNetwork",
|
||||
"wifi_password": "password123",
|
||||
"update_interval": 60
|
||||
}
|
||||
|
||||
data = [("config", config, "dictionary")]
|
||||
env, env_json_str = await smartsend(
|
||||
"/device/config",
|
||||
data,
|
||||
broker_url="nats://localhost:4222"
|
||||
)
|
||||
```
|
||||
|
||||
#### MicroPython
|
||||
|
||||
```python
|
||||
from natsbridge_mpy import NATSBridge
|
||||
|
||||
bridge = NATSBridge()
|
||||
|
||||
config = {
|
||||
"wifi_ssid": "MyNetwork",
|
||||
"wifi_password": "password123",
|
||||
"update_interval": 60
|
||||
}
|
||||
|
||||
data = [("config", config, "dictionary")]
|
||||
env, env_json_str = bridge.smartsend(
|
||||
"/device/config",
|
||||
data,
|
||||
size_threshold=100000
|
||||
)
|
||||
```
|
||||
|
||||
### Example 2: Sending Binary Data (Image)
|
||||
|
||||
#### Julia
|
||||
|
||||
```julia
|
||||
using NATSBridge
|
||||
|
||||
# Read image file
|
||||
image_data = read("image.png")
|
||||
|
||||
data = [("user_image", image_data, "binary")]
|
||||
env, env_json_str = smartsend("/chat/image", data, broker_url="nats://localhost:4222")
|
||||
```
|
||||
|
||||
#### JavaScript
|
||||
|
||||
```javascript
|
||||
const NATSBridge = require('./src/natsbridge.js');
|
||||
const fs = require('fs');
|
||||
|
||||
// Read image file
|
||||
const image_data = fs.readFileSync('image.png');
|
||||
|
||||
const data = [["user_image", image_data, "binary"]];
|
||||
const [env, env_json_str] = await NATSBridge.smartsend(
|
||||
"/chat/image",
|
||||
data,
|
||||
{ broker_url: "nats://localhost:4222" }
|
||||
);
|
||||
```
|
||||
|
||||
#### Python
|
||||
|
||||
```python
|
||||
from natsbridge import smartsend
|
||||
|
||||
# Read image file
|
||||
with open("image.png", "rb") as f:
|
||||
image_data = f.read()
|
||||
|
||||
data = [("user_image", image_data, "binary")]
|
||||
env, env_json_str = await smartsend(
|
||||
"/chat/image",
|
||||
data,
|
||||
broker_url="nats://localhost:4222"
|
||||
)
|
||||
```
|
||||
|
||||
#### MicroPython
|
||||
|
||||
```python
|
||||
from natsbridge_mpy import NATSBridge
|
||||
|
||||
bridge = NATSBridge()
|
||||
|
||||
# Read image file
|
||||
with open("image.png", "rb") as f:
|
||||
image_data = f.read()
|
||||
|
||||
data = [("user_image", image_data, "binary")]
|
||||
env, env_json_str = bridge.smartsend(
|
||||
"/chat/image",
|
||||
data,
|
||||
size_threshold=100000
|
||||
)
|
||||
```
|
||||
|
||||
### Example 3: Request-Response Pattern
|
||||
|
||||
#### Julia (Requester)
|
||||
|
||||
```julia
|
||||
using NATSBridge
|
||||
|
||||
# Send command with reply-to
|
||||
data = [("command", Dict("action" => "read_sensor"), "dictionary")]
|
||||
env, env_json_str = smartsend(
|
||||
"/device/command",
|
||||
data,
|
||||
broker_url="nats://localhost:4222",
|
||||
reply_to="/device/response",
|
||||
reply_to_msg_id="cmd-001"
|
||||
)
|
||||
```
|
||||
|
||||
#### JavaScript (Requester)
|
||||
|
||||
```javascript
|
||||
const NATSBridge = require('./src/natsbridge.js');
|
||||
|
||||
// Send command with reply-to
|
||||
const data = [["command", { action: "read_sensor" }, "dictionary"]];
|
||||
const [env, env_json_str] = await NATSBridge.smartsend(
|
||||
"/device/command",
|
||||
data,
|
||||
{
|
||||
broker_url: "nats://localhost:4222",
|
||||
reply_to: "/device/response",
|
||||
reply_to_msg_id: "cmd-001"
|
||||
}
|
||||
);
|
||||
```
|
||||
|
||||
#### Python (Requester)
|
||||
|
||||
```python
|
||||
from natsbridge import smartsend
|
||||
|
||||
# Send command with reply-to
|
||||
data = [("command", {"action": "read_sensor"}, "dictionary")]
|
||||
env, env_json_str = await smartsend(
|
||||
"/device/command",
|
||||
data,
|
||||
broker_url="nats://localhost:4222",
|
||||
reply_to="/device/response",
|
||||
reply_to_msg_id="cmd-001"
|
||||
)
|
||||
```
|
||||
|
||||
#### Julia (Responder)
|
||||
|
||||
```julia
|
||||
using NATSBridge, NATS
|
||||
|
||||
const SUBJECT = "/device/command"
|
||||
const NATS_URL = "nats://localhost:4222"
|
||||
|
||||
function test_responder()
|
||||
conn = NATS.connect(NATS_URL)
|
||||
NATS.subscribe(conn, SUBJECT) do msg
|
||||
env = smartreceive(msg, fileserver_download_handler=_fetch_with_backoff)
|
||||
|
||||
reply_to = env["reply_to"]
|
||||
|
||||
for (dataname, data, type) in env["payloads"]
|
||||
if dataname == "command" && data["action"] == "read_sensor"
|
||||
response = Dict("sensor_id" => "sensor-001", "value" => 42.5)
|
||||
if !isempty(reply_to)
|
||||
smartsend(reply_to, [("data", response, "dictionary")])
|
||||
end
|
||||
end
|
||||
end
|
||||
end
|
||||
|
||||
sleep(120)
|
||||
NATS.drain(conn)
|
||||
end
|
||||
|
||||
test_responder()
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Advanced Usage
|
||||
|
||||
### Example 4: Large Payloads (File Server)
|
||||
|
||||
For payloads larger than 1MB, NATSBridge automatically uses the file server:
|
||||
|
||||
#### Julia
|
||||
|
||||
```julia
|
||||
using NATSBridge
|
||||
|
||||
# Create large data (> 1MB)
|
||||
large_data = rand(UInt8, 2_000_000)
|
||||
|
||||
env, env_json_str = smartsend(
|
||||
"/data/large",
|
||||
[("large_file", large_data, "binary")],
|
||||
broker_url="nats://localhost:4222",
|
||||
fileserver_url="http://localhost:8080"
|
||||
)
|
||||
|
||||
println("File uploaded to: $(env.payloads[1].data)")
|
||||
# Note: For link transport, data field contains the URL string
|
||||
```
|
||||
|
||||
#### JavaScript
|
||||
|
||||
```javascript
|
||||
const NATSBridge = require('./src/natsbridge.js');
|
||||
|
||||
// Create large data (> 1MB)
|
||||
const large_data = Buffer.alloc(2_000_000);
|
||||
for (let i = 0; i < large_data.length; i++) {
|
||||
large_data[i] = Math.floor(Math.random() * 256);
|
||||
}
|
||||
|
||||
const [env, env_json_str] = await NATSBridge.smartsend(
|
||||
"/data/large",
|
||||
[["large_file", large_data, "binary"]],
|
||||
{
|
||||
broker_url: "nats://localhost:4222",
|
||||
fileserver_url: "http://localhost:8080"
|
||||
}
|
||||
);
|
||||
|
||||
console.log("File uploaded to:", env.payloads[0].data);
|
||||
// Note: For link transport, data field contains the URL string
|
||||
```
|
||||
|
||||
#### Python
|
||||
|
||||
```python
|
||||
from natsbridge import smartsend
|
||||
|
||||
# Create large data (> 1MB)
|
||||
import os
|
||||
large_data = os.urandom(2_000_000)
|
||||
|
||||
env, env_json_str = await smartsend(
|
||||
"/data/large",
|
||||
[("large_file", large_data, "binary")],
|
||||
broker_url="nats://localhost:4222",
|
||||
fileserver_url="http://localhost:8080"
|
||||
)
|
||||
|
||||
print(f"File uploaded to: {env['payloads'][0]['data']}")
|
||||
# Note: For link transport, data field contains the URL string
|
||||
```
|
||||
|
||||
#### MicroPython
|
||||
|
||||
MicroPython enforces a hard limit of 50KB per payload:
|
||||
|
||||
```python
|
||||
from natsbridge_mpy import NATSBridge
|
||||
|
||||
bridge = NATSBridge()
|
||||
|
||||
# MicroPython has a hard limit of 50KB per payload
|
||||
# Use streaming or chunking for larger data
|
||||
small_data = bytes(1000) # 1KB
|
||||
|
||||
data = [("small_file", small_data, "binary")]
|
||||
env, env_json_str = bridge.smartsend(
|
||||
"/data/small",
|
||||
data,
|
||||
size_threshold=100000 # Enforced max: 50000 bytes
|
||||
)
|
||||
```
|
||||
|
||||
### Example 5: Mixed Content (Chat with Text + Image)
|
||||
|
||||
NATSBridge supports sending multiple payloads with different types in a single message:
|
||||
|
||||
#### Julia
|
||||
|
||||
```julia
|
||||
using NATSBridge
|
||||
|
||||
image_data = read("avatar.png")
|
||||
|
||||
data = [
|
||||
("message_text", "Hello with image!", "text"),
|
||||
("user_avatar", image_data, "image")
|
||||
]
|
||||
|
||||
env, env_json_str = smartsend("/chat/mixed", data, broker_url="nats://localhost:4222")
|
||||
```
|
||||
|
||||
#### JavaScript
|
||||
|
||||
```javascript
|
||||
const NATSBridge = require('./src/natsbridge.js');
|
||||
const fs = require('fs');
|
||||
|
||||
const image_data = fs.readFileSync('avatar.png');
|
||||
|
||||
const data = [
|
||||
["message_text", "Hello with image!", "text"],
|
||||
["user_avatar", image_data, "image"]
|
||||
];
|
||||
|
||||
const [env, env_json_str] = await NATSBridge.smartsend(
|
||||
"/chat/mixed",
|
||||
data,
|
||||
{ broker_url: "nats://localhost:4222" }
|
||||
);
|
||||
```
|
||||
|
||||
#### Python
|
||||
|
||||
```python
|
||||
from natsbridge import smartsend
|
||||
|
||||
with open("avatar.png", "rb") as f:
|
||||
image_data = f.read()
|
||||
|
||||
data = [
|
||||
("message_text", "Hello with image!", "text"),
|
||||
("user_avatar", image_data, "image")
|
||||
]
|
||||
|
||||
env, env_json_str = await smartsend(
|
||||
"/chat/mixed",
|
||||
data,
|
||||
broker_url="nats://localhost:4222"
|
||||
)
|
||||
# env: Dict with all metadata and payloads
|
||||
```
|
||||
|
||||
### Example 6: Table Data (Arrow IPC)
|
||||
|
||||
For tabular data, NATSBridge uses Apache Arrow IPC format:
|
||||
|
||||
#### Julia
|
||||
|
||||
```julia
|
||||
using NATSBridge
|
||||
using DataFrames
|
||||
|
||||
# Create DataFrame
|
||||
df = DataFrame(
|
||||
id = [1, 2, 3],
|
||||
name = ["Alice", "Bob", "Charlie"],
|
||||
score = [95, 88, 92]
|
||||
)
|
||||
|
||||
data = [("students", df, "arrowtable")]
|
||||
env, env_json_str = smartsend("/data/students", data, broker_url="nats://localhost:4222")
|
||||
```
|
||||
|
||||
#### JavaScript
|
||||
|
||||
```javascript
|
||||
const NATSBridge = require('./src/natsbridge.js');
|
||||
|
||||
// Create table data (array of objects)
|
||||
const table_data = [
|
||||
{ id: 1, name: "Alice", score: 95 },
|
||||
{ id: 2, name: "Bob", score: 88 },
|
||||
{ id: 3, name: "Charlie", score: 92 }
|
||||
];
|
||||
|
||||
const data = [["students", table_data, "arrowtable"]];
|
||||
const [env, env_json_str] = await NATSBridge.smartsend(
|
||||
"/data/students",
|
||||
data,
|
||||
{ broker_url: "nats://localhost:4222" }
|
||||
);
|
||||
```
|
||||
|
||||
#### Python
|
||||
|
||||
```python
|
||||
from natsbridge import smartsend
|
||||
import pandas as pd
|
||||
|
||||
# Create DataFrame
|
||||
df = pd.DataFrame({
|
||||
'id': [1, 2, 3],
|
||||
'name': ['Alice', 'Bob', 'Charlie'],
|
||||
'score': [95, 88, 92]
|
||||
})
|
||||
|
||||
data = [("students", df, "table")]
|
||||
env, env_json_str = await smartsend(
|
||||
"/data/students",
|
||||
data,
|
||||
broker_url="nats://localhost:4222"
|
||||
)
|
||||
```
|
||||
|
||||
#### MicroPython
|
||||
|
||||
MicroPython does not support table type due to memory constraints. Use dictionary or binary instead.
|
||||
|
||||
---
|
||||
|
||||
## Next Steps
|
||||
|
||||
1. **Explore the test directory** for more examples
|
||||
2. **Check the documentation** for advanced configuration options
|
||||
3. **Read the walkthrough** for building real-world applications
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Connection Issues
|
||||
|
||||
- Ensure NATS server is running: `docker ps | grep nats`
|
||||
- Check firewall settings
|
||||
- Verify NATS URL configuration
|
||||
|
||||
### File Server Issues
|
||||
|
||||
- Ensure file server is running and accessible
|
||||
- Check upload permissions
|
||||
- Verify file server URL configuration
|
||||
|
||||
### Serialization Errors
|
||||
|
||||
- Verify data type matches the specified type
|
||||
- Check that binary data is in the correct format
|
||||
- MicroPython: Ensure payload size < 50KB
|
||||
|
||||
---
|
||||
|
||||
## License
|
||||
|
||||
MIT
|
||||
Reference in New Issue
Block a user