update readme
This commit is contained in:
226
README.md
226
README.md
@@ -1,6 +1,6 @@
|
|||||||
# NATSBridge - Cross-Platform Bi-Directional Data Bridge
|
# NATSBridge - Cross-Platform Bi-Directional Data Bridge
|
||||||
|
|
||||||
A high-performance, bi-directional data bridge for **Julia, JavaScript, Python, and MicroPython** applications using NATS (Core & JetStream), implementing the Claim-Check pattern for large payloads.
|
A high-performance, bi-directional data bridge for **Julia**, **JavaScript**, **Python**, and **MicroPython** applications using NATS (Core & JetStream), implementing the Claim-Check pattern for large payloads.
|
||||||
|
|
||||||
[](https://opensource.org/licenses/MIT)
|
[](https://opensource.org/licenses/MIT)
|
||||||
[](https://nats.io)
|
[](https://nats.io)
|
||||||
@@ -28,8 +28,8 @@ NATSBridge enables seamless communication across multiple platforms through NATS
|
|||||||
|
|
||||||
| Transport | Payload Size | Method |
|
| Transport | Payload Size | Method |
|
||||||
|-----------|--------------|--------|
|
|-----------|--------------|--------|
|
||||||
| **Direct** | < 1MB | Sent directly via NATS (Base64 encoded) |
|
| **Direct** | < 500KB | Sent directly via NATS (Base64 encoded) |
|
||||||
| **Link** | >= 1MB | Uploaded to HTTP file server, URL sent via NATS |
|
| **Link** | ≥ 500KB | Uploaded to HTTP file server, URL sent via NATS |
|
||||||
|
|
||||||
### Use Cases
|
### Use Cases
|
||||||
|
|
||||||
@@ -45,7 +45,7 @@ NATSBridge enables seamless communication across multiple platforms through NATS
|
|||||||
| Platform | Implementation | Features |
|
| Platform | Implementation | Features |
|
||||||
|----------|----------------|----------|
|
|----------|----------------|----------|
|
||||||
| **Julia** | [`src/NATSBridge.jl`](src/NATSBridge.jl) | Full feature set, Arrow IPC, multiple dispatch |
|
| **Julia** | [`src/NATSBridge.jl`](src/NATSBridge.jl) | Full feature set, Arrow IPC, multiple dispatch |
|
||||||
| **JavaScript** | [`src/natsbridge.js`](src/natsbridge.js) | Node.js, async/await |
|
| **JavaScript (Node.js)** | [`src/natsbridge.js`](src/natsbridge_ssr.js) | Node.js, async/await |
|
||||||
| **JavaScript (Browser)** | [`src/natsbridge_csr.js`](src/natsbridge_csr.js) | Browser, WebSocket NATS, async/await |
|
| **JavaScript (Browser)** | [`src/natsbridge_csr.js`](src/natsbridge_csr.js) | Browser, WebSocket NATS, async/await |
|
||||||
| **Python** | [`src/natsbridge.py`](src/natsbridge.py) | Desktop Python, asyncio, type hints |
|
| **Python** | [`src/natsbridge.py`](src/natsbridge.py) | Desktop Python, asyncio, type hints |
|
||||||
| **MicroPython** | [`src/natsbridge_mpy.py`](src/natsbridge_mpy.py) | Memory-constrained, synchronous API |
|
| **MicroPython** | [`src/natsbridge_mpy.py`](src/natsbridge_mpy.py) | Memory-constrained, synchronous API |
|
||||||
@@ -72,7 +72,7 @@ NATSBridge enables seamless communication across multiple platforms through NATS
|
|||||||
- ✅ **Bi-directional messaging** with request-reply patterns
|
- ✅ **Bi-directional messaging** with request-reply patterns
|
||||||
- ✅ **Multi-payload support** - send multiple payloads with different types in one message
|
- ✅ **Multi-payload support** - send multiple payloads with different types in one message
|
||||||
- ✅ **Automatic transport selection** - direct vs link based on payload size
|
- ✅ **Automatic transport selection** - direct vs link based on payload size
|
||||||
- ✅ **Claim-Check pattern** for payloads > 1MB
|
- ✅ **Claim-Check pattern** for payloads ≥ 500KB
|
||||||
- ✅ **Apache Arrow IPC** support for tabular data (zero-copy reading)
|
- ✅ **Apache Arrow IPC** support for tabular data (zero-copy reading)
|
||||||
- ✅ **Exponential backoff** for reliable file server downloads
|
- ✅ **Exponential backoff** for reliable file server downloads
|
||||||
- ✅ **Correlation ID tracking** for message tracing
|
- ✅ **Correlation ID tracking** for message tracing
|
||||||
@@ -83,23 +83,24 @@ NATSBridge enables seamless communication across multiple platforms through NATS
|
|||||||
|
|
||||||
## Quick Start
|
## Quick Start
|
||||||
|
|
||||||
### Step 1: Start NATS Server
|
### Prerequisites
|
||||||
|
|
||||||
```bash
|
1. **NATS Server** - Install and run a NATS server:
|
||||||
docker run -p 4222:4222 nats:latest
|
```bash
|
||||||
```
|
docker run -p 4222:4222 nats:latest
|
||||||
|
```
|
||||||
|
|
||||||
### Step 2: Start HTTP File Server (Optional)
|
2. **HTTP File Server** (optional, for large payloads) - Install and run a file server:
|
||||||
|
```bash
|
||||||
|
# Using Plik
|
||||||
|
docker run -p 8080:8080 -v /tmp/fileserver:/var/lib/plik -e PLIK_ADMIN_PASSWORD=admin plik/plik
|
||||||
|
|
||||||
```bash
|
# OR using simple Python HTTP server
|
||||||
# Create a directory for file uploads
|
mkdir -p /tmp/fileserver
|
||||||
mkdir -p /tmp/fileserver
|
python3 -m http.server 8080 --directory /tmp/fileserver
|
||||||
|
```
|
||||||
|
|
||||||
# Start HTTP file server
|
### Send Your First Message
|
||||||
python3 -m http.server 8080 --directory /tmp/fileserver
|
|
||||||
```
|
|
||||||
|
|
||||||
### Step 3: Send Your First Message
|
|
||||||
|
|
||||||
#### Julia
|
#### Julia
|
||||||
|
|
||||||
@@ -107,14 +108,14 @@ python3 -m http.server 8080 --directory /tmp/fileserver
|
|||||||
using NATSBridge
|
using NATSBridge
|
||||||
|
|
||||||
data = [("message", "Hello World", "text")]
|
data = [("message", "Hello World", "text")]
|
||||||
env, env_json_str = smartsend("/chat/room1", data, broker_url="nats://localhost:4222")
|
env, env_json_str = smartsend("/chat/room1", data; broker_url="nats://localhost:4222")
|
||||||
println("Message sent!")
|
println("Message sent!")
|
||||||
```
|
```
|
||||||
|
|
||||||
#### JavaScript
|
#### JavaScript (Node.js)
|
||||||
|
|
||||||
```javascript
|
```javascript
|
||||||
const NATSBridge = require('./src/natsbridge.js');
|
import NATSBridge from './src/natsbridge_ssr.js';
|
||||||
|
|
||||||
const data = [["message", "Hello World", "text"]];
|
const data = [["message", "Hello World", "text"]];
|
||||||
const [env, env_json_str] = await NATSBridge.smartsend(
|
const [env, env_json_str] = await NATSBridge.smartsend(
|
||||||
@@ -125,6 +126,20 @@ const [env, env_json_str] = await NATSBridge.smartsend(
|
|||||||
console.log("Message sent!");
|
console.log("Message sent!");
|
||||||
```
|
```
|
||||||
|
|
||||||
|
#### JavaScript (Browser)
|
||||||
|
|
||||||
|
```javascript
|
||||||
|
import NATSBridge from './src/natsbridge_csr.js';
|
||||||
|
|
||||||
|
const data = [["message", "Hello World", "text"]];
|
||||||
|
const [env, env_json_str] = await NATSBridge.smartsend(
|
||||||
|
"/chat/room1",
|
||||||
|
data,
|
||||||
|
{ broker_url: "ws://localhost:4222" }
|
||||||
|
);
|
||||||
|
console.log("Message sent!");
|
||||||
|
```
|
||||||
|
|
||||||
#### Python
|
#### Python
|
||||||
|
|
||||||
```python
|
```python
|
||||||
@@ -139,6 +154,21 @@ env, env_json_str = await smartsend(
|
|||||||
print("Message sent!")
|
print("Message sent!")
|
||||||
```
|
```
|
||||||
|
|
||||||
|
#### MicroPython
|
||||||
|
|
||||||
|
```python
|
||||||
|
from natsbridge import smartsend
|
||||||
|
|
||||||
|
data = [("message", "Hello World", "text")]
|
||||||
|
env, env_json_str = smartsend(
|
||||||
|
"/chat/room1",
|
||||||
|
data,
|
||||||
|
broker_url="nats://localhost:4222",
|
||||||
|
size_threshold=100000 # 100KB for MicroPython
|
||||||
|
)
|
||||||
|
print("Message sent!")
|
||||||
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## API Reference
|
## API Reference
|
||||||
@@ -147,13 +177,13 @@ print("Message sent!")
|
|||||||
|
|
||||||
All platforms use the same input/output format for payloads:
|
All platforms use the same input/output format for payloads:
|
||||||
|
|
||||||
**Input format for smartsend:**
|
**Input format for `smartsend`:**
|
||||||
```
|
```
|
||||||
[(dataname1, data1, type1), (dataname2, data2, type2), ...]
|
[(dataname1, data1, type1), (dataname2, data2, type2), ...]
|
||||||
```
|
```
|
||||||
|
|
||||||
**Output format for smartreceive:**
|
**Output format for `smartreceive`:**
|
||||||
```
|
```json
|
||||||
{
|
{
|
||||||
"correlation_id": "...",
|
"correlation_id": "...",
|
||||||
"msg_id": "...",
|
"msg_id": "...",
|
||||||
@@ -187,7 +217,7 @@ env, env_json_str = NATSBridge.smartsend(
|
|||||||
broker_url::String = "nats://localhost:4222",
|
broker_url::String = "nats://localhost:4222",
|
||||||
fileserver_url = "http://localhost:8080",
|
fileserver_url = "http://localhost:8080",
|
||||||
fileserver_upload_handler::Function = plik_oneshot_upload,
|
fileserver_upload_handler::Function = plik_oneshot_upload,
|
||||||
size_threshold::Int = 1_000_000,
|
size_threshold::Int = 500_000,
|
||||||
correlation_id::String = string(uuid4()),
|
correlation_id::String = string(uuid4()),
|
||||||
msg_purpose::String = "chat",
|
msg_purpose::String = "chat",
|
||||||
sender_name::String = "NATSBridge",
|
sender_name::String = "NATSBridge",
|
||||||
@@ -203,10 +233,10 @@ env, env_json_str = NATSBridge.smartsend(
|
|||||||
# Returns: ::Tuple{msg_envelope_v1, String}
|
# Returns: ::Tuple{msg_envelope_v1, String}
|
||||||
```
|
```
|
||||||
|
|
||||||
#### JavaScript
|
#### JavaScript (Node.js)
|
||||||
|
|
||||||
```javascript
|
```javascript
|
||||||
const NATSBridge = require('natsbridge');
|
import NATSBridge from './src/natsbridge_ssr.js';
|
||||||
|
|
||||||
const [env, env_json_str] = await NATSBridge.smartsend(
|
const [env, env_json_str] = await NATSBridge.smartsend(
|
||||||
subject,
|
subject,
|
||||||
@@ -215,7 +245,36 @@ const [env, env_json_str] = await NATSBridge.smartsend(
|
|||||||
broker_url: 'nats://localhost:4222',
|
broker_url: 'nats://localhost:4222',
|
||||||
fileserver_url: 'http://localhost:8080',
|
fileserver_url: 'http://localhost:8080',
|
||||||
fileserver_upload_handler: NATSBridge.plikOneshotUpload,
|
fileserver_upload_handler: NATSBridge.plikOneshotUpload,
|
||||||
size_threshold: 1_000_000,
|
size_threshold: 500_000,
|
||||||
|
correlation_id: uuidv4(),
|
||||||
|
msg_purpose: 'chat',
|
||||||
|
sender_name: 'NATSBridge',
|
||||||
|
receiver_name: '',
|
||||||
|
receiver_id: '',
|
||||||
|
reply_to: '',
|
||||||
|
reply_to_msg_id: '',
|
||||||
|
is_publish: true,
|
||||||
|
nats_connection: null,
|
||||||
|
msg_id: uuidv4(),
|
||||||
|
sender_id: uuidv4()
|
||||||
|
}
|
||||||
|
);
|
||||||
|
// Returns: Promise<[env, env_json_str]>
|
||||||
|
```
|
||||||
|
|
||||||
|
#### JavaScript (Browser)
|
||||||
|
|
||||||
|
```javascript
|
||||||
|
import NATSBridge from './src/natsbridge_csr.js';
|
||||||
|
|
||||||
|
const [env, env_json_str] = await NATSBridge.smartsend(
|
||||||
|
subject,
|
||||||
|
data,
|
||||||
|
{
|
||||||
|
broker_url: 'ws://localhost:4222',
|
||||||
|
fileserver_url: 'http://localhost:8080',
|
||||||
|
fileserver_upload_handler: NATSBridge.plikOneshotUpload,
|
||||||
|
size_threshold: 500_000,
|
||||||
correlation_id: uuidv4(),
|
correlation_id: uuidv4(),
|
||||||
msg_purpose: 'chat',
|
msg_purpose: 'chat',
|
||||||
sender_name: 'NATSBridge',
|
sender_name: 'NATSBridge',
|
||||||
@@ -243,7 +302,7 @@ env, env_json_str = await NATSBridge.smartsend(
|
|||||||
broker_url: str = "nats://localhost:4222",
|
broker_url: str = "nats://localhost:4222",
|
||||||
fileserver_url: str = "http://localhost:8080",
|
fileserver_url: str = "http://localhost:8080",
|
||||||
fileserver_upload_handler: Callable = plik_oneshot_upload,
|
fileserver_upload_handler: Callable = plik_oneshot_upload,
|
||||||
size_threshold: int = 1_000_000,
|
size_threshold: int = 500_000,
|
||||||
correlation_id: str = None,
|
correlation_id: str = None,
|
||||||
msg_purpose: str = "chat",
|
msg_purpose: str = "chat",
|
||||||
sender_name: str = "NATSBridge",
|
sender_name: str = "NATSBridge",
|
||||||
@@ -293,9 +352,28 @@ env = NATSBridge.smartreceive(
|
|||||||
# Returns: ::JSON.Object{String, Any}
|
# Returns: ::JSON.Object{String, Any}
|
||||||
```
|
```
|
||||||
|
|
||||||
#### JavaScript
|
#### JavaScript (Node.js)
|
||||||
|
|
||||||
```javascript
|
```javascript
|
||||||
|
import NATSBridge from './src/natsbridge_ssr.js';
|
||||||
|
|
||||||
|
const env = await NATSBridge.smartreceive(
|
||||||
|
msg,
|
||||||
|
{
|
||||||
|
fileserver_download_handler: NATSBridge.fetchWithBackoff,
|
||||||
|
max_retries: 5,
|
||||||
|
base_delay: 100,
|
||||||
|
max_delay: 5000
|
||||||
|
}
|
||||||
|
);
|
||||||
|
// Returns: Promise<env_object>
|
||||||
|
```
|
||||||
|
|
||||||
|
#### JavaScript (Browser)
|
||||||
|
|
||||||
|
```javascript
|
||||||
|
import NATSBridge from './src/natsbridge_csr.js';
|
||||||
|
|
||||||
const env = await NATSBridge.smartreceive(
|
const env = await NATSBridge.smartreceive(
|
||||||
msg,
|
msg,
|
||||||
{
|
{
|
||||||
@@ -311,6 +389,8 @@ const env = await NATSBridge.smartreceive(
|
|||||||
#### Python
|
#### Python
|
||||||
|
|
||||||
```python
|
```python
|
||||||
|
from natsbridge import NATSBridge
|
||||||
|
|
||||||
env = await NATSBridge.smartreceive(
|
env = await NATSBridge.smartreceive(
|
||||||
msg,
|
msg,
|
||||||
fileserver_download_handler=fetch_with_backoff,
|
fileserver_download_handler=fetch_with_backoff,
|
||||||
@@ -324,6 +404,8 @@ env = await NATSBridge.smartreceive(
|
|||||||
#### MicroPython
|
#### MicroPython
|
||||||
|
|
||||||
```python
|
```python
|
||||||
|
from natsbridge import NATSBridge
|
||||||
|
|
||||||
env = NATSBridge.smartreceive(
|
env = NATSBridge.smartreceive(
|
||||||
msg,
|
msg,
|
||||||
fileserver_download_handler=_sync_fileserver_download,
|
fileserver_download_handler=_sync_fileserver_download,
|
||||||
@@ -343,7 +425,7 @@ env = NATSBridge.smartreceive(
|
|||||||
| `text` | `String` | `string` | `str` | `str` | Plain text strings |
|
| `text` | `String` | `string` | `str` | `str` | Plain text strings |
|
||||||
| `dictionary` | `Dict`, `NamedTuple` | `Object`, `Array` | `dict`, `list` | `dict` | JSON-serializable dictionaries |
|
| `dictionary` | `Dict`, `NamedTuple` | `Object`, `Array` | `dict`, `list` | `dict` | JSON-serializable dictionaries |
|
||||||
| `arrowtable` | `DataFrame`, `Arrow.Table` | `Array<Object>` | `pandas.DataFrame` | ❌ | Tabular data (Arrow IPC) |
|
| `arrowtable` | `DataFrame`, `Arrow.Table` | `Array<Object>` | `pandas.DataFrame` | ❌ | Tabular data (Arrow IPC) |
|
||||||
| `jsontable` | `Vector{NamedTuple}` | `Array<Object>` | `list[dict]` | ❌ | Tabular data (JSON) |
|
| `jsontable` | `Vector{NamedTuple}` | `Array<Object>` | `list[dict]` | ⚠️ | Tabular data (JSON) |
|
||||||
| `image` | `Vector{UInt8}` | `Uint8Array`, `Buffer` | `bytes` | `bytearray` | Image data (PNG, JPG) |
|
| `image` | `Vector{UInt8}` | `Uint8Array`, `Buffer` | `bytes` | `bytearray` | Image data (PNG, JPG) |
|
||||||
| `audio` | `Vector{UInt8}` | `Uint8Array`, `Buffer` | `bytes` | `bytearray` | Audio data (WAV, MP3) |
|
| `audio` | `Vector{UInt8}` | `Uint8Array`, `Buffer` | `bytes` | `bytearray` | Audio data (WAV, MP3) |
|
||||||
| `video` | `Vector{UInt8}` | `Uint8Array`, `Buffer` | `bytes` | `bytearray` | Video data (MP4, AVI) |
|
| `video` | `Vector{UInt8}` | `Uint8Array`, `Buffer` | `bytes` | `bytearray` | Video data (MP4, AVI) |
|
||||||
@@ -368,13 +450,13 @@ data = [
|
|||||||
("large_document", large_file_data, "binary")
|
("large_document", large_file_data, "binary")
|
||||||
]
|
]
|
||||||
|
|
||||||
env, env_json_str = NATSBridge.smartsend("/chat/room1", data; fileserver_url="http://localhost:8080")
|
env, env_json_str = smartsend("/chat/room1", data; fileserver_url="http://localhost:8080")
|
||||||
```
|
```
|
||||||
|
|
||||||
#### JavaScript
|
#### JavaScript (Node.js)
|
||||||
|
|
||||||
```javascript
|
```javascript
|
||||||
const NATSBridge = require('natsbridge');
|
import NATSBridge from './src/natsbridge_ssr.js';
|
||||||
|
|
||||||
const data = [
|
const data = [
|
||||||
["message_text", "Hello!", "text"],
|
["message_text", "Hello!", "text"],
|
||||||
@@ -389,6 +471,24 @@ const [env, env_json_str] = await NATSBridge.smartsend(
|
|||||||
);
|
);
|
||||||
```
|
```
|
||||||
|
|
||||||
|
#### JavaScript (Browser)
|
||||||
|
|
||||||
|
```javascript
|
||||||
|
import NATSBridge from './src/natsbridge_csr.js';
|
||||||
|
|
||||||
|
const data = [
|
||||||
|
["message_text", "Hello!", "text"],
|
||||||
|
["user_avatar", imageData, "image"],
|
||||||
|
["large_document", largeFileData, "binary"]
|
||||||
|
];
|
||||||
|
|
||||||
|
const [env, env_json_str] = await NATSBridge.smartsend(
|
||||||
|
"/chat/room1",
|
||||||
|
data,
|
||||||
|
{ broker_url: 'ws://localhost:4222', fileserver_url: 'http://localhost:8080' }
|
||||||
|
);
|
||||||
|
```
|
||||||
|
|
||||||
#### Python
|
#### Python
|
||||||
|
|
||||||
```python
|
```python
|
||||||
@@ -423,13 +523,13 @@ config = Dict(
|
|||||||
)
|
)
|
||||||
|
|
||||||
data = [("config", config, "dictionary")]
|
data = [("config", config, "dictionary")]
|
||||||
env, env_json_str = NATSBridge.smartsend("/device/config", data)
|
env, env_json_str = smartsend("/device/config", data)
|
||||||
```
|
```
|
||||||
|
|
||||||
#### JavaScript
|
#### JavaScript (Node.js)
|
||||||
|
|
||||||
```javascript
|
```javascript
|
||||||
const NATSBridge = require('natsbridge');
|
import NATSBridge from './src/natsbridge_ssr.js';
|
||||||
|
|
||||||
const config = {
|
const config = {
|
||||||
wifi_ssid: "MyNetwork",
|
wifi_ssid: "MyNetwork",
|
||||||
@@ -475,13 +575,13 @@ df = DataFrame(
|
|||||||
)
|
)
|
||||||
|
|
||||||
data = [("students", df, "arrowtable")]
|
data = [("students", df, "arrowtable")]
|
||||||
env, env_json_str = NATSBridge.smartsend("/data/analysis", data)
|
env, env_json_str = smartsend("/data/analysis", data)
|
||||||
```
|
```
|
||||||
|
|
||||||
#### JavaScript
|
#### JavaScript (Node.js)
|
||||||
|
|
||||||
```javascript
|
```javascript
|
||||||
const NATSBridge = require('natsbridge');
|
import NATSBridge from './src/natsbridge_ssr.js';
|
||||||
|
|
||||||
const df = [
|
const df = [
|
||||||
{ id: 1, name: "Alice", score: 95 },
|
{ id: 1, name: "Alice", score: 95 },
|
||||||
@@ -521,18 +621,29 @@ Bi-directional communication with reply-to support.
|
|||||||
using NATSBridge
|
using NATSBridge
|
||||||
|
|
||||||
# Requester
|
# Requester
|
||||||
env, env_json_str = NATSBridge.smartsend(
|
env, env_json_str = smartsend(
|
||||||
"/device/command",
|
"/device/command",
|
||||||
[("command", Dict("action" => "read_sensor"), "dictionary")];
|
[("command", Dict("action" => "read_sensor"), "dictionary")];
|
||||||
broker_url="nats://localhost:4222",
|
broker_url="nats://localhost:4222",
|
||||||
reply_to="/device/response"
|
reply_to="/device/response"
|
||||||
)
|
)
|
||||||
|
|
||||||
|
# Receiver (in separate application)
|
||||||
|
msg = NATS.subscription.next()
|
||||||
|
env = smartreceive(msg)
|
||||||
|
# Process request and send response
|
||||||
|
response_env, response_json = smartsend(
|
||||||
|
"/device/response",
|
||||||
|
[("result", Dict("value" => 42), "dictionary")],
|
||||||
|
reply_to="/device/command",
|
||||||
|
reply_to_msg_id=env["msg_id"]
|
||||||
|
)
|
||||||
```
|
```
|
||||||
|
|
||||||
#### JavaScript
|
#### JavaScript (Node.js)
|
||||||
|
|
||||||
```javascript
|
```javascript
|
||||||
const NATSBridge = require('natsbridge');
|
import NATSBridge from './src/natsbridge_ssr.js';
|
||||||
|
|
||||||
// Requester
|
// Requester
|
||||||
const [env, env_json_str] = await NATSBridge.smartsend(
|
const [env, env_json_str] = await NATSBridge.smartsend(
|
||||||
@@ -540,6 +651,16 @@ const [env, env_json_str] = await NATSBridge.smartsend(
|
|||||||
[["command", { action: "read_sensor" }, "dictionary"]],
|
[["command", { action: "read_sensor" }, "dictionary"]],
|
||||||
{ broker_url: 'nats://localhost:4222', reply_to: '/device/response' }
|
{ broker_url: 'nats://localhost:4222', reply_to: '/device/response' }
|
||||||
);
|
);
|
||||||
|
|
||||||
|
// Receiver (in separate application)
|
||||||
|
// const msg = await natsConsumer.next();
|
||||||
|
// const env = await NATSBridge.smartreceive(msg);
|
||||||
|
// Process request and send response
|
||||||
|
// const response_env, response_json = await NATSBridge.smartsend(
|
||||||
|
// "/device/response",
|
||||||
|
// [["result", { value: 42 }, "dictionary"]],
|
||||||
|
// { reply_to: '/device/command', reply_to_msg_id: env.msg_id }
|
||||||
|
// );
|
||||||
```
|
```
|
||||||
|
|
||||||
#### Python
|
#### Python
|
||||||
@@ -554,6 +675,17 @@ env, env_json_str = await NATSBridge.smartsend(
|
|||||||
broker_url="nats://localhost:4222",
|
broker_url="nats://localhost:4222",
|
||||||
reply_to="/device/response"
|
reply_to="/device/response"
|
||||||
)
|
)
|
||||||
|
|
||||||
|
# Receiver (in separate application)
|
||||||
|
# msg = await nats_consumer.next()
|
||||||
|
# env = await NATSBridge.smartreceive(msg)
|
||||||
|
# Process request and send response
|
||||||
|
# response_env, response_json = await NATSBridge.smartsend(
|
||||||
|
# "/device/response",
|
||||||
|
# [("result", {"value": 42}, "dictionary")],
|
||||||
|
# reply_to="/device/command",
|
||||||
|
# reply_to_msg_id=env["msg_id"]
|
||||||
|
# )
|
||||||
```
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
@@ -640,10 +772,10 @@ python3 test/test_py_table_receiver.py
|
|||||||
|
|
||||||
For detailed architecture and implementation information, see:
|
For detailed architecture and implementation information, see:
|
||||||
|
|
||||||
- [Architecture Documentation](docs/architecture_updated.md) - Cross-platform architecture, API parity, platform-specific patterns
|
- [`docs/architecture.md`](docs/architecture.md) - Cross-platform architecture, API parity, platform-specific patterns
|
||||||
- [Implementation Guide](docs/implementation_updated.md) - Detailed implementation for each platform, handler functions, testing
|
- [`docs/requirements.md`](docs/requirements.md) - Business requirements and user stories
|
||||||
- [Tutorial](docs/tutorial_updated.md) - Step-by-step getting started guide
|
- [`docs/spec.md`](docs/spec.md) - Technical specification and contracts
|
||||||
- [Walkthrough](docs/walkthrough_updated.md) - Real-world application building guides
|
- [`docs/walkthrough.md`](docs/walkthrough.md) - Real-world application building guides
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
|||||||
741
docs/tutorial.md
741
docs/tutorial.md
@@ -1,741 +0,0 @@
|
|||||||
# Cross-Platform NATSBridge Tutorial
|
|
||||||
|
|
||||||
A step-by-step guide to get started with NATSBridge across **Julia**, **JavaScript**, and **Python/MicroPython**.
|
|
||||||
|
|
||||||
## Table of Contents
|
|
||||||
|
|
||||||
1. [Overview](#overview)
|
|
||||||
2. [Prerequisites](#prerequisites)
|
|
||||||
3. [Installation](#installation)
|
|
||||||
4. [Quick Start](#quick-start)
|
|
||||||
5. [Basic Examples](#basic-examples)
|
|
||||||
6. [Advanced Usage](#advanced-usage)
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Overview
|
|
||||||
|
|
||||||
NATSBridge enables seamless communication across platforms through NATS, with automatic transport selection based on payload size:
|
|
||||||
|
|
||||||
- **Direct Transport**: Payloads < 1MB are sent directly via NATS (Base64 encoded)
|
|
||||||
- **Link Transport**: Payloads >= 1MB are uploaded to an HTTP file server and referenced via URL
|
|
||||||
|
|
||||||
### Cross-Platform API Parity
|
|
||||||
|
|
||||||
All three platforms use the same high-level API:
|
|
||||||
|
|
||||||
```
|
|
||||||
# Input format
|
|
||||||
smartsend(subject, [(dataname, data, type), ...], options)
|
|
||||||
|
|
||||||
# Output format
|
|
||||||
(env, env_json_str) = smartsend(...)
|
|
||||||
env = smartreceive(msg, options)
|
|
||||||
```
|
|
||||||
|
|
||||||
**Important Platform Differences:**
|
|
||||||
|
|
||||||
1. **Encoding field:** Julia and JavaScript preserve the original serialization format in the encoding field (`"base64"`, `"json"`, or `"arrow-ipc"`), while Python and MicroPython always use `"base64"` for all direct transport payloads.
|
|
||||||
|
|
||||||
2. **Async vs Sync:** JavaScript and Python desktop use async/await, while MicroPython uses synchronous API.
|
|
||||||
|
|
||||||
### Supported Payload Types
|
|
||||||
|
|
||||||
| Type | Julia | JavaScript | Python | MicroPython |
|
|
||||||
|------|-------|------------|--------|-------------|
|
|
||||||
| `text` | `String` | `string` | `str` | `str` |
|
|
||||||
| `dictionary` | `Dict` | `Object` | `dict` | `dict` |
|
|
||||||
| `arrowtable` | `DataFrame` | `Array<Object>` | `pandas.DataFrame` | ❌ |
|
|
||||||
| `jsontable` | `Vector{NamedTuple}` | `Array<Object>` | `list[dict]` | ❌ |
|
|
||||||
| `table` | ❌ | ❌ | `pandas.DataFrame` | ❌ |
|
|
||||||
| `image` | `Vector{UInt8}` | `Uint8Array` | `bytes` | `bytearray` |
|
|
||||||
| `audio` | `Vector{UInt8}` | `Uint8Array` | `bytes` | `bytearray` |
|
|
||||||
| `video` | `Vector{UInt8}` | `Uint8Array` | `bytes` | `bytearray` |
|
|
||||||
| `binary` | `Vector{UInt8}` | `Uint8Array` | `bytes` | `bytearray` |
|
|
||||||
|
|
||||||
**Note on MicroPython:** MicroPython does not support table types (`arrowtable`, `jsontable`, or `table`) due to memory constraints. Use `dictionary` or `binary` instead.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Prerequisites
|
|
||||||
|
|
||||||
Before you begin, ensure you have:
|
|
||||||
|
|
||||||
1. **NATS Server** running (or accessible)
|
|
||||||
2. **HTTP File Server** (optional, for large payloads > 1MB)
|
|
||||||
3. **Platform-specific packages** installed
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Installation
|
|
||||||
|
|
||||||
### Julia
|
|
||||||
|
|
||||||
```julia
|
|
||||||
using Pkg
|
|
||||||
Pkg.add("NATS")
|
|
||||||
Pkg.add("Arrow")
|
|
||||||
Pkg.add("JSON3")
|
|
||||||
Pkg.add("HTTP")
|
|
||||||
Pkg.add("UUIDs")
|
|
||||||
Pkg.add("Dates")
|
|
||||||
```
|
|
||||||
|
|
||||||
### JavaScript (Node.js)
|
|
||||||
|
|
||||||
```bash
|
|
||||||
npm install nats uuid apache-arrow node-fetch
|
|
||||||
```
|
|
||||||
|
|
||||||
### JavaScript (Browser)
|
|
||||||
|
|
||||||
```html
|
|
||||||
<script src="https://unpkg.com/nats-js/dist/bundle/nats.min.js"></script>
|
|
||||||
<script src="https://unpkg.com/apache-arrow/arrow.min.js"></script>
|
|
||||||
```
|
|
||||||
|
|
||||||
### Python (Desktop)
|
|
||||||
|
|
||||||
```bash
|
|
||||||
pip install nats-py aiohttp pyarrow pandas
|
|
||||||
```
|
|
||||||
|
|
||||||
### MicroPython
|
|
||||||
|
|
||||||
Uses built-in modules: `network`, `socket`, `time`, `json`, `base64`
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Quick Start
|
|
||||||
|
|
||||||
### Step 1: Start NATS Server
|
|
||||||
|
|
||||||
```bash
|
|
||||||
docker run -p 4222:4222 nats:latest
|
|
||||||
```
|
|
||||||
|
|
||||||
### Step 2: Start HTTP File Server (Optional)
|
|
||||||
|
|
||||||
```bash
|
|
||||||
mkdir -p /tmp/fileserver
|
|
||||||
python3 -m http.server 8080 --directory /tmp/fileserver
|
|
||||||
```
|
|
||||||
|
|
||||||
### Step 3: Send Your First Message
|
|
||||||
|
|
||||||
#### Julia
|
|
||||||
|
|
||||||
```julia
|
|
||||||
using NATSBridge
|
|
||||||
|
|
||||||
# Send a text message
|
|
||||||
data = [("message", "Hello World", "text")]
|
|
||||||
env, env_json_str = smartsend("/chat/room1", data, broker_url="nats://localhost:4222")
|
|
||||||
# env: msg_envelope_v1 struct with all metadata and payloads
|
|
||||||
# env_json_str: JSON string representation of the envelope for publishing
|
|
||||||
println("Message sent!")
|
|
||||||
|
|
||||||
# Or use is_publish=false to get envelope and JSON without publishing
|
|
||||||
env, env_json_str = smartsend("/chat/room1", data, broker_url="nats://localhost:4222", is_publish=false)
|
|
||||||
# env: msg_envelope_v1 struct
|
|
||||||
# env_json_str: JSON string for publishing to NATS
|
|
||||||
```
|
|
||||||
|
|
||||||
#### JavaScript
|
|
||||||
|
|
||||||
```javascript
|
|
||||||
const NATSBridge = require('./src/natsbridge.js');
|
|
||||||
|
|
||||||
// Send a text message
|
|
||||||
const data = [["message", "Hello World", "text"]];
|
|
||||||
const [env, env_json_str] = await NATSBridge.smartsend(
|
|
||||||
"/chat/room1",
|
|
||||||
data,
|
|
||||||
{ broker_url: "nats://localhost:4222" }
|
|
||||||
);
|
|
||||||
// env: Object with all metadata and payloads
|
|
||||||
// env_json_str: JSON string for publishing
|
|
||||||
console.log("Message sent!");
|
|
||||||
|
|
||||||
// Or use is_publish=false
|
|
||||||
const [env, env_json_str] = await NATSBridge.smartsend(
|
|
||||||
"/chat/room1",
|
|
||||||
data,
|
|
||||||
{ broker_url: "nats://localhost:4222", is_publish: false }
|
|
||||||
);
|
|
||||||
```
|
|
||||||
|
|
||||||
#### Python
|
|
||||||
|
|
||||||
```python
|
|
||||||
from natsbridge import smartsend
|
|
||||||
|
|
||||||
# Send a text message
|
|
||||||
data = [("message", "Hello World", "text")]
|
|
||||||
env, env_json_str = await smartsend(
|
|
||||||
"/chat/room1",
|
|
||||||
data,
|
|
||||||
broker_url="nats://localhost:4222"
|
|
||||||
)
|
|
||||||
# env: Dict with all metadata and payloads
|
|
||||||
# env_json_str: JSON string for publishing
|
|
||||||
print("Message sent!")
|
|
||||||
|
|
||||||
# Or use is_publish=False
|
|
||||||
env, env_json_str = await smartsend(
|
|
||||||
"/chat/room1",
|
|
||||||
data,
|
|
||||||
broker_url="nats://localhost:4222",
|
|
||||||
is_publish=False
|
|
||||||
)
|
|
||||||
# env: Dict with all metadata and payloads
|
|
||||||
# env_json_str: JSON string for publishing to NATS
|
|
||||||
```
|
|
||||||
|
|
||||||
#### MicroPython
|
|
||||||
|
|
||||||
```python
|
|
||||||
from natsbridge_mpy import NATSBridge
|
|
||||||
|
|
||||||
bridge = NATSBridge()
|
|
||||||
|
|
||||||
# Send a text message (limited to small payloads)
|
|
||||||
data = [("message", "Hello World", "text")]
|
|
||||||
env, env_json_str = bridge.smartsend(
|
|
||||||
"/chat/room1",
|
|
||||||
data,
|
|
||||||
size_threshold=100000 # Lower threshold for MicroPython
|
|
||||||
)
|
|
||||||
print("Message sent!")
|
|
||||||
```
|
|
||||||
|
|
||||||
### Step 4: Receive Messages
|
|
||||||
|
|
||||||
#### Julia
|
|
||||||
|
|
||||||
```julia
|
|
||||||
using NATSBridge
|
|
||||||
|
|
||||||
# Receive and process message
|
|
||||||
env = smartreceive(msg; fileserver_download_handler=_fetch_with_backoff)
|
|
||||||
# Returns: ::JSON.Object{String, Any} with "payloads" field containing Vector{Tuple{String, Any, String}}
|
|
||||||
# Access payloads: for (dataname, data, type) in env["payloads"]
|
|
||||||
for (dataname, data, type) in env["payloads"]
|
|
||||||
println("Received $dataname: $data")
|
|
||||||
end
|
|
||||||
```
|
|
||||||
|
|
||||||
#### JavaScript
|
|
||||||
|
|
||||||
```javascript
|
|
||||||
const NATSBridge = require('./src/natsbridge.js');
|
|
||||||
|
|
||||||
// Receive and process message
|
|
||||||
const env = await NATSBridge.smartreceive(msg, {
|
|
||||||
fileserver_download_handler: NATSBridge.fetchWithBackoff
|
|
||||||
});
|
|
||||||
// env.payloads = [[dataname, data, type], ...]
|
|
||||||
for (const [dataname, data, type] of env.payloads) {
|
|
||||||
console.log(`Received ${dataname}:`, data);
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
#### Python
|
|
||||||
|
|
||||||
```python
|
|
||||||
from natsbridge import smartreceive, fetch_with_backoff
|
|
||||||
|
|
||||||
# Receive and process message
|
|
||||||
env = await smartreceive(
|
|
||||||
msg,
|
|
||||||
fileserver_download_handler=fetch_with_backoff
|
|
||||||
)
|
|
||||||
# env["payloads"] = [(dataname, data, type), ...]
|
|
||||||
for dataname, data, type_ in env["payloads"]:
|
|
||||||
print(f"Received {dataname}: {data}")
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Basic Examples
|
|
||||||
|
|
||||||
### Example 1: Sending a Dictionary
|
|
||||||
|
|
||||||
#### Julia
|
|
||||||
|
|
||||||
```julia
|
|
||||||
using NATSBridge
|
|
||||||
|
|
||||||
config = Dict(
|
|
||||||
"wifi_ssid" => "MyNetwork",
|
|
||||||
"wifi_password" => "password123",
|
|
||||||
"update_interval" => 60
|
|
||||||
)
|
|
||||||
|
|
||||||
data = [("config", config, "dictionary")]
|
|
||||||
env, env_json_str = smartsend("/device/config", data, broker_url="nats://localhost:4222")
|
|
||||||
```
|
|
||||||
|
|
||||||
#### JavaScript
|
|
||||||
|
|
||||||
```javascript
|
|
||||||
const NATSBridge = require('./src/natsbridge.js');
|
|
||||||
|
|
||||||
const config = {
|
|
||||||
wifi_ssid: "MyNetwork",
|
|
||||||
wifi_password: "password123",
|
|
||||||
update_interval: 60
|
|
||||||
};
|
|
||||||
|
|
||||||
const data = [["config", config, "dictionary"]];
|
|
||||||
const [env, env_json_str] = await NATSBridge.smartsend(
|
|
||||||
"/device/config",
|
|
||||||
data,
|
|
||||||
{ broker_url: "nats://localhost:4222" }
|
|
||||||
);
|
|
||||||
```
|
|
||||||
|
|
||||||
#### Python
|
|
||||||
|
|
||||||
```python
|
|
||||||
from natsbridge import smartsend
|
|
||||||
|
|
||||||
config = {
|
|
||||||
"wifi_ssid": "MyNetwork",
|
|
||||||
"wifi_password": "password123",
|
|
||||||
"update_interval": 60
|
|
||||||
}
|
|
||||||
|
|
||||||
data = [("config", config, "dictionary")]
|
|
||||||
env, env_json_str = await smartsend(
|
|
||||||
"/device/config",
|
|
||||||
data,
|
|
||||||
broker_url="nats://localhost:4222"
|
|
||||||
)
|
|
||||||
```
|
|
||||||
|
|
||||||
#### MicroPython
|
|
||||||
|
|
||||||
```python
|
|
||||||
from natsbridge_mpy import NATSBridge
|
|
||||||
|
|
||||||
bridge = NATSBridge()
|
|
||||||
|
|
||||||
config = {
|
|
||||||
"wifi_ssid": "MyNetwork",
|
|
||||||
"wifi_password": "password123",
|
|
||||||
"update_interval": 60
|
|
||||||
}
|
|
||||||
|
|
||||||
data = [("config", config, "dictionary")]
|
|
||||||
env, env_json_str = bridge.smartsend(
|
|
||||||
"/device/config",
|
|
||||||
data,
|
|
||||||
size_threshold=100000
|
|
||||||
)
|
|
||||||
```
|
|
||||||
|
|
||||||
### Example 2: Sending Binary Data (Image)
|
|
||||||
|
|
||||||
#### Julia
|
|
||||||
|
|
||||||
```julia
|
|
||||||
using NATSBridge
|
|
||||||
|
|
||||||
# Read image file
|
|
||||||
image_data = read("image.png")
|
|
||||||
|
|
||||||
data = [("user_image", image_data, "binary")]
|
|
||||||
env, env_json_str = smartsend("/chat/image", data, broker_url="nats://localhost:4222")
|
|
||||||
```
|
|
||||||
|
|
||||||
#### JavaScript
|
|
||||||
|
|
||||||
```javascript
|
|
||||||
const NATSBridge = require('./src/natsbridge.js');
|
|
||||||
const fs = require('fs');
|
|
||||||
|
|
||||||
// Read image file
|
|
||||||
const image_data = fs.readFileSync('image.png');
|
|
||||||
|
|
||||||
const data = [["user_image", image_data, "binary"]];
|
|
||||||
const [env, env_json_str] = await NATSBridge.smartsend(
|
|
||||||
"/chat/image",
|
|
||||||
data,
|
|
||||||
{ broker_url: "nats://localhost:4222" }
|
|
||||||
);
|
|
||||||
```
|
|
||||||
|
|
||||||
#### Python
|
|
||||||
|
|
||||||
```python
|
|
||||||
from natsbridge import smartsend
|
|
||||||
|
|
||||||
# Read image file
|
|
||||||
with open("image.png", "rb") as f:
|
|
||||||
image_data = f.read()
|
|
||||||
|
|
||||||
data = [("user_image", image_data, "binary")]
|
|
||||||
env, env_json_str = await smartsend(
|
|
||||||
"/chat/image",
|
|
||||||
data,
|
|
||||||
broker_url="nats://localhost:4222"
|
|
||||||
)
|
|
||||||
```
|
|
||||||
|
|
||||||
#### MicroPython
|
|
||||||
|
|
||||||
```python
|
|
||||||
from natsbridge_mpy import NATSBridge
|
|
||||||
|
|
||||||
bridge = NATSBridge()
|
|
||||||
|
|
||||||
# Read image file
|
|
||||||
with open("image.png", "rb") as f:
|
|
||||||
image_data = f.read()
|
|
||||||
|
|
||||||
data = [("user_image", image_data, "binary")]
|
|
||||||
env, env_json_str = bridge.smartsend(
|
|
||||||
"/chat/image",
|
|
||||||
data,
|
|
||||||
size_threshold=100000
|
|
||||||
)
|
|
||||||
```
|
|
||||||
|
|
||||||
### Example 3: Request-Response Pattern
|
|
||||||
|
|
||||||
#### Julia (Requester)
|
|
||||||
|
|
||||||
```julia
|
|
||||||
using NATSBridge
|
|
||||||
|
|
||||||
# Send command with reply-to
|
|
||||||
data = [("command", Dict("action" => "read_sensor"), "dictionary")]
|
|
||||||
env, env_json_str = smartsend(
|
|
||||||
"/device/command",
|
|
||||||
data,
|
|
||||||
broker_url="nats://localhost:4222",
|
|
||||||
reply_to="/device/response",
|
|
||||||
reply_to_msg_id="cmd-001"
|
|
||||||
)
|
|
||||||
```
|
|
||||||
|
|
||||||
#### JavaScript (Requester)
|
|
||||||
|
|
||||||
```javascript
|
|
||||||
const NATSBridge = require('./src/natsbridge.js');
|
|
||||||
|
|
||||||
// Send command with reply-to
|
|
||||||
const data = [["command", { action: "read_sensor" }, "dictionary"]];
|
|
||||||
const [env, env_json_str] = await NATSBridge.smartsend(
|
|
||||||
"/device/command",
|
|
||||||
data,
|
|
||||||
{
|
|
||||||
broker_url: "nats://localhost:4222",
|
|
||||||
reply_to: "/device/response",
|
|
||||||
reply_to_msg_id: "cmd-001"
|
|
||||||
}
|
|
||||||
);
|
|
||||||
```
|
|
||||||
|
|
||||||
#### Python (Requester)
|
|
||||||
|
|
||||||
```python
|
|
||||||
from natsbridge import smartsend
|
|
||||||
|
|
||||||
# Send command with reply-to
|
|
||||||
data = [("command", {"action": "read_sensor"}, "dictionary")]
|
|
||||||
env, env_json_str = await smartsend(
|
|
||||||
"/device/command",
|
|
||||||
data,
|
|
||||||
broker_url="nats://localhost:4222",
|
|
||||||
reply_to="/device/response",
|
|
||||||
reply_to_msg_id="cmd-001"
|
|
||||||
)
|
|
||||||
```
|
|
||||||
|
|
||||||
#### Julia (Responder)
|
|
||||||
|
|
||||||
```julia
|
|
||||||
using NATSBridge, NATS
|
|
||||||
|
|
||||||
const SUBJECT = "/device/command"
|
|
||||||
const NATS_URL = "nats://localhost:4222"
|
|
||||||
|
|
||||||
function test_responder()
|
|
||||||
conn = NATS.connect(NATS_URL)
|
|
||||||
NATS.subscribe(conn, SUBJECT) do msg
|
|
||||||
env = smartreceive(msg, fileserver_download_handler=_fetch_with_backoff)
|
|
||||||
|
|
||||||
reply_to = env["reply_to"]
|
|
||||||
|
|
||||||
for (dataname, data, type) in env["payloads"]
|
|
||||||
if dataname == "command" && data["action"] == "read_sensor"
|
|
||||||
response = Dict("sensor_id" => "sensor-001", "value" => 42.5)
|
|
||||||
if !isempty(reply_to)
|
|
||||||
smartsend(reply_to, [("data", response, "dictionary")])
|
|
||||||
end
|
|
||||||
end
|
|
||||||
end
|
|
||||||
end
|
|
||||||
|
|
||||||
sleep(120)
|
|
||||||
NATS.drain(conn)
|
|
||||||
end
|
|
||||||
|
|
||||||
test_responder()
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Advanced Usage
|
|
||||||
|
|
||||||
### Example 4: Large Payloads (File Server)
|
|
||||||
|
|
||||||
For payloads larger than 1MB, NATSBridge automatically uses the file server:
|
|
||||||
|
|
||||||
#### Julia
|
|
||||||
|
|
||||||
```julia
|
|
||||||
using NATSBridge
|
|
||||||
|
|
||||||
# Create large data (> 1MB)
|
|
||||||
large_data = rand(UInt8, 2_000_000)
|
|
||||||
|
|
||||||
env, env_json_str = smartsend(
|
|
||||||
"/data/large",
|
|
||||||
[("large_file", large_data, "binary")],
|
|
||||||
broker_url="nats://localhost:4222",
|
|
||||||
fileserver_url="http://localhost:8080"
|
|
||||||
)
|
|
||||||
|
|
||||||
println("File uploaded to: $(env.payloads[1].data)")
|
|
||||||
# Note: For link transport, data field contains the URL string
|
|
||||||
```
|
|
||||||
|
|
||||||
#### JavaScript
|
|
||||||
|
|
||||||
```javascript
|
|
||||||
const NATSBridge = require('./src/natsbridge.js');
|
|
||||||
|
|
||||||
// Create large data (> 1MB)
|
|
||||||
const large_data = Buffer.alloc(2_000_000);
|
|
||||||
for (let i = 0; i < large_data.length; i++) {
|
|
||||||
large_data[i] = Math.floor(Math.random() * 256);
|
|
||||||
}
|
|
||||||
|
|
||||||
const [env, env_json_str] = await NATSBridge.smartsend(
|
|
||||||
"/data/large",
|
|
||||||
[["large_file", large_data, "binary"]],
|
|
||||||
{
|
|
||||||
broker_url: "nats://localhost:4222",
|
|
||||||
fileserver_url: "http://localhost:8080"
|
|
||||||
}
|
|
||||||
);
|
|
||||||
|
|
||||||
console.log("File uploaded to:", env.payloads[0].data);
|
|
||||||
// Note: For link transport, data field contains the URL string
|
|
||||||
```
|
|
||||||
|
|
||||||
#### Python
|
|
||||||
|
|
||||||
```python
|
|
||||||
from natsbridge import smartsend
|
|
||||||
|
|
||||||
# Create large data (> 1MB)
|
|
||||||
import os
|
|
||||||
large_data = os.urandom(2_000_000)
|
|
||||||
|
|
||||||
env, env_json_str = await smartsend(
|
|
||||||
"/data/large",
|
|
||||||
[("large_file", large_data, "binary")],
|
|
||||||
broker_url="nats://localhost:4222",
|
|
||||||
fileserver_url="http://localhost:8080"
|
|
||||||
)
|
|
||||||
|
|
||||||
print(f"File uploaded to: {env['payloads'][0]['data']}")
|
|
||||||
# Note: For link transport, data field contains the URL string
|
|
||||||
```
|
|
||||||
|
|
||||||
#### MicroPython
|
|
||||||
|
|
||||||
MicroPython enforces a hard limit of 50KB per payload:
|
|
||||||
|
|
||||||
```python
|
|
||||||
from natsbridge_mpy import NATSBridge
|
|
||||||
|
|
||||||
bridge = NATSBridge()
|
|
||||||
|
|
||||||
# MicroPython has a hard limit of 50KB per payload
|
|
||||||
# Use streaming or chunking for larger data
|
|
||||||
small_data = bytes(1000) # 1KB
|
|
||||||
|
|
||||||
data = [("small_file", small_data, "binary")]
|
|
||||||
env, env_json_str = bridge.smartsend(
|
|
||||||
"/data/small",
|
|
||||||
data,
|
|
||||||
size_threshold=100000 # Enforced max: 50000 bytes
|
|
||||||
)
|
|
||||||
```
|
|
||||||
|
|
||||||
### Example 5: Mixed Content (Chat with Text + Image)
|
|
||||||
|
|
||||||
NATSBridge supports sending multiple payloads with different types in a single message:
|
|
||||||
|
|
||||||
#### Julia
|
|
||||||
|
|
||||||
```julia
|
|
||||||
using NATSBridge
|
|
||||||
|
|
||||||
image_data = read("avatar.png")
|
|
||||||
|
|
||||||
data = [
|
|
||||||
("message_text", "Hello with image!", "text"),
|
|
||||||
("user_avatar", image_data, "image")
|
|
||||||
]
|
|
||||||
|
|
||||||
env, env_json_str = smartsend("/chat/mixed", data, broker_url="nats://localhost:4222")
|
|
||||||
```
|
|
||||||
|
|
||||||
#### JavaScript
|
|
||||||
|
|
||||||
```javascript
|
|
||||||
const NATSBridge = require('./src/natsbridge.js');
|
|
||||||
const fs = require('fs');
|
|
||||||
|
|
||||||
const image_data = fs.readFileSync('avatar.png');
|
|
||||||
|
|
||||||
const data = [
|
|
||||||
["message_text", "Hello with image!", "text"],
|
|
||||||
["user_avatar", image_data, "image"]
|
|
||||||
];
|
|
||||||
|
|
||||||
const [env, env_json_str] = await NATSBridge.smartsend(
|
|
||||||
"/chat/mixed",
|
|
||||||
data,
|
|
||||||
{ broker_url: "nats://localhost:4222" }
|
|
||||||
);
|
|
||||||
```
|
|
||||||
|
|
||||||
#### Python
|
|
||||||
|
|
||||||
```python
|
|
||||||
from natsbridge import smartsend
|
|
||||||
|
|
||||||
with open("avatar.png", "rb") as f:
|
|
||||||
image_data = f.read()
|
|
||||||
|
|
||||||
data = [
|
|
||||||
("message_text", "Hello with image!", "text"),
|
|
||||||
("user_avatar", image_data, "image")
|
|
||||||
]
|
|
||||||
|
|
||||||
env, env_json_str = await smartsend(
|
|
||||||
"/chat/mixed",
|
|
||||||
data,
|
|
||||||
broker_url="nats://localhost:4222"
|
|
||||||
)
|
|
||||||
# env: Dict with all metadata and payloads
|
|
||||||
```
|
|
||||||
|
|
||||||
### Example 6: Table Data (Arrow IPC)
|
|
||||||
|
|
||||||
For tabular data, NATSBridge uses Apache Arrow IPC format:
|
|
||||||
|
|
||||||
#### Julia
|
|
||||||
|
|
||||||
```julia
|
|
||||||
using NATSBridge
|
|
||||||
using DataFrames
|
|
||||||
|
|
||||||
# Create DataFrame
|
|
||||||
df = DataFrame(
|
|
||||||
id = [1, 2, 3],
|
|
||||||
name = ["Alice", "Bob", "Charlie"],
|
|
||||||
score = [95, 88, 92]
|
|
||||||
)
|
|
||||||
|
|
||||||
data = [("students", df, "arrowtable")]
|
|
||||||
env, env_json_str = smartsend("/data/students", data, broker_url="nats://localhost:4222")
|
|
||||||
```
|
|
||||||
|
|
||||||
#### JavaScript
|
|
||||||
|
|
||||||
```javascript
|
|
||||||
const NATSBridge = require('./src/natsbridge.js');
|
|
||||||
|
|
||||||
// Create table data (array of objects)
|
|
||||||
const table_data = [
|
|
||||||
{ id: 1, name: "Alice", score: 95 },
|
|
||||||
{ id: 2, name: "Bob", score: 88 },
|
|
||||||
{ id: 3, name: "Charlie", score: 92 }
|
|
||||||
];
|
|
||||||
|
|
||||||
const data = [["students", table_data, "arrowtable"]];
|
|
||||||
const [env, env_json_str] = await NATSBridge.smartsend(
|
|
||||||
"/data/students",
|
|
||||||
data,
|
|
||||||
{ broker_url: "nats://localhost:4222" }
|
|
||||||
);
|
|
||||||
```
|
|
||||||
|
|
||||||
#### Python
|
|
||||||
|
|
||||||
```python
|
|
||||||
from natsbridge import smartsend
|
|
||||||
import pandas as pd
|
|
||||||
|
|
||||||
# Create DataFrame
|
|
||||||
df = pd.DataFrame({
|
|
||||||
'id': [1, 2, 3],
|
|
||||||
'name': ['Alice', 'Bob', 'Charlie'],
|
|
||||||
'score': [95, 88, 92]
|
|
||||||
})
|
|
||||||
|
|
||||||
data = [("students", df, "table")]
|
|
||||||
env, env_json_str = await smartsend(
|
|
||||||
"/data/students",
|
|
||||||
data,
|
|
||||||
broker_url="nats://localhost:4222"
|
|
||||||
)
|
|
||||||
```
|
|
||||||
|
|
||||||
#### MicroPython
|
|
||||||
|
|
||||||
MicroPython does not support table type due to memory constraints. Use dictionary or binary instead.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Next Steps
|
|
||||||
|
|
||||||
1. **Explore the test directory** for more examples
|
|
||||||
2. **Check the documentation** for advanced configuration options
|
|
||||||
3. **Read the walkthrough** for building real-world applications
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Troubleshooting
|
|
||||||
|
|
||||||
### Connection Issues
|
|
||||||
|
|
||||||
- Ensure NATS server is running: `docker ps | grep nats`
|
|
||||||
- Check firewall settings
|
|
||||||
- Verify NATS URL configuration
|
|
||||||
|
|
||||||
### File Server Issues
|
|
||||||
|
|
||||||
- Ensure file server is running and accessible
|
|
||||||
- Check upload permissions
|
|
||||||
- Verify file server URL configuration
|
|
||||||
|
|
||||||
### Serialization Errors
|
|
||||||
|
|
||||||
- Verify data type matches the specified type
|
|
||||||
- Check that binary data is in the correct format
|
|
||||||
- MicroPython: Ensure payload size < 50KB
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## License
|
|
||||||
|
|
||||||
MIT
|
|
||||||
Reference in New Issue
Block a user