update readme

This commit is contained in:
2026-03-14 10:41:16 +07:00
parent 50211b671d
commit 3c6e139ac0
2 changed files with 180 additions and 789 deletions

228
README.md
View File

@@ -1,6 +1,6 @@
# NATSBridge - Cross-Platform Bi-Directional Data Bridge
A high-performance, bi-directional data bridge for **Julia, JavaScript, Python, and MicroPython** applications using NATS (Core & JetStream), implementing the Claim-Check pattern for large payloads.
A high-performance, bi-directional data bridge for **Julia**, **JavaScript**, **Python**, and **MicroPython** applications using NATS (Core & JetStream), implementing the Claim-Check pattern for large payloads.
[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](https://opensource.org/licenses/MIT)
[![NATS](https://img.shields.io/badge/NATS-Enabled-green.svg)](https://nats.io)
@@ -28,8 +28,8 @@ NATSBridge enables seamless communication across multiple platforms through NATS
| Transport | Payload Size | Method |
|-----------|--------------|--------|
| **Direct** | < 1MB | Sent directly via NATS (Base64 encoded) |
| **Link** | >= 1MB | Uploaded to HTTP file server, URL sent via NATS |
| **Direct** | < 500KB | Sent directly via NATS (Base64 encoded) |
| **Link** | ≥ 500KB | Uploaded to HTTP file server, URL sent via NATS |
### Use Cases
@@ -45,7 +45,7 @@ NATSBridge enables seamless communication across multiple platforms through NATS
| Platform | Implementation | Features |
|----------|----------------|----------|
| **Julia** | [`src/NATSBridge.jl`](src/NATSBridge.jl) | Full feature set, Arrow IPC, multiple dispatch |
| **JavaScript** | [`src/natsbridge.js`](src/natsbridge.js) | Node.js, async/await |
| **JavaScript (Node.js)** | [`src/natsbridge.js`](src/natsbridge_ssr.js) | Node.js, async/await |
| **JavaScript (Browser)** | [`src/natsbridge_csr.js`](src/natsbridge_csr.js) | Browser, WebSocket NATS, async/await |
| **Python** | [`src/natsbridge.py`](src/natsbridge.py) | Desktop Python, asyncio, type hints |
| **MicroPython** | [`src/natsbridge_mpy.py`](src/natsbridge_mpy.py) | Memory-constrained, synchronous API |
@@ -72,7 +72,7 @@ NATSBridge enables seamless communication across multiple platforms through NATS
-**Bi-directional messaging** with request-reply patterns
-**Multi-payload support** - send multiple payloads with different types in one message
-**Automatic transport selection** - direct vs link based on payload size
-**Claim-Check pattern** for payloads > 1MB
-**Claim-Check pattern** for payloads ≥ 500KB
-**Apache Arrow IPC** support for tabular data (zero-copy reading)
-**Exponential backoff** for reliable file server downloads
-**Correlation ID tracking** for message tracing
@@ -83,23 +83,24 @@ NATSBridge enables seamless communication across multiple platforms through NATS
## Quick Start
### Step 1: Start NATS Server
### Prerequisites
```bash
docker run -p 4222:4222 nats:latest
```
1. **NATS Server** - Install and run a NATS server:
```bash
docker run -p 4222:4222 nats:latest
```
### Step 2: Start HTTP File Server (Optional)
2. **HTTP File Server** (optional, for large payloads) - Install and run a file server:
```bash
# Using Plik
docker run -p 8080:8080 -v /tmp/fileserver:/var/lib/plik -e PLIK_ADMIN_PASSWORD=admin plik/plik
# OR using simple Python HTTP server
mkdir -p /tmp/fileserver
python3 -m http.server 8080 --directory /tmp/fileserver
```
```bash
# Create a directory for file uploads
mkdir -p /tmp/fileserver
# Start HTTP file server
python3 -m http.server 8080 --directory /tmp/fileserver
```
### Step 3: Send Your First Message
### Send Your First Message
#### Julia
@@ -107,14 +108,14 @@ python3 -m http.server 8080 --directory /tmp/fileserver
using NATSBridge
data = [("message", "Hello World", "text")]
env, env_json_str = smartsend("/chat/room1", data, broker_url="nats://localhost:4222")
env, env_json_str = smartsend("/chat/room1", data; broker_url="nats://localhost:4222")
println("Message sent!")
```
#### JavaScript
#### JavaScript (Node.js)
```javascript
const NATSBridge = require('./src/natsbridge.js');
import NATSBridge from './src/natsbridge_ssr.js';
const data = [["message", "Hello World", "text"]];
const [env, env_json_str] = await NATSBridge.smartsend(
@@ -125,6 +126,20 @@ const [env, env_json_str] = await NATSBridge.smartsend(
console.log("Message sent!");
```
#### JavaScript (Browser)
```javascript
import NATSBridge from './src/natsbridge_csr.js';
const data = [["message", "Hello World", "text"]];
const [env, env_json_str] = await NATSBridge.smartsend(
"/chat/room1",
data,
{ broker_url: "ws://localhost:4222" }
);
console.log("Message sent!");
```
#### Python
```python
@@ -139,6 +154,21 @@ env, env_json_str = await smartsend(
print("Message sent!")
```
#### MicroPython
```python
from natsbridge import smartsend
data = [("message", "Hello World", "text")]
env, env_json_str = smartsend(
"/chat/room1",
data,
broker_url="nats://localhost:4222",
size_threshold=100000 # 100KB for MicroPython
)
print("Message sent!")
```
---
## API Reference
@@ -147,13 +177,13 @@ print("Message sent!")
All platforms use the same input/output format for payloads:
**Input format for smartsend:**
**Input format for `smartsend`:**
```
[(dataname1, data1, type1), (dataname2, data2, type2), ...]
```
**Output format for smartreceive:**
```
**Output format for `smartreceive`:**
```json
{
"correlation_id": "...",
"msg_id": "...",
@@ -187,7 +217,7 @@ env, env_json_str = NATSBridge.smartsend(
broker_url::String = "nats://localhost:4222",
fileserver_url = "http://localhost:8080",
fileserver_upload_handler::Function = plik_oneshot_upload,
size_threshold::Int = 1_000_000,
size_threshold::Int = 500_000,
correlation_id::String = string(uuid4()),
msg_purpose::String = "chat",
sender_name::String = "NATSBridge",
@@ -203,10 +233,10 @@ env, env_json_str = NATSBridge.smartsend(
# Returns: ::Tuple{msg_envelope_v1, String}
```
#### JavaScript
#### JavaScript (Node.js)
```javascript
const NATSBridge = require('natsbridge');
import NATSBridge from './src/natsbridge_ssr.js';
const [env, env_json_str] = await NATSBridge.smartsend(
subject,
@@ -215,7 +245,36 @@ const [env, env_json_str] = await NATSBridge.smartsend(
broker_url: 'nats://localhost:4222',
fileserver_url: 'http://localhost:8080',
fileserver_upload_handler: NATSBridge.plikOneshotUpload,
size_threshold: 1_000_000,
size_threshold: 500_000,
correlation_id: uuidv4(),
msg_purpose: 'chat',
sender_name: 'NATSBridge',
receiver_name: '',
receiver_id: '',
reply_to: '',
reply_to_msg_id: '',
is_publish: true,
nats_connection: null,
msg_id: uuidv4(),
sender_id: uuidv4()
}
);
// Returns: Promise<[env, env_json_str]>
```
#### JavaScript (Browser)
```javascript
import NATSBridge from './src/natsbridge_csr.js';
const [env, env_json_str] = await NATSBridge.smartsend(
subject,
data,
{
broker_url: 'ws://localhost:4222',
fileserver_url: 'http://localhost:8080',
fileserver_upload_handler: NATSBridge.plikOneshotUpload,
size_threshold: 500_000,
correlation_id: uuidv4(),
msg_purpose: 'chat',
sender_name: 'NATSBridge',
@@ -243,7 +302,7 @@ env, env_json_str = await NATSBridge.smartsend(
broker_url: str = "nats://localhost:4222",
fileserver_url: str = "http://localhost:8080",
fileserver_upload_handler: Callable = plik_oneshot_upload,
size_threshold: int = 1_000_000,
size_threshold: int = 500_000,
correlation_id: str = None,
msg_purpose: str = "chat",
sender_name: str = "NATSBridge",
@@ -293,9 +352,28 @@ env = NATSBridge.smartreceive(
# Returns: ::JSON.Object{String, Any}
```
#### JavaScript
#### JavaScript (Node.js)
```javascript
import NATSBridge from './src/natsbridge_ssr.js';
const env = await NATSBridge.smartreceive(
msg,
{
fileserver_download_handler: NATSBridge.fetchWithBackoff,
max_retries: 5,
base_delay: 100,
max_delay: 5000
}
);
// Returns: Promise<env_object>
```
#### JavaScript (Browser)
```javascript
import NATSBridge from './src/natsbridge_csr.js';
const env = await NATSBridge.smartreceive(
msg,
{
@@ -311,6 +389,8 @@ const env = await NATSBridge.smartreceive(
#### Python
```python
from natsbridge import NATSBridge
env = await NATSBridge.smartreceive(
msg,
fileserver_download_handler=fetch_with_backoff,
@@ -324,6 +404,8 @@ env = await NATSBridge.smartreceive(
#### MicroPython
```python
from natsbridge import NATSBridge
env = NATSBridge.smartreceive(
msg,
fileserver_download_handler=_sync_fileserver_download,
@@ -343,7 +425,7 @@ env = NATSBridge.smartreceive(
| `text` | `String` | `string` | `str` | `str` | Plain text strings |
| `dictionary` | `Dict`, `NamedTuple` | `Object`, `Array` | `dict`, `list` | `dict` | JSON-serializable dictionaries |
| `arrowtable` | `DataFrame`, `Arrow.Table` | `Array<Object>` | `pandas.DataFrame` | ❌ | Tabular data (Arrow IPC) |
| `jsontable` | `Vector{NamedTuple}` | `Array<Object>` | `list[dict]` | | Tabular data (JSON) |
| `jsontable` | `Vector{NamedTuple}` | `Array<Object>` | `list[dict]` | ⚠️ | Tabular data (JSON) |
| `image` | `Vector{UInt8}` | `Uint8Array`, `Buffer` | `bytes` | `bytearray` | Image data (PNG, JPG) |
| `audio` | `Vector{UInt8}` | `Uint8Array`, `Buffer` | `bytes` | `bytearray` | Audio data (WAV, MP3) |
| `video` | `Vector{UInt8}` | `Uint8Array`, `Buffer` | `bytes` | `bytearray` | Video data (MP4, AVI) |
@@ -368,13 +450,13 @@ data = [
("large_document", large_file_data, "binary")
]
env, env_json_str = NATSBridge.smartsend("/chat/room1", data; fileserver_url="http://localhost:8080")
env, env_json_str = smartsend("/chat/room1", data; fileserver_url="http://localhost:8080")
```
#### JavaScript
#### JavaScript (Node.js)
```javascript
const NATSBridge = require('natsbridge');
import NATSBridge from './src/natsbridge_ssr.js';
const data = [
["message_text", "Hello!", "text"],
@@ -389,6 +471,24 @@ const [env, env_json_str] = await NATSBridge.smartsend(
);
```
#### JavaScript (Browser)
```javascript
import NATSBridge from './src/natsbridge_csr.js';
const data = [
["message_text", "Hello!", "text"],
["user_avatar", imageData, "image"],
["large_document", largeFileData, "binary"]
];
const [env, env_json_str] = await NATSBridge.smartsend(
"/chat/room1",
data,
{ broker_url: 'ws://localhost:4222', fileserver_url: 'http://localhost:8080' }
);
```
#### Python
```python
@@ -423,13 +523,13 @@ config = Dict(
)
data = [("config", config, "dictionary")]
env, env_json_str = NATSBridge.smartsend("/device/config", data)
env, env_json_str = smartsend("/device/config", data)
```
#### JavaScript
#### JavaScript (Node.js)
```javascript
const NATSBridge = require('natsbridge');
import NATSBridge from './src/natsbridge_ssr.js';
const config = {
wifi_ssid: "MyNetwork",
@@ -475,13 +575,13 @@ df = DataFrame(
)
data = [("students", df, "arrowtable")]
env, env_json_str = NATSBridge.smartsend("/data/analysis", data)
env, env_json_str = smartsend("/data/analysis", data)
```
#### JavaScript
#### JavaScript (Node.js)
```javascript
const NATSBridge = require('natsbridge');
import NATSBridge from './src/natsbridge_ssr.js';
const df = [
{ id: 1, name: "Alice", score: 95 },
@@ -521,18 +621,29 @@ Bi-directional communication with reply-to support.
using NATSBridge
# Requester
env, env_json_str = NATSBridge.smartsend(
env, env_json_str = smartsend(
"/device/command",
[("command", Dict("action" => "read_sensor"), "dictionary")];
broker_url="nats://localhost:4222",
reply_to="/device/response"
)
# Receiver (in separate application)
msg = NATS.subscription.next()
env = smartreceive(msg)
# Process request and send response
response_env, response_json = smartsend(
"/device/response",
[("result", Dict("value" => 42), "dictionary")],
reply_to="/device/command",
reply_to_msg_id=env["msg_id"]
)
```
#### JavaScript
#### JavaScript (Node.js)
```javascript
const NATSBridge = require('natsbridge');
import NATSBridge from './src/natsbridge_ssr.js';
// Requester
const [env, env_json_str] = await NATSBridge.smartsend(
@@ -540,6 +651,16 @@ const [env, env_json_str] = await NATSBridge.smartsend(
[["command", { action: "read_sensor" }, "dictionary"]],
{ broker_url: 'nats://localhost:4222', reply_to: '/device/response' }
);
// Receiver (in separate application)
// const msg = await natsConsumer.next();
// const env = await NATSBridge.smartreceive(msg);
// Process request and send response
// const response_env, response_json = await NATSBridge.smartsend(
// "/device/response",
// [["result", { value: 42 }, "dictionary"]],
// { reply_to: '/device/command', reply_to_msg_id: env.msg_id }
// );
```
#### Python
@@ -554,6 +675,17 @@ env, env_json_str = await NATSBridge.smartsend(
broker_url="nats://localhost:4222",
reply_to="/device/response"
)
# Receiver (in separate application)
# msg = await nats_consumer.next()
# env = await NATSBridge.smartreceive(msg)
# Process request and send response
# response_env, response_json = await NATSBridge.smartsend(
# "/device/response",
# [("result", {"value": 42}, "dictionary")],
# reply_to="/device/command",
# reply_to_msg_id=env["msg_id"]
# )
```
---
@@ -640,10 +772,10 @@ python3 test/test_py_table_receiver.py
For detailed architecture and implementation information, see:
- [Architecture Documentation](docs/architecture_updated.md) - Cross-platform architecture, API parity, platform-specific patterns
- [Implementation Guide](docs/implementation_updated.md) - Detailed implementation for each platform, handler functions, testing
- [Tutorial](docs/tutorial_updated.md) - Step-by-step getting started guide
- [Walkthrough](docs/walkthrough_updated.md) - Real-world application building guides
- [`docs/architecture.md`](docs/architecture.md) - Cross-platform architecture, API parity, platform-specific patterns
- [`docs/requirements.md`](docs/requirements.md) - Business requirements and user stories
- [`docs/spec.md`](docs/spec.md) - Technical specification and contracts
- [`docs/walkthrough.md`](docs/walkthrough.md) - Real-world application building guides
---