ton/NATSBridge
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity

update

Browse Source
This commit is contained in:
narawat
2026-02-19 18:41:39 +07:00
parent e7c5e5f77f
commit 7dc7ab67e4
7 changed files with 1813 additions and 229 deletions
Download Patch File Download Diff File Expand all files Collapse all files

187
docs/IMPLEMENTATION.md → docs/implementation.md
View File

@@ -10,24 +10,26 @@ The implementation uses a **standardized list-of-tuples format** for all payload
**API Standard:**
```julia
# Input format for smartsend (always a list of tuples)
[(dataname1, data1), (dataname2, data2), ...]
# Input format for smartsend (always a list of tuples with type info)
[(dataname1, data1, type1), (dataname2, data2, type2), ...]
# Output format for smartreceive (always returns a list of tuples)
[(dataname1, data1), (dataname2, data2), ...]
# Output format for smartreceive (always returns a list of tuples with type info)
[(dataname1, data1, type1), (dataname2, data2, type2), ...]
```
Where `type` can be: `"text"`, `"dictionary"`, `"table"`, `"image"`, `"audio"`, `"video"`, `"binary"`
**Examples:**
```julia
# Single payload - still wrapped in a list
smartsend("/test", [(dataname1, data1)], ...)
# Single payload - still wrapped in a list (type is required as third element)
smartsend("/test", [(dataname1, data1, "text")], ...)
# Multiple payloads in one message
smartsend("/test", [(dataname1, data1), (dataname2, data2)], ...)
# Multiple payloads in one message (each payload has its own type)
smartsend("/test", [(dataname1, data1, "dictionary"), (dataname2, data2, "table")], ...)
# Receive always returns a list
# Receive always returns a list with type info
payloads = smartreceive(msg, ...)
# payloads = [(dataname1, data1), (dataname2, data2), ...]
# payloads = [(dataname1, data1, "text"), (dataname2, data2, "table"), ...]
```
## Architecture
@@ -139,26 +141,26 @@ node test/scenario3_julia_to_julia.js
```julia
using NATSBridge
# Send multiple payloads in one message
# Send multiple payloads in one message (type is required per payload)
smartsend(
"/test",
[("dataname1", data1), ("dataname2", data2)],
[("dataname1", data1, "dictionary"), ("dataname2", data2, "table")],
nats_url="nats://localhost:4222",
fileserver_url="http://localhost:8080/upload",
fileserver_url="http://localhost:8080",
metadata=Dict("custom_key" => "custom_value")
)
# Even single payload must be wrapped in a list
smartsend("/test", [("single_data", mydata)])
# Even single payload must be wrapped in a list with type
smartsend("/test", [("single_data", mydata, "dictionary")])
```
#### Julia (Receiver)
```julia
using NATSBridge
# Receive returns a list of payloads
payloads = smartreceive(msg, "http://localhost:8080/upload")
# payloads = [(dataname1, data1), (dataname2, data2), ...]
# Receive returns a list of payloads with type info
payloads = smartreceive(msg, "http://localhost:8080")
# payloads = [(dataname1, data1, "dictionary"), (dataname2, data2, "table"), ...]
```
### Scenario 1: Command & Control (Small JSON)
@@ -229,8 +231,8 @@ df = DataFrame(
category = rand(["A", "B", "C"], 10_000_000)
)
# Send via SmartSend - wrapped in a list
await SmartSend("analysis_results", [("table_data", df)], "table");
# Send via SmartSend - wrapped in a list (type is part of each tuple)
await SmartSend("analysis_results", [("table_data", df, "table")]);
```
#### JavaScript (Receiver)
@@ -287,9 +289,9 @@ end
using NATSBridge
function publish_health_status(nats_url)
# Send status wrapped in a list
# Send status wrapped in a list (type is part of each tuple)
status = Dict("cpu" => rand(), "memory" => rand())
smartsend("health", [("status", status)], "json", nats_url=nats_url)
smartsend("health", [("status", status, "dictionary")], nats_url=nats_url)
sleep(5) # Every 5 seconds
end
```
@@ -317,6 +319,145 @@ for await (const msg of consumer) {
}
```
### Scenario 5: Selection (Low Bandwidth)
**Focus:** Small Arrow tables, Julia to JavaScript. The Action: Julia wants to send a small DataFrame to show on a JavaScript dashboard for the user to choose.
**Julia (Sender):**
```julia
using NATSBridge
using DataFrames
# Create small DataFrame (e.g., 50KB - 500KB)
options_df = DataFrame(
id = 1:10,
name = ["Option A", "Option B", "Option C", "Option D", "Option E",
"Option F", "Option G", "Option H", "Option I", "Option J"],
description = ["Description A", "Description B", "Description C", "Description D", "Description E",
"Description F", "Description G", "Description H", "Description I", "Description J"]
)
# Convert to Arrow IPC stream
# Check payload size (< 1MB threshold)
# Publish directly to NATS with Base64-encoded payload
# Include metadata for dashboard selection context
smartsend(
"dashboard.selection",
[("options_table", options_df, "table")],
nats_url="nats://localhost:4222",
metadata=Dict("context" => "user_selection")
)
```
**JavaScript (Receiver):**
```javascript
const { SmartReceive } = require('./js_bridge');
// Receive NATS message with direct transport
const result = await SmartReceive(msg);
// Decode Base64 payload
// Parse Arrow IPC with zero-copy
// Load into selection UI component (e.g., dropdown, table)
const table = result[2]; // Get the DataFrame from the tuple
// User makes selection
const selection = uiComponent.getSelectedOption();
// Send selection back to Julia
await SmartSend("dashboard.response", [
("selected_option", selection, "dictionary")
]);
```
**Use Case:** Julia server generates a list of available options (e.g., file selections, configuration presets) as a small DataFrame and sends to JavaScript dashboard for user selection. The selection is then sent back to Julia for processing.
### Scenario 6: Chat System
**Focus:** Every conversational message is composed of any number and any combination of components, spanning the full spectrum from small to large. This includes text, images, audio, video, tables, and files—specifically accommodating everything from brief snippets to high-resolution images, large audio files, extensive tables, and massive documents. Support for claim-check delivery and full bi-directional messaging.
**Multi-Payload Support:** The system supports mixed-payload messages where a single message can contain multiple payloads with different transport strategies. The `smartreceive` function iterates through all payloads in the envelope and processes each according to its transport type.
**Julia (Sender/Receiver):**
```julia
using NATSBridge
using DataFrames
# Build chat message with mixed payloads:
# - Text: direct transport (Base64)
# - Small images: direct transport (Base64)
# - Large images: link transport (HTTP URL)
# - Audio/video: link transport (HTTP URL)
# - Tables: direct or link depending on size
# - Files: link transport (HTTP URL)
#
# Each payload uses appropriate transport strategy:
# - Size < 1MB → direct (NATS + Base64)
# - Size >= 1MB → link (HTTP upload + NATS URL)
#
# Include claim-check metadata for delivery tracking
# Support bidirectional messaging with replyTo fields
# Example: Chat with text, small image, and large file
chat_message = [
("message_text", "Hello, this is a test message!", "text"),
("user_avatar", image_bytes, "image"), # Small image, direct transport
("large_document", large_file_bytes, "binary") # Large file, link transport
]
smartsend(
"chat.room123",
chat_message,
nats_url="nats://localhost:4222",
msg_purpose="chat",
reply_to="chat.room123.responses"
)
```
**JavaScript (Sender/Receiver):**
```javascript
const { SmartSend, SmartReceive } = require('./js_bridge');
// Build chat message with mixed content:
// - User input text: direct transport
// - Selected image: check size, use appropriate transport
// - Audio recording: link transport for large files
// - File attachment: link transport
//
// Parse received message:
// - Direct payloads: decode Base64
// - Link payloads: fetch from HTTP with exponential backoff
// - Deserialize all payloads appropriately
//
// Render mixed content in chat interface
// Support bidirectional reply with claim-check delivery confirmation
// Example: Send chat with mixed content
const message = [
{
dataname: "text",
data: "Hello from JavaScript!",
type: "text"
},
{
dataname: "image",
data: selectedImageBuffer, // Small image
type: "image"
},
{
dataname: "audio",
data: audioUrl, // Large audio, link transport
type: "audio"
}
];
await SmartSend("chat.room123", message);
```
**Use Case:** Full-featured chat system supporting rich media. User can send text, small images directly, or upload large files that get uploaded to HTTP server and referenced via URLs. Claim-check pattern ensures reliable delivery tracking for all message components.
**Implementation Note:** The `smartreceive` function iterates through all payloads in the envelope and processes each according to its transport type. See the standard API format in Section 1: `msgEnvelope_v1` supports `AbstractArray{msgPayload_v1}` for multiple payloads.
## Configuration
### Environment Variables
@@ -324,7 +465,7 @@ for await (const msg of consumer) {
| Variable | Default | Description |
|----------|---------|-------------|
| `NATS_URL` | `nats://localhost:4222` | NATS server URL |
| `FILESERVER_URL` | `http://localhost:8080/upload` | HTTP file server URL |
| `FILESERVER_URL` | `http://localhost:8080` | HTTP file server URL (base URL without `/upload` suffix) |
| `SIZE_THRESHOLD` | `1_000_000` | Size threshold in bytes (1MB) |
### Message Envelope Schema