Compare commits
1 Commits
6da64092ca
...
v0.6.0-dev
| Author | SHA1 | Date | |
|---|---|---|---|
| 2a83c35852 |
@@ -1,7 +1,7 @@
|
|||||||
# Architecture Documentation: NATSBridge
|
# Architecture Documentation: NATSBridge
|
||||||
|
|
||||||
**Version**: 1.2.0
|
**Version**: 1.4.0
|
||||||
**Date**: 2026-05-13
|
**Date**: 2026-05-14
|
||||||
**Status**: Active
|
**Status**: Active
|
||||||
**Ground Truth**: [`src/NATSBridge.jl`](../src/NATSBridge.jl)
|
**Ground Truth**: [`src/NATSBridge.jl`](../src/NATSBridge.jl)
|
||||||
**Architecture Level**: C4 Container Level
|
**Architecture Level**: C4 Container Level
|
||||||
@@ -182,6 +182,7 @@ flowchart TD
|
|||||||
| **log_trace** | Log trace messages with correlation ID | All |
|
| **log_trace** | Log trace messages with correlation ID | All |
|
||||||
| **fileserver_upload_handler** | Upload large payloads to HTTP server | Desktop (Julia/JS/Python/Dart/Rust) |
|
| **fileserver_upload_handler** | Upload large payloads to HTTP server | Desktop (Julia/JS/Python/Dart/Rust) |
|
||||||
| **fileserver_download_handler** | Download payloads from HTTP server with exponential backoff | Desktop (Julia/JS/Python/Dart/Rust) |
|
| **fileserver_download_handler** | Download payloads from HTTP server with exponential backoff | Desktop (Julia/JS/Python/Dart/Rust) |
|
||||||
|
| **plik_upload_file** | Upload a local file to Plik server from disk | Rust |
|
||||||
|
|
||||||
### Data Flow
|
### Data Flow
|
||||||
|
|
||||||
@@ -563,7 +564,7 @@ pub enum Payload {
|
|||||||
pub struct SmartsendOptions {
|
pub struct SmartsendOptions {
|
||||||
pub broker_url: String,
|
pub broker_url: String,
|
||||||
pub fileserver_url: String,
|
pub fileserver_url: String,
|
||||||
pub fileserver_upload_handler: Option<UploadHandler>,
|
pub fileserver_upload_handler: Option<Arc<dyn FileUploadHandler>>,
|
||||||
pub size_threshold: usize,
|
pub size_threshold: usize,
|
||||||
pub correlation_id: String,
|
pub correlation_id: String,
|
||||||
pub msg_purpose: String,
|
pub msg_purpose: String,
|
||||||
@@ -577,13 +578,14 @@ let conn = nats::connect("nats://localhost:4222").await?;
|
|||||||
// Subscribe and process messages
|
// Subscribe and process messages
|
||||||
let mut sub = conn.subscribe("/agent/wine/api/v1/analyze")?;
|
let mut sub = conn.subscribe("/agent/wine/api/v1/analyze")?;
|
||||||
for msg in sub.messages() {
|
for msg in sub.messages() {
|
||||||
let envelope: MsgEnvelopeV1 = serde_json::from_slice(&msg.payload)?;
|
let envelope = smartreceive(&String::from_utf8_lossy(&msg.payload), &Default::default()).await?;
|
||||||
// Type-safe access to payloads
|
// Access deserialized payloads by type
|
||||||
for payload in &envelope.payloads {
|
for payload in &envelope.payloads {
|
||||||
match &payload.data {
|
match payload.payload_type.as_str() {
|
||||||
Payload::ArrowTable(bytes) => { /* process */ },
|
"arrowtable" => { /* payload.data is base64-encoded Arrow IPC */ },
|
||||||
Payload::Text(text) => { /* process */ },
|
"text" => { /* payload.data is decoded text string */ },
|
||||||
_ => {}
|
"binary" | "image" | "audio" | "video" => { /* payload.data is base64-encoded binary */ },
|
||||||
|
_ => { /* other types */ }
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -833,6 +835,13 @@ flowchart TD
|
|||||||
|
|
||||||
| Date | Version | Changes |
|
| Date | Version | Changes |
|
||||||
|------|---------|---------|
|
|------|---------|---------|
|
||||||
|
| 2026-05-14 | 1.4.0 | Updated Rust API to reflect `smartreceive` deserialization changes | All sections |
|
||||||
|
| - | - | `smartreceive` now stores deserialized data in `MsgPayloadV1.data` | specification.md:8 |
|
||||||
|
| - | - | Added `plik_upload_file` convenience function to component table | specification.md:13 |
|
||||||
|
| - | - | Fixed Rust payload access pattern (data is String, not Payload enum) | All sections |
|
||||||
|
| - | - | Fixed `SmartsendOptions.fileserver_upload_handler` type to `Arc<dyn FileUploadHandler>` | specification.md:13 |
|
||||||
|
| - | - | Removed `metadata` from link transport examples (now `None`/omitted) | specification.md:3 |
|
||||||
|
| - | - | Removed duplicate footer text | All sections |
|
||||||
| 2026-05-13 | 1.3.0 | Added Rust support with tokio, serde, and arrow2 | All sections |
|
| 2026-05-13 | 1.3.0 | Added Rust support with tokio, serde, and arrow2 | All sections |
|
||||||
| - | - | Added Rust to C4 diagrams (context, container) | All sections |
|
| - | - | Added Rust to C4 diagrams (context, container) | All sections |
|
||||||
| - | - | Added Rust platform-specific architecture section | specification.md:13 |
|
| - | - | Added Rust platform-specific architecture section | specification.md:13 |
|
||||||
@@ -874,7 +883,7 @@ flowchart TD
|
|||||||
| [`src/natsbridge_csr.js`](../src/natsbridge_csr.js) | Browser | JSON table only, WebSocket NATS | specification.md:2-19 (all sections) | FR-001 through FR-014, NFR-101 through NFR-405 |
|
| [`src/natsbridge_csr.js`](../src/natsbridge_csr.js) | Browser | JSON table only, WebSocket NATS | specification.md:2-19 (all sections) | FR-001 through FR-014, NFR-101 through NFR-405 |
|
||||||
| [`src/natsbridge.py`](../src/natsbridge.py) | Python | Arrow IPC, async/await | specification.md:2-19 (all sections) | FR-001 through FR-014, NFR-101 through NFR-405 |
|
| [`src/natsbridge.py`](../src/natsbridge.py) | Python | Arrow IPC, async/await | specification.md:2-19 (all sections) | FR-001 through FR-014, NFR-101 through NFR-405 |
|
||||||
| [`src/natsbridge.dart`](../src/natsbridge.dart) | Dart | Full feature set, Arrow IPC, async/await | specification.md:2-19 (all sections) | FR-001 through FR-014, NFR-101 through NFR-405 |
|
| [`src/natsbridge.dart`](../src/natsbridge.dart) | Dart | Full feature set, Arrow IPC, async/await | specification.md:2-19 (all sections) | FR-001 through FR-014, NFR-101 through NFR-405 |
|
||||||
| [`src/natsbridge.rs`](../src/natsbridge.rs) | Rust | Full feature set, Arrow IPC, async/await, type-safe | specification.md:2-19 (all sections) | FR-001 through FR-014, NFR-101 through NFR-405 |
|
| [`src/natsbridge.rs`](../src/natsbridge.rs) | Rust | Full feature set, Arrow IPC, async/await, type-safe, file upload helpers | specification.md:2-19 (all sections) | FR-001 through FR-014, NFR-101 through NFR-405 |
|
||||||
| [`src/natsbridge_mpy.py`](../src/natsbridge_mpy.py) | MicroPython | Limited to direct transport | specification.md:2-19 (all sections) | FR-005, FR-006, FR-012 |
|
| [`src/natsbridge_mpy.py`](../src/natsbridge_mpy.py) | MicroPython | Limited to direct transport | specification.md:2-19 (all sections) | FR-005, FR-006, FR-012 |
|
||||||
|
|
||||||
### 16.3 External Dependencies
|
### 16.3 External Dependencies
|
||||||
@@ -930,7 +939,3 @@ flowchart TD
|
|||||||
---
|
---
|
||||||
|
|
||||||
*This architecture document is versioned and maintained in git alongside the codebase. All implementations must adhere to this architecture.*
|
*This architecture document is versioned and maintained in git alongside the codebase. All implementations must adhere to this architecture.*
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
*This architecture document is versioned and maintained in git alongside the codebase. All implementations must adhere to this architecture.*
|
|
||||||
|
|||||||
@@ -1,7 +1,7 @@
|
|||||||
# Walkthrough: NATSBridge
|
# Walkthrough: NATSBridge
|
||||||
|
|
||||||
**Version**: 1.2.0
|
**Version**: 1.4.0
|
||||||
**Date**: 2026-05-13
|
**Date**: 2026-05-14
|
||||||
**Status**: Active
|
**Status**: Active
|
||||||
**Ground Truth**: [`src/NATSBridge.jl`](../src/NATSBridge.jl)
|
**Ground Truth**: [`src/NATSBridge.jl`](../src/NATSBridge.jl)
|
||||||
|
|
||||||
@@ -341,8 +341,7 @@ const response = await plikOneshotUpload(
|
|||||||
"transport": "link",
|
"transport": "link",
|
||||||
"encoding": "none",
|
"encoding": "none",
|
||||||
"size": 10000000,
|
"size": 10000000,
|
||||||
"data": "http://localhost:8080/file/UPLOAD_ID/FILE_ID/file",
|
"data": "http://localhost:8080/file/UPLOAD_ID/FILE_ID/file"
|
||||||
"metadata": {}
|
|
||||||
}
|
}
|
||||||
]
|
]
|
||||||
}
|
}
|
||||||
@@ -476,6 +475,7 @@ A Rust service needs to process messages from a Julia analytics pipeline and sen
|
|||||||
```rust
|
```rust
|
||||||
// Rust service - using tokio async runtime
|
// Rust service - using tokio async runtime
|
||||||
use natsbridge::{smartreceive, MsgEnvelopeV1};
|
use natsbridge::{smartreceive, MsgEnvelopeV1};
|
||||||
|
use base64::{Engine as _, engine::general_purpose::STANDARD as BASE64};
|
||||||
|
|
||||||
#[tokio::main]
|
#[tokio::main]
|
||||||
async fn main() {
|
async fn main() {
|
||||||
@@ -485,30 +485,33 @@ async fn main() {
|
|||||||
let mut sub = conn.subscribe("/agent/wine/api/v1/analyze").unwrap();
|
let mut sub = conn.subscribe("/agent/wine/api/v1/analyze").unwrap();
|
||||||
|
|
||||||
for msg in sub.messages() {
|
for msg in sub.messages() {
|
||||||
let envelope: MsgEnvelopeV1 = smartreceive(
|
let envelope = smartreceive(
|
||||||
&String::from_utf8_lossy(&msg.payload),
|
&String::from_utf8_lossy(&msg.payload),
|
||||||
&Default::default(),
|
&Default::default(),
|
||||||
).await.unwrap();
|
).await.unwrap();
|
||||||
|
|
||||||
// Type-safe payload access
|
// Access deserialized payloads by type
|
||||||
for payload in &envelope.payloads {
|
for payload in &envelope.payloads {
|
||||||
match &payload.data {
|
match payload.payload_type.as_str() {
|
||||||
Payload::ArrowTable(arrow_bytes) => {
|
"arrowtable" => {
|
||||||
// Process Arrow IPC data using arrow2
|
// Data is base64-encoded Arrow IPC bytes after smartreceive()
|
||||||
let table = arrow2::io::ipc::read::Reader::new(
|
let arrow_bytes = BASE64.decode(&payload.data).unwrap();
|
||||||
std::io::Cursor::new(arrow_bytes.clone()),
|
println!("Received arrowtable payload ({} bytes)", arrow_bytes.len());
|
||||||
);
|
|
||||||
println!("Received {} rows", table.len());
|
|
||||||
},
|
},
|
||||||
Payload::Text(text) => {
|
"text" => {
|
||||||
println!("Message: {}", text);
|
// Data is the decoded text string
|
||||||
|
println!("Message: {}", payload.data);
|
||||||
},
|
},
|
||||||
_ => println!("Received {} bytes of {} data",
|
"image" | "audio" | "video" | "binary" => {
|
||||||
match &payload.data {
|
// Data is base64-encoded binary content
|
||||||
Payload::Binary(b) => b.len(),
|
let bytes = BASE64.decode(&payload.data).unwrap();
|
||||||
_ => 0,
|
println!("Received {} bytes of {} data", bytes.len(), payload.payload_type);
|
||||||
},
|
},
|
||||||
payload.payload_type),
|
"dictionary" | "jsontable" => {
|
||||||
|
// Data is a JSON string
|
||||||
|
println!("Data: {}", payload.data);
|
||||||
|
},
|
||||||
|
_ => println!("Unknown payload type: {}", payload.payload_type),
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -516,10 +519,10 @@ async fn main() {
|
|||||||
```
|
```
|
||||||
|
|
||||||
**Rationale**:
|
**Rationale**:
|
||||||
- **Type-safe payloads**: Rust enum discriminates between payload types at compile time
|
|
||||||
- **serde serialization**: Automatic JSON deserialization to `MsgEnvelopeV1`
|
- **serde serialization**: Automatic JSON deserialization to `MsgEnvelopeV1`
|
||||||
- **tokio runtime**: Efficient async I/O for NATS and HTTP operations
|
- **tokio runtime**: Efficient async I/O for NATS and HTTP operations
|
||||||
- **arrow2 integration**: Direct Arrow IPC deserialization without intermediate format
|
- **smartreceive deserialization**: Payload data is deserialized and stored as strings in `payload.data`
|
||||||
|
- **Type dispatch**: `payload_type` field determines how to interpret the `data` string
|
||||||
|
|
||||||
#### Step 2: Rust Service Sends Processed Results
|
#### Step 2: Rust Service Sends Processed Results
|
||||||
|
|
||||||
@@ -911,7 +914,7 @@ log_trace(correlation_id, "Published to NATS")
|
|||||||
| [`src/natsbridge_csr.js`](../src/natsbridge_csr.js) | Browser | JSON table only, WebSocket NATS | specification.md:2-19 (all sections) |
|
| [`src/natsbridge_csr.js`](../src/natsbridge_csr.js) | Browser | JSON table only, WebSocket NATS | specification.md:2-19 (all sections) |
|
||||||
| [`src/natsbridge.py`](../src/natsbridge.py) | Python | Arrow IPC, async/await | specification.md:2-19 (all sections) |
|
| [`src/natsbridge.py`](../src/natsbridge.py) | Python | Arrow IPC, async/await | specification.md:2-19 (all sections) |
|
||||||
| [`src/natsbridge.dart`](../src/natsbridge.dart) | Dart | Full feature set, Arrow IPC, async/await | specification.md:2-19 (all sections) |
|
| [`src/natsbridge.dart`](../src/natsbridge.dart) | Dart | Full feature set, Arrow IPC, async/await | specification.md:2-19 (all sections) |
|
||||||
| [`src/natsbridge.rs`](../src/natsbridge.rs) | Rust | Full feature set, Arrow IPC, async/await, type-safe | specification.md:2-19 (all sections) |
|
| [`src/natsbridge.rs`](../src/natsbridge.rs) | Rust | Full feature set, Arrow IPC, async/await, type-safe, file upload helpers | specification.md:2-19 (all sections) |
|
||||||
| [`src/natsbridge_mpy.py`](../src/natsbridge_mpy.py) | MicroPython | Limited to direct transport | specification.md:2-19 (all sections) |
|
| [`src/natsbridge_mpy.py`](../src/natsbridge_mpy.py) | MicroPython | Limited to direct transport | specification.md:2-19 (all sections) |
|
||||||
|
|
||||||
---
|
---
|
||||||
@@ -920,6 +923,11 @@ log_trace(correlation_id, "Published to NATS")
|
|||||||
|
|
||||||
| Date | Version | Changes | Specification Reference |
|
| Date | Version | Changes | Specification Reference |
|
||||||
|------|---------|---------|------------------------|
|
|------|---------|---------|------------------------|
|
||||||
|
| 2026-05-14 | 1.4.0 | Updated Rust API to reflect `smartreceive` deserialization changes | All sections |
|
||||||
|
| - | - | `smartreceive` now stores deserialized data in `MsgPayloadV1.data` | specification.md:8 |
|
||||||
|
| - | - | Added `plik_upload_file` convenience function documentation | specification.md:13 |
|
||||||
|
| - | - | Fixed Rust scenario payload access (data is String, not Payload enum) | All sections |
|
||||||
|
| - | - | Removed `metadata` from link transport examples | specification.md:3 |
|
||||||
| 2026-05-13 | 1.3.0 | Added Rust support with tokio, serde, and arrow2 | All sections |
|
| 2026-05-13 | 1.3.0 | Added Rust support with tokio, serde, and arrow2 | All sections |
|
||||||
| - | - | Added Rust user scenario (User Scenario 4) | specification.md:11 (Rust API) |
|
| - | - | Added Rust user scenario (User Scenario 4) | specification.md:11 (Rust API) |
|
||||||
| - | - | Updated scenario numbering (MicroPython → Scenario 5, Cross-Platform → Scenario 6) | All sections |
|
| - | - | Updated scenario numbering (MicroPython → Scenario 5, Cross-Platform → Scenario 6) | All sections |
|
||||||
|
|||||||
Reference in New Issue
Block a user