2 Commits

2 changed files with 51 additions and 38 deletions

View File

@@ -1,7 +1,7 @@
# Architecture Documentation: NATSBridge # Architecture Documentation: NATSBridge
**Version**: 1.2.0 **Version**: 1.4.0
**Date**: 2026-05-13 **Date**: 2026-05-14
**Status**: Active **Status**: Active
**Ground Truth**: [`src/NATSBridge.jl`](../src/NATSBridge.jl) **Ground Truth**: [`src/NATSBridge.jl`](../src/NATSBridge.jl)
**Architecture Level**: C4 Container Level **Architecture Level**: C4 Container Level
@@ -182,6 +182,7 @@ flowchart TD
| **log_trace** | Log trace messages with correlation ID | All | | **log_trace** | Log trace messages with correlation ID | All |
| **fileserver_upload_handler** | Upload large payloads to HTTP server | Desktop (Julia/JS/Python/Dart/Rust) | | **fileserver_upload_handler** | Upload large payloads to HTTP server | Desktop (Julia/JS/Python/Dart/Rust) |
| **fileserver_download_handler** | Download payloads from HTTP server with exponential backoff | Desktop (Julia/JS/Python/Dart/Rust) | | **fileserver_download_handler** | Download payloads from HTTP server with exponential backoff | Desktop (Julia/JS/Python/Dart/Rust) |
| **plik_upload_file** | Upload a local file to Plik server from disk | Rust |
### Data Flow ### Data Flow
@@ -563,7 +564,7 @@ pub enum Payload {
pub struct SmartsendOptions { pub struct SmartsendOptions {
pub broker_url: String, pub broker_url: String,
pub fileserver_url: String, pub fileserver_url: String,
pub fileserver_upload_handler: Option<UploadHandler>, pub fileserver_upload_handler: Option<Arc<dyn FileUploadHandler>>,
pub size_threshold: usize, pub size_threshold: usize,
pub correlation_id: String, pub correlation_id: String,
pub msg_purpose: String, pub msg_purpose: String,
@@ -577,13 +578,14 @@ let conn = nats::connect("nats://localhost:4222").await?;
// Subscribe and process messages // Subscribe and process messages
let mut sub = conn.subscribe("/agent/wine/api/v1/analyze")?; let mut sub = conn.subscribe("/agent/wine/api/v1/analyze")?;
for msg in sub.messages() { for msg in sub.messages() {
let envelope: MsgEnvelopeV1 = serde_json::from_slice(&msg.payload)?; let envelope = smartreceive(&String::from_utf8_lossy(&msg.payload), &Default::default()).await?;
// Type-safe access to payloads // Access deserialized payloads by type
for payload in &envelope.payloads { for payload in &envelope.payloads {
match &payload.data { match payload.payload_type.as_str() {
Payload::ArrowTable(bytes) => { /* process */ }, "arrowtable" => { /* payload.data is base64-encoded Arrow IPC */ },
Payload::Text(text) => { /* process */ }, "text" => { /* payload.data is decoded text string */ },
_ => {} "binary" | "image" | "audio" | "video" => { /* payload.data is base64-encoded binary */ },
_ => { /* other types */ }
} }
} }
} }
@@ -833,6 +835,13 @@ flowchart TD
| Date | Version | Changes | | Date | Version | Changes |
|------|---------|---------| |------|---------|---------|
| 2026-05-14 | 1.4.0 | Updated Rust API to reflect `smartreceive` deserialization changes | All sections |
| - | - | `smartreceive` now stores deserialized data in `MsgPayloadV1.data` | specification.md:8 |
| - | - | Added `plik_upload_file` convenience function to component table | specification.md:13 |
| - | - | Fixed Rust payload access pattern (data is String, not Payload enum) | All sections |
| - | - | Fixed `SmartsendOptions.fileserver_upload_handler` type to `Arc<dyn FileUploadHandler>` | specification.md:13 |
| - | - | Removed `metadata` from link transport examples (now `None`/omitted) | specification.md:3 |
| - | - | Removed duplicate footer text | All sections |
| 2026-05-13 | 1.3.0 | Added Rust support with tokio, serde, and arrow2 | All sections | | 2026-05-13 | 1.3.0 | Added Rust support with tokio, serde, and arrow2 | All sections |
| - | - | Added Rust to C4 diagrams (context, container) | All sections | | - | - | Added Rust to C4 diagrams (context, container) | All sections |
| - | - | Added Rust platform-specific architecture section | specification.md:13 | | - | - | Added Rust platform-specific architecture section | specification.md:13 |
@@ -874,7 +883,7 @@ flowchart TD
| [`src/natsbridge_csr.js`](../src/natsbridge_csr.js) | Browser | JSON table only, WebSocket NATS | specification.md:2-19 (all sections) | FR-001 through FR-014, NFR-101 through NFR-405 | | [`src/natsbridge_csr.js`](../src/natsbridge_csr.js) | Browser | JSON table only, WebSocket NATS | specification.md:2-19 (all sections) | FR-001 through FR-014, NFR-101 through NFR-405 |
| [`src/natsbridge.py`](../src/natsbridge.py) | Python | Arrow IPC, async/await | specification.md:2-19 (all sections) | FR-001 through FR-014, NFR-101 through NFR-405 | | [`src/natsbridge.py`](../src/natsbridge.py) | Python | Arrow IPC, async/await | specification.md:2-19 (all sections) | FR-001 through FR-014, NFR-101 through NFR-405 |
| [`src/natsbridge.dart`](../src/natsbridge.dart) | Dart | Full feature set, Arrow IPC, async/await | specification.md:2-19 (all sections) | FR-001 through FR-014, NFR-101 through NFR-405 | | [`src/natsbridge.dart`](../src/natsbridge.dart) | Dart | Full feature set, Arrow IPC, async/await | specification.md:2-19 (all sections) | FR-001 through FR-014, NFR-101 through NFR-405 |
| [`src/natsbridge.rs`](../src/natsbridge.rs) | Rust | Full feature set, Arrow IPC, async/await, type-safe | specification.md:2-19 (all sections) | FR-001 through FR-014, NFR-101 through NFR-405 | | [`src/natsbridge.rs`](../src/natsbridge.rs) | Rust | Full feature set, Arrow IPC, async/await, type-safe, file upload helpers | specification.md:2-19 (all sections) | FR-001 through FR-014, NFR-101 through NFR-405 |
| [`src/natsbridge_mpy.py`](../src/natsbridge_mpy.py) | MicroPython | Limited to direct transport | specification.md:2-19 (all sections) | FR-005, FR-006, FR-012 | | [`src/natsbridge_mpy.py`](../src/natsbridge_mpy.py) | MicroPython | Limited to direct transport | specification.md:2-19 (all sections) | FR-005, FR-006, FR-012 |
### 16.3 External Dependencies ### 16.3 External Dependencies
@@ -930,7 +939,3 @@ flowchart TD
--- ---
*This architecture document is versioned and maintained in git alongside the codebase. All implementations must adhere to this architecture.* *This architecture document is versioned and maintained in git alongside the codebase. All implementations must adhere to this architecture.*
---
*This architecture document is versioned and maintained in git alongside the codebase. All implementations must adhere to this architecture.*

View File

@@ -1,7 +1,7 @@
# Walkthrough: NATSBridge # Walkthrough: NATSBridge
**Version**: 1.2.0 **Version**: 1.4.0
**Date**: 2026-05-13 **Date**: 2026-05-14
**Status**: Active **Status**: Active
**Ground Truth**: [`src/NATSBridge.jl`](../src/NATSBridge.jl) **Ground Truth**: [`src/NATSBridge.jl`](../src/NATSBridge.jl)
@@ -341,8 +341,7 @@ const response = await plikOneshotUpload(
"transport": "link", "transport": "link",
"encoding": "none", "encoding": "none",
"size": 10000000, "size": 10000000,
"data": "http://localhost:8080/file/UPLOAD_ID/FILE_ID/file", "data": "http://localhost:8080/file/UPLOAD_ID/FILE_ID/file"
"metadata": {}
} }
] ]
} }
@@ -476,6 +475,7 @@ A Rust service needs to process messages from a Julia analytics pipeline and sen
```rust ```rust
// Rust service - using tokio async runtime // Rust service - using tokio async runtime
use natsbridge::{smartreceive, MsgEnvelopeV1}; use natsbridge::{smartreceive, MsgEnvelopeV1};
use base64::{Engine as _, engine::general_purpose::STANDARD as BASE64};
#[tokio::main] #[tokio::main]
async fn main() { async fn main() {
@@ -485,30 +485,33 @@ async fn main() {
let mut sub = conn.subscribe("/agent/wine/api/v1/analyze").unwrap(); let mut sub = conn.subscribe("/agent/wine/api/v1/analyze").unwrap();
for msg in sub.messages() { for msg in sub.messages() {
let envelope: MsgEnvelopeV1 = smartreceive( let envelope = smartreceive(
&String::from_utf8_lossy(&msg.payload), &String::from_utf8_lossy(&msg.payload),
&Default::default(), &Default::default(),
).await.unwrap(); ).await.unwrap();
// Type-safe payload access // Access deserialized payloads by type
for payload in &envelope.payloads { for payload in &envelope.payloads {
match &payload.data { match payload.payload_type.as_str() {
Payload::ArrowTable(arrow_bytes) => { "arrowtable" => {
// Process Arrow IPC data using arrow2 // Data is base64-encoded Arrow IPC bytes after smartreceive()
let table = arrow2::io::ipc::read::Reader::new( let arrow_bytes = BASE64.decode(&payload.data).unwrap();
std::io::Cursor::new(arrow_bytes.clone()), println!("Received arrowtable payload ({} bytes)", arrow_bytes.len());
);
println!("Received {} rows", table.len());
}, },
Payload::Text(text) => { "text" => {
println!("Message: {}", text); // Data is the decoded text string
println!("Message: {}", payload.data);
}, },
_ => println!("Received {} bytes of {} data", "image" | "audio" | "video" | "binary" => {
match &payload.data { // Data is base64-encoded binary content
Payload::Binary(b) => b.len(), let bytes = BASE64.decode(&payload.data).unwrap();
_ => 0, println!("Received {} bytes of {} data", bytes.len(), payload.payload_type);
}, },
payload.payload_type), "dictionary" | "jsontable" => {
// Data is a JSON string
println!("Data: {}", payload.data);
},
_ => println!("Unknown payload type: {}", payload.payload_type),
} }
} }
} }
@@ -516,10 +519,10 @@ async fn main() {
``` ```
**Rationale**: **Rationale**:
- **Type-safe payloads**: Rust enum discriminates between payload types at compile time
- **serde serialization**: Automatic JSON deserialization to `MsgEnvelopeV1` - **serde serialization**: Automatic JSON deserialization to `MsgEnvelopeV1`
- **tokio runtime**: Efficient async I/O for NATS and HTTP operations - **tokio runtime**: Efficient async I/O for NATS and HTTP operations
- **arrow2 integration**: Direct Arrow IPC deserialization without intermediate format - **smartreceive deserialization**: Payload data is deserialized and stored as strings in `payload.data`
- **Type dispatch**: `payload_type` field determines how to interpret the `data` string
#### Step 2: Rust Service Sends Processed Results #### Step 2: Rust Service Sends Processed Results
@@ -911,7 +914,7 @@ log_trace(correlation_id, "Published to NATS")
| [`src/natsbridge_csr.js`](../src/natsbridge_csr.js) | Browser | JSON table only, WebSocket NATS | specification.md:2-19 (all sections) | | [`src/natsbridge_csr.js`](../src/natsbridge_csr.js) | Browser | JSON table only, WebSocket NATS | specification.md:2-19 (all sections) |
| [`src/natsbridge.py`](../src/natsbridge.py) | Python | Arrow IPC, async/await | specification.md:2-19 (all sections) | | [`src/natsbridge.py`](../src/natsbridge.py) | Python | Arrow IPC, async/await | specification.md:2-19 (all sections) |
| [`src/natsbridge.dart`](../src/natsbridge.dart) | Dart | Full feature set, Arrow IPC, async/await | specification.md:2-19 (all sections) | | [`src/natsbridge.dart`](../src/natsbridge.dart) | Dart | Full feature set, Arrow IPC, async/await | specification.md:2-19 (all sections) |
| [`src/natsbridge.rs`](../src/natsbridge.rs) | Rust | Full feature set, Arrow IPC, async/await, type-safe | specification.md:2-19 (all sections) | | [`src/natsbridge.rs`](../src/natsbridge.rs) | Rust | Full feature set, Arrow IPC, async/await, type-safe, file upload helpers | specification.md:2-19 (all sections) |
| [`src/natsbridge_mpy.py`](../src/natsbridge_mpy.py) | MicroPython | Limited to direct transport | specification.md:2-19 (all sections) | | [`src/natsbridge_mpy.py`](../src/natsbridge_mpy.py) | MicroPython | Limited to direct transport | specification.md:2-19 (all sections) |
--- ---
@@ -920,6 +923,11 @@ log_trace(correlation_id, "Published to NATS")
| Date | Version | Changes | Specification Reference | | Date | Version | Changes | Specification Reference |
|------|---------|---------|------------------------| |------|---------|---------|------------------------|
| 2026-05-14 | 1.4.0 | Updated Rust API to reflect `smartreceive` deserialization changes | All sections |
| - | - | `smartreceive` now stores deserialized data in `MsgPayloadV1.data` | specification.md:8 |
| - | - | Added `plik_upload_file` convenience function documentation | specification.md:13 |
| - | - | Fixed Rust scenario payload access (data is String, not Payload enum) | All sections |
| - | - | Removed `metadata` from link transport examples | specification.md:3 |
| 2026-05-13 | 1.3.0 | Added Rust support with tokio, serde, and arrow2 | All sections | | 2026-05-13 | 1.3.0 | Added Rust support with tokio, serde, and arrow2 | All sections |
| - | - | Added Rust user scenario (User Scenario 4) | specification.md:11 (Rust API) | | - | - | Added Rust user scenario (User Scenario 4) | specification.md:11 (Rust API) |
| - | - | Updated scenario numbering (MicroPython → Scenario 5, Cross-Platform → Scenario 6) | All sections | | - | - | Updated scenario numbering (MicroPython → Scenario 5, Cross-Platform → Scenario 6) | All sections |