Compare commits
2 Commits
v0.6.0-dev
...
v0.6.0-dev
| Author | SHA1 | Date | |
|---|---|---|---|
| 2a83c35852 | |||
| 6da64092ca |
@@ -1,7 +1,7 @@
|
||||
# Architecture Documentation: NATSBridge
|
||||
|
||||
**Version**: 1.2.0
|
||||
**Date**: 2026-05-13
|
||||
**Version**: 1.4.0
|
||||
**Date**: 2026-05-14
|
||||
**Status**: Active
|
||||
**Ground Truth**: [`src/NATSBridge.jl`](../src/NATSBridge.jl)
|
||||
**Architecture Level**: C4 Container Level
|
||||
@@ -182,6 +182,7 @@ flowchart TD
|
||||
| **log_trace** | Log trace messages with correlation ID | All |
|
||||
| **fileserver_upload_handler** | Upload large payloads to HTTP server | Desktop (Julia/JS/Python/Dart/Rust) |
|
||||
| **fileserver_download_handler** | Download payloads from HTTP server with exponential backoff | Desktop (Julia/JS/Python/Dart/Rust) |
|
||||
| **plik_upload_file** | Upload a local file to Plik server from disk | Rust |
|
||||
|
||||
### Data Flow
|
||||
|
||||
@@ -563,7 +564,7 @@ pub enum Payload {
|
||||
pub struct SmartsendOptions {
|
||||
pub broker_url: String,
|
||||
pub fileserver_url: String,
|
||||
pub fileserver_upload_handler: Option<UploadHandler>,
|
||||
pub fileserver_upload_handler: Option<Arc<dyn FileUploadHandler>>,
|
||||
pub size_threshold: usize,
|
||||
pub correlation_id: String,
|
||||
pub msg_purpose: String,
|
||||
@@ -577,13 +578,14 @@ let conn = nats::connect("nats://localhost:4222").await?;
|
||||
// Subscribe and process messages
|
||||
let mut sub = conn.subscribe("/agent/wine/api/v1/analyze")?;
|
||||
for msg in sub.messages() {
|
||||
let envelope: MsgEnvelopeV1 = serde_json::from_slice(&msg.payload)?;
|
||||
// Type-safe access to payloads
|
||||
let envelope = smartreceive(&String::from_utf8_lossy(&msg.payload), &Default::default()).await?;
|
||||
// Access deserialized payloads by type
|
||||
for payload in &envelope.payloads {
|
||||
match &payload.data {
|
||||
Payload::ArrowTable(bytes) => { /* process */ },
|
||||
Payload::Text(text) => { /* process */ },
|
||||
_ => {}
|
||||
match payload.payload_type.as_str() {
|
||||
"arrowtable" => { /* payload.data is base64-encoded Arrow IPC */ },
|
||||
"text" => { /* payload.data is decoded text string */ },
|
||||
"binary" | "image" | "audio" | "video" => { /* payload.data is base64-encoded binary */ },
|
||||
_ => { /* other types */ }
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -833,6 +835,13 @@ flowchart TD
|
||||
|
||||
| Date | Version | Changes |
|
||||
|------|---------|---------|
|
||||
| 2026-05-14 | 1.4.0 | Updated Rust API to reflect `smartreceive` deserialization changes | All sections |
|
||||
| - | - | `smartreceive` now stores deserialized data in `MsgPayloadV1.data` | specification.md:8 |
|
||||
| - | - | Added `plik_upload_file` convenience function to component table | specification.md:13 |
|
||||
| - | - | Fixed Rust payload access pattern (data is String, not Payload enum) | All sections |
|
||||
| - | - | Fixed `SmartsendOptions.fileserver_upload_handler` type to `Arc<dyn FileUploadHandler>` | specification.md:13 |
|
||||
| - | - | Removed `metadata` from link transport examples (now `None`/omitted) | specification.md:3 |
|
||||
| - | - | Removed duplicate footer text | All sections |
|
||||
| 2026-05-13 | 1.3.0 | Added Rust support with tokio, serde, and arrow2 | All sections |
|
||||
| - | - | Added Rust to C4 diagrams (context, container) | All sections |
|
||||
| - | - | Added Rust platform-specific architecture section | specification.md:13 |
|
||||
@@ -874,7 +883,7 @@ flowchart TD
|
||||
| [`src/natsbridge_csr.js`](../src/natsbridge_csr.js) | Browser | JSON table only, WebSocket NATS | specification.md:2-19 (all sections) | FR-001 through FR-014, NFR-101 through NFR-405 |
|
||||
| [`src/natsbridge.py`](../src/natsbridge.py) | Python | Arrow IPC, async/await | specification.md:2-19 (all sections) | FR-001 through FR-014, NFR-101 through NFR-405 |
|
||||
| [`src/natsbridge.dart`](../src/natsbridge.dart) | Dart | Full feature set, Arrow IPC, async/await | specification.md:2-19 (all sections) | FR-001 through FR-014, NFR-101 through NFR-405 |
|
||||
| [`src/natsbridge.rs`](../src/natsbridge.rs) | Rust | Full feature set, Arrow IPC, async/await, type-safe | specification.md:2-19 (all sections) | FR-001 through FR-014, NFR-101 through NFR-405 |
|
||||
| [`src/natsbridge.rs`](../src/natsbridge.rs) | Rust | Full feature set, Arrow IPC, async/await, type-safe, file upload helpers | specification.md:2-19 (all sections) | FR-001 through FR-014, NFR-101 through NFR-405 |
|
||||
| [`src/natsbridge_mpy.py`](../src/natsbridge_mpy.py) | MicroPython | Limited to direct transport | specification.md:2-19 (all sections) | FR-005, FR-006, FR-012 |
|
||||
|
||||
### 16.3 External Dependencies
|
||||
@@ -930,7 +939,3 @@ flowchart TD
|
||||
---
|
||||
|
||||
*This architecture document is versioned and maintained in git alongside the codebase. All implementations must adhere to this architecture.*
|
||||
|
||||
---
|
||||
|
||||
*This architecture document is versioned and maintained in git alongside the codebase. All implementations must adhere to this architecture.*
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
# Walkthrough: NATSBridge
|
||||
|
||||
**Version**: 1.2.0
|
||||
**Date**: 2026-05-13
|
||||
**Version**: 1.4.0
|
||||
**Date**: 2026-05-14
|
||||
**Status**: Active
|
||||
**Ground Truth**: [`src/NATSBridge.jl`](../src/NATSBridge.jl)
|
||||
|
||||
@@ -341,8 +341,7 @@ const response = await plikOneshotUpload(
|
||||
"transport": "link",
|
||||
"encoding": "none",
|
||||
"size": 10000000,
|
||||
"data": "http://localhost:8080/file/UPLOAD_ID/FILE_ID/file",
|
||||
"metadata": {}
|
||||
"data": "http://localhost:8080/file/UPLOAD_ID/FILE_ID/file"
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -476,6 +475,7 @@ A Rust service needs to process messages from a Julia analytics pipeline and sen
|
||||
```rust
|
||||
// Rust service - using tokio async runtime
|
||||
use natsbridge::{smartreceive, MsgEnvelopeV1};
|
||||
use base64::{Engine as _, engine::general_purpose::STANDARD as BASE64};
|
||||
|
||||
#[tokio::main]
|
||||
async fn main() {
|
||||
@@ -485,30 +485,33 @@ async fn main() {
|
||||
let mut sub = conn.subscribe("/agent/wine/api/v1/analyze").unwrap();
|
||||
|
||||
for msg in sub.messages() {
|
||||
let envelope: MsgEnvelopeV1 = smartreceive(
|
||||
let envelope = smartreceive(
|
||||
&String::from_utf8_lossy(&msg.payload),
|
||||
&Default::default(),
|
||||
).await.unwrap();
|
||||
|
||||
// Type-safe payload access
|
||||
// Access deserialized payloads by type
|
||||
for payload in &envelope.payloads {
|
||||
match &payload.data {
|
||||
Payload::ArrowTable(arrow_bytes) => {
|
||||
// Process Arrow IPC data using arrow2
|
||||
let table = arrow2::io::ipc::read::Reader::new(
|
||||
std::io::Cursor::new(arrow_bytes.clone()),
|
||||
);
|
||||
println!("Received {} rows", table.len());
|
||||
match payload.payload_type.as_str() {
|
||||
"arrowtable" => {
|
||||
// Data is base64-encoded Arrow IPC bytes after smartreceive()
|
||||
let arrow_bytes = BASE64.decode(&payload.data).unwrap();
|
||||
println!("Received arrowtable payload ({} bytes)", arrow_bytes.len());
|
||||
},
|
||||
Payload::Text(text) => {
|
||||
println!("Message: {}", text);
|
||||
"text" => {
|
||||
// Data is the decoded text string
|
||||
println!("Message: {}", payload.data);
|
||||
},
|
||||
_ => println!("Received {} bytes of {} data",
|
||||
match &payload.data {
|
||||
Payload::Binary(b) => b.len(),
|
||||
_ => 0,
|
||||
"image" | "audio" | "video" | "binary" => {
|
||||
// Data is base64-encoded binary content
|
||||
let bytes = BASE64.decode(&payload.data).unwrap();
|
||||
println!("Received {} bytes of {} data", bytes.len(), payload.payload_type);
|
||||
},
|
||||
payload.payload_type),
|
||||
"dictionary" | "jsontable" => {
|
||||
// Data is a JSON string
|
||||
println!("Data: {}", payload.data);
|
||||
},
|
||||
_ => println!("Unknown payload type: {}", payload.payload_type),
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -516,10 +519,10 @@ async fn main() {
|
||||
```
|
||||
|
||||
**Rationale**:
|
||||
- **Type-safe payloads**: Rust enum discriminates between payload types at compile time
|
||||
- **serde serialization**: Automatic JSON deserialization to `MsgEnvelopeV1`
|
||||
- **tokio runtime**: Efficient async I/O for NATS and HTTP operations
|
||||
- **arrow2 integration**: Direct Arrow IPC deserialization without intermediate format
|
||||
- **smartreceive deserialization**: Payload data is deserialized and stored as strings in `payload.data`
|
||||
- **Type dispatch**: `payload_type` field determines how to interpret the `data` string
|
||||
|
||||
#### Step 2: Rust Service Sends Processed Results
|
||||
|
||||
@@ -911,7 +914,7 @@ log_trace(correlation_id, "Published to NATS")
|
||||
| [`src/natsbridge_csr.js`](../src/natsbridge_csr.js) | Browser | JSON table only, WebSocket NATS | specification.md:2-19 (all sections) |
|
||||
| [`src/natsbridge.py`](../src/natsbridge.py) | Python | Arrow IPC, async/await | specification.md:2-19 (all sections) |
|
||||
| [`src/natsbridge.dart`](../src/natsbridge.dart) | Dart | Full feature set, Arrow IPC, async/await | specification.md:2-19 (all sections) |
|
||||
| [`src/natsbridge.rs`](../src/natsbridge.rs) | Rust | Full feature set, Arrow IPC, async/await, type-safe | specification.md:2-19 (all sections) |
|
||||
| [`src/natsbridge.rs`](../src/natsbridge.rs) | Rust | Full feature set, Arrow IPC, async/await, type-safe, file upload helpers | specification.md:2-19 (all sections) |
|
||||
| [`src/natsbridge_mpy.py`](../src/natsbridge_mpy.py) | MicroPython | Limited to direct transport | specification.md:2-19 (all sections) |
|
||||
|
||||
---
|
||||
@@ -920,6 +923,11 @@ log_trace(correlation_id, "Published to NATS")
|
||||
|
||||
| Date | Version | Changes | Specification Reference |
|
||||
|------|---------|---------|------------------------|
|
||||
| 2026-05-14 | 1.4.0 | Updated Rust API to reflect `smartreceive` deserialization changes | All sections |
|
||||
| - | - | `smartreceive` now stores deserialized data in `MsgPayloadV1.data` | specification.md:8 |
|
||||
| - | - | Added `plik_upload_file` convenience function documentation | specification.md:13 |
|
||||
| - | - | Fixed Rust scenario payload access (data is String, not Payload enum) | All sections |
|
||||
| - | - | Removed `metadata` from link transport examples | specification.md:3 |
|
||||
| 2026-05-13 | 1.3.0 | Added Rust support with tokio, serde, and arrow2 | All sections |
|
||||
| - | - | Added Rust user scenario (User Scenario 4) | specification.md:11 (Rust API) |
|
||||
| - | - | Updated scenario numbering (MicroPython → Scenario 5, Cross-Platform → Scenario 6) | All sections |
|
||||
|
||||
Reference in New Issue
Block a user