v1.0.0 #1

Merged
ton merged 14 commits from v1.0.0 into main 2026-05-22 22:16:17 +00:00
Showing only changes of commit 594eb32783 - Show all commits

View File

@@ -155,7 +155,7 @@ A JavaScript chat webapp wants to send mixed payloads (text message + user avata
### Step-by-Step Flow
#### 3.1 Step 1: JavaScript Webapp Sends Mixed Payloads
#### Step 1: JavaScript Webapp Sends Mixed Payloads
```javascript
// JavaScript (Browser or Node.js)
@@ -178,7 +178,7 @@ const [env, msgJson] = await msghandler.smartpack(
- **Why text first?** Text is smaller, sent via direct transport (fast, no file server needed)
- **Why image second?** Images may trigger link transport if >0.5MB
#### 3.1b Step 1b: JavaScript Publishes via Transport
#### Step 1b: JavaScript Publishes via Transport
```javascript
// Publish the JSON string via WebSocket (or NATS/MQTT)
@@ -186,7 +186,7 @@ const conn = await transportClient.connect({ servers: "ws://localhost:4222" });
await conn.publish("/agent/wine/api/v1/prompt", msgJson);
```
#### 3.2 Step 2: Transport Selection
#### Step 2: Transport Selection
For each payload, msghandler determines transport:
@@ -199,7 +199,7 @@ For each payload, msghandler determines transport:
- Direct transport is faster for small payloads (no file server round-trip)
- Link transport is used when payload ≥ 0.5MB (avoids transport size limits)
#### 3.3 Step 3: Serialization and Encoding
#### Step 3: Serialization and Encoding
Each payload is serialized:
@@ -213,7 +213,7 @@ Each payload is serialized:
- Images use raw bytes to preserve binary data integrity
- All payloads encoded as Base64 for JSON compatibility
#### 3.4 Step 4: Envelope Building
#### Step 4: Envelope Building
msghandler builds the message envelope:
@@ -262,7 +262,7 @@ msghandler builds the message envelope:
- **reply_to**: Tells backend where to send response
- **payloads array**: Contains all data with metadata for proper handling
#### 3.5 Step 5: Publish JSON String via Transport
#### Step 5: Publish JSON String via Transport
```javascript
// JavaScript: Publish via WebSocket (NATS/MQTT/HTTP work similarly)
@@ -270,7 +270,7 @@ const conn = await transportClient.connect({ servers: "ws://localhost:4222" });
await conn.publish("/agent/wine/api/v1/prompt", msgJson);
```
#### 3.6 Step 6: Julia Backend Receives and Unpacks
#### Step 6: Julia Backend Receives and Unpacks
```julia
# Julia backend
@@ -284,7 +284,7 @@ env = smartunpack(String(transport_msg.payload))
# ]
```
#### 3.7 Step 7: Julia Backend Sends Response
#### Step 7: Julia Backend Sends Response
```julia
# Julia backend processes the message
@@ -318,7 +318,7 @@ A JavaScript webapp wants to upload a large file (10MB) to a Julia backend for p
### Step-by-Step Flow
#### 4.1 Step 1: JavaScript Webapp Sends Large File
#### Step 1: JavaScript Webapp Sends Large File
```javascript
const [env, msgJson] = await msghandler.smartpack(
@@ -333,7 +333,7 @@ const [env, msgJson] = await msghandler.smartpack(
);
```
#### 4.1b Step 1b: JavaScript Publishes via Transport
#### Step 1b: JavaScript Publishes via Transport
```javascript
// Publish the JSON string via WebSocket (or NATS/MQTT)
@@ -341,7 +341,7 @@ const conn = await transportClient.connect({ servers: "ws://localhost:4222" });
await conn.publish("/agent/wine/api/v1/process", msgJson);
```
#### 4.2 Step 2: Transport Selection (Link)
#### Step 2: Transport Selection (Link)
| Payload | Size | Transport | Reason |
|---------|------|-----------|--------|
@@ -352,7 +352,7 @@ await conn.publish("/agent/wine/api/v1/process", msgJson);
- File server handles large file upload
- Transport only sends URL (small message)
#### 4.3 Step 3: File Server Upload
#### Step 3: File Server Upload
```javascript
// msghandler internally calls:
@@ -376,7 +376,7 @@ const response = await plikOneshotUpload(
- One-shot mode simplifies API
- Returns URL for download
#### 4.4 Step 4: Envelope with Link Transport
#### Step 4: Envelope with Link Transport
```json
{
@@ -400,7 +400,7 @@ const response = await plikOneshotUpload(
- `transport: "link"` signals URL-based download
- `encoding: "none"` indicates no additional encoding
#### 4.5 Step 5: Julia Backend Receives, Downloads, and Unpacks
#### Step 5: Julia Backend Receives, Downloads, and Unpacks
```julia
# Julia backend
@@ -428,7 +428,7 @@ A Python application sends tabular data (pandas DataFrame) to a Julia backend fo
### Step-by-Step Flow
#### 5.1 Step 1: Python Sends Tabular Data
#### Step 1: Python Sends Tabular Data
```python
# Python
@@ -454,14 +454,14 @@ env, msg_json = await smartpack(
- Arrow IPC format preserves data types
- Much faster than JSON serialization
#### 5.1b Step 1b: Python Publishes via Transport
#### Step 1b: Python Publishes via Transport
```python
# Publish the JSON string via WebSocket/NATS/MQTT
await transport_publisher.publish("/agent/wine/api/v1/analyze", msg_json)
```
#### 5.2 Step 2: Serialization to Arrow IPC
#### Step 2: Serialization to Arrow IPC
```python
# msghandler internally:
@@ -480,7 +480,7 @@ arrow_bytes = buf.getvalue();
- Binary format is compact
- No schema information loss
#### 5.3 Step 3: Julia Receives and Deserializes
#### Step 3: Julia Receives and Deserializes
```julia
# Julia backend
@@ -496,7 +496,7 @@ env = smartunpack(String(transport_msg.payload));
- DataFrame returned with correct types
- No manual parsing needed
#### 5.4 Step 4: Julia Sends Results (Complete Round-Trip)
#### Step 4: Julia Sends Results (Complete Round-Trip)
```julia
# Julia backend
@@ -526,7 +526,7 @@ A Rust service needs to process messages from a Julia analytics pipeline and sen
### Step-by-Step Flow
#### 6.1 Step 1: Rust Service Receives Message
#### Step 1: Rust Service Receives Message
```rust
// Rust service - using tokio async runtime
@@ -580,7 +580,7 @@ async fn main() {
- **smartunpack deserialization**: Payload data is deserialized and stored as strings in `payload.data`
- **Type dispatch**: `payload_type` field determines how to interpret the `data` string
#### 6.2 Step 2: Rust Service Sends Processed Results
#### Step 2: Rust Service Sends Processed Results
```rust
// Rust service sends results back with mixed payload types
@@ -621,7 +621,7 @@ conn.publish("/agent/wine/api/v1/results", &json_str)?;
- **Default options**: sensible defaults reduce boilerplate
- **Result<T, E>**: idiomatic Rust error handling
#### 6.3 Step 3: Python/Julia Receives Rust Response
#### Step 3: Python/Julia Receives Rust Response
```python
# Python backend receives Rust response
@@ -638,7 +638,7 @@ env = await smartunpack(str(transport_msg.payload));
- **Same JSON wire format**: No protocol translation needed
- **Type preservation**: Arrow IPC and text types preserved across all platforms
#### 6.4 Step 4: Large File Transfer from Rust
#### Step 4: Large File Transfer from Rust
```rust
// Rust service sends large binary file via link transport
@@ -681,7 +681,7 @@ A MicroPython sensor device sends sensor readings to a Python backend.
### Step-by-Step Flow
#### 7.1 Step 1: MicroPython Sends Sensor Data
#### Step 1: MicroPython Sends Sensor Data
```python
# MicroPython
@@ -706,14 +706,14 @@ env, msg_json = smartpack(
- Smaller threshold (100KB) for memory constraints
- Direct transport only (no file server support)
#### 7.1b Step 1b: MicroPython Publishes via Transport
#### Step 1b: MicroPython Publishes via Transport
```python
# Publish the JSON string via MQTT (common on IoT devices)
mqtt_client.publish("/sensor/device/v1/readings", msg_json)
```
#### 7.2 Step 2: Serialization
#### Step 2: Serialization
```python
# msghandler internally:
@@ -727,7 +727,7 @@ payload_b64 = base64.b64encode(json_bytes).decode('ascii');
- Base64 for transport compatibility
- UTF-8 for text encoding
#### 7.3 Step 3: Python Backend Receives
#### Step 3: Python Backend Receives
```python
# Python backend
@@ -743,7 +743,7 @@ env = await smartunpack(str(transport_msg.payload));
- Dictionary returned directly
- No Arrow support (memory constraints)
#### 7.4 Step 4: Complete MicroPython End-to-End Flow
#### Step 4: Complete MicroPython End-to-End Flow
```mermaid
flowchart LR
@@ -787,7 +787,7 @@ Multiple platforms (JavaScript, Python, Julia) communicate in a chat application
### Step-by-Step Flow
#### 8.1 Step 1: JavaScript Sends Chat Message
#### Step 1: JavaScript Sends Chat Message
```javascript
// JavaScript (Frontend)
@@ -810,7 +810,7 @@ const [env, msgJson] = await msghandler.smartpack(
- Chat messages often include text + images
- Transport wildcard subscriptions route to correct recipients
#### 8.1b Step 1b: JavaScript Publishes via Transport
#### Step 1b: JavaScript Publishes via Transport
```javascript
// Publish via WebSocket/NATS/MQTT
@@ -818,7 +818,7 @@ const conn = await transportClient.connect({ servers: "ws://localhost:4222" });
await conn.publish("/chat/user/v1/message", msgJson);
```
#### 8.2 Step 2: Python Backend Receives
#### Step 2: Python Backend Receives
```python
# Python (Backend)
@@ -837,7 +837,7 @@ env = await smartunpack(str(transport_msg.payload));
- Same payload structure regardless of sender
- Type information preserved
#### 8.3 Step 3: Julia Backend Receives
#### Step 3: Julia Backend Receives
```julia
# Julia (Backend)
@@ -856,7 +856,7 @@ env = smartunpack(String(transport_msg.payload));
- Same function signature across platforms
- Type information enables proper deserialization
#### 8.4 Step 4: Complete End-to-End Flow
#### Step 4: Complete End-to-End Flow
```mermaid
flowchart TB
@@ -893,7 +893,7 @@ flowchart TB
- `smartunpack()` handles transport detection, file downloads (if link), and deserialization
- Same API across all platforms ensures consistency
#### 8.6 Step 6: Complete Cross-Platform Flow
#### Step 5: Complete Cross-Platform Flow
```mermaid
flowchart TB