# Architecture Documentation: NATSBridge

**Version**: 1.0.0  
**Date**: 2026-03-13  
**Status**: Active  
**Ground Truth**: [`src/NATSBridge.jl`](../src/NATSBridge.jl)  
**Architecture Level**: C4 Container Level

---

## Executive Summary

This document defines the **blueprint** for NATSBridge - the cross-platform bi-directional data bridge that enables seamless communication between **Julia**, **JavaScript**, **Python**, and **MicroPython** applications using NATS as the message bus.

This architecture document serves as the single source of truth for:
- **System Structure**: How components fit together and interact
- **Scaling Considerations**: How the system scales horizontally and vertically
- **Failure Modes**: How the system handles failures and recovers
- **Trade-off Decisions**: The rationale behind architectural decisions

---

## Architecture Overview

### C4 Context Diagram

```mermaid
flowchart TD
    subgraph "External Systems"
        NATS_Server[NATS Server]
        File_Server[HTTP File Server<br/>Plik/AWS S3/Custom]
    end

    subgraph "Client Applications"
        Julia_App[Julia Application]
        JS_App[JavaScript Application<br/>Node.js/Browser]
        Python_App[Python Application<br/>Desktop]
        MicroPython_App[MicroPython Device]
    end

    Julia_App -->|NATS| NATS_Server
    JS_App -->|NATS| NATS_Server
    Python_App -->|NATS| NATS_Server
    MicroPython_App -->|NATS| NATS_Server

    Julia_App -->|HTTP| File_Server
    JS_App -->|HTTP| File_Server
    Python_App -->|HTTP| File_Server
    MicroPython_App -->|HTTP| File_Server

    style NATS_Server fill:#fff3e0,stroke:#f57c00
    style File_Server fill:#f3e5f5,stroke:#9c27b4
    style Julia_App fill:#e8f5e9,stroke:#4caf50
    style JS_App fill:#e3f2fd,stroke:#2196f3
    style Python_App fill:#e3f2fd,stroke:#2196f3
    style MicroPython_App fill:#fce4ec,stroke:#e91e63
```

### C4 Container Diagram

```mermaid
flowchart TD
    subgraph "Client Container"
        Julia_Module[Julia NATSBridge Module]
        JS_Module[JavaScript NATSBridge Module]
        Python_Module[Python NATSBridge Module]
        MicroPython_Module[MicroPython NATSBridge Module]
    end

    subgraph "NATS Container"
        NATS_Client[NATS Client]
        NATS_Broker[NATS Broker]
    end

    subgraph "File Server Container"
        File_Client[HTTP Client]
        File_Server[File Server]
    end

    Julia_Module --> NATS_Client
    JS_Module --> NATS_Client
    Python_Module --> NATS_Client
    MicroPython_Module --> NATS_Client

    NATS_Client --> NATS_Broker

    Julia_Module --> File_Client
    JS_Module --> File_Client
    Python_Module --> File_Client
    MicroPython_Module --> File_Client

    File_Client --> File_Server

    style Julia_Module fill:#e8f5e9,stroke:#4caf50
    style JS_Module fill:#e3f2fd,stroke:#2196f3
    style Python_Module fill:#e3f2fd,stroke:#2196f3
    style MicroPython_Module fill:#fce4ec,stroke:#e91e63
    style NATS_Broker fill:#fff3e0,stroke:#f57c00
    style File_Server fill:#f3e5f5,stroke:#9c27b4
```

### C4 Component Diagram (Julia Implementation)

```mermaid
flowchart TD
    subgraph "NATSBridge Module"
        SmartSend[smartsend Function]
        SmartReceive[smartreceive Function]
        
        Serialize[_serialize_data]
        Deserialize[_deserialize_data]
        
        BuildEnvelope[build_envelope]
        BuildPayload[build_payload]
        
        PublishMessage[publish_message]
        
        FileServerUpload[fileserver_upload_handler]
        FileServerDownload[fileserver_download_handler]
    end

    subgraph "Data Models"
        Payload[MsgPayloadV1 Struct]
        Envelope[MsgEnvelopeV1 Struct]
    end

    SmartSend --> Serialize
    SmartSend --> BuildEnvelope
    SmartSend --> BuildPayload
    SmartSend --> PublishMessage
    SmartSend --> FileServerUpload

    SmartReceive --> Deserialize
    SmartReceive --> FileServerDownload

    Serialize --> Payload
    BuildEnvelope --> Envelope
    BuildPayload --> Payload

    style SmartSend fill:#d1fae5,stroke:#10b981
    style SmartReceive fill:#d1fae5,stroke:#10b981
    style PublishMessage fill:#fef3c7,stroke:#f59e0b
    style FileServerUpload fill:#fef3c7,stroke:#f59e0b
    style FileServerDownload fill:#fef3c7,stroke:#f59e0b
```

---

## High-Level Architecture

### System Components

| Component | Purpose | Platform Support |
|-----------|---------|------------------|
| **smartsend** | Send data via NATS with automatic transport selection | All |
| **smartreceive** | Receive and process NATS messages | All |
| **_serialize_data** | Serialize data according to payload type | All |
| **_deserialize_data** | Deserialize bytes to native data types | All |
| **_build_envelope** | Build message envelope from payloads | All |
| **_build_payload** | Build payload object from serialized data | All |
| **publish_message** | Publish message to NATS subject | All |
| **fileserver_upload_handler** | Upload large payloads to HTTP server | Desktop |
| **fileserver_download_handler** | Download payloads from HTTP server | Desktop |

### Data Flow

```mermaid
flowchart TD
    A[User calls smartsend subject data] --> B[Process each payload]
    B --> C{Calculate serialized size}
    C -->|Size < Threshold| D[Direct Transport]
    C -->|Size >= Threshold| E[Link Transport]
    
    D --> F[Serialize data]
    F --> G[Base64 encode]
    G --> H[Build payload object]
    
    E --> I[Serialize data]
    I --> J[Upload to file server]
    J --> K[Get download URL]
    K --> H
    
    H --> L[Build envelope]
    L --> M[Convert to JSON]
    M --> N[Publish to NATS]
    
    style A fill:#f9f9f9,stroke:#333
    style N fill:#e0e7ff,stroke:#3b82f6
    style D fill:#d1fae5,stroke:#10b981
    style E fill:#fef3c7,stroke:#f59e0b
```

---

## Message Envelope Architecture

### msg_envelope_v1 Structure (Julia)

```julia
struct msg_envelope_v1
  correlation_id::String       # UUID v4 for distributed tracing
  msg_id::String               # UUID v4 for this message
  timestamp::String            # ISO 8601 UTC timestamp
  
  send_to::String              # NATS subject to publish to
  msg_purpose::String          # ACK, NACK, updateStatus, shutdown, chat
  sender_name::String          # Sender application name
  sender_id::String            # UUID v4 of sender
  receiver_name::String        # Receiver application name (empty = broadcast)
  receiver_id::String          # UUID v4 of receiver (empty = broadcast)
  
  reply_to::String             # Topic for reply messages
  reply_to_msg_id::String      # Message ID being replied to
  broker_url::String           # NATS broker URL
  
  metadata::Dict{String, Any}  # Message-level metadata
  payloads::Vector{msg_payload_v1}  # List of payloads
end
```

### msg_payload_v1 Structure (Julia)

```julia
struct msg_payload_v1
  id::String                    # UUID v4 for this payload
  dataname::String              # Name of the payload
  payload_type::String          # text, dictionary, arrowtable, etc.
  transport::String             # direct or link
  encoding::String              # none, json, base64, arrow-ipc
  size::Integer                 # Size in bytes
  data::Any                     # Base64 string or URL
  metadata::Dict{String, Any}   # Payload-level metadata
end
```

### JSON Schema (Cross-Platform)

```json
{
  "correlation_id": "string (UUID v4)",
  "msg_id": "string (UUID v4)",
  "timestamp": "string (ISO 8601 UTC)",
  "send_to": "string",
  "msg_purpose": "string",
  "sender_name": "string",
  "sender_id": "string (UUID v4)",
  "receiver_name": "string",
  "receiver_id": "string (UUID v4)",
  "reply_to": "string",
  "reply_to_msg_id": "string",
  "broker_url": "string",
  "metadata": "object",
  "payloads": [
    {
      "id": "string (UUID v4)",
      "dataname": "string",
      "payload_type": "string",
      "transport": "string",
      "encoding": "string",
      "size": "integer",
      "data": "string or URL",
      "metadata": "object"
    }
  ]
}
```

---

## Payload Type Architecture

### Supported Payload Types

| Type | Description | Serialization | Encoding | Platforms |
|------|-------------|---------------|----------|-----------|
| `text` | Plain text string | UTF-8 bytes | Base64 | All |
| `dictionary` | JSON object | JSON string | Base64/JSON | All |
| `arrowtable` | Apache Arrow IPC | Arrow IPC stream | Base64/arrow-ipc | Desktop |
| `jsontable` | JSON array of objects | JSON string | Base64/json | All |
| `image` | Binary image data | Raw bytes | Base64 | All |
| `audio` | Binary audio data | Raw bytes | Base64 | All |
| `video` | Binary video data | Raw bytes | Base64 | All |
| `binary` | Generic binary data | Raw bytes | Base64 | All |

### Serialization Logic

```mermaid
flowchart TD
    A[Input data + payload_type] --> B{Payload Type}
    
    B -->|"text"| C[UTF-8 encode]
    B -->|"dictionary"| D[JSON serialize]
    B -->|"arrowtable"| E[Arrow IPC serialize]
    B -->|"jsontable"| F[JSON serialize]
    B -->|"image"| G[Raw bytes]
    B -->|"audio"| H[Raw bytes]
    B -->|"video"| I[Raw bytes]
    B -->|"binary"| J[Raw bytes]
    
    C --> K[Return bytes]
    D --> K
    E --> K
    F --> K
    G --> K
    H --> K
    I --> K
    J --> K
    
    style A fill:#f9f9f9,stroke:#333
    style K fill:#e0e7ff,stroke:#3b82f6
```

---

## Transport Strategy Architecture

### Size Threshold Decision Logic

| Platform | Size Threshold | Notes |
|----------|----------------|-------|
| Desktop (Julia/JS/Python) | 500,000 bytes (0.5MB) | Default threshold |
| MicroPython | 100,000 bytes (100KB) | Lower threshold for memory constraints |

### Transport Selection Flow

```mermaid
flowchart TD
    A[smartsend called] --> B[Serialize payload]
    B --> C[Calculate size]
    C --> D{Size < Threshold?}
    
    D -->|Yes| E[Direct Transport]
    D -->|No| F[Link Transport]
    
    E --> G[Base64 encode]
    G --> H[Build payload with direct transport]
    
    F --> I[Upload to file server]
    I --> J[Get download URL]
    J --> K[Build payload with link transport]
    
    H --> L[Build envelope]
    K --> L
    
    style A fill:#f9f9f9,stroke:#333
    style L fill:#e0e7ff,stroke:#3b82f6
    style E fill:#d1fae5,stroke:#10b981
    style F fill:#fef3c7,stroke:#f59e0b
```

### Direct Transport Protocol

When `transport = "direct"`, the `data` field contains a Base64-encoded string of the serialized payload.

**Encoding Rules**:
- `text`: UTF-8 → Base64
- `dictionary`: JSON → Base64 (or direct JSON)
- `arrowtable`: Arrow IPC → Base64 (or arrow-ipc)
- `jsontable`: JSON → Base64 (or direct JSON)
- `image`/`audio`/`video`/`binary`: Raw bytes → Base64

### Link Transport Protocol

When `transport = "link"`, the `data` field contains a URL pointing to the uploaded payload.

**Upload Flow**:
1. Serialize payload according to `payload_type`
2. Upload to HTTP file server (e.g., Plik)
3. Include returned URL in `data` field

**Download Flow**:
1. Extract URL from payload
2. Fetch with exponential backoff (max 5 retries)
3. Deserialize based on `payload_type`

---

## Platform-Specific Architecture

### Julia Architecture

Julia leverages multiple dispatch for type-specific implementations:

- **Multiple Dispatch**: Function overloading based on argument types
- **Struct-based Data Models**: Explicit type definitions with `struct`
- **Native Arrow IPC**: Support via `Arrow.jl`
- **Async/Await**: Tasks for non-blocking I/O

```julia
# Multiple dispatch for serialization
function _serialize_data(data::String, payload_type::String)
    # Text serialization
end

function _serialize_data(data::Dict, payload_type::String)
    # Dictionary serialization
end

function _serialize_data(data::DataFrame, payload_type::String)
    # Arrow table serialization
end
```

### JavaScript Architecture

JavaScript uses async/await for non-blocking I/O:

- **Class-based NATS Client**: Connection management
- **Module-level Utilities**: Serialization functions
- **Native ArrayBuffer**: Binary data handling
- **Fetch API**: HTTP file server communication

```javascript
// Class-based NATS client
class NATSClient {
    constructor(url) {
        this.url = url;
        this.connection = null;
    }
    
    async connect() {
        this.connection = await nats.connect({ servers: this.url });
    }
}
```

### Python Architecture

Python uses classes for stateful operations:

- **Class-based NATSBridge**: Encapsulated API
- **Dataclasses**: Structured data (MsgPayloadV1, MsgEnvelopeV1)
- **Async/await**: I/O operations
- **pyarrow**: Arrow IPC support

```python
class NATSBridge:
    DEFAULT_SIZE_THRESHOLD = 500_000
    
    def __init__(self, broker_url=None, fileserver_url=None):
        self.broker_url = broker_url or self.DEFAULT_BROKER_URL
        self.fileserver_url = fileserver_url or self.DEFAULT_FILESERVER_URL
```

### MicroPython Architecture

MicroPython has significant constraints:

- **Synchronous API**: No async/await
- **Memory-constrained**: 256KB - 1MB
- **Limited payload support**: No tables, max 50KB
- **Simplified UUID generation**: Custom implementation

```python
# MicroPython constraints
DEFAULT_SIZE_THRESHOLD = 100_000  # 100KB
MAX_PAYLOAD_SIZE = 50_000  # 50KB hard limit
```

---

## Scaling Architecture

### Horizontal Scaling

| Component | Scaling Strategy |
|-----------|------------------|
| **NATS Server** | Cluster deployment with multiple nodes |
| **File Server** | Load balancer + multiple instances |
| **Client Applications** | Deploy multiple instances behind load balancer |

### Vertical Scaling

| Component | Scaling Strategy |
|-----------|------------------|
| **NATS Server** | Increase memory, CPU, disk I/O |
| **File Server** | Increase memory, CPU, disk capacity |
| **Client Applications** | Increase heap size (Python/JS) |

### Performance Considerations

| Metric | Target | Notes |
|--------|--------|-------|
| Message serialization overhead | <50ms | For 10KB payload |
| Message deserialization overhead | <50ms | For 10KB payload |
| NATS connection establishment | <100ms | Connection pool recommended |
| File upload latency | <1s | For 0.5MB file |
| File download latency | <1s | For 0.5MB file |

---

## Failure Modes and Recovery

### NATS Connection Failure

**Scenario**: NATS server unavailable

**Handler**:
- Connection auto-reconnect via TCP-level reconnection
- Retry with exponential backoff for publish operations

**Recovery**:
- NATS client automatically attempts reconnection
- Application can check connection status before publishing

### File Server Unavailable

**Scenario**: HTTP file server unavailable during upload/download

**Handler**:
- Retry up to 5 times with exponential backoff (100ms → 5000ms)
- Fallback to direct transport for upload (MicroPython)

**Recovery**:
- Exponential backoff: `delay = min(delay * 2, max_delay)`
- After max retries, throw error with correlation ID

### Deserialization Error

**Scenario**: Payload type mismatch or corrupted data

**Handler**:
- Log correlation ID and throw error
- No retry (data corruption)

**Recovery**:
- Application must validate payload_type matches data type
- Use proper serialization before sending

### Memory Overflow (MicroPython)

**Scenario**: Payload exceeds maximum size (50KB)

**Handler**:
- Reject payloads >50KB with MemoryError
- No retry (client-side check)

**Recovery**:
- Application must split large payloads
- Use direct transport only for small payloads

---

## Trade-off Decisions

### Decision 1: Direct vs Link Transport Threshold

**Trade-off**: Memory vs Network I/O

**Decision**: Use 0.5MB threshold for desktop, 100KB for MicroPython

**Rationale**:
- Direct transport uses more memory (Base64 encoding adds ~33% overhead)
- Link transport requires network I/O for upload/download
- 0.5MB is reasonable for desktop memory constraints
- 100KB is necessary for MicroPython memory constraints

### Decision 2: Base64 Encoding for Direct Transport

**Trade-off**: Bandwidth vs Simplicity

**Decision**: Use Base64 encoding for all direct transport payloads

**Rationale**:
- Simplifies JSON serialization (all data is string-compatible)
- Increases payload size by ~33%, but NATS can handle this
- Alternative would be binary payload support (more complex)

### Decision 3: Multiple Platform Implementations

**Trade-off**: Development effort vs Cross-platform support

**Decision**: Maintain separate implementations for each platform

**Rationale**:
- Each platform has idiomatic patterns (multiple dispatch, async/await, etc.)
- Maintains developer productivity and code quality
- API parity ensures cross-platform compatibility

### Decision 4: Handler Function Abstraction

**Trade-off**: Flexibility vs Simplicity

**Decision**: Abstract file server operations through handler functions

**Rationale**:
- Allows support for different file server implementations (Plik, AWS S3, custom)
- Maintains simplicity for common use cases
- Enables plug-in architecture for custom backends

---

## Deployment Architecture

### Minimum Infrastructure

| Component | Minimum | Notes |
|-----------|---------|-------|
| NATS Server | 1 instance | Single node for development |
| File Server | 1 instance | HTTP server for large payloads |
| Client Memory | 50MB | Desktop platforms |
| Client Memory | 256KB | MicroPython devices |

### Environment Variables

| Variable | Default | Description |
|----------|---------|-------------|
| `NATS_URL` | `nats://localhost:4222` | NATS server URL |
| `FILESERVER_URL` | `http://localhost:8080` | HTTP file server URL |
| `SIZE_THRESHOLD` | `1000000` | Size threshold in bytes |

### Container Deployment

```mermaid
flowchart TD
    subgraph "Docker Network"
        NATS_Container[NATS Server]
        FileServer_Container[Plik File Server]
        App_Container[Application Container]
    end
    
    App_Container -->|NATS| NATS_Container
    App_Container -->|HTTP| FileServer_Container
    
    style NATS_Container fill:#fff3e0,stroke:#f57c00
    style FileServer_Container fill:#f3e5f5,stroke:#9c27b4
    style App_Container fill:#e3f2fd,stroke:#2196f3
```

---

## Security Considerations

### Payload Integrity

**Mechanism**: SHA-256 checksum via metadata

**Implementation**:
- Sender calculates checksum and stores in payload metadata
- Receiver validates checksum on receipt

### Transport Security

**Mechanism**: TLS support for NATS connections

**Implementation**:
- Use `nats://` URL for plain text
- Use `tls://` URL for TLS-encrypted connections

### File Server Security

**Mechanism**: Authentication token for file uploads

**Implementation**:
- Plik uses upload token in `X-UploadToken` header
- Application can implement custom authentication

---

## Testing Architecture

### Unit Test Coverage

| Test Category | Coverage | Files |
|---------------|----------|-------|
| Serialization | All payload types | `test/test_*_sender.*` |
| Deserialization | All payload types | `test/test_*_receiver.*` |
| Transport selection | Direct vs link | `test/test_*_mix_payloads.*` |
| File server upload | Plik integration | Platform-specific |
| File server download | Exponential backoff | Platform-specific |

### Integration Test Scenarios

| Scenario | Platforms | Payloads | Transport | Expected Result |
|----------|-----------|----------|-----------|-----------------|
| Cross-platform text | Julia ↔ JS ↔ Python | text | direct | Round-trip successful |
| Arrow IPC round-trip | Julia ↔ JS ↔ Python | arrowtable | direct | Arrow IPC preserved |
| Large file transfer | All | image/audio/video | link | File server upload/download |
| Multi-payload mixed | All | text + image + file | direct/link | All payloads preserved |

---

## Versioning

### Architecture Versioning

| Component | Version | Notes |
|-----------|---------|-------|
| Architecture | 1.0.0 | Initial release |
| Protocol | v1 | Message envelope protocol version |

### Backward Compatibility

| Version | Supported Platforms |
|---------|---------------------|
| v1.0.x | Julia 1.7+, Node.js 16+, Python 3.8+, MicroPython 1.19+ |

---

## Change Log

| Date | Version | Changes |
|------|---------|---------|
| 2026-03-13 | 1.0.0 | Initial architecture documentation |

---

## References

- [`docs/requirements.md`](./requirements.md) - Business requirements and user stories
- [`docs/spec.md`](./spec.md) - Technical specification and contracts
- [`src/NATSBridge.jl`](../src/NATSBridge.jl) - Ground truth implementation
- [`README.md`](../README.md) - Project overview

---

*This architecture document is versioned and maintained in git alongside the codebase. All implementations must adhere to this architecture.*