diff --git a/README.md b/README.md index 3525db2..762bc15 100644 --- a/README.md +++ b/README.md @@ -60,33 +60,74 @@ NATSBridge enables seamless communication for Julia applications through NATS, w ### System Components +```mermaid +flowchart TB + subgraph Sender["Julia Application (Sender)"] + SenderApp[App Code] + NATSBridge_Send[NATSBridge] + NATS_Client[NATS.jl] + end + + subgraph Receiver["Julia Application (Receiver)"] + ReceiverApp[App Code] + NATSBridge_Recv[NATSBridge] + NATS_Client_Recv[NATS.jl] + end + + subgraph Infrastructure["Infrastructure"] + NATS[NATS Server
Message Broker] + FileServer[HTTP File Server
Upload/Download] + end + + SenderApp --> NATSBridge_Send + NATSBridge_Send --> NATS_Client + NATS_Client --> NATS + + NATS --> NATS_Client_Recv + NATS_Client_Recv --> NATSBridge_Recv + NATSBridge_Recv --> ReceiverApp + + NATSBridge_Send -.->|HTTP POST upload| FileServer + FileServer -.->|HTTP GET download| NATSBridge_Recv + + style SenderApp fill:#e8f5e9 + style ReceiverApp fill:#e8f5e9 + style NATS fill:#fff3e0 + style FileServer fill:#f3e5f5 ``` -┌─────────────────────────────────────────────────────────────────────┐ -│ NATSBridge Architecture │ -├─────────────────────────────────────────────────────────────────────┤ -│ ┌──────────────┐ │ │ -│ │ Julia │ ▼ │ -│ │ (NATS.jl) │ ┌─────────────────────────┐ │ -│ └──────────────┘ │ NATS │ │ -│ │ (Message Broker) │ │ -│ └─────────────────────────┘ │ -│ │ │ -│ ▼ │ -│ ┌──────────────────────┐ │ -│ │ File Server │ │ -│ │ (HTTP Upload/Get) │ │ -│ └──────────────────────┘ │ -└─────────────────────────────────────────────────────────────────────┘ -``` + +### Key Components + +| Component | Description | +|-----------|-------------| +| **Julia Application** | Sender and receiver applications using the NATSBridge module | +| **NATS Server** | Message broker for transporting message envelopes | +| **HTTP File Server** | Independent HTTP server for large payload storage (e.g., Plik) | ### Message Flow -1. **Sender** creates a message envelope with payloads -2. **NATSBridge** serializes and encodes payloads based on type -3. **Transport Decision**: Small payloads go directly to NATS, large payloads are uploaded to file server -4. **NATS** routes messages to subscribers -5. **Receiver** fetches payloads (from NATS or file server) -6. **NATSBridge** deserializes and decodes payloads +1. **Sender** creates a message envelope with payloads using `smartsend()` +2. **NATSBridge** serializes and encodes each payload based on type +3. **Transport Decision**: + - **Direct** (< 1MB): Payload encoded as Base64, published to NATS + - **Link** (≥ 1MB): Payload uploaded to HTTP file server, URL published to NATS +4. **NATS** routes message envelope to subscribers +5. **Receiver** receives message via NATS subscription callback +6. **NATSBridge** processes envelope: + - Decodes Base64 payloads from NATS message + - Fetches URLs from file server with exponential backoff +7. **Receiver** deserializes payloads based on their type + +### File Server Handler Abstraction + +The system uses handler functions to abstract file server operations: + +| Handler | Purpose | +|---------|---------| +| `plik_oneshot_upload()` | Uploads payload bytes to file server, returns URL | +| `_fetch_with_backoff()` | Downloads data from URL with exponential backoff retry | + +This abstraction allows support for different file server implementations (Plik, AWS S3, custom HTTP server). ---