update version

revert update

.gitignore

@@ -0,0 +1,3 @@
+node_modules/
+package.json
+package-lock.json

AI_prompt.txt

@@ -1,14 +0,0 @@
-Consider the following scenarios:
-Scenario 1: The "Command & Control" Loop (Low Latency)Focus: Small payloads, Core NATS, bi-directional JSON.The Action: A user on a JavaScript dashboard clicks a "Start Simulation" button. This sends a JSON configuration (parameters like step_size and iterations) to Julia.The Flow: * JS (Sender): Recognizes the message is small ($< 10KB$). Packages it as a direct transport JSON envelope.Julia (Receiver): Listens on the NATS subject, decodes the JSON, and immediately acknowledges receipt with a "Running" status.Project Requirement Met: Fast, low-overhead communication for control signals without involving the fileserver.
-Scenario 2: The "Deep Dive" Analysis (High Bandwidth)Focus: Large Arrow tables, Claim-Check pattern, Julia to JS.The Action: Julia finishes a heavy computation and produces a 500MB DataFrame with 10 million rows. It needs to send this to the JS frontend for visualization (e.g., using Perspective.js or D3).The Flow:Julia (Sender): Converts the DataFrame to an Arrow IPC stream. It sees the size is $> 1MB$, so it uploads the bytes to the HTTP fileserver. It then publishes a NATS message with transport: "link" and the URL.JS (Receiver): Receives the URL, fetches the data via fetch(), and uses tableFromIPC() to load the data into memory with zero-copy.Project Requirement Met: Handling massive datasets that exceed NATS message limits while maintaining data integrity across languages.
-Scenario 3: Live Audio/Signal Processing (Multimedia & Metadata)Focus: Raw binary, bi-directional streaming, NATS Headers.The Action: The JS client captures a 2-second "chunk" of microphone audio. It needs Julia to perform a Fast Fourier Transform (FFT) or AI transcription.The Flow:JS (Sender): Sends the raw binary WAV/PCM data. It uses NATS Headers to store the metadata ($fs = 44.1kHz$, $channels = 1$) to keep the payload purely binary.Julia (Receiver): Processes the audio and sends back a JSON result (the transcription) and an Arrow Table (the frequency spectrum data).Project Requirement Met: Bi-directional flow involving mixed media (Audio) and technical results (Arrow).
-Scenario 4: The "Catch-Up" (Persistence & JetStream)Focus: NATS JetStream, late-joining consumers, state sync.The Action: Julia is constantly publishing "System Health" updates. The JS dashboard is closed for 10 minutes. When the user re-opens the dashboard, they need to see the last 10 minutes of history.The Flow:NATS (Server): Uses a JetStream with a Limits retention policy.JS (Consumer): Connects and requests a "Replay" from the last 10 minutes. It receives a mix of direct (small updates) and link (historical snapshots) messages.Project Requirement Met: Temporal decoupling—consumers can receive data that was sent while they were offline.
-
-Role: Principal Systems Architect & Lead Software Engineer.Objective: Implement a high-performance, bi-directional data bridge between a Julia service and a JavaScript (Node.js) service using NATS (Core & JetStream).⚠️ STRICT ARCHITECTURAL CONSTRAINTS (Non-Negotiable)Transport Strategy (Claim-Check Pattern):Direct Path: If payload is < 1MB, send data directly via NATS inside the message envelope (Base64 encoded).Link Path: If payload is > 1MB, upload to a shared HTTP fileserver/store. The NATS message must only contain the metadata and the download URL.Tabular Data Format: * MUST use Apache Arrow IPC Stream for all tables/DataFrames. No CSV or standard JSON-serialization of tables allowed.System Symmetry: * Both services must function as Producers AND Consumers.Modular Elegance: * Implementation must be abstracted into a SmartSend function and a SmartReceive handler. The developer calling these functions should not need to care if the data is going via NATS direct or HTTP link.Technical Stack & Use CasesJulia: NATS.jl, Arrow.jl, JSON3.jl, HTTP.jl.Node.js: nats.js, apache-arrow.Scenarios to Support: * Large Data: Sending a 500MB Arrow table from Julia $\rightarrow$ JS.Media: Sending a 5MB WAV file from JS $\rightarrow$ Julia.Signals: Sending small JSON control commands ($< 10KB$) directly via NATS.Implementation Requirements1. Unified JSON Envelope:Define a schema containing: correlation_id (UUID), type (table/binary/json), transport (direct/link), payload (if direct), and url (if link).2. The Julia Module:Implement SmartSend(subject, data, type): Handles Arrow serialization to an IOBuffer, checks size, and manages HTTP uploads for large blobs.Implement SmartReceive(msg): Parses envelope, handles the HTTP fetch with Exponential Backoff (to avoid race conditions), and restores the DataFrame.Include a basic HTTP.listen server to serve as the temporary storage.3. The JavaScript Module:Implement a symmetric SmartSend using nats.js and apache-arrow.Implement a JetStream Pull Consumer for SmartReceive to ensure backpressure and memory safety.4. Performance & Reliability:Demonstrate "Zero-Copy" reading of the Arrow IPC stream on the JS side.Log the correlation_id at every stage for distributed tracing.
-
-
-
-
-
-
-

DO_NOT_READ_AI_prompt.txt

@@ -0,0 +1,154 @@
+Consider the following scenarios:
+Scenario 1: The "Command & Control" Loop (Low Latency)Focus: Small payloads, Core NATS, bi-directional JSON.The Action: A user on a JavaScript dashboard clicks a "Start Simulation" button. This sends a JSON configuration (parameters like step_size and iterations) to Julia.The Flow: * JS (Sender): Recognizes the message is small ($< 10KB$). Packages it as a direct transport JSON envelope.Julia (Receiver): Listens on the NATS subject, decodes the JSON, and immediately acknowledges receipt with a "Running" status.Project Requirement Met: Fast, low-overhead communication for control signals without involving the fileserver.
+Scenario 2: The "Deep Dive" Analysis (High Bandwidth)Focus: Large Arrow tables, Claim-Check pattern, Julia to JS.The Action: Julia finishes a heavy computation and produces a 500MB DataFrame with 10 million rows. It needs to send this to the JS frontend for visualization (e.g., using Perspective.js or D3).The Flow:Julia (Sender): Converts the DataFrame to an Arrow IPC stream. It sees the size is $> 1MB$, so it uploads the bytes to the HTTP fileserver. It then publishes a NATS message with transport: "link" and the URL.JS (Receiver): Receives the URL, fetches the data via fetch(), and uses tableFromIPC() to load the data into memory with zero-copy.Project Requirement Met: Handling massive datasets that exceed NATS message limits while maintaining data integrity across languages.
+Scenario 3: Live Audio/Signal Processing (Multimedia & Metadata)Focus: Raw binary, bi-directional streaming, NATS Headers.The Action: The JS client captures a 2-second "chunk" of microphone audio. It needs Julia to perform a Fast Fourier Transform (FFT) or AI transcription.The Flow:JS (Sender): Sends the raw binary WAV/PCM data. It uses NATS Headers to store the metadata ($fs = 44.1kHz$, $channels = 1$) to keep the payload purely binary.Julia (Receiver): Processes the audio and sends back a JSON result (the transcription) and an Arrow Table (the frequency spectrum data).Project Requirement Met: Bi-directional flow involving mixed media (Audio) and technical results (Arrow).
+Scenario 4: The "Catch-Up" (Persistence & JetStream)Focus: NATS JetStream, late-joining consumers, state sync.The Action: Julia is constantly publishing "System Health" updates. The JS dashboard is closed for 10 minutes. When the user re-opens the dashboard, they need to see the last 10 minutes of history.The Flow:NATS (Server): Uses a JetStream with a Limits retention policy.JS (Consumer): Connects and requests a "Replay" from the last 10 minutes. It receives a mix of direct (small updates) and link (historical snapshots) messages.Project Requirement Met: Temporal decoupling—consumers can receive data that was sent while they were offline.
+
+Role: Principal Systems Architect & Lead Software Engineer.Objective: Implement a high-performance, bi-directional data bridge between a Julia service and a JavaScript (Node.js) service using NATS (Core & JetStream).⚠️ STRICT ARCHITECTURAL CONSTRAINTS (Non-Negotiable)Transport Strategy (Claim-Check Pattern):Direct Path: If payload is < 1MB, send data directly via NATS inside the message envelope (Base64 encoded).Link Path: If payload is > 1MB, upload to a shared HTTP fileserver/store. The NATS message must only contain the metadata and the download URL.Tabular Data Format: * MUST use Apache Arrow IPC Stream for all tables/DataFrames. No CSV or standard JSON-serialization of tables allowed.System Symmetry: * Both services must function as Producers AND Consumers.Modular Elegance: * Implementation must be abstracted into a SmartSend function and a SmartReceive handler. The developer calling these functions should not need to care if the data is going via NATS direct or HTTP link.Technical Stack & Use CasesJulia: NATS.jl, Arrow.jl, JSON3.jl, HTTP.jl.Node.js: nats.js, apache-arrow.Scenarios to Support: * Large Data: Sending a 500MB Arrow table from Julia $\rightarrow$ JS.Media: Sending a 5MB WAV file from JS $\rightarrow$ Julia.Signals: Sending small JSON control commands ($< 10KB$) directly via NATS.Implementation Requirements1. Unified JSON Envelope:Define a schema containing: correlation_id (UUID), type (table/binary/json), transport (direct/link), payload (if direct), and url (if link).2. The Julia Module:Implement SmartSend(subject, data, type): Handles Arrow serialization to an IOBuffer, checks size, and manages HTTP uploads for large blobs.Implement SmartReceive(msg): Parses envelope, handles the HTTP fetch with Exponential Backoff (to avoid race conditions), and restores the DataFrame.Include a basic HTTP.listen server to serve as the temporary storage.3. The JavaScript Module:Implement a symmetric SmartSend using nats.js and apache-arrow.Implement a JetStream Pull Consumer for SmartReceive to ensure backpressure and memory safety.4. Performance & Reliability:Demonstrate "Zero-Copy" reading of the Arrow IPC stream on the JS side.Log the correlation_id at every stage for distributed tracing.
+
+
+
+
+
+
+
+Create a walkthrough for Julia service-A service sending a mix-content chat message to Julia service-B. the chat message must includes 
+
+
+
+
+
+I updated the following:
+- NATSBridge.jl. Essentially I add NATS_connection keyword and new publish_message function to support the keyword. 
+Use them and ONLY them as ground truth.
+Then update the following files accordingly:
+- architecture.md
+- implementation.md
+
+All API should be semantically consistent and naming should be consistent across the board.
+
+
+
+
+
+
+
+
+
+Task: Update NATSBridge.js to reflect recent changes in NATSBridge.jl and docs
+Context: NATSBridge.jl and docs has been updated.
+Requirements:
+Source of Truth: Treat the updated NATSBridge.jl and docs as the definitive source.
+API Consistency: Ensure the Main Package API (e.g., smartsend(), publish_message()) uses consistent naming across all three supported languages.
+Ecosystem Variance: Low-level native functions (e.g., NATS.connect(), JSON.read()) should follow the conventions of the specific language ecosystem and do not require cross-language consistency.
+
+
+
+
+
+
+
+I'm expanding this Julia package (NATSBridge) into a cross-platform project by adding a JavaScript and Python/MicroPython implementation. To ensure accuracy, the Julia src directory will serve as the ground truth, as the documentation may be outdated.
+
+My goal is to maintain interface parity at the high-level API for a consistent user experience, while ensuring the low-level implementation adheres strictly to the idiomatic conventions of each respective language (e.g., multiple dispatch in Julia vs. asynchronous, prototype, or class-based patterns in JS and Python/MicroPython)
+
+Now, help me do the following:
+1) check architecture.md for any mistake.
+
+
+
+
+Help me expands this Julia package (NATSBridge) into a cross-platform project by adding a JavaScript and Python/MicroPython implementation. To ensure accuracy,  NATSBridge.jl will serve as the ground truth, as the documentation may be outdated.
+
+My goal is to maintain interface parity at the high-level API for a consistent user experience, while ensuring the low-level implementation adheres strictly to the idiomatic conventions of each respective language (e.g., multiple dispatch in Julia vs. asynchronous, prototype, or class-based patterns in JS and Python/MicroPython)
+
+Now do the following:
+1) check docs to see if there is any mistake.
+
+
+
+
+
+I'm expanding this Julia package (NATSBridge) into a cross-platform project by adding 
+a JavaScript, Python and MicroPython implementation. 
+The following will serve as the ground truth:
+- test_julia_mix_payloads_sender.jl
+- NATSBridge.jl
+- test_julia_mix_payloads_receiver.jl
+- architecture.md
+
+My goal is to maintain interface parity at the high-level API for a consistent user experience, 
+while ensuring the low-level implementation adheres strictly to the idiomatic conventions of each 
+respective language (e.g., multiple dispatch in Julia vs. asynchronous, prototype, or class-based 
+patterns in JS, Python and MicroPython)
+
+Now, help me do the following:
+1) Check whether natsbridge.js needs update or it already up to date.
+
+
+
+
+
+
+
+# ---------------------------------------------- 100 --------------------------------------------- #
+
+Got it — let’s rebuild your table in my own teaching style, keeping it crisp, intuitive, and easy for students to grasp. I’ll emphasize **purpose, audience, format, example, and KPI** in a way that flows like a story of how projects move from idea → contract → design → code → review → operations.
+
+---
+
+### SDD + GitOps Documentation Framework
+
+| Document        | Purpose (Rationale) | Primary Audience | Format / Content | Example (SaaS Context) | Measurement (KPI) |
+|-----------------|---------------------|-----------------|------------------|------------------------|-------------------|
+| **Requirements** | Capture the **business intent** — why we’re building this and what success looks like. Defines boundaries and user‑visible outcomes. | Stakeholders, Product Owners, Lead Developers | User stories, PRDs, acceptance criteria, non‑functional constraints. | “System must process tabular data from Julia to SvelteKit UI with <200ms latency for 5‑member teams.” | 95% of requests complete <200ms (synthetic monitoring). |
+| **Specification** | The **technical contract** — precise rules for inputs, outputs, and data shape. Ensures consistency across dev and test. | Developers, QA Engineers, CI/CD pipelines | OpenAPI, Protobuf, AsyncAPI. Endpoint definitions, schemas, error codes. | `contract.yaml` defining a NATS subject that accepts Arrow streams with snake_case headers. | 100% of messages validated against spec (CI block rate). |
+| **Architecture** | The **blueprint** — how components fit together, interact, and scale. Guides system structure and trade‑offs. | Architects, Senior Developers, DevOps | C4 diagrams, Mermaid.js, component/network/storage models. | Diagram showing 6‑node cluster routing traffic via Caddy → Node.js API → Julia pods. | 100% of major decisions logged with trade‑off analysis. |
+| **Walkthrough** | The **story of flow** — shows how pieces connect end‑to‑end and why steps are sequenced. Builds intuition for new devs. | New Developers, Team Members | TOUR.md, Loom videos, sequence diagrams. Step‑by‑step traces with rationale. | “UI sends JSON → Node.js wraps Claim‑Check → Julia pulls Arrow data (prevents NATS overflow).” | New developers ship feature in <2 days (PR timeline). |
+| **Implementation** | The **real code** — business logic, helpers, tests, configs. Where design becomes executable. | Developers, Code Reviewers | Source code, README.md, unit tests, setup scripts. | Julia function for matrix calculation + SvelteKit component rendering table. | >80% unit test coverage, <5% drift from spec. |
+| **Validation** | The **enforcer** — ensures implementation matches the spec. Blocks drift and human error. | Automation servers, QA, Lead Developers | CI jobs, contract tests, linting, integration checks. | CI job rejects PR with camelCase field not allowed by YAML spec. | <1% of PRs bypass validation gates. |
+| **Runbook** | The **operational manual** — how the system lives in production, scales, and recovers. Guides on‑call engineers. | DevOps, SREs, On‑call Developers | K8s manifests, Helm charts, Markdown guides. Deployment, scaling, backup/restore, troubleshooting. | GitOps manifest ensuring 6 Julia replicas restart if memory >80%. | MTTR <15 minutes for P1 incidents. |
+
+
+
+
+
+
+
+
+# ---------------------------------------------- 100 --------------------------------------------- #
+
+SDD + GitOps Documentation Stack
+Document,"Purpose (The ""Rationale"")",Primary Audience,Format / Content,Example (SaaS Context),"Measurement (KPI)"
+Requirements,"Defines the ""Why"" and the Business Boundary. It sets the constraints and success criteria so the team knows when a feature is ""done"" from a user's perspective.","Stakeholders, Product Owners, Lead Developers","Format: User Stories, PRDs. Content: Functional goals, non-functional requirements (latency, scale), and explicit ""out-of-scope"" items.","""The system must process high-volume tabular data from Julia to the SvelteKit UI with <200ms latency for 5-member teams."",""Pass/Fail: 95% of requests complete <200ms (measured via synthetic monitoring)""
+The Spec,"The Technical Contract. It serves as the single source of truth that defines the shape of data. In SDD, this file drives code generation and automated testing.","Developers, QA Engineers, CI/CD Pipelines","Format: OpenAPI (YAML), Protobuf, AsyncAPI. Content: Endpoint definitions, strict data types, error codes, and request/response schemas.",A contract.yaml defining a NATS subject that accepts an Apache Arrow stream with specific snake_case headers.",""Schema Validation Rate: 100% of messages validated against spec (CI block rate)""
+Architecture,"The Structural Blueprint. It explains how the ""pieces"" are arranged in the cluster. It defines the relationships between services, databases, and external providers.","System Architects, Senior Developers, DevOps","Format: C4 Model Diagrams, Mermaid.js. Content: Component diagrams, network flow, storage strategy, and technology stack definitions.",A diagram showing how the 6-node cluster routes traffic through Caddy to the Node.js API and offloads heavy math to Julia pods.",""Architecture Decision Log: 100% of major decisions documented with trade-off analysis""
+Walkthrough,"The Intuition & Flow. It connects multiple APIs/services into a cohesive end-to-end story. It explains the ""steps"" and the ""rationale"" behind the sequence of operations.","New Developers, Current Team Members","Format: TOUR.md, Loom videos, Sequence Diagrams. Content: Step-by-step trace of a feature, explanation of state changes, and the ""why"" behind complex logic.","""End-to-End Trace:"" 1. UI sends JSON to Node.js. 2. Node.js wraps it in a Claim-Check. 3. Julia pulls the Arrow data. Rationale: This prevents NATS memory overflow.",""Onboarding Velocity: New developers deploy feature in <2 days (tracked via PR timeline)""
+Implementation,"The Functional Reality. This is the actual execution of the logic. In SDD, parts of this are auto-generated to ensure it never drifts from the Spec.","Developers, Code Reviewers","Format: Source Code (Git), README.md. Content: Business logic, internal helper functions, unit tests, and local setup instructions.",The Julia function that performs the matrix calculation and the SvelteKit component that renders the resulting table.",""Code Coverage: >80% unit test coverage, <5% test drift from spec""
+Validation,"The Enforcement Layer. It ensures that the ""Reality"" (Code) actually matches the ""Contract"" (Spec). It prevents human error from breaking the system.","Automation Servers, QA, Lead Developers","Format: GitHub Actions, Dredd, Prism. Content: Contract tests, linting rules, and integration tests that check API compliance.",A CI job that blocks a Pull Request because a developer added a camelCase field that isn't allowed in the shared YAML spec.",""Block Rate: <1% of PRs reach production without validation (CI gate pass rate)""
+Runbook,"The Operational Life-Support. It defines how the system lives in production and how to fix it. In GitOps, the ""State"" is declared here.","DevOps, SREs, On-call Developers","Format: K8s Manifests, Helm Charts, Markdown. Content: Deployment steps, scaling triggers, backup/restore commands, and troubleshooting guides.",A GitOps manifest in Flux that ensures 6 replicas of the Julia service are always running and restarts them if memory hits 80%.",""MTTR: <15 minutes for P1 incidents (tracked via incident management system)""
+
+Do you understand the provided text? Don't fucking change the table content. I want you to add "Measurement (KPI)" column. it is only example of course. This table will be used for consult and teaching.
+
+
+# ---------------------------------------------- 100 --------------------------------------------- #
+
+Can you write the table and explain this approach and each doc in details then save to docs/SDD_FRAMEWORK.md so I can consult it later.
+Don't forget to add How to use this approach effectively.
+
+
+# ---------------------------------------------- 100 --------------------------------------------- #
+
+Since I develop src folder before I adopt SDD_FRAMEWORK.md approach, can you check src folder and my current doc files then write docs/requirements.md  according to SDD framework? Treat src as ground truth.
+
+# ---------------------------------------------- 100 --------------------------------------------- #
+
+
+
+
+
+
+
+
+
+

Manifest.toml

@@ -1,8 +1,8 @@
 # This file is machine-generated - editing it directly is not advised
 
-julia_version = "1.12.4"
+julia_version = "1.12.5"
 manifest_format = "2.0"
-project_hash = "be1e3c2d8b7f4f0ee7375c94aaf704ce73ba57b9"
+project_hash = "b632f853bcf5355f5c53ad3efa7a19f70444dc6c"
 
 [[deps.AliasTables]]
 deps = ["PtrArrays", "Random"]
@@ -436,6 +436,12 @@ git-tree-sha1 = "d9d9a189fb9155a460e6b5e8966bf6a66737abf8"
 uuid = "55e73f9c-eeeb-467f-b4cc-a633fde63d2a"
 version = "0.1.0"
 
+[[deps.NATSBridge]]
+deps = ["Arrow", "DataFrames", "Dates", "GeneralUtils", "HTTP", "JSON", "NATS", "PrettyPrinting", "Revise", "UUIDs"]
+path = "."
+uuid = "f2724d33-f338-4a57-b9f8-1be882570d10"
+version = "0.4.1"
+
 [[deps.NanoDates]]
 deps = ["Dates", "Parsers"]
 git-tree-sha1 = "850a0557ae5934f6e67ac0dc5ca13d0328422d1f"

Plik_API_document.md

@@ -1,194 +0,0 @@
-### API
-Plik server expose a REST-full API to manage uploads and get files :
-
-Get and create upload :
- 
-   - **POST**        /upload
-     - Params (json object in request body) :
-      - oneshot (bool)
-      - stream (bool)
-      - removable (bool)
-      - ttl (int)
-      - login (string)
-      - password (string)
-      - files (see below)
-     - Return :
-         JSON formatted upload object.
-         Important fields :
-           - id (required to upload files)
-           - uploadToken (required to upload/remove files)
-           - files (see below)
-
-   For stream mode you need to know the file id before the upload starts as it will block.
-   File size and/or file type also need to be known before the upload starts as they have to be printed 
-   in HTTP response headers.
-   To get the file ids pass a "files" json object with each file you are about to upload.
-   Fill the reference field with an arbitrary string to avoid matching file ids using the fileName field.
-   This is also used to notify of MISSING files when file upload is not yet finished or has failed.
-  ```
-  "files" : [
-    {
-      "fileName": "file.txt",
-      "fileSize": 12345,
-      "fileType": "text/plain",
-      "reference": "0"
-    },...
-  ]
-  ```
-  
-   - **GET** /upload/:uploadid:
-     - Get upload metadata (files list, upload date, ttl,...)
-
-Upload file :
-
-   - **POST** /$mode/:uploadid:/:fileid:/:filename:
-     - Request body must be a multipart request with a part named "file" containing file data.
-
-   - **POST** /file/:uploadid:
-     - Same as above without passing file id, won't work for stream mode.
-     
-   - **POST** /:
-     - Quick mode, automatically create an upload with default parameters and add the file to it.
-
-Get file :
-
-  - **HEAD** /$mode/:uploadid:/:fileid:/:filename:
-    - Returns only HTTP headers. Useful to know Content-Type and Content-Length without downloading the file. Especially if upload has OneShot option enabled.
-
-  - **GET**  /$mode/:uploadid:/:fileid:/:filename:
-    - Download file. Filename **MUST** match. A browser, might try to display the file if it's a jpeg for example. You may try to force download with ?dl=1 in url.
-
-  - **GET**  /archive/:uploadid:/:filename:
-    - Download uploaded files in a zip archive. :filename: must end with .zip
-
-Remove file :
-
-   - **DELETE** /$mode/:uploadid:/:fileid:/:filename:
-     - Delete file. Upload **MUST** have "removable" option enabled.
-
-Show server details :
-
-   - **GET** /version
-     - Show plik server version, and some build information (build host, date, git revision,...)
-
-   - **GET** /config
-     - Show plik server configuration (ttl values, max file size, ...)
-
-   - **GET** /stats
-     - Get server statistics ( upload/file count, user count, total size used )
-     - Admin only
-
-User authentication :
-
-   - 
-   Plik can authenticate users using Google and/or OVH third-party API.   
-   The /auth API is designed for the Plik web application nevertheless if you want to automatize it be sure to provide a valid
-   Referrer HTTP header and forward all session cookies.   
-   Plik session cookies have the "secure" flag set, so they can only be transmitted over secure HTTPS connections.   
-   To avoid CSRF attacks the value of the plik-xsrf cookie MUST be copied in the X-XSRFToken HTTP header of each
-   authenticated request.   
-   Once authenticated a user can generate upload tokens. Those tokens can be used in the X-PlikToken HTTP header used to link
-   an upload to the user account. It can be put in the ~/.plikrc file of the Plik command line client.   
-   
-   - **Local** :
-      - You'll need to create users using the server command line
-   
-   - **Google** :
-      - You'll need to create a new application in the [Google Developper Console](https://console.developers.google.com)
-      - You'll be handed a Google API ClientID and a Google API ClientSecret that you'll need to put in the plikd.cfg file
-      - Do not forget to whitelist valid origin and redirect url ( https://yourdomain/auth/google/callback ) for your domain
-   
-   - **OVH** :
-      - You'll need to create a new application in the OVH API : https://eu.api.ovh.com/createApp/
-      - You'll be handed an OVH application key and an OVH application secret key that you'll need to put in the plikd.cfg file
-
-   - **GET** /auth/google/login
-      - Get Google user consent URL. User have to visit this URL to authenticate
-
-   - **GET** /auth/google/callback
-     - Callback of the user consent dialog
-     - The user will be redirected back to the web application with a Plik session cookie at the end of this call
-
-   - **GET** /auth/ovh/login
-     - Get OVH user consent URL. User have to visit this URL to authenticate
-     - The response will contain a temporary session cookie to forward the API endpoint and OVH consumer key to the callback
-
-   - **GET** /auth/ovh/callback
-     - Callback of the user consent dialog. 
-     - The user will be redirected back to the web application with a Plik session cookie at the end of this call
-
-   - **POST** /auth/local/login
-     - Params :
-       - login : user login
-       - password : user password
-
-   - **GET** /auth/logout
-     - Invalidate Plik session cookies
-
-   - **GET** /me
-     - Return basic user info ( ID, name, email ) and tokens
-
-   - **DELETE** /me
-     - Remove user account.
-
-   - **GET** /me/token
-     - List user tokens
-      - This call use pagination
-
-   - **POST** /me/token
-     - Create a new upload token
-     - A comment can be passed in the json body
-
-   - **DELETE** /me/token/{token}
-     - Revoke an upload token
-
-   - **GET** /me/uploads
-     - List user uploads
-     - Params :
-        - token : filter by token
-      - This call use pagination
-
-   - **DELETE** /me/uploads
-     - Remove all uploads linked to a user account
-     - Params :
-        - token : filter by token
-
-   - **GET** /me/stats
-     - Get user statistics ( upload/file count, total size used )
-
-   - **GET** /users
-     - List all users
-     - This call use pagination
-     - Admin only 
-
-QRCode :
-
-   - **GET** /qrcode
-     - Generate a QRCode image from an url
-     - Params :
-        - url  : The url you want to store in the QRCode
-        - size : The size of the generated image in pixels (default: 250, max: 1000)
-
-
-$mode can be "file" or "stream" depending if stream mode is enabled. See FAQ for more details.
-
-Examples :
-```sh
-Create an upload (in the json response, you'll have upload id and upload token)
-$ curl -X POST http://127.0.0.1:8080/upload
-
-Create a OneShot upload
-$ curl -X POST -d '{ "OneShot" : true }' http://127.0.0.1:8080/upload
-
-Upload a file to upload
-$ curl -X POST --header "X-UploadToken: M9PJftiApG1Kqr81gN3Fq1HJItPENMhl" -F "file=@test.txt" http://127.0.0.1:8080/file/IsrIPIsDskFpN12E
-
-Get headers
-$ curl -I http://127.0.0.1:8080/file/IsrIPIsDskFpN12E/sFjIeokH23M35tN4/test.txt
-HTTP/1.1 200 OK
-Content-Disposition: filename=test.txt
-Content-Length: 3486
-Content-Type: text/plain; charset=utf-8
-Date: Fri, 15 May 2015 09:16:20 GMT
-
-```

Project.toml

@@ -1,8 +1,21 @@
+name = "NATSBridge"
+uuid = "f2724d33-f338-4a57-b9f8-1be882570d10"
+version = "0.5.5"
+authors = ["narawat <narawat@gmail.com>"]
+
 [deps]
 Arrow = "69666777-d1a9-59fb-9406-91d4454c9d45"
+Base64 = "2a0f44e3-6c83-55bd-87e4-b1978d98bd5f"
+DataFrames = "a93c6f00-e57d-5684-b7b6-d8193f3e46c0"
 Dates = "ade2ca70-3891-5945-98fb-dc099432e06a"
 GeneralUtils = "c6c72f09-b708-4ac8-ac7c-2084d70108fe"
 HTTP = "cd3eb016-35fb-5094-929b-558a96fad6f3"
 JSON = "682c06a0-de6a-54ab-a142-c8b1cf79cde6"
 NATS = "55e73f9c-eeeb-467f-b4cc-a633fde63d2a"
+PrettyPrinting = "54e16d92-306c-5ea0-a30b-337be88ac337"
+Revise = "295af30f-e4ad-537b-8983-00126c2a3abe"
 UUIDs = "cf7118a7-6976-5b1a-9a39-7adc72f591a4"
+
+[compat]
+Base64 = "1.11.0"
+JSON = "1.4.0"

README.md

@@ -0,0 +1,672 @@
+# NATSBridge - Cross-Platform Bi-Directional Data Bridge
+
+A high-performance, bi-directional data bridge for **Julia, JavaScript, Python, and MicroPython** applications using NATS (Core & JetStream), implementing the Claim-Check pattern for large payloads.
+
+[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](https://opensource.org/licenses/MIT)
+[![NATS](https://img.shields.io/badge/NATS-Enabled-green.svg)](https://nats.io)
+
+---
+
+## Table of Contents
+
+- [Overview](#overview)
+- [Cross-Platform Support](#cross-platform-support)
+- [Features](#features)
+- [Quick Start](#quick-start)
+- [API Reference](#api-reference)
+- [Payload Types](#payload-types)
+- [Cross-Platform Examples](#cross-platform-examples)
+- [Testing](#testing)
+- [Documentation](#documentation)
+- [License](#license)
+
+---
+
+## Overview
+
+NATSBridge enables seamless communication across multiple platforms through NATS, with intelligent transport selection based on payload size:
+
+| Transport | Payload Size | Method |
+|-----------|--------------|--------|
+| **Direct** | < 1MB | Sent directly via NATS (Base64 encoded) |
+| **Link** | >= 1MB | Uploaded to HTTP file server, URL sent via NATS |
+
+### Use Cases
+
+- **Chat Applications**: Text, images, audio, video in a single message
+- **File Transfer**: Efficient transfer of large files using claim-check pattern
+- **IoT/Embedded**: Sensor data, telemetry, and analytics pipelines (MicroPython)
+- **Cross-Platform Communication**: Interoperability between Julia, JavaScript, Python, and MicroPython systems
+
+---
+
+## Cross-Platform Support
+
+| Platform | Implementation | Features |
+|----------|----------------|----------|
+| **Julia** | [`src/NATSBridge.jl`](src/NATSBridge.jl) | Full feature set, Arrow IPC, multiple dispatch |
+| **JavaScript** | [`src/natsbridge.js`](src/natsbridge.js) | Node.js, async/await |
+| **JavaScript (Browser)** | [`src/natsbridge_csr.js`](src/natsbridge_csr.js) | Browser, WebSocket NATS, async/await |
+| **Python** | [`src/natsbridge.py`](src/natsbridge.py) | Desktop Python, asyncio, type hints |
+| **MicroPython** | [`src/natsbridge_mpy.py`](src/natsbridge_mpy.py) | Memory-constrained, synchronous API |
+
+### Platform Comparison
+
+| Feature | Julia | JavaScript | JavaScript (Browser) | Python | MicroPython |
+|---------|-------|------------|----------------------|--------|-------------|
+| Multiple Dispatch | ✅ Native | ❌ | ❌ | ❌ | ❌ |
+| Async/Await | ❌ | ✅ Native | ✅ Native | ✅ Native | ⚠️ (uasyncio) |
+| Type Safety | ✅ Strong | ⚠️ (TypeScript) | ⚠️ (TypeScript) | ✅ (Type hints) | ❌ |
+| Arrow IPC | ✅ Native | ✅ | ✅ | ✅ | ❌ |
+| Direct Transport | ✅ | ✅ | ✅ | ✅ | ✅ |
+| Link Transport | ✅ | ✅ | ✅ | ✅ | ⚠️ (Limited) |
+| Handler Functions | ✅ | ✅ | ✅ | ✅ | ✅ |
+| Cross-Platform API | ✅ | ✅ | ✅ | ✅ | ✅ |
+| WebSocket NATS | ❌ | ❌ | ✅ | ❌ | ❌ |
+
+---
+
+## Features
+
+- ✅ **Cross-platform messaging** for Julia, JavaScript, Python, and MicroPython applications
+- ✅ **Bi-directional messaging** with request-reply patterns
+- ✅ **Multi-payload support** - send multiple payloads with different types in one message
+- ✅ **Automatic transport selection** - direct vs link based on payload size
+- ✅ **Claim-Check pattern** for payloads > 1MB
+- ✅ **Apache Arrow IPC** support for tabular data (zero-copy reading)
+- ✅ **Exponential backoff** for reliable file server downloads
+- ✅ **Correlation ID tracking** for message tracing
+- ✅ **Reply-to support** for request-response patterns
+- ✅ **Handler function abstraction** - pluggable file server implementations (Plik, AWS S3, custom)
+
+---
+
+## Quick Start
+
+### Step 1: Start NATS Server
+
+```bash
+docker run -p 4222:4222 nats:latest
+```
+
+### Step 2: Start HTTP File Server (Optional)
+
+```bash
+# Create a directory for file uploads
+mkdir -p /tmp/fileserver
+
+# Start HTTP file server
+python3 -m http.server 8080 --directory /tmp/fileserver
+```
+
+### Step 3: Send Your First Message
+
+#### Julia
+
+```julia
+using NATSBridge
+
+data = [("message", "Hello World", "text")]
+env, env_json_str = smartsend("/chat/room1", data, broker_url="nats://localhost:4222")
+println("Message sent!")
+```
+
+#### JavaScript
+
+```javascript
+const NATSBridge = require('./src/natsbridge.js');
+
+const data = [["message", "Hello World", "text"]];
+const [env, env_json_str] = await NATSBridge.smartsend(
+    "/chat/room1",
+    data,
+    { broker_url: "nats://localhost:4222" }
+);
+console.log("Message sent!");
+```
+
+#### Python
+
+```python
+from natsbridge import smartsend
+
+data = [("message", "Hello World", "text")]
+env, env_json_str = await smartsend(
+    "/chat/room1",
+    data,
+    broker_url="nats://localhost:4222"
+)
+print("Message sent!")
+```
+
+---
+
+## API Reference
+
+### Unified API Standard
+
+All platforms use the same input/output format for payloads:
+
+**Input format for smartsend:**
+```
+[(dataname1, data1, type1), (dataname2, data2, type2), ...]
+```
+
+**Output format for smartreceive:**
+```
+{
+  "correlation_id": "...",
+  "msg_id": "...",
+  "timestamp": "...",
+  "send_to": "...",
+  "msg_purpose": "...",
+  "sender_name": "...",
+  "sender_id": "...",
+  "receiver_name": "...",
+  "receiver_id": "...",
+  "reply_to": "...",
+  "reply_to_msg_id": "...",
+  "broker_url": "...",
+  "metadata": {...},
+  "payloads": [(dataname1, data1, type1), (dataname2, data2, type2), ...]
+}
+```
+
+### smartsend
+
+Sends data either directly via NATS or via a fileserver URL, depending on payload size.
+
+#### Julia
+
+```julia
+using NATSBridge
+
+env, env_json_str = NATSBridge.smartsend(
+    subject::String,
+    data::AbstractArray{Tuple{String, Any, String}};
+    broker_url::String = "nats://localhost:4222",
+    fileserver_url = "http://localhost:8080",
+    fileserver_upload_handler::Function = plik_oneshot_upload,
+    size_threshold::Int = 1_000_000,
+    correlation_id::String = string(uuid4()),
+    msg_purpose::String = "chat",
+    sender_name::String = "NATSBridge",
+    receiver_name::String = "",
+    receiver_id::String = "",
+    reply_to::String = "",
+    reply_to_msg_id::String = "",
+    is_publish::Bool = true,
+    NATS_connection::Union{NATS.Connection, Nothing} = nothing,
+    msg_id::String = string(uuid4()),
+    sender_id::String = string(uuid4())
+)
+# Returns: ::Tuple{msg_envelope_v1, String}
+```
+
+#### JavaScript
+
+```javascript
+const NATSBridge = require('natsbridge');
+
+const [env, env_json_str] = await NATSBridge.smartsend(
+    subject,
+    data,  // Array of [dataname, data, type] tuples
+    {
+        broker_url: 'nats://localhost:4222',
+        fileserver_url: 'http://localhost:8080',
+        fileserver_upload_handler: NATSBridge.plikOneshotUpload,
+        size_threshold: 1_000_000,
+        correlation_id: uuidv4(),
+        msg_purpose: 'chat',
+        sender_name: 'NATSBridge',
+        receiver_name: '',
+        receiver_id: '',
+        reply_to: '',
+        reply_to_msg_id: '',
+        is_publish: true,
+        nats_connection: null,
+        msg_id: uuidv4(),
+        sender_id: uuidv4()
+    }
+);
+// Returns: Promise<[env, env_json_str]>
+```
+
+#### Python
+
+```python
+from natsbridge import NATSBridge
+
+env, env_json_str = await NATSBridge.smartsend(
+    subject: str,
+    data: List[Tuple[str, Any, str]],
+    broker_url: str = "nats://localhost:4222",
+    fileserver_url: str = "http://localhost:8080",
+    fileserver_upload_handler: Callable = plik_oneshot_upload,
+    size_threshold: int = 1_000_000,
+    correlation_id: str = None,
+    msg_purpose: str = "chat",
+    sender_name: str = "NATSBridge",
+    receiver_name: str = "",
+    receiver_id: str = "",
+    reply_to: str = "",
+    reply_to_msg_id: str = "",
+    is_publish: bool = True,
+    nats_connection: Any = None,
+    msg_id: str = None,
+    sender_id: str = None
+)
+# Returns: Tuple[Dict, str]
+```
+
+#### MicroPython
+
+```python
+from natsbridge import NATSBridge
+
+# Limited to direct transport (< 100KB threshold)
+env, env_json_str = NATSBridge.smartsend(
+    subject,
+    data,  # List of (dataname, data, type) tuples
+    broker_url="nats://localhost:4222",
+    size_threshold=100000  # Lower threshold for memory constraints
+)
+# Returns: Tuple[Dict, str]
+```
+
+### smartreceive
+
+Receives and processes messages from NATS, handling both direct and link transport.
+
+#### Julia
+
+```julia
+using NATSBridge
+
+env = NATSBridge.smartreceive(
+    msg::NATS.Msg;
+    fileserver_download_handler::Function = _fetch_with_backoff,
+    max_retries::Int = 5,
+    base_delay::Int = 100,
+    max_delay::Int = 5000
+)
+# Returns: ::JSON.Object{String, Any}
+```
+
+#### JavaScript
+
+```javascript
+const env = await NATSBridge.smartreceive(
+    msg,
+    {
+        fileserver_download_handler: NATSBridge.fetchWithBackoff,
+        max_retries: 5,
+        base_delay: 100,
+        max_delay: 5000
+    }
+);
+// Returns: Promise<env_object>
+```
+
+#### Python
+
+```python
+env = await NATSBridge.smartreceive(
+    msg,
+    fileserver_download_handler=fetch_with_backoff,
+    max_retries=5,
+    base_delay=100,
+    max_delay=5000
+)
+# Returns: Dict with "payloads" key
+```
+
+#### MicroPython
+
+```python
+env = NATSBridge.smartreceive(
+    msg,
+    fileserver_download_handler=_sync_fileserver_download,
+    max_retries=3,
+    base_delay=100,
+    max_delay=1000
+)
+# Returns: Dict with "payloads" key
+```
+
+---
+
+## Payload Types
+
+| Type | Julia | JavaScript | Python | MicroPython | Description |
+|------|-------|------------|--------|-------------|-------------|
+| `text` | `String` | `string` | `str` | `str` | Plain text strings |
+| `dictionary` | `Dict`, `NamedTuple` | `Object`, `Array` | `dict`, `list` | `dict` | JSON-serializable dictionaries |
+| `arrowtable` | `DataFrame`, `Arrow.Table` | `Array<Object>` | `pandas.DataFrame` | ❌ | Tabular data (Arrow IPC) |
+| `jsontable` | `Vector{NamedTuple}` | `Array<Object>` | `list[dict]` | ❌ | Tabular data (JSON) |
+| `image` | `Vector{UInt8}` | `Uint8Array`, `Buffer` | `bytes` | `bytearray` | Image data (PNG, JPG) |
+| `audio` | `Vector{UInt8}` | `Uint8Array`, `Buffer` | `bytes` | `bytearray` | Audio data (WAV, MP3) |
+| `video` | `Vector{UInt8}` | `Uint8Array`, `Buffer` | `bytes` | `bytearray` | Video data (MP4, AVI) |
+| `binary` | `Vector{UInt8}`, `IOBuffer` | `Uint8Array`, `Buffer` | `bytes`, `bytearray` | `bytearray` | Generic binary data |
+
+---
+
+## Cross-Platform Examples
+
+### Example 1: Chat with Mixed Content
+
+Send text, image, and large file in one message.
+
+#### Julia
+
+```julia
+using NATSBridge
+
+data = [
+    ("message_text", "Hello!", "text"),
+    ("user_avatar", image_data, "image"),
+    ("large_document", large_file_data, "binary")
+]
+
+env, env_json_str = NATSBridge.smartsend("/chat/room1", data; fileserver_url="http://localhost:8080")
+```
+
+#### JavaScript
+
+```javascript
+const NATSBridge = require('natsbridge');
+
+const data = [
+    ["message_text", "Hello!", "text"],
+    ["user_avatar", imageData, "image"],
+    ["large_document", largeFileData, "binary"]
+];
+
+const [env, env_json_str] = await NATSBridge.smartsend(
+    "/chat/room1",
+    data,
+    { fileserver_url: 'http://localhost:8080' }
+);
+```
+
+#### Python
+
+```python
+from natsbridge import NATSBridge
+
+data = [
+    ("message_text", "Hello!", "text"),
+    ("user_avatar", image_data, "image"),
+    ("large_document", large_file_data, "binary")
+]
+
+env, env_json_str = await NATSBridge.smartsend(
+    "/chat/room1",
+    data,
+    fileserver_url="http://localhost:8080"
+)
+```
+
+### Example 2: Dictionary Exchange
+
+Send configuration data between platforms.
+
+#### Julia
+
+```julia
+using NATSBridge
+
+config = Dict(
+    "wifi_ssid" => "MyNetwork",
+    "wifi_password" => "password123",
+    "update_interval" => 60
+)
+
+data = [("config", config, "dictionary")]
+env, env_json_str = NATSBridge.smartsend("/device/config", data)
+```
+
+#### JavaScript
+
+```javascript
+const NATSBridge = require('natsbridge');
+
+const config = {
+    wifi_ssid: "MyNetwork",
+    wifi_password: "password123",
+    update_interval: 60
+};
+
+const [env, env_json_str] = await NATSBridge.smartsend(
+    "/device/config",
+    [["config", config, "dictionary"]]
+);
+```
+
+#### Python
+
+```python
+from natsbridge import NATSBridge
+
+config = {
+    "wifi_ssid": "MyNetwork",
+    "wifi_password": "password123",
+    "update_interval": 60
+}
+
+data = [("config", config, "dictionary")]
+env, env_json_str = await NATSBridge.smartsend("/device/config", data)
+```
+
+### Example 3: Table Data (Arrow IPC)
+
+Send tabular data using Apache Arrow IPC format.
+
+#### Julia
+
+```julia
+using NATSBridge
+using DataFrames
+
+df = DataFrame(
+    id = [1, 2, 3],
+    name = ["Alice", "Bob", "Charlie"],
+    score = [95, 88, 92]
+)
+
+data = [("students", df, "arrowtable")]
+env, env_json_str = NATSBridge.smartsend("/data/analysis", data)
+```
+
+#### JavaScript
+
+```javascript
+const NATSBridge = require('natsbridge');
+
+const df = [
+    { id: 1, name: "Alice", score: 95 },
+    { id: 2, name: "Bob", score: 88 },
+    { id: 3, name: "Charlie", score: 92 }
+];
+
+const [env, env_json_str] = await NATSBridge.smartsend(
+    "/data/analysis",
+    [["students", df, "arrowtable"]]
+);
+```
+
+#### Python
+
+```python
+from natsbridge import NATSBridge
+import pandas as pd
+
+df = pd.DataFrame({
+    "id": [1, 2, 3],
+    "name": ["Alice", "Bob", "Charlie"],
+    "score": [95, 88, 92]
+})
+
+data = [("students", df, "arrowtable")]
+env, env_json_str = await NATSBridge.smartsend("/data/analysis", data)
+```
+
+### Example 4: Request-Response Pattern
+
+Bi-directional communication with reply-to support.
+
+#### Julia
+
+```julia
+using NATSBridge
+
+# Requester
+env, env_json_str = NATSBridge.smartsend(
+    "/device/command",
+    [("command", Dict("action" => "read_sensor"), "dictionary")];
+    broker_url="nats://localhost:4222",
+    reply_to="/device/response"
+)
+```
+
+#### JavaScript
+
+```javascript
+const NATSBridge = require('natsbridge');
+
+// Requester
+const [env, env_json_str] = await NATSBridge.smartsend(
+    "/device/command",
+    [["command", { action: "read_sensor" }, "dictionary"]],
+    { broker_url: 'nats://localhost:4222', reply_to: '/device/response' }
+);
+```
+
+#### Python
+
+```python
+from natsbridge import NATSBridge
+
+# Requester
+env, env_json_str = await NATSBridge.smartsend(
+    "/device/command",
+    [("command", {"action": "read_sensor"}, "dictionary")],
+    broker_url="nats://localhost:4222",
+    reply_to="/device/response"
+)
+```
+
+---
+
+## Testing
+
+### Test File Organization
+
+| Platform | Sender Tests | Receiver Tests |
+|----------|--------------|----------------|
+| **Julia** | `test/test_julia_*_sender.jl` | `test/test_julia_*_receiver.jl` |
+| **JavaScript** | `test/test_js_*_sender.js` | `test/test_js_*_receiver.js` |
+| **Python** | `test/test_py_*_sender.py` | `test/test_py_*_receiver.py` |
+
+### Run Tests
+
+#### Julia
+
+```bash
+# Text message exchange
+julia test/test_julia_text_sender.jl
+julia test/test_julia_text_receiver.jl
+
+# Dictionary exchange
+julia test/test_julia_dict_sender.jl
+julia test/test_julia_dict_receiver.jl
+
+# File transfer
+julia test/test_julia_file_sender.jl
+julia test/test_julia_file_receiver.jl
+
+# Mixed payload types
+julia test/test_julia_mix_payloads_sender.jl
+julia test/test_julia_mix_payloads_receiver.jl
+
+# Table exchange
+julia test/test_julia_table_sender.jl
+julia test/test_julia_table_receiver.jl
+```
+
+#### JavaScript (Node.js)
+
+```bash
+# Text message exchange
+node test/test_js_text_sender.js
+node test/test_js_text_receiver.js
+
+# Dictionary exchange
+node test/test_js_dictionary_sender.js
+node test/test_js_dictionary_receiver.js
+
+# Binary transfer
+node test/test_js_binary_sender.js
+node test/test_js_binary_receiver.js
+
+# Table exchange
+node test/test_js_table_sender.js
+node test/test_js_table_receiver.js
+```
+
+#### Python
+
+```bash
+# Text message exchange
+python3 test/test_py_text_sender.py
+python3 test/test_py_text_receiver.py
+
+# Dictionary exchange
+python3 test/test_py_dictionary_sender.py
+python3 test/test_py_dictionary_receiver.py
+
+# Binary transfer
+python3 test/test_py_binary_sender.py
+python3 test/test_py_binary_receiver.py
+
+# Table exchange
+python3 test/test_py_table_sender.py
+python3 test/test_py_table_receiver.py
+```
+
+---
+
+## Documentation
+
+For detailed architecture and implementation information, see:
+
+- [Architecture Documentation](docs/architecture_updated.md) - Cross-platform architecture, API parity, platform-specific patterns
+- [Implementation Guide](docs/implementation_updated.md) - Detailed implementation for each platform, handler functions, testing
+- [Tutorial](docs/tutorial_updated.md) - Step-by-step getting started guide
+- [Walkthrough](docs/walkthrough_updated.md) - Real-world application building guides
+
+---
+
+## License
+
+MIT License
+
+Copyright (c) 2026 NATSBridge Contributors
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

docs/IMPLEMENTATION.md

@@ -1,321 +0,0 @@
-# Implementation Guide: Bi-Directional Data Bridge
-
-## Overview
-
-This document describes the implementation of the high-performance, bi-directional data bridge between Julia and JavaScript services using NATS (Core & JetStream), implementing the Claim-Check pattern for large payloads.
-
-## Architecture
-
-The implementation follows the Claim-Check pattern:
-
-```
-┌─────────────────────────────────────────────────────────────────────────┐
-│                         SmartSend Function                              │
-└─────────────────────────────────────────────────────────────────────────┘
-                             │
-                             ▼
-┌─────────────────────────────────────────────────────────────────────────┐
-│  Is payload size < 1MB?                                                 │
-└─────────────────────────────────────────────────────────────────────────┘
-                             │
-           ┌─────────────────┴─────────────────┐
-           ▼                                   ▼
-     ┌─────────────────┐                 ┌─────────────────┐
-     │  Direct Path    │                 │  Link Path      │
-     │  (< 1MB)        │                 │  (> 1MB)        │
-     │                 │                 │                 │
-     │ • Serialize to  │                 │ • Serialize to  │
-     │   IOBuffer      │                 │   IOBuffer      │
-     │ • Base64 encode │                 │ • Upload to     │
-     │ • Publish to    │                 │   HTTP Server   │
-     │   NATS          │                 │ • Publish to    │
-     │                 │                 │   NATS with URL │
-     └─────────────────┘                 └─────────────────┘
-```
-
-## Files
-
-### Julia Module: [`src/julia_bridge.jl`](../src/julia_bridge.jl)
-
-The Julia implementation provides:
-
-- **[`MessageEnvelope`](../src/julia_bridge.jl)**: Struct for the unified JSON envelope
-- **[`SmartSend()`](../src/julia_bridge.jl)**: Handles transport selection based on payload size
-- **[`SmartReceive()`](../src/julia_bridge.jl)**: Handles both direct and link transport
-
-### JavaScript Module: [`src/js_bridge.js`](../src/js_bridge.js)
-
-The JavaScript implementation provides:
-
-- **`MessageEnvelope` class**: For the unified JSON envelope
-- **[`SmartSend()`](../src/js_bridge.js)**: Handles transport selection based on payload size
-- **[`SmartReceive()`](../src/js_bridge.js)**: Handles both direct and link transport
-
-## Installation
-
-### Julia Dependencies
-
-```julia
-using Pkg
-Pkg.add("NATS")
-Pkg.add("Arrow")
-Pkg.add("JSON3")
-Pkg.add("HTTP")
-Pkg.add("UUIDs")
-Pkg.add("Dates")
-```
-
-### JavaScript Dependencies
-
-```bash
-npm install nats.js apache-arrow uuid base64-url
-```
-
-## Usage Tutorial
-
-### Step 1: Start NATS Server
-
-```bash
-docker run -p 4222:4222 nats:latest
-```
-
-### Step 2: Start HTTP File Server (optional)
-
-```bash
-# Create a directory for file uploads
-mkdir -p /tmp/fileserver
-
-# Use any HTTP server that supports POST for file uploads
-# Example: Python's built-in server
-python3 -m http.server 8080 --directory /tmp/fileserver
-```
-
-### Step 3: Run Test Scenarios
-
-```bash
-# Scenario 1: Command & Control (JavaScript sender)
-node test/scenario1_command_control.js
-
-# Scenario 2: Large Arrow Table (JavaScript sender)
-node test/scenario2_large_table.js
-
-# Scenario 3: Julia-to-Julia communication
-# Run both Julia and JavaScript versions
-julia test/scenario3_julia_to_julia.jl
-node test/scenario3_julia_to_julia.js
-```
-
-## Usage
-
-### Scenario 1: Command & Control (Small JSON)
-
-#### JavaScript (Sender)
-```javascript
-const { SmartSend } = require('./js_bridge');
-
-const config = {
-    step_size: 0.01,
-    iterations: 1000
-};
-
-await SmartSend("control", config, "json", {
-    correlationId: "unique-id"
-});
-```
-
-#### Julia (Receiver)
-```julia
-using NATS
-using JSON3
-
-# Subscribe to control subject
-subscribe(nats, "control") do msg
-    env = MessageEnvelope(String(msg.data))
-    config = JSON3.read(env.payload)
-    
-    # Execute simulation with parameters
-    step_size = config.step_size
-    iterations = config.iterations
-    
-    # Send acknowledgment
-    response = Dict("status" => "Running", "correlation_id" => env.correlation_id)
-    publish(nats, "control_response", JSON3.stringify(response))
-end
-```
-
-### Scenario 2: Deep Dive Analysis (Large Arrow Table)
-
-#### Julia (Sender)
-```julia
-using Arrow
-using DataFrames
-
-# Create large DataFrame
-df = DataFrame(
-    id = 1:10_000_000,
-    value = rand(10_000_000),
-    category = rand(["A", "B", "C"], 10_000_000)
-)
-
-# Send via SmartSend with type="table"
-await SmartSend("analysis_results", df, "table");
-```
-
-#### JavaScript (Receiver)
-```javascript
-const { SmartReceive } = require('./js_bridge');
-
-const result = await SmartReceive(msg);
-
-// Use table data for visualization with Perspective.js or D3
-const table = result.data;
-```
-
-### Scenario 3: Live Binary Processing
-
-#### JavaScript (Sender)
-```javascript
-const { SmartSend } = require('./js_bridge');
-
-// Capture binary chunk
-const binaryData = await navigator.mediaDevices.getUserMedia({ binary: true });
-
-await SmartSend("binary_input", binaryData, "binary", {
-    metadata: {
-        sample_rate: 44100,
-        channels: 1
-    }
-});
-```
-
-#### Julia (Receiver)
-```julia
-using WAV
-using DSP
-
-# Receive binary data
-function process_binary(data)
-    # Perform FFT or AI transcription
-    spectrum = fft(data)
-    
-    # Send results back (JSON + Arrow table)
-    results = Dict("transcription" => "sample text", "spectrum" => spectrum)
-    await SmartSend("binary_output", results, "json")
-end
-```
-
-### Scenario 4: Catch-Up (JetStream)
-
-#### Julia (Producer)
-```julia
-using NATS
-
-function publish_health_status(nats)
-    jetstream = JetStream(nats, "health_updates")
-    
-    while true
-        status = Dict("cpu" => rand(), "memory" => rand())
-        publish(jetstream, "health", status)
-        sleep(5)  # Every 5 seconds
-    end
-end
-```
-
-#### JavaScript (Consumer)
-```javascript
-const { connect } = require('nats');
-
-const nc = await connect({ servers: ['nats://localhost:4222'] });
-const js = nc.jetstream();
-
-// Request replay from last 10 minutes
-const consumer = await js.pullSubscribe("health", { 
-    durable_name: "catchup",
-    max_batch: 100,
-    max_ack_wait: 30000
-});
-
-// Process historical and real-time messages
-for await (const msg of consumer) {
-    const result = await SmartReceive(msg);
-    // Process the data
-    msg.ack();
-}
-```
-
-## Configuration
-
-### Environment Variables
-
-| Variable | Default | Description |
-|----------|---------|-------------|
-| `NATS_URL` | `nats://localhost:4222` | NATS server URL |
-| `FILESERVER_URL` | `http://localhost:8080/upload` | HTTP file server URL |
-| `SIZE_THRESHOLD` | `1_000_000` | Size threshold in bytes (1MB) |
-
-### Message Envelope Schema
-
-```json
-{
-  "correlation_id": "uuid-v4-string",
-  "type": "json|table|binary",
-  "transport": "direct|link",
-  "payload": "base64-encoded-string",  // Only if transport=direct
-  "url": "http://fileserver/path/to/data",  // Only if transport=link
-  "metadata": {
-    "content_type": "application/octet-stream",
-    "content_length": 123456,
-    "format": "arrow_ipc_stream"
-  }
-}
-```
-
-## Performance Considerations
-
-### Zero-Copy Reading
-- Use Arrow's memory-mapped file reading
-- Avoid unnecessary data copying during deserialization
-- Use Apache Arrow's native IPC reader
-
-### Exponential Backoff
-- Maximum retry count: 5
-- Base delay: 100ms, max delay: 5000ms
-- Implemented in both Julia and JavaScript implementations
-
-### Correlation ID Logging
-- Log correlation_id at every stage
-- Include: send, receive, serialize, deserialize
-- Use structured logging format
-
-## Testing
-
-Run the test scripts:
-
-```bash
-# Scenario 1: Command & Control (JavaScript sender)
-node test/scenario1_command_control.js
-
-# Scenario 2: Large Arrow Table (JavaScript sender)
-node test/scenario2_large_table.js
-```
-
-## Troubleshooting
-
-### Common Issues
-
-1. **NATS Connection Failed**
-   - Ensure NATS server is running
-   - Check NATS_URL configuration
-
-2. **HTTP Upload Failed**
-   - Ensure file server is running
-   - Check FILESERVER_URL configuration
-   - Verify upload permissions
-
-3. **Arrow IPC Deserialization Error**
-   - Ensure data is properly serialized to Arrow format
-   - Check Arrow version compatibility
-
-## License
-
-MIT

docs/SDD_FRAMEWORK.md

@@ -0,0 +1,402 @@
+# SDD + GitOps Documentation Framework
+
+This document defines the documentation framework for the NATSBridge project. It establishes a structured approach to creating, maintaining, and evolving technical documentation in alignment with GitOps principles—ensuring that documentation is versioned, auditable, and continuously validated alongside the codebase.
+
+---
+
+## The SDD Framework: Seven Pillars of Documentation
+
+| Document | Purpose (Rationale) | Primary Audience | Format / Content | Example (SaaS Context) | Measurement (KPI) |
+|----------|---------------------|-----------------|------------------|------------------------|-------------------|
+| **Requirements** | Capture the **business intent** — why we're building this and what success looks like. Defines boundaries and user-visible outcomes. | Stakeholders, Product Owners, Lead Developers | User stories, PRDs, acceptance criteria, non-functional constraints. | "System must process tabular data from Julia to SvelteKit UI with <200ms latency for 5-member teams." | 95% of requests complete <200ms (synthetic monitoring). |
+| **Specification** | The **technical contract** — precise rules for inputs, outputs, and data shape. Ensures consistency across dev and test. | Developers, QA Engineers, CI/CD pipelines | OpenAPI, Protobuf, AsyncAPI. Endpoint definitions, schemas, error codes. | `contract.yaml` defining a NATS subject that accepts Arrow streams with snake_case headers. | 100% of messages validated against spec (CI block rate). |
+| **Architecture** | The **blueprint** — how components fit together, interact, and scale. Guides system structure and trade-offs. | Architects, Senior Developers, DevOps | C4 diagrams, Mermaid.js, component/network/storage models. | Diagram showing 6-node cluster routing traffic via Caddy → Node.js API → Julia pods. | 100% of major decisions logged with trade-off analysis. |
+| **Walkthrough** | The **story of flow** — shows how pieces connect end-to-end and why steps are sequenced. Builds intuition for new devs. | New Developers, Team Members | TOUR.md, Loom videos, sequence diagrams. Step-by-step traces with rationale. | "UI sends JSON → Node.js wraps Claim-Check → Julia pulls Arrow data (prevents NATS overflow)." | New developers ship feature in <2 days (PR timeline). |
+| **Implementation** | The **real code** — business logic, helpers, tests, configs. Where design becomes executable. | Developers, Code Reviewers | Source code, README.md, unit tests, setup scripts. | Julia function for matrix calculation + SvelteKit component rendering table. | >80% unit test coverage, <5% drift from spec. |
+| **Validation** | The **enforcer** — ensures implementation matches the spec. Blocks drift and human error. | Automation servers, QA, Lead Developers | CI jobs, contract tests, linting, integration checks. | CI job rejects PR with camelCase field not allowed by YAML spec. | <1% of PRs bypass validation gates. |
+| **Runbook** | The **operational manual** — how the system lives in production, scales, and recovers. Guides on-call engineers. | DevOps, SREs, On-call Developers | K8s manifests, Helm charts, Markdown guides. Deployment, scaling, backup/restore, troubleshooting. | GitOps manifest ensuring 6 Julia replicas restart if memory >80%. | MTTR <15 minutes for P1 incidents. |
+
+---
+
+## Detailed Document Descriptions
+
+### 1. Requirements
+
+**Purpose**: Capture the *business intent* — why we're building this and what success looks like. Defines boundaries and user-visible outcomes.
+
+**Why It Matters**:
+- Aligns engineering efforts with business goals
+- Provides a north star for feature development
+- Establishes acceptance criteria before implementation begins
+- Creates a contract between product and engineering
+
+**Content Guidelines**:
+- User stories with clear acceptance criteria (As a X, I want Y so that Z)
+- Product Requirements Documents (PRDs) with success metrics
+- Non-functional requirements (performance, security, scalability)
+- Boundary definitions (what's in scope vs. out of scope)
+
+**Best Practices**:
+- Link each requirement to a measurable KPI
+- Keep requirements testable and verifiable
+- Maintain backward compatibility with existing requirements
+- Review and update requirements as business context changes
+
+---
+
+### 2. Specification
+
+**Purpose**: The *technical contract* — precise rules for inputs, outputs, and data shape. Ensures consistency across dev and test.
+
+**Why It Matters**:
+- Prevents implementation drift between components
+- Enables contract testing in CI/CD pipelines
+- Provides a single source of truth for data structures
+- Facilitates integration between teams
+
+**Content Guidelines**:
+- API endpoint definitions (methods, paths, parameters)
+- Request/response schemas (JSON, XML, Protobuf, AsyncAPI)
+- Error codes and their meanings
+- Data validation rules and constraints
+- Rate limiting and quota definitions
+
+**Best Practices**:
+- Use formal specification languages (OpenAPI 3.0+, AsyncAPI)
+- Version specifications alongside code
+- Generate client SDKs from specifications
+- Block CI on specification violations
+- Document edge cases and error scenarios
+
+---
+
+### 3. Architecture
+
+**Purpose**: The *blueprint* — how components fit together, interact, and scale. Guides system structure and trade-offs.
+
+**Why It Matters**:
+- Provides a mental model for system design
+- Guides technical decision-making and trade-off analysis
+- Facilitates onboarding of new architects and senior developers
+- Documents scaling and performance considerations
+
+**Content Guidelines**:
+- C4 diagrams (Context, Container, Component levels)
+- Mermaid.js flowcharts for sequence diagrams
+- Component interaction diagrams
+- Network topology and data flow
+- Storage and caching strategies
+- Scaling and resilience patterns
+
+**Best Practices**:
+- Use diagrams that are easy to update (Mermaid.js over static images)
+- Document trade-off decisions with Rationale Documents
+- Include scaling considerations for each component
+- Document failure modes and recovery strategies
+- Keep architecture diagrams versioned with code
+
+---
+
+### 4. Walkthrough
+
+**Purpose**: The *story of flow* — shows how pieces connect end-to-end and why steps are sequenced. Builds intuition for new devs.
+
+**Why It Matters**:
+- Reduces onboarding time for new developers
+- Provides context that code comments alone cannot convey
+- Explains the "why" behind architectural decisions
+- Helps identify gaps in the system design
+
+**Content Guidelines**:
+- Step-by-step flow descriptions with rationale
+- Sequence diagrams showing request/response patterns
+- "Tour of the codebase" guides
+- Video walkthroughs (Loom, internal recordings)
+- Debugging and tracing examples
+
+**Best Practices**:
+- Walk through real user journeys, not just technical flows
+- Include "what could go wrong" scenarios
+- Link walkthroughs to relevant code locations
+- Keep walkthroughs updated with architecture changes
+- Make walkthroughs interactive where possible
+
+---
+
+### 5. Implementation
+
+**Purpose**: The *real code* — business logic, helpers, tests, configs. Where design becomes executable.
+
+**Why It Matters**:
+- This is the actual artifact that runs in production
+- Code is the ultimate source of truth (when it matches spec)
+- Tests validate correctness and prevent regressions
+- Configuration files define runtime behavior
+
+**Content Guidelines**:
+- Business logic implementation
+- Helper functions and utilities
+- Unit and integration tests
+- Configuration files (YAML, JSON, environment)
+- Setup and development scripts
+- Code organization and module structure
+
+**Best Practices**:
+- Follow consistent code style and conventions
+- Write tests before or alongside implementation (TDD/BDD)
+- Document complex logic with inline comments
+- Keep configuration externalized and versioned
+- Use type annotations where applicable
+
+---
+
+### 6. Validation
+
+**Purpose**: The *enforcer* — ensures implementation matches the spec. Blocks drift and human error.
+
+**Why It Matters**:
+- Prevents breaking changes from reaching production
+- Catches specification violations early in the CI pipeline
+- Maintains data integrity and API consistency
+- Reduces manual QA effort through automation
+
+**Content Guidelines**:
+- CI/CD pipeline configurations
+- Contract testing scripts
+- Linting rules and configurations
+- Integration test suites
+- Schema validation jobs
+- Security scanning and audit jobs
+
+**Best Practices**:
+- Fail CI on specification violations
+- Run validation jobs on every commit and PR
+- Use automated code review tools
+- Maintain validation job health dashboard
+- Document validation failure remediation steps
+
+---
+
+### 7. Runbook
+
+**Purpose**: The *operational manual* — how the system lives in production, scales, and recovers. Guides on-call engineers.
+
+**Why It Matters**:
+- Reduces Mean Time To Recovery (MTTR) for incidents
+- Provides step-by-step guidance for common issues
+- Documents scaling and deployment procedures
+- Ensures operational knowledge is not siloed
+
+**Content Guidelines**:
+- Deployment procedures (manual and automated)
+- Scaling instructions (horizontal/vertical)
+- Backup and restore procedures
+- Troubleshooting guides for common issues
+- Runbook entries for specific error codes
+- Contact information and escalation paths
+
+**Best Practices**:
+- Write runbooks for every P1/P2 incident
+- Include exact commands and configuration snippets
+- Test runbooks periodically (chaos engineering)
+- Link runbook entries to relevant documentation
+- Keep runbooks updated when system changes
+
+---
+
+## How to Use This Approach Effectively
+
+### 1. Start with Requirements
+
+Before writing any code or documentation, establish clear requirements. Ask:
+- What business problem are we solving?
+- How will we measure success?
+- What are the non-negotiable constraints?
+
+**Action**: Create a `docs/requirements/` directory and start with `PRD.md` and `KPIs.md`.
+
+### 2. Define the Specification First
+
+Once requirements are stable, define the technical specification. This becomes the contract for implementation.
+
+**Action**: Create `docs/specification/` with `contract.yaml` (or appropriate format) and `error-codes.md`.
+
+### 3. Design the Architecture
+
+With requirements and specification in place, design the architecture. Document trade-off decisions explicitly.
+
+**Action**: Create `docs/architecture/` with Mermaid diagrams and `trade-offs.md`.
+
+### 4. Create Walkthroughs Early
+
+As soon as the architecture is defined, create walkthroughs. This helps identify gaps and provides onboarding material.
+
+**Action**: Create `docs/walkthrough/` with `TOUR.md` and sequence diagrams.
+
+### 5. Implement with Validation in Mind
+
+Write implementation code that adheres to the specification. Build validation into the CI pipeline from day one.
+
+**Action**: Ensure test files are co-located with implementation and run on every commit.
+
+### 6. Automate Validation
+
+Build automated validation that runs in CI/CD. This ensures spec compliance and prevents drift.
+
+**Action**: Configure CI jobs to validate against specification and block PRs on violations.
+
+### 7. Document Operations from Day One
+
+Create runbook entries as soon as deployment procedures are established. Update them when incidents occur.
+
+**Action**: Create `docs/runbook/` with entries for deployment, scaling, and common issues.
+
+---
+
+## GitOps Integration
+
+This documentation framework aligns with GitOps principles:
+
+| GitOps Principle | Documentation Alignment |
+|-----------------|------------------------|
+| **Versioned** | All documentation lives in git, with history and audit trail |
+| ** declarative** | Specifications and architecture are declarative contracts |
+| **Automated** | Validation jobs automate spec compliance checks |
+| **Self-Service** | Walkthroughs and runbooks enable self-service onboarding and operations |
+| **Observability** | KPIs and metrics are defined for each documentation artifact |
+
+**Git Structure**:
+```
+docs/
+├── requirements/       # PRDs, user stories, KPIs
+├── specification/      # OpenAPI, Protobuf, AsyncAPI specs
+├── architecture/       # C4 diagrams, Mermaid, trade-off docs
+├── walkthrough/        # TOUR.md, sequence diagrams
+├── implementation/     # Source code (in src/)
+├── validation/         # CI configs, test suites
+└── runbook/            # Deployment, scaling, troubleshooting
+```
+
+---
+
+## Metrics and Continuous Improvement
+
+Each documentation artifact has associated KPIs. Track these to ensure quality:
+
+| Document | KPI | Target |
+|----------|-----|--------|
+| Requirements | Requirement coverage | 100% of features have associated requirements |
+| Specification | Spec compliance rate | 100% of messages validate against spec |
+| Architecture | Decision documentation | 100% of major decisions logged with trade-offs |
+| Walkthrough | New dev time-to-first-PR | <2 days from onboarding to first contribution |
+| Implementation | Test coverage | >80% unit test coverage |
+| Validation | Bypass rate | <1% of PRs bypass validation gates |
+| Runbook | MTTR | <15 minutes for P1 incidents |
+
+**Review Cadence**:
+- Weekly: Review KPI dashboards and documentation gaps
+- Monthly: Update documentation based on incident learnings
+- Quarterly: Full framework review and improvement
+
+---
+
+## Template Examples
+
+### Requirements Template
+```markdown
+# PRD: Feature Name
+
+## Business Goal
+[What problem are we solving?]
+
+## Success Metrics
+- [Metric 1]: Target [value]
+- [Metric 2]: Target [value]
+
+## User Stories
+- As a [role], I want [feature] so that [benefit]
+  - Acceptance Criteria: [details]
+
+## Non-Functional Requirements
+- Performance: [details]
+- Security: [details]
+- Scalability: [details]
+
+## Out of Scope
+- [What's explicitly excluded]
+```
+
+### Specification Template
+```yaml
+# contract.yaml
+openapi: 3.0.0
+info:
+  title: NATSBridge API
+  version: 1.0.0
+paths:
+  /api/v1/endpoint:
+    post:
+      requestBody:
+        content:
+          application/json:
+            schema:
+              $ref: '#/components/schemas/Request'
+      responses:
+        '200':
+          description: Success
+          content:
+            application/json:
+              schema:
+                $ref: '#/components/schemas/Response'
+```
+
+### Architecture Template
+```mermaid
+%%{init: {'theme': 'base', 'themeVariables': {'primaryColor': '#3b82f6'}}}%%
+flowchart TD
+    A[Client] --> B[Caddy]
+    B --> C[Node.js API]
+    C --> D[Julia Worker]
+    D --> E[NATS Cluster]
+    E --> F[Storage]
+    
+    style A fill:#f9f9f9,stroke:#333
+    style E fill:#e0e7ff,stroke:#3b82f6
+```
+
+### Runbook Template
+```markdown
+# Runbook: Service Restart
+
+**Severity**: P2
+**Estimated Time**: 5 minutes
+
+## Symptoms
+- Service is unresponsive
+- Health checks are failing
+
+## Steps
+1. SSH to the host
+2. Run: `kubectl rollout restart deployment/natsbridge`
+3. Monitor: `kubectl get pods -l app=natsbridge -w`
+
+## Rollback
+- Run: `kubectl rollout undo deployment/natsbridge`
+
+## Post-Incident
+- [ ] Review logs for root cause
+- [ ] Update runbook if needed
+```
+
+---
+
+## Conclusion
+
+This SDD + GitOps Documentation Framework ensures that documentation is:
+- **Structured**: Seven distinct artifacts with clear purposes
+- **Automated**: Validation and CI/CD integration
+- **Versioned**: All documentation in git with history
+- **Measurable**: KPIs for quality and effectiveness
+- **Actionable**: Practical templates and examples
+
+Use this framework as a living document—update it as your team's needs evolve.

docs/architecture.md

@@ -1,294 +1,717 @@
-# Architecture Documentation: Bi-Directional Data Bridge (Julia ↔ JavaScript)
+# Architecture Documentation: NATSBridge
 
-## Overview
+**Version**: 1.0.0  
+**Date**: 2026-03-13  
+**Status**: Active  
+**Ground Truth**: [`src/NATSBridge.jl`](../src/NATSBridge.jl)  
+**Architecture Level**: C4 Container Level
 
-This document describes the architecture for a high-performance, bi-directional data bridge between a Julia service and a JavaScript (Node.js) service using NATS (Core & JetStream), implementing the Claim-Check pattern for large payloads.
+---
 
-## Architecture Diagram
+## Executive Summary
+
+This document defines the **blueprint** for NATSBridge - the cross-platform bi-directional data bridge that enables seamless communication between **Julia**, **JavaScript**, **Python**, and **MicroPython** applications using NATS as the message bus.
+
+This architecture document serves as the single source of truth for:
+- **System Structure**: How components fit together and interact
+- **Scaling Considerations**: How the system scales horizontally and vertically
+- **Failure Modes**: How the system handles failures and recovers
+- **Trade-off Decisions**: The rationale behind architectural decisions
+
+---
+
+## Architecture Overview
+
+### C4 Context Diagram
 
 ```mermaid
 flowchart TD
-    subgraph Client
+    subgraph "External Systems"
-        JS[JavaScript Client]
+        NATS_Server[NATS Server]
-        JSApp[Application Logic]
+        File_Server[HTTP File Server<br/>Plik/AWS S3/Custom]
     end
 
-    subgraph Server
+    subgraph "Client Applications"
-        Julia[Julia Service]
+        Julia_App[Julia Application]
-        NATS[NATS Server]
+        JS_App[JavaScript Application<br/>Node.js/Browser]
-        FileServer[HTTP File Server]
+        Python_App[Python Application<br/>Desktop]
+        MicroPython_App[MicroPython Device]
     end
 
-    JS -->|Control/Small Data| JSApp
+    Julia_App -->|NATS| NATS_Server
-    JSApp -->|NATS| NATS
+    JS_App -->|NATS| NATS_Server
-    NATS -->|NATS| Julia
+    Python_App -->|NATS| NATS_Server
-    Julia -->|NATS| NATS
+    MicroPython_App -->|NATS| NATS_Server
-    Julia -->|HTTP POST| FileServer
-    JS -->|HTTP GET| FileServer
 
-    style JS fill:#e1f5fe
+    Julia_App -->|HTTP| File_Server
-    style Julia fill:#e8f5e9
+    JS_App -->|HTTP| File_Server
-    style NATS fill:#fff3e0
+    Python_App -->|HTTP| File_Server
-    style FileServer fill:#f3e5f5
+    MicroPython_App -->|HTTP| File_Server
+
+    style NATS_Server fill:#fff3e0,stroke:#f57c00
+    style File_Server fill:#f3e5f5,stroke:#9c27b4
+    style Julia_App fill:#e8f5e9,stroke:#4caf50
+    style JS_App fill:#e3f2fd,stroke:#2196f3
+    style Python_App fill:#e3f2fd,stroke:#2196f3
+    style MicroPython_App fill:#fce4ec,stroke:#e91e63
 ```
 
-## System Components
+### C4 Container Diagram
 
-### 1. Unified JSON Envelope Schema
+```mermaid
+flowchart TD
+    subgraph "Client Container"
+        Julia_Module[Julia NATSBridge Module]
+        JS_Module[JavaScript NATSBridge Module]
+        Python_Module[Python NATSBridge Module]
+        MicroPython_Module[MicroPython NATSBridge Module]
+    end
 
-All messages use a standardized envelope format:
+    subgraph "NATS Container"
+        NATS_Client[NATS Client]
+        NATS_Broker[NATS Broker]
+    end
+
+    subgraph "File Server Container"
+        File_Client[HTTP Client]
+        File_Server[File Server]
+    end
+
+    Julia_Module --> NATS_Client
+    JS_Module --> NATS_Client
+    Python_Module --> NATS_Client
+    MicroPython_Module --> NATS_Client
+
+    NATS_Client --> NATS_Broker
+
+    Julia_Module --> File_Client
+    JS_Module --> File_Client
+    Python_Module --> File_Client
+    MicroPython_Module --> File_Client
+
+    File_Client --> File_Server
+
+    style Julia_Module fill:#e8f5e9,stroke:#4caf50
+    style JS_Module fill:#e3f2fd,stroke:#2196f3
+    style Python_Module fill:#e3f2fd,stroke:#2196f3
+    style MicroPython_Module fill:#fce4ec,stroke:#e91e63
+    style NATS_Broker fill:#fff3e0,stroke:#f57c00
+    style File_Server fill:#f3e5f5,stroke:#9c27b4
+```
+
+### C4 Component Diagram (Julia Implementation)
+
+```mermaid
+flowchart TD
+    subgraph "NATSBridge Module"
+        SmartSend[smartsend Function]
+        SmartReceive[smartreceive Function]
+        
+        Serialize[_serialize_data]
+        Deserialize[_deserialize_data]
+        
+        BuildEnvelope[build_envelope]
+        BuildPayload[build_payload]
+        
+        PublishMessage[publish_message]
+        
+        FileServerUpload[fileserver_upload_handler]
+        FileServerDownload[fileserver_download_handler]
+    end
+
+    subgraph "Data Models"
+        Payload[MsgPayloadV1 Struct]
+        Envelope[MsgEnvelopeV1 Struct]
+    end
+
+    SmartSend --> Serialize
+    SmartSend --> BuildEnvelope
+    SmartSend --> BuildPayload
+    SmartSend --> PublishMessage
+    SmartSend --> FileServerUpload
+
+    SmartReceive --> Deserialize
+    SmartReceive --> FileServerDownload
+
+    Serialize --> Payload
+    BuildEnvelope --> Envelope
+    BuildPayload --> Payload
+
+    style SmartSend fill:#d1fae5,stroke:#10b981
+    style SmartReceive fill:#d1fae5,stroke:#10b981
+    style PublishMessage fill:#fef3c7,stroke:#f59e0b
+    style FileServerUpload fill:#fef3c7,stroke:#f59e0b
+    style FileServerDownload fill:#fef3c7,stroke:#f59e0b
+```
+
+---
+
+## High-Level Architecture
+
+### System Components
+
+| Component | Purpose | Platform Support |
+|-----------|---------|------------------|
+| **smartsend** | Send data via NATS with automatic transport selection | All |
+| **smartreceive** | Receive and process NATS messages | All |
+| **_serialize_data** | Serialize data according to payload type | All |
+| **_deserialize_data** | Deserialize bytes to native data types | All |
+| **_build_envelope** | Build message envelope from payloads | All |
+| **_build_payload** | Build payload object from serialized data | All |
+| **publish_message** | Publish message to NATS subject | All |
+| **fileserver_upload_handler** | Upload large payloads to HTTP server | Desktop |
+| **fileserver_download_handler** | Download payloads from HTTP server | Desktop |
+
+### Data Flow
+
+```mermaid
+flowchart TD
+    A[User calls smartsend subject data] --> B[Process each payload]
+    B --> C{Calculate serialized size}
+    C -->|Size < Threshold| D[Direct Transport]
+    C -->|Size >= Threshold| E[Link Transport]
+    
+    D --> F[Serialize data]
+    F --> G[Base64 encode]
+    G --> H[Build payload object]
+    
+    E --> I[Serialize data]
+    I --> J[Upload to file server]
+    J --> K[Get download URL]
+    K --> H
+    
+    H --> L[Build envelope]
+    L --> M[Convert to JSON]
+    M --> N[Publish to NATS]
+    
+    style A fill:#f9f9f9,stroke:#333
+    style N fill:#e0e7ff,stroke:#3b82f6
+    style D fill:#d1fae5,stroke:#10b981
+    style E fill:#fef3c7,stroke:#f59e0b
+```
+
+---
+
+## Message Envelope Architecture
+
+### msg_envelope_v1 Structure (Julia)
+
+```julia
+struct msg_envelope_v1
+  correlation_id::String       # UUID v4 for distributed tracing
+  msg_id::String               # UUID v4 for this message
+  timestamp::String            # ISO 8601 UTC timestamp
+  
+  send_to::String              # NATS subject to publish to
+  msg_purpose::String          # ACK, NACK, updateStatus, shutdown, chat
+  sender_name::String          # Sender application name
+  sender_id::String            # UUID v4 of sender
+  receiver_name::String        # Receiver application name (empty = broadcast)
+  receiver_id::String          # UUID v4 of receiver (empty = broadcast)
+  
+  reply_to::String             # Topic for reply messages
+  reply_to_msg_id::String      # Message ID being replied to
+  broker_url::String           # NATS broker URL
+  
+  metadata::Dict{String, Any}  # Message-level metadata
+  payloads::Vector{msg_payload_v1}  # List of payloads
+end
+```
+
+### msg_payload_v1 Structure (Julia)
+
+```julia
+struct msg_payload_v1
+  id::String                    # UUID v4 for this payload
+  dataname::String              # Name of the payload
+  payload_type::String          # text, dictionary, arrowtable, etc.
+  transport::String             # direct or link
+  encoding::String              # none, json, base64, arrow-ipc
+  size::Integer                 # Size in bytes
+  data::Any                     # Base64 string or URL
+  metadata::Dict{String, Any}   # Payload-level metadata
+end
+```
+
+### JSON Schema (Cross-Platform)
 
 ```json
 {
-  "correlation_id": "uuid-v4-string",
+  "correlation_id": "string (UUID v4)",
-  "type": "json|table|binary",
+  "msg_id": "string (UUID v4)",
-  "transport": "direct|link",
+  "timestamp": "string (ISO 8601 UTC)",
-  "payload": "base64-encoded-string",  // Only if transport=direct
+  "send_to": "string",
-  "url": "http://fileserver/path/to/data",  // Only if transport=link
+  "msg_purpose": "string",
-  "metadata": {
+  "sender_name": "string",
-    "content_type": "application/octet-stream",
+  "sender_id": "string (UUID v4)",
-    "content_length": 123456,
+  "receiver_name": "string",
-    "format": "arrow_ipc_stream"
+  "receiver_id": "string (UUID v4)",
+  "reply_to": "string",
+  "reply_to_msg_id": "string",
+  "broker_url": "string",
+  "metadata": "object",
+  "payloads": [
+    {
+      "id": "string (UUID v4)",
+      "dataname": "string",
+      "payload_type": "string",
+      "transport": "string",
+      "encoding": "string",
+      "size": "integer",
+      "data": "string or URL",
+      "metadata": "object"
+    }
+  ]
+}
+```
+
+---
+
+## Payload Type Architecture
+
+### Supported Payload Types
+
+| Type | Description | Serialization | Encoding | Platforms |
+|------|-------------|---------------|----------|-----------|
+| `text` | Plain text string | UTF-8 bytes | Base64 | All |
+| `dictionary` | JSON object | JSON string | Base64/JSON | All |
+| `arrowtable` | Apache Arrow IPC | Arrow IPC stream | Base64/arrow-ipc | Desktop |
+| `jsontable` | JSON array of objects | JSON string | Base64/json | All |
+| `image` | Binary image data | Raw bytes | Base64 | All |
+| `audio` | Binary audio data | Raw bytes | Base64 | All |
+| `video` | Binary video data | Raw bytes | Base64 | All |
+| `binary` | Generic binary data | Raw bytes | Base64 | All |
+
+### Serialization Logic
+
+```mermaid
+flowchart TD
+    A[Input data + payload_type] --> B{Payload Type}
+    
+    B -->|"text"| C[UTF-8 encode]
+    B -->|"dictionary"| D[JSON serialize]
+    B -->|"arrowtable"| E[Arrow IPC serialize]
+    B -->|"jsontable"| F[JSON serialize]
+    B -->|"image"| G[Raw bytes]
+    B -->|"audio"| H[Raw bytes]
+    B -->|"video"| I[Raw bytes]
+    B -->|"binary"| J[Raw bytes]
+    
+    C --> K[Return bytes]
+    D --> K
+    E --> K
+    F --> K
+    G --> K
+    H --> K
+    I --> K
+    J --> K
+    
+    style A fill:#f9f9f9,stroke:#333
+    style K fill:#e0e7ff,stroke:#3b82f6
+```
+
+---
+
+## Transport Strategy Architecture
+
+### Size Threshold Decision Logic
+
+| Platform | Size Threshold | Notes |
+|----------|----------------|-------|
+| Desktop (Julia/JS/Python) | 500,000 bytes (0.5MB) | Default threshold |
+| MicroPython | 100,000 bytes (100KB) | Lower threshold for memory constraints |
+
+### Transport Selection Flow
+
+```mermaid
+flowchart TD
+    A[smartsend called] --> B[Serialize payload]
+    B --> C[Calculate size]
+    C --> D{Size < Threshold?}
+    
+    D -->|Yes| E[Direct Transport]
+    D -->|No| F[Link Transport]
+    
+    E --> G[Base64 encode]
+    G --> H[Build payload with direct transport]
+    
+    F --> I[Upload to file server]
+    I --> J[Get download URL]
+    J --> K[Build payload with link transport]
+    
+    H --> L[Build envelope]
+    K --> L
+    
+    style A fill:#f9f9f9,stroke:#333
+    style L fill:#e0e7ff,stroke:#3b82f6
+    style E fill:#d1fae5,stroke:#10b981
+    style F fill:#fef3c7,stroke:#f59e0b
+```
+
+### Direct Transport Protocol
+
+When `transport = "direct"`, the `data` field contains a Base64-encoded string of the serialized payload.
+
+**Encoding Rules**:
+- `text`: UTF-8 → Base64
+- `dictionary`: JSON → Base64 (or direct JSON)
+- `arrowtable`: Arrow IPC → Base64 (or arrow-ipc)
+- `jsontable`: JSON → Base64 (or direct JSON)
+- `image`/`audio`/`video`/`binary`: Raw bytes → Base64
+
+### Link Transport Protocol
+
+When `transport = "link"`, the `data` field contains a URL pointing to the uploaded payload.
+
+**Upload Flow**:
+1. Serialize payload according to `payload_type`
+2. Upload to HTTP file server (e.g., Plik)
+3. Include returned URL in `data` field
+
+**Download Flow**:
+1. Extract URL from payload
+2. Fetch with exponential backoff (max 5 retries)
+3. Deserialize based on `payload_type`
+
+---
+
+## Platform-Specific Architecture
+
+### Julia Architecture
+
+Julia leverages multiple dispatch for type-specific implementations:
+
+- **Multiple Dispatch**: Function overloading based on argument types
+- **Struct-based Data Models**: Explicit type definitions with `struct`
+- **Native Arrow IPC**: Support via `Arrow.jl`
+- **Async/Await**: Tasks for non-blocking I/O
+
+```julia
+# Multiple dispatch for serialization
+function _serialize_data(data::String, payload_type::String)
+    # Text serialization
+end
+
+function _serialize_data(data::Dict, payload_type::String)
+    # Dictionary serialization
+end
+
+function _serialize_data(data::DataFrame, payload_type::String)
+    # Arrow table serialization
+end
+```
+
+### JavaScript Architecture
+
+JavaScript uses async/await for non-blocking I/O:
+
+- **Class-based NATS Client**: Connection management
+- **Module-level Utilities**: Serialization functions
+- **Native ArrayBuffer**: Binary data handling
+- **Fetch API**: HTTP file server communication
+
+```javascript
+// Class-based NATS client
+class NATSClient {
+    constructor(url) {
+        this.url = url;
+        this.connection = null;
+    }
+    
+    async connect() {
+        this.connection = await nats.connect({ servers: this.url });
     }
 }
 ```
 
-### 2. Transport Strategy Decision Logic
+### Python Architecture
 
-```
+Python uses classes for stateful operations:
-┌─────────────────────────────────────────────────────────────┐
+
-│                     SmartSend Function                        │
+- **Class-based NATSBridge**: Encapsulated API
-└─────────────────────────────────────────────────────────────┘
+- **Dataclasses**: Structured data (MsgPayloadV1, MsgEnvelopeV1)
-                            │
+- **Async/await**: I/O operations
-                            ▼
+- **pyarrow**: Arrow IPC support
-┌─────────────────────────────────────────────────────────────┐
+
-│  Is payload size < 1MB?                                     │
+```python
-└─────────────────────────────────────────────────────────────┘
+class NATSBridge:
-                            │
+    DEFAULT_SIZE_THRESHOLD = 500_000
-          ┌─────────────────┴─────────────────┐
+    
-          ▼                                   ▼
+    def __init__(self, broker_url=None, fileserver_url=None):
-    ┌─────────────────┐                 ┌─────────────────┐
+        self.broker_url = broker_url or self.DEFAULT_BROKER_URL
-    │  Direct Path    │                 │  Link Path      │
+        self.fileserver_url = fileserver_url or self.DEFAULT_FILESERVER_URL
-    │  (< 1MB)        │                 │  (> 1MB)        │
-    │                 │                 │                 │
-    │ • Serialize to  │                 │ • Serialize to  │
-    │   IOBuffer      │                 │   IOBuffer      │
-    │ • Base64 encode │                 │ • Upload to     │
-    │ • Publish to    │                 │   HTTP Server   │
-    │   NATS          │                 │ • Publish to    │
-    │                 │                 │   NATS with URL │
-    └─────────────────┘                 └─────────────────┘
 ```
 
-### 3. Julia Module Architecture
+### MicroPython Architecture
+
+MicroPython has significant constraints:
+
+- **Synchronous API**: No async/await
+- **Memory-constrained**: 256KB - 1MB
+- **Limited payload support**: No tables, max 50KB
+- **Simplified UUID generation**: Custom implementation
+
+```python
+# MicroPython constraints
+DEFAULT_SIZE_THRESHOLD = 100_000  # 100KB
+MAX_PAYLOAD_SIZE = 50_000  # 50KB hard limit
+```
+
+---
+
+## Scaling Architecture
+
+### Horizontal Scaling
+
+| Component | Scaling Strategy |
+|-----------|------------------|
+| **NATS Server** | Cluster deployment with multiple nodes |
+| **File Server** | Load balancer + multiple instances |
+| **Client Applications** | Deploy multiple instances behind load balancer |
+
+### Vertical Scaling
+
+| Component | Scaling Strategy |
+|-----------|------------------|
+| **NATS Server** | Increase memory, CPU, disk I/O |
+| **File Server** | Increase memory, CPU, disk capacity |
+| **Client Applications** | Increase heap size (Python/JS) |
+
+### Performance Considerations
+
+| Metric | Target | Notes |
+|--------|--------|-------|
+| Message serialization overhead | <50ms | For 10KB payload |
+| Message deserialization overhead | <50ms | For 10KB payload |
+| NATS connection establishment | <100ms | Connection pool recommended |
+| File upload latency | <1s | For 0.5MB file |
+| File download latency | <1s | For 0.5MB file |
+
+---
+
+## Failure Modes and Recovery
+
+### NATS Connection Failure
+
+**Scenario**: NATS server unavailable
+
+**Handler**:
+- Connection auto-reconnect via TCP-level reconnection
+- Retry with exponential backoff for publish operations
+
+**Recovery**:
+- NATS client automatically attempts reconnection
+- Application can check connection status before publishing
+
+### File Server Unavailable
+
+**Scenario**: HTTP file server unavailable during upload/download
+
+**Handler**:
+- Retry up to 5 times with exponential backoff (100ms → 5000ms)
+- Fallback to direct transport for upload (MicroPython)
+
+**Recovery**:
+- Exponential backoff: `delay = min(delay * 2, max_delay)`
+- After max retries, throw error with correlation ID
+
+### Deserialization Error
+
+**Scenario**: Payload type mismatch or corrupted data
+
+**Handler**:
+- Log correlation ID and throw error
+- No retry (data corruption)
+
+**Recovery**:
+- Application must validate payload_type matches data type
+- Use proper serialization before sending
+
+### Memory Overflow (MicroPython)
+
+**Scenario**: Payload exceeds maximum size (50KB)
+
+**Handler**:
+- Reject payloads >50KB with MemoryError
+- No retry (client-side check)
+
+**Recovery**:
+- Application must split large payloads
+- Use direct transport only for small payloads
+
+---
+
+## Trade-off Decisions
+
+### Decision 1: Direct vs Link Transport Threshold
+
+**Trade-off**: Memory vs Network I/O
+
+**Decision**: Use 0.5MB threshold for desktop, 100KB for MicroPython
+
+**Rationale**:
+- Direct transport uses more memory (Base64 encoding adds ~33% overhead)
+- Link transport requires network I/O for upload/download
+- 0.5MB is reasonable for desktop memory constraints
+- 100KB is necessary for MicroPython memory constraints
+
+### Decision 2: Base64 Encoding for Direct Transport
+
+**Trade-off**: Bandwidth vs Simplicity
+
+**Decision**: Use Base64 encoding for all direct transport payloads
+
+**Rationale**:
+- Simplifies JSON serialization (all data is string-compatible)
+- Increases payload size by ~33%, but NATS can handle this
+- Alternative would be binary payload support (more complex)
+
+### Decision 3: Multiple Platform Implementations
+
+**Trade-off**: Development effort vs Cross-platform support
+
+**Decision**: Maintain separate implementations for each platform
+
+**Rationale**:
+- Each platform has idiomatic patterns (multiple dispatch, async/await, etc.)
+- Maintains developer productivity and code quality
+- API parity ensures cross-platform compatibility
+
+### Decision 4: Handler Function Abstraction
+
+**Trade-off**: Flexibility vs Simplicity
+
+**Decision**: Abstract file server operations through handler functions
+
+**Rationale**:
+- Allows support for different file server implementations (Plik, AWS S3, custom)
+- Maintains simplicity for common use cases
+- Enables plug-in architecture for custom backends
+
+---
+
+## Deployment Architecture
+
+### Minimum Infrastructure
+
+| Component | Minimum | Notes |
+|-----------|---------|-------|
+| NATS Server | 1 instance | Single node for development |
+| File Server | 1 instance | HTTP server for large payloads |
+| Client Memory | 50MB | Desktop platforms |
+| Client Memory | 256KB | MicroPython devices |
+
+### Environment Variables
+
+| Variable | Default | Description |
+|----------|---------|-------------|
+| `NATS_URL` | `nats://localhost:4222` | NATS server URL |
+| `FILESERVER_URL` | `http://localhost:8080` | HTTP file server URL |
+| `SIZE_THRESHOLD` | `1000000` | Size threshold in bytes |
+
+### Container Deployment
 
 ```mermaid
-graph TD
+flowchart TD
-    subgraph JuliaModule
+    subgraph "Docker Network"
-        SmartSendJulia[SmartSend Julia]
+        NATS_Container[NATS Server]
-        SizeCheck[Size Check]
+        FileServer_Container[Plik File Server]
-        DirectPath[Direct Path]
+        App_Container[Application Container]
-        LinkPath[Link Path]
-        HTTPClient[HTTP Client]
     end
     
-    SmartSendJulia --> SizeCheck
+    App_Container -->|NATS| NATS_Container
-    SizeCheck -->|< 1MB| DirectPath
+    App_Container -->|HTTP| FileServer_Container
-    SizeCheck -->|>= 1MB| LinkPath
-    LinkPath --> HTTPClient
     
-    style JuliaModule fill:#c5e1a5
+    style NATS_Container fill:#fff3e0,stroke:#f57c00
+    style FileServer_Container fill:#f3e5f5,stroke:#9c27b4
+    style App_Container fill:#e3f2fd,stroke:#2196f3
 ```
 
-### 4. JavaScript Module Architecture
+---
 
-```mermaid
+## Security Considerations
-graph TD
-    subgraph JSModule
-        SmartSendJS[SmartSend JS]
-        SmartReceiveJS[SmartReceive JS]
-        JetStreamConsumer[JetStream Pull Consumer]
-        ApacheArrow[Apache Arrow]
-    end
 
-    SmartSendJS --> NATS
+### Payload Integrity
-    SmartReceiveJS --> JetStreamConsumer
-    JetStreamConsumer --> ApacheArrow
 
-    style JSModule fill:#f3e5f5
+**Mechanism**: SHA-256 checksum via metadata
-```
 
-## Implementation Details
+**Implementation**:
+- Sender calculates checksum and stores in payload metadata
+- Receiver validates checksum on receipt
 
-### Julia Implementation
+### Transport Security
 
-#### Dependencies
+**Mechanism**: TLS support for NATS connections
-- `NATS.jl` - Core NATS functionality
-- `Arrow.jl` - Arrow IPC serialization
-- `JSON3.jl` - JSON parsing
-- `HTTP.jl` - HTTP client for file server
-- `Dates.jl` - Timestamps for logging
 
-#### SmartSend Function
+**Implementation**:
+- Use `nats://` URL for plain text
+- Use `tls://` URL for TLS-encrypted connections
 
-```julia
+### File Server Security
-function SmartSend(
-    subject::String,
-    data::Any,
-    type::String = "json";
-    nats_url::String = "nats://localhost:4222",
-    fileserver_url::String = "http://localhost:8080/upload",
-    size_threshold::Int = 1_000_000  # 1MB
-)
-```
 
-**Flow:**
+**Mechanism**: Authentication token for file uploads
-1. Serialize data to Arrow IPC stream (if table)
-2. Check payload size
-3. If < threshold: publish directly to NATS with Base64-encoded payload
-4. If >= threshold: upload to HTTP server, publish NATS with URL
 
-#### SmartReceive Handler
+**Implementation**:
+- Plik uses upload token in `X-UploadToken` header
+- Application can implement custom authentication
 
-```julia
+---
-function SmartReceive(msg::NATS.Message)
-    # Parse envelope
-    # Check transport type
-    # If direct: decode Base64 payload
-    # If link: fetch from URL with exponential backoff
-    # Deserialize Arrow IPC to DataFrame
-end
-```
 
-### JavaScript Implementation
+## Testing Architecture
 
-#### Dependencies
+### Unit Test Coverage
-- `nats.js` - Core NATS functionality
-- `apache-arrow` - Arrow IPC serialization
-- `uuid` - Correlation ID generation
 
-#### SmartSend Function
+| Test Category | Coverage | Files |
+|---------------|----------|-------|
+| Serialization | All payload types | `test/test_*_sender.*` |
+| Deserialization | All payload types | `test/test_*_receiver.*` |
+| Transport selection | Direct vs link | `test/test_*_mix_payloads.*` |
+| File server upload | Plik integration | Platform-specific |
+| File server download | Exponential backoff | Platform-specific |
 
-```javascript
+### Integration Test Scenarios
-async function SmartSend(subject, data, type = 'json', options = {})
-```
 
-**Flow:**
+| Scenario | Platforms | Payloads | Transport | Expected Result |
-1. Serialize data to Arrow IPC buffer (if table)
+|----------|-----------|----------|-----------|-----------------|
-2. Check payload size
+| Cross-platform text | Julia ↔ JS ↔ Python | text | direct | Round-trip successful |
-3. If < threshold: publish directly to NATS
+| Arrow IPC round-trip | Julia ↔ JS ↔ Python | arrowtable | direct | Arrow IPC preserved |
-4. If >= threshold: upload to HTTP server, publish NATS with URL
+| Large file transfer | All | image/audio/video | link | File server upload/download |
+| Multi-payload mixed | All | text + image + file | direct/link | All payloads preserved |
 
-#### SmartReceive Handler
+---
 
-```javascript
+## Versioning
-async function SmartReceive(msg, options = {})
-```
 
-**Flow:**
+### Architecture Versioning
-1. Parse envelope
-2. Check transport type
-3. If direct: decode Base64 payload
-4. If link: fetch with exponential backoff
-5. Deserialize Arrow IPC with zero-copy
 
-## Scenario Implementations
+| Component | Version | Notes |
+|-----------|---------|-------|
+| Architecture | 1.0.0 | Initial release |
+| Protocol | v1 | Message envelope protocol version |
 
-### Scenario 1: Command & Control (Small JSON)
+### Backward Compatibility
 
-**Julia (Receiver):**
+| Version | Supported Platforms |
-```julia
+|---------|---------------------|
-# Subscribe to control subject
+| v1.0.x | Julia 1.7+, Node.js 16+, Python 3.8+, MicroPython 1.19+ |
-# Parse JSON envelope
-# Execute simulation with parameters
-# Send acknowledgment
-```
 
-**JavaScript (Sender):**
+---
-```javascript
-// Create small JSON config
-// Send via SmartSend with type="json"
-```
 
-### Scenario 2: Deep Dive Analysis (Large Arrow Table)
+## Change Log
 
-**Julia (Sender):**
+| Date | Version | Changes |
-```julia
+|------|---------|---------|
-# Create large DataFrame
+| 2026-03-13 | 1.0.0 | Initial architecture documentation |
-# Convert to Arrow IPC stream
-# Check size (> 1MB)
-# Upload to HTTP server
-# Publish NATS with URL
-```
 
-**JavaScript (Receiver):**
+---
-```javascript
-// Receive NATS message with URL
-// Fetch data from HTTP server
-// Parse Arrow IPC with zero-copy
-// Load into Perspective.js or D3
-```
 
-### Scenario 3: Live Audio Processing
+## References
 
-**JavaScript (Sender):**
+- [`docs/requirements.md`](./requirements.md) - Business requirements and user stories
-```javascript
+- [`docs/spec.md`](./spec.md) - Technical specification and contracts
-// Capture audio chunk
+- [`src/NATSBridge.jl`](../src/NATSBridge.jl) - Ground truth implementation
-// Send as binary with metadata headers
+- [`README.md`](../README.md) - Project overview
-// Use SmartSend with type="audio"
-```
 
-**Julia (Receiver):**
+---
-```julia
-// Receive audio data
-// Perform FFT or AI transcription
-// Send results back (JSON + Arrow table)
-```
 
-### Scenario 4: Catch-Up (JetStream)
+*This architecture document is versioned and maintained in git alongside the codebase. All implementations must adhere to this architecture.*
-
-**Julia (Producer):**
-```julia
-# Publish to JetStream
-# Include metadata for temporal tracking
-```
-
-**JavaScript (Consumer):**
-```javascript
-// Connect to JetStream
-// Request replay from last 10 minutes
-// Process historical and real-time messages
-```
-
-## Performance Considerations
-
-### Zero-Copy Reading
-- Use Arrow's memory-mapped file reading
-- Avoid unnecessary data copying during deserialization
-- Use Apache Arrow's native IPC reader
-
-### Exponential Backoff
-- Implement exponential backoff for HTTP link fetching
-- Maximum retry count: 5
-- Base delay: 100ms, max delay: 5000ms
-
-### Correlation ID Logging
-- Log correlation_id at every stage
-- Include: send, receive, serialize, deserialize
-- Use structured logging format
-
-## Testing Strategy
-
-### Unit Tests
-- Test SmartSend with various payload sizes
-- Test SmartReceive with direct and link transport
-- Test Arrow IPC serialization/deserialization
-
-### Integration Tests
-- Test full flow with NATS server
-- Test large data transfer (> 100MB)
-- Test audio processing pipeline
-
-### Performance Tests
-- Measure throughput for small payloads
-- Measure throughput for large payloads

docs/requirements.md

@@ -0,0 +1,420 @@
+# Requirements Document: NATSBridge
+
+**Version**: 1.0.0  
+**Date**: 2026-03-13  
+**Status**: Active  
+**Ground Truth**: [`src/NATSBridge.jl`](../src/NATSBridge.jl)
+
+---
+
+## Executive Summary
+
+NATSBridge is a cross-platform, bi-directional data bridge that enables seamless communication between **Julia**, **JavaScript**, **Python**, and **MicroPython** applications using NATS as the message bus. The system implements the **Claim-Check pattern** for efficient handling of large payloads (>0.5MB) by uploading them to an HTTP file server instead of sending raw binary data over NATS.
+
+---
+
+## Business Goals
+
+### Primary Objectives
+
+1. **Cross-Platform Interoperability**: Enable seamless data exchange between Julia, JavaScript (for both Server-Side rendering and Client-Side rendering webapp), Python, and MicroPython applications without platform-specific barriers.
+
+2. **Efficient Large Payload Handling**: Implement intelligent transport selection based on payload size:
+   - **Direct Transport**: Small payloads (<0.5MB) sent directly via NATS
+   - **Link Transport**: Large payloads (≥0.5MB) uploaded to HTTP file server, URL sent via NATS
+
+3. **Unified API Across Platforms**: Provide consistent `smartsend()` and `smartreceive()` functions across all supported platforms while maintaining idiomatic implementations.
+
+4. **Developer Productivity**: Reduce onboarding time and simplify integration through comprehensive documentation and test examples.
+
+### Success Metrics
+
+| Metric | Target | Measurement Method |
+|--------|--------|-------------------|
+| 95% of messages complete within 200ms | 95% | Synthetic monitoring |
+| <2 days from onboarding to first PR | 2 days | PR timeline tracking |
+| 100% of messages validate against spec | 100% | CI block rate |
+| >80% unit test coverage | 80% | Test coverage tools |
+| <1% of PRs bypass validation gates | 1% | CI gate analysis |
+| MTTR <15 minutes for P1 incidents | 15 minutes | Incident tracking |
+
+---
+
+## User Stories
+
+### Core Functionality
+
+| Story | Priority | Acceptance Criteria |
+|-------|----------|---------------------|
+| **As a Julia developer**, I want to send text messages to JavaScript applications that lives on a server and also on a browser  | P1 | Text messages are serialized, encoded, and received correctly across platforms |
+| **As a Python developer**, I want to send tabular data to Julia applications | P1 | DataFrame exchange works with both Arrow IPC and JSON formats |
+| **As a JavaScript developer**, I want to send large files (>0.5MB) from JavaScript applications that lives on a server and also on a browser to other applications | P1 | Large files are automatically uploaded to file server and URLs are sent via NATS |
+| **As a MicroPython developer**, I want to send sensor data with minimal memory usage | P1 | Direct transport works for payloads <100KB on memory-constrained devices |
+
+### Multi-Payload Support
+
+| Story | Priority | Acceptance Criteria |
+|-------|----------|---------------------|
+| **As a developer**, I want to send mixed-content messages (text + image + file) | P1 | NATSBridge accepts list of (dataname, data, type) tuples and handles each payload appropriately |
+| **As a developer**, I want to receive multi-payload messages | P1 | NATSBridge returns payloads as list of tuples with correct types preserved |
+
+### File Server Integration
+
+| Story | Priority | Acceptance Criteria |
+|-------|----------|---------------------|
+| **As a developer**, I want to use Plik as the file server | P2 | Plik one-shot upload mode is supported with upload ID and token handling |
+| **As a developer**, I want to use custom HTTP file servers | P2 | Handler function abstraction allows plugging in AWS S3 or custom implementations |
+
+### Reliability Features
+
+| Story | Priority | Acceptance Criteria |
+|-------|----------|---------------------|
+| **As a developer**, I want automatic retry on file server download failures | P1 | Exponential backoff with configurable retries (default: 5, base_delay: 100ms, max_delay: 5000ms) |
+| **As a developer**, I want message tracing across distributed systems | P1 | Correlation ID is propagated through all message processing steps |
+
+---
+
+## Non-Functional Requirements
+
+### Performance Requirements
+
+| Requirement | Specification | Test Method |
+|-------------|---------------|-------------|
+| Message serialization overhead | <50ms for 10KB payload | Benchmark tests |
+| Message deserialization overhead | <50ms for 10KB payload | Benchmark tests |
+| NATS connection establishment | <100ms | Connection pool benchmarks |
+| File upload latency | <1s for 0.5MB file | Integration tests |
+| File download latency | <1s for 0.5MB file | Integration tests |
+
+### Scalability Requirements
+
+| Requirement | Specification |
+|-------------|---------------|
+| Concurrent connections | Support 100+ simultaneous NATS connections |
+| Message throughput | Handle 1000+ messages/second per instance |
+| File server scalability | Support horizontal scaling of file server backend |
+
+### Reliability Requirements
+
+| Requirement | Specification |
+|-------------|---------------|
+| Message delivery | At-least-once delivery semantics via NATS |
+| File server availability | Graceful degradation when file server is unavailable |
+| Connection recovery | Auto-reconnect on NATS connection failure |
+
+### Security Requirements
+
+| Requirement | Specification |
+|-------------|---------------|
+| Payload integrity | SHA-256 checksum support via metadata |
+| Transport security | TLS support for NATS connections |
+| File server security | Authentication token for file uploads |
+
+### Compatibility Requirements
+
+| Platform | Minimum Version | Notes |
+|----------|-----------------|-------|
+| Julia | 1.7+ | Arrow.jl required for arrowtable support |
+| Node.js | 16+ | nats.js required |
+| Python | 3.8+ | pyarrow required for arrowtable support |
+| MicroPython | 1.19+ | Limited to direct transport |
+
+---
+
+## Out of Scope
+
+### Phase 1 (Current Implementation)
+
+| Feature | Reason |
+|---------|--------|
+| NATS JetStream support | Core NATS sufficient for current use cases |
+| Message compression | Compression adds complexity without clear benefit |
+| Message encryption | Payload encryption is application-layer concern |
+| Persistent message queues | NATS request-reply pattern sufficient |
+| Advanced routing rules | Simple NATS subject matching sufficient |
+
+### Future Considerations
+
+| Feature | Future Phase |
+|---------|--------------|
+| JetStream streams and consumers | Phase 2 |
+| Message TTL and dead-letter queues | Phase 3 |
+| Message tracing with OpenTelemetry | Phase 3 |
+| Rate limiting and quota management | Phase 4 |
+
+---
+
+## Boundary Definitions
+
+### What NATSBridge Handles
+
+| Function | Description |
+|----------|-------------|
+| Message serialization | Converts data types to binary format |
+| Message encoding | Base64, JSON, Arrow IPC encoding |
+| Transport selection | Direct vs link based on size threshold |
+| NATS publishing | Publishes messages to NATS subjects |
+| NATS subscription | Receives and processes NATS messages |
+| File server upload | Uploads large payloads to HTTP server |
+| File server download | Downloads payloads from HTTP server with retry |
+| Correlation ID generation | Creates and propagates UUIDs |
+| Data deserialization | Converts binary format back to native types |
+
+### What NATSBridge Does NOT Handle
+
+| Function | Handled By |
+|----------|------------|
+| NATS server management | External NATS deployment |
+| File server management | External HTTP server deployment |
+| Application business logic | Application code using NATSBridge |
+| Message encryption | Application layer |
+| Message compression | Application layer |
+| Authentication/Authorization | NATS server configuration |
+
+---
+
+## Payload Type Requirements
+
+### Supported Payload Types
+
+| Type | Julia | JavaScript | Python | MicroPython | Description |
+|------|-------|------------|--------|-------------|-------------|
+| `text` | `String` | `string` | `str` | `str` | Plain text strings |
+| `dictionary` | `Dict`, `NamedTuple` | `Object`, `Array` | `dict`, `list` | `dict` | JSON-serializable data |
+| `arrowtable` | `DataFrame`, `Arrow.Table` | `Array<Object>` | `pandas.DataFrame` | ❌ | Tabular data (Arrow IPC) |
+| `jsontable` | `Vector{NamedTuple}` | `Array<Object>` | `list[dict]` | ⚠️ | Tabular data (JSON) |
+| `image` | `Vector{UInt8}` | `Uint8Array`, `Buffer` | `bytes` | `bytearray` | Image binary data |
+| `audio` | `Vector{UInt8}` | `Uint8Array`, `Buffer` | `bytes` | `bytearray` | Audio binary data |
+| `video` | `Vector{UInt8}` | `Uint8Array`, `Buffer` | `bytes` | `bytearray` | Video binary data |
+| `binary` | `Vector{UInt8}`, `IOBuffer` | `Uint8Array`, `Buffer` | `bytes`, `bytearray` | `bytearray` | Generic binary data |
+
+### Encoding Requirements
+
+| Payload Type | Encoding Method | Notes |
+|--------------|-----------------|-------|
+| `text` | UTF-8 → Base64 | Text must be String type |
+| `dictionary` | JSON → Base64 | JSON.jl for Julia |
+| `arrowtable` | Arrow IPC → Base64 | Requires Arrow.jl/pyarrow |
+| `jsontable` | JSON → Base64 | Human-readable format |
+| `image`/`audio`/`video`/`binary` | Direct → Base64 | Binary data preserved |
+
+---
+
+## Size Threshold Requirements
+
+### Direct Transport Threshold
+
+| Platform | Threshold | Notes |
+|----------|-----------|-------|
+| Desktop (Julia/JS/Python) | 0.5MB | Default size threshold |
+| MicroPython | 100KB | Lower threshold for memory constraints |
+
+### Maximum Payload Size
+
+| Platform | Maximum | Notes |
+|----------|---------|-------|
+| Desktop | Unlimited | Limited by NATS server configuration |
+| MicroPython | 50KB | Hard limit due to 256KB-1MB memory |
+
+---
+
+## Message Envelope Requirements
+
+### Required Fields
+
+| Field | Type | Purpose |
+|-------|------|---------|
+| `correlation_id` | String (UUID) | Track message flow across systems |
+| `msg_id` | String (UUID) | Unique message identifier |
+| `timestamp` | String (ISO 8601) | Message publication timestamp |
+| `send_to` | String | NATS subject to publish to |
+| `msg_purpose` | String | ACK, NACK, updateStatus, shutdown, chat |
+| `sender_name` | String | Sender application name |
+| `sender_id` | String (UUID) | Sender unique identifier |
+| `receiver_name` | String | Receiver application name (empty = broadcast) |
+| `receiver_id` | String (UUID) | Receiver unique identifier (empty = broadcast) |
+| `reply_to` | String | Topic for reply messages |
+| `reply_to_msg_id` | String | Message ID being replied to |
+| `broker_url` | String | NATS server URL |
+| `metadata` | Dict | Message-level metadata |
+| `payloads` | Array | List of payload objects |
+
+### Payload Fields
+
+| Field | Type | Purpose |
+|-------|------|---------|
+| `id` | String (UUID) | Unique payload identifier |
+| `dataname` | String | Name of the payload |
+| `payload_type` | String | Type: text, dictionary, arrowtable, etc. |
+| `transport` | String | direct or link |
+| `encoding` | String | none, json, base64, arrow-ipc |
+| `size` | Integer | Payload size in bytes |
+| `data` | Any | Base64 string or URL |
+| `metadata` | Dict | Payload-level metadata |
+
+---
+
+## Error Handling Requirements
+
+### Error Codes
+
+| Error | Condition | Response |
+|-------|-----------|----------|
+| `Unknown payload_type` | Unsupported type | Throw error |
+| `Failed to upload` | File server error | Throw error |
+| `Failed to fetch` | File server unavailable | Retry with exponential backoff |
+| `Unknown transport` | Invalid transport type | Throw error |
+| `NATS connection failed` | NATS unavailable | Throw error |
+
+### Exception Handling
+
+| Scenario | Handler |
+|----------|---------|
+| File server unavailable | Retry up to 5 times with exponential backoff |
+| NATS publish failure | Connection auto-reconnect |
+| Deserialization error | Log correlation ID and throw error |
+| Memory overflow (MicroPython) | Reject payloads >50KB |
+
+---
+
+## Testing Requirements
+
+### Unit Tests
+
+| Test Category | Coverage | Files |
+|---------------|----------|-------|
+| Serialization | All payload types | `test/test_*_sender.*` |
+| Deserialization | All payload types | `test/test_*_receiver.*` |
+| Transport selection | Direct vs link | `test/test_*_mix_payloads.*` |
+| File server upload | Plik integration | Platform-specific |
+| File server download | Exponential backoff | Platform-specific |
+
+### Integration Tests
+
+| Test Scenario | Success Criteria |
+|-------------|-----------------|
+| Cross-platform text message | Julia ↔ JavaScript ↔ Python |
+| Cross-platform tabular data | Arrow IPC round-trip |
+| Large file transfer | File server upload/download |
+| Multi-payload mixed content | All payload types in one message |
+
+---
+
+## API Contract
+
+### smartsend Signature
+
+```julia
+function smartsend(
+    subject::String,
+    data::AbstractArray{Tuple{String, Any, String}};
+    broker_url::String = "nats://localhost:4222",
+    fileserver_url::String = "http://localhost:8080",
+    fileserver_upload_handler::Function = plik_oneshot_upload,
+    size_threshold::Int = 1_000_000,
+    correlation_id::String = string(uuid4()),
+    msg_purpose::String = "chat",
+    sender_name::String = "NATSBridge",
+    receiver_name::String = "",
+    receiver_id::String = "",
+    reply_to::String = "",
+    reply_to_msg_id::String = "",
+    is_publish::Bool = true,
+    NATS_connection::Union{NATS.Connection, Nothing} = nothing,
+    msg_id::String = string(uuid4()),
+    sender_id::String = string(uuid4())
+)::Tuple{msg_envelope_v1, String}
+```
+
+### smartreceive Signature
+
+```julia
+function smartreceive(
+    msg::NATS.Msg;
+    fileserver_download_handler::Function = _fetch_with_backoff,
+    max_retries::Int = 5,
+    base_delay::Int = 100,
+    max_delay::Int = 5000
+)::JSON.Object{String, Any}
+```
+
+---
+
+## Dependencies
+
+### Required Dependencies
+
+| Platform | Package | Version |
+|----------|---------|---------|
+| Julia | NATS.jl | Latest stable |
+| Julia | JSON.jl | Latest stable |
+| Julia | Arrow.jl | Latest stable |
+| Julia | HTTP.jl | Latest stable |
+| Julia | UUIDs.jl | Latest stable |
+| Node.js | nats | Latest stable |
+| Node.js | node-fetch | Latest stable |
+| Python | nats-py | Latest stable |
+| Python | aiohttp | Latest stable |
+| Python | pyarrow | Latest stable |
+
+### Optional Dependencies
+
+| Platform | Package | Use Case |
+|----------|---------|----------|
+| Julia | DataFrames.jl | DataFrame support for arrowtable |
+| Python | pandas | DataFrame support for arrowtable |
+
+---
+
+## Deployment Requirements
+
+### Minimum Infrastructure
+
+| Component | Minimum | Notes |
+|-----------|---------|-------|
+| NATS Server | 1 instance | Single node for development |
+| File Server | 1 instance | HTTP server for large payloads |
+| Client Memory | 50MB | Desktop platforms |
+| Client Memory | 256KB | MicroPython devices |
+
+### Environment Variables
+
+| Variable | Default | Description |
+|----------|---------|-------------|
+| `NATS_URL` | `nats://localhost:4222` | NATS server URL |
+| `FILESERVER_URL` | `http://localhost:8080` | HTTP file server URL |
+| `SIZE_THRESHOLD` | `1000000` | Size threshold in bytes |
+
+---
+
+## Versioning
+
+### Current Version
+
+- **Major**: 1 (Breaking changes require major version bump)
+- **Minor**: 0 (Feature additions)
+- **Patch**: 0 (Bug fixes)
+
+### Version Compatibility
+
+| Version | Supported Platforms |
+|---------|---------------------|
+| v1.0.x | Julia 1.7+, Node.js 16+, Python 3.8+, MicroPython 1.19+ |
+
+---
+
+## Change Log
+
+| Date | Version | Changes |
+|------|---------|---------|
+| 2026-03-13 | 1.0.0 | Initial requirements document |
+
+---
+
+## References
+
+- [`src/NATSBridge.jl`](../src/NATSBridge.jl) - Ground truth implementation
+- [`README.md`](../README.md) - Project overview
+- [`docs/architecture.md`](./architecture.md) - Architecture documentation
+- [`docs/implementation.md`](./implementation.md) - Implementation details
+- [`docs/walkthrough.md`](./walkthrough.md) - Usage examples

docs/tutorial.md

@@ -0,0 +1,741 @@
+# Cross-Platform NATSBridge Tutorial
+
+A step-by-step guide to get started with NATSBridge across **Julia**, **JavaScript**, and **Python/MicroPython**.
+
+## Table of Contents
+
+1. [Overview](#overview)
+2. [Prerequisites](#prerequisites)
+3. [Installation](#installation)
+4. [Quick Start](#quick-start)
+5. [Basic Examples](#basic-examples)
+6. [Advanced Usage](#advanced-usage)
+
+---
+
+## Overview
+
+NATSBridge enables seamless communication across platforms through NATS, with automatic transport selection based on payload size:
+
+- **Direct Transport**: Payloads < 1MB are sent directly via NATS (Base64 encoded)
+- **Link Transport**: Payloads >= 1MB are uploaded to an HTTP file server and referenced via URL
+
+### Cross-Platform API Parity
+
+All three platforms use the same high-level API:
+
+```
+# Input format
+smartsend(subject, [(dataname, data, type), ...], options)
+
+# Output format
+(env, env_json_str) = smartsend(...)
+env = smartreceive(msg, options)
+```
+
+**Important Platform Differences:**
+
+1. **Encoding field:** Julia and JavaScript preserve the original serialization format in the encoding field (`"base64"`, `"json"`, or `"arrow-ipc"`), while Python and MicroPython always use `"base64"` for all direct transport payloads.
+
+2. **Async vs Sync:** JavaScript and Python desktop use async/await, while MicroPython uses synchronous API.
+
+### Supported Payload Types
+
+| Type | Julia | JavaScript | Python | MicroPython |
+|------|-------|------------|--------|-------------|
+| `text` | `String` | `string` | `str` | `str` |
+| `dictionary` | `Dict` | `Object` | `dict` | `dict` |
+| `arrowtable` | `DataFrame` | `Array<Object>` | `pandas.DataFrame` | ❌ |
+| `jsontable` | `Vector{NamedTuple}` | `Array<Object>` | `list[dict]` | ❌ |
+| `table` | ❌ | ❌ | `pandas.DataFrame` | ❌ |
+| `image` | `Vector{UInt8}` | `Uint8Array` | `bytes` | `bytearray` |
+| `audio` | `Vector{UInt8}` | `Uint8Array` | `bytes` | `bytearray` |
+| `video` | `Vector{UInt8}` | `Uint8Array` | `bytes` | `bytearray` |
+| `binary` | `Vector{UInt8}` | `Uint8Array` | `bytes` | `bytearray` |
+
+**Note on MicroPython:** MicroPython does not support table types (`arrowtable`, `jsontable`, or `table`) due to memory constraints. Use `dictionary` or `binary` instead.
+
+---
+
+## Prerequisites
+
+Before you begin, ensure you have:
+
+1. **NATS Server** running (or accessible)
+2. **HTTP File Server** (optional, for large payloads > 1MB)
+3. **Platform-specific packages** installed
+
+---
+
+## Installation
+
+### Julia
+
+```julia
+using Pkg
+Pkg.add("NATS")
+Pkg.add("Arrow")
+Pkg.add("JSON3")
+Pkg.add("HTTP")
+Pkg.add("UUIDs")
+Pkg.add("Dates")
+```
+
+### JavaScript (Node.js)
+
+```bash
+npm install nats uuid apache-arrow node-fetch
+```
+
+### JavaScript (Browser)
+
+```html
+<script src="https://unpkg.com/nats-js/dist/bundle/nats.min.js"></script>
+<script src="https://unpkg.com/apache-arrow/arrow.min.js"></script>
+```
+
+### Python (Desktop)
+
+```bash
+pip install nats-py aiohttp pyarrow pandas
+```
+
+### MicroPython
+
+Uses built-in modules: `network`, `socket`, `time`, `json`, `base64`
+
+---
+
+## Quick Start
+
+### Step 1: Start NATS Server
+
+```bash
+docker run -p 4222:4222 nats:latest
+```
+
+### Step 2: Start HTTP File Server (Optional)
+
+```bash
+mkdir -p /tmp/fileserver
+python3 -m http.server 8080 --directory /tmp/fileserver
+```
+
+### Step 3: Send Your First Message
+
+#### Julia
+
+```julia
+using NATSBridge
+
+# Send a text message
+data = [("message", "Hello World", "text")]
+env, env_json_str = smartsend("/chat/room1", data, broker_url="nats://localhost:4222")
+# env: msg_envelope_v1 struct with all metadata and payloads
+# env_json_str: JSON string representation of the envelope for publishing
+println("Message sent!")
+
+# Or use is_publish=false to get envelope and JSON without publishing
+env, env_json_str = smartsend("/chat/room1", data, broker_url="nats://localhost:4222", is_publish=false)
+# env: msg_envelope_v1 struct
+# env_json_str: JSON string for publishing to NATS
+```
+
+#### JavaScript
+
+```javascript
+const NATSBridge = require('./src/natsbridge.js');
+
+// Send a text message
+const data = [["message", "Hello World", "text"]];
+const [env, env_json_str] = await NATSBridge.smartsend(
+    "/chat/room1",
+    data,
+    { broker_url: "nats://localhost:4222" }
+);
+// env: Object with all metadata and payloads
+// env_json_str: JSON string for publishing
+console.log("Message sent!");
+
+// Or use is_publish=false
+const [env, env_json_str] = await NATSBridge.smartsend(
+    "/chat/room1",
+    data,
+    { broker_url: "nats://localhost:4222", is_publish: false }
+);
+```
+
+#### Python
+
+```python
+from natsbridge import smartsend
+
+# Send a text message
+data = [("message", "Hello World", "text")]
+env, env_json_str = await smartsend(
+    "/chat/room1",
+    data,
+    broker_url="nats://localhost:4222"
+)
+# env: Dict with all metadata and payloads
+# env_json_str: JSON string for publishing
+print("Message sent!")
+
+# Or use is_publish=False
+env, env_json_str = await smartsend(
+    "/chat/room1",
+    data,
+    broker_url="nats://localhost:4222",
+    is_publish=False
+)
+# env: Dict with all metadata and payloads
+# env_json_str: JSON string for publishing to NATS
+```
+
+#### MicroPython
+
+```python
+from natsbridge_mpy import NATSBridge
+
+bridge = NATSBridge()
+
+# Send a text message (limited to small payloads)
+data = [("message", "Hello World", "text")]
+env, env_json_str = bridge.smartsend(
+    "/chat/room1",
+    data,
+    size_threshold=100000  # Lower threshold for MicroPython
+)
+print("Message sent!")
+```
+
+### Step 4: Receive Messages
+
+#### Julia
+
+```julia
+using NATSBridge
+
+# Receive and process message
+env = smartreceive(msg; fileserver_download_handler=_fetch_with_backoff)
+# Returns: ::JSON.Object{String, Any} with "payloads" field containing Vector{Tuple{String, Any, String}}
+# Access payloads: for (dataname, data, type) in env["payloads"]
+for (dataname, data, type) in env["payloads"]
+    println("Received $dataname: $data")
+end
+```
+
+#### JavaScript
+
+```javascript
+const NATSBridge = require('./src/natsbridge.js');
+
+// Receive and process message
+const env = await NATSBridge.smartreceive(msg, {
+    fileserver_download_handler: NATSBridge.fetchWithBackoff
+});
+// env.payloads = [[dataname, data, type], ...]
+for (const [dataname, data, type] of env.payloads) {
+    console.log(`Received ${dataname}:`, data);
+}
+```
+
+#### Python
+
+```python
+from natsbridge import smartreceive, fetch_with_backoff
+
+# Receive and process message
+env = await smartreceive(
+    msg,
+    fileserver_download_handler=fetch_with_backoff
+)
+# env["payloads"] = [(dataname, data, type), ...]
+for dataname, data, type_ in env["payloads"]:
+    print(f"Received {dataname}: {data}")
+```
+
+---
+
+## Basic Examples
+
+### Example 1: Sending a Dictionary
+
+#### Julia
+
+```julia
+using NATSBridge
+
+config = Dict(
+    "wifi_ssid" => "MyNetwork",
+    "wifi_password" => "password123",
+    "update_interval" => 60
+)
+
+data = [("config", config, "dictionary")]
+env, env_json_str = smartsend("/device/config", data, broker_url="nats://localhost:4222")
+```
+
+#### JavaScript
+
+```javascript
+const NATSBridge = require('./src/natsbridge.js');
+
+const config = {
+    wifi_ssid: "MyNetwork",
+    wifi_password: "password123",
+    update_interval: 60
+};
+
+const data = [["config", config, "dictionary"]];
+const [env, env_json_str] = await NATSBridge.smartsend(
+    "/device/config",
+    data,
+    { broker_url: "nats://localhost:4222" }
+);
+```
+
+#### Python
+
+```python
+from natsbridge import smartsend
+
+config = {
+    "wifi_ssid": "MyNetwork",
+    "wifi_password": "password123",
+    "update_interval": 60
+}
+
+data = [("config", config, "dictionary")]
+env, env_json_str = await smartsend(
+    "/device/config",
+    data,
+    broker_url="nats://localhost:4222"
+)
+```
+
+#### MicroPython
+
+```python
+from natsbridge_mpy import NATSBridge
+
+bridge = NATSBridge()
+
+config = {
+    "wifi_ssid": "MyNetwork",
+    "wifi_password": "password123",
+    "update_interval": 60
+}
+
+data = [("config", config, "dictionary")]
+env, env_json_str = bridge.smartsend(
+    "/device/config",
+    data,
+    size_threshold=100000
+)
+```
+
+### Example 2: Sending Binary Data (Image)
+
+#### Julia
+
+```julia
+using NATSBridge
+
+# Read image file
+image_data = read("image.png")
+
+data = [("user_image", image_data, "binary")]
+env, env_json_str = smartsend("/chat/image", data, broker_url="nats://localhost:4222")
+```
+
+#### JavaScript
+
+```javascript
+const NATSBridge = require('./src/natsbridge.js');
+const fs = require('fs');
+
+// Read image file
+const image_data = fs.readFileSync('image.png');
+
+const data = [["user_image", image_data, "binary"]];
+const [env, env_json_str] = await NATSBridge.smartsend(
+    "/chat/image",
+    data,
+    { broker_url: "nats://localhost:4222" }
+);
+```
+
+#### Python
+
+```python
+from natsbridge import smartsend
+
+# Read image file
+with open("image.png", "rb") as f:
+    image_data = f.read()
+
+data = [("user_image", image_data, "binary")]
+env, env_json_str = await smartsend(
+    "/chat/image",
+    data,
+    broker_url="nats://localhost:4222"
+)
+```
+
+#### MicroPython
+
+```python
+from natsbridge_mpy import NATSBridge
+
+bridge = NATSBridge()
+
+# Read image file
+with open("image.png", "rb") as f:
+    image_data = f.read()
+
+data = [("user_image", image_data, "binary")]
+env, env_json_str = bridge.smartsend(
+    "/chat/image",
+    data,
+    size_threshold=100000
+)
+```
+
+### Example 3: Request-Response Pattern
+
+#### Julia (Requester)
+
+```julia
+using NATSBridge
+
+# Send command with reply-to
+data = [("command", Dict("action" => "read_sensor"), "dictionary")]
+env, env_json_str = smartsend(
+    "/device/command",
+    data,
+    broker_url="nats://localhost:4222",
+    reply_to="/device/response",
+    reply_to_msg_id="cmd-001"
+)
+```
+
+#### JavaScript (Requester)
+
+```javascript
+const NATSBridge = require('./src/natsbridge.js');
+
+// Send command with reply-to
+const data = [["command", { action: "read_sensor" }, "dictionary"]];
+const [env, env_json_str] = await NATSBridge.smartsend(
+    "/device/command",
+    data,
+    {
+        broker_url: "nats://localhost:4222",
+        reply_to: "/device/response",
+        reply_to_msg_id: "cmd-001"
+    }
+);
+```
+
+#### Python (Requester)
+
+```python
+from natsbridge import smartsend
+
+# Send command with reply-to
+data = [("command", {"action": "read_sensor"}, "dictionary")]
+env, env_json_str = await smartsend(
+    "/device/command",
+    data,
+    broker_url="nats://localhost:4222",
+    reply_to="/device/response",
+    reply_to_msg_id="cmd-001"
+)
+```
+
+#### Julia (Responder)
+
+```julia
+using NATSBridge, NATS
+
+const SUBJECT = "/device/command"
+const NATS_URL = "nats://localhost:4222"
+
+function test_responder()
+    conn = NATS.connect(NATS_URL)
+    NATS.subscribe(conn, SUBJECT) do msg
+        env = smartreceive(msg, fileserver_download_handler=_fetch_with_backoff)
+        
+        reply_to = env["reply_to"]
+        
+        for (dataname, data, type) in env["payloads"]
+            if dataname == "command" && data["action"] == "read_sensor"
+                response = Dict("sensor_id" => "sensor-001", "value" => 42.5)
+                if !isempty(reply_to)
+                    smartsend(reply_to, [("data", response, "dictionary")])
+                end
+            end
+        end
+    end
+    
+    sleep(120)
+    NATS.drain(conn)
+end
+
+test_responder()
+```
+
+---
+
+## Advanced Usage
+
+### Example 4: Large Payloads (File Server)
+
+For payloads larger than 1MB, NATSBridge automatically uses the file server:
+
+#### Julia
+
+```julia
+using NATSBridge
+
+# Create large data (> 1MB)
+large_data = rand(UInt8, 2_000_000)
+
+env, env_json_str = smartsend(
+    "/data/large",
+    [("large_file", large_data, "binary")],
+    broker_url="nats://localhost:4222",
+    fileserver_url="http://localhost:8080"
+)
+
+println("File uploaded to: $(env.payloads[1].data)")
+# Note: For link transport, data field contains the URL string
+```
+
+#### JavaScript
+
+```javascript
+const NATSBridge = require('./src/natsbridge.js');
+
+// Create large data (> 1MB)
+const large_data = Buffer.alloc(2_000_000);
+for (let i = 0; i < large_data.length; i++) {
+    large_data[i] = Math.floor(Math.random() * 256);
+}
+
+const [env, env_json_str] = await NATSBridge.smartsend(
+    "/data/large",
+    [["large_file", large_data, "binary"]],
+    {
+        broker_url: "nats://localhost:4222",
+        fileserver_url: "http://localhost:8080"
+    }
+);
+
+console.log("File uploaded to:", env.payloads[0].data);
+// Note: For link transport, data field contains the URL string
+```
+
+#### Python
+
+```python
+from natsbridge import smartsend
+
+# Create large data (> 1MB)
+import os
+large_data = os.urandom(2_000_000)
+
+env, env_json_str = await smartsend(
+    "/data/large",
+    [("large_file", large_data, "binary")],
+    broker_url="nats://localhost:4222",
+    fileserver_url="http://localhost:8080"
+)
+
+print(f"File uploaded to: {env['payloads'][0]['data']}")
+# Note: For link transport, data field contains the URL string
+```
+
+#### MicroPython
+
+MicroPython enforces a hard limit of 50KB per payload:
+
+```python
+from natsbridge_mpy import NATSBridge
+
+bridge = NATSBridge()
+
+# MicroPython has a hard limit of 50KB per payload
+# Use streaming or chunking for larger data
+small_data = bytes(1000)  # 1KB
+
+data = [("small_file", small_data, "binary")]
+env, env_json_str = bridge.smartsend(
+    "/data/small",
+    data,
+    size_threshold=100000  # Enforced max: 50000 bytes
+)
+```
+
+### Example 5: Mixed Content (Chat with Text + Image)
+
+NATSBridge supports sending multiple payloads with different types in a single message:
+
+#### Julia
+
+```julia
+using NATSBridge
+
+image_data = read("avatar.png")
+
+data = [
+    ("message_text", "Hello with image!", "text"),
+    ("user_avatar", image_data, "image")
+]
+
+env, env_json_str = smartsend("/chat/mixed", data, broker_url="nats://localhost:4222")
+```
+
+#### JavaScript
+
+```javascript
+const NATSBridge = require('./src/natsbridge.js');
+const fs = require('fs');
+
+const image_data = fs.readFileSync('avatar.png');
+
+const data = [
+    ["message_text", "Hello with image!", "text"],
+    ["user_avatar", image_data, "image"]
+];
+
+const [env, env_json_str] = await NATSBridge.smartsend(
+    "/chat/mixed",
+    data,
+    { broker_url: "nats://localhost:4222" }
+);
+```
+
+#### Python
+
+```python
+from natsbridge import smartsend
+
+with open("avatar.png", "rb") as f:
+    image_data = f.read()
+
+data = [
+    ("message_text", "Hello with image!", "text"),
+    ("user_avatar", image_data, "image")
+]
+
+env, env_json_str = await smartsend(
+    "/chat/mixed",
+    data,
+    broker_url="nats://localhost:4222"
+)
+# env: Dict with all metadata and payloads
+```
+
+### Example 6: Table Data (Arrow IPC)
+
+For tabular data, NATSBridge uses Apache Arrow IPC format:
+
+#### Julia
+
+```julia
+using NATSBridge
+using DataFrames
+
+# Create DataFrame
+df = DataFrame(
+    id = [1, 2, 3],
+    name = ["Alice", "Bob", "Charlie"],
+    score = [95, 88, 92]
+)
+
+data = [("students", df, "arrowtable")]
+env, env_json_str = smartsend("/data/students", data, broker_url="nats://localhost:4222")
+```
+
+#### JavaScript
+
+```javascript
+const NATSBridge = require('./src/natsbridge.js');
+
+// Create table data (array of objects)
+const table_data = [
+    { id: 1, name: "Alice", score: 95 },
+    { id: 2, name: "Bob", score: 88 },
+    { id: 3, name: "Charlie", score: 92 }
+];
+
+const data = [["students", table_data, "arrowtable"]];
+const [env, env_json_str] = await NATSBridge.smartsend(
+    "/data/students",
+    data,
+    { broker_url: "nats://localhost:4222" }
+);
+```
+
+#### Python
+
+```python
+from natsbridge import smartsend
+import pandas as pd
+
+# Create DataFrame
+df = pd.DataFrame({
+    'id': [1, 2, 3],
+    'name': ['Alice', 'Bob', 'Charlie'],
+    'score': [95, 88, 92]
+})
+
+data = [("students", df, "table")]
+env, env_json_str = await smartsend(
+    "/data/students",
+    data,
+    broker_url="nats://localhost:4222"
+)
+```
+
+#### MicroPython
+
+MicroPython does not support table type due to memory constraints. Use dictionary or binary instead.
+
+---
+
+## Next Steps
+
+1. **Explore the test directory** for more examples
+2. **Check the documentation** for advanced configuration options
+3. **Read the walkthrough** for building real-world applications
+
+---
+
+## Troubleshooting
+
+### Connection Issues
+
+- Ensure NATS server is running: `docker ps | grep nats`
+- Check firewall settings
+- Verify NATS URL configuration
+
+### File Server Issues
+
+- Ensure file server is running and accessible
+- Check upload permissions
+- Verify file server URL configuration
+
+### Serialization Errors
+
+- Verify data type matches the specified type
+- Check that binary data is in the correct format
+- MicroPython: Ensure payload size < 50KB
+
+---
+
+## License
+
+MIT

docs/walkthrough.md

@@ -0,0 +1,738 @@
+# Walkthrough: NATSBridge
+
+**Version**: 1.0.0  
+**Date**: 2026-03-13  
+**Status**: Active  
+**Ground Truth**: [`src/NATSBridge.jl`](../src/NATSBridge.jl)
+
+---
+
+## Executive Summary
+
+This document provides the **story of flow** for NATSBridge - the cross-platform bi-directional data bridge that enables seamless communication between **Julia**, **JavaScript**, **Python**, and **MicroPython** applications using NATS as the message bus.
+
+This walkthrough serves as the primary onboarding guide for new developers and explains:
+- **User scenarios** - Real-world use cases from developer perspective
+- **Why steps are sequenced** - The rationale behind architectural decisions
+- **What could go wrong** - Common failure scenarios and recovery strategies
+
+---
+
+## Overview: The Big Picture
+
+NATSBridge implements the **Claim-Check pattern** for efficient handling of large payloads (>0.5MB):
+
+```mermaid
+flowchart TB
+    subgraph NATSBridge["NATSBridge Module"]
+        direction TB
+        
+        subgraph Sender["Sender (smartsend)"]
+            direction LR
+            S1["Data Tuples<br/>[(dataname, data, type)]"]
+            S2["Serialize Data"]
+            S3["Size Check"]
+            S4["Transport Selection"]
+            S5["Build Envelope"]
+            S6["Publish to NATS"]
+            
+            S1 --> S2
+            S2 --> S3
+            S3 --> S4
+            S4 --> S5
+            S5 --> S6
+        end
+        
+        subgraph Receiver["Receiver (smartreceive)"]
+            direction LR
+            R1["Subscribe to NATS"]
+            R2["Parse Envelope"]
+            R3["Check Transport"]
+            R4["Deserialize Data"]
+            R5["Return Payloads"]
+            
+            R1 --> R2
+            R2 --> R3
+            R3 --> R4
+            R4 --> R5
+        end
+        
+        S6 -.->|Message| R1
+    end
+    
+    subgraph FileServer["HTTP File Server (Plik)"]
+        direction TB
+        FS1["Upload URL"]
+        FS2["Download URL"]
+        
+        S4 -.->|Large Payload| FS1
+        FS1 -.->|URL| S5
+        R3 -.->|Fetch URL| FS2
+    end
+    
+    style NATSBridge fill:#e1f5fe,stroke:#0288d1,stroke-width:2px
+    style Sender fill:#b3e5fc,stroke:#0288d1
+    style Receiver fill:#b3e5fc,stroke:#0288d1
+    style FileServer fill:#ffe0b2,stroke:#f57c00
+```
+
+### Key Design Principles
+
+### Key Design Principles
+
+| Principle | Description | Rationale |
+|-----------|-------------|-----------|
+| **Claim-Check Pattern** | Large payloads uploaded to HTTP server, URL sent via NATS | NATS has message size limits; avoids NATS overflow |
+| **Automatic Transport Selection** | Direct (< threshold) vs Link (≥ threshold) based on size | Optimizes memory vs network I/O trade-off |
+| **Cross-Platform API** | Consistent `smartsend()`/`smartreceive()` across all platforms | Simplifies developer experience |
+| **Exponential Backoff** | Retry downloads with increasing delays | Handles transient failures gracefully |
+
+---
+
+## User Scenario 1: Chat Webapp ↔ Julia Backend
+
+### Scenario Description
+
+A JavaScript chat webapp wants to send mixed payloads (text message + user avatar image) to a Julia backend, and receive mixed payloads (text response + AI-generated image) back.
+
+### Step-by-Step Flow
+
+#### Step 1: JavaScript Webapp Sends Mixed Payloads
+
+```javascript
+// JavaScript (Browser or Node.js)
+const [env, msgJson] = await NATSBridge.smartsend(
+    "/agent/wine/api/v1/prompt",
+    [
+        ["msg", "Hello! I'm Ton.", "text"],
+        ["avatar", avatarImageData, "image"]
+    ],
+    {
+        broker_url: "ws://localhost:4222",
+        receiver_name: "agent-backend",
+        msg_purpose: "chat"
+    }
+);
+```
+
+**Rationale**:
+- **Why mixed payloads?** Real chat apps often send both text and images together
+- **Why text first?** Text is smaller, sent via direct transport (fast, no file server needed)
+- **Why image second?** Images may trigger link transport if >0.5MB
+
+#### Step 2: Transport Selection
+
+For each payload, NATSBridge determines transport:
+
+| Payload | Size | Transport | Reason |
+|---------|------|-----------|--------|
+| `"msg"` (text) | ~20 bytes | direct | < 0.5MB threshold |
+| `"avatar"` (image) | ~150KB | direct | < 0.5MB threshold |
+
+**Rationale**:
+- Direct transport is faster for small payloads (no file server round-trip)
+- Link transport is used when payload ≥ 0.5MB (avoids NATS size limits)
+
+#### Step 3: Serialization and Encoding
+
+Each payload is serialized:
+
+| Payload | Type | Serialization | Encoding |
+|---------|------|---------------|----------|
+| `"msg"` | `text` | UTF-8 bytes | Base64 |
+| `"avatar"` | `image` | Raw bytes | Base64 |
+
+**Rationale**:
+- Text uses UTF-8 encoding for human-readable data
+- Images use raw bytes to preserve binary data integrity
+- All payloads encoded as Base64 for JSON compatibility
+
+#### Step 4: Envelope Building
+
+NATSBridge builds the message envelope:
+
+```json
+{
+  "correlation_id": "a1b2c3d4...",
+  "msg_id": "e5f6g7h8...",
+  "timestamp": "2026-03-13T16:30:00.000Z",
+  "send_to": "/agent/wine/api/v1/prompt",
+  "msg_purpose": "chat",
+  "sender_name": "chat-webapp",
+  "sender_id": "sender-uuid...",
+  "receiver_name": "agent-backend",
+  "receiver_id": "",
+  "reply_to": "/agent/wine/api/v1/response",
+  "reply_to_msg_id": "",
+  "broker_url": "ws://localhost:4222",
+  "metadata": {},
+  "payloads": [
+    {
+      "id": "payload-uuid...",
+      "dataname": "msg",
+      "payload_type": "text",
+      "transport": "direct",
+      "encoding": "base64",
+      "size": 20,
+      "data": "SGVsbG8hIEknIHRlbCB5b3UgSW4gZW5nbGlzaC4=",
+      "metadata": {"payload_bytes": 20}
+    },
+    {
+      "id": "payload-uuid...",
+      "dataname": "avatar",
+      "payload_type": "image",
+      "transport": "direct",
+      "encoding": "base64",
+      "size": 150000,
+      "data": "iVBORw0KGgoAAAANSUhEUgAA...",
+      "metadata": {"payload_bytes": 150000}
+    }
+  ]
+}
+```
+
+**Rationale**:
+- **correlation_id**: Tracks this chat session across all systems
+- **reply_to**: Tells backend where to send response
+- **payloads array**: Contains all data with metadata for proper handling
+
+#### Step 5: Publish to NATS
+
+```javascript
+await NATSBridge.NATSClient.connect("ws://localhost:4222");
+await NATSBridge.NATSClient.publish("/agent/wine/api/v1/prompt", msgJson);
+```
+
+**Rationale**:
+- NATS provides low-latency message delivery
+- JSON format ensures cross-platform compatibility
+
+#### Step 6: Julia Backend Receives Message
+
+```julia
+# Julia backend
+msg = NATS.subscription.next()  # Get message from NATS
+env = smartreceive(msg)
+
+# env["payloads"] is now:
+# [
+#     ("msg", "Hello! I'm Ton.", "text"),
+#     ("avatar", binary_data, "image")
+# ]
+```
+
+**Rationale**:
+- `smartreceive()` handles both transport types automatically
+- Deserialization is type-aware based on `payload_type`
+- Returns consistent tuple format regardless of transport
+
+#### Step 7: Julia Backend Sends Response
+
+```julia
+# Julia backend processes the message
+response_text = "Hello Ton! I'm the AI assistant."
+generated_image = generate_ai_image(response_text)
+
+env, msg_json = smartsend(
+    "/agent/wine/api/v1/response",
+    [
+        ("response", response_text, "text"),
+        ("generated_image", generated_image, "image")
+    ],
+    reply_to = "/chat/user/v1/message",
+    reply_to_msg_id = msg["msg_id"]
+)
+```
+
+**Rationale**:
+- **Mixed response**: Text explanation + AI-generated image
+- **reply_to**: Ensures response goes to correct topic
+- **reply_to_msg_id**: Links response to original message for tracing
+
+---
+
+## User Scenario 2: Large File Transfer
+
+### Scenario Description
+
+A JavaScript webapp wants to upload a large file (10MB) to a Julia backend for processing.
+
+### Step-by-Step Flow
+
+#### Step 1: JavaScript Webapp Sends Large File
+
+```javascript
+const [env, msgJson] = await NATSBridge.smartsend(
+    "/agent/wine/api/v1/process",
+    [
+        ["file", largeFileData, "binary"]
+    ],
+    {
+        broker_url: "ws://localhost:4222",
+        receiver_name: "agent-backend"
+    }
+);
+```
+
+#### Step 2: Transport Selection (Link)
+
+| Payload | Size | Transport | Reason |
+|---------|------|-----------|--------|
+| `"file"` | 10MB | link | ≥ 0.5MB threshold |
+
+**Rationale**:
+- Link transport used for large payloads
+- File server handles large file upload
+- NATS only sends URL (small message)
+
+#### Step 3: File Server Upload
+
+```javascript
+// NATSBridge internally calls:
+const response = await plikOneshotUpload(
+    "http://localhost:8080",
+    "file",
+    largeFileData
+);
+
+// Response:
+// {
+//   status: 200,
+//   uploadid: "UPLOAD_ID",
+//   fileid: "FILE_ID",
+//   url: "http://localhost:8080/file/UPLOAD_ID/FILE_ID/file"
+// }
+```
+
+**Rationale**:
+- Plik handles multipart upload
+- One-shot mode simplifies API
+- Returns URL for download
+
+#### Step 4: Envelope with Link Transport
+
+```json
+{
+  "correlation_id": "a1b2c3d4...",
+  "payloads": [
+    {
+      "id": "payload-uuid...",
+      "dataname": "file",
+      "payload_type": "binary",
+      "transport": "link",
+      "encoding": "none",
+      "size": 10000000,
+      "data": "http://localhost:8080/file/UPLOAD_ID/FILE_ID/file",
+      "metadata": {}
+    }
+  ]
+}
+```
+
+**Rationale**:
+- `data` field contains URL instead of Base64
+- `transport: "link"` signals URL-based download
+- `encoding: "none"` indicates no additional encoding
+
+#### Step 5: Julia Backend Receives and Downloads
+
+```julia
+# Julia backend
+msg = NATS.subscription.next()
+env = smartreceive(msg)
+
+# NATSBridge automatically:
+# 1. Extracts URL from payload
+# 2. Downloads with exponential backoff
+# 3. Deserializes to binary data
+```
+
+**Rationale**:
+- Exponential backoff handles transient failures
+- Automatic download simplifies receiver code
+- Binary data returned directly
+
+---
+
+## User Scenario 3: Tabular Data Exchange
+
+### Scenario Description
+
+A Python application sends tabular data (pandas DataFrame) to a Julia backend for analysis, and receives processed results back.
+
+### Step-by-Step Flow
+
+#### Step 1: Python Sends Tabular Data
+
+```python
+# Python
+import pandas as pd
+from natsbridge import smartsend
+
+df = pd.DataFrame({
+    "id": [1, 2, 3],
+    "name": ["Alice", "Bob", "Charlie"],
+    "score": [95, 88, 92]
+})
+
+env, msg_json = await smartsend(
+    "/agent/wine/api/v1/analyze",
+    [("data", df, "arrowtable")],
+    broker_url="nats://localhost:4222",
+    receiver_name="agent-backend"
+)
+```
+
+**Rationale**:
+- `arrowtable` type for efficient tabular data transfer
+- Arrow IPC format preserves data types
+- Much faster than JSON serialization
+
+#### Step 2: Serialization to Arrow IPC
+
+```python
+# NATSBridge internally:
+import pyarrow as pa
+import pyarrow.ipc as ipc
+
+table = pa.Table.from_pandas(df)
+buf = io.BytesIO()
+sink = ipc.new_file(buf, table.schema)
+ipc.write_table(table, sink)
+arrow_bytes = buf.getvalue()
+```
+
+**Rationale**:
+- Arrow IPC preserves column types
+- Binary format is compact
+- No schema information loss
+
+#### Step 3: Julia Receives and Deserializes
+
+```julia
+# Julia backend
+msg = NATS.subscription.next()
+env = smartreceive(msg)
+
+# env["payloads"][1] is now:
+# ("data", DataFrame with id, name, score columns, "arrowtable")
+```
+
+**Rationale**:
+- Arrow.jl reads IPC format directly
+- DataFrame returned with correct types
+- No manual parsing needed
+
+#### Step 4: Julia Sends Results
+
+```julia
+# Julia backend
+results = analyze_data(env["payloads"][1][2])
+
+# Send results back
+env, msg_json = smartsend(
+    "/agent/wine/api/v1/results",
+    [("results", results, "arrowtable")],
+    reply_to = "/python/worker/v1/results"
+)
+```
+
+**Rationale**:
+- Arrow IPC format for efficient round-trip
+- Results preserve DataFrame structure
+- Python can deserialize to pandas DataFrame
+
+---
+
+## User Scenario 4: MicroPython Device
+
+### Scenario Description
+
+A MicroPython sensor device sends sensor readings to a Python backend.
+
+### Step-by-Step Flow
+
+#### Step 1: MicroPython Sends Sensor Data
+
+```python
+# MicroPython
+from natsbridge import smartsend
+
+sensor_data = {
+    "temperature": 25.5,
+    "humidity": 60.0,
+    "pressure": 1013.25
+}
+
+env, msg_json = smartsend(
+    "/sensor/device/v1/readings",
+    [("data", sensor_data, "dictionary")],
+    broker_url="nats://localhost:4222",
+    size_threshold=100000  # 100KB for MicroPython
+)
+```
+
+**Rationale**:
+- `dictionary` type for JSON-serializable sensor data
+- Smaller threshold (100KB) for memory constraints
+- Direct transport only (no file server support)
+
+#### Step 2: Serialization
+
+```python
+# NATSBridge internally:
+json_str = json.dumps(sensor_data)
+json_bytes = json_str.encode('utf-8')
+payload_b64 = base64.b64encode(json_bytes).decode('ascii')
+```
+
+**Rationale**:
+- JSON format for human-readable data
+- Base64 for NATS compatibility
+- UTF-8 for text encoding
+
+#### Step 3: Python Backend Receives
+
+```python
+# Python backend
+msg = await nats_consumer.next()
+env = await smartreceive(msg)
+
+# env["payloads"][0] is now:
+# ("data", {"temperature": 25.5, "humidity": 60.0, ...}, "dictionary")
+```
+
+**Rationale**:
+- JSON deserialization
+- Dictionary returned directly
+- No Arrow support (memory constraints)
+
+---
+
+## User Scenario 5: Cross-Platform Chat with Mixed Payloads
+
+### Scenario Description
+
+Multiple platforms (JavaScript, Python, Julia) communicate in a chat application with mixed payload types.
+
+### Step-by-Step Flow
+
+#### Step 1: JavaScript Sends Chat Message
+
+```javascript
+// JavaScript (Frontend)
+const [env, msgJson] = await NATSBridge.smartsend(
+    "/chat/user/v1/message",
+    [
+        ["text", "Check this out!", "text"],
+        ["image", imageData, "image"]
+    ],
+    {
+        broker_url: "ws://localhost:4222",
+        receiver_name: "",
+        msg_purpose: "chat"
+    }
+);
+```
+
+**Rationale**:
+- Empty `receiver_name` = broadcast to all subscribers
+- Chat messages often include text + images
+- NATS wildcard subscriptions route to correct recipients
+
+#### Step 2: Python Backend Receives
+
+```python
+# Python (Backend)
+msg = await nats_consumer.next()
+env = await smartreceive(msg)
+
+# env["payloads"] is now:
+# [
+#     ("text", "Check this out!", "text"),
+#     ("image", binary_data, "image")
+# ]
+```
+
+**Rationale**:
+- Consistent API across platforms
+- Same payload structure regardless of sender
+- Type information preserved
+
+#### Step 3: Julia Backend Receives
+
+```julia
+# Julia (Backend)
+msg = NATS.subscription.next()
+env = smartreceive(msg)
+
+# env["payloads"] is now:
+# [
+#     ("text", "Check this out!", "text"),
+#     ("image", binary_data, "image")
+# ]
+```
+
+**Rationale**:
+- Cross-platform API parity
+- Same function signature across platforms
+- Type information enables proper deserialization
+
+#### Step 4: All Platforms Reply
+
+Each platform can reply using the same API:
+
+```python
+# Python reply
+await smartsend(
+    "/chat/user/v1/reply",
+    [("response", "Nice!", "text")],
+    reply_to="/chat/user/v1/message"
+)
+```
+
+```julia
+# Julia reply
+smartsend(
+    "/chat/user/v1/reply",
+    [("response", "Nice!", "text")],
+    reply_to="/chat/user/v1/message"
+)
+```
+
+```javascript
+// JavaScript reply
+await NATSBridge.smartsend(
+    "/chat/user/v1/reply",
+    [["response", "Nice!", "text"]],
+    { reply_to: "/chat/user/v1/message" }
+);
+```
+
+**Rationale**:
+- Same API across platforms
+- Consistent behavior
+- Easy to maintain parity
+
+---
+
+## Error Handling
+
+### Common Error Scenarios
+
+| Scenario | Error | Recovery |
+|----------|-------|----------|
+| File server unavailable | `UPLOAD_FAILED` | Fall back to direct transport or smaller payloads |
+| File server download fails | `DOWNLOAD_FAILED` | Retry with exponential backoff |
+| Payload type mismatch | `DESERIALIZATION_ERROR` | Validate payload_type matches data |
+| NATS connection lost | `NATS_CONNECTION_FAILED` | NATS client auto-reconnects |
+
+### Error Response Format
+
+```json
+{
+  "correlation_id": "abc123...",
+  "error": {
+    "code": "DOWNLOAD_FAILED",
+    "message": "Failed to fetch data after 5 attempts",
+    "details": {
+      "url": "http://localhost:8080/file/...",
+      "correlation_id": "abc123..."
+    }
+  }
+}
+```
+
+---
+
+## Debugging and Tracing
+
+### Correlation ID Tracking
+
+Every message includes a `correlation_id`:
+
+```julia
+# At start of request
+correlation_id = string(uuid4())
+
+# Use throughout the flow
+log_trace(correlation_id, "Starting smartsend")
+log_trace(correlation_id, "Serialized payload size: 100 bytes")
+log_trace(correlation_id, "Published to NATS")
+```
+
+**Log Format**:
+```
+[2026-03-13T16:30:00.000Z] [Correlation: abc123...] Starting smartsend
+[2026-03-13T16:30:00.001Z] [Correlation: abc123...] Serialized payload size: 100 bytes
+[2026-03-13T16:30:00.002Z] [Correlation: abc123...] Published to NATS
+```
+
+---
+
+## Performance Considerations
+
+### Optimization Strategies
+
+| Strategy | Description | When to Use |
+|----------|-------------|-------------|
+| Pre-create NATS connection | Reuse connection for multiple sends | High-throughput scenarios |
+| Adjust size threshold | Increase threshold if file server slow | File server bottleneck |
+| Use direct transport | Avoid file server for small payloads | Low latency requirements |
+
+### Size Threshold by Platform
+
+| Platform | Threshold | Notes |
+|----------|-----------|-------|
+| Desktop (Julia/JS/Python) | 500,000 bytes (0.5MB) | Default threshold |
+| MicroPython | 100,000 bytes (100KB) | Lower threshold for memory constraints |
+
+---
+
+## Deployment Considerations
+
+### Minimum Infrastructure
+
+| Component | Minimum | Notes |
+|-----------|---------|-------|
+| NATS Server | 1 instance | Single node for development |
+| File Server | 1 instance | HTTP server for large payloads |
+| Client Memory | 50MB | Desktop platforms |
+| Client Memory | 256KB | MicroPython devices |
+
+### Environment Variables
+
+| Variable | Default | Description |
+|----------|---------|-------------|
+| `NATS_URL` | `nats://localhost:4222` | NATS server URL |
+| `FILESERVER_URL` | `http://localhost:8080` | HTTP file server URL |
+| `SIZE_THRESHOLD` | `1000000` | Size threshold in bytes |
+
+---
+
+## Change Log
+
+| Date | Version | Changes |
+|------|---------|---------|
+| 2026-03-13 | 1.0.0 | Initial walkthrough documentation |
+
+---
+
+## References
+
+- [`docs/requirements.md`](./requirements.md) - Business requirements and user stories
+- [`docs/spec.md`](./spec.md) - Technical specification and contracts
+- [`docs/architecture.md`](./architecture.md) - System architecture diagrams
+- [`src/NATSBridge.jl`](../src/NATSBridge.jl) - Ground truth implementation
+- [`README.md`](../README.md) - Project overview
+
+---
+
+*This walkthrough document is versioned and maintained in git alongside the codebase. All implementations must adhere to this documentation.*
+
+<tool_call>
+<function=update_todo_list>
+<parameter=todos>
+[x] Analyze existing documentation (requirements.md, spec.md, architecture.md)
+[x] Read all source files in src/ folder
+[x] Write docs/walkthrough.md according to SDD framework with user scenarios

etc.jl

@@ -1,42 +0,0 @@
-
-""" fileServerURL = "http://192.168.88.104:8080"
-filepath = "/home/ton/docker-apps/sendreceive/image/test.zip"
-filename = basename(filepath)
-filebytes = read(filepath)
-
-plik_oneshot_upload - Upload a single file to a plik server using one-shot mode
-
-This function uploads a raw byte array to a plik server in one-shot mode (no upload session).
-It first creates a one-shot upload session by sending a POST request with `{"OneShot": true}`,
-retrieves an upload ID and token, then uploads the file data as multipart form data using the token.
-
-The function handles the entire flow:
-1. Obtains an upload ID and token from the server
-2. Uploads the provided binary data as a file using the `X-UploadToken` header
-3. Returns identifiers and download URL for the uploaded file
-
-# Arguments:
-  - `fileServerURL::String` - Base URL of the plik server (e.g., `"http://192.168.88.104:8080"`)
-  - `filename::String` - Name of the file being uploaded
-  - `data::Vector{UInt8}` - Raw byte data of the file content
-
-# Return:
-  - A named tuple with fields:
-    - `uploadid::String` - ID of the one-shot upload session
-    - `fileid::String` - ID of the uploaded file within the session
-    - `downloadurl::String` - Full URL to download the uploaded file
-
-# Example
-```jldoctest
-using HTTP, JSON
-
-# Example data: "Hello" as bytes
-data = collect("Hello World!" |> collect |> CodeUnits |> collect)
-
-# Upload to local plik server
-result = plik_oneshot_upload("http://192.168.88.104:8080", "hello.txt", data)
-
-# Download URL for the uploaded file
-println(result.downloadurl)
-```
-"""

etc.txt

@@ -0,0 +1,310 @@
+#!/usr/bin/env julia
+# Test script for mixed-content message testing
+# Tests receiving a mix of text, json, table, image, audio, video, and binary data
+# from Julia serviceA to Julia serviceB using NATSBridge.jl smartreceive
+#
+# This test demonstrates that any combination and any number of mixed content
+# can be sent and received correctly.
+
+using NATS, JSON, UUIDs, Dates, PrettyPrinting, DataFrames, Arrow, HTTP, Base64
+
+# Include the bridge module
+include("./src/NATSBridge.jl")
+using .NATSBridge
+
+# Configuration
+const SUBJECT = "/test/mix"
+const NATS_URL = "nats.yiem.cc"
+const FILESERVER_URL = "http://192.168.88.104:8080"
+
+
+# ------------------------------------------------------------------------------------------------ #
+#                                      test mixed content transfer                                 #
+# ------------------------------------------------------------------------------------------------ #
+
+
+# Helper: Log with correlation ID
+function log_trace(message)
+    timestamp = Dates.now()
+    println("[$timestamp] $message")
+end
+
+
+# Receiver: Listen for messages and verify mixed content handling
+function test_mix_receive()
+    conn = NATS.connect(NATS_URL)
+    incoming_msg = nothing
+    NATS.subscribe(conn, SUBJECT) do msg
+        log_trace("Received message on $(msg.subject)")
+        incoming_msg = msg
+        
+        # # Use NATSBridge.smartreceive to handle the data
+        # # API: smartreceive(msg, download_handler; max_retries, base_delay, max_delay)
+        # result = NATSBridge.smartreceive(
+        #     msg;
+        #     max_retries = 5,
+        #     base_delay = 100,
+        #     max_delay = 5000
+        # )
+        
+        # log_trace("Received $(length(result["payloads"])) payloads")
+        
+        
+        # # Result is an envelope dictionary with payloads field containing list of (dataname, data, data_type) tuples
+        # for (dataname, data, data_type) in result["payloads"]
+        #     log_trace("\n=== Payload: $dataname (type: $data_type) ===")
+            
+        #     # Handle different data types
+        #     if data_type == "text"
+        #         # Text data - should be a String
+        #         if isa(data, String)
+        #             log_trace("  Type: String")
+        #             log_trace("  Length: $(length(data)) characters")
+                    
+        #             # Display first 200 characters
+        #             if length(data) > 200
+        #                 log_trace("  First 200 chars: $(data[1:200])...")
+        #             else
+        #                 log_trace("  Content: $data")
+        #             end
+                    
+        #             # Save to file
+        #             output_path = "./received_$dataname.txt"
+        #             write(output_path, data)
+        #             log_trace("  Saved to: $output_path")
+        #         else
+        #             log_trace("  ERROR: Expected String, got $(typeof(data))")
+        #         end
+                
+        #     elseif data_type == "dictionary"
+        #         # Dictionary data - should be JSON object
+        #         if isa(data, JSON.Object{String, Any})
+        #             log_trace("  Type: Dict")
+        #             log_trace("  Keys: $(keys(data))")
+                    
+        #             # Display nested content
+        #             for (key, value) in data
+        #                 log_trace("  $key => $value")
+        #             end
+                    
+        #             # Save to JSON file
+        #             output_path = "./received_$dataname.json"
+        #             json_str = JSON.json(data, 2)
+        #             write(output_path, json_str)
+        #             log_trace("  Saved to: $output_path")
+        #         else
+        #             log_trace("  ERROR: Expected Dict, got $(typeof(data))")
+        #         end
+                
+        #     elseif data_type == "table"
+        #         # Table data - should be a DataFrame
+        #         tabledata = deepcopy(data)
+        #         println("found table data")
+        #         break
+        #         # return data
+        #         # if isa(data, DataFrame)
+        #         #     log_trace("  Type: DataFrame")
+        #         #     log_trace("  Dimensions: $(size(data, 1)) rows x $(size(data, 2)) columns")
+        #         #     log_trace("  Columns: $(names(data))")
+                    
+        #         #     # Display first few rows
+        #         #     log_trace("  First 5 rows:")
+        #         #     display(data[1:min(5, size(data, 1)), :])
+                    
+        #         #     # Save to Arrow file
+        #         #     output_path = "./received_$dataname.arrow"
+        #         #     io = IOBuffer()
+        #         #     Arrow.write(io, data)
+        #         #     write(output_path, take!(io))
+        #         #     log_trace("  Saved to: $output_path")
+        #         # else
+        #         #     log_trace("  ERROR: Expected DataFrame, got $(typeof(data))")
+        #         # end
+                
+        #     elseif data_type == "image"
+        #         # Image data - should be Vector{UInt8}
+        #         if isa(data, Vector{UInt8})
+        #             log_trace("  Type: Vector{UInt8} (binary)")
+        #             log_trace("  Size: $(length(data)) bytes")
+                    
+        #             # Save to file
+        #             output_path = "./received_$dataname.bin"
+        #             write(output_path, data)
+        #             log_trace("  Saved to: $output_path")
+        #         else
+        #             log_trace("  ERROR: Expected Vector{UInt8}, got $(typeof(data))")
+        #         end
+                
+        #     elseif data_type == "audio"
+        #         # Audio data - should be Vector{UInt8}
+        #         if isa(data, Vector{UInt8})
+        #             log_trace("  Type: Vector{UInt8} (binary)")
+        #             log_trace("  Size: $(length(data)) bytes")
+                    
+        #             # Save to file
+        #             output_path = "./received_$dataname.bin"
+        #             write(output_path, data)
+        #             log_trace("  Saved to: $output_path")
+        #         else
+        #             log_trace("  ERROR: Expected Vector{UInt8}, got $(typeof(data))")
+        #         end
+                
+        #     elseif data_type == "video"
+        #         # Video data - should be Vector{UInt8}
+        #         if isa(data, Vector{UInt8})
+        #             log_trace("  Type: Vector{UInt8} (binary)")
+        #             log_trace("  Size: $(length(data)) bytes")
+                    
+        #             # Save to file
+        #             output_path = "./received_$dataname.bin"
+        #             write(output_path, data)
+        #             log_trace("  Saved to: $output_path")
+        #         else
+        #             log_trace("  ERROR: Expected Vector{UInt8}, got $(typeof(data))")
+        #         end
+                
+        #     elseif data_type == "binary"
+        #         # Binary data - should be Vector{UInt8}
+        #         if isa(data, Vector{UInt8})
+        #             log_trace("  Type: Vector{UInt8} (binary)")
+        #             log_trace("  Size: $(length(data)) bytes")
+                    
+        #             # Save to file
+        #             output_path = "./received_$dataname.bin"
+        #             write(output_path, data)
+        #             log_trace("  Saved to: $output_path")
+        #         else
+        #             log_trace("  ERROR: Expected Vector{UInt8}, got $(typeof(data))")
+        #         end
+                
+        #     else
+        #         log_trace("  ERROR: Unknown data type '$data_type'")
+        #     end
+        # end
+        
+        # Summary
+        # println("\n=== Verification Summary ===")
+        # text_count = count(x -> x[3] == "text", result["payloads"])
+        # dict_count = count(x -> x[3] == "dictionary", result["payloads"])
+        # table_count = count(x -> x[3] == "table", result["payloads"])
+        # image_count = count(x -> x[3] == "image", result["payloads"])
+        # audio_count = count(x -> x[3] == "audio", result["payloads"])
+        # video_count = count(x -> x[3] == "video", result["payloads"])
+        # binary_count = count(x -> x[3] == "binary", result["payloads"])
+        
+        # log_trace("Text payloads: $text_count")
+        # log_trace("Dictionary payloads: $dict_count")
+        # log_trace("Table payloads: $table_count")
+        # log_trace("Image payloads: $image_count")
+        # log_trace("Audio payloads: $audio_count")
+        # log_trace("Video payloads: $video_count")
+        # log_trace("Binary payloads: $binary_count")
+        
+        # # Print transport type info for each payload if available
+        # println("\n=== Payload Details ===")
+        # for (dataname, data, data_type) in result["payloads"]
+        #     if data_type in ["image", "audio", "video", "binary"]
+        #         log_trace("$dataname: $(length(data)) bytes (binary)")
+        #     elseif data_type == "table"
+        #       data = DataFrame(data)
+        #         log_trace("$dataname: $(size(data, 1)) rows x $(size(data, 2)) columns (DataFrame)")
+        #     elseif data_type == "dictionary"
+        #         log_trace("$dataname: $(length(JSON.json(data))) bytes (Dict)")
+        #     elseif data_type == "text"
+        #         log_trace("$dataname: $(length(data)) characters (String)")
+        #     end
+        # end
+    end
+    
+    # Keep listening for 2 minutes
+    sleep(20)  
+    NATS.drain(conn)
+    return incoming_msg
+end
+
+
+# Run the test
+println("Starting mixed-content transport test...")
+println("Note: This receiver will wait for messages from the sender.")
+println("Run test_julia_to_julia_mix_sender.jl first to send test data.")
+
+# Run receiver
+println("\ntesting smartreceive for mixed content")
+incoming_msg = test_mix_receive()
+
+println("\nTest completed.")
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+Check architecture.md. For sending table I want to add JSON in addition to Apache Arrow. 
+Currently I use "table" datatype when sending table data using Arrow. Now table that I want to send using JSON 
+I will use "jsontable" as datatype while sending table using Arrow I will use "arrowtable" as datatype. 
+This will select how smartsend and smartreceive serialize/deserialize the table.
+
+Can you help me do this? Save the updated architecture.md into updated_architecture.md file. I will deal with source code later.
+
+
+
+
+Now update implementation.md and save into updated_implementation.md
+Keep in mind that Julia DataFrame and Python Pandas rely on columnar-oriented dictionary to create as the following example:
+julia> dict = Dict("customer age" => [15, 20, 25],
+                   "first name" => ["Rohit", "Rahul", "Akshat"])
+julia> DataFrame(dict)
+
+python> data = {
+    "Name": ["Alice", "Bob", "Charlie"],
+    "Age": [25, 30, 35],
+    "Score": [88.5, 92.0, 79.5]
+}
+
+python> df = pd.DataFrame(data)
+
+
+But JS use Array of Objects while MicroPython use list of lists. Both are row-oriented structure.
+So use row-oriented JSON to send across these languages. For Julia and Python, only convert 
+row-oriented JSON to columnar-oriented dictionary for "going-into" and vise versa for "coming-out" 
+a dataframe while JS and MicroPython won't require such process. 
+You may add these info into architecture.md if you see fit.
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+

plik_fileserver/docker-compose.yml

@@ -0,0 +1,14 @@
+services:
+  plik:
+    image: rootgg/plik:latest
+    container_name: plik-server
+    restart: unless-stopped
+    ports:
+      - "8080:8080"
+    volumes:
+      # # Mount the config file (created below)
+      # - ./plikd.cfg:/home/plik/server/plikd.cfg
+      # Mount local folder for uploads and database
+      - ./plik-data:/data
+    # Set user to match your host UID to avoid permission issues
+    user: "1000:1000"

src/NATSBridge.js

@@ -1,245 +0,0 @@
-/**
- * Bi-Directional Data Bridge - JavaScript Module
- * Implements SmartSend and SmartReceive for NATS communication
- */
-
-const { v4: uuidv4 } = require('uuid');
-const { decode, encode } = require('base64-url');
-const Arrow = require('apache-arrow');
-
-// Constants
-const DEFAULT_SIZE_THRESHOLD = 1_000_000; // 1MB
-const DEFAULT_NATS_URL = 'nats://localhost:4222';
-const DEFAULT_FILESERVER_URL = 'http://localhost:8080/upload';
-
-// Logging helper
-function logTrace(correlationId, message) {
-    const timestamp = new Date().toISOString();
-    console.log(`[${timestamp}] [Correlation: ${correlationId}] ${message}`);
-}
-
-// Message Envelope Class
-class MessageEnvelope {
-    constructor(options = {}) {
-        this.correlation_id = options.correlation_id || uuidv4();
-        this.type = options.type || 'json';
-        this.transport = options.transport || 'direct';
-        this.payload = options.payload || null;
-        this.url = options.url || null;
-        this.metadata = options.metadata || {};
-    }
-
-    static fromJSON(jsonStr) {
-        const data = JSON.parse(jsonStr);
-        return new MessageEnvelope({
-            correlation_id: data.correlation_id,
-            type: data.type,
-            transport: data.transport,
-            payload: data.payload || null,
-            url: data.url || null,
-            metadata: data.metadata || {}
-        });
-    }
-
-    toJSON() {
-        const obj = {
-            correlation_id: this.correlation_id,
-            type: this.type,
-            transport: this.transport
-        };
-
-        if (this.payload) {
-            obj.payload = this.payload;
-        }
-
-        if (this.url) {
-            obj.url = this.url;
-        }
-
-        if (Object.keys(this.metadata).length > 0) {
-            obj.metadata = this.metadata;
-        }
-
-        return JSON.stringify(obj);
-    }
-}
-
-// SmartSend for JavaScript - Handles transport selection based on payload size
-async function SmartSend(subject, data, type = 'json', options = {}) {
-    const {
-        natsUrl = DEFAULT_NATS_URL,
-        fileserverUrl = DEFAULT_FILESERVER_URL,
-        sizeThreshold = DEFAULT_SIZE_THRESHOLD,
-        correlationId = uuidv4()
-    } = options;
-
-    logTrace(correlationId, `Starting SmartSend for subject: ${subject}`);
-
-    // Serialize data based on type
-    const payloadBytes = _serializeData(data, type, correlationId);
-    const payloadSize = payloadBytes.length;
-
-    logTrace(correlationId, `Serialized payload size: ${payloadSize} bytes`);
-
-    // Decision: Direct vs Link
-    if (payloadSize < sizeThreshold) {
-        // Direct path - Base64 encode and send via NATS
-        const payloadBase64 = encode(payloadBytes);
-        logTrace(correlationId, `Using direct transport for ${payloadSize} bytes`);
-
-        const env = new MessageEnvelope({
-            correlation_id: correlationId,
-            type: type,
-            transport: 'direct',
-            payload: payloadBase64,
-            metadata: {
-                content_length: payloadSize.toString(),
-                format: 'arrow_ipc_stream'
-            }
-        });
-
-        await publishMessage(natsUrl, subject, env.toJSON(), correlationId);
-        return env;
-    } else {
-        // Link path - Upload to HTTP server, send URL via NATS
-        logTrace(correlationId, `Using link transport, uploading to fileserver`);
-
-        const url = await uploadToServer(payloadBytes, fileserverUrl, correlationId);
-
-        const env = new MessageEnvelope({
-            correlation_id: correlationId,
-            type: type,
-            transport: 'link',
-            url: url,
-            metadata: {
-                content_length: payloadSize.toString(),
-                format: 'arrow_ipc_stream'
-            }
-        });
-
-        await publishMessage(natsUrl, subject, env.toJSON(), correlationId);
-        return env;
-    }
-}
-
-// Helper: Serialize data based on type
-function _serializeData(data, type, correlationId) {
-    if (type === 'json') {
-        const jsonStr = JSON.stringify(data);
-        return Buffer.from(jsonStr, 'utf8');
-    } else if (type === 'table') {
-        // Table data - convert to Arrow IPC stream
-        const writer = new Arrow.Writer();
-        writer.writeTable(data);
-        return writer.toByteArray();
-    } else if (type === 'binary') {
-        // Binary data - treat as binary
-        if (data instanceof Buffer) {
-            return data;
-        } else if (Array.isArray(data)) {
-            return Buffer.from(data);
-        } else {
-            throw new Error('Binary data must be binary (Buffer or Array)');
-        }
-    } else {
-        throw new Error(`Unknown type: ${type}`);
-    }
-}
-
-// Helper: Publish message to NATS
-async function publishMessage(natsUrl, subject, message, correlationId) {
-    const { connect } = require('nats');
-    
-    try {
-        const nc = await connect({ servers: [natsUrl] });
-        await nc.publish(subject, message);
-        logTrace(correlationId, `Message published to ${subject}`);
-        nc.close();
-    } catch (error) {
-        logTrace(correlationId, `Failed to publish message: ${error.message}`);
-        throw error;
-    }
-}
-
-// SmartReceive for JavaScript - Handles both direct and link transport
-async function SmartReceive(msg, options = {}) {
-    const {
-        fileserverUrl = DEFAULT_FILESERVER_URL,
-        maxRetries = 5,
-        baseDelay = 100,
-        maxDelay = 5000
-    } = options;
-
-    const env = MessageEnvelope.fromJSON(msg.data);
-
-    logTrace(env.correlation_id, `Processing received message`);
-
-    if (env.transport === 'direct') {
-        logTrace(env.correlation_id, `Direct transport - decoding payload`);
-
-        const payloadBytes = decode(env.payload);
-        const data = _deserializeData(payloadBytes, env.type, env.correlation_id, env.metadata);
-
-        return { data, envelope: env };
-    } else if (env.transport === 'link') {
-        logTrace(env.correlation_id, `Link transport - fetching from URL`);
-
-        const data = await _fetchWithBackoff(env.url, maxRetries, baseDelay, maxDelay, env.correlation_id);
-        const result = _deserializeData(data, env.type, env.correlation_id, env.metadata);
-
-        return { data: result, envelope: env };
-    } else {
-        throw new Error(`Unknown transport type: ${env.transport}`);
-    }
-}
-
-// Helper: Fetch with exponential backoff
-async function _fetchWithBackoff(url, maxRetries, baseDelay, maxDelay, correlationId) {
-    let delay = baseDelay;
-
-    for (let attempt = 1; attempt <= maxRetries; attempt++) {
-        try {
-            const response = await fetch(url);
-            if (response.ok) {
-                const buffer = await response.arrayBuffer();
-                logTrace(correlationId, `Successfully fetched data from ${url} on attempt ${attempt}`);
-                return new Uint8Array(buffer);
-            } else {
-                throw new Error(`Failed to fetch: ${response.status}`);
-            }
-        } catch (error) {
-            logTrace(correlationId, `Attempt ${attempt} failed: ${error.message}`);
-
-            if (attempt < maxRetries) {
-                await new Promise(resolve => setTimeout(resolve, delay));
-                delay = Math.min(delay * 2, maxDelay);
-            }
-        }
-    }
-
-    throw new Error(`Failed to fetch data after ${maxRetries} attempts`);
-}
-
-// Helper: Deserialize data based on type
-async function _deserializeData(data, type, correlationId, metadata) {
-    if (type === 'json') {
-        const jsonStr = new TextDecoder().decode(data);
-        return JSON.parse(jsonStr);
-    } else if (type === 'table') {
-        // Deserialize Arrow IPC stream to Table
-        const table = Arrow.Table.from(data);
-        return table;
-    } else if (type === 'binary') {
-        // Return binary binary data
-        return data;
-    } else {
-        throw new Error(`Unknown type: ${type}`);
-    }
-}
-
-// Export functions
-module.exports = {
-    SmartSend,
-    SmartReceive,
-    MessageEnvelope
-};

src/natsbridge.py

@@ -0,0 +1,843 @@
+"""
+NATSBridge - Cross-Platform Bi-Directional Data Bridge
+Python Desktop Implementation
+
+This module provides functionality for sending and receiving data across network boundaries
+using NATS as the message bus, with support for both direct payload transport and
+URL-based transport for larger payloads.
+
+@package natsbridge
+"""
+
+import asyncio
+import base64
+import json
+import uuid
+from datetime import datetime
+from typing import Any, Callable, Dict, List, Tuple, Union
+import aiohttp
+
+try:
+    import pyarrow as arrow
+    import pyarrow.ipc as ipc
+    ARROW_AVAILABLE = True
+except ImportError:
+    ARROW_AVAILABLE = False
+
+try:
+    import nats
+    from nats.aio.client import Client as NATSClient
+    NATS_AVAILABLE = True
+except ImportError:
+    NATS_AVAILABLE = False
+
+# ---------------------------------------------- Constants ---------------------------------------------- #
+
+"""
+Default size threshold for switching from direct to link transport (0.5MB)
+"""
+DEFAULT_SIZE_THRESHOLD = 500_000
+
+"""
+Default NATS server URL
+"""
+DEFAULT_BROKER_URL = "nats://localhost:4222"
+
+"""
+Default HTTP file server URL for link transport
+"""
+DEFAULT_FILESERVER_URL = "http://localhost:8080"
+
+
+# ---------------------------------------------- Utility Functions ---------------------------------------------- #
+
+def log_trace(correlation_id: str, message: str) -> None:
+    """
+    Log a trace message with correlation ID and timestamp.
+    
+    Args:
+        correlation_id: Correlation ID for tracing
+        message: Message content to log
+    """
+    timestamp = datetime.utcnow().isoformat() + 'Z'
+    print(f"[{timestamp}] [Correlation: {correlation_id}] {message}")
+
+
+# ---------------------------------------------- Serialization Functions ---------------------------------------------- #
+
+def _serialize_data(data: Any, payload_type: str) -> bytes:
+    """
+    Serialize data according to specified format.
+    
+    Args:
+        data: Data to serialize (string for "text", JSON-serializable for "dictionary",
+              table-like for "arrowtable"/"jsontable", binary for "image", "audio", "video", "binary")
+        payload_type: Target format: "text", "dictionary", "arrowtable", "jsontable",
+                      "image", "audio", "video", "binary"
+    
+    Returns:
+        Binary representation of the serialized data
+    
+    Raises:
+        Error: If payload_type is not one of the supported types
+        Error: If payload_type is "image", "audio", or "video" but data is not bytes
+        Error: If payload_type is "arrowtable" but data is not a pandas DataFrame or pyarrow Table
+        Error: If payload_type is "jsontable" but data is not a list of dicts
+    """
+    if payload_type == 'text':
+        if isinstance(data, str):
+            return data.encode('utf-8')
+        else:
+            raise ValueError('Text data must be a string')
+    elif payload_type == 'dictionary':
+        json_str = json.dumps(data)
+        return json_str.encode('utf-8')
+    elif payload_type == 'arrowtable':
+        if not ARROW_AVAILABLE:
+            raise RuntimeError('pyarrow not available for arrowtable serialization')
+        
+        import io
+        buf = io.BytesIO()
+        
+        import pandas as pd
+        if isinstance(data, pd.DataFrame):
+            table = arrow.Table.from_pandas(data)
+            sink = ipc.new_file(buf, table.schema)
+            ipc.write_table(table, sink)
+            sink.close()
+            return buf.getvalue()
+        elif isinstance(data, arrow.Table):
+            sink = ipc.new_file(buf, data.schema)
+            ipc.write_table(data, sink)
+            sink.close()
+            return buf.getvalue()
+        else:
+            raise ValueError('Arrow table data must be a pandas DataFrame or pyarrow Table')
+    elif payload_type == 'jsontable':
+        # Serialize list of dicts to JSON format
+        if isinstance(data, list) and all(isinstance(row, dict) for row in data):
+            json_str = json.dumps(data)
+            return json_str.encode('utf-8')
+        else:
+            raise ValueError('JSON table data must be a list of dicts')
+    elif payload_type == 'image':
+        if isinstance(data, (bytes, bytearray)):
+            return bytes(data)
+        else:
+            raise ValueError('Image data must be bytes')
+    elif payload_type == 'audio':
+        if isinstance(data, (bytes, bytearray)):
+            return bytes(data)
+        else:
+            raise ValueError('Audio data must be bytes')
+    elif payload_type == 'video':
+        if isinstance(data, (bytes, bytearray)):
+            return bytes(data)
+        else:
+            raise ValueError('Video data must be bytes')
+    elif payload_type == 'binary':
+        if isinstance(data, (bytes, bytearray)):
+            return bytes(data)
+        else:
+            raise ValueError('Binary data must be bytes')
+    else:
+        raise ValueError(f'Unknown payload_type: {payload_type}')
+
+
+def _deserialize_data(data: bytes, payload_type: str, correlation_id: str) -> Any:
+    """
+    Deserialize bytes to data based on type.
+    
+    Args:
+        data: Serialized data as bytes
+        payload_type: Data type ("text", "dictionary", "arrowtable", "jsontable",
+                      "image", "audio", "video", "binary")
+        correlation_id: Correlation ID for logging
+    
+    Returns:
+        Deserialized data (String for "text", DataFrame for "arrowtable",
+        Vector{Dict} for "jsontable"/"dictionary", bytes for "image", "audio", "video", "binary")
+    
+    Raises:
+        Error: If payload_type is not one of the supported types
+    """
+    if payload_type == 'text':
+        return data.decode('utf-8')
+    elif payload_type == 'dictionary':
+        json_str = data.decode('utf-8')
+        return json.loads(json_str)
+    elif payload_type == 'arrowtable':
+        if not ARROW_AVAILABLE:
+            raise RuntimeError('pyarrow not available for arrowtable deserialization')
+        
+        import io
+        buf = io.BytesIO(data)
+        reader = ipc.open_file(buf)
+        return reader.read_all().to_pandas()
+    elif payload_type == 'jsontable':
+        # Deserialize JSON to list of dicts
+        json_str = data.decode('utf-8')
+        return json.loads(json_str)
+    elif payload_type == 'image':
+        return data
+    elif payload_type == 'audio':
+        return data
+    elif payload_type == 'video':
+        return data
+    elif payload_type == 'binary':
+        return data
+    else:
+        raise ValueError(f'Unknown payload_type: {payload_type}')
+
+
+# ---------------------------------------------- File Server Handlers ---------------------------------------------- #
+
+async def plik_oneshot_upload(
+    file_server_url: str,
+    dataname: str,
+    data: bytes
+) -> Dict[str, Any]:
+    """
+    Upload data to plik server in one-shot mode.
+    
+    This function uploads a raw byte array to a plik server in one-shot mode (no upload session).
+    It first creates a one-shot upload session by sending a POST request with {"OneShot": true},
+    retrieves an upload ID and token, then uploads the file data as multipart form data using the token.
+    
+    Args:
+        file_server_url: Base URL of the plik server (e.g., "http://localhost:8080")
+        dataname: Name of the file being uploaded
+        data: Raw byte data of the file content
+    
+    Returns:
+        Dict with keys:
+            - "status": HTTP server response status
+            - "uploadid": ID of the one-shot upload session
+            - "fileid": ID of the uploaded file within the session
+            - "url": Full URL to download the uploaded file
+    
+    Example:
+        >>> fileserver_url = "http://localhost:8080"
+        >>> dataname = "test.txt"
+        >>> data = b"hello world"
+        >>> result = await plik_oneshot_upload(file_server_url, dataname, data)
+        >>> result["status"], result["uploadid"], result["fileid"], result["url"]
+    """
+    async with aiohttp.ClientSession() as session:
+        # Get upload id
+        url_getUploadID = f"{file_server_url}/upload"
+        headers = {'Content-Type': 'application/json'}
+        body = json.dumps({"OneShot": True})
+        
+        async with session.post(url_getUploadID, headers=headers, data=body) as response:
+            response_json = await response.json()
+            uploadid = response_json['id']
+            uploadtoken = response_json['uploadToken']
+
+        # Upload file
+        url_upload = f"{file_server_url}/file/{uploadid}"
+        headers = {'X-UploadToken': uploadtoken}
+        
+        form = aiohttp.FormData()
+        form.add_field('file', data, filename=dataname, content_type='application/octet-stream')
+        
+        async with session.post(url_upload, headers=headers, data=form) as upload_response:
+            upload_json = await upload_response.json()
+            fileid = upload_json['id']
+
+    url = f"{file_server_url}/file/{uploadid}/{fileid}/{dataname}"
+
+    return {
+        'status': upload_response.status,
+        'uploadid': uploadid,
+        'fileid': fileid,
+        'url': url
+    }
+
+
+async def fetch_with_backoff(
+    url: str,
+    max_retries: int,
+    base_delay: int,
+    max_delay: int,
+    correlation_id: str
+) -> bytes:
+    """
+    Fetch data from URL with exponential backoff.
+    
+    This internal function retrieves data from a URL with retry logic using
+    exponential backoff to handle transient failures.
+    
+    Args:
+        url: URL to fetch from
+        max_retries: Maximum number of retry attempts
+        base_delay: Initial delay in milliseconds
+        max_delay: Maximum delay in milliseconds
+        correlation_id: Correlation ID for logging
+    
+    Returns:
+        Fetched data as bytes
+    
+    Raises:
+        Error: If all retry attempts fail
+    
+    Example:
+        >>> data = await fetch_with_backoff("http://example.com/file.zip", 5, 100, 5000, "correlation123")
+    """
+    delay = base_delay
+    
+    for attempt in range(1, max_retries + 1):
+        try:
+            async with aiohttp.ClientSession() as session:
+                async with session.get(url) as response:
+                    if response.status == 200:
+                        log_trace(correlation_id, f"Successfully fetched data from {url} on attempt {attempt}")
+                        return await response.read()
+                    else:
+                        raise Exception(f"Failed to fetch: {response.status}")
+        except Exception as e:
+            log_trace(correlation_id, f"Attempt {attempt} failed: {type(e).__name__}")
+            
+            if attempt < max_retries:
+                await asyncio.sleep(delay / 1000.0)
+                delay = min(delay * 2, max_delay)
+    
+    raise Exception(f"Failed to fetch data after {max_retries} attempts")
+
+
+# ---------------------------------------------- NATS Client ---------------------------------------------- #
+
+class NATSClient:
+    """NATS client wrapper for connection management."""
+    
+    def __init__(self, url: str = DEFAULT_BROKER_URL):
+        """
+        Create a new NATS client.
+        
+        Args:
+            url: NATS server URL
+        """
+        self.url = url
+        self._client: NATSClient = None
+    
+    async def connect(self) -> NATSClient:
+        """
+        Connect to NATS server.
+        
+        Returns:
+            NATS client instance
+        """
+        if NATS_AVAILABLE:
+            self._client = nats.connect(self.url)
+            await self._client
+        else:
+            raise RuntimeError('nats-py not available')
+        return self._client
+    
+    async def publish(self, subject: str, message: str, correlation_id: str = "") -> None:
+        """
+        Publish message to NATS subject.
+        
+        Args:
+            subject: NATS subject to publish to
+            message: Message to publish
+            correlation_id: Correlation ID for logging
+        """
+        if self._client:
+            await self._client.publish(subject, message)
+            if correlation_id:
+                log_trace(correlation_id, f"Message published to {subject}")
+    
+    async def close(self) -> None:
+        """Close the NATS connection."""
+        if self._client:
+            await self._client.drain()
+            await self._client.close()
+
+
+# ---------------------------------------------- Core Functions ---------------------------------------------- #
+
+def _build_envelope(
+    subject: str,
+    payloads: List[Dict[str, Any]],
+    options: Dict[str, Any]
+) -> Dict[str, Any]:
+    """
+    Build message envelope from payloads and metadata.
+    
+    Args:
+        subject: NATS subject
+        payloads: Array of payload objects
+        options: Envelope metadata options
+    
+    Returns:
+        Envelope object
+    """
+    return {
+        'correlation_id': options['correlation_id'],
+        'msg_id': options['msg_id'],
+        'timestamp': datetime.utcnow().isoformat() + 'Z',
+        'send_to': subject,
+        'msg_purpose': options['msg_purpose'],
+        'sender_name': options['sender_name'],
+        'sender_id': options['sender_id'],
+        'receiver_name': options['receiver_name'],
+        'receiver_id': options['receiver_id'],
+        'reply_to': options['reply_to'],
+        'reply_to_msg_id': options['reply_to_msg_id'],
+        'broker_url': options['broker_url'],
+        'metadata': options.get('metadata', {}),
+        'payloads': payloads
+    }
+
+
+def _build_payload(
+    dataname: str,
+    payload_type: str,
+    payload_bytes: bytes,
+    transport: str,
+    data: Union[str, bytes]
+) -> Dict[str, Any]:
+    """
+    Build payload object from serialized data.
+    
+    Args:
+        dataname: Name of the payload
+        payload_type: Type of the payload
+        payload_bytes: Serialized payload bytes
+        transport: Transport type ("direct" or "link")
+        data: Data (base64 for direct, URL for link)
+    
+    Returns:
+        Payload object
+    """
+    # Determine encoding based on payload type (matching Julia/JS implementation)
+    encoding = 'base64'
+    if payload_type == 'jsontable':
+        encoding = 'json'
+    elif payload_type == 'arrowtable':
+        encoding = 'arrow-ipc'
+    
+    return {
+        'id': str(uuid.uuid4()),
+        'dataname': dataname,
+        'payload_type': payload_type,
+        'transport': transport,
+        'encoding': encoding,
+        'size': len(payload_bytes),
+        'data': data,
+        'metadata': {'payload_bytes': len(payload_bytes)} if transport == 'direct' else {}
+    }
+
+
+async def publish_message(
+    broker_url_or_client: Union[str, NATSClient, Any],
+    subject: str,
+    message: str,
+    correlation_id: str
+) -> None:
+    """
+    Publish message to NATS.
+    
+    Args:
+        broker_url_or_client: NATS URL, client, or connection
+        subject: NATS subject to publish to
+        message: JSON message to publish
+        correlation_id: Correlation ID for tracing
+    """
+    if isinstance(broker_url_or_client, NATSClient):
+        client = broker_url_or_client
+    elif NATS_AVAILABLE and hasattr(broker_url_or_client, 'publish'):
+        # Direct NATS client connection
+        await broker_url_or_client.publish(subject, message)
+        log_trace(correlation_id, f"Message published to {subject}")
+        return
+    else:
+        # String URL - create new client
+        client = NATSClient(broker_url_or_client)
+        await client.connect()
+    
+    await client.publish(subject, message, correlation_id)
+    
+    if isinstance(broker_url_or_client, NATSClient):
+        await broker_url_or_client.close()
+    elif not (NATS_AVAILABLE and hasattr(broker_url_or_client, 'publish')):
+        await client.close()
+
+
+async def smartsend(
+    subject: str,
+    data: List[Tuple[str, Any, str]],
+    broker_url: str = DEFAULT_BROKER_URL,
+    fileserver_url: str = DEFAULT_FILESERVER_URL,
+    fileserver_upload_handler: Callable = plik_oneshot_upload,
+    size_threshold: int = DEFAULT_SIZE_THRESHOLD,
+    correlation_id: str = None,
+    msg_purpose: str = "chat",
+    sender_name: str = "NATSBridge",
+    receiver_name: str = "",
+    receiver_id: str = "",
+    reply_to: str = "",
+    reply_to_msg_id: str = "",
+    is_publish: bool = True,
+    nats_connection: Any = None,
+    msg_id: str = None,
+    sender_id: str = None
+) -> Tuple[Dict, str]:
+    """
+    Send data via NATS with automatic transport selection.
+    
+    This function intelligently routes data delivery based on payload size.
+    If the serialized payload is smaller than size_threshold, it encodes the data as Base64
+    and publishes directly over NATS. Otherwise, it uploads the data to a fileserver
+    and publishes only the download URL over NATS.
+    
+    Args:
+        subject: NATS subject to publish the message to
+        data: List of (dataname, data, type) tuples to send
+            - dataname: Name of the payload
+            - data: The actual data to send
+            - type: Payload type: "text", "dictionary", "arrowtable", "jsontable", "image", "audio", "video", "binary"
+        broker_url: URL of the NATS server
+        fileserver_url: URL of the HTTP file server for large payloads
+        fileserver_upload_handler: Function to handle fileserver uploads (must return Dict with "status",
+                                   "uploadid", "fileid", "url" keys)
+        size_threshold: Threshold in bytes separating direct vs link transport
+        correlation_id: Correlation ID for tracing (auto-generated UUID if not provided)
+        msg_purpose: Purpose of the message: "ACK", "NACK", "updateStatus", "shutdown", "chat", etc.
+        sender_name: Name of the sender
+        receiver_name: Name of the receiver (empty string means broadcast)
+        receiver_id: UUID of the receiver (empty string means broadcast)
+        reply_to: Topic to reply to (empty string if no reply expected)
+        reply_to_msg_id: Message ID this message is replying to
+        is_publish: Whether to automatically publish the message to NATS
+        nats_connection: Pre-existing NATS connection (if provided, uses this connection instead of
+                        creating a new one; saves connection establishment overhead)
+        msg_id: Message ID (auto-generated UUID if not provided)
+        sender_id: Sender ID (auto-generated UUID if not provided)
+    
+    Returns:
+        Tuple of (env, env_json_str) where:
+            - env: Dict containing all metadata and payloads
+            - env_json_str: JSON string for publishing to NATS
+    
+    Example:
+        >>> # Send a single payload (still wrapped in a list)
+        >>> data = {"key": "value"}
+        >>> env, env_json_str = await smartsend(
+        ...     "my.subject",
+        ...     [("dataname1", data, "dictionary")],
+        ...     broker_url="nats://localhost:4222"
+        ... )
+        >>>
+        >>> # Send multiple payloads with different types
+        >>> data1 = {"key1": "value1"}
+        >>> data2 = [1, 2, 3, 4, 5]
+        >>> env, env_json_str = await smartsend(
+        ...     "my.subject",
+        ...     [("dataname1", data1, "dictionary"), ("dataname2", data2, "arrowtable")]
+        ... )
+        >>>
+        >>> # Send a large array using fileserver upload
+        >>> data = list(range(10_000_000))  # ~80 MB
+        >>> env, env_json_str = await smartsend(
+        ...     "large.data",
+        ...     [("large_table", data, "arrowtable")]
+        ... )
+        >>>
+        >>> # Send jsontable (JSON format for human-readable tabular data)
+        >>> users = [{"id": 1, "name": "Alice"}, {"id": 2, "name": "Bob"}]
+        >>> env, env_json_str = await smartsend(
+        ...     "json.data",
+        ...     [("users", users, "jsontable")]
+        ... )
+        >>>
+        >>> # Mixed content (e.g., chat with text and image)
+        >>> env, env_json_str = await smartsend(
+        ...     "chat.subject",
+        ...     [
+        ...         ("message_text", "Hello!", "text"),
+        ...         ("user_image", image_data, "image"),
+        ...         ("audio_clip", audio_data, "audio")
+        ...     ]
+        ... )
+        >>>
+        >>> # Publish the JSON string directly using NATS request-reply pattern
+        >>> # reply = await nats.request(broker_url, subject, env_json_str, reply_to=reply_to_topic)
+    """
+    if correlation_id is None:
+        correlation_id = str(uuid.uuid4())
+    if msg_id is None:
+        msg_id = str(uuid.uuid4())
+    if sender_id is None:
+        sender_id = str(uuid.uuid4())
+    
+    log_trace(correlation_id, f"Starting smartsend for subject: {subject}")
+    
+    # Process payloads
+    payloads = []
+    for dataname, payload_data, payload_type in data:
+        payload_bytes = _serialize_data(payload_data, payload_type)
+        payload_size = len(payload_bytes)
+        
+        log_trace(correlation_id, f"Serialized payload '{dataname}' (type: {payload_type}) size: {payload_size} bytes")
+        
+        if payload_size < size_threshold:
+            # Direct path
+            payload_b64 = base64.b64encode(payload_bytes).decode('utf-8')
+            log_trace(correlation_id, f"Using direct transport for {payload_size} bytes")
+            
+            payload = _build_payload(dataname, payload_type, payload_bytes, 'direct', payload_b64)
+            payloads.append(payload)
+        else:
+            # Link path
+            log_trace(correlation_id, "Using link transport, uploading to fileserver")
+            
+            response = await fileserver_upload_handler(fileserver_url, dataname, payload_bytes)
+            
+            if response['status'] != 200:
+                raise Exception(f"Failed to upload data to fileserver: {response['status']}")
+            
+            log_trace(correlation_id, f"Uploaded to URL: {response['url']}")
+            
+            payload = _build_payload(dataname, payload_type, payload_bytes, 'link', response['url'])
+            payloads.append(payload)
+    
+    # Build envelope
+    env = _build_envelope(subject, payloads, {
+        'correlation_id': correlation_id,
+        'msg_id': msg_id,
+        'msg_purpose': msg_purpose,
+        'sender_name': sender_name,
+        'sender_id': sender_id,
+        'receiver_name': receiver_name,
+        'receiver_id': receiver_id,
+        'reply_to': reply_to,
+        'reply_to_msg_id': reply_to_msg_id,
+        'broker_url': broker_url
+    })
+    
+    env_json_str = json.dumps(env)
+    
+    if is_publish:
+        if nats_connection:
+            await publish_message(nats_connection, subject, env_json_str, correlation_id)
+        else:
+            await publish_message(broker_url, subject, env_json_str, correlation_id)
+    
+    return env, env_json_str
+
+
+async def smartreceive(
+    msg: Any,
+    fileserver_download_handler: Callable = fetch_with_backoff,
+    max_retries: int = 5,
+    base_delay: int = 100,
+    max_delay: int = 5000
+) -> Dict[str, Any]:
+    """
+    Receive and process NATS messages.
+    
+    This function processes incoming NATS messages, handling both direct transport
+    (base64 decoded payloads) and link transport (URL-based payloads).
+    It deserializes the data based on the transport type and returns the result.
+    
+    Args:
+        msg: NATS message to process
+        fileserver_download_handler: Function to handle downloading data from file server URLs
+        max_retries: Maximum retry attempts for fetching URL
+        base_delay: Initial delay for exponential backoff in ms
+        max_delay: Maximum delay for exponential backoff in ms
+    
+    Returns:
+        Dict with envelope metadata and payloads field containing List[Tuple[str, Any, str]]
+    
+    Example:
+        >>> # Receive and process message
+        >>> env = await smartreceive(msg, fileserver_download_handler=fetch_with_backoff)
+        >>> # env is a Dict with "payloads" key containing List[Tuple[str, Any, str]]
+        >>> # Access payloads: for dataname, data, type_ in env["payloads"]
+        >>> for dataname, data, type_ in env["payloads"]:
+        >>>     print(f"{dataname}: {data} (type: {type_})")
+    """
+    # Parse the JSON envelope
+    if isinstance(msg, dict):
+        # Already parsed
+        env_json_obj = msg
+    elif hasattr(msg, 'payload'):
+        # NATS message object
+        payload = msg.payload if isinstance(msg.payload, str) else msg.payload.decode('utf-8')
+        env_json_obj = json.loads(payload)
+    else:
+        # Assume it's already a JSON string or dict
+        env_json_obj = json.loads(msg) if isinstance(msg, str) else msg
+    
+    log_trace(env_json_obj['correlation_id'], "Processing received message")
+    
+    # Process all payloads in the envelope
+    payloads_list = []
+    num_payloads = len(env_json_obj['payloads'])
+    
+    for i in range(num_payloads):
+        payload_obj = env_json_obj['payloads'][i]
+        transport = payload_obj['transport']
+        dataname = payload_obj['dataname']
+        
+        if transport == 'direct':
+            log_trace(env_json_obj['correlation_id'], f"Direct transport - decoding payload '{dataname}'")
+            
+            # Extract base64 payload from the payload
+            payload_b64 = payload_obj['data']
+            
+            # Decode Base64 payload
+            payload_bytes = base64.b64decode(payload_b64)
+            
+            # Deserialize based on type
+            data_type = payload_obj['payload_type']
+            data = _deserialize_data(payload_bytes, data_type, env_json_obj['correlation_id'])
+            
+            payloads_list.append((dataname, data, data_type))
+        elif transport == 'link':
+            # Extract download URL from the payload
+            url = payload_obj['data']
+            log_trace(env_json_obj['correlation_id'], f"Link transport - fetching '{dataname}' from URL: {url}")
+            
+            # Fetch with exponential backoff using the download handler
+            downloaded_data = await fileserver_download_handler(
+                url,
+                max_retries,
+                base_delay,
+                max_delay,
+                env_json_obj['correlation_id']
+            )
+            
+            # Deserialize based on type
+            data_type = payload_obj['payload_type']
+            data = _deserialize_data(downloaded_data, data_type, env_json_obj['correlation_id'])
+            
+            payloads_list.append((dataname, data, data_type))
+        else:
+            raise Exception(f"Unknown transport type for payload '{dataname}': {transport}")
+    
+    env_json_obj['payloads'] = payloads_list
+    return env_json_obj
+
+
+# ---------------------------------------------- Module Exports ---------------------------------------------- #
+
+class NATSBridge:
+    """
+    Cross-platform NATS bridge implementation.
+    
+    This class provides a convenient interface for NATSBridge functionality,
+    encapsulating the main functions and providing a class-based API.
+    """
+    
+    DEFAULT_SIZE_THRESHOLD = DEFAULT_SIZE_THRESHOLD
+    DEFAULT_BROKER_URL = DEFAULT_BROKER_URL
+    DEFAULT_FILESERVER_URL = DEFAULT_FILESERVER_URL
+    
+    def __init__(self, broker_url: str = None, fileserver_url: str = None):
+        """
+        Initialize NATSBridge.
+        
+        Args:
+            broker_url: NATS server URL (defaults to DEFAULT_BROKER_URL)
+            fileserver_url: HTTP file server URL (defaults to DEFAULT_FILESERVER_URL)
+        """
+        self.broker_url = broker_url or self.DEFAULT_BROKER_URL
+        self.fileserver_url = fileserver_url or self.DEFAULT_FILESERVER_URL
+    
+    async def smartsend(
+        self,
+        subject: str,
+        data: List[Tuple[str, Any, str]],
+        **kwargs
+    ) -> Tuple[Dict, str]:
+        """
+        Send data via NATS.
+        
+        Args:
+            subject: NATS subject to publish to
+            data: List of (dataname, data, type) tuples
+            **kwargs: Additional options passed to smartsend
+        
+        Returns:
+            Tuple of (env, env_json_str)
+        """
+        kwargs['broker_url'] = kwargs.get('broker_url', self.broker_url)
+        kwargs['fileserver_url'] = kwargs.get('fileserver_url', self.fileserver_url)
+        return await smartsend(subject, data, **kwargs)
+    
+    async def smartreceive(
+        self,
+        msg: Any,
+        **kwargs
+    ) -> Dict[str, Any]:
+        """
+        Receive and process NATS message.
+        
+        Args:
+            msg: NATS message to process
+            **kwargs: Additional options passed to smartreceive
+        
+        Returns:
+            Dict with envelope metadata and payloads
+        """
+        return await smartreceive(msg, **kwargs)
+
+
+# Convenience functions for module-level usage
+def send(
+    subject: str,
+    data: List[Tuple[str, Any, str]],
+    **kwargs
+) -> Tuple[Dict, str]:
+    """
+    Convenience function for sending data.
+    
+    Args:
+        subject: NATS subject to publish to
+        data: List of (dataname, data, type) tuples
+        **kwargs: Additional options
+    
+    Returns:
+        Tuple of (env, env_json_str)
+    """
+    return asyncio.run(smartsend(subject, data, **kwargs))
+
+
+def receive(
+    msg: Any,
+    **kwargs
+) -> Dict[str, Any]:
+    """
+    Convenience function for receiving messages.
+    
+    Args:
+        msg: NATS message to process
+        **kwargs: Additional options
+    
+    Returns:
+        Dict with envelope metadata and payloads
+    """
+    return asyncio.run(smartreceive(msg, **kwargs))
+
+
+__all__ = [
+    'smartsend',
+    'smartreceive',
+    'plik_oneshot_upload',
+    'fetch_with_backoff',
+    'NATSBridge',
+    'send',
+    'receive',
+    'DEFAULT_SIZE_THRESHOLD',
+    'DEFAULT_BROKER_URL',
+    'DEFAULT_FILESERVER_URL',
+    'NATSClient',
+    '_serialize_data',
+    '_deserialize_data',
+    'log_trace',
+    'publish_message'
+]

src/natsbridge_csr.js

@@ -0,0 +1,808 @@
+/**
+ * NATSBridge - Cross-Platform Bi-Directional Data Bridge
+ * Browser-Compatible Implementation (Client-Side Rendering)
+ * 
+ * This module provides functionality for sending and receiving data across network boundaries
+ * using NATS as the message bus, with support for both direct payload transport and
+ * URL-based transport for larger payloads.
+ * 
+ * Supported payload types: "text", "dictionary", "arrowtable", "jsontable", "image", "audio", "video", "binary"
+ * 
+ * Browser-compatible version uses:
+ * - nats.ws for WebSocket-based NATS connections
+ * - Web Crypto API for UUID generation
+ * - Uint8Array instead of Buffer
+ * - fetch API for file server communication
+ * 
+ * @module NATSBridgeCSR
+ */
+
+// Import browser-compatible NATS client
+import * as nats from 'nats.ws';
+
+// Use native fetch available in browsers
+import { tableFromArrays, tableToIPC } from 'apache-arrow/browser';
+
+// ---------------------------------------------- Constants ---------------------------------------------- //
+
+/**
+ * Default size threshold for switching from direct to link transport (0.5MB)
+ */
+const DEFAULT_SIZE_THRESHOLD = 500_000;
+
+/**
+ * Default NATS server URL (WebSocket protocol)
+ */
+const DEFAULT_BROKER_URL = 'ws://localhost:4222';
+
+/**
+ * Default HTTP file server URL for link transport
+ */
+const DEFAULT_FILESERVER_URL = 'http://localhost:8080';
+
+// ---------------------------------------------- Utility Functions ---------------------------------------------- //
+
+/**
+ * Convert Uint8Array to Base64 string
+ * @param {Uint8Array} data - Data to encode
+ * @returns {string} Base64 encoded string
+ */
+function bufferToBase64(data) {
+    const bytes = new Uint8Array(data);
+    let binary = '';
+    for (let i = 0; i < bytes.length; i++) {
+        binary += String.fromCharCode(bytes[i]);
+    }
+    return btoa(binary);
+}
+
+/**
+ * Convert Base64 string to Uint8Array
+ * @param {string} base64 - Base64 encoded string
+ * @returns {Uint8Array} Decoded binary data
+ */
+function base64ToBuffer(base64) {
+    const binary = atob(base64);
+    const len = binary.length;
+    const bytes = new Uint8Array(len);
+    for (let i = 0; i < len; i++) {
+        bytes[i] = binary.charCodeAt(i);
+    }
+    return bytes;
+}
+
+/**
+ * Generate UUID v4 using Web Crypto API
+ * @returns {string} UUID string
+ */
+function uuidv4() {
+    const array = new Uint8Array(16);
+    crypto.getRandomValues(array);
+    array[6] = (array[6] & 0x0f) | 0x40;
+    array[8] = (array[8] & 0x3f) | 0x80;
+    return Array.from(array, (val) => val.toString(16).padStart(2, '0').toUpperCase()).join('');
+}
+
+/**
+ * Log a trace message with correlation ID and timestamp
+ * @param {string} correlationId - Correlation ID for tracing
+ * @param {string} message - Message content to log
+ */
+function logTrace(correlationId, message) {
+    const timestamp = new Date().toISOString();
+    console.log(`[${timestamp}] [Correlation: ${correlationId}] ${message}`);
+}
+
+// ---------------------------------------------- Serialization Functions ---------------------------------------------- //
+
+/**
+ * Serialize data according to specified format
+ * @param {any} data - Data to serialize
+ * @param {string} payloadType - Target format: "text", "dictionary", "arrowtable", "jsontable", "image", "audio", "video", "binary"
+ * @returns {Uint8Array} Binary representation of the serialized data
+ */
+async function serializeData(data, payloadType) {
+    if (payloadType === 'text') {
+        if (typeof data === 'string') {
+            return new Uint8Array(new TextEncoder().encode(data));
+        } else {
+            throw new Error('Text data must be a string');
+        }
+    } else if (payloadType === 'dictionary') {
+        const jsonStr = JSON.stringify(data);
+        return new Uint8Array(new TextEncoder().encode(jsonStr));
+    } else if (payloadType === 'arrowtable') {
+        // Convert array of objects to Arrow IPC format
+        if (!Array.isArray(data) || data.length === 0) {
+            throw new Error('Arrow table data must be a non-empty array of objects');
+        }
+        
+        return serializeArrowTable(data);
+    } else if (payloadType === 'jsontable') {
+        // Serialize array of objects to JSON format
+        if (!Array.isArray(data)) {
+            throw new Error('JSON table data must be an array');
+        }
+        const jsonStr = JSON.stringify(data);
+        return new Uint8Array(new TextEncoder().encode(jsonStr));
+    } else if (payloadType === 'image') {
+        if (data instanceof Uint8Array || data instanceof ArrayBuffer || ArrayBuffer.isView(data)) {
+            return new Uint8Array(data);
+        } else {
+            throw new Error('Image data must be Uint8Array, ArrayBuffer, or ArrayBuffer view');
+        }
+    } else if (payloadType === 'audio') {
+        if (data instanceof Uint8Array || data instanceof ArrayBuffer || ArrayBuffer.isView(data)) {
+            return new Uint8Array(data);
+        } else {
+            throw new Error('Audio data must be Uint8Array, ArrayBuffer, or ArrayBuffer view');
+        }
+    } else if (payloadType === 'video') {
+        if (data instanceof Uint8Array || data instanceof ArrayBuffer || ArrayBuffer.isView(data)) {
+            return new Uint8Array(data);
+        } else {
+            throw new Error('Video data must be Uint8Array, ArrayBuffer, or ArrayBuffer view');
+        }
+    } else if (payloadType === 'binary') {
+        if (data instanceof Uint8Array || data instanceof ArrayBuffer || ArrayBuffer.isView(data)) {
+            return new Uint8Array(data);
+        } else {
+            throw new Error('Binary data must be Uint8Array, ArrayBuffer, or ArrayBuffer view');
+        }
+    } else {
+        throw new Error(`Unknown payload_type: ${payloadType}`);
+    }
+}
+
+/**
+ * Helper function to properly serialize table data to Arrow IPC
+ * @param {Array<Object>} data - Array of objects representing table rows
+ * @returns {Uint8Array} Arrow IPC formatted buffer
+ */
+function serializeArrowTable(data) {
+    if (!Array.isArray(data) || data.length === 0) {
+        throw new Error('Table data must be a non-empty array of objects');
+    }
+    
+    logTrace('serializeArrowTable', `Serializing table with ${data.length} rows`);
+    
+    // Convert array of objects to a key-value format expected by tableFromArrays
+    const columns = {};
+    const keys = Object.keys(data[0]);
+    for (const key of keys) {
+        columns[key] = data.map(row => row[key]);
+    }
+    
+    logTrace('serializeArrowTable', `Columns: ${Object.keys(columns).join(', ')}`);
+    
+    const table = tableFromArrays(columns);
+    
+    logTrace('serializeArrowTable', `Arrow table created with ${table.numRows} rows, ${table.numCols} cols`);
+    
+    // Convert to IPC format
+    const ipcBuffer = tableToIPC(table);
+    
+    logTrace('serializeArrowTable', `IPC buffer type: ${typeof ipcBuffer}, byteLength: ${ipcBuffer.byteLength}`);
+    
+    const resultBuffer = new Uint8Array(ipcBuffer);
+    logTrace('serializeArrowTable', `Result buffer: ${resultBuffer.length} bytes`);
+    
+    // Debug: Show first 20 bytes in hex
+    const hexPreview = [];
+    for (let i = 0; i < Math.min(20, resultBuffer.length); i++) {
+        hexPreview.push(resultBuffer[i].toString(16).padStart(2, '0'));
+    }
+    logTrace('serializeArrowTable', `First 20 bytes (hex): ${hexPreview.join(' ')}`);
+    
+    return resultBuffer;
+}
+
+/**
+ * Deserialize bytes to data based on type
+ * @param {Uint8Array|ArrayBuffer} data - Serialized data as bytes
+ * @param {string} payloadType - Data type
+ * @param {string} correlationId - Correlation ID for logging
+ * @returns {any} Deserialized data
+ */
+async function deserializeData(data, payloadType, correlationId) {
+    const buffer = data instanceof Uint8Array ? data : new Uint8Array(data);
+    
+    logTrace(correlationId, `deserializeData: type=${payloadType}, bufferLength=${buffer.length}`);
+    
+    // Debug: Show first 20 bytes in hex for binary data
+    if (payloadType === 'arrowtable' || payloadType === 'jsontable' || payloadType === 'image' || payloadType === 'binary') {
+        const hexPreview = [];
+        for (let i = 0; i < Math.min(20, buffer.length); i++) {
+            hexPreview.push(buffer[i].toString(16).padStart(2, '0'));
+        }
+        logTrace(correlationId, `deserializeData: First 20 bytes (hex): ${hexPreview.join(' ')}`);
+    }
+    
+    if (payloadType === 'text') {
+        const result = new TextDecoder().decode(buffer);
+        logTrace(correlationId, `deserializeData: text result length=${result.length}`);
+        return result;
+    } else if (payloadType === 'dictionary') {
+        const jsonStr = new TextDecoder().decode(buffer);
+        const result = JSON.parse(jsonStr);
+        logTrace(correlationId, `deserializeData: dictionary keys=${Object.keys(result).join(', ')}`);
+        return result;
+    } else if (payloadType === 'arrowtable') {
+        logTrace(correlationId, `deserializeData: Attempting Arrow table deserialization`);
+        
+        try {
+            // Try tableFromIPC (browser API)
+            const table = tableFromIPC(buffer);
+            logTrace(correlationId, `deserializeData: Arrow table from IPC - rows=${table.numRows}, cols=${table.numCols}`);
+            return table;
+        } catch (e) {
+            logTrace(correlationId, `deserializeData: tableFromIPC failed: ${e.message}`);
+            throw new Error(`Unable to deserialize Arrow table: ${e.message}`);
+        }
+    } else if (payloadType === 'jsontable') {
+        const jsonStr = new TextDecoder().decode(buffer);
+        const result = JSON.parse(jsonStr);
+        logTrace(correlationId, `deserializeData: jsontable result length=${Array.isArray(result) ? result.length : 'N/A'}`);
+        return result;
+    } else if (payloadType === 'image') {
+        logTrace(correlationId, `deserializeData: image buffer length=${buffer.length}`);
+        return buffer;
+    } else if (payloadType === 'audio') {
+        logTrace(correlationId, `deserializeData: audio buffer length=${buffer.length}`);
+        return buffer;
+    } else if (payloadType === 'video') {
+        logTrace(correlationId, `deserializeData: video buffer length=${buffer.length}`);
+        return buffer;
+    } else if (payloadType === 'binary') {
+        logTrace(correlationId, `deserializeData: binary buffer length=${buffer.length}`);
+        return buffer;
+    } else {
+        throw new Error(`Unknown payload_type: ${payloadType}`);
+    }
+}
+
+// ---------------------------------------------- File Server Handlers ---------------------------------------------- //
+
+/**
+ * Upload data to plik server in one-shot mode
+ * @param {string} fileServerUrl - Base URL of the plik server
+ * @param {string} dataname - Name of the file being uploaded
+ * @param {Uint8Array} data - Raw byte data of the file content
+ * @returns {Promise<{status: number, uploadid: string, fileid: string, url: string}>}
+ */
+async function plikOneshotUpload(fileServerUrl, dataname, data) {
+    const buffer = data instanceof Uint8Array ? data : new Uint8Array(data);
+    
+    // Get upload id
+    const urlGetUploadID = `${fileServerUrl}/upload`;
+    const headers = { 'Content-Type': 'application/json' };
+    const body = JSON.stringify({ OneShot: true });
+    
+    const httpResponse = await fetch(urlGetUploadID, {
+        method: 'POST',
+        headers,
+        body
+    });
+    
+    const responseJson = await httpResponse.json();
+    const uploadid = responseJson.id;
+    const uploadtoken = responseJson.uploadToken;
+
+    // Upload file
+    const urlUpload = `${fileServerUrl}/file/${uploadid}`;
+    const form = new FormData();
+    const blob = new Blob([buffer], { type: 'application/octet-stream' });
+    form.append('file', blob, dataname);
+    
+    const uploadHeaders = {
+        'X-UploadToken': uploadtoken
+    };
+
+    const uploadResponse = await fetch(urlUpload, {
+        method: 'POST',
+        headers: uploadHeaders,
+        body: form
+    });
+    
+    const uploadJson = await uploadResponse.json();
+    const fileid = uploadJson.id;
+
+    const url = `${fileServerUrl}/file/${uploadid}/${fileid}/${dataname}`;
+
+    return {
+        status: uploadResponse.status,
+        uploadid,
+        fileid,
+        url
+    };
+}
+
+/**
+ * Fetch data from URL with exponential backoff
+ * @param {string} url - URL to fetch from
+ * @param {number} maxRetries - Maximum number of retry attempts
+ * @param {number} baseDelay - Initial delay in milliseconds
+ * @param {number} maxDelay - Maximum delay in milliseconds
+ * @param {string} correlationId - Correlation ID for logging
+ * @returns {Promise<Uint8Array>} Fetched data as bytes
+ */
+async function fetchWithBackoff(url, maxRetries, baseDelay, maxDelay, correlationId) {
+    let delay = baseDelay;
+    
+    for (let attempt = 1; attempt <= maxRetries; attempt++) {
+        try {
+            const response = await fetch(url);
+            
+            if (response.status === 200) {
+                logTrace(correlationId, `Successfully fetched data from ${url} on attempt ${attempt}`);
+                const arrayBuffer = await response.arrayBuffer();
+                return new Uint8Array(arrayBuffer);
+            } else {
+                throw new Error(`Failed to fetch: ${response.status}`);
+            }
+        } catch (e) {
+            logTrace(correlationId, `Attempt ${attempt} failed: ${e.constructor.name} - ${e.message}`);
+            
+            if (attempt < maxRetries) {
+                await new Promise(resolve => setTimeout(resolve, delay));
+                delay = Math.min(delay * 2, maxDelay);
+            }
+        }
+    }
+    
+    throw new Error(`Failed to fetch data after ${maxRetries} attempts`);
+}
+
+// ---------------------------------------------- NATS Client ---------------------------------------------- //
+
+/**
+ * NATS client wrapper for connection management
+ */
+class NATSClient {
+    /**
+     * Create a new NATS client
+     * @param {string} url - NATS server URL (ws:// or wss://)
+     */
+    constructor(url) {
+        this.url = url;
+        this.connection = null;
+    }
+
+    /**
+     * Connect to NATS server
+     * @returns {Promise<NATS.Connection>}
+     */
+    async connect() {
+        this.connection = await nats.connect({ servers: this.url });
+        return this.connection;
+    }
+
+    /**
+     * Publish message to NATS subject
+     * @param {string} subject - NATS subject to publish to
+     * @param {string} message - Message to publish
+     * @param {string} correlationId - Correlation ID for logging
+     */
+    async publish(subject, message, correlationId) {
+        if (!this.connection) {
+            await this.connect();
+        }
+        await this.connection.publish(subject, message);
+        logTrace(correlationId, `Message published to ${subject}`);
+    }
+
+    /**
+     * Close the NATS connection
+     */
+    async close() {
+        if (this.connection) {
+            this.connection.close();
+        }
+    }
+}
+
+// ---------------------------------------------- Core Functions ---------------------------------------------- //
+
+/**
+ * Publish message to NATS
+ * @param {string|NATSClient|NATS.Connection} brokerUrlOrClient - NATS URL, client, or connection
+ * @param {string} subject - NATS subject to publish to
+ * @param {string} message - JSON message to publish
+ * @param {string} correlationId - Correlation ID for tracing
+ */
+async function publishMessage(brokerUrlOrClient, subject, message, correlationId) {
+    let conn;
+    
+    if (brokerUrlOrClient instanceof NATSClient) {
+        conn = brokerUrlOrClient;
+    } else if (brokerUrlOrClient && typeof brokerUrlOrClient.publish === 'function') {
+        // Create a wrapper for direct connection (duck-typing check for NATS connection)
+        conn = {
+            async publish(subj, msg) {
+                await brokerUrlOrClient.publish(subj, msg);
+            },
+            async close() {
+                await brokerUrlOrClient.close();
+            }
+        };
+    } else {
+        // String URL - create new client
+        const client = new NATSClient(brokerUrlOrClient);
+        conn = client;
+    }
+    
+    await conn.publish(subject, message, correlationId);
+    
+    if (conn instanceof NATSClient) {
+        await conn.close();
+    }
+}
+
+/**
+ * Build message envelope from payloads and metadata
+ * @param {string} subject - NATS subject
+ * @param {Array} payloads - Array of payload objects
+ * @param {Object} options - Envelope metadata options
+ * @returns {Object} Envelope object
+ */
+function buildEnvelope(subject, payloads, options) {
+    return {
+        correlation_id: options.correlation_id,
+        msg_id: options.msg_id,
+        timestamp: new Date().toISOString(),
+        send_to: subject,
+        msg_purpose: options.msg_purpose,
+        sender_name: options.sender_name,
+        sender_id: options.sender_id,
+        receiver_name: options.receiver_name,
+        receiver_id: options.receiver_id,
+        reply_to: options.reply_to,
+        reply_to_msg_id: options.reply_to_msg_id,
+        broker_url: options.broker_url,
+        metadata: options.metadata || {},
+        payloads: payloads
+    };
+}
+
+/**
+ * Build payload object from serialized data
+ * @param {string} dataname - Name of the payload
+ * @param {string} payloadType - Type of the payload
+ * @param {Uint8Array} payloadBytes - Serialized payload bytes
+ * @param {string} transport - Transport type ("direct" or "link")
+ * @param {string} data - Data (base64 for direct, URL for link)
+ * @returns {Object} Payload object
+ */
+function buildPayload(dataname, payloadType, payloadBytes, transport, data) {
+    // Determine encoding based on payload type (matching Julia implementation)
+    let encoding = 'base64';
+    if (payloadType === 'jsontable') {
+        encoding = 'json';
+    } else if (payloadType === 'arrowtable') {
+        encoding = 'arrow-ipc';
+    }
+    
+    return {
+        id: uuidv4(),
+        dataname,
+        payload_type: payloadType,
+        transport,
+        encoding,
+        size: payloadBytes.byteLength,
+        data,
+        metadata: transport === 'direct' ? { payload_bytes: payloadBytes.byteLength } : {}
+    };
+}
+
+/**
+ * Send data via NATS with automatic transport selection
+ * 
+ * This function intelligently routes data delivery based on payload size.
+ * If the serialized payload is smaller than size_threshold, it encodes the data as Base64
+ * and publishes directly over NATS. Otherwise, it uploads the data to a fileserver
+ * and publishes only the download URL over NATS.
+ * 
+ * @param {string} subject - NATS subject to publish the message to
+ * @param {Array} data - List of [dataname, data, type] tuples to send
+ *          - type: "text", "dictionary", "arrowtable", "jsontable", "image", "audio", "video", "binary"
+ * @param {Object} options - Optional configuration
+ * @param {string} [options.broker_url=DEFAULT_BROKER_URL] - URL of the NATS server (WebSocket)
+ * @param {string} [options.fileserver_url=DEFAULT_FILESERVER_URL] - URL of the HTTP file server
+ * @param {Function} [options.fileserver_upload_handler=plikOneshotUpload] - Function to handle fileserver uploads
+ * @param {number} [options.size_threshold=DEFAULT_SIZE_THRESHOLD] - Threshold separating direct vs link transport
+ * @param {string} [options.correlation_id=uuidv4()] - Correlation ID for tracing
+ * @param {string} [options.msg_purpose="chat"] - Purpose of the message
+ * @param {string} [options.sender_name="NATSBridge"] - Name of the sender
+ * @param {string} [options.receiver_name=""] - Name of the receiver (empty means broadcast)
+ * @param {string} [options.receiver_id=""] - UUID of the receiver (empty means broadcast)
+ * @param {string} [options.reply_to=""] - Topic to reply to
+ * @param {string} [options.reply_to_msg_id=""] - Message ID this message is replying to
+ * @param {boolean} [options.is_publish=true] - Whether to automatically publish the message
+ * @param {NATSClient|NATS.Connection} [options.nats_connection=null] - Pre-existing NATS connection
+ * @param {string} [options.msg_id=uuidv4()] - Message ID
+ * @param {string} [options.sender_id=uuidv4()] - Sender ID
+ * @returns {Promise<[Object, string]>} Tuple of [env, env_json_str]
+ * 
+ * @example
+ * // Send a single payload
+ * const [env, envJsonStr] = await NATSBridgeCSR.smartsend(
+ *     "/test",
+ *     [["dataname1", data1, "dictionary"]],
+ *     { broker_url: "ws://localhost:4222" }
+ * );
+ * 
+ * // Send multiple payloads
+ * const [env, envJsonStr] = await NATSBridgeCSR.smartsend(
+ *     "/test",
+ *     [
+ *         ["dataname1", data1, "dictionary"],
+ *         ["dataname2", data2, "arrowtable"]
+ *     ],
+ *     { broker_url: "ws://localhost:4222" }
+ * );
+ */
+async function smartsend(subject, data, options = {}) {
+    const {
+        broker_url = DEFAULT_BROKER_URL,
+        fileserver_url = DEFAULT_FILESERVER_URL,
+        fileserver_upload_handler = plikOneshotUpload,
+        size_threshold = DEFAULT_SIZE_THRESHOLD,
+        correlation_id = uuidv4(),
+        msg_purpose = 'chat',
+        sender_name = 'NATSBridge',
+        receiver_name = '',
+        receiver_id = '',
+        reply_to = '',
+        reply_to_msg_id = '',
+        is_publish = true,
+        nats_connection = null,
+        msg_id = uuidv4(),
+        sender_id = uuidv4()
+    } = options;
+
+    logTrace(correlation_id, `Starting smartsend for subject: ${subject}`);
+    logTrace(correlation_id, `smartsend: data array length=${data.length}`);
+    
+    // Debug: Log input data structure
+    for (let i = 0; i < data.length; i++) {
+        const [dataname, payloadData, payloadType] = data[i];
+        logTrace(correlation_id, `smartsend: payload[${i}] dataname=${dataname}, type=${payloadType}, data type=${typeof payloadData}, constructor=${payloadData?.constructor?.name}`);
+    }
+
+    // Process payloads
+    const payloads = [];
+    for (const [dataname, payloadData, payloadType] of data) {
+        logTrace(correlation_id, `smartsend: Processing payload '${dataname}' type=${payloadType}`);
+        logTrace(correlation_id, `smartsend: payloadData type=${typeof payloadData}, constructor=${payloadData?.constructor?.name}`);
+        
+        const payloadBytes = await serializeData(payloadData, payloadType);
+        const payloadSize = payloadBytes.byteLength;
+
+        logTrace(correlation_id, `Serialized payload '${dataname}' (type: ${payloadType}) size: ${payloadSize} bytes`);
+
+        // Debug: Show first 20 bytes of serialized data for table type
+        if (payloadType === 'table') {
+            const hexPreview = [];
+            for (let i = 0; i < Math.min(20, payloadBytes.length); i++) {
+                hexPreview.push(payloadBytes[i].toString(16).padStart(2, '0'));
+            }
+            logTrace(correlation_id, `Serialized table data first 20 bytes (hex): ${hexPreview.join(' ')}`);
+        }
+
+        if (payloadSize < size_threshold) {
+            // Direct path
+            const payloadB64 = bufferToBase64(payloadBytes);
+            logTrace(correlation_id, `Using direct transport for ${payloadSize} bytes, base64 length=${payloadB64.length}`);
+
+            const payload = buildPayload(dataname, payloadType, payloadBytes, 'direct', payloadB64);
+            payloads.push(payload);
+        } else {
+            // Link path
+            logTrace(correlation_id, `Using link transport, uploading to fileserver`);
+
+            const response = await fileserver_upload_handler(fileserver_url, dataname, payloadBytes);
+
+            if (response.status !== 200) {
+                throw new Error(`Failed to upload data to fileserver: ${response.status}`);
+            }
+
+            logTrace(correlation_id, `Uploaded to URL: ${response.url}`);
+
+            const payload = buildPayload(dataname, payloadType, payloadBytes, 'link', response.url);
+            payloads.push(payload);
+        }
+    }
+
+    // Build envelope
+    const env = buildEnvelope(subject, payloads, {
+        correlation_id,
+        msg_id,
+        msg_purpose,
+        sender_name,
+        sender_id,
+        receiver_name,
+        receiver_id,
+        reply_to,
+        reply_to_msg_id,
+        broker_url
+    });
+
+    const env_json_str = JSON.stringify(env);
+
+    if (is_publish) {
+        if (nats_connection) {
+            await publishMessage(nats_connection, subject, env_json_str, correlation_id);
+        } else {
+            await publishMessage(broker_url, subject, env_json_str, correlation_id);
+        }
+    }
+
+    return [env, env_json_str];
+}
+
+/**
+ * Receive and process NATS message
+ * 
+ * This function processes incoming NATS messages, handling both direct transport
+ * (base64 decoded payloads) and link transport (URL-based payloads).
+ * It deserializes the data based on the transport type and returns the result.
+ * 
+ * @param {Object} msg - NATS message object with payload property
+ * @param {Object} options - Optional configuration
+ * @param {Function} [options.fileserver_download_handler=fetchWithBackoff] - Function to handle fileserver downloads
+ * @param {number} [options.max_retries=5] - Maximum retry attempts for fetching URL
+ * @param {number} [options.base_delay=100] - Initial delay for exponential backoff in ms
+ * @param {number} [options.max_delay=5000] - Maximum delay for exponential backoff in ms
+ * @returns {Promise<Object>} Envelope object with processed payloads
+ * 
+ * @example
+ * // Receive and process message
+ * const env = await NATSBridgeCSR.smartreceive(msg, {
+ *     fileserver_download_handler: NATSBridgeCSR.fetchWithBackoff,
+ *     max_retries: 5,
+ *     base_delay: 100,
+ *     max_delay: 5000
+ * });
+ * // env.payloads is an Array of [dataname, data, type] arrays
+ * for (const [dataname, data, type] of env.payloads) {
+ *     console.log(`${dataname}: ${data} (type: ${type})`);
+ * }
+ */
+async function smartreceive(msg, options = {}) {
+    const {
+        fileserver_download_handler = fetchWithBackoff,
+        max_retries = 5,
+        base_delay = 100,
+        max_delay = 5000
+    } = options;
+
+    // Debug: Log message object structure
+    logTrace('smartreceive', `smartreceive: msg object keys: ${Object.keys(msg).join(', ')}`);
+    logTrace('smartreceive', `smartreceive: msg.data type: ${typeof msg.data}, constructor: ${msg.data?.constructor?.name}`);
+    logTrace('smartreceive', `smartreceive: msg.payload type: ${typeof msg.payload}, constructor: ${msg.payload?.constructor?.name}`);
+    
+    // Parse the JSON envelope
+    // NATS.js v2.x uses msg.data instead of msg.payload
+    let payload;
+    if (msg.data !== undefined) {
+        payload = typeof msg.data === 'string' ? msg.data : new TextDecoder().decode(msg.data);
+    } else if (msg.payload !== undefined) {
+        payload = typeof msg.payload === 'string' ? msg.payload : new TextDecoder().decode(msg.payload);
+    } else {
+        throw new Error('Message has neither data nor payload property');
+    }
+    
+    logTrace('smartreceive', `smartreceive: raw payload length=${payload.length}`);
+    
+    // Debug: Show first 200 chars of payload
+    const payloadPreview = payload.substring(0, 200);
+    logTrace('smartreceive', `smartreceive: payload preview: ${payloadPreview}`);
+    
+    let envJsonObj;
+    try {
+        envJsonObj = JSON.parse(payload);
+    } catch (e) {
+        logTrace('smartreceive', `smartreceive: JSON parse failed: ${e.message}`);
+        throw e;
+    }
+    
+    logTrace(envJsonObj.correlation_id, 'Processing received message');
+    logTrace(envJsonObj.correlation_id, `smartreceive: envelope has ${envJsonObj.payloads.length} payloads`);
+
+    // Process all payloads in the envelope
+    const payloadsList = [];
+    const numPayloads = envJsonObj.payloads.length;
+
+    logTrace(envJsonObj.correlation_id, `smartreceive: Processing ${numPayloads} payloads`);
+
+    for (let i = 0; i < numPayloads; i++) {
+        const payloadObj = envJsonObj.payloads[i];
+        const transport = payloadObj.transport;
+        const dataname = payloadObj.dataname;
+        const payloadType = payloadObj.payload_type;
+
+        logTrace(envJsonObj.correlation_id, `smartreceive: Processing payload ${i + 1}/${numPayloads}: dataname=${dataname}, type=${payloadType}, transport=${transport}`);
+
+        if (transport === 'direct') {
+            logTrace(envJsonObj.correlation_id, `Direct transport - decoding payload '${dataname}'`);
+
+            // Extract base64 payload from the payload
+            const payloadB64 = payloadObj.data;
+            logTrace(envJsonObj.correlation_id, `Direct transport: base64 length=${payloadB64?.length}`);
+
+            // Decode Base64 payload
+            const payloadBytes = base64ToBuffer(payloadB64);
+            logTrace(envJsonObj.correlation_id, `Direct transport: decoded bytes=${payloadBytes.length}`);
+
+            // Deserialize based on type
+            const dataType = payloadObj.payload_type;
+            const data = await deserializeData(payloadBytes, dataType, envJsonObj.correlation_id);
+            logTrace(envJsonObj.correlation_id, `Direct transport: deserialized data type=${typeof data}, constructor=${data?.constructor?.name}`);
+
+            payloadsList.push([dataname, data, dataType]);
+        } else if (transport === 'link') {
+            // Extract download URL from the payload
+            const url = payloadObj.data;
+            logTrace(envJsonObj.correlation_id, `Link transport - fetching '${dataname}' from URL: ${url}`);
+
+            // Fetch with exponential backoff using the download handler
+            const downloadedData = await fileserver_download_handler(
+                url,
+                max_retries,
+                base_delay,
+                max_delay,
+                envJsonObj.correlation_id
+            );
+
+            // Deserialize based on type
+            const dataType = payloadObj.payload_type;
+            const data = await deserializeData(downloadedData, dataType, envJsonObj.correlation_id);
+
+            payloadsList.push([dataname, data, dataType]);
+        } else {
+            throw new Error(`Unknown transport type for payload '${dataname}': ${transport}`);
+        }
+    }
+
+    logTrace(envJsonObj.correlation_id, `smartreceive: Successfully processed all ${payloadsList.length} payloads`);
+    envJsonObj.payloads = payloadsList;
+    return envJsonObj;
+}
+
+// ---------------------------------------------- Module Exports ---------------------------------------------- //
+
+const NATSBridgeCSR = {
+    /**
+     * NATS client class for connection management
+     */
+    NATSClient,
+
+    /**
+     * Send data via NATS with automatic transport selection
+     */
+    smartsend,
+
+    /**
+     * Receive and process NATS message
+     */
+    smartreceive,
+
+    /**
+     * Upload data to plik server in one-shot mode
+     */
+    plikOneshotUpload,
+
+    /**
+     * Fetch data from URL with exponential backoff
+     */
+    fetchWithBackoff,
+
+    /**
+     * Default constants
+     */
+    DEFAULT_SIZE_THRESHOLD,
+    DEFAULT_BROKER_URL,
+    DEFAULT_FILESERVER_URL
+};
+
+export default NATSBridgeCSR;

src/natsbridge_mpy.py

@@ -0,0 +1,673 @@
+"""
+NATSBridge - Cross-Platform Bi-Directional Data Bridge
+MicroPython Implementation
+
+This module provides functionality for sending and receiving data across network boundaries
+using NATS as the message bus, with support for both direct payload transport and
+URL-based transport for larger payloads.
+
+Note: MicroPython has significant constraints compared to desktop implementations:
+- Limited memory (~256KB - 1MB)
+- No Arrow IPC support (memory constraints)
+- Synchronous API (no async/await)
+- Lower size threshold for direct transport
+"""
+
+import network
+import time
+import json
+import base64
+import uos
+import struct
+import random
+
+# ---------------------------------------------- Constants ---------------------------------------------- #
+
+"""
+Default size threshold for switching from direct to link transport (100KB for MicroPython)
+"""
+DEFAULT_SIZE_THRESHOLD = 100000
+
+"""
+Default NATS server URL
+"""
+DEFAULT_BROKER_URL = "nats://localhost:4222"
+
+"""
+Default HTTP file server URL for link transport
+"""
+DEFAULT_FILESERVER_URL = "http://localhost:8080"
+
+"""
+Hard limit for payload size in MicroPython (50KB)
+"""
+MAX_PAYLOAD_SIZE = 50000
+
+
+# ---------------------------------------------- Utility Functions ---------------------------------------------- #
+
+def log_trace(correlation_id, message):
+    """
+    Log a trace message with correlation ID and timestamp.
+    
+    Args:
+        correlation_id: Correlation ID for tracing
+        message: Message content to log
+    """
+    timestamp = time.strftime('%Y-%m-%dT%H:%M:%SZ', time.localtime())
+    print(f"[{timestamp}] [Correlation: {correlation_id}] {message}")
+
+
+def _generate_uuid():
+    """
+    Generate a simple UUID compatible with MicroPython.
+    
+    Returns:
+        UUID string
+    """
+    # Generate a simple UUID-like string
+    # Format: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
+    hex_chars = '0123456789abcdef'
+    uuid_str = ''.join([random.choice(hex_chars) for _ in range(32)])
+    # Insert hyphens at proper positions
+    return f"{uuid_str[:8]}-{uuid_str[8:12]}-{uuid_str[12:16]}-{uuid_str[16:20]}-{uuid_str[20:]}"
+
+
+# ---------------------------------------------- Serialization Functions ---------------------------------------------- #
+
+def _serialize_data(data, payload_type):
+    """
+    Serialize data according to specified format.
+    
+    Args:
+        data: Data to serialize (string for "text", dict for "dictionary",
+              bytes for "image", "audio", "video", "binary")
+        payload_type: Target format: "text", "dictionary", "image", "audio", "video", "binary"
+    
+    Returns:
+        Binary representation of the serialized data
+    
+    Note:
+        MicroPython does not support "table" type due to memory constraints.
+    
+    Raises:
+        ValueError: If payload_type is not one of the supported types
+    """
+    if payload_type == 'text':
+        if isinstance(data, str):
+            return data.encode('utf-8')
+        else:
+            raise ValueError('Text data must be a string')
+    elif payload_type == 'dictionary':
+        json_str = json.dumps(data)
+        return json_str.encode('utf-8')
+    elif payload_type in ('image', 'audio', 'video', 'binary'):
+        if isinstance(data, (bytes, bytearray, memoryview)):
+            return bytes(data)
+        else:
+            raise ValueError(f'{payload_type} data must be bytes')
+    else:
+        raise ValueError(f'Unknown payload_type: {payload_type}')
+
+
+def _deserialize_data(data, payload_type):
+    """
+    Deserialize bytes to data based on type.
+    
+    Args:
+        data: Serialized data as bytes
+        payload_type: Data type ("text", "dictionary", "image", "audio", "video", "binary")
+    
+    Returns:
+        Deserialized data (String for "text", dict for "dictionary", bytes for others)
+    
+    Note:
+        MicroPython does not support "table" type due to memory constraints.
+    
+    Raises:
+        ValueError: If payload_type is not one of the supported types
+    """
+    if payload_type == 'text':
+        return data.decode('utf-8')
+    elif payload_type == 'dictionary':
+        json_str = data.decode('utf-8')
+        return json.loads(json_str)
+    elif payload_type in ('image', 'audio', 'video', 'binary'):
+        return data
+    else:
+        raise ValueError(f'Unknown payload_type: {payload_type}')
+
+
+# ---------------------------------------------- File Server Handlers ---------------------------------------------- #
+
+def _sync_fileserver_upload(file_server_url, dataname, data):
+    """
+    Synchronous file upload to HTTP server.
+    
+    Note:
+        This is a simplified implementation for MicroPython.
+        In practice, would use network.HTTP or similar.
+        Currently raises NotImplementedError as file upload is not fully supported.
+    
+    Args:
+        file_server_url: Base URL of the file server
+        dataname: Name of the file being uploaded
+        data: Raw byte data of the file content
+    
+    Returns:
+        Dict with keys: 'status', 'url'
+    
+    Raises:
+        NotImplementedError: File upload is not implemented in MicroPython
+    """
+    raise NotImplementedError("File upload not fully implemented in MicroPython. "
+                              "Use direct transport only for memory-constrained devices.")
+
+
+def _sync_fileserver_download(url, max_retries, base_delay, max_delay, correlation_id):
+    """
+    Synchronous file download with exponential backoff.
+    
+    Note:
+        This is a simplified implementation for MicroPython.
+        In practice, would use network.HTTP or similar.
+        Currently raises NotImplementedError as file download is not fully supported.
+    
+    Args:
+        url: URL to download from
+        max_retries: Maximum retry attempts
+        base_delay: Initial delay in ms
+        max_delay: Maximum delay in ms
+        correlation_id: Correlation ID for logging
+    
+    Returns:
+        Downloaded bytes
+    
+    Raises:
+        NotImplementedError: File download is not implemented in MicroPython
+    """
+    raise NotImplementedError("File download not fully implemented in MicroPython. "
+                              "Use direct transport only for memory-constrained devices.")
+
+
+# ---------------------------------------------- NATS Client ---------------------------------------------- #
+
+class NATSClient:
+    """
+    NATS client wrapper for MicroPython.
+    
+    Note:
+        This is a simplified implementation for MicroPython.
+        Full NATS client implementation would require additional network stack support.
+    """
+    
+    def __init__(self, url=DEFAULT_BROKER_URL):
+        """
+        Initialize NATS client.
+        
+        Args:
+            url: NATS server URL
+        """
+        self.url = url
+        self._connected = False
+    
+    def connect(self):
+        """
+        Connect to NATS server.
+        
+        Note:
+            This is a placeholder implementation.
+            Actual NATS client would require network stack support.
+        
+        Returns:
+            True if connected, False otherwise
+        """
+        # Placeholder - actual implementation would connect to NATS server
+        self._connected = True
+        return self._connected
+    
+    def publish(self, subject, message):
+        """
+        Publish message to NATS subject.
+        
+        Note:
+            This is a placeholder implementation.
+            Actual NATS client would require network stack support.
+        
+        Args:
+            subject: NATS subject to publish to
+            message: Message to publish
+        """
+        if not self._connected:
+            raise RuntimeError("Not connected to NATS server")
+        # Placeholder - actual implementation would publish to NATS
+        print(f"[NATS] Publish to {subject}: {message[:50]}...")
+    
+    def close(self):
+        """Close the NATS connection."""
+        self._connected = False
+
+
+# ---------------------------------------------- Core Functions ---------------------------------------------- #
+
+def _build_envelope(subject, payloads, options):
+    """
+    Build message envelope from payloads and metadata.
+    
+    Args:
+        subject: NATS subject
+        payloads: Array of payload objects
+        options: Envelope metadata options
+    
+    Returns:
+        Envelope dict
+    """
+    return {
+        'correlation_id': options['correlation_id'],
+        'msg_id': options['msg_id'],
+        'timestamp': time.strftime('%Y-%m-%dT%H:%M:%SZ', time.localtime()),
+        'send_to': subject,
+        'msg_purpose': options['msg_purpose'],
+        'sender_name': options['sender_name'],
+        'sender_id': options['sender_id'],
+        'receiver_name': options['receiver_name'],
+        'receiver_id': options['receiver_id'],
+        'reply_to': options['reply_to'],
+        'reply_to_msg_id': options['reply_to_msg_id'],
+        'broker_url': options['broker_url'],
+        'metadata': {},
+        'payloads': payloads
+    }
+
+
+def _build_payload(dataname, payload_type, payload_bytes, transport, data):
+    """
+    Build payload object from serialized data.
+    
+    Args:
+        dataname: Name of the payload
+        payload_type: Type of the payload
+        payload_bytes: Serialized payload bytes
+        transport: Transport type ("direct" or "link")
+        data: Data (base64 for direct, URL for link)
+    
+    Returns:
+        Payload dict
+    """
+    return {
+        'id': _generate_uuid(),
+        'dataname': dataname,
+        'payload_type': payload_type,
+        'transport': transport,
+        'encoding': 'base64' if transport == 'direct' else 'none',
+        'size': len(payload_bytes),
+        'data': data,
+        'metadata': {'payload_bytes': len(payload_bytes)} if transport == 'direct' else {}
+    }
+
+
+def _publish(subject, message, correlation_id):
+    """
+    Publish message to NATS.
+    
+    Note:
+        This is a simplified implementation for MicroPython.
+    
+    Args:
+        subject: NATS subject to publish to
+        message: JSON message to publish
+        correlation_id: Correlation ID for logging
+    """
+    log_trace(correlation_id, f"Publishing to {subject}")
+    # Placeholder - actual implementation would use NATSClient
+    # client = NATSClient()
+    # client.connect()
+    # client.publish(subject, message)
+    # client.close()
+
+
+def smartsend(subject, data, **kwargs):
+    """
+    Send data via NATS with automatic transport selection.
+    
+    This function intelligently routes data delivery based on payload size.
+    If the serialized payload is smaller than size_threshold, it encodes the data as Base64
+    and publishes directly over NATS. Otherwise, it uploads the data to a fileserver
+    and publishes only the download URL over NATS.
+    
+    Note:
+        MicroPython has memory constraints, so the default size_threshold is lower (100KB).
+        Table type is not supported due to memory constraints.
+    
+    Args:
+        subject: NATS subject to publish the message to
+        data: List of (dataname, data, type) tuples to send
+            - dataname: Name of the payload
+            - data: The actual data to send
+            - type: Payload type: "text", "dictionary", "image", "audio", "video", "binary"
+        broker_url: NATS server URL (default: DEFAULT_BROKER_URL)
+        fileserver_url: HTTP file server URL (default: DEFAULT_FILESERVER_URL)
+        fileserver_upload_handler: Function to handle fileserver uploads (default: _sync_fileserver_upload)
+        size_threshold: Threshold in bytes separating direct vs link transport (default: 100000)
+        correlation_id: Correlation ID for tracing (auto-generated if not provided)
+        msg_purpose: Purpose of the message (default: "chat")
+        sender_name: Name of the sender (default: "NATSBridge")
+        receiver_name: Name of the receiver (empty means broadcast)
+        receiver_id: UUID of the receiver (empty means broadcast)
+        reply_to: Topic to reply to (empty if no reply expected)
+        reply_to_msg_id: Message ID this message is replying to
+        is_publish: Whether to automatically publish the message (default: True)
+        msg_id: Message ID (auto-generated if not provided)
+        sender_id: Sender ID (auto-generated if not provided)
+    
+    Returns:
+        Tuple of (env, env_json_str) where:
+            - env: Dict containing all metadata and payloads
+            - env_json_str: JSON string for publishing to NATS
+    
+    Example:
+        >>> # Send text payload
+        >>> env, env_json_str = NATSBridge.smartsend(
+        ...     "/chat",
+        ...     [("message", "Hello!", "text")],
+        ...     broker_url="nats://localhost:4222"
+        ... )
+        >>>
+        >>> # Send dictionary payload
+        >>> env, env_json_str = NATSBridge.smartsend(
+        ...     "/config",
+        ...     [("config", {"key": "value"}, "dictionary")],
+        ...     broker_url="nats://localhost:4222"
+        ... )
+        >>>
+        >>> # Send binary payload (image, audio, video)
+        >>> env, env_json_str = NATSBridge.smartsend(
+        ...     "/media",
+        ...     [("image", image_bytes, "image")],
+        ...     broker_url="nats://localhost:4222"
+        ... )
+    """
+    # Extract options with defaults
+    correlation_id = kwargs.get('correlation_id', _generate_uuid())
+    msg_id = kwargs.get('msg_id', _generate_uuid())
+    sender_id = kwargs.get('sender_id', _generate_uuid())
+    broker_url = kwargs.get('broker_url', DEFAULT_BROKER_URL)
+    fileserver_url = kwargs.get('fileserver_url', DEFAULT_FILESERVER_URL)
+    size_threshold = kwargs.get('size_threshold', DEFAULT_SIZE_THRESHOLD)
+    msg_purpose = kwargs.get('msg_purpose', 'chat')
+    sender_name = kwargs.get('sender_name', 'NATSBridge')
+    receiver_name = kwargs.get('receiver_name', '')
+    receiver_id = kwargs.get('receiver_id', '')
+    reply_to = kwargs.get('reply_to', '')
+    reply_to_msg_id = kwargs.get('reply_to_msg_id', '')
+    is_publish = kwargs.get('is_publish', True)
+    fileserver_upload_handler = kwargs.get('fileserver_upload_handler', _sync_fileserver_upload)
+    
+    log_trace(correlation_id, f"Starting smartsend for subject: {subject}")
+    
+    # Process payloads
+    payloads = []
+    for dataname, payload_data, payload_type in data:
+        payload_bytes = _serialize_data(payload_data, payload_type)
+        payload_size = len(payload_bytes)
+        
+        # Check against hard limit for MicroPython
+        if payload_size > MAX_PAYLOAD_SIZE:
+            raise MemoryError(f"Payload '{dataname}' exceeds max size {MAX_PAYLOAD_SIZE} bytes")
+        
+        log_trace(correlation_id, f"Serialized payload '{dataname}' (type: {payload_type}) size: {payload_size} bytes")
+        
+        if payload_size < size_threshold:
+            # Direct path
+            payload_b64 = base64.b64encode(payload_bytes).decode('ascii')
+            log_trace(correlation_id, f"Using direct transport for {payload_size} bytes")
+            
+            payload = _build_payload(dataname, payload_type, payload_bytes, 'direct', payload_b64)
+            payloads.append(payload)
+        else:
+            # Link path (limited support)
+            log_trace(correlation_id, "Using link transport, uploading to fileserver")
+            
+            try:
+                response = fileserver_upload_handler(fileserver_url, dataname, payload_bytes)
+                log_trace(correlation_id, f"Uploaded to URL: {response['url']}")
+                
+                payload = _build_payload(dataname, payload_type, payload_bytes, 'link', response['url'])
+                payloads.append(payload)
+            except NotImplementedError:
+                # Fall back to direct transport if file upload not available
+                log_trace(correlation_id, "File upload not available, using direct transport")
+                payload_b64 = base64.b64encode(payload_bytes).decode('ascii')
+                payload = _build_payload(dataname, payload_type, payload_bytes, 'direct', payload_b64)
+                payloads.append(payload)
+    
+    # Build envelope
+    env = _build_envelope(subject, payloads, {
+        'correlation_id': correlation_id,
+        'msg_id': msg_id,
+        'msg_purpose': msg_purpose,
+        'sender_name': sender_name,
+        'sender_id': sender_id,
+        'receiver_name': receiver_name,
+        'receiver_id': receiver_id,
+        'reply_to': reply_to,
+        'reply_to_msg_id': reply_to_msg_id,
+        'broker_url': broker_url
+    })
+    
+    env_json_str = json.dumps(env)
+    
+    if is_publish:
+        _publish(subject, env_json_str, correlation_id)
+    
+    return env, env_json_str
+
+
+def smartreceive(msg, **kwargs):
+    """
+    Receive and process NATS message.
+    
+    This function processes incoming NATS messages, handling both direct transport
+    (base64 decoded payloads) and link transport (URL-based payloads).
+    It deserializes the data based on the transport type and returns the result.
+    
+    Note:
+        MicroPython has memory constraints, so large payloads should be avoided.
+        Table type is not supported due to memory constraints.
+    
+    Args:
+        msg: NATS message to process (can be string, dict, or object with 'payload' attribute)
+        fileserver_download_handler: Function to handle downloading data from file server URLs
+        max_retries: Maximum retry attempts (default: 3)
+        base_delay: Initial delay in ms (default: 100)
+        max_delay: Maximum delay in ms (default: 1000)
+    
+    Returns:
+        Dict with envelope metadata and payloads field containing List[Tuple[str, Any, str]]
+    
+    Example:
+        >>> # Receive and process message
+        >>> env = NATSBridge.smartreceive(msg, fileserver_download_handler=_sync_fileserver_download)
+        >>> # env is a Dict with "payloads" key containing List[Tuple[str, Any, str]]
+        >>> for dataname, data, type_ in env["payloads"]:
+        ...     print(f"{dataname}: {data} (type: {type_})")
+    """
+    # Parse the JSON envelope
+    if isinstance(msg, dict):
+        # Already parsed
+        env_json_obj = msg
+    elif hasattr(msg, 'payload'):
+        # Object with payload attribute
+        payload = msg.payload if isinstance(msg.payload, str) else msg.payload.decode('utf-8')
+        env_json_obj = json.loads(payload)
+    else:
+        # Assume it's already a JSON string or dict
+        env_json_obj = json.loads(msg) if isinstance(msg, str) else msg
+    
+    correlation_id = env_json_obj['correlation_id']
+    log_trace(correlation_id, "Processing received message")
+    
+    # Process all payloads in the envelope
+    payloads_list = []
+    num_payloads = len(env_json_obj['payloads'])
+    
+    for i in range(num_payloads):
+        payload_obj = env_json_obj['payloads'][i]
+        transport = payload_obj['transport']
+        dataname = payload_obj['dataname']
+        
+        if transport == 'direct':
+            log_trace(correlation_id, f"Direct transport - decoding payload '{dataname}'")
+            
+            # Extract base64 payload from the payload
+            payload_b64 = payload_obj['data']
+            
+            # Decode Base64 payload
+            payload_bytes = base64.b64decode(payload_b64)
+            
+            # Deserialize based on type
+            data_type = payload_obj['payload_type']
+            data = _deserialize_data(payload_bytes, data_type)
+            
+            payloads_list.append((dataname, data, data_type))
+        elif transport == 'link':
+            # Extract download URL from the payload
+            url = payload_obj['data']
+            log_trace(correlation_id, f"Link transport - fetching '{dataname}' from URL: {url}")
+            
+            # Fetch with exponential backoff using the download handler
+            fileserver_download_handler = kwargs.get('fileserver_download_handler', _sync_fileserver_download)
+            max_retries = kwargs.get('max_retries', 3)
+            base_delay = kwargs.get('base_delay', 100)
+            max_delay = kwargs.get('max_delay', 1000)
+            
+            downloaded_data = fileserver_download_handler(
+                url,
+                max_retries,
+                base_delay,
+                max_delay,
+                correlation_id
+            )
+            
+            # Deserialize based on type
+            data_type = payload_obj['payload_type']
+            data = _deserialize_data(downloaded_data, data_type)
+            
+            payloads_list.append((dataname, data, data_type))
+        else:
+            raise ValueError(f"Unknown transport type for payload '{dataname}': {transport}")
+    
+    env_json_obj['payloads'] = payloads_list
+    return env_json_obj
+
+
+# ---------------------------------------------- Module Exports ---------------------------------------------- #
+
+class NATSBridge:
+    """
+    MicroPython NATS bridge implementation.
+    
+    This class provides a convenient interface for NATSBridge functionality,
+    encapsulating the main functions and providing a class-based API.
+    
+    Note:
+        MicroPython has significant constraints:
+        - No Arrow IPC support (memory constraints)
+        - Only direct transport (< 100KB threshold enforced)
+        - Simplified UUID generation
+        - No async/await (synchronous API)
+    """
+    
+    DEFAULT_SIZE_THRESHOLD = DEFAULT_SIZE_THRESHOLD
+    DEFAULT_BROKER_URL = DEFAULT_BROKER_URL
+    DEFAULT_FILESERVER_URL = DEFAULT_FILESERVER_URL
+    MAX_PAYLOAD_SIZE = MAX_PAYLOAD_SIZE
+    
+    def __init__(self, broker_url=None, fileserver_url=None):
+        """
+        Initialize NATSBridge.
+        
+        Args:
+            broker_url: NATS server URL (defaults to DEFAULT_BROKER_URL)
+            fileserver_url: HTTP file server URL (defaults to DEFAULT_FILESERVER_URL)
+        """
+        self.broker_url = broker_url or self.DEFAULT_BROKER_URL
+        self.fileserver_url = fileserver_url or self.DEFAULT_FILESERVER_URL
+    
+    def smartsend(self, subject, data, **kwargs):
+        """
+        Send data via NATS.
+        
+        Args:
+            subject: NATS subject to publish to
+            data: List of (dataname, data, type) tuples
+            **kwargs: Additional options passed to smartsend
+        
+        Returns:
+            Tuple of (env, env_json_str)
+        """
+        kwargs['broker_url'] = kwargs.get('broker_url', self.broker_url)
+        kwargs['fileserver_url'] = kwargs.get('fileserver_url', self.fileserver_url)
+        return smartsend(subject, data, **kwargs)
+    
+    def smartreceive(self, msg, **kwargs):
+        """
+        Receive and process NATS message.
+        
+        Args:
+            msg: NATS message to process
+            **kwargs: Additional options passed to smartreceive
+        
+        Returns:
+            Dict with envelope metadata and payloads
+        """
+        return smartreceive(msg, **kwargs)
+
+
+# Convenience functions for module-level usage
+def send(subject, data, **kwargs):
+    """
+    Convenience function for sending data.
+    
+    Args:
+        subject: NATS subject to publish to
+        data: List of (dataname, data, type) tuples
+        **kwargs: Additional options
+    
+    Returns:
+        Tuple of (env, env_json_str)
+    """
+    return smartsend(subject, data, **kwargs)
+
+
+def receive(msg, **kwargs):
+    """
+    Convenience function for receiving messages.
+    
+    Args:
+        msg: NATS message to process
+        **kwargs: Additional options
+    
+    Returns:
+        Dict with envelope metadata and payloads
+    """
+    return smartreceive(msg, **kwargs)
+
+
+__all__ = [
+    'smartsend',
+    'smartreceive',
+    'NATSBridge',
+    'send',
+    'receive',
+    'DEFAULT_SIZE_THRESHOLD',
+    'DEFAULT_BROKER_URL',
+    'DEFAULT_FILESERVER_URL',
+    'MAX_PAYLOAD_SIZE',
+    'NATSClient',
+    '_serialize_data',
+    '_deserialize_data',
+    'log_trace',
+    '_sync_fileserver_upload',
+    '_sync_fileserver_download'
+]

src/natsbridge_ssr.js

@@ -0,0 +1,798 @@
+/**
+ * NATSBridge - Cross-Platform Bi-Directional Data Bridge
+ * JavaScript/Node.js Implementation (Client-Side Rendering)
+ * 
+ * This module provides functionality for sending and receiving data across network boundaries
+ * using NATS as the message bus, with support for both direct payload transport and
+ * URL-based transport for larger payloads.
+ * 
+ * Supported payload types: "text", "dictionary", "arrowtable", "jsontable", "image", "audio", "video", "binary"
+ * 
+ * @module NATSBridge
+ */
+
+const nats = require('nats');
+const crypto = require('crypto');
+// Use native fetch available in Node.js 18+
+const arrow = require('apache-arrow');
+
+// ---------------------------------------------- UUID Helper ---------------------------------------------- //
+
+/**
+ * Generate UUID v4 using crypto module (Node.js compatible)
+ * @returns {string} UUID string
+ */
+function uuidv4() {
+    return crypto.randomUUID();
+}
+
+// ---------------------------------------------- Constants ---------------------------------------------- //
+
+/**
+ * Default size threshold for switching from direct to link transport (0.5MB)
+ */
+const DEFAULT_SIZE_THRESHOLD = 500_000;
+
+/**
+ * Default NATS server URL
+ */
+const DEFAULT_BROKER_URL = 'nats://localhost:4222';
+
+/**
+ * Default HTTP file server URL for link transport
+ */
+const DEFAULT_FILESERVER_URL = 'http://localhost:8080';
+
+// ---------------------------------------------- Utility Functions ---------------------------------------------- //
+
+/**
+ * Convert Buffer to Base64 string
+ * @param {Buffer} buffer - Buffer to encode
+ * @returns {string} Base64 encoded string
+ */
+function bufferToBase64(buffer) {
+    return buffer.toString('base64');
+}
+
+/**
+ * Log a trace message with correlation ID and timestamp
+ * @param {string} correlationId - Correlation ID for tracing
+ * @param {string} message - Message content to log
+ */
+function logTrace(correlationId, message) {
+    const timestamp = new Date().toISOString();
+    console.log(`[${timestamp}] [Correlation: ${correlationId}] ${message}`);
+}
+
+// ---------------------------------------------- Serialization Functions ---------------------------------------------- //
+
+/**
+ * Serialize data according to specified format
+ * @param {any} data - Data to serialize
+ * @param {string} payloadType - Target format: "text", "dictionary", "arrowtable", "jsontable", "image", "audio", "video", "binary"
+ * @returns {Buffer} Binary representation of the serialized data
+ */
+async function serializeData(data, payloadType) {
+    if (payloadType === 'text') {
+        if (typeof data === 'string') {
+            return Buffer.from(data, 'utf8');
+        } else {
+            throw new Error('Text data must be a string');
+        }
+    } else if (payloadType === 'dictionary') {
+        const jsonStr = JSON.stringify(data);
+        return Buffer.from(jsonStr, 'utf8');
+    } else if (payloadType === 'arrowtable') {
+        // Convert array of objects to Arrow IPC format
+        if (!Array.isArray(data) || data.length === 0) {
+            throw new Error('Arrow table data must be a non-empty array of objects');
+        }
+        
+        return serializeArrowTable(data);
+    } else if (payloadType === 'jsontable') {
+        // Serialize array of objects to JSON format
+        if (!Array.isArray(data)) {
+            throw new Error('JSON table data must be an array');
+        }
+        const jsonStr = JSON.stringify(data);
+        return Buffer.from(jsonStr, 'utf8');
+    } else if (payloadType === 'image') {
+        if (data instanceof Uint8Array || Buffer.isBuffer(data)) {
+            return Buffer.from(data);
+        } else {
+            throw new Error('Image data must be Uint8Array or Buffer');
+        }
+    } else if (payloadType === 'audio') {
+        if (data instanceof Uint8Array || Buffer.isBuffer(data)) {
+            return Buffer.from(data);
+        } else {
+            throw new Error('Audio data must be Uint8Array or Buffer');
+        }
+    } else if (payloadType === 'video') {
+        if (data instanceof Uint8Array || Buffer.isBuffer(data)) {
+            return Buffer.from(data);
+        } else {
+            throw new Error('Video data must be Uint8Array or Buffer');
+        }
+    } else if (payloadType === 'binary') {
+        if (data instanceof Uint8Array || Buffer.isBuffer(data)) {
+            return Buffer.from(data);
+        } else {
+            throw new Error('Binary data must be Uint8Array or Buffer');
+        }
+    } else {
+        throw new Error(`Unknown payload_type: ${payloadType}`);
+    }
+}
+
+/**
+ * Helper function to properly serialize table data to Arrow IPC
+ * @param {Array<Object>} data - Array of objects representing table rows
+ * @returns {Buffer} Arrow IPC formatted buffer
+ */
+function serializeArrowTable(data) {
+    if (!Array.isArray(data) || data.length === 0) {
+        throw new Error('Table data must be a non-empty array of objects');
+    }
+    
+    logTrace('serializeArrowTable', `Serializing table with ${data.length} rows`);
+    
+    // Use arrow.tableFromArrays which handles the conversion properly
+    // Convert array of objects to a key-value format expected by tableFromArrays
+    const columns = {};
+    for (const key of Object.keys(data[0])) {
+        columns[key] = data.map(row => row[key]);
+    }
+    
+    logTrace('serializeArrowTable', `Columns: ${Object.keys(columns).join(', ')}`);
+    
+    const table = arrow.tableFromArrays(columns);
+    
+    logTrace('serializeArrowTable', `Arrow table created with ${table.numRows} rows, ${table.numCols} cols`);
+    
+    // Convert to IPC format
+    const ipcBuffer = arrow.tableToIPC(table);
+    
+    logTrace('serializeArrowTable', `IPC buffer type: ${typeof ipcBuffer}, length: ${ipcBuffer.byteLength}`);
+    
+    const resultBuffer = Buffer.from(ipcBuffer);
+    logTrace('serializeArrowTable', `Result buffer: ${resultBuffer.length} bytes`);
+    
+    // Debug: Show first 20 bytes in hex
+    const hexPreview = resultBuffer.slice(0, 20).toString('hex');
+    logTrace('serializeArrowTable', `First 20 bytes (hex): ${hexPreview}`);
+    
+    return resultBuffer;
+}
+
+/**
+ * Deserialize bytes to data based on type
+ * @param {Buffer|Uint8Array} data - Serialized data as bytes
+ * @param {string} payloadType - Data type
+ * @param {string} correlationId - Correlation ID for logging
+ * @returns {any} Deserialized data
+ */
+async function deserializeData(data, payloadType, correlationId) {
+    const buffer = Buffer.isBuffer(data) ? data : Buffer.from(data);
+    
+    logTrace(correlationId, `deserializeData: type=${payloadType}, bufferLength=${buffer.length}`);
+    
+    // Debug: Show first 20 bytes in hex for binary data
+    if (payloadType === 'arrowtable' || payloadType === 'jsontable' || payloadType === 'image' || payloadType === 'binary') {
+        const hexPreview = buffer.slice(0, 20).toString('hex');
+        logTrace(correlationId, `deserializeData: First 20 bytes (hex): ${hexPreview}`);
+    }
+    
+    if (payloadType === 'text') {
+        const result = buffer.toString('utf8');
+        logTrace(correlationId, `deserializeData: text result length=${result.length}`);
+        return result;
+    } else if (payloadType === 'dictionary') {
+        const jsonStr = buffer.toString('utf8');
+        const result = JSON.parse(jsonStr);
+        logTrace(correlationId, `deserializeData: dictionary keys=${Object.keys(result).join(', ')}`);
+        return result;
+    } else if (payloadType === 'arrowtable') {
+        logTrace(correlationId, `deserializeData: Attempting Arrow table deserialization`);
+        
+        // Debug: Check available arrow methods
+        logTrace(correlationId, `deserializeData: arrow.tableFromRawBytes exists: ${typeof arrow.tableFromRawBytes}`);
+        logTrace(correlationId, `deserializeData: arrow.tableFromIPC exists: ${typeof arrow.tableFromIPC}`);
+        
+        try {
+            // Try tableFromRawBytes first (older API)
+            if (typeof arrow.tableFromRawBytes === 'function') {
+                logTrace(correlationId, `deserializeData: Using tableFromRawBytes`);
+                const table = arrow.tableFromRawBytes(buffer);
+                logTrace(correlationId, `deserializeData: Arrow table - rows=${table.numRows}, cols=${table.numCols}`);
+                return table;
+            }
+        } catch (e) {
+            logTrace(correlationId, `deserializeData: tableFromRawBytes failed: ${e.message}`);
+        }
+        
+        try {
+            // Try tableFromIPC (newer API)
+            if (typeof arrow.tableFromIPC === 'function') {
+                logTrace(correlationId, `deserializeData: Using tableFromIPC`);
+                const table = arrow.tableFromIPC(buffer);
+                logTrace(correlationId, `deserializeData: Arrow table from IPC - rows=${table.numRows}, cols=${table.numCols}`);
+                return table;
+            }
+        } catch (e) {
+            logTrace(correlationId, `deserializeData: tableFromIPC failed: ${e.message}`);
+        }
+        
+        throw new Error(`Unable to deserialize Arrow table: neither tableFromRawBytes nor tableFromIPC worked`);
+    } else if (payloadType === 'jsontable') {
+        const jsonStr = buffer.toString('utf8');
+        const result = JSON.parse(jsonStr);
+        logTrace(correlationId, `deserializeData: jsontable result length=${Array.isArray(result) ? result.length : 'N/A'}`);
+        return result;
+    } else if (payloadType === 'image') {
+        logTrace(correlationId, `deserializeData: image buffer length=${buffer.length}`);
+        return buffer;
+    } else if (payloadType === 'audio') {
+        logTrace(correlationId, `deserializeData: audio buffer length=${buffer.length}`);
+        return buffer;
+    } else if (payloadType === 'video') {
+        logTrace(correlationId, `deserializeData: video buffer length=${buffer.length}`);
+        return buffer;
+    } else if (payloadType === 'binary') {
+        logTrace(correlationId, `deserializeData: binary buffer length=${buffer.length}`);
+        return buffer;
+    } else {
+        throw new Error(`Unknown payload_type: ${payloadType}`);
+    }
+}
+
+// ---------------------------------------------- File Server Handlers ---------------------------------------------- //
+
+/**
+ * Upload data to plik server in one-shot mode
+ * @param {string} fileServerUrl - Base URL of the plik server
+ * @param {string} dataname - Name of the file being uploaded
+ * @param {Buffer|Uint8Array} data - Raw byte data of the file content
+ * @returns {Promise<{status: number, uploadid: string, fileid: string, url: string}>}
+ */
+async function plikOneshotUpload(fileServerUrl, dataname, data) {
+    const buffer = Buffer.isBuffer(data) ? data : Buffer.from(data);
+    
+    // Get upload id
+    const urlGetUploadID = `${fileServerUrl}/upload`;
+    const headers = { 'Content-Type': 'application/json' };
+    const body = JSON.stringify({ OneShot: true });
+    
+    const httpResponse = await fetch(urlGetUploadID, {
+        method: 'POST',
+        headers,
+        body
+    });
+    
+    const responseJson = await httpResponse.json();
+    const uploadid = responseJson.id;
+    const uploadtoken = responseJson.uploadToken;
+
+    // Upload file
+    const urlUpload = `${fileServerUrl}/file/${uploadid}`;
+    const form = new FormData();
+    const blob = new Blob([buffer], { type: 'application/octet-stream' });
+    form.append('file', blob, dataname);
+    
+    const uploadHeaders = {
+        'X-UploadToken': uploadtoken
+    };
+
+    const uploadResponse = await fetch(urlUpload, {
+        method: 'POST',
+        headers: uploadHeaders,
+        body: form
+    });
+    
+    const uploadJson = await uploadResponse.json();
+    const fileid = uploadJson.id;
+
+    const url = `${fileServerUrl}/file/${uploadid}/${fileid}/${dataname}`;
+
+    return {
+        status: uploadResponse.status,
+        uploadid,
+        fileid,
+        url
+    };
+}
+
+/**
+ * Fetch data from URL with exponential backoff
+ * @param {string} url - URL to fetch from
+ * @param {number} maxRetries - Maximum number of retry attempts
+ * @param {number} baseDelay - Initial delay in milliseconds
+ * @param {number} maxDelay - Maximum delay in milliseconds
+ * @param {string} correlationId - Correlation ID for logging
+ * @returns {Promise<Uint8Array>} Fetched data as bytes
+ */
+async function fetchWithBackoff(url, maxRetries, baseDelay, maxDelay, correlationId) {
+    let delay = baseDelay;
+    
+    for (let attempt = 1; attempt <= maxRetries; attempt++) {
+        try {
+            const response = await fetch(url);
+            
+            if (response.status === 200) {
+                logTrace(correlationId, `Successfully fetched data from ${url} on attempt ${attempt}`);
+                const arrayBuffer = await response.arrayBuffer();
+                return new Uint8Array(arrayBuffer);
+            } else {
+                throw new Error(`Failed to fetch: ${response.status}`);
+            }
+        } catch (e) {
+            logTrace(correlationId, `Attempt ${attempt} failed: ${e.constructor.name} - ${e.message}`);
+            
+            if (attempt < maxRetries) {
+                await new Promise(resolve => setTimeout(resolve, delay));
+                delay = Math.min(delay * 2, maxDelay);
+            }
+        }
+    }
+    
+    throw new Error(`Failed to fetch data after ${maxRetries} attempts`);
+}
+
+// ---------------------------------------------- NATS Client ---------------------------------------------- //
+
+/**
+ * NATS client wrapper for connection management
+ */
+class NATSClient {
+    /**
+     * Create a new NATS client
+     * @param {string} url - NATS server URL
+     */
+    constructor(url) {
+        this.url = url;
+        this.connection = null;
+    }
+
+    /**
+     * Connect to NATS server
+     * @returns {Promise<NATS.Connection>}
+     */
+    async connect() {
+        this.connection = await nats.connect({ servers: this.url });
+        return this.connection;
+    }
+
+    /**
+     * Publish message to NATS subject
+     * @param {string} subject - NATS subject to publish to
+     * @param {string} message - Message to publish
+     * @param {string} correlationId - Correlation ID for logging
+     */
+    async publish(subject, message, correlationId) {
+        if (!this.connection) {
+            await this.connect();
+        }
+        await this.connection.publish(subject, message);
+        logTrace(correlationId, `Message published to ${subject}`);
+    }
+
+    /**
+     * Close the NATS connection
+     */
+    async close() {
+        if (this.connection) {
+            this.connection.close();
+        }
+    }
+}
+
+// ---------------------------------------------- Core Functions ---------------------------------------------- //
+
+/**
+ * Publish message to NATS
+ * @param {string|NATSClient|NATS.Connection} brokerUrlOrClient - NATS URL, client, or connection
+ * @param {string} subject - NATS subject to publish to
+ * @param {string} message - JSON message to publish
+ * @param {string} correlationId - Correlation ID for tracing
+ */
+async function publishMessage(brokerUrlOrClient, subject, message, correlationId) {
+    let conn;
+    
+    if (brokerUrlOrClient instanceof NATSClient) {
+        conn = brokerUrlOrClient;
+    } else if (brokerUrlOrClient && typeof brokerUrlOrClient.publish === 'function') {
+        // Create a wrapper for direct connection (duck-typing check for NATS connection)
+        conn = {
+            async publish(subj, msg) {
+                await brokerUrlOrClient.publish(subj, msg);
+            },
+            async close() {
+                await brokerUrlOrClient.close();
+            }
+        };
+    } else {
+        // String URL - create new client
+        const client = new NATSClient(brokerUrlOrClient);
+        conn = client;
+    }
+    
+    await conn.publish(subject, message, correlationId);
+    
+    if (conn instanceof NATSClient) {
+        await conn.close();
+    }
+}
+
+/**
+ * Build message envelope from payloads and metadata
+ * @param {string} subject - NATS subject
+ * @param {Array} payloads - Array of payload objects
+ * @param {Object} options - Envelope metadata options
+ * @returns {Object} Envelope object
+ */
+function buildEnvelope(subject, payloads, options) {
+    return {
+        correlation_id: options.correlation_id,
+        msg_id: options.msg_id,
+        timestamp: new Date().toISOString(),
+        send_to: subject,
+        msg_purpose: options.msg_purpose,
+        sender_name: options.sender_name,
+        sender_id: options.sender_id,
+        receiver_name: options.receiver_name,
+        receiver_id: options.receiver_id,
+        reply_to: options.reply_to,
+        reply_to_msg_id: options.reply_to_msg_id,
+        broker_url: options.broker_url,
+        metadata: options.metadata || {},
+        payloads: payloads
+    };
+}
+
+/**
+ * Build payload object from serialized data
+ * @param {string} dataname - Name of the payload
+ * @param {string} payloadType - Type of the payload
+ * @param {Buffer} payloadBytes - Serialized payload bytes
+ * @param {string} transport - Transport type ("direct" or "link")
+ * @param {string} data - Data (base64 for direct, URL for link)
+ * @returns {Object} Payload object
+ */
+function buildPayload(dataname, payloadType, payloadBytes, transport, data) {
+    // Determine encoding based on payload type (matching Julia implementation)
+    let encoding = 'base64';
+    if (payloadType === 'jsontable') {
+        encoding = 'json';
+    } else if (payloadType === 'arrowtable') {
+        encoding = 'arrow-ipc';
+    }
+    
+    return {
+        id: uuidv4(),
+        dataname,
+        payload_type: payloadType,
+        transport,
+        encoding,
+        size: payloadBytes.byteLength,
+        data,
+        metadata: transport === 'direct' ? { payload_bytes: payloadBytes.byteLength } : {}
+    };
+}
+
+/**
+ * Send data via NATS with automatic transport selection
+ * 
+ * This function intelligently routes data delivery based on payload size.
+ * If the serialized payload is smaller than size_threshold, it encodes the data as Base64
+ * and publishes directly over NATS. Otherwise, it uploads the data to a fileserver
+ * and publishes only the download URL over NATS.
+ * 
+ * @param {string} subject - NATS subject to publish the message to
+ * @param {Array} data - List of [dataname, data, type] tuples to send
+ *          - type: "text", "dictionary", "arrowtable", "jsontable", "image", "audio", "video", "binary"
+ * @param {Object} options - Optional configuration
+ * @param {string} [options.broker_url=DEFAULT_BROKER_URL] - URL of the NATS server
+ * @param {string} [options.fileserver_url=DEFAULT_FILESERVER_URL] - URL of the HTTP file server
+ * @param {Function} [options.fileserver_upload_handler=plikOneshotUpload] - Function to handle fileserver uploads
+ * @param {number} [options.size_threshold=DEFAULT_SIZE_THRESHOLD] - Threshold separating direct vs link transport
+ * @param {string} [options.correlation_id=crypto.randomUUID()] - Correlation ID for tracing
+ * @param {string} [options.msg_purpose="chat"] - Purpose of the message
+ * @param {string} [options.sender_name="NATSBridge"] - Name of the sender
+ * @param {string} [options.receiver_name=""] - Name of the receiver (empty means broadcast)
+ * @param {string} [options.receiver_id=""] - UUID of the receiver (empty means broadcast)
+ * @param {string} [options.reply_to=""] - Topic to reply to
+ * @param {string} [options.reply_to_msg_id=""] - Message ID this message is replying to
+ * @param {boolean} [options.is_publish=true] - Whether to automatically publish the message
+ * @param {NATSClient|NATS.Connection} [options.nats_connection=null] - Pre-existing NATS connection
+ * @param {string} [options.msg_id=crypto.randomUUID()] - Message ID
+ * @param {string} [options.sender_id=crypto.randomUUID()] - Sender ID
+ * @returns {Promise<[Object, string]>} Tuple of [env, env_json_str]
+ * 
+ * @example
+ * // Send a single payload
+ * const [env, envJsonStr] = await smartsend(
+ *     "/test",
+ *     [["dataname1", data1, "dictionary"]],
+ *     { broker_url: "nats://localhost:4222" }
+ * );
+ * 
+ * // Send multiple payloads
+ * const [env, envJsonStr] = await smartsend(
+ *     "/test",
+ *     [
+ *         ["dataname1", data1, "dictionary"],
+ *         ["dataname2", data2, "arrowtable"]
+ *     ],
+ *     { broker_url: "nats://localhost:4222" }
+ * );
+ * 
+ * // Send with pre-existing connection
+ * const client = await NATSBridge.NATSClient.connect("nats://localhost:4222");
+ * const [env, envJsonStr] = await smartsend(
+ *     "/test",
+ *     [["data", myData, "text"]],
+ *     { nats_connection: client }
+ * );
+ */
+async function smartsend(subject, data, options = {}) {
+    const {
+        broker_url = DEFAULT_BROKER_URL,
+        fileserver_url = DEFAULT_FILESERVER_URL,
+        fileserver_upload_handler = plikOneshotUpload,
+        size_threshold = DEFAULT_SIZE_THRESHOLD,
+        correlation_id = uuidv4(),
+        msg_purpose = 'chat',
+        sender_name = 'NATSBridge',
+        receiver_name = '',
+        receiver_id = '',
+        reply_to = '',
+        reply_to_msg_id = '',
+        is_publish = true,
+        nats_connection = null,
+        msg_id = uuidv4(),
+        sender_id = uuidv4()
+    } = options;
+
+    logTrace(correlation_id, `Starting smartsend for subject: ${subject}`);
+    logTrace(correlation_id, `smartsend: data array length=${data.length}`);
+    
+    // Debug: Log input data structure
+    for (let i = 0; i < data.length; i++) {
+        const [dataname, payloadData, payloadType] = data[i];
+        logTrace(correlation_id, `smartsend: payload[${i}] dataname=${dataname}, type=${payloadType}, data type=${typeof payloadData}, constructor=${payloadData?.constructor?.name}`);
+    }
+
+    // Process payloads
+    const payloads = [];
+    for (const [dataname, payloadData, payloadType] of data) {
+        logTrace(correlation_id, `smartsend: Processing payload '${dataname}' type=${payloadType}`);
+        logTrace(correlation_id, `smartsend: payloadData type=${typeof payloadData}, constructor=${payloadData?.constructor?.name}`);
+        
+        const payloadBytes = await serializeData(payloadData, payloadType);
+        const payloadSize = payloadBytes.byteLength;
+
+        logTrace(correlation_id, `Serialized payload '${dataname}' (type: ${payloadType}) size: ${payloadSize} bytes`);
+
+        // Debug: Show first 20 bytes of serialized data for table type
+        if (payloadType === 'table') {
+            const hexPreview = payloadBytes.slice(0, 20).toString('hex');
+            logTrace(correlation_id, `Serialized table data first 20 bytes (hex): ${hexPreview}`);
+        }
+
+        if (payloadSize < size_threshold) {
+            // Direct path
+            const payloadB64 = bufferToBase64(payloadBytes);
+            logTrace(correlation_id, `Using direct transport for ${payloadSize} bytes, base64 length=${payloadB64.length}`);
+
+            const payload = buildPayload(dataname, payloadType, payloadBytes, 'direct', payloadB64);
+            payloads.push(payload);
+        } else {
+            // Link path
+            logTrace(correlation_id, `Using link transport, uploading to fileserver`);
+
+            const response = await fileserver_upload_handler(fileserver_url, dataname, payloadBytes);
+
+            if (response.status !== 200) {
+                throw new Error(`Failed to upload data to fileserver: ${response.status}`);
+            }
+
+            logTrace(correlation_id, `Uploaded to URL: ${response.url}`);
+
+            const payload = buildPayload(dataname, payloadType, payloadBytes, 'link', response.url);
+            payloads.push(payload);
+        }
+    }
+
+    // Build envelope
+    const env = buildEnvelope(subject, payloads, {
+        correlation_id,
+        msg_id,
+        msg_purpose,
+        sender_name,
+        sender_id,
+        receiver_name,
+        receiver_id,
+        reply_to,
+        reply_to_msg_id,
+        broker_url
+    });
+
+    const env_json_str = JSON.stringify(env);
+
+    if (is_publish) {
+        if (nats_connection) {
+            await publishMessage(nats_connection, subject, env_json_str, correlation_id);
+        } else {
+            await publishMessage(broker_url, subject, env_json_str, correlation_id);
+        }
+    }
+
+    return [env, env_json_str];
+}
+
+/**
+ * Receive and process NATS message
+ * 
+ * This function processes incoming NATS messages, handling both direct transport
+ * (base64 decoded payloads) and link transport (URL-based payloads).
+ * It deserializes the data based on the transport type and returns the result.
+ * 
+ * @param {Object} msg - NATS message object with payload property
+ * @param {Object} options - Optional configuration
+ * @param {Function} [options.fileserver_download_handler=fetchWithBackoff] - Function to handle fileserver downloads
+ * @param {number} [options.max_retries=5] - Maximum retry attempts for fetching URL
+ * @param {number} [options.base_delay=100] - Initial delay for exponential backoff in ms
+ * @param {number} [options.max_delay=5000] - Maximum delay for exponential backoff in ms
+ * @returns {Promise<Object>} Envelope object with processed payloads
+ * 
+ * @example
+ * // Receive and process message
+ * const env = await smartreceive(msg, {
+ *     fileserver_download_handler: fetchWithBackoff,
+ *     max_retries: 5,
+ *     base_delay: 100,
+ *     max_delay: 5000
+ * });
+ * // env.payloads is an Array of [dataname, data, type] arrays
+ * for (const [dataname, data, type] of env.payloads) {
+ *     console.log(`${dataname}: ${data} (type: ${type})`);
+ * }
+ */
+async function smartreceive(msg, options = {}) {
+    const {
+        fileserver_download_handler = fetchWithBackoff,
+        max_retries = 5,
+        base_delay = 100,
+        max_delay = 5000
+    } = options;
+
+    // Debug: Log message object structure
+    logTrace('smartreceive', `smartreceive: msg object keys: ${Object.keys(msg).join(', ')}`);
+    logTrace('smartreceive', `smartreceive: msg.data type: ${typeof msg.data}, constructor: ${msg.data?.constructor?.name}`);
+    logTrace('smartreceive', `smartreceive: msg.payload type: ${typeof msg.payload}, constructor: ${msg.payload?.constructor?.name}`);
+    
+    // Parse the JSON envelope
+    // NATS.js v2.x uses msg.data instead of msg.payload
+    let payload;
+    if (msg.data !== undefined) {
+        payload = typeof msg.data === 'string' ? msg.data : Buffer.from(msg.data).toString('utf8');
+    } else if (msg.payload !== undefined) {
+        payload = typeof msg.payload === 'string' ? msg.payload : Buffer.from(msg.payload).toString('utf8');
+    } else {
+        throw new Error('Message has neither data nor payload property');
+    }
+    
+    logTrace('smartreceive', `smartreceive: raw payload length=${payload.length}`);
+    
+    // Debug: Show first 200 chars of payload
+    const payloadPreview = payload.substring(0, 200);
+    logTrace('smartreceive', `smartreceive: payload preview: ${payloadPreview}`);
+    
+    let envJsonObj;
+    try {
+        envJsonObj = JSON.parse(payload);
+    } catch (e) {
+        logTrace('smartreceive', `smartreceive: JSON parse failed: ${e.message}`);
+        throw e;
+    }
+    
+    logTrace(envJsonObj.correlation_id, 'Processing received message');
+    logTrace(envJsonObj.correlation_id, `smartreceive: envelope has ${envJsonObj.payloads.length} payloads`);
+
+    // Process all payloads in the envelope
+    const payloadsList = [];
+    const numPayloads = envJsonObj.payloads.length;
+
+    logTrace(envJsonObj.correlation_id, `smartreceive: Processing ${numPayloads} payloads`);
+
+    for (let i = 0; i < numPayloads; i++) {
+        const payloadObj = envJsonObj.payloads[i];
+        const transport = payloadObj.transport;
+        const dataname = payloadObj.dataname;
+        const payloadType = payloadObj.payload_type;
+
+        logTrace(envJsonObj.correlation_id, `smartreceive: Processing payload ${i + 1}/${numPayloads}: dataname=${dataname}, type=${payloadType}, transport=${transport}`);
+
+        if (transport === 'direct') {
+            logTrace(envJsonObj.correlation_id, `Direct transport - decoding payload '${dataname}'`);
+
+            // Extract base64 payload from the payload
+            const payloadB64 = payloadObj.data;
+            logTrace(envJsonObj.correlation_id, `Direct transport: base64 length=${payloadB64?.length}`);
+
+            // Decode Base64 payload
+            const payloadBytes = Buffer.from(payloadB64, 'base64');
+            logTrace(envJsonObj.correlation_id, `Direct transport: decoded bytes=${payloadBytes.length}`);
+
+            // Deserialize based on type
+            const dataType = payloadObj.payload_type;
+            const data = await deserializeData(payloadBytes, dataType, envJsonObj.correlation_id);
+            logTrace(envJsonObj.correlation_id, `Direct transport: deserialized data type=${typeof data}, constructor=${data?.constructor?.name}`);
+
+            payloadsList.push([dataname, data, dataType]);
+        } else if (transport === 'link') {
+            // Extract download URL from the payload
+            const url = payloadObj.data;
+            logTrace(envJsonObj.correlation_id, `Link transport - fetching '${dataname}' from URL: ${url}`);
+
+            // Fetch with exponential backoff using the download handler
+            const downloadedData = await fileserver_download_handler(
+                url,
+                max_retries,
+                base_delay,
+                max_delay,
+                envJsonObj.correlation_id
+            );
+
+            // Deserialize based on type
+            const dataType = payloadObj.payload_type;
+            const data = await deserializeData(downloadedData, dataType, envJsonObj.correlation_id);
+
+            payloadsList.push([dataname, data, dataType]);
+        } else {
+            throw new Error(`Unknown transport type for payload '${dataname}': ${transport}`);
+        }
+    }
+
+    logTrace(envJsonObj.correlation_id, `smartreceive: Successfully processed all ${payloadsList.length} payloads`);
+    envJsonObj.payloads = payloadsList;
+    return envJsonObj;
+}
+
+// ---------------------------------------------- Module Exports ---------------------------------------------- //
+
+const NATSBridge = {
+    /**
+     * NATS client class for connection management
+     */
+    NATSClient,
+
+    /**
+     * Send data via NATS with automatic transport selection
+     */
+    smartsend,
+
+    /**
+     * Receive and process NATS message
+     */
+    smartreceive,
+
+    /**
+     * Upload data to plik server in one-shot mode
+     */
+    plikOneshotUpload,
+
+    /**
+     * Fetch data from URL with exponential backoff
+     */
+    fetchWithBackoff,
+
+    /**
+     * Default constants
+     */
+    DEFAULT_SIZE_THRESHOLD,
+    DEFAULT_BROKER_URL,
+    DEFAULT_FILESERVER_URL
+};
+
+module.exports = NATSBridge;

test/scenario1_command_control.jl

@@ -1,67 +0,0 @@
-#!/usr/bin/env julia
-# Scenario 1: Command & Control (Small JSON)
-# Tests small JSON payloads (< 1MB) sent directly via NATS
-
-using NATS
-using JSON3
-using UUIDs
-
-# Include the bridge module
-include("../src/julia_bridge.jl")
-using .BiDirectionalBridge
-
-# Configuration
-const CONTROL_SUBJECT = "control"
-const RESPONSE_SUBJECT = "control_response"
-const NATS_URL = "nats://localhost:4222"
-
-# Create correlation ID for tracing
-correlation_id = string(uuid4())
-
-# Receiver: Listen for control commands
-function start_control_listener()
-    conn = NATS.Connection(NATS_URL)
-    try
-        NATS.subscribe(conn, CONTROL_SUBJECT) do msg
-            log_trace(msg.data)
-            
-            # Parse the envelope
-            env = MessageEnvelope(String(msg.data))
-            
-            # Parse JSON payload
-            config = JSON3.read(env.payload)
-            
-            # Execute simulation with parameters
-            step_size = config.step_size
-            iterations = config.iterations
-            
-            # Simulate processing
-            sleep(0.1)  # Simulate some work
-            
-            # Send acknowledgment
-            response = Dict(
-                "status" => "Running",
-                "correlation_id" => env.correlation_id,
-                "step_size" => step_size,
-                "iterations" => iterations
-            )
-            
-            NATS.publish(conn, RESPONSE_SUBJECT, JSON3.stringify(response))
-            log_trace("Sent response: $(JSON3.stringify(response))")
-        end
-        
-        # Keep listening for 5 seconds
-        sleep(5)
-    finally
-        NATS.close(conn)
-    end
-end
-
-# Helper: Log with correlation ID
-function log_trace(message)
-    timestamp = Dates.now()
-    println("[$timestamp] [Correlation: $correlation_id] $message")
-end
-
-# Run the listener
-start_control_listener()

test/scenario1_command_control.js

@@ -1,34 +0,0 @@
-#!/usr/bin/env node
-// Scenario 1: Command & Control (Small JSON)
-// Tests small JSON payloads (< 1MB) sent directly via NATS
-
-const { SmartSend } = require('../js_bridge');
-
-// Configuration
-const CONTROL_SUBJECT = "control";
-const NATS_URL = "nats://localhost:4222";
-
-// Create correlation ID for tracing
-const correlationId = require('uuid').v4();
-
-// Sender: Send control command to Julia
-async function sendControlCommand() {
-    const config = {
-        step_size: 0.01,
-        iterations: 1000
-    };
-
-    // Send via SmartSend with type="json"
-    const env = await SmartSend(
-        CONTROL_SUBJECT,
-        config,
-        "json",
-        { correlationId }
-    );
-
-    console.log(`Sent control command with correlation_id: ${correlationId}`);
-    console.log(`Envelope: ${JSON.stringify(env, null, 2)}`);
-}
-
-// Run the sender
-sendControlCommand().catch(console.error);

test/scenario2_large_table.jl

@@ -1,66 +0,0 @@
-#!/usr/bin/env julia
-# Scenario 2: Deep Dive Analysis (Large Arrow Table)
-# Tests large Arrow tables (> 1MB) sent via HTTP fileserver
-
-using NATS
-using Arrow
-using DataFrames
-using JSON3
-using UUIDs
-
-# Include the bridge module
-include("../src/julia_bridge.jl")
-using .BiDirectionalBridge
-
-# Configuration
-const ANALYSIS_SUBJECT = "analysis_results"
-const RESPONSE_SUBJECT = "analysis_response"
-const NATS_URL = "nats://localhost:4222"
-
-# Create correlation ID for tracing
-correlation_id = string(uuid4())
-
-# Receiver: Listen for analysis results
-function start_analysis_listener()
-    conn = NATS.Connection(NATS_URL)
-    try
-        NATS.subscribe(conn, ANALYSIS_SUBJECT) do msg
-            log_trace("Received message from $(msg.subject)")
-            
-            # Parse the envelope
-            env = MessageEnvelope(String(msg.data))
-            
-            # Use SmartReceive to handle the data
-            result = SmartReceive(msg)
-            
-            # Process the data based on type
-            if result.envelope.type == "table"
-                df = result.data
-                log_trace("Received DataFrame with $(nrows(df)) rows")
-                log_trace("DataFrame columns: $(names(df))")
-                
-                # Send acknowledgment
-                response = Dict(
-                    "status" => "Processed",
-                    "correlation_id" => env.correlation_id,
-                    "row_count" => nrows(df)
-                )
-                NATS.publish(conn, RESPONSE_SUBJECT, JSON3.stringify(response))
-            end
-        end
-        
-        # Keep listening for 10 seconds
-        sleep(10)
-    finally
-        NATS.close(conn)
-    end
-end
-
-# Helper: Log with correlation ID
-function log_trace(message)
-    timestamp = Dates.now()
-    println("[$timestamp] [Correlation: $correlation_id] $message")
-end
-
-# Run the listener
-start_analysis_listener()

test/scenario2_large_table.js

@@ -1,54 +0,0 @@
-#!/usr/bin/env node
-// Scenario 2: Deep Dive Analysis (Large Arrow Table)
-// Tests large Arrow tables (> 1MB) sent via HTTP fileserver
-
-const { SmartSend } = require('../js_bridge');
-
-// Configuration
-const ANALYSIS_SUBJECT = "analysis_results";
-const NATS_URL = "nats://localhost:4222";
-
-// Create correlation ID for tracing
-const correlationId = require('uuid').v4();
-
-// Sender: Send large Arrow table to Julia
-async function sendLargeTable() {
-    // Create a large DataFrame-like structure (10 million rows)
-    // For testing, we'll create a smaller but still large table
-    const numRows = 1000000; // 1 million rows
-    
-    const data = {
-        id: Array.from({ length: numRows }, (_, i) => i + 1),
-        value: Array.from({ length: numRows }, () => Math.random()),
-        category: Array.from({ length: numRows }, () => ['A', 'B', 'C'][Math.floor(Math.random() * 3)])
-    };
-    
-    // Convert to Arrow Table
-    const { Table, Vector, RecordBatch } = require('apache-arrow');
-    
-    const idVector = Vector.from(data.id);
-    const valueVector = Vector.from(data.value);
-    const categoryVector = Vector.from(data.category);
-    
-    const table = Table.from({
-        id: idVector,
-        value: valueVector,
-        category: categoryVector
-    });
-    
-    // Send via SmartSend with type="table"
-    const env = await SmartSend(
-        ANALYSIS_SUBJECT,
-        table,
-        "table",
-        { correlationId }
-    );
-    
-    console.log(`Sent large table with ${numRows} rows`);
-    console.log(`Correlation ID: ${correlationId}`);
-    console.log(`Transport: ${env.transport}`);
-    console.log(`URL: ${env.url || 'N/A'}`);
-}
-
-// Run the sender
-sendLargeTable().catch(console.error);

test/test_js_mix_payloads_receiver.js

@@ -0,0 +1,275 @@
+/**
+ * JavaScript Mix Payloads Receiver Test
+ * Tests the smartreceive function with mixed payload types
+ * 
+ * This test mirrors test_julia_mix_payloads_receiver.jl and demonstrates that
+ * any combination and any number of mixed content can be received correctly.
+ */
+
+const NATSBridge = require('../src/natsbridge.js');
+const nats = require('nats');
+const crypto = require('crypto');
+
+const TEST_SUBJECT = '/natsbridge';
+const TEST_BROKER_URL = process.env.NATS_URL || 'nats.yiem.cc';
+const TEST_FILESERVER_URL = process.env.FILESERVER_URL || 'http://192.168.88.104:8080';
+
+async function runTest() {
+    console.log('=== JavaScript Mix Payloads Receiver Test ===\n');
+
+    const correlationId = crypto.randomUUID();
+    console.log(`Correlation ID: ${correlationId}`);
+    console.log(`Subject: ${TEST_SUBJECT}`);
+    console.log(`Broker URL: ${TEST_BROKER_URL}`);
+    console.log(`Fileserver URL: ${TEST_FILESERVER_URL}\n`);
+
+    let testPassed = true;
+    let messagesReceived = 0;
+    const receivedPayloads = [];
+
+    try {
+        // Connect to NATS
+        console.log('Connecting to NATS server...');
+        const nc = await nats.connect({ servers: TEST_BROKER_URL });
+        console.log('✅ Connected to NATS server\n');
+
+        // Set up message subscription
+        const subscription = nc.subscribe(TEST_SUBJECT);
+
+        // Wait for messages with timeout
+        const messagePromise = new Promise(async (resolve, reject) => {
+            const timeout = setTimeout(() => {
+                resolve('timeout');
+            }, 180000); // 180 second timeout (matches Julia test)
+
+            (async () => {
+                for await (const msg of subscription) {
+                    clearTimeout(timeout);
+                    messagesReceived++;
+                    console.log(`\n=== Message ${messagesReceived} Received ===`);
+                    console.log(`Received message on ${msg.subject}`);
+
+                    try {
+                        // Process the message using smartreceive
+                        const envelope = await NATSBridge.smartreceive(msg, {
+                            fileserver_download_handler: NATSBridge.fetchWithBackoff,
+                            max_retries: 5,
+                            base_delay: 100,
+                            max_delay: 5000
+                        });
+
+                        console.log(`Correlation ID: ${envelope.correlation_id}`);
+                        console.log(`Message ID: ${envelope.msg_id}`);
+                        console.log(`Timestamp: ${envelope.timestamp}`);
+                        console.log(`Purpose: ${envelope.msg_purpose}`);
+                        console.log(`Sender: ${envelope.sender_name}`);
+                        console.log(`Number of payloads: ${envelope.payloads.length}`);
+
+                        receivedPayloads.push(envelope);
+
+                        // Validate envelope structure
+                        console.log('\n=== Envelope Validation ===');
+
+                        if (envelope.payloads.length < 1) {
+                            console.log(`❌ Expected at least 1 payload, got ${envelope.payloads.length}`);
+                            testPassed = false;
+                        } else {
+                            console.log(`✅ Correct number of payloads: ${envelope.payloads.length}`);
+                        }
+
+                        // Process all payloads in the envelope
+                        console.log('\n=== Processing Payloads ===');
+                        for (let i = 0; i < envelope.payloads.length; i++) {
+                            const [dataname, data, dataType] = envelope.payloads[i];
+
+                            console.log(`\n--- Payload ${i + 1}: ${dataname} (type: ${dataType}) ---`);
+
+                            // Validate data based on type
+                            if (dataType === 'text') {
+                                if (typeof data === 'string') {
+                                    console.log(`✅ Text data received (${data.length} chars)`);
+                                    console.log(`   First 200 chars: "${data.substring(0, 200)}${data.length > 200 ? '...' : ''}"`);
+                                    
+                                    // Save to file
+                                    const outputPath = `./received_${dataname}.txt`;
+                                    require('fs').writeFileSync(outputPath, data);
+                                    console.log(`   Saved to: ${outputPath}`);
+                                } else {
+                                    console.log(`❌ Text data is not a string, got: ${typeof data}`);
+                                    testPassed = false;
+                                }
+                            } else if (dataType === 'dictionary') {
+                                if (typeof data === 'object' && data !== null && !Array.isArray(data)) {
+                                    console.log(`✅ Dictionary data received`);
+                                    console.log(`   Keys: ${Object.keys(data).join(', ')}`);
+                                    
+                                    // Save to JSON file
+                                    const outputPath = `./received_${dataname}.json`;
+                                    require('fs').writeFileSync(outputPath, JSON.stringify(data, null, 2));
+                                    console.log(`   Saved to: ${outputPath}`);
+                                } else {
+                                    console.log(`❌ Dictionary data is not an object, got: ${typeof data}`);
+                                    testPassed = false;
+                                }
+                            } else if (dataType === 'arrowtable') {
+                                // Arrow tables have numRows and numCols properties
+                                if (data && typeof data === 'object' && 
+                                    (data.numRows !== undefined || data.numRows !== null) &&
+                                    (data.numCols !== undefined || data.numCols !== null)) {
+                                    console.log(`✅ Arrow table data received`);
+                                    console.log(`   Rows: ${data.numRows}, Columns: ${data.numCols}`);
+                                    
+                                    // Save to file
+                                    const outputPath = `./received_${dataname}.arrow`;
+                                    // Note: Actual Arrow IPC serialization would require apache-arrow library
+                                    console.log(`   Saved to: ${outputPath}`);
+                                } else if (data && typeof data === 'object') {
+                                    // Some Arrow implementations may have different properties
+                                    console.log(`✅ Arrow table data received (non-standard format)`);
+                                    console.log(`   Keys: ${Object.keys(data).join(', ')}`);
+                                } else {
+                                    console.log(`❌ Arrow table data is not a valid object, got: ${typeof data}`);
+                                    testPassed = false;
+                                }
+                            } else if (dataType === 'jsontable') {
+                                if (Array.isArray(data)) {
+                                    console.log(`✅ JSON table data received`);
+                                    console.log(`   Rows: ${data.length}`);
+                                    if (data.length > 0) {
+                                        console.log(`   Columns: ${Object.keys(data[0]).join(', ')}`);
+                                    }
+                                    
+                                    // Save to JSON file
+                                    const outputPath = `./received_${dataname}.json`;
+                                    require('fs').writeFileSync(outputPath, JSON.stringify(data, null, 2));
+                                    console.log(`   Saved to: ${outputPath}`);
+                                } else {
+                                    console.log(`❌ JSON table data is not an array, got: ${typeof data}`);
+                                    testPassed = false;
+                                }
+                            } else if (dataType === 'image') {
+                                if (data instanceof Buffer || data instanceof Uint8Array) {
+                                    const dataBuffer = Buffer.isBuffer(data) ? data : Buffer.from(data);
+                                    console.log(`✅ Image data received (${dataBuffer.length} bytes)`);
+                                    
+                                    // Save to file
+                                    const outputPath = `./received_${dataname}.bin`;
+                                    require('fs').writeFileSync(outputPath, dataBuffer);
+                                    console.log(`   Saved to: ${outputPath}`);
+                                } else {
+                                    console.log(`❌ Image data is not a Buffer or Uint8Array, got: ${typeof data}`);
+                                    testPassed = false;
+                                }
+                            } else if (dataType === 'audio') {
+                                if (data instanceof Buffer || data instanceof Uint8Array) {
+                                    const dataBuffer = Buffer.isBuffer(data) ? data : Buffer.from(data);
+                                    console.log(`✅ Audio data received (${dataBuffer.length} bytes)`);
+                                    
+                                    // Save to file
+                                    const outputPath = `./received_${dataname}.bin`;
+                                    require('fs').writeFileSync(outputPath, dataBuffer);
+                                    console.log(`   Saved to: ${outputPath}`);
+                                } else {
+                                    console.log(`❌ Audio data is not a Buffer or Uint8Array, got: ${typeof data}`);
+                                    testPassed = false;
+                                }
+                            } else if (dataType === 'video') {
+                                if (data instanceof Buffer || data instanceof Uint8Array) {
+                                    const dataBuffer = Buffer.isBuffer(data) ? data : Buffer.from(data);
+                                    console.log(`✅ Video data received (${dataBuffer.length} bytes)`);
+                                    
+                                    // Save to file
+                                    const outputPath = `./received_${dataname}.bin`;
+                                    require('fs').writeFileSync(outputPath, dataBuffer);
+                                    console.log(`   Saved to: ${outputPath}`);
+                                } else {
+                                    console.log(`❌ Video data is not a Buffer or Uint8Array, got: ${typeof data}`);
+                                    testPassed = false;
+                                }
+                            } else if (dataType === 'binary') {
+                                if (data instanceof Buffer || data instanceof Uint8Array) {
+                                    const dataBuffer = Buffer.isBuffer(data) ? data : Buffer.from(data);
+                                    console.log(`✅ Binary data received (${dataBuffer.length} bytes)`);
+                                    
+                                    // Save to file
+                                    const outputPath = `./received_${dataname}`;
+                                    require('fs').writeFileSync(outputPath, dataBuffer);
+                                    console.log(`   Saved to: ${outputPath}`);
+                                } else {
+                                    console.log(`❌ Binary data is not a Buffer or Uint8Array, got: ${typeof data}`);
+                                    testPassed = false;
+                                }
+                            } else {
+                                console.log(`❌ Unknown data type: ${dataType}`);
+                                testPassed = false;
+                            }
+                        }
+
+                        // Print summary
+                        console.log('\n=== Verification Summary ===');
+                        const textCount = envelope.payloads.filter(p => p[2] === 'text').length;
+                        const dictCount = envelope.payloads.filter(p => p[2] === 'dictionary').length;
+                        const arrowtableCount = envelope.payloads.filter(p => p[2] === 'arrowtable').length;
+                        const jsontableCount = envelope.payloads.filter(p => p[2] === 'jsontable').length;
+                        const imageCount = envelope.payloads.filter(p => p[2] === 'image').length;
+                        const audioCount = envelope.payloads.filter(p => p[2] === 'audio').length;
+                        const videoCount = envelope.payloads.filter(p => p[2] === 'video').length;
+                        const binaryCount = envelope.payloads.filter(p => p[2] === 'binary').length;
+
+                        console.log(`Text payloads: ${textCount}`);
+                        console.log(`Dictionary payloads: ${dictCount}`);
+                        console.log(`Arrow table payloads: ${arrowtableCount}`);
+                        console.log(`JSON table payloads: ${jsontableCount}`);
+                        console.log(`Image payloads: ${imageCount}`);
+                        console.log(`Audio payloads: ${audioCount}`);
+                        console.log(`Video payloads: ${videoCount}`);
+                        console.log(`Binary payloads: ${binaryCount}`);
+
+                        // Stop after receiving at least one valid message
+                        if (messagesReceived >= 1) {
+                            resolve('done');
+                        }
+                    } catch (error) {
+                        console.error(`❌ Error processing message: ${error.message}`);
+                        console.error(error.stack);
+                        testPassed = false;
+                        resolve('error');
+                    }
+                }
+            })();
+        });
+
+        console.log('Waiting for messages...\n');
+
+        // Wait for message or timeout
+        const result = await messagePromise;
+
+        // Close NATS connection
+        await nc.close();
+        console.log('\n✅ NATS connection closed');
+
+        // Final result
+        console.log('\n=== Test Result ===');
+        if (messagesReceived === 0) {
+            console.log('❌ NO MESSAGES RECEIVED');
+            console.log('Make sure to run the sender test first: node test/test_js_mix_payloads_sender.js');
+            process.exit(1);
+        } else if (result === 'error') {
+            console.log('❌ ERROR PROCESSING MESSAGES');
+            process.exit(1);
+        } else if (testPassed) {
+            console.log('✅ ALL TESTS PASSED');
+            process.exit(0);
+        } else {
+            console.log('❌ SOME TESTS FAILED');
+            process.exit(1);
+        }
+
+    } catch (error) {
+        console.error('❌ Test failed with error:', error.message);
+        console.error(error.stack);
+        process.exit(1);
+    }
+}
+
+runTest();

test/test_js_mix_payloads_sender.js

@@ -0,0 +1,207 @@
+/**
+ * JavaScript Mix Payloads Sender Test
+ * Tests the smartsend function with mixed payload types
+ * 
+ * This test mirrors test_julia_mix_payloads_sender.jl and demonstrates that
+ * any combination and any number of mixed content can be sent correctly.
+ */
+
+const NATSBridge = require('../src/natsbridge.js');
+const crypto = require('crypto');
+const fs = require('fs');
+const path = require('path');
+
+const TEST_SUBJECT = '/natsbridge';
+const TEST_BROKER_URL = process.env.NATS_URL || 'nats.yiem.cc';
+const TEST_FILESERVER_URL = process.env.FILESERVER_URL || 'http://192.168.88.104:8080';
+const SIZE_THRESHOLD = 1_000_000; // 1MB threshold
+
+async function runTest() {
+    console.log('=== JavaScript Mix Payloads Sender Test ===\n');
+    
+    const correlationId = crypto.randomUUID();
+    console.log(`Correlation ID: ${correlationId}`);
+    console.log(`Subject: ${TEST_SUBJECT}`);
+    console.log(`Broker URL: ${TEST_BROKER_URL}`);
+    console.log(`Fileserver URL: ${TEST_FILESERVER_URL}`);
+    console.log(`Size Threshold: ${SIZE_THRESHOLD} bytes (1MB)\n`);
+    
+    // Helper: Log with correlation ID
+    function logTrace(message) {
+        const timestamp = new Date().toISOString();
+        console.log(`[${timestamp}] [Correlation: ${correlationId}] ${message}`);
+    }
+    
+    // Create sample data for each type (mirroring Julia test)
+    const textData = 'Hello! This is a test chat message. 🎉\nHow are you doing today? 😊';
+    
+    const dictData = {
+        type: 'chat',
+        sender: 'serviceA',
+        receiver: 'serviceB',
+        metadata: {
+            timestamp: new Date().toISOString(),
+            priority: 'high',
+            tags: ['urgent', 'chat', 'test']
+        },
+        content: {
+            text: 'This is a JSON-formatted chat message with nested structure.',
+            format: 'markdown',
+            mentions: ['user1', 'user2']
+        }
+    };
+    
+    // Arrow table data (small - direct transport)
+    const arrowTableSmall = [
+        { id: 1, name: 'Alice', score: 95, active: true },
+        { id: 2, name: 'Bob', score: 88, active: false },
+        { id: 3, name: 'Charlie', score: 92, active: true },
+        { id: 4, name: 'Diana', score: 78, active: true },
+        { id: 5, name: 'Eve', score: 85, active: false },
+        { id: 6, name: 'Frank', score: 91, active: true },
+        { id: 7, name: 'Grace', score: 89, active: true },
+        { id: 8, name: 'Henry', score: 76, active: false },
+        { id: 9, name: 'Ivy', score: 94, active: true },
+        { id: 10, name: 'Jack', score: 82, active: true }
+    ];
+    
+    // Json table data (small - direct transport)
+    const jsonTableSmall = [
+        { id: 1, name: 'Alice', score: 95, active: true },
+        { id: 2, name: 'Bob', score: 88, active: false },
+        { id: 3, name: 'Charlie', score: 92, active: true },
+        { id: 4, name: 'Diana', score: 78, active: true },
+        { id: 5, name: 'Eve', score: 85, active: false },
+        { id: 6, name: 'Frank', score: 91, active: true },
+        { id: 7, name: 'Grace', score: 89, active: true },
+        { id: 8, name: 'Henry', score: 76, active: false },
+        { id: 9, name: 'Ivy', score: 94, active: true },
+        { id: 10, name: 'Jack', score: 82, active: true }
+    ];
+    
+    // Audio data (small binary - direct transport)
+    const audioData = Buffer.alloc(100);
+    for (let i = 0; i < 100; i++) {
+        audioData[i] = Math.floor(Math.random() * 255);
+    }
+    
+    // Video data (small binary - direct transport)
+    const videoData = Buffer.alloc(150);
+    for (let i = 0; i < 150; i++) {
+        videoData[i] = Math.floor(Math.random() * 255);
+    }
+    
+    // Binary data (small - direct transport)
+    const binaryData = Buffer.alloc(200);
+    for (let i = 0; i < 200; i++) {
+        binaryData[i] = Math.floor(Math.random() * 255);
+    }
+    
+    // Large data for link transport testing
+    const largeArrowTable = [];
+    for (let i = 1; i <= 20000; i++) {
+        largeArrowTable.push({
+            id: i,
+            name: `user_${i}`,
+            score: Math.floor(Math.random() * 51) + 50,
+            active: Math.random() > 0.5,
+            timestamp: new Date().toISOString()
+        });
+    }
+    
+    const largeJsonTable = [];
+    for (let i = 1; i <= 50000; i++) {
+        largeJsonTable.push({
+            id: i,
+            name: `user_${i}`,
+            score: Math.floor(Math.random() * 51) + 50,
+            active: Math.random() > 0.5
+        });
+    }
+    
+    const largeAudioData = Buffer.alloc(1_500_000);
+    for (let i = 0; i < 1_500_000; i++) {
+        largeAudioData[i] = Math.floor(Math.random() * 255);
+    }
+    
+    const largeVideoData = Buffer.alloc(1_500_000);
+    for (let i = 0; i < 1_500_000; i++) {
+        largeVideoData[i] = Math.floor(Math.random() * 255);
+    }
+    
+    const largeBinaryData = Buffer.alloc(1_500_000);
+    for (let i = 0; i < 1_500_000; i++) {
+        largeBinaryData[i] = Math.floor(Math.random() * 255);
+    }
+    
+    // Read image files from disk (following Julia test pattern)
+    const file_path_small_image = path.join(__dirname, 'small_image.jpg');
+    const file_data_small_image = fs.readFileSync(file_path_small_image);
+    const filename_small_image = path.basename(file_path_small_image);
+    
+    const file_path_large_image = path.join(__dirname, 'large_image.png');
+    const file_data_large_image = fs.readFileSync(file_path_large_image);
+    const filename_large_image = path.basename(file_path_large_image);
+    
+    logTrace('Creating payloads list with mixed content');
+    
+    // Create payloads list - mixed content with both small and large data
+    // Small data uses direct transport, large data uses link transport
+    const payloads = [
+        // Small data (direct transport) - text, dictionary, arrowtable, jsontable, small image
+        ['chat_text', textData, 'text'],
+        ['chat_json', dictData, 'dictionary'],
+        // ['arrow_table_small', arrowTableSmall, 'arrowtable'],
+        ['json_table_small', jsonTableSmall, 'jsontable'],
+        [filename_small_image, file_data_small_image, 'binary'],
+        
+        // Large data (link transport) - large arrowtable, large jsontable, large image, large audio, large video, large binary
+        // ['arrow_table_large', largeArrowTable, 'arrowtable'],
+        ['json_table_large', largeJsonTable, 'jsontable'],
+        [filename_large_image, file_data_large_image, 'binary'],
+        // ['audio_clip_large', largeAudioData, 'audio'],
+        // ['video_clip_large', largeVideoData, 'video'],
+        // ['binary_file_large', largeBinaryData, 'binary']
+    ];
+    
+    logTrace(`Total payloads: ${payloads.length}`);
+    
+    try {
+        // Send the message
+        console.log('Sending mixed payloads...\n');
+        const [env, envJsonStr] = await NATSBridge.smartsend(
+            TEST_SUBJECT,
+            payloads,
+            {
+                broker_url: TEST_BROKER_URL,
+                fileserver_url: TEST_FILESERVER_URL,
+                fileserver_upload_handler: NATSBridge.plikOneshotUpload,
+                size_threshold: SIZE_THRESHOLD,
+                correlation_id: correlationId,
+                msg_purpose: 'chat',
+                sender_name: 'js-mix-test',
+                receiver_name: '',
+                receiver_id: '',
+                reply_to: '',
+                reply_to_msg_id: '',
+                is_publish: true
+            }
+        );
+        
+        console.log('\n=== Envelope Created ===');
+        console.log(`Correlation ID: ${env.correlation_id}`);
+        console.log(`Message ID: ${env.msg_id}`);
+        console.log(`Timestamp: ${env.timestamp}`);
+        console.log(`Subject: ${env.send_to}`);
+        console.log(`Purpose: ${env.msg_purpose}`);
+        console.log(`Sender: ${env.sender_name}`);
+        console.log(`Payloads: ${env.payloads.length}\n`);
+        
+    } catch (error) {
+        console.error('\n❌ Test failed with error:', error.message);
+        console.error(error.stack);
+        process.exit(1);
+    }
+}
+
+runTest();

test/test_julia_julia_data_transfer.jl

@@ -1,190 +0,0 @@
-#!/usr/bin/env julia
-# Test script for large payload testing using binary transport
-# Tests sending a large file (> 1MB) via smartsend with binary type
-
-using NATS, JSON, UUIDs, Dates
-
-# Include the bridge module
-include("../src/NATSBridge.jl")
-using .NATSBridge
-
-# Configuration
-const SUBJECT = "/large_binary_test"
-const NATS_URL = "nats.yiem.cc"
-const FILESERVER_URL = "http://192.168.88.104:8080"
-
-# Create correlation ID for tracing
-correlation_id = string(uuid4())
-
-
-# ------------------------------------------------------------------------------------------------ #
-#                                        test file transfer                                        #
-# ------------------------------------------------------------------------------------------------ #
-
-# File path for large binary payload test
-const FILE_PATH = "./testFile_small.zip"
-const filename = basename(FILE_PATH)
-
-# Helper: Log with correlation ID
-function log_trace(message)
-    timestamp = Dates.now()
-    println("[$timestamp] [Correlation: $correlation_id] $message")
-end
-
-# Sender: Send large binary file via smartsend
-function test_large_binary_send()
-    conn = NATS.connect(NATS_URL)
-    # Read the large file as binary data
-    log_trace("Reading large file: $FILE_PATH")
-    file_data = read(FILE_PATH)
-    
-    file_size = length(file_data)
-    log_trace("File size: $file_size bytes")
-    
-    # Use smartsend with binary type - will automatically use link transport
-    # if file size exceeds the threshold (1MB by default)
-    env = NATSBridge.smartsend(
-        SUBJECT,
-        file_data,
-        "binary",
-        nats_url = NATS_URL,
-        fileserver_url = FILESERVER_URL;
-        dataname=filename
-    )
-    
-    log_trace("Sent message with transport: $(env.transport)")
-    log_trace("Envelope type: $(env.type)")
-    
-    # Check if link transport was used
-    if env.transport == "link"
-        log_trace("Using link transport - file uploaded to HTTP server")
-        log_trace("URL: $(env.url)")
-    else
-        log_trace("Using direct transport - payload sent via NATS")
-    end
-    
-    NATS.drain(conn)
-end
-
-# Receiver: Listen for messages and verify large payload handling
-function test_large_binary_receive()
-  conn = NATS.connect(NATS_URL)
-  NATS.subscribe(conn, SUBJECT) do msg
-    log_trace("Received message on $(msg.subject)")
-    
-    # Use NATSBridge.smartreceive to handle the data
-    result = NATSBridge.smartreceive(msg)
-    # Check transport type
-    if result.envelope.transport == "direct"
-        log_trace("Received direct transport")
-    else
-        # For link transport, result.data is the URL
-        log_trace("Received link transport")
-    end
-              
-    # Verify the received data matches the original
-    if result.envelope.type == "binary"
-      if isa(result.data, Vector{UInt8})
-        file_size = length(result.data)
-        log_trace("Received $(file_size) bytes of binary data")
-        
-        # Save received data to a test file
-        println("metadata ", result.envelope.metadata)
-        dataname = result.envelope.metadata["dataname"]
-        if dataname != "NA"
-          output_path = "./new_$dataname"
-          write(output_path, result.data)
-          log_trace("Saved received data to $output_path")
-        end
-        
-        # Verify file size
-        original_size = length(read(FILE_PATH))
-        if file_size == result.envelope.metadata["content_length"]
-            log_trace("SUCCESS: File size matches! Original: $(result.envelope.metadata["content_length"]) bytes")
-        else
-            log_trace("WARNING: File size mismatch! Original: $(result.envelope.metadata["content_length"]), Received: $file_size")
-        end
-      end
-    end
-  end
-
-  # Keep listening for 10 seconds
-  sleep(120)  
-  NATS.drain(conn)
-end
-
-
-# Run the test
-println("Starting large binary payload test...")
-println("Correlation ID: $correlation_id")
-println("File: $FILE_PATH")
-
-# Run sender first
-println("start smartsend")
-test_large_binary_send()
-
-# # Run receiver
-# println("testing smartreceive")
-# test_large_binary_receive()
-
-println("Test completed.")
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-

test/test_julia_mix_payloads_receiver.jl

@@ -0,0 +1,251 @@
+#!/usr/bin/env julia
+# Test script for mixed-content message testing
+# Tests receiving a mix of text, json, table, image, audio, video, and binary data
+# from Julia serviceA to Julia serviceB using NATSBridge.jl smartreceive
+#
+# This test demonstrates that any combination and any number of mixed content
+# can be sent and received correctly.
+
+using NATS, JSON, UUIDs, Dates, PrettyPrinting, DataFrames, Arrow, HTTP, Base64
+
+# Include the bridge module
+include("../src/NATSBridge.jl")
+using .NATSBridge
+
+# Configuration
+const SUBJECT = "/natsbridge"
+const NATS_URL = "nats.yiem.cc"
+const FILESERVER_URL = "http://192.168.88.104:8080"
+
+
+# ------------------------------------------------------------------------------------------------ #
+#                                      test mixed content transfer                                 #
+# ------------------------------------------------------------------------------------------------ #
+
+
+# Helper: Log with correlation ID
+function log_trace(message)
+    timestamp = Dates.now()
+    println("[$timestamp] $message")
+end
+
+
+# Receiver: Listen for messages and verify mixed content handling
+function test_mix_receive()
+    conn = NATS.connect(NATS_URL)
+    NATS.subscribe(conn, SUBJECT) do msg
+        log_trace("Received message on $(msg.subject)")
+        
+        # Use NATSBridge.smartreceive to handle the data
+        # API: smartreceive(msg, download_handler; max_retries, base_delay, max_delay)
+        result = NATSBridge.smartreceive(
+            msg;
+            max_retries = 5,
+            base_delay = 100,
+            max_delay = 5000
+        )
+        
+        log_trace("Received $(length(result["payloads"])) payloads")
+        
+        # Result is an envelope dictionary with payloads field containing list of (dataname, data, data_type) tuples
+        for (dataname, data, data_type) in result["payloads"]
+            log_trace("\n=== Payload: $dataname (type: $data_type) ===")
+            
+            # Handle different data types
+            if data_type == "text"
+                # Text data - should be a String
+                if isa(data, String)
+                    log_trace("  Type: String")
+                    log_trace("  Length: $(length(data)) characters")
+                    
+                    # Display first 200 characters
+                    if length(data) > 200
+                        log_trace("  First 200 chars: $(data[1:200])...")
+                    else
+                        log_trace("  Content: $data")
+                    end
+                    
+                    # Save to file
+                    output_path = "./received_$dataname.txt"
+                    write(output_path, data)
+                    log_trace("  Saved to: $output_path")
+                else
+                    log_trace("  ERROR: Expected String, got $(typeof(data))")
+                end
+                
+            elseif data_type == "dictionary"
+                # Dictionary data - should be JSON object
+                if isa(data, JSON.Object{String, Any})
+                    log_trace("  Type: Dict")
+                    log_trace("  Keys: $(keys(data))")
+                    
+                    # Display nested content
+                    for (key, value) in data
+                        log_trace("  $key => $value")
+                    end
+                    
+                    # Save to JSON file
+                    output_path = "./received_$dataname.json"
+                    json_str = JSON.json(data, 2)
+                    write(output_path, json_str)
+                    log_trace("  Saved to: $output_path")
+                else
+                    log_trace("  ERROR: Expected Dict, got $(typeof(data))")
+                end
+                
+            elseif data_type == "arrowtable"
+                # Arrow table data - should be Arrow.Table
+                if isa(data, Arrow.Table)
+                    log_trace("  Type: Arrow.Table")
+                    
+                    # Convert to DataFrame for display and save
+                    df = DataFrame(data)
+                    @show df[1:3, :]
+                    output_path = "./received_$dataname.arrow"
+                    io = IOBuffer()
+                    Arrow.write(io, data)
+                    write(output_path, take!(io))
+                    log_trace("  Saved to: $output_path")
+                else
+                    log_trace("  ERROR: Expected Arrow.Table, got $(typeof(data))")
+                end
+                
+            elseif data_type == "jsontable"
+                # JSON table data - should be Vector{Dict} or Vector{NamedTuple}
+                @show "jsontable" typeof(data)
+                if isa(data, Vector{Any})
+                    log_trace("  Type: Vector{Dict/NamedTuple}")
+                    
+                    # Convert to DataFrame for display and save
+                    df = DataFrame(data)
+                    @show df[1:3, :]
+                    log_trace("  Converted to DataFrame: $(size(df, 1)) rows x $(size(df, 2)) columns")
+                    
+                    # Save as JSON file
+                    output_path = "./received_$dataname.json"
+                    json_str = JSON.json(data, 2)
+                    write(output_path, json_str)
+                    log_trace("  Saved to: $output_path")
+                else
+                    log_trace("  ERROR: Expected Vector{Dict/NamedTuple}, got $(typeof(data))")
+                end
+                
+            elseif data_type == "image"
+                # Image data - should be Vector{UInt8}
+                if isa(data, Vector{UInt8})
+                    log_trace("  Type: Vector{UInt8} (binary)")
+                    log_trace("  Size: $(length(data)) bytes")
+                    
+                    # Save to file
+                    output_path = "./received_$dataname.bin"
+                    write(output_path, data)
+                    log_trace("  Saved to: $output_path")
+                else
+                    log_trace("  ERROR: Expected Vector{UInt8}, got $(typeof(data))")
+                end
+                
+            elseif data_type == "audio"
+                # Audio data - should be Vector{UInt8}
+                if isa(data, Vector{UInt8})
+                    log_trace("  Type: Vector{UInt8} (binary)")
+                    log_trace("  Size: $(length(data)) bytes")
+                    
+                    # Save to file
+                    output_path = "./received_$dataname.bin"
+                    write(output_path, data)
+                    log_trace("  Saved to: $output_path")
+                else
+                    log_trace("  ERROR: Expected Vector{UInt8}, got $(typeof(data))")
+                end
+                
+            elseif data_type == "video"
+                # Video data - should be Vector{UInt8}
+                if isa(data, Vector{UInt8})
+                    log_trace("  Type: Vector{UInt8} (binary)")
+                    log_trace("  Size: $(length(data)) bytes")
+                    
+                    # Save to file
+                    output_path = "./received_$dataname.bin"
+                    write(output_path, data)
+                    log_trace("  Saved to: $output_path")
+                else
+                    log_trace("  ERROR: Expected Vector{UInt8}, got $(typeof(data))")
+                end
+                
+            elseif data_type == "binary"
+                # Binary data - should be Vector{UInt8}
+                if isa(data, Vector{UInt8})
+                    log_trace("  Type: Vector{UInt8} (binary)")
+                    log_trace("  Size: $(length(data)) bytes")
+                    
+                    # Save to file
+                    output_path = "./received_$dataname"
+                    write(output_path, data)
+                    log_trace("  Saved to: $output_path")
+                else
+                    log_trace("  ERROR: Expected Vector{UInt8}, got $(typeof(data))")
+                end
+                
+            else
+                log_trace("  ERROR: Unknown data type '$data_type'")
+            end
+        end
+        
+        # Summary
+        println("\n=== Verification Summary ===")
+        text_count = count(x -> x[3] == "text", result["payloads"])
+        dict_count = count(x -> x[3] == "dictionary", result["payloads"])
+        arrowtable_count = count(x -> x[3] == "arrowtable", result["payloads"])
+        jsontable_count = count(x -> x[3] == "jsontable", result["payloads"])
+        table_count = count(x -> x[3] == "table", result["payloads"])  # backward compatibility
+        image_count = count(x -> x[3] == "image", result["payloads"])
+        audio_count = count(x -> x[3] == "audio", result["payloads"])
+        video_count = count(x -> x[3] == "video", result["payloads"])
+        binary_count = count(x -> x[3] == "binary", result["payloads"])
+        
+        log_trace("Text payloads: $text_count")
+        log_trace("Dictionary payloads: $dict_count")
+        log_trace("Arrow table payloads: $arrowtable_count")
+        log_trace("JSON table payloads: $jsontable_count")
+        log_trace("Table payloads (backward compat): $table_count")
+        log_trace("Image payloads: $image_count")
+        log_trace("Audio payloads: $audio_count")
+        log_trace("Video payloads: $video_count")
+        log_trace("Binary payloads: $binary_count")
+        
+        # Print transport type info for each payload if available
+        println("\n=== Payload Details ===")
+        for (dataname, data, data_type) in result["payloads"]
+            if data_type in ["image", "audio", "video", "binary"]
+                log_trace("$dataname: $(length(data)) bytes (binary)")
+            elseif data_type == "arrowtable"
+                # log_trace("$dataname: $(size(data, 1)) rows x $(size(data, 2)) columns (Arrow.Table)")
+            elseif data_type == "jsontable"
+                log_trace("$dataname: $(length(data)) rows (Vector{Dict/NamedTuple})")
+            elseif data_type == "table"
+                data = DataFrame(data)
+                # log_trace("$dataname: $(size(data, 1)) rows x $(size(data, 2)) columns (DataFrame)")
+            elseif data_type == "dictionary"
+                log_trace("$dataname: $(length(JSON.json(data))) bytes (Dict)")
+            elseif data_type == "text"
+                log_trace("$dataname: $(length(data)) characters (String)")
+            end
+        end
+    end
+    
+    # Keep listening for 2 minutes
+    sleep(180)  
+    NATS.drain(conn)
+end
+
+
+# Run the test
+println("Starting mixed-content transport test...")
+println("Note: This receiver will wait for messages from the sender.")
+println("Run test_julia_to_julia_mix_sender.jl first to send test data.")
+
+# Run receiver
+println("\ntesting smartreceive for mixed content")
+test_mix_receive()
+
+println("\nTest completed.")

test/test_julia_mix_payloads_sender.jl

@@ -0,0 +1,258 @@
+#!/usr/bin/env julia
+# Test script for mixed-content message testing
+# Tests sending a mix of text, dictionary, arrowtable, jsontable, image, audio, video, and binary data
+# from Julia serviceA to Julia serviceB using NATSBridge.jl smartsend
+# 
+# This test demonstrates that any combination and any number of mixed content
+# can be sent and received correctly.
+# 
+# Key concept: DataFrames are the main table representation in Julia.
+# The NATSBridge.jl library handles serialization:
+# - For "arrowtable" type: DataFrame is serialized to Arrow IPC format
+# - For "jsontable" type: DataFrame is converted to Vector{Dict} and then to JSON
+
+using NATS, JSON, UUIDs, Dates, PrettyPrinting, DataFrames, Arrow, HTTP, Base64
+
+# Include the bridge module
+include("../src/NATSBridge.jl")
+using .NATSBridge
+
+# Configuration
+const SUBJECT = "/natsbridge"
+const NATS_URL = "nats.yiem.cc"
+const FILESERVER_URL = "http://192.168.88.104:8080"
+
+# Create correlation ID for tracing
+correlation_id = string(uuid4())
+
+
+# ------------------------------------------------------------------------------------------------ #
+#                                      test mixed content transfer                                 #
+# ------------------------------------------------------------------------------------------------ #
+
+
+# Helper: Log with correlation ID
+function log_trace(message)
+    timestamp = Dates.now()
+    println("[$timestamp] [Correlation: $correlation_id] $message")
+end
+
+
+# File upload handler for plik server
+function plik_upload_handler(fileserver_url::String, dataname::String, data::Vector{UInt8})::Dict{String, Any}
+    # Get upload ID
+    url_getUploadID = "$fileserver_url/upload"
+    headers = ["Content-Type" => "application/json"]
+    body = """{ "OneShot" : true }"""
+    httpResponse = HTTP.request("POST", url_getUploadID, headers, body; body_is_form=false)
+    responseJson = JSON.parse(String(httpResponse.body))
+    uploadid = responseJson["id"]
+    uploadtoken = responseJson["uploadToken"]
+    
+    # Upload file
+    file_multipart = HTTP.Multipart(dataname, IOBuffer(data), "application/octet-stream")
+    url_upload = "$fileserver_url/file/$uploadid"
+    headers = ["X-UploadToken" => uploadtoken]
+    
+    form = HTTP.Form(Dict("file" => file_multipart))
+    httpResponse = HTTP.post(url_upload, headers, form)
+    responseJson = JSON.parse(String(httpResponse.body))
+    
+    fileid = responseJson["id"]
+    url = "$fileserver_url/file/$uploadid/$fileid/$dataname"
+    
+    return Dict("status" => httpResponse.status, "uploadid" => uploadid, "fileid" => fileid, "url" => url)
+end
+
+
+# Helper: Create sample data for each type
+function create_sample_data()
+    # Text data (small - direct transport)
+    text_data = "Hello! This is a test chat message. 🎉\nHow are you doing today? 😊"
+    
+    # Dictionary/JSON data (medium - could be direct or link)
+    dict_data = Dict(
+        "type" => "chat",
+        "sender" => "serviceA",
+        "receiver" => "serviceB",
+        "metadata" => Dict(
+            "timestamp" => string(Dates.now()),
+            "priority" => "high",
+            "tags" => ["urgent", "chat", "test"]
+        ),
+        "content" => Dict(
+            "text" => "This is a JSON-formatted chat message with nested structure.",
+            "format" => "markdown",
+            "mentions" => ["user1", "user2"]
+        )
+    )
+    
+    # Arrow table data (DataFrame - small - direct transport)
+    # Uses Arrow IPC format for efficient binary serialization
+    # NATSBridge.jl handles serialization: DataFrame -> Arrow IPC
+    arrow_table_small = DataFrame(
+        id = 1:10,
+        name = ["Alice", "Bob", "Charlie", "Diana", "Eve", "Frank", "Grace", "Henry", "Ivy", "Jack"],
+        score = rand(50:100, 10),
+        active = rand([true, false], 10)
+    )
+    
+    # Arrow table data (DataFrame - large - link transport)
+    # ~1.5MB of Arrow data (200,000 rows) - should trigger link transport
+    # NATSBridge.jl handles serialization: DataFrame -> Arrow IPC
+    arrow_table_large = DataFrame(
+        id = 1:2_000_000,
+        name = ["user_$i" for i in 1:2_000_000],
+        score = rand(50:100, 2_000_000),
+        active = rand([true, false], 2_000_000),
+        timestamp = [string(Dates.now()) for _ in 1:2_000_000]
+    )
+    
+    # Json table data (DataFrame - small - direct transport)
+    # Uses JSON format for human-readable tabular data
+    # NATSBridge.jl handles serialization: DataFrame -> Vector{Dict} -> JSON
+    json_table_small = DataFrame(
+        id = 1:10,
+        name = ["Alice", "Bob", "Charlie", "Diana", "Eve", "Frank", "Grace", "Henry", "Ivy", "Jack"],
+        score = rand(50:100, 10),
+        active = rand([true, false], 10)
+    )
+    
+    # Json table data (DataFrame - large - link transport)
+    # ~1.5MB of JSON data (150,000 rows) - should trigger link transport
+    # NATSBridge.jl handles serialization: DataFrame -> Vector{Dict} -> JSON
+    json_table_large = DataFrame(
+        id = 1:2_000_000,
+        name = ["user_$i" for i in 1:2_000_000],
+        score = rand(50:100, 2_000_000),
+        active = rand([true, false], 2_000_000)
+    )
+    
+    # Audio data (small binary - direct transport)
+    audio_data = UInt8[rand(1:255) for _ in 1:100]
+    
+    # Audio data (large - link transport)
+    # ~1.5MB of audio-like data
+    large_audio_data = UInt8[rand(1:255) for _ in 1:1_500_000]
+    
+    # Video data (small binary - direct transport)
+    video_data = UInt8[rand(1:255) for _ in 1:150]
+    
+    # Video data (large - link transport)
+    # ~1.5MB of video-like data
+    large_video_data = UInt8[rand(1:255) for _ in 1:1_500_000]
+    
+    # Binary data (small - direct transport)
+    binary_data = UInt8[rand(1:255) for _ in 1:200]
+    
+    # Binary data (large - link transport)
+    # ~1.5MB of binary data
+    large_binary_data = UInt8[rand(1:255) for _ in 1:1_500_000]
+    
+    return (
+        text_data,
+        dict_data,
+        arrow_table_small,
+        arrow_table_large,
+        json_table_small,
+        json_table_large,
+        audio_data,
+        large_audio_data,
+        video_data,
+        large_video_data,
+        binary_data,
+        large_binary_data
+    )
+end
+
+
+# Sender: Send mixed content via smartsend
+function test_mix_send()
+    # Create sample data
+    (text_data, dict_data, arrow_table_small, arrow_table_large, json_table_small, json_table_large, audio_data, large_audio_data, video_data, large_video_data, binary_data, large_binary_data) = create_sample_data()
+
+    # Read image files from disk (following test_julia_file_sender.jl pattern)
+    # Small image - should use direct transport
+    file_path_small_image = "./test/small_image.jpg"
+    file_data_small_image = read(file_path_small_image)
+    filename_small_image = basename(file_path_small_image)
+
+    # Large image - should use link transport
+    file_path_large_image = "./test/large_image.png"
+    file_data_large_image = read(file_path_large_image)
+    filename_large_image = basename(file_path_large_image)
+
+    # Create payloads list - mixed content with both small and large data
+    # Small data uses direct transport, large data uses link transport
+    # Key: Pass DataFrame directly and specify type as "arrowtable" or "jsontable"
+    # NATSBridge.jl handles the serialization internally
+    payloads = [
+        # Small data (direct transport) - text, dictionary, arrowtable, jsontable, small image
+        ("chat_text", text_data, "text"),
+        ("chat_json", dict_data, "dictionary"),
+        # ("arrow_table_small", arrow_table_small, "arrowtable"),
+        ("json_table_small", json_table_small, "jsontable"),
+        (filename_small_image, file_data_small_image, "binary"),
+
+        # Large data (link transport) - large arrowtable, large jsontable, large image, large audio, large video, large binary
+        # ("arrow_table_large", arrow_table_large, "arrowtable"),
+        ("json_table_large", json_table_large, "jsontable"),
+        (filename_large_image, file_data_large_image, "binary"),
+        ("audio_clip_large", large_audio_data, "audio"),
+        ("video_clip_large", large_video_data, "video"),
+        ("binary_file_large", large_binary_data, "binary")
+    ]
+    
+    # Use smartsend with mixed content
+    sendinfo = NATSBridge.smartsend(
+        SUBJECT,
+        payloads;  # List of (dataname, data, type) tuples
+        broker_url = NATS_URL,
+        fileserver_url = FILESERVER_URL,
+        fileserver_upload_handler = plik_upload_handler,
+        size_threshold = 1_000_000,  # 1MB threshold
+        correlation_id = correlation_id,
+        msg_purpose = "chat",
+        sender_name = "mix_sender",
+        receiver_name = "",
+        receiver_id = "",
+        reply_to = "",
+        reply_to_msg_id = "",
+        is_publish = true  # Publish the message to NATS
+    )
+
+    env, env_json_str = sendinfo
+    log_trace("Sent message with $(length(env.payloads)) payloads")
+    
+    # Log transport type for each payload
+    for (i, payload) in enumerate(env.payloads)
+        log_trace("Payload $i ('$payload.dataname'):")
+        log_trace("  Transport: $(payload.transport)")
+        log_trace("  Type: $(payload.payload_type)")
+        log_trace("  Size: $(payload.size) bytes")
+        log_trace("  Encoding: $(payload.encoding)")
+        
+        if payload.transport == "link"
+            log_trace("  URL: $(payload.data)")
+        end
+    end
+    
+    # Summary
+    println("\n--- Transport Summary ---")
+    direct_count = count(p -> p.transport == "direct", env.payloads)
+    link_count = count(p -> p.transport == "link", env.payloads)
+    log_trace("Direct transport: $direct_count payloads")
+    log_trace("Link transport: $link_count payloads")
+end
+
+
+# Run the test
+println("Starting mixed-content transport test...")
+println("Correlation ID: $correlation_id")
+
+# Run sender
+println("start smartsend for mixed content")
+test_mix_send()
+
+println("\nTest completed.")
+println("Note: Run test_julia_to_julia_mix_receiver.jl to receive the messages.")

test/test_py_mix_payloads_sender.py

@@ -0,0 +1,199 @@
+"""
+Python Mix Payloads Sender Test
+Tests the smartsend function with mixed payload types
+"""
+
+import asyncio
+import sys
+import os
+import base64
+
+# Add parent directory to path
+sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
+
+from natsbridge import smartsend, DEFAULT_BROKER_URL, DEFAULT_FILESERVER_URL
+
+TEST_SUBJECT = '/test/mix'
+TEST_BROKER_URL = os.environ.get('NATS_URL', 'nats://localhost:4222')
+TEST_FILESERVER_URL = os.environ.get('FILESERVER_URL', 'http://localhost:8080')
+
+
+async def run_test():
+    print('=== Python Mix Payloads Sender Test ===\n')
+    
+    correlation_id = 'py-mix-test-' + str(asyncio.get_event_loop().time() * 1000000)
+    print(f'Correlation ID: {correlation_id}')
+    print(f'Subject: {TEST_SUBJECT}')
+    print(f'Broker URL: {TEST_BROKER_URL}\n')
+    
+    # Test data - mixed payload types
+    text_data = 'Hello, NATSBridge!'
+    dict_data = {'key1': 'value1', 'key2': 42, 'nested': {'a': 1, 'b': 2}}
+    image_data = bytes([0x89, 0x50, 0x4E, 0x47, 0x0D, 0x0A, 0x1A, 0x0A])  # PNG header
+    
+    # Table data
+    try:
+        import pandas as pd
+        table_data = pd.DataFrame({
+            'id': [1, 2, 3],
+            'name': ['Alice', 'Bob', 'Charlie'],
+            'age': [30, 25, 35]
+        })
+        table_available = True
+    except ImportError:
+        table_available = False
+        table_data = None
+    
+    test_data = [
+        ('message', text_data, 'text'),
+        ('config', dict_data, 'dictionary'),
+        ('image', image_data, 'image')
+    ]
+    
+    if table_available:
+        test_data.append(('users', table_data, 'table'))
+    
+    try:
+        # Send the message
+        print('Sending mixed payloads...')
+        env, env_json_str = await smartsend(
+            TEST_SUBJECT,
+            test_data,
+            broker_url=TEST_BROKER_URL,
+            fileserver_url=TEST_FILESERVER_URL,
+            correlation_id=correlation_id,
+            msg_purpose='test',
+            sender_name='py-mix-test',
+            is_publish=False
+        )
+        
+        print('\n=== Envelope Created ===')
+        print(f'Correlation ID: {env["correlation_id"]}')
+        print(f'Message ID: {env["msg_id"]}')
+        print(f'Timestamp: {env["timestamp"]}')
+        print(f'Subject: {env["send_to"]}')
+        print(f'Purpose: {env["msg_purpose"]}')
+        print(f'Sender: {env["sender_name"]}')
+        print(f'Payloads: {len(env["payloads"])}\n')
+        
+        # Validate envelope structure
+        print('=== Validation ===')
+        passed = True
+        
+        expected_count = 4 if table_available else 3
+        if len(env['payloads']) != expected_count:
+            print(f'❌ Expected {expected_count} payloads, got {len(env["payloads"])}')
+            passed = False
+        else:
+            print('✅ Correct number of payloads')
+        
+        # Test each payload
+        expected_datanames = ['message', 'config', 'image']
+        expected_types = ['text', 'dictionary', 'image']
+        expected_data = [text_data, dict_data, image_data]
+        
+        if table_available:
+            expected_datanames.append('users')
+            expected_types.append('table')
+        
+        for i in range(len(env['payloads'])):
+            payload = env['payloads'][i]
+            
+            if payload['dataname'] != expected_datanames[i]:
+                print(f"❌ Payload {i + 1}: Expected dataname '{expected_datanames[i]}', got '{payload['dataname']}'")
+                passed = False
+            else:
+                print(f'✅ Payload {i + 1}: Correct dataname')
+            
+            if payload['payload_type'] != expected_types[i]:
+                print(f"❌ Payload {i + 1}: Expected type '{expected_types[i]}', got '{payload['payload_type']}'")
+                passed = False
+            else:
+                print(f'✅ Payload {i + 1}: Correct type')
+            
+            if payload['transport'] != 'direct':
+                print(f"❌ Payload {i + 1}: Expected transport 'direct', got '{payload['transport']}'")
+                passed = False
+            else:
+                print(f'✅ Payload {i + 1}: Correct transport')
+            
+            if payload['encoding'] != 'base64':
+                print(f"❌ Payload {i + 1}: Expected encoding 'base64', got '{payload['encoding']}'")
+                passed = False
+            else:
+                print(f'✅ Payload {i + 1}: Correct encoding')
+            
+            # Verify data integrity based on type
+            decoded_data = base64.b64decode(payload['data'])
+            
+            if expected_types[i] == 'text':
+                decoded_text = decoded_data.decode('utf8')
+                if decoded_text != expected_data[i]:
+                    print(f'❌ Payload {i + 1}: Data integrity mismatch')
+                    passed = False
+                else:
+                    print(f'✅ Payload {i + 1}: Data integrity verified')
+            elif expected_types[i] == 'dictionary':
+                import json
+                decoded_dict = json.loads(decoded_data.decode('utf8'))
+                if json.dumps(decoded_dict, sort_keys=True) != json.dumps(expected_data[i], sort_keys=True):
+                    print(f'❌ Payload {i + 1}: Data integrity mismatch')
+                    passed = False
+                else:
+                    print(f'✅ Payload {i + 1}: Data integrity verified')
+            elif expected_types[i] == 'image':
+                if decoded_data != expected_data[i]:
+                    print(f'❌ Payload {i + 1}: Data integrity mismatch')
+                    passed = False
+                else:
+                    print(f'✅ Payload {i + 1}: Data integrity verified')
+            elif expected_types[i] == 'table':
+                if len(decoded_data) > 0:
+                    print(f'✅ Payload {i + 1}: Arrow IPC data present ({len(decoded_data)} bytes)')
+                else:
+                    print(f'❌ Payload {i + 1}: Arrow IPC data is empty')
+                    passed = False
+            
+            print(f'   Size: {payload["size"]} bytes\n')
+        
+        # Test with chat-like payload (text + image + audio)
+        print('=== Chat-like Payload Test ===')
+        chat_data = [
+            ('text', 'Hello!', 'text'),
+            ('image', bytes([0xFF, 0xD8, 0xFF, 0xE0]), 'image'),
+            ('audio', bytes([0x46, 0x4C, 0x41, 0x43]), 'audio')
+        ]
+        
+        chat_env, _ = await smartsend(
+            TEST_SUBJECT,
+            chat_data,
+            broker_url=TEST_BROKER_URL,
+            fileserver_url=TEST_FILESERVER_URL,
+            correlation_id='chat-' + correlation_id,
+            is_publish=False
+        )
+        
+        if len(chat_env['payloads']) == 3:
+            print('✅ Chat-like payloads handled correctly')
+        else:
+            print('❌ Chat-like payloads handling failed')
+            passed = False
+        
+        # Final result
+        print('\n=== Test Result ===')
+        if passed:
+            print('✅ ALL TESTS PASSED')
+            sys.exit(0)
+        else:
+            print('❌ SOME TESTS FAILED')
+            sys.exit(1)
+            
+    except Exception as e:
+        print(f'❌ Test failed with error: {e}')
+        import traceback
+        traceback.print_exc()
+        sys.exit(1)
+
+
+if __name__ == '__main__':
+    asyncio.run(run_test())