update readme
This commit is contained in:
173
README.md
173
README.md
@@ -14,7 +14,6 @@ A high-performance, bi-directional data bridge for **Julia**, **JavaScript**, **
|
|||||||
- [Features](#features)
|
- [Features](#features)
|
||||||
- [Quick Start](#quick-start)
|
- [Quick Start](#quick-start)
|
||||||
- [API Reference](#api-reference)
|
- [API Reference](#api-reference)
|
||||||
- [Payload Types](#payload-types)
|
|
||||||
- [Cross-Platform Examples](#cross-platform-examples)
|
- [Cross-Platform Examples](#cross-platform-examples)
|
||||||
- [Testing](#testing)
|
- [Testing](#testing)
|
||||||
- [Documentation](#documentation)
|
- [Documentation](#documentation)
|
||||||
@@ -22,6 +21,73 @@ A high-performance, bi-directional data bridge for **Julia**, **JavaScript**, **
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
## Quick Data Format Reference
|
||||||
|
|
||||||
|
All platforms use the same input format for `smartpack()`:
|
||||||
|
|
||||||
|
### Format: `[(dataname, data, type), ...]`
|
||||||
|
|
||||||
|
| Element | Type | Description |
|
||||||
|
|---------|------|-------------|
|
||||||
|
| `dataname` | String | Name for the payload (e.g., `"message"`, `"config.json"`) |
|
||||||
|
| `data` | Any | The actual data (type depends on the `type` parameter below) |
|
||||||
|
| `type` | String | Payload type identifier |
|
||||||
|
|
||||||
|
### Supported Payload Types
|
||||||
|
|
||||||
|
| Type | Julia Data | JavaScript Data | Python Data | Description |
|
||||||
|
|------|------------|-----------------|-------------|-------------|
|
||||||
|
| `"text"` | `String` | `string` | `str` | Plain text |
|
||||||
|
| `"dictionary"` | `Dict` | `Object` | `dict` | JSON object |
|
||||||
|
| `"arrowtable"` | `DataFrame` | ❌ | `pandas.DataFrame` | Arrow IPC table (Desktop only) |
|
||||||
|
| `"jsontable"` | `DataFrame` | `Array<Object>` | `list[dict]` | JSON table |
|
||||||
|
| `"image"` | `Vector{UInt8}` | `Uint8Array` | `bytes` | Image data |
|
||||||
|
| `"audio"` | `Vector{UInt8}` | `Uint8Array` | `bytes` | Audio data |
|
||||||
|
| `"video"` | `Vector{UInt8}` | `Uint8Array` | `bytes` | Video data |
|
||||||
|
| `"binary"` | `Vector{UInt8}` | `Uint8Array` | `bytes` | Binary data |
|
||||||
|
|
||||||
|
### Examples
|
||||||
|
|
||||||
|
**Sending multiple payloads:**
|
||||||
|
```julia
|
||||||
|
# Julia
|
||||||
|
data = [
|
||||||
|
("message", "Hello!", "text"),
|
||||||
|
("config", Dict("key" => "value"), "dictionary"),
|
||||||
|
("file", file_bytes, "binary")
|
||||||
|
]
|
||||||
|
env, json_str = msghandler.smartpack("subject", data, options...)
|
||||||
|
```
|
||||||
|
|
||||||
|
```python
|
||||||
|
# Python
|
||||||
|
data = [
|
||||||
|
("message", "Hello!", "text"),
|
||||||
|
("config", {"key": "value"}, "dictionary"),
|
||||||
|
("file", file_bytes, "binary")
|
||||||
|
]
|
||||||
|
env, json_str = await smartpack("subject", data, **options)
|
||||||
|
```
|
||||||
|
|
||||||
|
```javascript
|
||||||
|
// JavaScript
|
||||||
|
const data = [
|
||||||
|
["message", "Hello!", "text"],
|
||||||
|
["config", {key: "value"}, "dictionary"],
|
||||||
|
["file", fileBytes, "binary"]
|
||||||
|
];
|
||||||
|
const [env, json_str] = await smartpack("subject", data, options);
|
||||||
|
```
|
||||||
|
|
||||||
|
**Important Notes:**
|
||||||
|
- Always wrap payloads in a list `[]`, even for single payloads
|
||||||
|
- Type must be one of the supported types above
|
||||||
|
- Large payloads (≥500KB) automatically use link transport
|
||||||
|
- Browser only supports: `"text"`, `"dictionary"`, `"jsontable"`, `"image"`, `"audio"`, `"video"`, `"binary"`
|
||||||
|
(No Arrow IPC due to browser incompatibility)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
## Overview
|
## Overview
|
||||||
|
|
||||||
msghandler enables seamless communication across multiple platforms through NATS, with intelligent transport selection based on payload size:
|
msghandler enables seamless communication across multiple platforms through NATS, with intelligent transport selection based on payload size:
|
||||||
@@ -109,6 +175,14 @@ msghandler enables seamless communication across multiple platforms through NATS
|
|||||||
```julia
|
```julia
|
||||||
using msghandler, NATS
|
using msghandler, NATS
|
||||||
|
|
||||||
|
# Data format: [(dataname, data, type), ...]
|
||||||
|
# Each tuple contains:
|
||||||
|
# 1. dataname: String name for the payload (e.g., "message")
|
||||||
|
# 2. data: The actual data (String, bytes, DataFrame, etc.)
|
||||||
|
# 3. type: String type indicator (e.g., "text", "binary", "dictionary")
|
||||||
|
# Supported types: "text", "dictionary", "arrowtable", "jsontable",
|
||||||
|
# "image", "audio", "video", "binary"
|
||||||
|
|
||||||
payload_1 = ("test_message", "Hello World", "text")
|
payload_1 = ("test_message", "Hello World", "text")
|
||||||
|
|
||||||
file_path_large_image = "./test/large_image.png"
|
file_path_large_image = "./test/large_image.png"
|
||||||
@@ -116,7 +190,7 @@ file_data_large_image = read(file_path_large_image)
|
|||||||
filename_large_image = basename(file_path_large_image)
|
filename_large_image = basename(file_path_large_image)
|
||||||
payload_2 = (filename_large_image, file_data_large_image, "binary")
|
payload_2 = (filename_large_image, file_data_large_image, "binary")
|
||||||
|
|
||||||
payloads = [payload_1, payload_2] # payloads to be send format is [(dataname, data, type), ...] tuples.
|
payloads = [payload_1, payload_2] # List of tuples
|
||||||
envelope, envelope_json_str = msghandler.smartpack("test.topic",
|
envelope, envelope_json_str = msghandler.smartpack("test.topic",
|
||||||
payloads;
|
payloads;
|
||||||
broker_url="nats.yiem.cc",
|
broker_url="nats.yiem.cc",
|
||||||
@@ -159,6 +233,13 @@ console.log("Message sent!");
|
|||||||
```python
|
```python
|
||||||
from msghandler import smartpack
|
from msghandler import smartpack
|
||||||
|
|
||||||
|
# Data format: [(dataname, data, type), ...]
|
||||||
|
# Each tuple contains:
|
||||||
|
# 1. dataname: String name for the payload (e.g., "message")
|
||||||
|
# 2. data: The actual data (str, bytes, dict, list, etc.)
|
||||||
|
# 3. type: String type indicator (e.g., "text", "binary", "dictionary")
|
||||||
|
# Supported types: "text", "dictionary", "jsontable", "image", "audio", "video", "binary"
|
||||||
|
|
||||||
data = [("message", "Hello World", "text")]
|
data = [("message", "Hello World", "text")]
|
||||||
env, env_json_str = await smartpack(
|
env, env_json_str = await smartpack(
|
||||||
"/chat/room1",
|
"/chat/room1",
|
||||||
@@ -211,14 +292,77 @@ end
|
|||||||
|
|
||||||
### Unified API Standard
|
### Unified API Standard
|
||||||
|
|
||||||
All platforms use the same input/output format for payloads:
|
All platforms use the same input/output format for payloads.
|
||||||
|
|
||||||
**Input format for `smartpack`:**
|
#### Input Format for `smartpack`
|
||||||
```
|
|
||||||
[(dataname1, data1, type1), (dataname2, data2, type2), ...]
|
**Format:** `[(dataname1, data1, type1), (dataname2, data2, type2), ...]`
|
||||||
|
|
||||||
|
**Each tuple contains 3 elements:**
|
||||||
|
1. **`dataname`** (String) - Name for the payload (e.g., `"message"`, `"config.json"`, `"avatar.png"`)
|
||||||
|
2. **`data`** (Any) - The actual data to send (type depends on `type`):
|
||||||
|
- `text` → String
|
||||||
|
- `dictionary` → Dict/Object
|
||||||
|
- `arrowtable` → DataFrame/Arrow.Table
|
||||||
|
- `jsontable` → List of dicts/Vector{Dict}
|
||||||
|
- `image/audio/video/binary` → bytes/Uint8Array/Vector{UInt8}
|
||||||
|
3. **`type`** (String) - Payload type (required):
|
||||||
|
- `"text"` - Plain text string
|
||||||
|
- `"dictionary"` - JSON-serializable dictionary
|
||||||
|
- `"arrowtable"` - Apache Arrow IPC table (Desktop only)
|
||||||
|
- `"jsontable"` - JSON table (all platforms)
|
||||||
|
- `"image"` - Image data (PNG, JPG)
|
||||||
|
- `"audio"` - Audio data (WAV, MP3)
|
||||||
|
- `"video"` - Video data (MP4, AVI)
|
||||||
|
- `"binary"` - Generic binary data
|
||||||
|
|
||||||
|
**Important:**
|
||||||
|
- Always wrap payloads in a list, even for single payloads
|
||||||
|
- Type must be one of the supported types above
|
||||||
|
- Large payloads (≥500KB by default) automatically use link transport
|
||||||
|
|
||||||
|
**Example:**
|
||||||
|
```julia
|
||||||
|
# Julia
|
||||||
|
data = [
|
||||||
|
("message", "Hello World", "text"),
|
||||||
|
("config", Dict("key" => "value"), "dictionary"),
|
||||||
|
("file", file_bytes, "binary")
|
||||||
|
]
|
||||||
```
|
```
|
||||||
|
|
||||||
**Output format for `smartunpack`:**
|
```python
|
||||||
|
# Python
|
||||||
|
data = [
|
||||||
|
("message", "Hello World", "text"),
|
||||||
|
("config", {"key": "value"}, "dictionary"),
|
||||||
|
("file", file_bytes, "binary")
|
||||||
|
]
|
||||||
|
```
|
||||||
|
|
||||||
|
```javascript
|
||||||
|
// JavaScript
|
||||||
|
const data = [
|
||||||
|
["message", "Hello World", "text"],
|
||||||
|
["config", {key: "value"}, "dictionary"],
|
||||||
|
["file", fileBytes, "binary"]
|
||||||
|
];
|
||||||
|
```
|
||||||
|
|
||||||
|
### Common Payload Examples
|
||||||
|
|
||||||
|
| Use Case | Data Format | Type |
|
||||||
|
|----------|-------------|------|
|
||||||
|
| Simple text message | `("message", "Hello World", "text")` | `"text"` |
|
||||||
|
| Configuration JSON | `("config", Dict("key" => "value"), "dictionary")` | `"dictionary"` |
|
||||||
|
| Table as Arrow | `("data", dataframe, "arrowtable")` | `"arrowtable"` |
|
||||||
|
| Table as JSON | `("data", [{"col": 1}], "jsontable")` | `"jsontable"` |
|
||||||
|
| Image file | `("avatar.png", read("file.png"), "image")` | `"image"` |
|
||||||
|
| Audio file | `("audio.mp3", audio_bytes, "audio")` | `"audio"` |
|
||||||
|
| Video file | `("video.mp4", video_bytes, "video")` | `"video"` |
|
||||||
|
| Binary file | `("data.bin", file_bytes, "binary")` | `"binary"` |
|
||||||
|
|
||||||
|
#### Output Format for `smartunpack`
|
||||||
```json
|
```json
|
||||||
{
|
{
|
||||||
"correlation_id": "...",
|
"correlation_id": "...",
|
||||||
@@ -454,21 +598,6 @@ print("Received payloads:", env["payloads"])
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Payload Types
|
|
||||||
|
|
||||||
| Type | Julia | JavaScript | Python | MicroPython | Description |
|
|
||||||
|------|-------|------------|--------|-------------|-------------|
|
|
||||||
| `text` | `String` | `string` | `str` | `str` | Plain text strings |
|
|
||||||
| `dictionary` | `Dict`, `NamedTuple` | `Object`, `Array` | `dict`, `list` | `dict` | JSON-serializable dictionaries |
|
|
||||||
| `arrowtable` | `DataFrame`, `Arrow.Table` | ❌ (Browser), ✅ (Node.js) | `pandas.DataFrame` | ❌ | Tabular data (Arrow IPC) |
|
|
||||||
| `jsontable` | `DataFrame`, `Vector{NamedTuple}` | `Array<Object>` | `list[dict]` | ⚠️ | Tabular data (JSON) - **Only table type in Browser** |
|
|
||||||
| `image` | `Vector{UInt8}` | `Uint8Array`, `Buffer` | `bytes` | `bytearray` | Image data (PNG, JPG) |
|
|
||||||
| `audio` | `Vector{UInt8}` | `Uint8Array`, `Buffer` | `bytes` | `bytearray` | Audio data (WAV, MP3) |
|
|
||||||
| `video` | `Vector{UInt8}` | `Uint8Array`, `Buffer` | `bytes` | `bytearray` | Video data (MP4, AVI) |
|
|
||||||
| `binary` | `Vector{UInt8}`, `IOBuffer` | `Uint8Array`, `Buffer` | `bytes`, `bytearray` | `bytearray` | Generic binary data |
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Cross-Platform Examples
|
## Cross-Platform Examples
|
||||||
|
|
||||||
### Example 1: Chat with Mixed Content
|
### Example 1: Chat with Mixed Content
|
||||||
|
|||||||
Reference in New Issue
Block a user