update readme
This commit is contained in:
173
README.md
173
README.md
@@ -14,7 +14,6 @@ A high-performance, bi-directional data bridge for **Julia**, **JavaScript**, **
|
||||
- [Features](#features)
|
||||
- [Quick Start](#quick-start)
|
||||
- [API Reference](#api-reference)
|
||||
- [Payload Types](#payload-types)
|
||||
- [Cross-Platform Examples](#cross-platform-examples)
|
||||
- [Testing](#testing)
|
||||
- [Documentation](#documentation)
|
||||
@@ -22,6 +21,73 @@ A high-performance, bi-directional data bridge for **Julia**, **JavaScript**, **
|
||||
|
||||
---
|
||||
|
||||
## Quick Data Format Reference
|
||||
|
||||
All platforms use the same input format for `smartpack()`:
|
||||
|
||||
### Format: `[(dataname, data, type), ...]`
|
||||
|
||||
| Element | Type | Description |
|
||||
|---------|------|-------------|
|
||||
| `dataname` | String | Name for the payload (e.g., `"message"`, `"config.json"`) |
|
||||
| `data` | Any | The actual data (type depends on the `type` parameter below) |
|
||||
| `type` | String | Payload type identifier |
|
||||
|
||||
### Supported Payload Types
|
||||
|
||||
| Type | Julia Data | JavaScript Data | Python Data | Description |
|
||||
|------|------------|-----------------|-------------|-------------|
|
||||
| `"text"` | `String` | `string` | `str` | Plain text |
|
||||
| `"dictionary"` | `Dict` | `Object` | `dict` | JSON object |
|
||||
| `"arrowtable"` | `DataFrame` | ❌ | `pandas.DataFrame` | Arrow IPC table (Desktop only) |
|
||||
| `"jsontable"` | `DataFrame` | `Array<Object>` | `list[dict]` | JSON table |
|
||||
| `"image"` | `Vector{UInt8}` | `Uint8Array` | `bytes` | Image data |
|
||||
| `"audio"` | `Vector{UInt8}` | `Uint8Array` | `bytes` | Audio data |
|
||||
| `"video"` | `Vector{UInt8}` | `Uint8Array` | `bytes` | Video data |
|
||||
| `"binary"` | `Vector{UInt8}` | `Uint8Array` | `bytes` | Binary data |
|
||||
|
||||
### Examples
|
||||
|
||||
**Sending multiple payloads:**
|
||||
```julia
|
||||
# Julia
|
||||
data = [
|
||||
("message", "Hello!", "text"),
|
||||
("config", Dict("key" => "value"), "dictionary"),
|
||||
("file", file_bytes, "binary")
|
||||
]
|
||||
env, json_str = msghandler.smartpack("subject", data, options...)
|
||||
```
|
||||
|
||||
```python
|
||||
# Python
|
||||
data = [
|
||||
("message", "Hello!", "text"),
|
||||
("config", {"key": "value"}, "dictionary"),
|
||||
("file", file_bytes, "binary")
|
||||
]
|
||||
env, json_str = await smartpack("subject", data, **options)
|
||||
```
|
||||
|
||||
```javascript
|
||||
// JavaScript
|
||||
const data = [
|
||||
["message", "Hello!", "text"],
|
||||
["config", {key: "value"}, "dictionary"],
|
||||
["file", fileBytes, "binary"]
|
||||
];
|
||||
const [env, json_str] = await smartpack("subject", data, options);
|
||||
```
|
||||
|
||||
**Important Notes:**
|
||||
- Always wrap payloads in a list `[]`, even for single payloads
|
||||
- Type must be one of the supported types above
|
||||
- Large payloads (≥500KB) automatically use link transport
|
||||
- Browser only supports: `"text"`, `"dictionary"`, `"jsontable"`, `"image"`, `"audio"`, `"video"`, `"binary"`
|
||||
(No Arrow IPC due to browser incompatibility)
|
||||
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
msghandler enables seamless communication across multiple platforms through NATS, with intelligent transport selection based on payload size:
|
||||
@@ -109,6 +175,14 @@ msghandler enables seamless communication across multiple platforms through NATS
|
||||
```julia
|
||||
using msghandler, NATS
|
||||
|
||||
# Data format: [(dataname, data, type), ...]
|
||||
# Each tuple contains:
|
||||
# 1. dataname: String name for the payload (e.g., "message")
|
||||
# 2. data: The actual data (String, bytes, DataFrame, etc.)
|
||||
# 3. type: String type indicator (e.g., "text", "binary", "dictionary")
|
||||
# Supported types: "text", "dictionary", "arrowtable", "jsontable",
|
||||
# "image", "audio", "video", "binary"
|
||||
|
||||
payload_1 = ("test_message", "Hello World", "text")
|
||||
|
||||
file_path_large_image = "./test/large_image.png"
|
||||
@@ -116,7 +190,7 @@ file_data_large_image = read(file_path_large_image)
|
||||
filename_large_image = basename(file_path_large_image)
|
||||
payload_2 = (filename_large_image, file_data_large_image, "binary")
|
||||
|
||||
payloads = [payload_1, payload_2] # payloads to be send format is [(dataname, data, type), ...] tuples.
|
||||
payloads = [payload_1, payload_2] # List of tuples
|
||||
envelope, envelope_json_str = msghandler.smartpack("test.topic",
|
||||
payloads;
|
||||
broker_url="nats.yiem.cc",
|
||||
@@ -159,6 +233,13 @@ console.log("Message sent!");
|
||||
```python
|
||||
from msghandler import smartpack
|
||||
|
||||
# Data format: [(dataname, data, type), ...]
|
||||
# Each tuple contains:
|
||||
# 1. dataname: String name for the payload (e.g., "message")
|
||||
# 2. data: The actual data (str, bytes, dict, list, etc.)
|
||||
# 3. type: String type indicator (e.g., "text", "binary", "dictionary")
|
||||
# Supported types: "text", "dictionary", "jsontable", "image", "audio", "video", "binary"
|
||||
|
||||
data = [("message", "Hello World", "text")]
|
||||
env, env_json_str = await smartpack(
|
||||
"/chat/room1",
|
||||
@@ -211,14 +292,77 @@ end
|
||||
|
||||
### Unified API Standard
|
||||
|
||||
All platforms use the same input/output format for payloads:
|
||||
All platforms use the same input/output format for payloads.
|
||||
|
||||
**Input format for `smartpack`:**
|
||||
```
|
||||
[(dataname1, data1, type1), (dataname2, data2, type2), ...]
|
||||
#### Input Format for `smartpack`
|
||||
|
||||
**Format:** `[(dataname1, data1, type1), (dataname2, data2, type2), ...]`
|
||||
|
||||
**Each tuple contains 3 elements:**
|
||||
1. **`dataname`** (String) - Name for the payload (e.g., `"message"`, `"config.json"`, `"avatar.png"`)
|
||||
2. **`data`** (Any) - The actual data to send (type depends on `type`):
|
||||
- `text` → String
|
||||
- `dictionary` → Dict/Object
|
||||
- `arrowtable` → DataFrame/Arrow.Table
|
||||
- `jsontable` → List of dicts/Vector{Dict}
|
||||
- `image/audio/video/binary` → bytes/Uint8Array/Vector{UInt8}
|
||||
3. **`type`** (String) - Payload type (required):
|
||||
- `"text"` - Plain text string
|
||||
- `"dictionary"` - JSON-serializable dictionary
|
||||
- `"arrowtable"` - Apache Arrow IPC table (Desktop only)
|
||||
- `"jsontable"` - JSON table (all platforms)
|
||||
- `"image"` - Image data (PNG, JPG)
|
||||
- `"audio"` - Audio data (WAV, MP3)
|
||||
- `"video"` - Video data (MP4, AVI)
|
||||
- `"binary"` - Generic binary data
|
||||
|
||||
**Important:**
|
||||
- Always wrap payloads in a list, even for single payloads
|
||||
- Type must be one of the supported types above
|
||||
- Large payloads (≥500KB by default) automatically use link transport
|
||||
|
||||
**Example:**
|
||||
```julia
|
||||
# Julia
|
||||
data = [
|
||||
("message", "Hello World", "text"),
|
||||
("config", Dict("key" => "value"), "dictionary"),
|
||||
("file", file_bytes, "binary")
|
||||
]
|
||||
```
|
||||
|
||||
**Output format for `smartunpack`:**
|
||||
```python
|
||||
# Python
|
||||
data = [
|
||||
("message", "Hello World", "text"),
|
||||
("config", {"key": "value"}, "dictionary"),
|
||||
("file", file_bytes, "binary")
|
||||
]
|
||||
```
|
||||
|
||||
```javascript
|
||||
// JavaScript
|
||||
const data = [
|
||||
["message", "Hello World", "text"],
|
||||
["config", {key: "value"}, "dictionary"],
|
||||
["file", fileBytes, "binary"]
|
||||
];
|
||||
```
|
||||
|
||||
### Common Payload Examples
|
||||
|
||||
| Use Case | Data Format | Type |
|
||||
|----------|-------------|------|
|
||||
| Simple text message | `("message", "Hello World", "text")` | `"text"` |
|
||||
| Configuration JSON | `("config", Dict("key" => "value"), "dictionary")` | `"dictionary"` |
|
||||
| Table as Arrow | `("data", dataframe, "arrowtable")` | `"arrowtable"` |
|
||||
| Table as JSON | `("data", [{"col": 1}], "jsontable")` | `"jsontable"` |
|
||||
| Image file | `("avatar.png", read("file.png"), "image")` | `"image"` |
|
||||
| Audio file | `("audio.mp3", audio_bytes, "audio")` | `"audio"` |
|
||||
| Video file | `("video.mp4", video_bytes, "video")` | `"video"` |
|
||||
| Binary file | `("data.bin", file_bytes, "binary")` | `"binary"` |
|
||||
|
||||
#### Output Format for `smartunpack`
|
||||
```json
|
||||
{
|
||||
"correlation_id": "...",
|
||||
@@ -454,21 +598,6 @@ print("Received payloads:", env["payloads"])
|
||||
|
||||
---
|
||||
|
||||
## Payload Types
|
||||
|
||||
| Type | Julia | JavaScript | Python | MicroPython | Description |
|
||||
|------|-------|------------|--------|-------------|-------------|
|
||||
| `text` | `String` | `string` | `str` | `str` | Plain text strings |
|
||||
| `dictionary` | `Dict`, `NamedTuple` | `Object`, `Array` | `dict`, `list` | `dict` | JSON-serializable dictionaries |
|
||||
| `arrowtable` | `DataFrame`, `Arrow.Table` | ❌ (Browser), ✅ (Node.js) | `pandas.DataFrame` | ❌ | Tabular data (Arrow IPC) |
|
||||
| `jsontable` | `DataFrame`, `Vector{NamedTuple}` | `Array<Object>` | `list[dict]` | ⚠️ | Tabular data (JSON) - **Only table type in Browser** |
|
||||
| `image` | `Vector{UInt8}` | `Uint8Array`, `Buffer` | `bytes` | `bytearray` | Image data (PNG, JPG) |
|
||||
| `audio` | `Vector{UInt8}` | `Uint8Array`, `Buffer` | `bytes` | `bytearray` | Audio data (WAV, MP3) |
|
||||
| `video` | `Vector{UInt8}` | `Uint8Array`, `Buffer` | `bytes` | `bytearray` | Video data (MP4, AVI) |
|
||||
| `binary` | `Vector{UInt8}`, `IOBuffer` | `Uint8Array`, `Buffer` | `bytes`, `bytearray` | `bytearray` | Generic binary data |
|
||||
|
||||
---
|
||||
|
||||
## Cross-Platform Examples
|
||||
|
||||
### Example 1: Chat with Mixed Content
|
||||
|
||||
Reference in New Issue
Block a user