update docs

2026-05-22 06:17:30 +07:00
parent 396e0848da
commit 312d14b28f
4 changed files with 851 additions and 995 deletions
--- a/docs/implementation-plan.md
+++ b/docs/implementation-plan.md
@@ -0,0 +1,416 @@
+# Implementation Plan: msghandler
+
+**Version**: 1.3.0  
+**Date**: 2026-05-19  
+**Status**: Active  
+**Ground Truth**: [`src/msghandler.jl`](../src/msghandler.jl)
+
+---
+
+## 1. Implementation Phases and Timeline
+
+### Phase 1: Core API Implementation (Week 1-2)
+| Task | Priority | Estimated Effort | Status |
+|------|----------|-----------------|--------|
+| Core `smartpack()` implementation | P0 | 3 days | ✅ Complete |
+| Core `smartunpack()` implementation | P0 | 3 days | ✅ Complete |
+| Message envelope structure | P0 | 2 days | ✅ Complete |
+| Payload type handling | P0 | 2 days | ✅ Complete |
+| Transport adapter layer | P0 | 3 days | ✅ Complete |
+
+**Deliverables**:
+- Julia module: `src/msghandler.jl`
+- Node.js module: `src/msghandler_ssr.js`
+- Browser module: `src/msghandler_csr.js`
+- Python module: `src/msghandler.py`
+- MicroPython module: `src/msghandler_mpy.py`
+
+### Phase 2: File Server Integration (Week 3)
+| Task | Priority | Estimated Effort | Status |
+|------|----------|-----------------|--------|
+| File server upload handler | P1 | 2 days | ✅ Complete |
+| File server download handler | P1 | 2 days | ✅ Complete |
+| Exponential backoff logic | P1 | 1 day | ✅ Complete |
+| Plik integration | P1 | 2 days | ✅ Complete |
+
+**Deliverables**:
+- Upload handler with plik_oneshot_upload
+- Download handler with retry logic
+- Configurable file server URL
+
+### Phase 3: Platform-Specific Features (Week 4)
+| Task | Priority | Estimated Effort | Status |
+|------|----------|-----------------|--------|
+| Arrow IPC support (Desktop) | P1 | 3 days | ✅ Complete |
+| JSON table support (Browser) | P1 | 2 days | ✅ Complete |
+| Browser WebSocket transport | P1 | 2 days | ✅ Complete |
+| MicroPython optimizations | P2 | 2 days | ✅ Complete |
+
+**Deliverables**:
+- Arrow IPC serialization for tabular data
+- JSON table format for browser compatibility
+- Browser-specific transport layer
+- Memory-optimized MicroPython implementation
+
+### Phase 4: Cross-Platform Testing (Week 5)
+| Task | Priority | Estimated Effort | Status |
+|------|----------|-----------------|--------|
+| Text message tests | P1 | 1 day | ✅ Complete |
+| Dictionary tests | P1 | 1 day | ✅ Complete |
+| Tabular data tests | P1 | 2 days | ✅ Complete |
+| Mixed payload tests | P1 | 2 days | ✅ Complete |
+| Large file tests | P1 | 2 days | ✅ Complete |
+
+**Deliverables**:
+- Platform-specific test suites
+- Integration test scenarios
+- Performance benchmarks
+
+### Phase 5: Documentation & Examples (Week 6)
+| Task | Priority | Estimated Effort | Status |
+|------|----------|-----------------|--------|
+| API documentation | P2 | 2 days | ✅ Complete |
+| Walkthrough examples | P2 | 2 days | ✅ Complete |
+| Architecture diagrams | P2 | 1 day | ✅ Complete |
+| Deployment guides | P2 | 1 day | ✅ Complete |
+
+**Deliverables**:
+- Comprehensive documentation
+- Code examples for all platforms
+- Deployment runbooks
+
+---
+
+## 2. Module/Component Breakdown
+
+### Core Modules
+
+#### msghandler.jl (Julia)
+```
+src/
+└── msghandler.jl
+    ├── Constants (DEFAULT_SIZE_THRESHOLD, etc.)
+    ├── msg_payload_v1 struct
+    ├── msg_envelope_v1 struct
+    ├── Serialization functions
+    │   ├── serialize_text()
+    │   ├── serialize_dictionary()
+    │   ├── serialize_arrowtable()
+    │   ├── serialize_jsontable()
+    │   └── serialize_binary()
+    ├── Deserialization functions
+    │   ├── deserialize_text()
+    │   ├── deserialize_dictionary()
+    │   ├── deserialize_arrowtable()
+    │   ├── deserialize_jsontable()
+    │   └── deserialize_binary()
+    ├── File server handlers
+    │   ├── plik_oneshot_upload()
+    │   └── _fetch_with_backoff()
+    ├── smartpack() - Main sender function
+    └── smartunpack() - Main receiver function
+```
+
+**Dependencies**:
+- JSON.jl (JSON serialization)
+- Arrow.jl (Arrow IPC)
+- HTTP.jl (File server)
+- UUIDs.jl (IDs)
+- DataFrames.jl (DataFrame support)
+
+#### msghandler_ssr.js (Node.js)
+```
+src/
+├── msghandler_ssr.js
+│   ├── Constants
+│   ├── msg_payload_v1 class
+│   ├── msg_envelope_v1 class
+│   ├── Serialization methods
+│   ├── Deserialization methods
+│   ├── File server handlers
+│   ├── smartpack() function
+│   └── smartunpack() function
+└── nats/
+    ├── NATSClient.js
+    └── NATSConnectionPool.js
+```
+
+**Dependencies**:
+- nats (NATS client)
+- node-fetch (HTTP file server)
+
+#### msghandler_csr.js (Browser)
+```
+src/
+└── msghandler_csr.js
+    ├── Constants
+    ├── msg_payload_v1 class
+    ├── msg_envelope_v1 class
+    ├── Serialization methods (JSON table only)
+    ├── Deserialization methods
+    ├── File server handlers (browser-compatible)
+    ├── smartpack() function
+    └── smartunpack() function
+```
+
+**Dependencies**:
+- nats.ws (Browser NATS client)
+
+#### msghandler.py (Python)
+```
+src/
+└── msghandler.py
+    ├── Constants
+    ├── msg_payload_v1 class
+    ├── msg_envelope_v1 class
+    ├── Serialization methods
+    ├── Deserialization methods
+    ├── File server handlers
+    ├── smartpack() async function
+    └── smartunpack() async function
+```
+
+**Dependencies**:
+- aiohttp (HTTP file server)
+- pyarrow (Arrow IPC)
+- uuid (IDs)
+
+#### msghandler.rs (Rust)
+```
+src/
+├── msghandler.rs
+│   ├── Constants
+│   ├── msg_payload_v1 struct
+│   ├── msg_envelope_v1 struct
+│   ├── Serialization traits
+│   ├── Deserialization traits
+│   ├── File server handlers
+│   ├── smartpack() async function
+│   └── smartunpack() async function
+├── Payload enum
+├── smartpackOptions struct
+└── smartunpackOptions struct
+```
+
+**Dependencies**:
+- tokio (Async runtime)
+- serde (JSON serialization)
+- reqwest (HTTP file server)
+- arrow2 (Arrow IPC)
+
+#### msghandler_mpy.py (MicroPython)
+```
+src/
+└── msghandler_mpy.py
+    ├── Constants (lower thresholds)
+    ├── msg_payload_v1 class
+    ├── msg_envelope_v1 class
+    ├── serialize_text()
+    ├── deserialize_text()
+    ├── serialize_dictionary()
+    ├── deserialize_dictionary()
+    └── smartpack()/smartunpack() functions
+```
+
+**Constraints**:
+- Limited to text and dictionary types
+- Direct transport only (no file server)
+- 100KB threshold for memory constraints
+
+---
+
+## 3. Task List
+
+### Core API Tasks
+
+| Task ID | Description | Assignee | Priority | Status |
+|---------|-------------|----------|----------|--------|
+| T-001 | Implement `smartpack()` with tuple format | Developer A | P0 | ✅ Complete |
+| T-002 | Implement `smartunpack()` with type handling | Developer A | P0 | ✅ Complete |
+| T-003 | Create message envelope structure | Developer A | P0 | ✅ Complete |
+| T-004 | Implement transport adapter | Developer B | P0 | ✅ Complete |
+| T-005 | Add correlation ID support | Developer A | P0 | ✅ Complete |
+
+### File Server Tasks
+
+| Task ID | Description | Assignee | Priority | Status |
+|---------|-------------|----------|----------|--------|
+| T-006 | Implement Plik upload handler | Developer B | P1 | ✅ Complete |
+| T-007 | Implement file download with retry | Developer B | P1 | ✅ Complete |
+| T-008 | Add exponential backoff logic | Developer B | P1 | ✅ Complete |
+
+### Platform Tasks
+
+| Task ID | Description | Assignee | Priority | Status |
+|---------|-------------|----------|----------|--------|
+| T-009 | Implement Arrow IPC (Julia/Python/Node.js) | Developer A | P1 | ✅ Complete |
+| T-010 | Implement JSON table (Browser) | Developer B | P1 | ✅ Complete |
+| T-011 | Implement MicroPython optimizations | Developer C | P2 | ✅ Complete |
+| T-012 | Browser WebSocket transport | Developer B | P1 | ✅ Complete |
+
+### Testing Tasks
+
+| Task ID | Description | Assignee | Priority | Status |
+|---------|-------------|----------|------------------|
+| T-013 | Text message tests | QA Team | P1 | ✅ Complete |
+| T-014 | Dictionary tests | QA Team | P1 | ✅ Complete |
+| T-015 | Tabular data tests | QA Team | P1 | ✅ Complete |
+| T-016 | Mixed payload tests | QA Team | P1 | ✅ Complete |
+| T-017 | Large file tests | QA Team | P1 | ✅ Complete |
+
+---
+
+## 4. Test Strategy
+
+### Unit Tests
+
+| Test Category | Coverage | Files | Requirements |
+|---------------|----------|-------|--------------|
+| Serialization | All payload types | `test/test_*_sender.*` | FR-001 through FR-012 |
+| Deserialization | All payload types | `test/test_*_receiver.*` | FR-001 through FR-012 |
+| Transport selection | Direct vs link | `test/test_*_mix_payloads.*` | FR-003, FR-004, FR-006 |
+| File server upload | Plik integration | Platform-specific | FR-008, FR-009 |
+| File server download | Exponential backoff | Platform-specific | FR-010, FR-011 |
+
+### Integration Tests
+
+| Scenario | Platforms | Payloads | Transport | Requirements |
+|----------|-----------|----------|-----------|--------------|
+| Single text (small) | All | text | direct | FR-001, FR-012 |
+| Single dictionary (small) | All | dictionary | direct | FR-002, FR-012 |
+| Single arrow table (small) | Desktop | arrowtable | direct | FR-002, FR-012 |
+| Single JSON table (small) | All | jsontable | direct | FR-001, FR-002, FR-006 |
+| Single image (small) | All | image | direct | FR-001, FR-006 |
+| Single text (large) | All | text | link | FR-003, FR-008, FR-009 |
+| Mixed payloads | All | text + dictionary + image | mixed | FR-006, FR-007 |
+
+### Test Coverage Targets
+
+| Phase | Coverage Target | Method |
+|-------|----------------|--------|
+| Phase 1 | 70% | Unit tests per platform |
+| Phase 2 | 80% | Add integration tests |
+| Phase 3 | 85% | Add edge case tests |
+| Phase 4 | 90% | Add performance tests |
+
+---
+
+## 5. Build and Deployment Preparation
+
+### Continuous Integration
+
+| Check | Command | Purpose |
+|-------|---------|---------|
+| Linting | `npm run lint` | Code style enforcement |
+| Type checking | `npx tsc --noEmit` | Type safety (JavaScript/TypeScript) |
+| Unit tests | `npm test` | Functionality validation |
+| Integration tests | `npm run test:integration` | Cross-platform validation |
+| Coverage | `npm run coverage` | Test coverage tracking |
+
+### Deployment Pipeline
+
+```
+GitHub Push
+    ↓
+CI/CD Pipeline
+    ↓
+├──→ Linting (all platforms)
+├──→ Unit tests (all platforms)
+├──→ Integration tests (cross-platform)
+├──→ Coverage report
+└──→ Build documentation
+    ↓
+Release (if all checks pass)
+    ↓
+├──→ GitHub Releases
+├──→ Package registry (npm, PyPI)
+└──→ Documentation site
+```
+
+---
+
+## 6. Risk Mitigation
+
+### Known Blockers
+
+| Risk | Mitigation Step | Owner |
+|------|----------------|-------|
+| **Browser Arrow IPC** | Use JSON table as fallback | Developer B |
+| **MicroPython memory** | 100KB threshold, direct transport only | Developer C |
+| **File server availability** | Exponential backoff with graceful degradation | Developer B |
+
+### Known Unknowns
+
+| Unknown | Monitoring Strategy | Response Plan |
+|---------|-------------------|---------------|
+| Platform-specific bugs | Comprehensive test coverage | Hotfix with platform-specific handling |
+| Performance bottlenecks | Load testing and profiling | Optimized serialization/deserialization |
+
+---
+
+## 7. Requirements Traceability
+
+### Functional Requirements
+
+| Requirement ID | Implementation Location | Status |
+|---------------|------------------------|--------|
+| FR-001 | All platform modules | ✅ Complete |
+| FR-002 | All platform modules | ✅ Complete |
+| FR-003 | All platform modules (size_threshold logic) | ✅ Complete |
+| FR-004 | All platform modules | ✅ Complete |
+| FR-005 | MicroPython module | ✅ Complete |
+| FR-006 | All platform modules | ✅ Complete |
+| FR-007 | All platform modules | ✅ Complete |
+| FR-008 | All platform modules | ✅ Complete |
+| FR-009 | All platform modules | ✅ Complete |
+| FR-010 | All platform modules | ✅ Complete |
+| FR-011 | All platform modules | ✅ Complete |
+| FR-012 | All platform modules | ✅ Complete |
+| FR-013 | All platform modules | ✅ Complete |
+| FR-014 | All platform modules | ✅ Complete |
+
+### Non-Functional Requirements
+
+| NFR ID | Implementation Location | Status |
+|--------|------------------------|--------|
+| NFR-101 | Serialization functions | ✅ Complete |
+| NFR-102 | Deserialization functions | ✅ Complete |
+| NFR-103 | Transport adapter | ✅ Complete |
+| NFR-104 | File upload handler | ✅ Complete |
+| NFR-105 | File download handler | ✅ Complete |
+| NFR-106 | MicroPython module | ✅ Complete |
+| NFR-107 | Performance benchmarks | ✅ Complete |
+| NFR-201 | Transport adapter | ✅ Complete |
+| NFR-202 | File download retry logic | ✅ Complete |
+| NFR-203 | Transport adapter | ✅ Complete |
+| NFR-401 | Message envelope | ✅ Complete |
+| NFR-402 | Metrics instrumentation | ✅ Complete |
+| NFR-403 | Correlation ID propagation | ✅ Complete |
+
+---
+
+## 8. Validation Gates
+
+### Pre-Release Checklist
+
+| Gate | Check | Pass Criteria |
+|------|-------|--------------|
+| **G-001** | All unit tests pass | 100% pass rate per platform |
+| **G-002** | Integration tests pass | Cross-platform round-trip successful |
+| **G-003** | Coverage threshold | ≥80% line coverage |
+| **G-004** | Linting clean | No warnings or errors |
+| **G-005** | Specification compliance | All spec rules validated |
+| **G-006** | Documentation complete | All required docs present |
+
+### CI/CD Validation
+
+| Check | Command | Failure Action |
+|-------|---------|---------------|
+| Syntax | `julia --check-base` | Block PR |
+| Unit tests | `julia test/runtests.jl` | Block PR |
+| Integration | `npm run test:integration` | Block PR |
+| Coverage | `codecov` | Report only |
+
+---
+
+*This implementation plan is versioned and maintained in git alongside the codebase. All implementations must adhere to this plan.*