# Implementation Plan: msghandler

**Version**: 1.3.0  
**Date**: 2026-05-19  
**Status**: Active  
**Ground Truth**: [`src/msghandler.jl`](../src/msghandler.jl)

---

## 1. Implementation Phases and Timeline

### Phase 1: Core API Implementation (Week 1-2)
| Task | Priority | Estimated Effort | Status |
|------|----------|-----------------|--------|
| Core `smartpack()` implementation | P0 | 3 days | ✅ Complete |
| Core `smartunpack()` implementation | P0 | 3 days | ✅ Complete |
| Message envelope structure | P0 | 2 days | ✅ Complete |
| Payload type handling | P0 | 2 days | ✅ Complete |
| Transport adapter layer | P0 | 3 days | ✅ Complete |

**Deliverables**:
- Julia module: `src/msghandler.jl`
- Node.js module: `src/msghandler_ssr.js`
- Browser module: `src/msghandler_csr.js`
- Python module: `src/msghandler.py`
- MicroPython module: `src/msghandler_mpy.py`

### Phase 2: File Server Integration (Week 3)
| Task | Priority | Estimated Effort | Status |
|------|----------|-----------------|--------|
| File server upload handler | P1 | 2 days | ✅ Complete |
| File server download handler | P1 | 2 days | ✅ Complete |
| Exponential backoff logic | P1 | 1 day | ✅ Complete |
| Plik integration | P1 | 2 days | ✅ Complete |

**Deliverables**:
- Upload handler with plik_oneshot_upload
- Download handler with retry logic
- Configurable file server URL

### Phase 3: Platform-Specific Features (Week 4)
| Task | Priority | Estimated Effort | Status |
|------|----------|-----------------|--------|
| Arrow IPC support (Desktop) | P1 | 3 days | ✅ Complete |
| JSON table support (Browser) | P1 | 2 days | ✅ Complete |
| Browser WebSocket transport | P1 | 2 days | ✅ Complete |
| MicroPython optimizations | P2 | 2 days | ✅ Complete |

**Deliverables**:
- Arrow IPC serialization for tabular data
- JSON table format for browser compatibility
- Browser-specific transport layer
- Memory-optimized MicroPython implementation

### Phase 4: Cross-Platform Testing (Week 5)
| Task | Priority | Estimated Effort | Status |
|------|----------|-----------------|--------|
| Text message tests | P1 | 1 day | ✅ Complete |
| Dictionary tests | P1 | 1 day | ✅ Complete |
| Tabular data tests | P1 | 2 days | ✅ Complete |
| Mixed payload tests | P1 | 2 days | ✅ Complete |
| Large file tests | P1 | 2 days | ✅ Complete |

**Deliverables**:
- Platform-specific test suites
- Integration test scenarios
- Performance benchmarks

### Phase 5: Documentation & Examples (Week 6)
| Task | Priority | Estimated Effort | Status |
|------|----------|-----------------|--------|
| API documentation | P2 | 2 days | ✅ Complete |
| Walkthrough examples | P2 | 2 days | ✅ Complete |
| Architecture diagrams | P2 | 1 day | ✅ Complete |
| Deployment guides | P2 | 1 day | ✅ Complete |

**Deliverables**:
- Comprehensive documentation
- Code examples for all platforms
- Deployment runbooks

---

## 2. Module/Component Breakdown

### Core Modules

#### msghandler.jl (Julia)
```
src/
└── msghandler.jl
    ├── Constants (DEFAULT_SIZE_THRESHOLD, etc.)
    ├── msg_payload_v1 struct
    ├── msg_envelope_v1 struct
    ├── Serialization functions
    │   ├── serialize_text()
    │   ├── serialize_dictionary()
    │   ├── serialize_arrowtable()
    │   ├── serialize_jsontable()
    │   └── serialize_binary()
    ├── Deserialization functions
    │   ├── deserialize_text()
    │   ├── deserialize_dictionary()
    │   ├── deserialize_arrowtable()
    │   ├── deserialize_jsontable()
    │   └── deserialize_binary()
    ├── File server handlers
    │   ├── plik_oneshot_upload()
    │   └── _fetch_with_backoff()
    ├── smartpack() - Main sender function
    └── smartunpack() - Main receiver function
```

**Dependencies**:
- JSON.jl (JSON serialization)
- Arrow.jl (Arrow IPC)
- HTTP.jl (File server)
- UUIDs.jl (IDs)
- DataFrames.jl (DataFrame support)

#### msghandler_ssr.js (Node.js)
```
src/
├── msghandler_ssr.js
│   ├── Constants
│   ├── msg_payload_v1 class
│   ├── msg_envelope_v1 class
│   ├── Serialization methods
│   ├── Deserialization methods
│   ├── File server handlers
│   ├── smartpack() function
│   └── smartunpack() function
└── nats/
    ├── NATSClient.js
    └── NATSConnectionPool.js
```

**Dependencies**:
- nats (NATS client)
- node-fetch (HTTP file server)

#### msghandler_csr.js (Browser)
```
src/
└── msghandler_csr.js
    ├── Constants
    ├── msg_payload_v1 class
    ├── msg_envelope_v1 class
    ├── Serialization methods (JSON table only)
    ├── Deserialization methods
    ├── File server handlers (browser-compatible)
    ├── smartpack() function
    └── smartunpack() function
```

**Dependencies**:
- nats.ws (Browser NATS client)

#### msghandler.py (Python)
```
src/
└── msghandler.py
    ├── Constants
    ├── msg_payload_v1 class
    ├── msg_envelope_v1 class
    ├── Serialization methods
    ├── Deserialization methods
    ├── File server handlers
    ├── smartpack() async function
    └── smartunpack() async function
```

**Dependencies**:
- aiohttp (HTTP file server)
- pyarrow (Arrow IPC)
- uuid (IDs)

#### msghandler.rs (Rust)
```
src/
├── msghandler.rs
│   ├── Constants
│   ├── msg_payload_v1 struct
│   ├── msg_envelope_v1 struct
│   ├── Serialization traits
│   ├── Deserialization traits
│   ├── File server handlers
│   ├── smartpack() async function
│   └── smartunpack() async function
├── Payload enum
├── smartpackOptions struct
└── smartunpackOptions struct
```

**Dependencies**:
- tokio (Async runtime)
- serde (JSON serialization)
- reqwest (HTTP file server)
- arrow2 (Arrow IPC)

#### msghandler_mpy.py (MicroPython)
```
src/
└── msghandler_mpy.py
    ├── Constants (lower thresholds)
    ├── msg_payload_v1 class
    ├── msg_envelope_v1 class
    ├── serialize_text()
    ├── deserialize_text()
    ├── serialize_dictionary()
    ├── deserialize_dictionary()
    └── smartpack()/smartunpack() functions
```

**Constraints**:
- Limited to text and dictionary types
- Direct transport only (no file server)
- 100KB threshold for memory constraints

---

## 3. Task List

### Core API Tasks

| Task ID | Description | Assignee | Priority | Status |
|---------|-------------|----------|----------|--------|
| T-001 | Implement `smartpack()` with tuple format | Developer A | P0 | ✅ Complete |
| T-002 | Implement `smartunpack()` with type handling | Developer A | P0 | ✅ Complete |
| T-003 | Create message envelope structure | Developer A | P0 | ✅ Complete |
| T-004 | Implement transport adapter | Developer B | P0 | ✅ Complete |
| T-005 | Add correlation ID support | Developer A | P0 | ✅ Complete |

### File Server Tasks

| Task ID | Description | Assignee | Priority | Status |
|---------|-------------|----------|----------|--------|
| T-006 | Implement Plik upload handler | Developer B | P1 | ✅ Complete |
| T-007 | Implement file download with retry | Developer B | P1 | ✅ Complete |
| T-008 | Add exponential backoff logic | Developer B | P1 | ✅ Complete |

### Platform Tasks

| Task ID | Description | Assignee | Priority | Status |
|---------|-------------|----------|----------|--------|
| T-009 | Implement Arrow IPC (Julia/Python/Node.js) | Developer A | P1 | ✅ Complete |
| T-010 | Implement JSON table (Browser) | Developer B | P1 | ✅ Complete |
| T-011 | Implement MicroPython optimizations | Developer C | P2 | ✅ Complete |
| T-012 | Browser WebSocket transport | Developer B | P1 | ✅ Complete |

### Testing Tasks

| Task ID | Description | Assignee | Priority | Status |
|---------|-------------|----------|------------------|
| T-013 | Text message tests | QA Team | P1 | ✅ Complete |
| T-014 | Dictionary tests | QA Team | P1 | ✅ Complete |
| T-015 | Tabular data tests | QA Team | P1 | ✅ Complete |
| T-016 | Mixed payload tests | QA Team | P1 | ✅ Complete |
| T-017 | Large file tests | QA Team | P1 | ✅ Complete |

---

## 4. Test Strategy

### Unit Tests

| Test Category | Coverage | Files | Requirements |
|---------------|----------|-------|--------------|
| Serialization | All payload types | `test/test_*_sender.*` | FR-001 through FR-012 |
| Deserialization | All payload types | `test/test_*_receiver.*` | FR-001 through FR-012 |
| Transport selection | Direct vs link | `test/test_*_mix_payloads.*` | FR-003, FR-004, FR-006 |
| File server upload | Plik integration | Platform-specific | FR-008, FR-009 |
| File server download | Exponential backoff | Platform-specific | FR-010, FR-011 |

### Integration Tests

| Scenario | Platforms | Payloads | Transport | Requirements |
|----------|-----------|----------|-----------|--------------|
| Single text (small) | All | text | direct | FR-001, FR-012 |
| Single dictionary (small) | All | dictionary | direct | FR-002, FR-012 |
| Single arrow table (small) | Desktop | arrowtable | direct | FR-002, FR-012 |
| Single JSON table (small) | All | jsontable | direct | FR-001, FR-002, FR-006 |
| Single image (small) | All | image | direct | FR-001, FR-006 |
| Single text (large) | All | text | link | FR-003, FR-008, FR-009 |
| Mixed payloads | All | text + dictionary + image | mixed | FR-006, FR-007 |

### Test Coverage Targets

| Phase | Coverage Target | Method |
|-------|----------------|--------|
| Phase 1 | 70% | Unit tests per platform |
| Phase 2 | 80% | Add integration tests |
| Phase 3 | 85% | Add edge case tests |
| Phase 4 | 90% | Add performance tests |

---

## 5. Build and Deployment Preparation

### Continuous Integration

| Check | Command | Purpose |
|-------|---------|---------|
| Linting | `npm run lint` | Code style enforcement |
| Type checking | `npx tsc --noEmit` | Type safety (JavaScript/TypeScript) |
| Unit tests | `npm test` | Functionality validation |
| Integration tests | `npm run test:integration` | Cross-platform validation |
| Coverage | `npm run coverage` | Test coverage tracking |

### Deployment Pipeline

```
GitHub Push
    ↓
CI/CD Pipeline
    ↓
├──→ Linting (all platforms)
├──→ Unit tests (all platforms)
├──→ Integration tests (cross-platform)
├──→ Coverage report
└──→ Build documentation
    ↓
Release (if all checks pass)
    ↓
├──→ GitHub Releases
├──→ Package registry (npm, PyPI)
└──→ Documentation site
```

---

## 6. Risk Mitigation

### Known Blockers

| Risk | Mitigation Step | Owner |
|------|----------------|-------|
| **Browser Arrow IPC** | Use JSON table as fallback | Developer B |
| **MicroPython memory** | 100KB threshold, direct transport only | Developer C |
| **File server availability** | Exponential backoff with graceful degradation | Developer B |

### Known Unknowns

| Unknown | Monitoring Strategy | Response Plan |
|---------|-------------------|---------------|
| Platform-specific bugs | Comprehensive test coverage | Hotfix with platform-specific handling |
| Performance bottlenecks | Load testing and profiling | Optimized serialization/deserialization |

---

## 7. Requirements Traceability

### Functional Requirements

| Requirement ID | Implementation Location | Status |
|---------------|------------------------|--------|
| FR-001 | All platform modules | ✅ Complete |
| FR-002 | All platform modules | ✅ Complete |
| FR-003 | All platform modules (size_threshold logic) | ✅ Complete |
| FR-004 | All platform modules | ✅ Complete |
| FR-005 | MicroPython module | ✅ Complete |
| FR-006 | All platform modules | ✅ Complete |
| FR-007 | All platform modules | ✅ Complete |
| FR-008 | All platform modules | ✅ Complete |
| FR-009 | All platform modules | ✅ Complete |
| FR-010 | All platform modules | ✅ Complete |
| FR-011 | All platform modules | ✅ Complete |
| FR-012 | All platform modules | ✅ Complete |
| FR-013 | All platform modules | ✅ Complete |
| FR-014 | All platform modules | ✅ Complete |

### Non-Functional Requirements

| NFR ID | Implementation Location | Status |
|--------|------------------------|--------|
| NFR-101 | Serialization functions | ✅ Complete |
| NFR-102 | Deserialization functions | ✅ Complete |
| NFR-103 | Transport adapter | ✅ Complete |
| NFR-104 | File upload handler | ✅ Complete |
| NFR-105 | File download handler | ✅ Complete |
| NFR-106 | MicroPython module | ✅ Complete |
| NFR-107 | Performance benchmarks | ✅ Complete |
| NFR-201 | Transport adapter | ✅ Complete |
| NFR-202 | File download retry logic | ✅ Complete |
| NFR-203 | Transport adapter | ✅ Complete |
| NFR-401 | Message envelope | ✅ Complete |
| NFR-402 | Metrics instrumentation | ✅ Complete |
| NFR-403 | Correlation ID propagation | ✅ Complete |

---

## 8. Validation Gates

### Pre-Release Checklist

| Gate | Check | Pass Criteria |
|------|-------|--------------|
| **G-001** | All unit tests pass | 100% pass rate per platform |
| **G-002** | Integration tests pass | Cross-platform round-trip successful |
| **G-003** | Coverage threshold | ≥80% line coverage |
| **G-004** | Linting clean | No warnings or errors |
| **G-005** | Specification compliance | All spec rules validated |
| **G-006** | Documentation complete | All required docs present |

### CI/CD Validation

| Check | Command | Failure Action |
|-------|---------|---------------|
| Syntax | `julia --check-base` | Block PR |
| Unit tests | `julia test/runtests.jl` | Block PR |
| Integration | `npm run test:integration` | Block PR |
| Coverage | `codecov` | Report only |

---

*This implementation plan is versioned and maintained in git alongside the codebase. All implementations must adhere to this plan.*