243 lines
8.7 KiB
Markdown
243 lines
8.7 KiB
Markdown
# Implementation Plan: Email Migration (013)
|
|
|
|
**Branch:** `013-email-server-migration`
|
|
**Date:** 2026-06-10
|
|
**Spec:** [spec.md](./spec.md)
|
|
|
|
---
|
|
|
|
## Summary
|
|
|
|
Orquestrador de migração de e-mail no VM122: API REST + worker assíncrono + UI Desk. Executa **imapsync**, **readpst+imap-upload** e **zmmailbox TGZ** conforme `source_type`. **Gate DNS** impede cutover até validação.
|
|
|
|
---
|
|
|
|
## Módulo técnico — mapa de componentes
|
|
|
|
```
|
|
┌──────────────────────────────────────────────────────────────────────────┐
|
|
│ FRONTEND (Desk VM122) │
|
|
│ view-email-migration │ migration-job-detail │ gate-badge no ticket │
|
|
└────────────────────────────────┬─────────────────────────────────────────┘
|
|
│ REST + JWT
|
|
┌────────────────────────────────▼─────────────────────────────────────────┐
|
|
│ API (FastAPI) │
|
|
│ app/migration/ │
|
|
│ ├── router.py # rotas /api/v1/migration/* │
|
|
│ ├── store.py # CRUD SQLite jobs/mailboxes/runs │
|
|
│ ├── gate.py # migration_gate logic + DNS block │
|
|
│ ├── credentials.py # encrypt/decrypt origem (Fernet) │
|
|
│ └── schemas.py # Pydantic models │
|
|
└────────────────────────────────┬─────────────────────────────────────────┘
|
|
│ Redis queue (existente) ou SQLite jobs
|
|
┌────────────────────────────────▼─────────────────────────────────────────┐
|
|
│ WORKER (VM122 piloto · VM123 futuro — ver infrastructure) │
|
|
│ worker/migration_runner.py │
|
|
│ ├── run_imapsync() │
|
|
│ ├── run_pst_pipeline() # readpst → imap-upload │
|
|
│ ├── run_tgz_import() # ssh/zmmailbox no VM112 │
|
|
│ ├── run_verify() # contagens IMAP │
|
|
│ └── parse_logs() # imapsync LOG file → DB │
|
|
└────────────────────────────────┬─────────────────────────────────────────┘
|
|
│
|
|
┌────────────────────────┼────────────────────────┐
|
|
▼ ▼ ▼
|
|
┌─────────────┐ ┌─────────────┐ ┌─────────────┐
|
|
│ Servidor │ │ Carbonio │ │ Cloudflare │
|
|
│ origem IMAP │ │ VM112 │ │ / pfSense │
|
|
│ PST/mbox │ │ (destino) │ │ (DNS gate) │
|
|
└─────────────┘ └─────────────┘ └─────────────┘
|
|
```
|
|
|
|
---
|
|
|
|
## Estrutura de ficheiros (a criar)
|
|
|
|
```text
|
|
api/app/migration/
|
|
├── __init__.py
|
|
├── router.py
|
|
├── store.py
|
|
├── gate.py
|
|
├── credentials.py
|
|
├── schemas.py
|
|
├── verify.py
|
|
└── tools/
|
|
├── imapsync_runner.py
|
|
├── pst_runner.py
|
|
├── tgz_runner.py
|
|
└── log_parser.py
|
|
|
|
worker/
|
|
├── migration_runner.py
|
|
└── migration_config.example.env
|
|
|
|
frontend/
|
|
├── index.html # + nav Email Migration
|
|
├── assets/app.js # renderEmailMigration(), job detail
|
|
└── assets/styles.css # .migration-*
|
|
|
|
scripts/
|
|
├── verify-migration.sh
|
|
└── install-migration-tools.sh # imapsync, pst-utils, imap-upload
|
|
|
|
data/migrations/ # PST uploads, logs (volume Docker)
|
|
├── uploads/
|
|
├── logs/
|
|
└── quarantine/
|
|
```
|
|
|
|
---
|
|
|
|
## Fluxo DNS gate (integração)
|
|
|
|
```mermaid
|
|
sequenceDiagram
|
|
participant T as Técnico Desk
|
|
participant W as Worker
|
|
participant API as API VM122
|
|
participant CF as DNS/Cloudflare
|
|
participant VM as Wizard VM112
|
|
|
|
T->>API: POST /migration/jobs/{id}/sync (delta)
|
|
W->>W: imapsync origem → Carbonio
|
|
W->>API: PATCH run status + counts
|
|
T->>API: GET /migration/jobs/{id}/verify
|
|
API-->>T: 99.2% OK, gate=warning
|
|
T->>API: POST /migration/jobs/{id}/sync (final)
|
|
T->>API: POST approve-gate
|
|
API-->>T: gate=ready_for_dns
|
|
VM->>API: GET /migration/gate?domain=cliente.com
|
|
API-->>VM: ready_for_dns
|
|
T->>CF: Alterar MX
|
|
VM->>API: dns.applied (webhook)
|
|
```
|
|
|
|
**Bloqueio wizard (Fase B):** VM112 chama gate antes de passo DNS definitivo. MVP: bloqueio só no Desk (alerta manual).
|
|
|
|
---
|
|
|
|
## Variáveis de ambiente
|
|
|
|
```env
|
|
# Migration module
|
|
MIGRATION_ENABLED=true
|
|
MIGRATION_TOOLS_PATH=/opt/migration-tools
|
|
MIGRATION_DATA_PATH=/data/migrations
|
|
MIGRATION_GATE_MIN_RATIO=0.99
|
|
MIGRATION_GATE_OVERRIDE_ROLES=super_admin
|
|
MIGRATION_CREDENTIALS_KEY=<fernet key>
|
|
MIGRATION_MAX_PST_GB=50
|
|
MIGRATION_IMAPSYNC_BIN=/usr/bin/imapsync
|
|
MIGRATION_READPST_BIN=/usr/bin/readpst
|
|
MIGRATION_IMAP_UPLOAD=/opt/migration-tools/imap-upload/imap_upload.py
|
|
|
|
# Destino default (Carbonio)
|
|
MIGRATION_DEST_IMAP_HOST=mail.cliente.com
|
|
MIGRATION_DEST_IMAP_PORT=993
|
|
MIGRATION_DEST_IMAP_SSL=true
|
|
|
|
# VM112 admin (TGZ path)
|
|
MIGRATION_CARBONIO_SSH=root@10.10.10.112
|
|
MIGRATION_ZMMAILBOX_USER=zextras
|
|
```
|
|
|
|
---
|
|
|
|
## Permissões RBAC
|
|
|
|
```python
|
|
def can_manage_migration(role: str) -> bool:
|
|
return role in ("super_admin", "ops_lead", "technician")
|
|
|
|
def can_approve_migration_gate(role: str) -> bool:
|
|
return role in ("super_admin", "ops_lead")
|
|
|
|
def can_override_migration_gate(role: str) -> bool:
|
|
return role == "super_admin"
|
|
```
|
|
|
|
---
|
|
|
|
## Comandos executados pelo worker (referência)
|
|
|
|
### IMAP (imapsync)
|
|
|
|
```bash
|
|
imapsync \
|
|
--host1 "${SRC_HOST}" --user1 "${SRC_USER}" --password1 "${SRC_PASS}" \
|
|
--host2 "${DST_HOST}" --user2 "${DST_USER}" --password2 "${DST_PASS}" \
|
|
--ssl1 --ssl2 --automap --syncinternaldates \
|
|
--useheader "Message-Id" \
|
|
--logdir "/data/migrations/logs/${RUN_ID}" \
|
|
--errorsmax 100
|
|
```
|
|
|
|
OAuth (O365):
|
|
```bash
|
|
imapsync --host1 outlook.office365.com --user1 user@domain.com \
|
|
--oauthaccesstoken1 /path/token.txt \
|
|
--host2 mail.dest.com --user2 user@domain.com --password2 '...'
|
|
```
|
|
|
|
### PST
|
|
|
|
```bash
|
|
readpst -o "/data/migrations/work/${MBX_ID}/mbox" -r "/data/migrations/uploads/file.pst"
|
|
python3 /opt/migration-tools/imap-upload/imap_upload.py \
|
|
--ssl --host "${DST_HOST}" --port 993 \
|
|
--user "${DST_USER}" --password "${DST_PASS}" \
|
|
--error "/data/migrations/quarantine/${RUN_ID}_errors.mbox" \
|
|
-r "/data/migrations/work/${MBX_ID}/mbox"
|
|
```
|
|
|
|
### TGZ (Carbonio)
|
|
|
|
```bash
|
|
# export na origem (SSH)
|
|
zmmailbox -z -m user@domain.com getRestURL '/?fmt=tgz' > user.tgz
|
|
# import no destino
|
|
zmmailbox -z -m user@domain.com postRestURL "/?fmt=tgz&resolve=skip" user.tgz
|
|
```
|
|
|
|
### Verificação
|
|
|
|
```bash
|
|
# Script Python verify.py — IMAP STATUS + SEARCH ALL por pasta
|
|
python3 -m app.migration.verify --job-id 42 --mailbox-id 7
|
|
```
|
|
|
|
---
|
|
|
|
## Constitution Check
|
|
|
|
| Princípio | Status |
|
|
|-----------|--------|
|
|
| Spec-Driven | ✅ |
|
|
| VM112 fora compose | ✅ worker SSH para zmmailbox |
|
|
| Mail vs Ops separation | ✅ orquestração no Ops; mail no Carbonio |
|
|
| YAGNI MVP | ✅ 3 pipelines; sem calendários |
|
|
|
|
---
|
|
|
|
## Fases de implementação
|
|
|
|
Ver [tasks.md](./tasks.md):
|
|
|
|
- **Fase A (P0):** schema + API CRUD + imapsync runner + gate básico + UI lista
|
|
- **Fase B (P0):** PST pipeline + verify + approve gate
|
|
- **Fase C (P1):** TGZ + webhook gate VM112 + relatório PDF
|
|
- **Fase D (P2):** pst2mbox wrapper, OAuth UI, agendamento cron
|
|
|
|
---
|
|
|
|
## Testes
|
|
|
|
```bash
|
|
./scripts/verify-migration.sh
|
|
# 1. Criar job teste
|
|
# 2. Preflight imap test account
|
|
# 3. Sync mini mailbox
|
|
# 4. Verify counts
|
|
# 5. Gate blocked → approve → ready
|
|
```
|