Ligbox Spec Hub 3a2c64834b Initial import: ligbox-ops-platform + specs + LAPTOP + obsidian merge (CT130)

Source: VM122 /opt + obsidian-infra + LAPTOP
Hub: CT130 spec-hub 10.10.10.130

2026-06-19 17:26:41 +00:00

8.7 KiB

Raw Permalink Blame History

Implementation Plan: Email Migration (013)

Branch: 013-email-server-migration
Date: 2026-06-10
Spec: spec.md

Summary

Orquestrador de migração de e-mail no VM122: API REST + worker assíncrono + UI Desk. Executa imapsync, readpst+imap-upload e zmmailbox TGZ conforme source_type. Gate DNS impede cutover até validação.

Módulo técnico — mapa de componentes

┌──────────────────────────────────────────────────────────────────────────┐
│                         FRONTEND (Desk VM122)                            │
│  view-email-migration │ migration-job-detail │ gate-badge no ticket      │
└────────────────────────────────┬─────────────────────────────────────────┘
                                 │ REST + JWT
┌────────────────────────────────▼─────────────────────────────────────────┐
│                         API (FastAPI)                                    │
│  app/migration/                                                          │
│    ├── router.py          # rotas /api/v1/migration/*                    │
│    ├── store.py           # CRUD SQLite jobs/mailboxes/runs               │
│    ├── gate.py            # migration_gate logic + DNS block              │
│    ├── credentials.py     # encrypt/decrypt origem (Fernet)               │
│    └── schemas.py         # Pydantic models                               │
└────────────────────────────────┬─────────────────────────────────────────┘
                                 │ Redis queue (existente) ou SQLite jobs
┌────────────────────────────────▼─────────────────────────────────────────┐
│              WORKER (VM122 piloto · VM123 futuro — ver infrastructure)   │
│  worker/migration_runner.py                                              │
│    ├── run_imapsync()                                                     │
│    ├── run_pst_pipeline()    # readpst → imap-upload                     │
│    ├── run_tgz_import()      # ssh/zmmailbox no VM112                      │
│    ├── run_verify()          # contagens IMAP                              │
│    └── parse_logs()          # imapsync LOG file → DB                      │
└────────────────────────────────┬─────────────────────────────────────────┘
                                 │
        ┌────────────────────────┼────────────────────────┐
        ▼                        ▼                        ▼
 ┌─────────────┐         ┌─────────────┐         ┌─────────────┐
 │ Servidor    │         │ Carbonio    │         │ Cloudflare  │
 │ origem IMAP │         │ VM112       │         │ / pfSense   │
 │ PST/mbox    │         │ (destino)   │         │ (DNS gate)  │
 └─────────────┘         └─────────────┘         └─────────────┘

Estrutura de ficheiros (a criar)

api/app/migration/
├── __init__.py
├── router.py
├── store.py
├── gate.py
├── credentials.py
├── schemas.py
├── verify.py
└── tools/
    ├── imapsync_runner.py
    ├── pst_runner.py
    ├── tgz_runner.py
    └── log_parser.py

worker/
├── migration_runner.py
└── migration_config.example.env

frontend/
├── index.html              # + nav Email Migration
├── assets/app.js           # renderEmailMigration(), job detail
└── assets/styles.css       # .migration-*

scripts/
├── verify-migration.sh
└── install-migration-tools.sh   # imapsync, pst-utils, imap-upload

data/migrations/            # PST uploads, logs (volume Docker)
├── uploads/
├── logs/
└── quarantine/

Fluxo DNS gate (integração)

sequenceDiagram
    participant T as Técnico Desk
    participant W as Worker
    participant API as API VM122
    participant CF as DNS/Cloudflare
    participant VM as Wizard VM112

    T->>API: POST /migration/jobs/{id}/sync (delta)
    W->>W: imapsync origem → Carbonio
    W->>API: PATCH run status + counts
    T->>API: GET /migration/jobs/{id}/verify
    API-->>T: 99.2% OK, gate=warning
    T->>API: POST /migration/jobs/{id}/sync (final)
    T->>API: POST approve-gate
    API-->>T: gate=ready_for_dns
    VM->>API: GET /migration/gate?domain=cliente.com
    API-->>VM: ready_for_dns
    T->>CF: Alterar MX
    VM->>API: dns.applied (webhook)

Bloqueio wizard (Fase B): VM112 chama gate antes de passo DNS definitivo. MVP: bloqueio só no Desk (alerta manual).

Variáveis de ambiente

# Migration module
MIGRATION_ENABLED=true
MIGRATION_TOOLS_PATH=/opt/migration-tools
MIGRATION_DATA_PATH=/data/migrations
MIGRATION_GATE_MIN_RATIO=0.99
MIGRATION_GATE_OVERRIDE_ROLES=super_admin
MIGRATION_CREDENTIALS_KEY=<fernet key>
MIGRATION_MAX_PST_GB=50
MIGRATION_IMAPSYNC_BIN=/usr/bin/imapsync
MIGRATION_READPST_BIN=/usr/bin/readpst
MIGRATION_IMAP_UPLOAD=/opt/migration-tools/imap-upload/imap_upload.py

# Destino default (Carbonio)
MIGRATION_DEST_IMAP_HOST=mail.cliente.com
MIGRATION_DEST_IMAP_PORT=993
MIGRATION_DEST_IMAP_SSL=true

# VM112 admin (TGZ path)
MIGRATION_CARBONIO_SSH=root@10.10.10.112
MIGRATION_ZMMAILBOX_USER=zextras

Permissões RBAC

def can_manage_migration(role: str) -> bool:
    return role in ("super_admin", "ops_lead", "technician")

def can_approve_migration_gate(role: str) -> bool:
    return role in ("super_admin", "ops_lead")

def can_override_migration_gate(role: str) -> bool:
    return role == "super_admin"

Comandos executados pelo worker (referência)

IMAP (imapsync)

imapsync \
  --host1 "${SRC_HOST}" --user1 "${SRC_USER}" --password1 "${SRC_PASS}" \
  --host2 "${DST_HOST}" --user2 "${DST_USER}" --password2 "${DST_PASS}" \
  --ssl1 --ssl2 --automap --syncinternaldates \
  --useheader "Message-Id" \
  --logdir "/data/migrations/logs/${RUN_ID}" \
  --errorsmax 100

OAuth (O365):

imapsync --host1 outlook.office365.com --user1 user@domain.com \
  --oauthaccesstoken1 /path/token.txt \
  --host2 mail.dest.com --user2 user@domain.com --password2 '...'

PST

readpst -o "/data/migrations/work/${MBX_ID}/mbox" -r "/data/migrations/uploads/file.pst"
python3 /opt/migration-tools/imap-upload/imap_upload.py \
  --ssl --host "${DST_HOST}" --port 993 \
  --user "${DST_USER}" --password "${DST_PASS}" \
  --error "/data/migrations/quarantine/${RUN_ID}_errors.mbox" \
  -r "/data/migrations/work/${MBX_ID}/mbox"

TGZ (Carbonio)

# export na origem (SSH)
zmmailbox -z -m user@domain.com getRestURL '/?fmt=tgz' > user.tgz
# import no destino
zmmailbox -z -m user@domain.com postRestURL "/?fmt=tgz&resolve=skip" user.tgz

Verificação

# Script Python verify.py — IMAP STATUS + SEARCH ALL por pasta
python3 -m app.migration.verify --job-id 42 --mailbox-id 7

Constitution Check

Princípio	Status
Spec-Driven	✅
VM112 fora compose	✅ worker SSH para zmmailbox
Mail vs Ops separation	✅ orquestração no Ops; mail no Carbonio
YAGNI MVP	✅ 3 pipelines; sem calendários

Fases de implementação

Ver tasks.md:

Fase A (P0): schema + API CRUD + imapsync runner + gate básico + UI lista
Fase B (P0): PST pipeline + verify + approve gate
Fase C (P1): TGZ + webhook gate VM112 + relatório PDF
Fase D (P2): pst2mbox wrapper, OAuth UI, agendamento cron

Testes

./scripts/verify-migration.sh
# 1. Criar job teste
# 2. Preflight imap test account
# 3. Sync mini mailbox
# 4. Verify counts
# 5. Gate blocked → approve → ready

8.7 KiB Raw Permalink Blame History