diff --git a/LAPTOP/20260616_SERVICOS_ORQUESTRACAO_PURGE_ASPECTOS.md b/LAPTOP/20260616_SERVICOS_ORQUESTRACAO_PURGE_ASPECTOS.md index fc948e9..e498544 100644 --- a/LAPTOP/20260616_SERVICOS_ORQUESTRACAO_PURGE_ASPECTOS.md +++ b/LAPTOP/20260616_SERVICOS_ORQUESTRACAO_PURGE_ASPECTOS.md @@ -153,3 +153,13 @@ docker-compose -f docker-compose.mvp.yml up -d api frontend - Cada oferta = wizard próprio (Proxmox, servidor físico, etc.). - Modelo comercial Pizza as a Service documentado na Spec 018. - Purge Spec 017 mantido; UI evolui (drawer + jobs). + +--- + +## 12. Continuação — 2026-06-25 (performance + UX Escopo OPS) + +Ver documento dedicado: + +**`docs/anais-referencia/20260625_SERVICOS_IAAS_PERF_UX_ESCOPO.md`** + +Inclui: cache VM112/VM122, loading modal purge, cards Escopo OPS clicáveis, blocklist UI, versão frontend `20260625align1`. diff --git a/LAPTOP/20260625_SERVICOS_IAAS_PERF_UX_ESCOPO.md b/LAPTOP/20260625_SERVICOS_IAAS_PERF_UX_ESCOPO.md new file mode 100644 index 0000000..2e546cd --- /dev/null +++ b/LAPTOP/20260625_SERVICOS_IAAS_PERF_UX_ESCOPO.md @@ -0,0 +1,124 @@ +# Anais de Referência — Serviços IaaS: Performance, UX Escopo OPS, Blocklist + +**Data:** 2026-06-25 +**Utilizador:** Roger +**Sistema:** Desk VM122 (`10.10.10.122`) · Wizard VM112 (`10.10.10.112`) +**Specs:** 017 · 018 · 026 · 028 · 032 · 037 + +--- + +## 1. Resumo executivo + +Sessão de melhorias na página **Serviços IaaS** (Desk): + +1. **Performance** — cache VM112 lista + detalhe domínio; cache proxy VM122; SWR frontend. +2. **Modal purge** — skeleton imediato + loading animado (barra, etapas, spinner). +3. **Escopo OPS** — cards clicáveis com camada (tenant/app/infra), Spec e navegação. +4. **Blocklist UI** — `ligbox.com.br` e `itecnologys.com` visíveis na lista e painel Escopo. +5. **Alinhamento lista clientes** — badges agrupados (fix desalinhamento `itecnologys.com`). + +--- + +## 2. Performance — resultados + +| Pedido | Antes | Depois (cache quente) | +|--------|-------|------------------------| +| VM112 `GET /api/admin/domains` | ~5–11s | **~0,03s** | +| VM112 `GET /api/admin/domains/{domain}` | ~15–20s | **~0,04s** | +| Desk `GET /api/v1/vm112/domains` | ~16s | **~0,08s** | +| Desk detalhe domínio (2.º hit) | ~15s+ | **~0,1s** | +| Modal purge 1.º hit (cold) | ecrã branco | skeleton + ~10–15s | + +**Patch VM112:** `deploy/vm112-wizard/perf-domains-list-20260625/` +**Desk API:** `VM112_DOMAINS_CACHE_TTL=60`, `VM112_DOMAIN_DETAIL_TTL=45` +**Poll Serviços:** 90s (`app.js`) + +--- + +## 3. UX Escopo OPS — mapa cards + +| Card | Camada | Spec | Destino | +|------|--------|------|---------| +| Carbonio | Aplicação MOSP | 017 | Modal Serviços | +| ligbox-sites | Tenant utilizador | 017 | Modal Serviços | +| Portal users | Tenant utilizador | 028 | Console `/admin/dominio` | +| Cloudflare | Infraestrutura | 037 | Console DNS Viewer | +| Traefik / SNI | Infraestrutura | 026 | Desk Infra CODE | +| Desk OPS | Aplicação MOSP | 017 | Eventos › Histórico purges | + +**Helper navegação:** `window.DeskNavigate.go(view, { eventsTab })` — `app.js` + +--- + +## 4. Blocklist purge + +| Domínio | Purge API | UI | +|---------|-----------|-----| +| `ligbox.com.br` | HTTP 400 | 🔒 badge + banner Escopo + modal desactivado | +| `itecnologys.com` | HTTP 400 | idem | + +Constante: `PURGE_BLOCKLIST` — `vm112_domains.py` + espelho `servicos.js`. + +--- + +## 5. Ficheiros principais + +| Ficheiro | Função | +|----------|--------| +| `frontend/assets/servicos.js` | Escopo OPS, loading modal, blocklist, cache SWR | +| `frontend/assets/styles.css` | `.vm112-load-*`, `.servicos-scope-*`, client row grid | +| `frontend/assets/app.js` | `DeskNavigate`, poll 90s | +| `frontend/assets/auth.js` | sessão resiliente rede | +| `frontend/index.html` | cache bust `20260625align1` | +| `api/app/vm112_domains.py` | cache proxy, `purge_blocked` | + +--- + +## 6. Deploy VM122 (referência) + +```bash +sshpass -p '805353' scp servicos.js styles.css app.js index.html root@10.10.10.122:/tmp/ +ssh root@10.10.10.122 ' + C=ligbox-ops-platform_frontend_1 + docker cp /tmp/servicos.js $C:/usr/share/nginx/html/assets/servicos.js + docker cp /tmp/styles.css $C:/usr/share/nginx/html/assets/styles.css + docker cp /tmp/app.js $C:/usr/share/nginx/html/assets/app.js + docker cp /tmp/index.html $C:/usr/share/nginx/html/index.html +' +``` + +**Hard refresh:** Ctrl+Shift+R em `https://desk.ligbox.com.br` + +--- + +## 7. Versão frontend (cache bust) + +| Asset | Query string | +|-------|----------------| +| `styles.css` | `?v=20260625align1` | +| `servicos.js` | `?v=20260625align1` | +| `app.js` | `?v=20260625align1` | + +--- + +## 8. Specs actualizadas + +| Spec | Secção | +|------|--------| +| 017 | Performance cache, UI loading/blocklist/Escopo | +| 018 | Performance, Escopo OPS cards, critérios aceite | +| 032 | UI blocklist | +| 026 | Destino card Traefik | +| 028 | Destino card Portal | +| 037 | Destino card Cloudflare | + +--- + +## 9. Canais de arquivo + +| Canal | Path | +|-------|------| +| Anais | `docs/anais-referencia/20260625_SERVICOS_IAAS_PERF_UX_ESCOPO.md` | +| Spec 017 | `specs/017-vm112-domain-orchestration/spec.md` | +| Spec 018 | `specs/018-service-orchestration/spec.md` | +| Patch VM112 | `deploy/vm112-wizard/perf-domains-list-20260625/` | diff --git a/LAPTOP/anais-referencia/20260616_SERVICOS_ORQUESTRACAO_PURGE_ASPECTOS.md b/LAPTOP/anais-referencia/20260616_SERVICOS_ORQUESTRACAO_PURGE_ASPECTOS.md index fc948e9..e498544 100644 --- a/LAPTOP/anais-referencia/20260616_SERVICOS_ORQUESTRACAO_PURGE_ASPECTOS.md +++ b/LAPTOP/anais-referencia/20260616_SERVICOS_ORQUESTRACAO_PURGE_ASPECTOS.md @@ -153,3 +153,13 @@ docker-compose -f docker-compose.mvp.yml up -d api frontend - Cada oferta = wizard próprio (Proxmox, servidor físico, etc.). - Modelo comercial Pizza as a Service documentado na Spec 018. - Purge Spec 017 mantido; UI evolui (drawer + jobs). + +--- + +## 12. Continuação — 2026-06-25 (performance + UX Escopo OPS) + +Ver documento dedicado: + +**`docs/anais-referencia/20260625_SERVICOS_IAAS_PERF_UX_ESCOPO.md`** + +Inclui: cache VM112/VM122, loading modal purge, cards Escopo OPS clicáveis, blocklist UI, versão frontend `20260625align1`. diff --git a/LAPTOP/anais-referencia/20260625_SERVICOS_IAAS_PERF_UX_ESCOPO.md b/LAPTOP/anais-referencia/20260625_SERVICOS_IAAS_PERF_UX_ESCOPO.md new file mode 100644 index 0000000..2e546cd --- /dev/null +++ b/LAPTOP/anais-referencia/20260625_SERVICOS_IAAS_PERF_UX_ESCOPO.md @@ -0,0 +1,124 @@ +# Anais de Referência — Serviços IaaS: Performance, UX Escopo OPS, Blocklist + +**Data:** 2026-06-25 +**Utilizador:** Roger +**Sistema:** Desk VM122 (`10.10.10.122`) · Wizard VM112 (`10.10.10.112`) +**Specs:** 017 · 018 · 026 · 028 · 032 · 037 + +--- + +## 1. Resumo executivo + +Sessão de melhorias na página **Serviços IaaS** (Desk): + +1. **Performance** — cache VM112 lista + detalhe domínio; cache proxy VM122; SWR frontend. +2. **Modal purge** — skeleton imediato + loading animado (barra, etapas, spinner). +3. **Escopo OPS** — cards clicáveis com camada (tenant/app/infra), Spec e navegação. +4. **Blocklist UI** — `ligbox.com.br` e `itecnologys.com` visíveis na lista e painel Escopo. +5. **Alinhamento lista clientes** — badges agrupados (fix desalinhamento `itecnologys.com`). + +--- + +## 2. Performance — resultados + +| Pedido | Antes | Depois (cache quente) | +|--------|-------|------------------------| +| VM112 `GET /api/admin/domains` | ~5–11s | **~0,03s** | +| VM112 `GET /api/admin/domains/{domain}` | ~15–20s | **~0,04s** | +| Desk `GET /api/v1/vm112/domains` | ~16s | **~0,08s** | +| Desk detalhe domínio (2.º hit) | ~15s+ | **~0,1s** | +| Modal purge 1.º hit (cold) | ecrã branco | skeleton + ~10–15s | + +**Patch VM112:** `deploy/vm112-wizard/perf-domains-list-20260625/` +**Desk API:** `VM112_DOMAINS_CACHE_TTL=60`, `VM112_DOMAIN_DETAIL_TTL=45` +**Poll Serviços:** 90s (`app.js`) + +--- + +## 3. UX Escopo OPS — mapa cards + +| Card | Camada | Spec | Destino | +|------|--------|------|---------| +| Carbonio | Aplicação MOSP | 017 | Modal Serviços | +| ligbox-sites | Tenant utilizador | 017 | Modal Serviços | +| Portal users | Tenant utilizador | 028 | Console `/admin/dominio` | +| Cloudflare | Infraestrutura | 037 | Console DNS Viewer | +| Traefik / SNI | Infraestrutura | 026 | Desk Infra CODE | +| Desk OPS | Aplicação MOSP | 017 | Eventos › Histórico purges | + +**Helper navegação:** `window.DeskNavigate.go(view, { eventsTab })` — `app.js` + +--- + +## 4. Blocklist purge + +| Domínio | Purge API | UI | +|---------|-----------|-----| +| `ligbox.com.br` | HTTP 400 | 🔒 badge + banner Escopo + modal desactivado | +| `itecnologys.com` | HTTP 400 | idem | + +Constante: `PURGE_BLOCKLIST` — `vm112_domains.py` + espelho `servicos.js`. + +--- + +## 5. Ficheiros principais + +| Ficheiro | Função | +|----------|--------| +| `frontend/assets/servicos.js` | Escopo OPS, loading modal, blocklist, cache SWR | +| `frontend/assets/styles.css` | `.vm112-load-*`, `.servicos-scope-*`, client row grid | +| `frontend/assets/app.js` | `DeskNavigate`, poll 90s | +| `frontend/assets/auth.js` | sessão resiliente rede | +| `frontend/index.html` | cache bust `20260625align1` | +| `api/app/vm112_domains.py` | cache proxy, `purge_blocked` | + +--- + +## 6. Deploy VM122 (referência) + +```bash +sshpass -p '805353' scp servicos.js styles.css app.js index.html root@10.10.10.122:/tmp/ +ssh root@10.10.10.122 ' + C=ligbox-ops-platform_frontend_1 + docker cp /tmp/servicos.js $C:/usr/share/nginx/html/assets/servicos.js + docker cp /tmp/styles.css $C:/usr/share/nginx/html/assets/styles.css + docker cp /tmp/app.js $C:/usr/share/nginx/html/assets/app.js + docker cp /tmp/index.html $C:/usr/share/nginx/html/index.html +' +``` + +**Hard refresh:** Ctrl+Shift+R em `https://desk.ligbox.com.br` + +--- + +## 7. Versão frontend (cache bust) + +| Asset | Query string | +|-------|----------------| +| `styles.css` | `?v=20260625align1` | +| `servicos.js` | `?v=20260625align1` | +| `app.js` | `?v=20260625align1` | + +--- + +## 8. Specs actualizadas + +| Spec | Secção | +|------|--------| +| 017 | Performance cache, UI loading/blocklist/Escopo | +| 018 | Performance, Escopo OPS cards, critérios aceite | +| 032 | UI blocklist | +| 026 | Destino card Traefik | +| 028 | Destino card Portal | +| 037 | Destino card Cloudflare | + +--- + +## 9. Canais de arquivo + +| Canal | Path | +|-------|------| +| Anais | `docs/anais-referencia/20260625_SERVICOS_IAAS_PERF_UX_ESCOPO.md` | +| Spec 017 | `specs/017-vm112-domain-orchestration/spec.md` | +| Spec 018 | `specs/018-service-orchestration/spec.md` | +| Patch VM112 | `deploy/vm112-wizard/perf-domains-list-20260625/` | diff --git a/deploy/vm112-wizard/perf-domains-list-20260625/README.md b/deploy/vm112-wizard/perf-domains-list-20260625/README.md new file mode 100644 index 0000000..2a8fc5b --- /dev/null +++ b/deploy/vm112-wizard/perf-domains-list-20260625/README.md @@ -0,0 +1,63 @@ +# VM112 — cache `GET /api/admin/domains` (2026-06-25) + +**Problema:** Serviços IaaS (Desk) demorava 10–16s — `zmprov gad` ~5s por pedido, sem cache. + +**Patch:** cache em memória no wizard VM112 + invalidação em cd/dd/purge. + +## Ficheiros + +| Ficheiro | Destino VM112 | +|----------|----------------| +| `carbonio_cache.py` | `/opt/ligbox-wizard/backend/app/services/` | +| `carbonio.py` | idem | +| `domain_orchestration.py` | idem | +| `infrastructure.py` | idem | +| `admin_domains.py` | `/opt/ligbox-wizard/backend/app/routers/` | + +## Deploy + +```bash +scp perf-domains-list-20260625/*.py root@10.10.10.112:/tmp/ +ssh root@10.10.10.112 ' + W=/opt/ligbox-wizard/backend/app + cp /tmp/carbonio_cache.py $W/services/ + cp /tmp/carbonio.py $W/services/ + cp /tmp/domain_orchestration.py $W/services/ + cp /tmp/infrastructure.py $W/services/ + cp /tmp/admin_domains.py $W/routers/ + systemctl restart ligbox-wizard +' +``` + +## Env (opcional) + +| Variável | Default | Descrição | +|----------|---------|-----------| +| — | `TTL_ALL_DOMAINS=90` | Cache `zmprov gad` (carbonio_cache.py) | +| — | `TTL_ORCH_LIST=60` | Cache lista orquestrada montada | +| Desk `VM112_DOMAINS_CACHE_TTL` | 60 | Cache proxy VM122 | + +## Validação + +```bash +# 2.º hit lista — deve ser <1s +curl -s -H "X-Api-Key: $KEY" http://10.10.10.112:8090/api/admin/domains | jq '.cached, .cache_age_sec' + +# Desk detalhe — purge_blocked para blocklist +curl -s -H "Authorization: Bearer $TOKEN" \ + https://api.ops.ligbox.com.br/api/v1/vm112/domains/ligbox.com.br | jq '.purge_blocked' +``` + +**Documentação:** Spec 017/018 § Performance · anais `docs/anais-referencia/20260625_SERVICOS_IAAS_PERF_UX_ESCOPO.md` · frontend `?v=20260625align1` + +```bash +# Lista — 1.º ~5–6s, 2.º <0.1s +curl -s -H "X-Api-Key: $KEY" http://10.10.10.112:8090/api/admin/domains | jq '.cached, .cache_age_sec, (.domains|length)' + +# Detalhe purge modal — 1.º ~16s, 2.º <0.1s +curl -s -H "X-Api-Key: $KEY" http://10.10.10.112:8090/api/admin/domains/diarissima.com | jq '.domain, (.accounts|length), (.infra_status.steps|length)' +``` + +## Rollback + +Restaurar backup ou git anterior em `/opt/ligbox-wizard/backend/app/services/` + `systemctl restart ligbox-wizard`. diff --git a/deploy/vm112-wizard/perf-domains-list-20260625/admin_domains.py b/deploy/vm112-wizard/perf-domains-list-20260625/admin_domains.py new file mode 100644 index 0000000..28d6c75 --- /dev/null +++ b/deploy/vm112-wizard/perf-domains-list-20260625/admin_domains.py @@ -0,0 +1,53 @@ +"""API admin — domínios orquestrados (Desk Spec 017).""" + +from fastapi import APIRouter, BackgroundTasks, Depends, Header, HTTPException, Query + +from app.config import settings +from app.services import domain_orchestration, purge_jobs + +router = APIRouter(prefix="/admin/domains", tags=["admin-domains"]) + + +def require_api_key(x_api_key: str | None = Header(default=None, alias="X-Api-Key")): + if x_api_key != settings.admin_api_key: + raise HTTPException(401, "API key inválida") + return True + + +@router.get("/purge-jobs/{job_id}", dependencies=[Depends(require_api_key)]) +def get_purge_job(job_id: str): + job = purge_jobs.get_job(job_id) + if not job: + raise HTTPException(404, "Job de purge não encontrado") + return job + + +@router.get("", dependencies=[Depends(require_api_key)]) +def list_domains(q: str = Query(default="")): + return domain_orchestration.list_orchestrated_domains_meta(q) + + +@router.get("/{domain}", dependencies=[Depends(require_api_key)]) +def domain_detail(domain: str): + try: + return domain_orchestration.get_domain_detail(domain.lower().strip()) + except ValueError as e: + raise HTTPException(404, str(e)) from e + + +@router.post("/{domain}/purge", dependencies=[Depends(require_api_key)]) +def purge_domain(domain: str, background_tasks: BackgroundTasks, sync: bool = Query(default=False)): + """Inicia purge assíncrono (Fase 3). Desk faz poll em purge-jobs/{job_id}.""" + domain = domain.lower().strip() + try: + domain_orchestration.assert_purge_allowed(domain) + if sync: + result = domain_orchestration.purge_domain(domain) + return {"ok": True, "steps": result.get("steps", []), "result": result} + meta = purge_jobs.create_job(domain) + background_tasks.add_task(domain_orchestration.run_purge_job, meta["job_id"]) + return {"ok": True, **meta} + except ValueError as e: + raise HTTPException(400, str(e)) from e + except Exception as e: + raise HTTPException(500, str(e)) from e diff --git a/deploy/vm112-wizard/perf-domains-list-20260625/carbonio.py b/deploy/vm112-wizard/perf-domains-list-20260625/carbonio.py new file mode 100644 index 0000000..2db4781 --- /dev/null +++ b/deploy/vm112-wizard/perf-domains-list-20260625/carbonio.py @@ -0,0 +1,485 @@ +"""Provisionamento local via zmprov (Carbonio).""" + +from __future__ import annotations + +import re +import subprocess +from pathlib import Path + +from app.config import settings + +NGINX_WEB_HTTPS = Path("/opt/zextras/conf/nginx/includes/nginx.conf.web.https") +ZMPROXYCONFGEN = "/opt/zextras/libexec/zmproxyconfgen" +from app.services import activity_log, carbonio_cache + +EMAIL_RE = re.compile(r"^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}$") +_NO_SUCH_DOMAIN = "NO_SUCH_DOMAIN" +_NO_SUCH_ACCOUNT = "NO_SUCH_ACCOUNT" + + +class CarbonioError(Exception): + pass + + +def _zmprov_run(*args: str, log_cmd: bool = True) -> tuple[int, str, str]: + cmd = ["sudo", "-u", settings.zextras_user, settings.zmprov, *args] + if log_cmd: + activity_log.cmd(" ".join(cmd), source="vm112") + proc = subprocess.run(cmd, capture_output=True, text=True, timeout=120) + return proc.returncode, (proc.stdout or "").strip(), (proc.stderr or "").strip() + + +def _run_zmprov(*args: str) -> str: + """Comando zmprov que deve ter sucesso (criar domínio/conta, etc.).""" + code, out, err = _zmprov_run(*args, log_cmd=True) + if code != 0: + msg = err or out or "zmprov failed" + activity_log.error(f"zmprov falhou: {msg}", source="vm112") + raise CarbonioError(msg) + if out: + activity_log.ok(out[:500], source="vm112") + return out + + +def _is_missing_domain(stderr: str, stdout: str) -> bool: + blob = f"{stderr}\n{stdout}" + return _NO_SUCH_DOMAIN in blob + + +def _is_missing_account(stderr: str, stdout: str) -> bool: + blob = f"{stderr}\n{stdout}" + return _NO_SUCH_ACCOUNT in blob or _NO_SUCH_DOMAIN in blob + + +def _is_account_exists(stderr: str, stdout: str) -> bool: + blob = f"{stderr}\n{stdout}" + return "ACCOUNT_EXISTS" in blob + + +def _carbonio_unreachable(err: str, out: str) -> bool: + blob = f"{err}\n{out}".lower() + return "connection refused" in blob or "io_error" in blob + + +def domain_exists(domain: str, *, use_cache: bool = True) -> bool: + """Verificação silenciosa — domínio inexistente NÃO é erro de onboarding.""" + domain = domain.lower().strip() + cache_key = f"domain_exists:{domain}" + if use_cache: + cached = carbonio_cache.get(cache_key) + if cached is not None: + return bool(cached) + code, out, err = _zmprov_run("gd", domain, log_cmd=False) + if code == 0: + exists = f"# name {domain}" in out or f"name {domain}" in out + elif _is_missing_domain(err, out): + exists = False + elif _carbonio_unreachable(err, out): + exists = False + else: + activity_log.warn(f"zmprov gd {domain}: {err or out}", source="vm112") + exists = False + if use_cache: + carbonio_cache.set(cache_key, exists, carbonio_cache.TTL_DOMAIN_EXISTS) + return exists + + +def list_all_domains(*, use_cache: bool = True) -> list[str]: + """zmprov gad (~5s) — cache TTL curto para Serviços IaaS / Desk.""" + cache_key = carbonio_cache.CACHE_KEY_ALL_DOMAINS + if use_cache: + cached = carbonio_cache.get(cache_key) + if cached is not None: + return list(cached) + code, out, _err = _zmprov_run("gad", log_cmd=False) + if code != 0: + return [] + domains = [ln.strip().lower() for ln in out.splitlines() if ln.strip()] + if use_cache: + carbonio_cache.set(cache_key, domains, carbonio_cache.TTL_ALL_DOMAINS) + return domains + + +def invalidate_domain_list_cache() -> None: + carbonio_cache.invalidate_all_domains() + + +def set_domain_public_hostname(domain: str) -> str: + """Webmail mail.{domínio} — evita redirect para mail.ligbox.com.br.""" + domain = domain.lower().strip() + mail_host = f"mail.{domain}" + logo = f"https://{mail_host}/public/logos/ligbox-logo.png" + admin = f"admin@{domain}" + _run_zmprov( + "md", + domain, + "zimbraPublicServiceHostname", + mail_host, + "zimbraVirtualHostname", + mail_host, + "zimbraPublicServiceProtocol", + "https", + "zimbraPublicServicePort", + "443", + "zimbraSkinLogoURL", + logo, + "carbonioLogoUrl", + logo, + "carbonioNotificationFrom", + admin, + "carbonioNotificationRecipients", + admin, + ) + return mail_host + + +def add_mail_alias_hostnames(domain: str, hostnames: list[str]) -> None: + """Adiciona hostnames virtuais (alias webmail) ao domínio Carbonio.""" + domain = domain.lower().strip() + primary_mail = f"mail.{domain}" + for host in hostnames: + h = host.lower().strip().rstrip(".") + if not h or h == primary_mail: + continue + try: + _run_zmprov("md", domain, "zimbraVirtualHostname", h) + activity_log.ok(f"Alias hostname Carbonio: {h}", source="vm112") + except CarbonioError as e: + activity_log.warn(f"Alias {h}: {e}", source="vm112") + + +def create_domain(domain: str) -> str: + domain = domain.lower().strip() + if domain_exists(domain): + activity_log.info(f"Domínio {domain} já existe no Carbonio", source="vm112") + set_domain_public_hostname(domain) + return f"domain {domain} already exists" + activity_log.info(f"Criar domínio no Carbonio: {domain}", source="vm112") + out = _run_zmprov("cd", domain) + set_domain_public_hostname(domain) + invalidate_domain_list_cache() + activity_log.ok(f"Webmail: https://mail.{domain}/", source="vm112") + return out + + +def create_account(email: str, password: str, display_name: str | None = None) -> str: + return create_account_full( + email, + password, + display_name=display_name, + quota_mb=0, + must_change_password=False, + ) + + +def ensure_onboarding_account( + email: str, + password: str, + display_name: str | None = None, +) -> tuple[str, bool]: + """ + Cria conta admin no onboarding ou reutiliza se já existir (retry do wizard). + Returns (message, reused). + """ + email = email.lower().strip() + domain = email.split("@", 1)[1] + if account_exists(email): + activity_log.info( + f"Conta {email} já existe no Carbonio — actualizar senha e continuar", + source="vm112", + ) + if display_name: + try: + _run_zmprov("ma", email, "displayName", display_name) + except CarbonioError: + pass + set_password(email, password) + set_password_must_change(email, False) + invalidate_domain_cache(domain) + return f"account {email} already exists (password updated)", True + msg = create_account(email, password, display_name) + return msg, False + + +def create_account_full( + email: str, + password: str, + *, + display_name: str | None = None, + quota_mb: int = 0, + must_change_password: bool = True, +) -> str: + """Uma única chamada zmprov ca (evita 3–4 round-trips de ~5s cada).""" + if not EMAIL_RE.match(email): + raise CarbonioError("invalid email") + domain = email.split("@", 1)[1] + if not domain_exists(domain): + create_domain(domain) + activity_log.info(f"Criar conta: {email}", source="vm112") + args = ["ca", email, password] + if display_name: + args.extend(["displayName", display_name]) + if quota_mb > 0: + args.extend(["zimbraMailQuota", str(int(quota_mb) * 1024 * 1024)]) + if must_change_password: + args.extend(["zimbraPasswordMustChange", "TRUE"]) + out = _run_zmprov(*args) + invalidate_domain_cache(domain) + return out + + +def account_summary_fast( + email: str, + *, + display_name: str | None = None, + quota_mb: int = 0, + must_change_password: bool = True, +) -> dict: + """Resposta imediata sem zmprov ga (~5s).""" + quota_bytes = int(quota_mb) * 1024 * 1024 if quota_mb > 0 else 0 + return { + "email": email.lower(), + "display_name": display_name or "", + "status": "active", + "active": True, + "suspended": False, + "quota_bytes": quota_bytes, + "quota_label": _quota_label(str(quota_bytes)), + "two_factor_enabled": False, + "password_must_change": must_change_password, + "last_logon": None, + } + + +def list_accounts(domain: str, *, use_cache: bool = True) -> list[str]: + """Lista contas do domínio (zmprov -l gaa ~3–5s — cache TTL curto).""" + domain = domain.lower().strip() + cache_key = f"accounts_list:{domain}" + if use_cache: + cached = carbonio_cache.get(cache_key) + if cached is not None: + return list(cached) + code, out, err = _zmprov_run("-l", "gaa", domain, log_cmd=False) + if code != 0: + msg = err or out or "zmprov gaa failed" + raise CarbonioError(msg) + result = sorted(line.strip() for line in out.splitlines() if line.strip()) + if use_cache: + carbonio_cache.set(cache_key, result, carbonio_cache.TTL_ACCOUNTS_LIST) + return result + + +def set_password(email: str, password: str) -> str: + return _run_zmprov("sp", email, password) + + +def account_exists(email: str) -> bool: + code, out, err = _zmprov_run("ga", email, log_cmd=False) + if code == 0: + return email.lower() in out.lower() + if _is_missing_account(err, out): + return False + activity_log.warn(f"zmprov ga {email}: {err or out}", source="vm112") + return False + + +def _parse_ga_attributes(out: str) -> dict[str, str]: + attrs: dict[str, str] = {} + for line in out.splitlines(): + if ":" not in line: + continue + key, _, val = line.partition(":") + attrs[key.strip()] = val.strip() + return attrs + + +def get_account_attributes(email: str) -> dict[str, str]: + code, out, err = _zmprov_run("ga", email, log_cmd=False) + if code != 0: + raise CarbonioError(err or out or f"Conta {email} não encontrada") + return _parse_ga_attributes(out) + + +def get_domain_attributes(domain: str) -> dict[str, str]: + code, out, err = _zmprov_run("gd", domain, log_cmd=False) + if code != 0: + raise CarbonioError(err or out or f"Domínio {domain} não encontrado") + return _parse_ga_attributes(out) + + +def _quota_label(bytes_raw: str) -> str: + try: + n = int(bytes_raw) + except (TypeError, ValueError): + return bytes_raw or "—" + if n <= 0: + return "Ilimitada (COS)" + if n >= 1024**3: + return f"{n / 1024**3:.1f} GB" + if n >= 1024**2: + return f"{n / 1024**2:.0f} MB" + return f"{n} B" + + +def account_summary(email: str, *, use_cache: bool = True) -> dict: + email = email.lower().strip() + cache_key = f"account_summary:{email}" + if use_cache: + cached = carbonio_cache.get(cache_key) + if cached is not None: + return cached + attrs = get_account_attributes(email) + status = attrs.get("zimbraAccountStatus", "unknown") + quota_b = attrs.get("zimbraMailQuota", "0") + tfa = attrs.get("zimbraTwoFactorAuthEnabled", "").upper() == "TRUE" + must_change = attrs.get("zimbraPasswordMustChange", "").upper() == "TRUE" + last_logon = attrs.get("zimbraLastLogonTimestamp", "") + if last_logon and len(last_logon) >= 8: + last_logon = f"{last_logon[0:4]}-{last_logon[4:6]}-{last_logon[6:8]}" + summary = { + "email": email.lower(), + "display_name": attrs.get("displayName") or attrs.get("cn") or "", + "status": status, + "active": status == "active", + "suspended": status in ("locked", "closed", "maintenance"), + "quota_bytes": int(quota_b) if quota_b.isdigit() else 0, + "quota_label": _quota_label(quota_b), + "two_factor_enabled": tfa, + "password_must_change": must_change, + "last_logon": last_logon or None, + "_summary": True, + } + if use_cache: + carbonio_cache.set(cache_key, summary, carbonio_cache.TTL_ACCOUNT_SUMMARY) + return summary + + +def list_domain_accounts_fast(domain: str, *, use_cache: bool = True) -> list[dict]: + """Uma chamada zmprov -l gaa (~5s) em vez de ga por conta (~5s × N).""" + domain = domain.lower().strip() + cache_key = f"accounts_fast:{domain}" + if use_cache: + cached = carbonio_cache.get(cache_key) + if cached is not None: + return cached + emails = list_accounts(domain) + result = [ + { + "email": e, + "display_name": "", + "status": "—", + "active": True, + "suspended": False, + "quota_label": "—", + "two_factor_enabled": False, + "password_must_change": False, + "last_logon": None, + "_summary": False, + } + for e in emails + ] + if use_cache: + carbonio_cache.set(cache_key, result, carbonio_cache.TTL_ACCOUNTS_LIST) + return result + + +def list_domain_accounts(domain: str, *, full: bool = False, use_cache: bool = True) -> list[dict]: + if not full: + return list_domain_accounts_fast(domain, use_cache=use_cache) + domain = domain.lower().strip() + emails = list_accounts(domain) + result = [] + for email in emails: + try: + result.append(account_summary(email, use_cache=use_cache)) + except CarbonioError: + result.append( + { + "email": email, + "status": "unknown", + "active": False, + "suspended": False, + "_summary": False, + } + ) + return result + + +def invalidate_domain_cache(domain: str) -> None: + carbonio_cache.invalidate_domain(domain.lower().strip()) + + +def set_account_status(email: str, active: bool) -> str: + status = "active" if active else "locked" + activity_log.info(f"Conta {email} → {status}", source="vm112") + out = _run_zmprov("ma", email, "zimbraAccountStatus", status) + carbonio_cache.invalidate_account(email) + return out + + +def set_mail_quota(email: str, quota_mb: int) -> str: + if quota_mb <= 0: + bytes_q = "0" + else: + bytes_q = str(int(quota_mb) * 1024 * 1024) + activity_log.info(f"Quota {email} → {quota_mb} MB", source="vm112") + out = _run_zmprov("ma", email, "zimbraMailQuota", bytes_q) + carbonio_cache.invalidate_account(email) + return out + + +def set_password_must_change(email: str, must_change: bool = True) -> str: + val = "TRUE" if must_change else "FALSE" + out = _run_zmprov("ma", email, "zimbraPasswordMustChange", val) + carbonio_cache.invalidate_account(email) + return out + + +def set_password_with_policy( + email: str, + password: str, + *, + must_change: bool = True, +) -> str: + out = set_password(email, password) + if must_change: + set_password_must_change(email, True) + else: + carbonio_cache.invalidate_account(email) + return out + + +def domain_two_factor_capabilities(domain: str) -> dict: + attrs = get_domain_attributes(domain) + avail = attrs.get("zimbraFeatureTwoFactorAuthAvailable", "FALSE").upper() == "TRUE" + required = attrs.get("zimbraFeatureTwoFactorAuthRequired", "FALSE").upper() == "TRUE" + return { + "available": avail, + "required": required, + "automated": avail, + "message": ( + "2FA disponível neste domínio (política via painel activa)." + if avail + else ( + "Política 2FA via painel inactiva no COS/domínio. " + "Utilizadores podem activar OTP em webmail: Settings → Auth → OTP Authentication → NEW OTP." + ) + ), + } + + +def set_user_two_factor(email: str, enabled: bool) -> str: + caps = domain_two_factor_capabilities(email.split("@", 1)[1]) + if not caps["available"]: + raise CarbonioError(caps["message"]) + val = "TRUE" if enabled else "FALSE" + activity_log.info(f"2FA {email} → {val}", source="vm112") + return _run_zmprov("ma", email, "zimbraTwoFactorAuthEnabled", val) + + +def set_domain_two_factor_required(domain: str, required: bool) -> str: + caps = domain_two_factor_capabilities(domain) + if not caps["available"]: + raise CarbonioError(caps["message"]) + val = "TRUE" if required else "FALSE" + return _run_zmprov("md", domain, "zimbraFeatureTwoFactorAuthRequired", val) diff --git a/deploy/vm112-wizard/perf-domains-list-20260625/carbonio_cache.py b/deploy/vm112-wizard/perf-domains-list-20260625/carbonio_cache.py new file mode 100644 index 0000000..7fd1bed --- /dev/null +++ b/deploy/vm112-wizard/perf-domains-list-20260625/carbonio_cache.py @@ -0,0 +1,75 @@ +"""Cache em memória para reduzir chamadas lentas ao zmprov.""" + +from __future__ import annotations + +import threading +import time +from typing import Any + +_LOCK = threading.Lock() +_STORE: dict[str, tuple[float, Any, float]] = {} + +TTL_ACCOUNTS_LIST = 45 +TTL_DOMAIN_EXISTS = 120 +TTL_ACCOUNT_SUMMARY = 60 +TTL_ALL_DOMAINS = 90 +TTL_ORCH_LIST = 60 +TTL_DOMAIN_DETAIL = 120 +TTL_INFRA_STATUS = 90 + +CACHE_KEY_ALL_DOMAINS = "gad_all_domains" +CACHE_KEY_ORCH_LIST = "orchestrated_domains_list" + + +def get(key: str) -> Any | None: + with _LOCK: + item = _STORE.get(key) + if not item: + return None + expires, value, _set_at = item + if time.monotonic() > expires: + del _STORE[key] + return None + return value + + +def set(key: str, value: Any, ttl: int) -> None: + now = time.monotonic() + with _LOCK: + _STORE[key] = (now + ttl, value, now) + + +def entry_age_sec(key: str) -> int | None: + with _LOCK: + item = _STORE.get(key) + if not item: + return None + expires, _value, set_at = item + now = time.monotonic() + if now > expires: + return None + return int(now - set_at) + + +def invalidate_domain(domain: str) -> None: + domain = domain.lower().strip() + with _LOCK: + for key in list(_STORE): + if domain in key or key.endswith(f":{domain}") or f":{domain}:" in key: + del _STORE[key] + + +def invalidate_account(email: str) -> None: + email = email.lower().strip() + domain = email.split("@", 1)[1] if "@" in email else "" + with _LOCK: + _STORE.pop(f"account_summary:{email}", None) + if domain: + invalidate_domain(domain) + + +def invalidate_all_domains() -> None: + """Após zmprov cd/dd/purge — lista gad + orquestração Desk.""" + with _LOCK: + _STORE.pop(CACHE_KEY_ALL_DOMAINS, None) + _STORE.pop(CACHE_KEY_ORCH_LIST, None) diff --git a/deploy/vm112-wizard/perf-domains-list-20260625/domain_orchestration.py b/deploy/vm112-wizard/perf-domains-list-20260625/domain_orchestration.py new file mode 100644 index 0000000..a00bfcf --- /dev/null +++ b/deploy/vm112-wizard/perf-domains-list-20260625/domain_orchestration.py @@ -0,0 +1,598 @@ +"""Listagem e purge de domínios orquestrados na VM112 (Spec 017 + 026).""" + +from __future__ import annotations + +import json +import re +import shutil +import subprocess +from concurrent.futures import ThreadPoolExecutor +from pathlib import Path +from typing import Any, Callable + +from app.config import settings +from app.services import activity_log, carbonio, carbonio_cache, domain_registry, purge_jobs +from app.services.domain_site_layout import PLATFORM_DOMAINS, SITES_ROOT +from app.services.infrastructure import _mail_host +from app.services.traefik_purge import ( + _collect_purge_hosts, + purge_traefik_routers, + purge_traefik_sni, + rollback_traefik_backup, + validate_traefik_after_purge, +) +from app.services.purge_snapshot import build_audit_snapshot +from app.services.purge_openpanel import purge_openpanel +from app.services.purge_pmg import purge_pmg + +_LOG_DIR = Path("/var/log/ibytera-mail-portal/sessions") +_WIZARD_PORTAL_USERS_DIR = Path("/var/lib/ligbox-wizard/portal_users") +_TENANT_BRANDING = Path("/opt/ligbox-wizard/backend/app/services/tenant_branding.py") +_DEPLOY_SCRIPTS = Path("/opt/ligbox-deploy/scripts/admin-login-check") +_TRAEFIK_CERT_EXPORT = Path("/opt/zextras/ssl/letsencrypt/traefik-export") +_NGINX_INCLUDES = Path("/opt/zextras/conf/nginx/includes") +_PURGE_BLOCKLIST = PLATFORM_DOMAINS | frozenset({"itecnologys.com"}) + + +def _domain_slug(domain: str) -> str: + return domain.lower().strip().replace(".", "-") + + +def _list_carbonio_domains() -> list[str]: + return carbonio.list_all_domains() + + +def _list_site_domains() -> list[str]: + if not SITES_ROOT.is_dir(): + return [] + return [ + p.name.lower() + for p in SITES_ROOT.iterdir() + if p.is_dir() and (p / "domain.json").is_file() + ] + + +def _portal_users_for_domain(domain: str, users_dir: Path | None = None) -> list[dict[str, Any]]: + domain = domain.lower().strip() + base = users_dir or Path(settings.portal_users_dir) + if not base.is_dir(): + return [] + found: list[dict[str, Any]] = [] + for f in base.glob("*.json"): + try: + data = json.loads(f.read_text(encoding="utf-8")) + except (json.JSONDecodeError, OSError): + continue + email = (data.get("planned_corporate_email") or "").lower().strip() + if email.endswith(f"@{domain}"): + found.append({ + "login_id": data.get("login_id") or f.stem, + "planned_corporate_email": email, + "path": str(f), + }) + return found + + +def _build_orchestrated_domains(query: str = "") -> list[dict[str, Any]]: + carbonio_set = set(_list_carbonio_domains()) + names = sorted(carbonio_set | set(_list_site_domains())) + users_by_domain: dict[str, list[dict[str, Any]]] = {} + users_dir = Path(settings.portal_users_dir) + if users_dir.is_dir(): + for f in users_dir.glob("*.json"): + try: + data = json.loads(f.read_text(encoding="utf-8")) + except (json.JSONDecodeError, OSError): + continue + email = (data.get("planned_corporate_email") or "").lower().strip() + if "@" not in email: + continue + dom = email.split("@", 1)[1] + users_by_domain.setdefault(dom, []).append({ + "login_id": data.get("login_id") or f.stem, + "planned_corporate_email": email, + "path": str(f), + }) + if _WIZARD_PORTAL_USERS_DIR.is_dir(): + for f in _WIZARD_PORTAL_USERS_DIR.glob("*.json"): + try: + data = json.loads(f.read_text(encoding="utf-8")) + except (json.JSONDecodeError, OSError): + continue + email = (data.get("planned_corporate_email") or "").lower().strip() + if "@" not in email: + continue + dom = email.split("@", 1)[1] + entry = { + "login_id": data.get("login_id") or f.stem, + "planned_corporate_email": email, + "path": str(f), + } + if entry not in users_by_domain.get(dom, []): + users_by_domain.setdefault(dom, []).append(entry) + q = (query or "").strip().lower() + items: list[dict[str, Any]] = [] + for domain in names: + if domain in PLATFORM_DOMAINS: + continue + rec = domain_registry.get_domain_record(domain) or {} + admin_email = rec.get("portal_admin_email") or "" + exists = domain in carbonio_set + item = { + "domain": domain, + "mail_host": _mail_host(domain), + "portal_admin_email": admin_email, + "carbonio_exists": exists, + "site_folder_exists": (SITES_ROOT / domain / "domain.json").is_file(), + "account_count": None, + "accounts_preview": [], + "portal_users": users_by_domain.get(domain, []), + "updated_at": rec.get("updated_at"), + } + if q: + blob = " ".join([ + domain, + admin_email, + " ".join(u.get("login_id", "") for u in item["portal_users"]), + ]).lower() + if q not in blob: + continue + items.append(item) + return items + + +def list_orchestrated_domains(query: str = "") -> list[dict[str, Any]]: + q = (query or "").strip().lower() + if not q: + cached = carbonio_cache.get(carbonio_cache.CACHE_KEY_ORCH_LIST) + if cached is not None: + return cached + items = _build_orchestrated_domains(q) + if not q: + carbonio_cache.set( + carbonio_cache.CACHE_KEY_ORCH_LIST, + items, + carbonio_cache.TTL_ORCH_LIST, + ) + return items + + +def list_orchestrated_domains_meta(query: str = "") -> dict[str, Any]: + items = list_orchestrated_domains(query) + age = carbonio_cache.entry_age_sec(carbonio_cache.CACHE_KEY_ALL_DOMAINS) + return { + "domains": items, + "cached": age is not None, + "cache_age_sec": age, + } + + +def get_domain_detail(domain: str) -> dict[str, Any]: + domain = domain.lower().strip() + detail_key = f"domain_detail:{domain}" + cached = carbonio_cache.get(detail_key) + if cached is not None: + return dict(cached) + + matches = [i for i in list_orchestrated_domains() if i["domain"] == domain] + if not matches: + raise ValueError(f"Domínio {domain} não encontrado na orquestração") + base = dict(matches[0]) + from app.services.infrastructure import get_status + + rec = domain_registry.get_domain_record(domain) or {} + aliases = list(rec.get("mail_aliases") or []) + base["domain_record"] = rec + carbonio_exists = bool(base.get("carbonio_exists")) + + def fetch_accounts() -> list[str]: + if not carbonio_exists: + return [] + try: + return carbonio.list_accounts(domain) + except carbonio.CarbonioError: + return [] + + def fetch_infra() -> dict[str, Any]: + return get_status(domain, aliases) + + def fetch_cf() -> dict[str, Any]: + try: + from app.services.cloudflare import CloudflareDNS + + return CloudflareDNS().get_zone_by_name(domain) + except Exception as e: + return {"error": str(e)} + + with ThreadPoolExecutor(max_workers=3) as pool: + fut_acc = pool.submit(fetch_accounts) + fut_inf = pool.submit(fetch_infra) + fut_cf = pool.submit(fetch_cf) + base["accounts"] = fut_acc.result() + base["infra_status"] = fut_inf.result() + base["cloudflare_zone"] = fut_cf.result() + + carbonio_cache.set(detail_key, base, carbonio_cache.TTL_DOMAIN_DETAIL) + return base + + +def _delete_carbonio_accounts(domain: str) -> list[str]: + logs: list[str] = [] + accounts: list[str] = [] + try: + accounts = carbonio.list_accounts(domain) + except carbonio.CarbonioError: + pass + for email in accounts: + code, _out, _err = carbonio._zmprov_run("da", email, log_cmd=True) + logs.append(f"da {email}: rc={code}") + return logs + + +def _delete_carbonio_domain_only(domain: str) -> str | None: + if carbonio.domain_exists(domain, use_cache=False): + code, _out, _err = carbonio._zmprov_run("dd", domain, log_cmd=True) + carbonio.invalidate_domain_list_cache() + return f"dd {domain}: rc={code}" + return None + + +def _purge_portal_users(domain: str) -> list[str]: + removed: list[str] = [] + for users_dir in (Path(settings.portal_users_dir), _WIZARD_PORTAL_USERS_DIR): + for u in _portal_users_for_domain(domain, users_dir): + try: + Path(u["path"]).unlink(missing_ok=True) + label = f"{u['login_id']}@{users_dir.name}" + if label not in removed: + removed.append(label) + except OSError as e: + removed.append(f"ERR:{u['login_id']}:{e}") + return removed + + +def _purge_site_folder(domain: str) -> bool: + path = SITES_ROOT / domain + if path.is_dir(): + shutil.rmtree(path) + return True + return False + + +def _purge_cloudflare_zone(domain: str) -> str: + try: + from app.services.cloudflare import CloudflareDNS, CloudflareError + + cf = CloudflareDNS() + zone = cf.get_zone_by_name(domain) + if not zone: + return "no_zone" + zid = zone.get("id") + with cf._client() as c: + r = c.delete(f"https://api.cloudflare.com/client/v4/zones/{zid}") + data = r.json() + if data.get("success"): + return f"deleted:{zid}" + raise CloudflareError(str(data.get("errors"))) + except Exception as e: + return f"error:{e}" + + +def _purge_tenant_branding(domain: str) -> str: + if not _TENANT_BRANDING.is_file(): + return "skip" + text = _TENANT_BRANDING.read_text(encoding="utf-8") + needle = f'"{domain}"' + if needle not in text: + return "absent" + new_text = re.sub(rf'^\s*"{re.escape(domain)}".*\n', "", text, flags=re.MULTILINE) + _TENANT_BRANDING.write_text(new_text, encoding="utf-8") + return "removed" + + +def _purge_deploy_script_refs(domain: str) -> str: + mail_host = _mail_host(domain) + changed = [] + for name in ("apply-admin-nginx-overrides.py", "sync-traefik-admin-certs.sh"): + path = _DEPLOY_SCRIPTS / name + if not path.is_file(): + continue + text = path.read_text(encoding="utf-8") + if mail_host not in text: + continue + new_text = re.sub(rf'^\s*"{re.escape(mail_host)}".*\n', "", text, flags=re.MULTILINE) + new_text = new_text.replace(f" {mail_host}", "") + new_text = new_text.replace(f"'{mail_host}'", "") + path.write_text(new_text, encoding="utf-8") + changed.append(name) + return ",".join(changed) if changed else "none" + + +def _purge_traefik_export_certs(domain: str) -> str: + slug = _domain_slug(domain) + pattern = f"mail-{slug}*" + removed = 0 + if _TRAEFIK_CERT_EXPORT.is_dir(): + for f in _TRAEFIK_CERT_EXPORT.glob(pattern): + f.unlink(missing_ok=True) + removed += 1 + return str(removed) + + +def _remove_nginx_server_block(text: str, server_name: str) -> tuple[str, bool]: + pattern = re.compile( + rf"server\s*\{{\s*\n\s*server_name\s+{re.escape(server_name)}\b", + re.MULTILINE, + ) + match = pattern.search(text) + if not match: + return text, False + start = match.start() + brace = 0 + i = text.find("{", start) + while i < len(text): + if text[i] == "{": + brace += 1 + elif text[i] == "}": + brace -= 1 + if brace == 0: + end = i + 1 + while end < len(text) and text[end] in "\r\n": + end += 1 + return text[:start] + text[end:], True + i += 1 + return text, False + + +def _purge_nginx_vhosts(domain: str, hosts: list[str]) -> str: + if not _NGINX_INCLUDES.is_dir(): + return "skip" + targets = [ + _NGINX_INCLUDES / "nginx.conf.web.https", + _NGINX_INCLUDES / "nginx.conf.web.admin", + _NGINX_INCLUDES / "nginx.conf.web.http", + _NGINX_INCLUDES / "nginx.conf.mail.imap", + _NGINX_INCLUDES / "nginx.conf.mail.imaps", + _NGINX_INCLUDES / "nginx.conf.mail.pop3", + _NGINX_INCLUDES / "nginx.conf.mail.pop3s", + _NGINX_INCLUDES / "nginx.conf.map.key", + _NGINX_INCLUDES / "nginx.conf.map.crt", + ] + edited = 0 + for fp in targets: + if not fp.is_file(): + continue + text = fp.read_text(encoding="utf-8") + orig = text + for host in hosts: + while True: + text, removed = _remove_nginx_server_block(text, host) + if not removed: + break + for host in hosts: + text = re.sub(rf"^{re.escape(host)} .*\n", "", text, flags=re.MULTILINE) + if text != orig: + fp.with_suffix(fp.suffix + f".bak-purge-{_domain_slug(domain)}").write_text(orig, encoding="utf-8") + fp.write_text(text, encoding="utf-8") + edited += 1 + if edited == 0: + return "none" + proc = subprocess.run( + ["/opt/zextras/common/sbin/nginx", "-t", "-c", "/opt/zextras/conf/nginx.conf"], + capture_output=True, + text=True, + timeout=60, + ) + if proc.returncode != 0: + return f"nginx_test_fail:{(proc.stderr or proc.stdout)[:120]}" + subprocess.run( + ["su", "-", "zextras", "-c", "/opt/zextras/common/sbin/nginx -s reload"], + capture_output=True, + timeout=60, + ) + return f"updated:{edited}" + + +def _purge_session_logs(domain: str) -> int: + count = 0 + if not _LOG_DIR.is_dir(): + return 0 + for f in _LOG_DIR.glob("*.jsonl"): + try: + if domain in f.read_text(encoding="utf-8", errors="ignore").lower(): + f.unlink() + count += 1 + except OSError: + pass + return count + + +def _execute_purge( + domain: str, + report: Callable[[str, str, str | None], None] | None = None, + *, + job_id: str | None = None, + requested_by: str | None = None, +) -> dict[str, Any]: + def _step(step_id: str, status: str, detail: str | None = None) -> None: + if report: + report(step_id, status, detail) + if status == "running": + activity_log.info(f"PURGE [{step_id}] {detail or '…'}", source="admin") + elif status == "done": + activity_log.ok(f"PURGE [{step_id}] {detail or 'OK'}", source="admin") + elif status == "error": + activity_log.error(f"PURGE [{step_id}] {detail or 'erro'}", source="admin") + + rec = domain_registry.get_domain_record(domain) or {} + aliases = list(rec.get("mail_aliases") or []) + hosts = _collect_purge_hosts(domain, aliases) + result: dict[str, Any] = {"domain": domain} + traefik_backup: str | None = None + + _step("audit_snapshot", "running") + snapshot = build_audit_snapshot(domain, job_id=job_id, requested_by=requested_by) + result["audit_snapshot"] = snapshot + ac = snapshot.get("carbonio", {}).get("account_count", 0) + _step("audit_snapshot", "done", f"{ac} conta(s), inventário gravado") + + _step("carbonio_accounts", "running") + result["carbonio_accounts"] = _delete_carbonio_accounts(domain) + _step("carbonio_accounts", "done", "; ".join(result["carbonio_accounts"]) or "nenhuma conta") + + _step("carbonio_domain", "running") + dd = _delete_carbonio_domain_only(domain) + result["carbonio_domain"] = dd or "domínio já ausente" + _step("carbonio_domain", "done", result["carbonio_domain"]) + + _step("portal_users", "running") + result["portal_users_removed"] = _purge_portal_users(domain) + _step("portal_users", "done", ", ".join(result["portal_users_removed"]) or "nenhum") + + _step("site_folder", "running") + result["site_folder_removed"] = _purge_site_folder(domain) + _step("site_folder", "done", "removido" if result["site_folder_removed"] else "já ausente") + + _step("cloudflare", "running") + result["cloudflare"] = _purge_cloudflare_zone(domain) + _step("cloudflare", "done", result["cloudflare"]) + + _step("session_logs", "running") + result["session_logs_removed"] = _purge_session_logs(domain) + _step("session_logs", "done", str(result["session_logs_removed"])) + + _step("tenant_branding", "running") + result["tenant_branding"] = _purge_tenant_branding(domain) + _step("tenant_branding", "done", result["tenant_branding"]) + + _step("deploy_scripts", "running") + result["deploy_scripts"] = _purge_deploy_script_refs(domain) + _step("deploy_scripts", "done", result["deploy_scripts"]) + + _step("traefik_export_certs", "running") + result["traefik_export_certs"] = _purge_traefik_export_certs(domain) + _step("traefik_export_certs", "done", result["traefik_export_certs"]) + + _step("nginx_vhosts", "running") + result["nginx_vhosts"] = _purge_nginx_vhosts(domain, hosts) + nginx_ok = not str(result["nginx_vhosts"]).startswith("nginx_test_fail") + _step("nginx_vhosts", "done" if nginx_ok else "error", result["nginx_vhosts"]) + + try: + _step("traefik_sni", "running") + result["traefik_sni"] = purge_traefik_sni(hosts) + sni_ok = not str(result["traefik_sni"]).startswith("sni_fail") + _step("traefik_sni", "done" if sni_ok else "error", result["traefik_sni"]) + + _step("traefik_routers", "running") + tr = purge_traefik_routers(domain, hosts) + result["traefik_routers"] = tr.get("detail") or ("traefik_ok" if tr.get("ok") else "traefik_fail") + traefik_backup = tr.get("backup") + result["traefik_backup"] = traefik_backup + tr_ok = bool(tr.get("ok")) + _step("traefik_routers", "done" if tr_ok else "error", result["traefik_routers"]) + + _step("traefik_validate", "running") + validation = validate_traefik_after_purge(domain) + result["traefik_validate"] = validation + val_detail = json.dumps(validation.get("checks") or {}, ensure_ascii=False)[:400] + val_ok = bool(validation.get("ok")) + if not val_ok and traefik_backup: + rb = rollback_traefik_backup(traefik_backup) + result["traefik_rollback"] = rb + validation = validate_traefik_after_purge(domain) + result["traefik_validate_after_rollback"] = validation + _step("traefik_validate", "done" if val_ok else "error", val_detail) + + if not sni_ok or not tr_ok or not val_ok: + raise RuntimeError( + f"Traefik purge incompleto (sni={result['traefik_sni']}, " + f"routers={result['traefik_routers']}, validate={val_detail})" + ) + except Exception as e: + result["traefik_error"] = str(e) + if report: + _step("traefik_validate", "error", str(e)) + raise + + _step("openpanel_dns", "running") + op = purge_openpanel(domain) + result["openpanel"] = op + _step("openpanel_dns", "done", op.get("dns", "ok")) + _step("openpanel_user", "running") + _step("openpanel_user", "done", op.get("user", "ok")) + + _step("pmg_domain", "running") + pmg = purge_pmg(domain) + result["pmg"] = pmg + _step("pmg_domain", "done", pmg.get("domains", "ok")) + _step("pmg_transport", "running") + _step("pmg_transport", "done", pmg.get("transport", "ok")) + + result["carbonio"] = result.get("carbonio_accounts", []) + ( + [result["carbonio_domain"]] if result.get("carbonio_domain") else [] + ) + return result + + +def run_purge_job(job_id: str) -> None: + job = purge_jobs.get_job(job_id) + if not job: + return + domain = job["domain"] + report = purge_jobs.make_reporter(job_id) + try: + if domain in _PURGE_BLOCKLIST: + raise ValueError(f"Domínio {domain} está na blocklist de purge") + activity_log.info(f"PURGE job {job_id} iniciado: {domain}", source="admin") + result = _execute_purge(domain, report=report, job_id=job_id) + purge_jobs.complete_job(job_id, result) + carbonio.invalidate_domain_list_cache() + carbonio_cache.invalidate_domain(domain) + activity_log.ok(f"PURGE job {job_id} concluído: {domain}", source="admin") + except Exception as e: + purge_jobs.fail_job(job_id, str(e)) + activity_log.error(f"PURGE job {job_id} falhou: {e}", source="admin") + + +def assert_purge_allowed(domain: str) -> None: + domain = domain.lower().strip() + if domain in _PURGE_BLOCKLIST: + raise ValueError(f"Domínio {domain} está na blocklist de purge") + + +def purge_domain(domain: str) -> dict[str, Any]: + domain = domain.lower().strip() + assert_purge_allowed(domain) + activity_log.info(f"PURGE iniciado: {domain}", source="admin") + job = purge_jobs.create_job(domain) + job_id = job["job_id"] + report = purge_jobs.make_reporter(job_id) + try: + result = _execute_purge(domain, report=report) + purge_jobs.complete_job(job_id, result) + carbonio.invalidate_domain_list_cache() + carbonio_cache.invalidate_domain(domain) + activity_log.ok(f"PURGE concluído: {domain}", source="admin") + job_data = purge_jobs.get_job(job_id) or {} + steps = purge_jobs.steps_for_desk(job_data.get("steps") or []) + return {"domain": domain, "steps": steps, **result} + except Exception as e: + purge_jobs.fail_job(job_id, str(e)) + raise + + +def delete_carbonio_account(email: str) -> dict: + """Remove uma conta Carbonio (zmprov da) — Spec 022.""" + email = email.lower().strip() + if "@" not in email: + raise ValueError("e-mail inválido") + domain = email.split("@", 1)[1] + assert_purge_allowed(domain) + if not carbonio.account_exists(email): + return {"ok": True, "email": email, "message": "Conta já não existia", "skipped": True} + code, out, err = carbonio._zmprov_run("da", email, log_cmd=True) + if code != 0 and not carbonio._is_missing_account(err, out): + raise carbonio.CarbonioError(err or out or f"zmprov da falhou para {email}") + carbonio.invalidate_domain_cache(domain) + return {"ok": True, "email": email, "message": f"Conta {email} removida do Carbonio", "rc": code} diff --git a/deploy/vm112-wizard/perf-domains-list-20260625/infrastructure.py b/deploy/vm112-wizard/perf-domains-list-20260625/infrastructure.py new file mode 100644 index 0000000..d5b2a52 --- /dev/null +++ b/deploy/vm112-wizard/perf-domains-list-20260625/infrastructure.py @@ -0,0 +1,380 @@ +"""Provisionamento infra: HAProxy SNI (CT114), Traefik, certificado LE, Carbonio.""" + +from __future__ import annotations + +import re +import subprocess +from typing import Any + +import httpx + +from app.config import settings +from app.services import activity_log, carbonio, dns_verify, carbonio_cache +from app.services.domain_registry import get_domain_record +from app.services.mail_aliases import sanitize_mail_aliases + +CERT_NAME = "mail-vm112-multi" +TRAEFIK_SERVICE = "dratcoin-Service" +SNI_MAP = "/root/traefik/haproxy-mail-sni/maps/sni_vm112.lst" +TRAEFIK_DYNAMIC = "/root/traefik/dynamic.yml" +DEPLOY_SCRIPT = "/usr/local/bin/carbonio-cert-deploy.sh" + + +def _mail_host(domain: str) -> str: + return f"mail.{domain.lower().strip()}" + + +def _router_key_for_host(mail_host: str) -> str: + slug = re.sub(r"[^a-z0-9]+", "-", mail_host.lower()).strip("-") + return f"mail-{slug}-Router" + + +def _router_key(domain: str) -> str: + return _router_key_for_host(_mail_host(domain)) + + +def _collect_mail_hosts(domain: str, mail_aliases: list[str] | None = None) -> list[str]: + primary = _mail_host(domain) + hosts = [primary] + for h in mail_aliases or []: + hh = h.lower().strip().rstrip(".") + if hh and hh not in hosts: + hosts.append(hh) + return hosts + + +def _cert_san_hostnames(all_hosts: list[str]) -> list[str]: + """ + Hostnames para certificado LE — só webmail (mail.* / webmail.*). + Exclui portal.* e outros aliases que não servem Carbonio directamente. + """ + cert: list[str] = [] + for h in all_hosts: + hl = h.lower().strip() + if hl.startswith("portal."): + continue + if hl.startswith("mail.") or hl.startswith("webmail."): + cert.append(hl) + return sorted(set(cert)) + + +def _normalize_cert_sans(existing_sans: list[str], cert_hosts: list[str]) -> list[str]: + """Mantém SANs activos no certificado multi-domínio.""" + result = set(cert_hosts) + for san in existing_sans: + s = san.lower().strip() + if not (s.startswith("mail.") or s.startswith("webmail.")): + continue + if s.startswith("portal."): + continue + result.add(s) + return sorted(result) + + +def _ssh_ct114(command: str, timeout: int = 90) -> tuple[bool, str]: + host = settings.traefik_ssh_host + try: + proc = subprocess.run( + [ + "ssh", + "-o", + "BatchMode=yes", + "-o", + "StrictHostKeyChecking=accept-new", + host, + command, + ], + capture_output=True, + text=True, + timeout=timeout, + ) + out = (proc.stdout or "") + (proc.stderr or "") + return proc.returncode == 0, out.strip() + except (subprocess.TimeoutExpired, OSError) as e: + return False, str(e) + + +def _read_ct114_file(path: str) -> tuple[bool, str]: + ok, out = _ssh_ct114(f"cat {path} 2>/dev/null || true") + return ok, out + + +def _cert_domains() -> list[str]: + try: + proc = subprocess.run( + ["certbot", "certificates"], + capture_output=True, + text=True, + timeout=60, + ) + text = proc.stdout or "" + for block in text.split("Certificate Name:"): + if CERT_NAME in block: + m = re.search(r"Domains:\s*(.+)", block) + if m: + return m.group(1).split() + except (subprocess.TimeoutExpired, OSError): + pass + return [] + + +def get_status(domain: str, mail_aliases: list[str] | None = None) -> dict[str, Any]: + domain = domain.lower().strip() + cache_key = f"infra_status:{domain}" + cached = carbonio_cache.get(cache_key) + if cached is not None: + return dict(cached) + mail = _mail_host(domain) + if mail_aliases is not None: + aliases = sanitize_mail_aliases(mail_aliases, domain) + else: + rec = get_domain_record(domain) + aliases = list(rec.get("mail_aliases") or []) if rec else [] + all_hosts = _collect_mail_hosts(domain, aliases) + steps: list[dict[str, Any]] = [] + + # 1 Carbonio + c_ok = carbonio.domain_exists(domain) + c_msg = ( + "Domínio activo no Carbonio" + if c_ok + else "Aguarda «Criar conta agora» (zmprov cd + ca)" + ) + steps.append({"id": "carbonio_domain", "label": "Carbonio (domínio)", "ok": c_ok, "message": c_msg}) + + # 2 DNS público + try: + v = dns_verify.verify_mail_dns(domain, settings.mail_public_ip) + d_ok = bool(v.get("ready")) + d_msg = "MX e A mail OK" if d_ok else "DNS mail incompleto" + except Exception as e: + d_ok = False + d_msg = str(e) + steps.append({"id": "dns_mail", "label": "DNS público (MX/A)", "ok": d_ok, "message": d_msg}) + + # 3 HAProxy SNI + ok_ssh, sni_body = _read_ct114_file(SNI_MAP) + if not ok_ssh: + s_ok, s_msg = False, f"SSH CT114 indisponível: {sni_body[:120]}" + else: + sni_lines = {ln.strip() for ln in sni_body.splitlines() if ln.strip()} + missing = [h for h in all_hosts if h not in sni_lines] + s_ok = len(missing) == 0 + s_msg = ( + f"{len(all_hosts)} hostname(s) no SNI" + if s_ok + else f"Faltam no SNI: {', '.join(missing)}" + ) + steps.append({"id": "haproxy_sni", "label": "HAProxy SNI (CT114)", "ok": s_ok, "message": s_msg}) + + # 4 Traefik router + ok_ssh, dyn = _read_ct114_file(TRAEFIK_DYNAMIC) + if not ok_ssh: + t_ok, t_msg = False, f"SSH CT114: {dyn[:120]}" + else: + missing_t = [h for h in all_hosts if f"Host(`{h}`)" not in dyn] + t_ok = len(missing_t) == 0 + t_msg = ( + f"Routers Traefik ({len(all_hosts)} hostnames)" + if t_ok + else f"Faltam routers: {', '.join(missing_t)}" + ) + steps.append({"id": "traefik_router", "label": "Traefik HTTPS (CT114)", "ok": t_ok, "message": t_msg}) + + # 5 Cert SAN (só hostnames mail.* — não portal.*) + cert_hosts = _cert_san_hostnames(all_hosts) + sans = _cert_domains() + missing_cert = [h for h in cert_hosts if h not in sans] + cert_ok = len(missing_cert) == 0 + steps.append( + { + "id": "cert_san", + "label": "Certificado LE (VM112)", + "ok": cert_ok, + "message": ( + f"SAN inclui {len(cert_hosts)} hostname(s) webmail" + if cert_ok + else f"Faltam no cert: {', '.join(missing_cert)}" + ), + } + ) + + # 6 Webmail HTTPS + try: + with httpx.Client(timeout=12.0, verify=False) as client: + r = client.get(f"https://{mail}/", follow_redirects=True) + w_ok = r.status_code < 500 + w_msg = f"HTTPS {r.status_code}" if w_ok else f"HTTPS erro {r.status_code}" + except Exception as e: + w_ok = False + w_msg = str(e)[:80] + steps.append({"id": "webmail_https", "label": "Webmail HTTPS", "ok": w_ok, "message": w_msg}) + + all_ok = all(s["ok"] for s in steps) + result = { + "domain": domain, + "mail_host": mail, + "mail_aliases": aliases, + "mail_hosts": all_hosts, + "steps": steps, + "ready": all_ok, + } + carbonio_cache.set(cache_key, result, carbonio_cache.TTL_INFRA_STATUS) + return result + + +def provision( + domain: str, + step_id: str | None = None, + mail_aliases: list[str] | None = None, +) -> dict[str, Any]: + domain = domain.lower().strip() + if mail_aliases is not None: + aliases = sanitize_mail_aliases(mail_aliases, domain) + else: + rec = get_domain_record(domain) + aliases = list(rec.get("mail_aliases") or []) if rec else [] + all_hosts = _collect_mail_hosts(domain, aliases) + results: list[dict[str, Any]] = [] + + def run(step: str, fn) -> None: + if step_id and step_id != step: + return + activity_log.info(f"Infra: iniciar {step}", source="traefik") + try: + msg = fn() + activity_log.ok(f"Infra {step}: {msg}", source="traefik") + results.append({"id": step, "ok": True, "message": msg}) + except Exception as e: + activity_log.error(f"Infra {step}: {e}", source="traefik") + results.append({"id": step, "ok": False, "message": str(e)}) + + def do_sni() -> str: + ok, body = _read_ct114_file(SNI_MAP) + if not ok: + raise RuntimeError(body) + added: list[str] = [] + for host in all_hosts: + if host in body.splitlines(): + continue + cmd = f"grep -qxF '{host}' {SNI_MAP} || echo '{host}' >> {SNI_MAP}" + ok2, out = _ssh_ct114(cmd, timeout=60) + if not ok2: + raise RuntimeError(out) + added.append(host) + if added: + ok3, out = _ssh_ct114( + "cd /root/traefik && docker compose restart mail-sni-proxy 2>&1 | tail -2", + timeout=120, + ) + if not ok3: + raise RuntimeError(out) + return f"SNI: {len(all_hosts)} hostname(s)" + (f" (+{len(added)} novos)" if added else " (já OK)") + + def do_traefik() -> str: + ok, body = _read_ct114_file(TRAEFIK_DYNAMIC) + if not ok: + raise RuntimeError(body) + added: list[str] = [] + for host in all_hosts: + if f"Host(`{host}`)" in body: + continue + r_key = _router_key_for_host(host) + remote = f"""python3 <<'PY' +from pathlib import Path +p = Path("{TRAEFIK_DYNAMIC}") +text = p.read_text() +mail = "{host}" +rkey = "{r_key}" +svc = "{TRAEFIK_SERVICE}" +if f"Host(`{{mail}}`)" in text: + print("exists") +else: + block = f''' + {{rkey}}: + rule: Host(`{{mail}}`) + service: {{svc}} + entryPoints: + - websecure + tls: + certResolver: letsencrypt + middlewares: + - default-headers +''' + idx = text.find(" services:") + if idx < 0: + raise SystemExit("services: não encontrado") + p.write_text(text[:idx] + block + text[idx:]) + print("added") +PY""" + ok2, out = _ssh_ct114(remote, timeout=90) + if not ok2: + raise RuntimeError(out) + added.append(host) + if added: + ok3, out2 = _ssh_ct114( + "cd /root/traefik && docker compose restart traefik 2>&1 | tail -3", + timeout=120, + ) + if not ok3: + raise RuntimeError(out2) + return f"Traefik: {len(all_hosts)} hostname(s)" + (f" (+{len(added)} routers)" if added else " (já OK)") + + def do_cert() -> str: + cert_hosts = _cert_san_hostnames(all_hosts) + sans = _cert_domains() + missing = [h for h in cert_hosts if h not in sans] + if not missing: + return f"SAN já inclui {len(cert_hosts)} hostname(s) webmail" + + try: + from app.services.cloudflare import CloudflareDNS + + zone = CloudflareDNS().get_zone_by_name(domain) + if zone and zone.get("status") != "active": + ns = ", ".join(zone.get("name_servers") or []) + raise RuntimeError( + f"Zona Cloudflare «{domain}» ainda não activa (status: {zone.get('status')}). " + f"Altere os nameservers no registrador para: {ns}" + ) + except RuntimeError: + raise + except Exception: + pass + + new_domains = _normalize_cert_sans(sans, cert_hosts) + dom_args = " ".join(f"-d {d}" for d in new_domains) + creds = settings.certbot_cloudflare_credentials + cmd = ( + f"certbot certonly --dns-cloudflare " + f"--dns-cloudflare-credentials {creds} " + f"--dns-cloudflare-propagation-seconds 90 " + f"--cert-name {CERT_NAME} --expand {dom_args} " + f"--non-interactive --agree-tos" + ) + proc = subprocess.run(cmd, shell=True, capture_output=True, text=True, timeout=600) + if proc.returncode != 0: + err = (proc.stderr or proc.stdout or "certbot falhou")[:500] + raise RuntimeError( + f"{err} — confirme NS Cloudflare do domínio e aguarde propagação DNS." + ) + lineage = f"/etc/letsencrypt/live/{CERT_NAME}" + deploy = subprocess.run( + f"RENEWED_LINEAGE={lineage} {DEPLOY_SCRIPT}", + shell=True, + capture_output=True, + text=True, + timeout=180, + ) + if deploy.returncode != 0: + raise RuntimeError((deploy.stderr or deploy.stdout or "deploy falhou")[:300]) + return f"certificado expandido ({len(new_domains)} SANs) e deploy Carbonio" + + run("haproxy_sni", do_sni) + run("traefik_router", do_traefik) + if step_id in (None, "cert_san", "carbonio_cert"): + if not step_id or step_id == "cert_san": + run("cert_san", do_cert) + + status = get_status(domain, aliases) + return {"domain": domain, "results": results, "status": status, "mail_aliases": aliases} diff --git a/docs/anais-referencia/20260616_SERVICOS_ORQUESTRACAO_PURGE_ASPECTOS.md b/docs/anais-referencia/20260616_SERVICOS_ORQUESTRACAO_PURGE_ASPECTOS.md index fc948e9..e498544 100644 --- a/docs/anais-referencia/20260616_SERVICOS_ORQUESTRACAO_PURGE_ASPECTOS.md +++ b/docs/anais-referencia/20260616_SERVICOS_ORQUESTRACAO_PURGE_ASPECTOS.md @@ -153,3 +153,13 @@ docker-compose -f docker-compose.mvp.yml up -d api frontend - Cada oferta = wizard próprio (Proxmox, servidor físico, etc.). - Modelo comercial Pizza as a Service documentado na Spec 018. - Purge Spec 017 mantido; UI evolui (drawer + jobs). + +--- + +## 12. Continuação — 2026-06-25 (performance + UX Escopo OPS) + +Ver documento dedicado: + +**`docs/anais-referencia/20260625_SERVICOS_IAAS_PERF_UX_ESCOPO.md`** + +Inclui: cache VM112/VM122, loading modal purge, cards Escopo OPS clicáveis, blocklist UI, versão frontend `20260625align1`. diff --git a/docs/anais-referencia/20260625_SERVICOS_IAAS_PERF_UX_ESCOPO.md b/docs/anais-referencia/20260625_SERVICOS_IAAS_PERF_UX_ESCOPO.md new file mode 100644 index 0000000..2e546cd --- /dev/null +++ b/docs/anais-referencia/20260625_SERVICOS_IAAS_PERF_UX_ESCOPO.md @@ -0,0 +1,124 @@ +# Anais de Referência — Serviços IaaS: Performance, UX Escopo OPS, Blocklist + +**Data:** 2026-06-25 +**Utilizador:** Roger +**Sistema:** Desk VM122 (`10.10.10.122`) · Wizard VM112 (`10.10.10.112`) +**Specs:** 017 · 018 · 026 · 028 · 032 · 037 + +--- + +## 1. Resumo executivo + +Sessão de melhorias na página **Serviços IaaS** (Desk): + +1. **Performance** — cache VM112 lista + detalhe domínio; cache proxy VM122; SWR frontend. +2. **Modal purge** — skeleton imediato + loading animado (barra, etapas, spinner). +3. **Escopo OPS** — cards clicáveis com camada (tenant/app/infra), Spec e navegação. +4. **Blocklist UI** — `ligbox.com.br` e `itecnologys.com` visíveis na lista e painel Escopo. +5. **Alinhamento lista clientes** — badges agrupados (fix desalinhamento `itecnologys.com`). + +--- + +## 2. Performance — resultados + +| Pedido | Antes | Depois (cache quente) | +|--------|-------|------------------------| +| VM112 `GET /api/admin/domains` | ~5–11s | **~0,03s** | +| VM112 `GET /api/admin/domains/{domain}` | ~15–20s | **~0,04s** | +| Desk `GET /api/v1/vm112/domains` | ~16s | **~0,08s** | +| Desk detalhe domínio (2.º hit) | ~15s+ | **~0,1s** | +| Modal purge 1.º hit (cold) | ecrã branco | skeleton + ~10–15s | + +**Patch VM112:** `deploy/vm112-wizard/perf-domains-list-20260625/` +**Desk API:** `VM112_DOMAINS_CACHE_TTL=60`, `VM112_DOMAIN_DETAIL_TTL=45` +**Poll Serviços:** 90s (`app.js`) + +--- + +## 3. UX Escopo OPS — mapa cards + +| Card | Camada | Spec | Destino | +|------|--------|------|---------| +| Carbonio | Aplicação MOSP | 017 | Modal Serviços | +| ligbox-sites | Tenant utilizador | 017 | Modal Serviços | +| Portal users | Tenant utilizador | 028 | Console `/admin/dominio` | +| Cloudflare | Infraestrutura | 037 | Console DNS Viewer | +| Traefik / SNI | Infraestrutura | 026 | Desk Infra CODE | +| Desk OPS | Aplicação MOSP | 017 | Eventos › Histórico purges | + +**Helper navegação:** `window.DeskNavigate.go(view, { eventsTab })` — `app.js` + +--- + +## 4. Blocklist purge + +| Domínio | Purge API | UI | +|---------|-----------|-----| +| `ligbox.com.br` | HTTP 400 | 🔒 badge + banner Escopo + modal desactivado | +| `itecnologys.com` | HTTP 400 | idem | + +Constante: `PURGE_BLOCKLIST` — `vm112_domains.py` + espelho `servicos.js`. + +--- + +## 5. Ficheiros principais + +| Ficheiro | Função | +|----------|--------| +| `frontend/assets/servicos.js` | Escopo OPS, loading modal, blocklist, cache SWR | +| `frontend/assets/styles.css` | `.vm112-load-*`, `.servicos-scope-*`, client row grid | +| `frontend/assets/app.js` | `DeskNavigate`, poll 90s | +| `frontend/assets/auth.js` | sessão resiliente rede | +| `frontend/index.html` | cache bust `20260625align1` | +| `api/app/vm112_domains.py` | cache proxy, `purge_blocked` | + +--- + +## 6. Deploy VM122 (referência) + +```bash +sshpass -p '805353' scp servicos.js styles.css app.js index.html root@10.10.10.122:/tmp/ +ssh root@10.10.10.122 ' + C=ligbox-ops-platform_frontend_1 + docker cp /tmp/servicos.js $C:/usr/share/nginx/html/assets/servicos.js + docker cp /tmp/styles.css $C:/usr/share/nginx/html/assets/styles.css + docker cp /tmp/app.js $C:/usr/share/nginx/html/assets/app.js + docker cp /tmp/index.html $C:/usr/share/nginx/html/index.html +' +``` + +**Hard refresh:** Ctrl+Shift+R em `https://desk.ligbox.com.br` + +--- + +## 7. Versão frontend (cache bust) + +| Asset | Query string | +|-------|----------------| +| `styles.css` | `?v=20260625align1` | +| `servicos.js` | `?v=20260625align1` | +| `app.js` | `?v=20260625align1` | + +--- + +## 8. Specs actualizadas + +| Spec | Secção | +|------|--------| +| 017 | Performance cache, UI loading/blocklist/Escopo | +| 018 | Performance, Escopo OPS cards, critérios aceite | +| 032 | UI blocklist | +| 026 | Destino card Traefik | +| 028 | Destino card Portal | +| 037 | Destino card Cloudflare | + +--- + +## 9. Canais de arquivo + +| Canal | Path | +|-------|------| +| Anais | `docs/anais-referencia/20260625_SERVICOS_IAAS_PERF_UX_ESCOPO.md` | +| Spec 017 | `specs/017-vm112-domain-orchestration/spec.md` | +| Spec 018 | `specs/018-service-orchestration/spec.md` | +| Patch VM112 | `deploy/vm112-wizard/perf-domains-list-20260625/` | diff --git a/projects/ops-desk/api/app/main.py b/projects/ops-desk/api/app/main.py index df48aab..1c3ef63 100644 --- a/projects/ops-desk/api/app/main.py +++ b/projects/ops-desk/api/app/main.py @@ -126,8 +126,19 @@ ASSIST_LIFECYCLE_EVENTS = frozenset({"onboarding.assist.started", "onboarding.as TICKET_ACTIVE_STATUSES = frozenset({"open", "escalated", "assisting", "resolved"}) +_cors_raw = os.getenv( + "CORS_ORIGINS", + "https://console.ligbox.com.br,https://desk.ligbox.com.br,http://localhost:5173,http://127.0.0.1:5173", +) +_cors_origins = [o.strip() for o in _cors_raw.split(",") if o.strip()] + app = FastAPI(title="Ligbox Ops Platform API", version="0.9.7-spec029-agentic") -app.add_middleware(CORSMiddleware, allow_origins=["*"], allow_methods=["*"], allow_headers=["*"]) +app.add_middleware( + CORSMiddleware, + allow_origins=_cors_origins or ["*"], + allow_methods=["*"], + allow_headers=["*"], +) app.include_router(auth_router) app.include_router(registration_router) app.include_router(mfa_recovery_router) @@ -1067,6 +1078,24 @@ def list_webhook_events( return {"events": [_enrich_event(r) for r in rows[:50]]} +@app.get("/api/v1/webhooks/events/{event_id}") +def get_webhook_event( + event_id: int, + user: auth.DeskUser = Depends(auth.get_current_user), +): + """Detalhe de um evento webhook — usado pelo Auditor de Eventos (Spec 017).""" + with db() as conn: + row = conn.execute( + "SELECT id, event_type, source, payload, created_at FROM webhook_events WHERE id = ?", + (event_id,), + ).fetchone() + if not row: + raise HTTPException(404, "Evento não encontrado") + if not can_list_webhook_events(user.role, row["source"]): + raise HTTPException(403, "insufficient permissions") + return {"event": _enrich_event(row)} + + @app.get("/api/v1/onboard/funnel") def onboard_funnel( window_hours: int = Query(default=48, ge=1, le=168), diff --git a/projects/ops-desk/api/app/vm112_domains.py b/projects/ops-desk/api/app/vm112_domains.py index 52b6e78..652614d 100644 --- a/projects/ops-desk/api/app/vm112_domains.py +++ b/projects/ops-desk/api/app/vm112_domains.py @@ -4,7 +4,9 @@ from __future__ import annotations import os import sqlite3 +import time from datetime import datetime, timezone +from threading import Lock from typing import Any import httpx @@ -13,6 +15,16 @@ from app import auth VM112_API = os.getenv("VM112_API_URL", "http://10.10.10.112:8090") VM112_ADMIN_API_KEY = os.getenv("VM112_ADMIN_API_KEY", "ibytera-corp-api-key-change-later") +# Purge Carbonio/CF/Traefik pode demorar vários minutos — evitar httpx "timed out" prematuro. +VM112_PURGE_HTTP_TIMEOUT = float(os.getenv("VM112_PURGE_HTTP_TIMEOUT", "300")) +VM112_LIST_TIMEOUT = float(os.getenv("VM112_LIST_TIMEOUT", "30")) +VM112_DOMAINS_CACHE_TTL = float(os.getenv("VM112_DOMAINS_CACHE_TTL", "60")) +VM112_DOMAIN_DETAIL_TTL = float(os.getenv("VM112_DOMAIN_DETAIL_TTL", "45")) + +_DOMAINS_CACHE: dict[str, Any] | None = None +_DOMAINS_CACHE_AT = 0.0 +_DOMAINS_CACHE_LOCK = Lock() +_DOMAIN_DETAIL_CACHE: dict[str, tuple[float, dict[str, Any]]] = {} PURGE_BLOCKLIST = frozenset({"ligbox.com.br", "itecnologys.com"}) @@ -29,6 +41,21 @@ VM112_PURGE_STEP_LABELS = ( "Logs de sessão wizard", ) +# Origem da solicitação de expurgo (user = JWT / sessão; source = canal) +PURGE_SOURCE_LABELS: dict[str, str] = { + "desk.ui.purge": "Desk › Purge domínio (UI)", + "desk.api.purge": "Desk › Purge domínio (API sync)", + "desk.api.purge-jobs": "Desk › Purge domínio (job async)", + "desk.api.purge-stream": "Desk › Purge domínio (SSE)", + "cursor.agent": "Cursor Agent (comando chat)", + "domain-console.sandbox": "Domain Console › sandbox", + "desk.agentic": "Desk › rotina agentic", +} + + +def purge_source_label(source: str) -> str: + return PURGE_SOURCE_LABELS.get(source, source or "desconhecido") + def _ts() -> str: return datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ") @@ -80,26 +107,46 @@ def delete_carbonio_account(email: str) -> dict[str, Any]: } -def list_domains(query: str = "") -> dict[str, Any]: - with httpx.Client(timeout=60.0) as client: +def list_domains(query: str = "", *, force_refresh: bool = False) -> dict[str, Any]: + """Lista domínios VM112 — cache TTL curto (Serviços IaaS poll frequente).""" + global _DOMAINS_CACHE, _DOMAINS_CACHE_AT + if not query and not force_refresh: + with _DOMAINS_CACHE_LOCK: + if _DOMAINS_CACHE is not None and time.time() - _DOMAINS_CACHE_AT < VM112_DOMAINS_CACHE_TTL: + return {**_DOMAINS_CACHE, "cached": True, "cache_age_sec": int(time.time() - _DOMAINS_CACHE_AT)} + + with httpx.Client(timeout=VM112_LIST_TIMEOUT) as client: r = client.get( f"{VM112_API}/api/admin/domains", params={"q": query} if query else None, headers=_vm112_headers(), ) r.raise_for_status() - return r.json() + data = r.json() + + if not query: + with _DOMAINS_CACHE_LOCK: + _DOMAINS_CACHE = data + _DOMAINS_CACHE_AT = time.time() + return data def get_domain(domain: str) -> dict[str, Any]: domain = domain.lower().strip() - with httpx.Client(timeout=180.0) as client: - r = client.get( - f"{VM112_API}/api/admin/domains/{domain}", - headers=_vm112_headers(), - ) - r.raise_for_status() - data = r.json() + now = time.time() + cached = _DOMAIN_DETAIL_CACHE.get(domain) + if cached and now - cached[0] < VM112_DOMAIN_DETAIL_TTL: + data = dict(cached[1]) + else: + with httpx.Client(timeout=VM112_LIST_TIMEOUT) as client: + r = client.get( + f"{VM112_API}/api/admin/domains/{domain}", + headers=_vm112_headers(), + ) + r.raise_for_status() + data = r.json() + if isinstance(data, dict): + _DOMAIN_DETAIL_CACHE[domain] = (now, data) if isinstance(data, dict): data["purge_extra_auth_required"] = requires_purge_extra_auth(domain) data["purge_blocked"] = domain in PURGE_BLOCKLIST @@ -129,7 +176,7 @@ def domain_exists_on_vm112(domain: str) -> bool: def start_purge_vm112(domain: str) -> dict[str, Any]: """Inicia purge assíncrono na VM112 (Spec 017 Fase 3).""" domain = domain.lower().strip() - with httpx.Client(timeout=60.0) as client: + with httpx.Client(timeout=VM112_PURGE_HTTP_TIMEOUT) as client: r = client.post( f"{VM112_API}/api/admin/domains/{domain}/purge", headers=_vm112_headers(), @@ -139,7 +186,7 @@ def start_purge_vm112(domain: str) -> dict[str, Any]: def poll_purge_vm112_job(job_id: str) -> dict[str, Any]: - with httpx.Client(timeout=60.0) as client: + with httpx.Client(timeout=VM112_PURGE_HTTP_TIMEOUT) as client: r = client.get( f"{VM112_API}/api/admin/domains/purge-jobs/{job_id}", headers=_vm112_headers(), @@ -183,8 +230,37 @@ def purge_vm112_with_poll(domain: str, poll_interval: float = 1.5, timeout: floa t0 = time.monotonic() deadline = t0 + timeout seen = 0 + poll_errors = 0 while time.monotonic() < deadline: - job = poll_purge_vm112_job(job_id) + try: + job = poll_purge_vm112_job(job_id) + poll_errors = 0 + except Exception as exc: + poll_errors += 1 + err = str(exc) or "erro poll VM112" + yield ( + "heartbeat", + { + "elapsed": int(time.monotonic() - t0), + "job_id": job_id, + "warn": err, + "poll_errors": poll_errors, + }, + ) + if poll_errors >= 5 and not domain_exists_on_vm112(domain): + yield ( + "final", + { + "ok": True, + "job_id": job_id, + "recovered": True, + "steps": [], + "result": {"message": "Domínio ausente na VM112 após falhas de poll"}, + }, + ) + return + time.sleep(poll_interval) + continue steps = vm112_job_steps_timeline(job) if len(steps) > seen: for step in steps[seen:]: @@ -217,7 +293,26 @@ def purge_vm112_with_poll(domain: str, poll_interval: float = 1.5, timeout: floa yield ("heartbeat", {"elapsed": int(time.monotonic() - t0), "job_id": job_id}) time.sleep(poll_interval) - yield ("final", {"ok": False, "error": "Timeout purge VM112", "job_id": job_id}) + if not domain_exists_on_vm112(domain): + yield ( + "final", + { + "ok": True, + "job_id": job_id, + "recovered": True, + "steps": [], + "result": {"message": "Domínio removido na VM112 (timeout poll — recuperado)"}, + }, + ) + return + yield ( + "final", + { + "ok": False, + "error": "Timeout purge VM112 — domínio ainda presente; use Recuperar ou repita", + "job_id": job_id, + }, + ) def purge_vm112(domain: str) -> dict[str, Any]: @@ -255,46 +350,114 @@ def vm112_purge_timeline(vm112_result: dict[str, Any]) -> list[dict[str, str]]: def purge_desk_records(conn: sqlite3.Connection, domain: str) -> dict[str, int]: - domain = domain.lower().strip() - like = f"%{domain}%" - counts = {} - counts["webhook_events"] = conn.execute( - "DELETE FROM webhook_events WHERE payload LIKE ?", (like,) - ).rowcount - counts["tickets"] = conn.execute( - "DELETE FROM tickets WHERE subject LIKE ? OR payload LIKE ?", (like, like) - ).rowcount - counts["audit_domains"] = conn.execute( - "DELETE FROM audit_domains WHERE domain = ?", (domain,) - ).rowcount - counts["assist_sessions"] = conn.execute( - "DELETE FROM assist_sessions WHERE domain = ?", (domain,) - ).rowcount - counts["audit_checks"] = conn.execute( - "DELETE FROM audit_checks WHERE domain = ?", (domain,) - ).rowcount - conn.commit() + counts, _ = purge_desk_timeline( + conn, domain, by_user="system", purge_source="desk.api.purge", record_event=False + ) return counts -def purge_desk_timeline(conn: sqlite3.Connection, domain: str) -> tuple[dict[str, int], list[dict[str, str]]]: - """Purge Desk com uma linha de timeline por tabela.""" +def purge_domain_console_scenario(conn: sqlite3.Connection, domain: str) -> int: + domain = domain.lower().strip() + return conn.execute( + "DELETE FROM domain_console_scenarios WHERE domain = ?", (domain,) + ).rowcount + + +def record_domain_purged_event( + conn: sqlite3.Connection, + domain: str, + *, + by_user: str, + purge_source: str, + desk_removed: dict[str, int] | None = None, + vm112_ok: bool | None = None, + vm112_error: str | None = None, + job_id: str | None = None, +) -> int: + """Marca expurgo em eventos — histórico preservado (não apaga webhook_events).""" + import json + + now = datetime.now(timezone.utc).isoformat() + payload = json.dumps( + { + "domain": domain.lower().strip(), + "event": "domain.purged", + "purged_at": now, + "by_user": by_user, + "purge_source": purge_source, + "purge_tool": purge_source_label(purge_source), + "message": ( + f"Domínio expurgado em {now} pelo utilizador {by_user} " + f"via {purge_source_label(purge_source)}" + ), + "desk_removed": desk_removed or {}, + "vm112_ok": vm112_ok, + "vm112_error": vm112_error, + "job_id": job_id, + "accounts_hint": [f"admin@{domain.lower().strip()}", f"mail.{domain.lower().strip()}"], + }, + ensure_ascii=False, + ) + cur = conn.execute( + "INSERT INTO webhook_events (event_type, source, payload, created_at) VALUES (?,?,?,?)", + ("domain.purged", "desk.purge", payload, now), + ) + conn.commit() + return int(cur.lastrowid) + + +def purge_desk_timeline( + conn: sqlite3.Connection, + domain: str, + *, + by_user: str = "system", + purge_source: str = "desk.api.purge", + record_event: bool = True, + vm112_ok: bool | None = None, + vm112_error: str | None = None, + job_id: str | None = None, +) -> tuple[dict[str, int], list[dict[str, str]]]: + """Purge Desk — mantém webhook_events; regista domain.purged no fim.""" domain = domain.lower().strip() like = f"%{domain}%" timeline: list[dict[str, str]] = [] counts: dict[str, int] = {} desk_steps = ( - ("Desk — webhook_events", "webhook_events", "DELETE FROM webhook_events WHERE payload LIKE ?", (like,)), + ("Desk — domain_console_scenarios", "domain_console_scenarios", None, None), ("Desk — tickets", "tickets", "DELETE FROM tickets WHERE subject LIKE ? OR payload LIKE ?", (like, like)), ("Desk — audit_domains", "audit_domains", "DELETE FROM audit_domains WHERE domain = ?", (domain,)), ("Desk — assist_sessions", "assist_sessions", "DELETE FROM assist_sessions WHERE domain = ?", (domain,)), ("Desk — audit_checks", "audit_checks", "DELETE FROM audit_checks WHERE domain = ?", (domain,)), ) for label, key, sql, params in desk_steps: - n = conn.execute(sql, params).rowcount + if key == "domain_console_scenarios": + n = purge_domain_console_scenario(conn, domain) + else: + n = conn.execute(sql, params).rowcount counts[key] = n timeline.append(_timeline_entry(label, "ok", f"{n} registo(s) removido(s)")) + + event_id = 0 + if record_event: + event_id = record_domain_purged_event( + conn, + domain, + by_user=by_user, + purge_source=purge_source, + desk_removed=counts, + vm112_ok=vm112_ok, + vm112_error=vm112_error, + job_id=job_id, + ) + counts["webhook_events_purged_marker"] = 1 + timeline.append( + _timeline_entry( + "Desk — evento domain.purged", + "ok", + f"event_id={event_id} · histórico webhook preservado", + ) + ) conn.commit() return counts, timeline diff --git a/projects/ops-desk/api/app/vm112_domains_routes.py b/projects/ops-desk/api/app/vm112_domains_routes.py index abfa2a8..f12f29e 100644 --- a/projects/ops-desk/api/app/vm112_domains_routes.py +++ b/projects/ops-desk/api/app/vm112_domains_routes.py @@ -26,6 +26,11 @@ class DomainPurgeRequest(BaseModel): confirm_domain: str = Field(..., min_length=3) root_password: str = Field(..., min_length=1) purge_auth_code: str = Field("", max_length=32) + purge_source: str = Field( + "desk.api.purge", + max_length=64, + description="Canal: desk.ui.purge, cursor.agent, domain-console.sandbox, desk.agentic, …", + ) def _require_admin(user: auth.DeskUser = Depends(auth.get_current_user)) -> auth.DeskUser: @@ -108,7 +113,14 @@ def purge_vm112_domain( raise HTTPException(502, f"Purge VM112 falhou: {e}") from e conn = auth.db() try: - desk_counts, desk_timeline = vm112_domains.purge_desk_timeline(conn, domain) + desk_counts, desk_timeline = vm112_domains.purge_desk_timeline( + conn, + domain, + by_user=user.username, + purge_source=body.purge_source or "desk.api.purge", + vm112_ok=vm112_result.get("ok"), + vm112_error=vm112_result.get("error"), + ) finally: conn.close() timeline = vm112_domains.build_purge_timeline(vm112_result, desk_counts, desk_timeline) @@ -119,6 +131,8 @@ def purge_vm112_domain( "desk": desk_counts, "timeline": timeline, "by": user.username, + "purge_source": body.purge_source, + "purge_tool": vm112_domains.purge_source_label(body.purge_source), } @@ -150,7 +164,9 @@ def start_purge_job( """Inicia purge em background; consultar GET /purge/jobs/{id} (recomendado via Traefik).""" domain = _validate_purge_request(domain, body, username=user.username) try: - job_id = start_job(domain, body.root_password, user.username) + job_id = start_job( + domain, body.root_password, user.username, purge_source=body.purge_source + ) except PurgeJobConflictError as exc: raise HTTPException(409, str(exc)) from exc except sqlite3.OperationalError as exc: diff --git a/projects/ops-desk/api/app/vm112_purge_jobs.py b/projects/ops-desk/api/app/vm112_purge_jobs.py index 6235fa6..cf62cc9 100644 --- a/projects/ops-desk/api/app/vm112_purge_jobs.py +++ b/projects/ops-desk/api/app/vm112_purge_jobs.py @@ -292,11 +292,17 @@ def create_job(domain: str, username: str) -> str: return job_id -def start_job(domain: str, root_password: str, username: str) -> str: +def start_job( + domain: str, + root_password: str, + username: str, + purge_source: str = "desk.api.purge-jobs", +) -> str: job_id = create_job(domain, username) + _set_job(job_id, desk={"purge_source": purge_source}) thread = threading.Thread( target=_execute_job, - args=(job_id, domain, root_password, username), + args=(job_id, domain, root_password, username, purge_source), daemon=True, ) thread.start() @@ -320,9 +326,20 @@ def _finish_desk_phase(job_id: str) -> dict[str, Any] | None: return _load_job(job_id) domain = job["domain"] + purge_source = (job.get("desk") or {}).get("purge_source") or "desk.api.purge-jobs" + vm112_ok = (job.get("vm112") or {}).get("ok") + vm112_error = job.get("error") conn = auth.db() try: - desk_counts, desk_timeline = vm112_domains.purge_desk_timeline(conn, domain) + desk_counts, desk_timeline = vm112_domains.purge_desk_timeline( + conn, + domain, + by_user=job.get("by") or "system", + purge_source=purge_source, + vm112_ok=vm112_ok, + vm112_error=vm112_error, + job_id=job_id, + ) finally: conn.close() @@ -339,7 +356,7 @@ def _finish_desk_phase(job_id: str) -> dict[str, Any] | None: def recover_job(job_id: str, domain: str | None = None) -> dict[str, Any] | None: - """Finaliza job quando VM112 já removeu o domínio (ex.: API reiniciada).""" + """Finaliza job quando VM112 já removeu o domínio (ex.: timeout poll ou API reiniciada).""" job = _load_job(job_id) if not job: if not domain: @@ -375,7 +392,7 @@ def recover_job(job_id: str, domain: str | None = None) -> dict[str, Any] | None "by": None, } - if job["status"] in ("done", "error"): + if job["status"] == "done": return job domain = (domain or job["domain"]).lower().strip() @@ -390,10 +407,25 @@ def recover_job(job_id: str, domain: str | None = None) -> dict[str, Any] | None "Domínio já removido na VM112 (recuperação)", ), ) - return _finish_desk_phase(job_id) + finished = _finish_desk_phase(job_id) + if finished: + _set_job( + job_id, + status="done", + error=None, + vm112={**(finished.get("vm112") or {}), "ok": True, "recovered": True}, + ) + return _load_job(job_id) + return finished -def _execute_job(job_id: str, domain: str, root_password: str, username: str) -> None: +def _execute_job( + job_id: str, + domain: str, + root_password: str, + username: str, + purge_source: str = "desk.api.purge-jobs", +) -> None: domain = domain.lower().strip() try: _set_job(job_id, status="running") @@ -438,19 +470,44 @@ def _execute_job(job_id: str, domain: str, root_password: str, username: str) -> break if not vm112_result.get("ok", False): - step = vm112_domains._timeline_entry( - "Purge VM112", - "fail", - str(vm112_result.get("error") or "falhou"), - ) + err = str(vm112_result.get("error") or "falhou") + if not vm112_domains.domain_exists_on_vm112(domain): + _upsert_step( + job_id, + vm112_domains._timeline_entry( + "Purge VM112", + "ok", + "Domínio removido na VM112 (recuperação automática)", + ), + ) + _set_job( + job_id, + vm112={**vm112_result, "ok": True, "recovered": True, "previous_error": err}, + error=None, + ) + _finish_desk_phase(job_id) + return + step = vm112_domains._timeline_entry("Purge VM112", "fail", err) _upsert_step(job_id, step) - _set_job(job_id, status="error", error=str(vm112_result.get("error") or "falhou")) + _set_job(job_id, status="error", error=err, vm112=vm112_result) return _set_job(job_id, vm112=vm112_result) _finish_desk_phase(job_id) except Exception as exc: err = str(exc) or "erro inesperado" + if not vm112_domains.domain_exists_on_vm112(domain): + _upsert_step( + job_id, + vm112_domains._timeline_entry( + "Purge VM112", + "ok", + f"Recuperação após excepção: {err}", + ), + ) + _set_job(job_id, vm112={"ok": True, "recovered": True, "previous_error": err}, error=None) + _finish_desk_phase(job_id) + return _upsert_step( job_id, vm112_domains._timeline_entry("Purge VM112", "fail", err), diff --git a/projects/ops-desk/api/app/vm112_purge_stream.py b/projects/ops-desk/api/app/vm112_purge_stream.py index 4bca5b6..a99cc98 100644 --- a/projects/ops-desk/api/app/vm112_purge_stream.py +++ b/projects/ops-desk/api/app/vm112_purge_stream.py @@ -65,31 +65,21 @@ def purge_sse_generator(domain: str, root_password: str, username: str) -> Itera break conn = auth.db() - desk_counts: dict[str, int] = {} try: - domain_l = domain.lower().strip() - like = f"%{domain_l}%" - desk_steps = ( - ("Desk — webhook_events", "webhook_events", "DELETE FROM webhook_events WHERE payload LIKE ?", (like,)), - ("Desk — tickets", "tickets", "DELETE FROM tickets WHERE subject LIKE ? OR payload LIKE ?", (like, like)), - ("Desk — audit_domains", "audit_domains", "DELETE FROM audit_domains WHERE domain = ?", (domain_l,)), - ("Desk — assist_sessions", "assist_sessions", "DELETE FROM assist_sessions WHERE domain = ?", (domain_l,)), - ("Desk — audit_checks", "audit_checks", "DELETE FROM audit_checks WHERE domain = ?", (domain_l,)), + desk_counts, desk_timeline = vm112_domains.purge_desk_timeline( + conn, + domain, + by_user=username, + purge_source="desk.api.purge-stream", + vm112_ok=vm112_result.get("ok"), + vm112_error=vm112_result.get("error"), ) - for label, key, sql, params in desk_steps: - yield _sse({"type": "step", "step": vm112_domains._timeline_entry(label, "running")}) - n = conn.execute(sql, params).rowcount - desk_counts[key] = n - yield _sse({ - "type": "step", - "step": vm112_domains._timeline_entry(label, "ok", f"{n} registo(s) removido(s)"), - "phase": "desk", - }) - conn.commit() finally: conn.close() total_desk = sum(desk_counts.values()) + for step in desk_timeline: + yield _sse({"type": "step", "step": step, "phase": "desk"}) done_step = vm112_domains._timeline_entry("Purge concluído", "ok", f"Desk: {total_desk} registo(s)") yield _sse({ "type": "done", diff --git a/projects/ops-desk/frontend/assets/app.js b/projects/ops-desk/frontend/assets/app.js index 892feeb..bf210c6 100644 --- a/projects/ops-desk/frontend/assets/app.js +++ b/projects/ops-desk/frontend/assets/app.js @@ -276,11 +276,35 @@ function setView(name) { refresh(); } +/** Navegação cross-módulo (Spec 018 — cards Escopo OPS). */ +window.DeskNavigate = { + go(view, opts = {}) { + if (opts.eventsTab) state.eventsTab = opts.eventsTab; + if (opts.ticketFilter) state.ticketFilter = opts.ticketFilter; + setView(view); + if (view === 'events' && opts.eventsTab) { + document.querySelectorAll('[data-events-tab]').forEach((b) => { + b.classList.toggle('active', b.dataset.eventsTab === opts.eventsTab); + }); + renderEvents(); + } + if (view === 'tickets' && opts.ticketFilter) { + document.querySelectorAll('.filter-btn[data-filter]').forEach((b) => { + b.classList.toggle('active', b.dataset.filter === opts.ticketFilter); + }); + renderTickets(); + } + }, +}; + let pollTimer = null; let socRenderInFlight = false; function reschedulePoll() { if (pollTimer) clearInterval(pollTimer); - const ms = state.view === 'infra2' ? 15000 : 30000; + let ms = 30000; + if (state.view === 'infra2') ms = 15000; + // VM112 /admin/domains ~10–15s — poll menos agressivo (Spec 018 Serviços IaaS) + if (state.view === 'overview-home') ms = 90000; pollTimer = setInterval(() => refresh({ poll: true }), ms); } @@ -2174,12 +2198,14 @@ async function renderEvents(options = {}) { const rows = (data.events || []).map((e) => { const p = e.payload || {}; const dataObj = p.data || {}; - return ` + const domain = p.domain || e.domain || '—'; + const ref = (p.session_id || p.job_id || '').slice(0, 16); + return ` ${e.id} ${sourceBadge(e.source)} ${esc(e.event_type)} ${severityBadge(dataObj.level || e.severity)} - ${esc(p.domain || '—')} - ${esc((p.session_id || '').slice(0, 16))} + ${esc(domain)} + ${esc(ref || '—')} ${fmtDate(e.created_at)} `; }).join(''); @@ -2190,6 +2216,13 @@ async function renderEvents(options = {}) { ${rows || 'Sem eventos'} `; + el.querySelectorAll('.event-auditor-row').forEach((row) => { + const open = () => openEventAuditorModal(Number(row.dataset.eventId)); + row.addEventListener('click', open); + row.addEventListener('keydown', (ev) => { + if (ev.key === 'Enter' || ev.key === ' ') { ev.preventDefault(); open(); } + }); + }); } catch (e) { el.innerHTML = `

Erro: ${esc(e.message)}

`; } finally { @@ -2501,6 +2534,126 @@ function renderPurgeTimelineHtml(steps) { }).join('')}`; } +const PURGE_DESK_LABELS = { + webhook_events_purged_marker: 'Marcador domain.purged (histórico preservado)', + domain_console_scenarios: 'Cenários Domain Console', + tickets: 'Tickets', + audit_domains: 'Domínios audit', + assist_sessions: 'Sessões assist', + audit_checks: 'Checks audit', +}; + +function purgeDeskRowsHtml(desk) { + const keys = Object.keys(PURGE_DESK_LABELS); + const extra = Object.keys(desk || {}).filter((k) => !keys.includes(k) && k !== 'purge_source'); + const allKeys = [...keys, ...extra]; + const rows = allKeys.map((key) => { + const n = Number(desk?.[key] || 0); + if (!n && !keys.includes(key)) return ''; + const label = PURGE_DESK_LABELS[key] || key; + return `${esc(label)}${n}`; + }).filter(Boolean).join(''); + const total = Object.entries(desk || {}) + .filter(([k]) => k !== 'purge_source') + .reduce((a, [, v]) => a + Number(v || 0), 0); + return { rows, total }; +} + +function closeEventAuditorModal() { + const modal = document.getElementById('event-auditor-modal'); + if (!modal) return; + modal.classList.add('hidden'); + modal.setAttribute('aria-hidden', 'true'); +} + +function renderDomainPurgedAuditor(ev) { + const p = ev.payload || {}; + const desk = p.desk_removed || {}; + const deskRows = Object.entries(desk).map(([k, v]) => + `${esc(PURGE_DESK_LABELS[k] || k)}${Number(v)}`, + ).join(''); + const jobLink = p.job_id + ? `` + : ''; + return ` +
+

Expurgo de domínio

+
+
Domínio
${esc(p.domain || ev.domain || '—')}
+
Utilizador
${esc(p.by_user || '—')}
+
Ferramenta
${esc(p.purge_tool || p.purge_source || '—')}
+
Expurgado em
${fmtDate(p.purged_at || ev.created_at)}
+
VM112
${p.vm112_ok === false ? `falhou ${esc(p.vm112_error || '')}` : 'OK'}
+ ${jobLink ? `
Job purge
${jobLink}
` : ''} +
+ ${p.message ? `

${esc(p.message)}

` : ''} +
+ ${deskRows ? `

Removido no Desk

${deskRows}
` : ''} +

Payload completo

+
${esc(JSON.stringify(p, null, 2))}
`; +} + +function openEventAuditorModal(eventId) { + const modal = document.getElementById('event-auditor-modal'); + const title = document.getElementById('event-auditor-modal-title'); + const sub = document.getElementById('event-auditor-modal-sub'); + const body = document.getElementById('event-auditor-modal-body'); + if (!modal || !body) return; + modal.classList.remove('hidden'); + modal.setAttribute('aria-hidden', 'false'); + title.textContent = 'Auditor de Eventos'; + sub.textContent = `webhook · ${eventId}`; + body.innerHTML = '

Carregando…

'; + api(`/v1/webhooks/events/${encodeURIComponent(eventId)}`) + .then(async (data) => { + const ev = data.event || {}; + sub.textContent = `${esc(ev.source || 'webhook')} · ${ev.id}`; + if (ev.event_type === 'domain.purged') { + body.innerHTML = renderDomainPurgedAuditor(ev); + body.querySelector('[data-open-purge-job]')?.addEventListener('click', (btn) => { + closeEventAuditorModal(); + openPurgeHistoryModal(btn.currentTarget.dataset.openPurgeJob); + }); + return; + } + const p = ev.payload || {}; + const sid = (p.session_id || ev.session_id || '').trim(); + let timelineBlock = ''; + if (sid && ev.source === 'vm112-onboard') { + try { + const tl = await api(`/v1/onboard/sessions/${encodeURIComponent(sid)}/timeline`); + const events = tl.events || tl.timeline || []; + if (events.length) { + timelineBlock = ` +

Timeline onboard (${esc(sid.slice(0, 16))}…)

+ ${timelineHtml(events, null, { compact: true })}`; + } + } catch { + /* timeline opcional */ + } + } + body.innerHTML = ` +
+
+
Tipo
${esc(ev.event_type)}
+
Origem
${sourceBadge(ev.source)}
+
Domínio
${esc(p.domain || ev.domain || '—')}
+
Sessão
${esc(sid || '—')}
+
Data
${fmtDate(ev.created_at)}
+
+
+ ${timelineBlock} +

Payload

+
${esc(JSON.stringify(p, null, 2))}
`; + }) + .catch((e) => { + body.innerHTML = `

Erro: ${esc(e.message)}

`; + }); + document.querySelectorAll('[data-close-event-auditor-modal]').forEach((el) => { + el.onclick = closeEventAuditorModal; + }); +} + function closePurgeHistoryModal() { const modal = document.getElementById('purge-history-modal'); if (!modal) return; @@ -2525,23 +2678,19 @@ function openPurgeHistoryModal(jobId) { sub.innerHTML = `${purgeStatusBadge(job.status)} · ${esc(job.by || '—')} · ${fmtDate(job.created_at)} · job ${esc(job.id)}`; const desk = job.desk || {}; const vm112 = job.vm112 || {}; - const deskRows = Object.entries({ - webhook_events: 'Eventos webhook', - tickets: 'Tickets', - audit_domains: 'Domínios audit', - assist_sessions: 'Sessões assist', - audit_checks: 'Checks audit', - }).map(([key, label]) => ` - ${esc(label)}${Number(desk[key] || 0)}`).join(''); + const { rows: deskRows, total: deskTotal } = purgeDeskRowsHtml(desk); const vm112Steps = Array.isArray(vm112.steps) ? vm112.steps : []; const timeline = (job.timeline || []).length ? job.timeline : vm112Steps; + const recoverBtn = job.status === 'error' + ? `` + : ''; body.innerHTML = `

Removido no Desk (VM122)

- ${deskRows} - + ${deskRows || ''} +
Total${Object.values(desk).reduce((a, b) => a + Number(b || 0), 0)}
Nenhum registo Desk removido
Total${deskTotal}
@@ -2549,12 +2698,28 @@ function openPurgeHistoryModal(jobId) {

${vm112RemovedSummary(vm112)}

${job.elapsed_vm112 ? `

Duração VM112: ${job.elapsed_vm112}s

` : ''} ${job.error ? `

${esc(job.error)}

` : ''} + ${recoverBtn}

Timeline completa

${timeline.length ? renderPurgeTimelineHtml(timeline) : '

Sem passos registados

'}
`; + body.querySelector('#purge-history-recover-btn')?.addEventListener('click', async () => { + const btn = body.querySelector('#purge-history-recover-btn'); + if (btn) { btn.disabled = true; btn.textContent = 'A recuperar…'; } + try { + const recovered = await api( + `/v1/vm112/purge/jobs/${encodeURIComponent(jobId)}/recover`, + { method: 'POST' }, + ); + openPurgeHistoryModal(recovered.id || jobId); + await renderPurgeHistory(); + } catch (e) { + if (btn) { btn.disabled = false; btn.textContent = 'Recuperar purge (VM112 já limpa?)'; } + alert(e.message || 'Recuperação falhou'); + } + }); }) .catch((e) => { body.innerHTML = `

Erro: ${esc(e.message)}

`; @@ -4330,7 +4495,7 @@ async function renderPurgeAuthInfraPanel() { async function refresh(options = {}) { const { poll = false } = options; - await loadHealth(); + void loadHealth(); if (poll && state.view === 'account') { return; } @@ -4399,6 +4564,10 @@ document.querySelectorAll('[data-close-purge-history-modal]').forEach((el) => { el.addEventListener('click', closePurgeHistoryModal); }); +document.querySelectorAll('[data-close-event-auditor-modal]').forEach((el) => { + el.addEventListener('click', closeEventAuditorModal); +}); + document.getElementById('btn-refresh')?.addEventListener('click', () => { if (state.view === 'account') { state.accountLoaded = false; @@ -4406,6 +4575,19 @@ document.getElementById('btn-refresh')?.addEventListener('click', () => { refresh(); }); +document.addEventListener('click', (ev) => { + const btn = ev.target.closest('.js-console-dns'); + if (!btn || typeof openConsoleDnsViewer !== 'function') return; + ev.preventDefault(); + openConsoleDnsViewer(btn.dataset.domain || ''); +}); + +document.getElementById('nav-console')?.addEventListener('click', (ev) => { + if (typeof openConsolePath !== 'function') return; + ev.preventDefault(); + openConsolePath('/admin/dominio'); +}); + (async function boot() { const dash = document.getElementById('dashboard-content'); try { diff --git a/projects/ops-desk/frontend/assets/auth.js b/projects/ops-desk/frontend/assets/auth.js index 5a89084..c81c42f 100644 --- a/projects/ops-desk/frontend/assets/auth.js +++ b/projects/ops-desk/frontend/assets/auth.js @@ -1,5 +1,22 @@ const AUTH_TOKEN_KEY = 'ligbox_ops_token'; const AUTH_USER_KEY = 'ligbox_ops_user'; +const SSO_COOKIE = 'ligbox_sso_token'; +const SSO_MAX_AGE = 8 * 3600; + +function readCookie(name) { + const re = new RegExp(`(?:^|; )${name.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\$&')}=([^;]*)`); + const m = document.cookie.match(re); + return m ? decodeURIComponent(m[1]) : null; +} + +function writeSsoCookie(token, maxAgeSec = SSO_MAX_AGE) { + if (!token) return; + document.cookie = `${SSO_COOKIE}=${encodeURIComponent(token)}; domain=.ligbox.com.br; path=/; max-age=${maxAgeSec}; secure; samesite=lax`; +} + +function clearSsoCookie() { + document.cookie = `${SSO_COOKIE}=; domain=.ligbox.com.br; path=/; max-age=0; secure; samesite=lax`; +} function getToken() { return sessionStorage.getItem(AUTH_TOKEN_KEY); @@ -13,14 +30,16 @@ function getUser() { } } -function setSession(token, user) { +function setSession(token, user, opts = {}) { sessionStorage.setItem(AUTH_TOKEN_KEY, token); sessionStorage.setItem(AUTH_USER_KEY, JSON.stringify(user)); + writeSsoCookie(token, opts.maxAgeSec ?? SSO_MAX_AGE); } function clearSession() { sessionStorage.removeItem(AUTH_TOKEN_KEY); sessionStorage.removeItem(AUTH_USER_KEY); + clearSsoCookie(); } function isLoggedIn() { @@ -68,9 +87,9 @@ async function ensureValidSession() { }); return true; } catch (err) { - console.warn('ensureValidSession:', err?.name || err); - clearSession(); - return false; + // API reiniciando / rede — não expulsar sessão (evita flash login em Serviços IaaS) + console.warn('ensureValidSession (rede, sessão mantida):', err?.name || err); + return true; } } @@ -125,11 +144,15 @@ async function login(username, password) { if (data.mfa_required) { return { mfaRequired: true, mfaToken: data.mfa_token, username: data.username }; } - setSession(data.access_token, { - username: data.username, - role: data.role, - display_name: data.display_name, - }); + setSession( + data.access_token, + { + username: data.username, + role: data.role, + display_name: data.display_name, + }, + { maxAgeSec: data.expires_in || SSO_MAX_AGE }, + ); return data; } @@ -146,11 +169,15 @@ async function loginMfa(mfaToken, totpCode, backupCode) { if (!res.ok) { throw new Error(data.detail || `Código 2FA inválido (${res.status})`); } - setSession(data.access_token, { - username: data.username, - role: data.role, - display_name: data.display_name, - }); + setSession( + data.access_token, + { + username: data.username, + role: data.role, + display_name: data.display_name, + }, + { maxAgeSec: data.expires_in || SSO_MAX_AGE }, + ); return data; } @@ -185,11 +212,47 @@ async function mfaRecoveryComplete(recoveryToken, totpCode) { const data = await res.json().catch(() => ({})); if (!res.ok) throw new Error(data.detail || `Recuperação falhou (${res.status})`); if (data.access_token) { - setSession(data.access_token, { - username: data.username, - role: data.role, - display_name: data.display_name, - }); + setSession( + data.access_token, + { + username: data.username, + role: data.role, + display_name: data.display_name, + }, + { maxAgeSec: data.expires_in || SSO_MAX_AGE }, + ); } return data; } + +function canReadDnsViewer() { + return hasRole('super_admin', 'ops_lead', 'technician', 'noc', 'seo', 'devops', 'developer'); +} + +/** Desk → Console (handoff + cookie SSO .ligbox.com.br). */ +async function openConsolePath(path, { domain } = {}) { + const url = new URL(path, 'https://console.ligbox.com.br'); + const dom = (domain || '').trim().toLowerCase(); + if (dom) url.searchParams.set('domain', dom); + if (!isLoggedIn()) { + window.open(url.toString(), '_blank', 'noopener'); + return; + } + try { + const res = await fetchWithTimeout('/api/v1/auth/console-handoff', { + method: 'POST', + headers: authHeaders(), + }); + const data = await res.json().catch(() => ({})); + if (res.ok && data.handoff_token) { + url.searchParams.set('desk_handoff', data.handoff_token); + } + } catch (err) { + console.warn('openConsolePath handoff:', err?.name || err); + } + window.open(url.toString(), '_blank', 'noopener'); +} + +async function openConsoleDnsViewer(domain) { + return openConsolePath('/admin/dominio', { domain }); +} diff --git a/projects/ops-desk/frontend/assets/servicos.js b/projects/ops-desk/frontend/assets/servicos.js index aeb42b7..212cf61 100644 --- a/projects/ops-desk/frontend/assets/servicos.js +++ b/projects/ops-desk/frontend/assets/servicos.js @@ -10,14 +10,88 @@ const DeskServices = (() => { let _selectedClientId = null; let _selectedServiceId = 'email_tenant'; let _billingByDomain = {}; + let _loadInFlight = null; + + const DOMAINS_CACHE_KEY = 'desk_servicos_domains_v1'; + const DOMAINS_CACHE_TTL_MS = 5 * 60 * 1000; + const DOMAINS_FETCH_TIMEOUT_MS = 35000; + const DOMAINS_POLL_TIMEOUT_MS = 22000; + + /** Domínios onde purge é proibido (Spec 017 — espelha API VM122). */ + const PURGE_BLOCKLIST = new Set(['ligbox.com.br', 'itecnologys.com']); + + const LAYER_LABELS = { + tenant: 'Tenant utilizador', + app: 'Aplicação MOSP', + infra: 'Infraestrutura', + }; const OPS_SCOPES = [ - { id: 'carbonio', label: 'Carbonio', desc: 'Domínio e contas de e-mail (zmprov)' }, - { id: 'site', label: 'ligbox-sites', desc: 'Pasta do tenant em /opt/ligbox-sites/domains/' }, - { id: 'portal', label: 'Portal users', desc: 'Contas Self-Service ligadas ao domínio' }, - { id: 'cloudflare', label: 'Cloudflare', desc: 'Zona DNS na conta Ibytera' }, - { id: 'traefik', label: 'Traefik / SNI', desc: 'Routers mail.* no CT114' }, - { id: 'desk', label: 'Desk OPS', desc: 'Webhooks, tickets e audit_domains' }, + { + id: 'carbonio', + label: 'Carbonio', + desc: 'Domínio e contas de e-mail (zmprov)', + layer: 'app', + spec: '017', + specTitle: 'Purge domínio VM112', + action: 'modal', + goto: 'Modal Serviços — contas Carbonio', + }, + { + id: 'site', + label: 'ligbox-sites', + desc: 'Pasta do tenant em /opt/ligbox-sites/domains/', + layer: 'tenant', + spec: '017', + specTitle: 'Purge domínio VM112', + action: 'modal', + goto: 'Modal Serviços — pasta site', + }, + { + id: 'portal', + label: 'Portal users', + desc: 'Contas Self-Service ligadas ao domínio', + layer: 'tenant', + spec: '028', + specTitle: 'OpenPanel VM123', + action: 'console', + consolePath: '/admin/dominio', + goto: 'Console › Domínio', + }, + { + id: 'cloudflare', + label: 'Cloudflare', + desc: 'Zona DNS na conta Ibytera', + layer: 'infra', + spec: '037', + specTitle: 'DNS multi-Cloudflare', + action: 'console', + consolePath: '/admin/dominio', + goto: 'Console › DNS Viewer', + }, + { + id: 'traefik', + label: 'Traefik / SNI', + desc: 'Routers mail.* no CT114', + layer: 'infra', + spec: '026', + specTitle: 'Purge Traefik validation', + action: 'view', + view: 'infra', + goto: 'Desk › Infra CODE (CT114)', + }, + { + id: 'desk', + label: 'Desk OPS', + desc: 'Webhooks, tickets e audit_domains', + layer: 'app', + spec: '017', + specTitle: 'Purge + histórico Desk', + action: 'view', + view: 'events', + eventsTab: 'purges', + goto: 'Desk › Eventos › Histórico purges', + }, ]; const SERVICE_CATALOG = [ @@ -30,6 +104,7 @@ const DeskServices = (() => { wizardHost: 'VM112', deliveryModel: 'saas', category: 'apps', + layerScope: 'app', phase: 'active', }, { @@ -41,6 +116,7 @@ const DeskServices = (() => { wizardHost: 'VM112', deliveryModel: 'saas', category: 'apps', + layerScope: 'tenant', phase: 'active', }, { @@ -52,6 +128,7 @@ const DeskServices = (() => { wizardHost: 'futuro', deliveryModel: 'saas', category: 'apps', + layerScope: 'infra', phase: 'planned', }, { @@ -63,6 +140,7 @@ const DeskServices = (() => { wizardHost: 'futuro', deliveryModel: 'iaas', category: 'security', + layerScope: 'infra', phase: 'planned', }, { @@ -74,6 +152,7 @@ const DeskServices = (() => { wizardHost: 'futuro', deliveryModel: 'iaas', category: 'infra', + layerScope: 'infra', phase: 'planned', }, { @@ -85,6 +164,7 @@ const DeskServices = (() => { wizardHost: 'futuro', deliveryModel: 'saas', category: 'security', + layerScope: 'app', phase: 'planned', }, ]; @@ -167,24 +247,69 @@ const DeskServices = (() => { })); } + function readDomainsCache() { + try { + const raw = sessionStorage.getItem(DOMAINS_CACHE_KEY); + if (!raw) return null; + const parsed = JSON.parse(raw); + if (!parsed || !Array.isArray(parsed.domains)) return null; + const ageMs = Date.now() - (parsed.at || 0); + return { domains: parsed.domains, ageMs, fresh: ageMs < DOMAINS_CACHE_TTL_MS }; + } catch { + return null; + } + } + + function writeDomainsCache(domains) { + try { + sessionStorage.setItem( + DOMAINS_CACHE_KEY, + JSON.stringify({ at: Date.now(), domains: domains || [] }), + ); + } catch { + /* quota — ignorar */ + } + } + + function applyDomains(domains) { + _domains = domains || []; + _clients = buildClients(_domains); + if (_selectedClientId && !_clients.some((c) => c.id === _selectedClientId)) { + _selectedClientId = null; + } + } + + function hydrateFromCache() { + const cached = readDomainsCache(); + if (!cached?.domains?.length) return null; + applyDomains(cached.domains); + return cached; + } + async function loadBillingMap() { if (!window.DeskModules?.isEnabled('billing-recurrence')) return; try { - const data = await apiFetch('/v1/billing/accounts?limit=500'); + const data = await apiFetch('/v1/billing/accounts?limit=500', {}, 15000); _billingByDomain = {}; for (const a of data.accounts || []) _billingByDomain[a.domain] = a; } catch { _billingByDomain = {}; } } - async function loadDomains() { - const data = await apiFetch('/v1/vm112/domains', {}, 120000); - _domains = data.domains || []; - await loadBillingMap(); - _clients = buildClients(_domains); - if (_selectedClientId && !_clients.some((c) => c.id === _selectedClientId)) { - _selectedClientId = null; - } - return _domains; + async function loadDomains(options = {}) { + const timeoutMs = options.timeoutMs ?? DOMAINS_FETCH_TIMEOUT_MS; + if (_loadInFlight) return _loadInFlight; + _loadInFlight = (async () => { + try { + const data = await apiFetch('/v1/vm112/domains', {}, timeoutMs); + applyDomains(data.domains || []); + writeDomainsCache(_domains); + await loadBillingMap(); + return _domains; + } finally { + _loadInFlight = null; + } + })(); + return _loadInFlight; } function filteredClients() { @@ -201,6 +326,10 @@ const DeskServices = (() => { }); } + function isPurgeBlocked(domain) { + return PURGE_BLOCKLIST.has(String(domain || '').toLowerCase().trim()); + } + function selectedClient() { return _clients.find((c) => c.id === _selectedClientId) || null; } @@ -247,15 +376,19 @@ const DeskServices = (() => { const sel = _selectedClientId === c.id ? ' servicos-client-row--selected' : ''; const emailSt = serviceStatus('email_tenant', c); const offers = activeOfferCount(c); + const protectedDom = isPurgeBlocked(c.domain); return ` `; } @@ -275,11 +408,13 @@ const DeskServices = (() => { const wizNote = service.wizard ? `wizard: ${service.wizard} (${service.wizardHost})` : (service.phase === 'planned' ? 'wizard próprio — planeamento MOSP' : ''); + const layerLabel = LAYER_LABELS[service.layerScope] || LAYER_LABELS.tenant; return `
${esc(dm)} + ${esc(layerLabel)}

${esc(service.label)}

${esc(service.desc)}

@@ -320,6 +455,31 @@ const DeskServices = (() => {

Cada oferta MOSP terá wizard próprio (VM112 = só e-mail). Orquestração aqui no Desk VM122 — sem alterar a Hero do portal.

`; } + function scopeCardHtml(s) { + const layer = LAYER_LABELS[s.layer] || s.layer; + return ` + `; + } + + function blocklistBannerHtml(domain) { + if (!isPurgeBlocked(domain)) return ''; + return ` +
+ 🔒 Blocklist purge (Spec 017) +

${esc(domain)} está protegido — purge desactivado. Domínios fixos: ${[...PURGE_BLOCKLIST].map((d) => `${esc(d)}`).join(', ')}.

+
`; + } + function scopesHtml() { const client = selectedClient(); if (!client) { @@ -328,11 +488,7 @@ const DeskServices = (() => { if (_selectedServiceId !== 'email_tenant') { return `

Escopo OPS detalhado disponível para E-mail Tenant. Serviço actual: ${esc(SERVICE_CATALOG.find((s) => s.id === _selectedServiceId)?.label || _selectedServiceId)}.

`; } - return OPS_SCOPES.map((s) => ` -
- ${esc(s.label)} - ${esc(s.desc)} -
`).join(''); + return `${blocklistBannerHtml(client.domain)}${OPS_SCOPES.map(scopeCardHtml).join('')}`; } function pageHtml() { @@ -381,6 +537,7 @@ const DeskServices = (() => { if (stats) stats.innerHTML = statsHtml(); bindClientClicks(root); bindServiceClicks(root); + bindScopeClicks(root); } function bindPage(root) { @@ -395,6 +552,7 @@ const DeskServices = (() => { }); bindClientClicks(root); bindServiceClicks(root); + bindScopeClicks(root); } function bindClientClicks(root) { @@ -407,6 +565,42 @@ const DeskServices = (() => { }); } + function bindScopeClicks(root) { + root.querySelectorAll('[data-scope-id]').forEach((btn) => { + btn.addEventListener('click', () => navigateScope(btn.dataset.scopeId)); + }); + } + + function navigateScope(scopeId) { + const scope = OPS_SCOPES.find((s) => s.id === scopeId); + const client = selectedClient(); + if (!scope || !client) return; + const domain = client.domain; + + if (scope.action === 'modal') { + openModal(domain); + return; + } + if (scope.action === 'console') { + if (typeof openConsoleDnsViewer === 'function' && scope.id === 'cloudflare') { + openConsoleDnsViewer(domain); + return; + } + if (typeof openConsolePath === 'function') { + openConsolePath(scope.consolePath || '/admin/dominio', { domain }); + } + return; + } + if (scope.action === 'view' && window.DeskNavigate?.go) { + window.DeskNavigate.go(scope.view, { + eventsTab: scope.eventsTab, + ticketFilter: scope.ticketFilter, + }); + } else if (scope.action === 'view') { + document.querySelector(`[data-view="${scope.view}"]`)?.click(); + } + } + function bindServiceClicks(root) { root.querySelectorAll('[data-service-id]').forEach((tile) => { tile.addEventListener('click', () => { @@ -783,6 +977,7 @@ const DeskServices = (() => { } function closeModal() { + stopVm112LoadStepAnimation(); closePurgeDrawer(); const modal = document.getElementById('vm112-domain-modal'); if (!modal) return; @@ -790,46 +985,29 @@ const DeskServices = (() => { modal.setAttribute('aria-hidden', 'true'); } - function openModal(domain) { - const modal = document.getElementById('vm112-domain-modal'); - const body = document.getElementById('vm112-domain-modal-body'); - const title = document.getElementById('vm112-domain-modal-title'); - const sub = document.getElementById('vm112-domain-modal-sub'); - if (!modal || !body) return; - modal.classList.remove('hidden'); - modal.setAttribute('aria-hidden', 'false'); - title.textContent = domain; - sub.textContent = 'E-mail Tenant · a carregar…'; - body.innerHTML = '

A carregar detalhe VM112…

'; - loadModal(domain); - modal.querySelectorAll('[data-close-vm112-modal]').forEach((el) => { - el.onclick = closeModal; - }); - } - - async function loadModal(domain) { - const body = document.getElementById('vm112-domain-modal-body'); - const sub = document.getElementById('vm112-domain-modal-sub'); - try { - const d = await apiFetch(`/v1/vm112/domains/${encodeURIComponent(domain)}`, {}, 180000); - const accN = (d.accounts || []).length; - sub.textContent = `E-mail Tenant · ${accN} conta(s) · ${d.mail_host || ''}`; - const steps = (d.infra_status?.steps || []) - .map((s) => `
  • ${esc(s.label)} — ${esc(s.message)}
  • `) - .join(''); - const accounts = (d.accounts || []) - .map((a) => `
  • ${esc(a)}
  • `).join('') || '
  • Nenhuma
  • '; - const cf = d.cloudflare_zone; - const cfTxt = cf?.name ? `Zona ${cf.name} (${cf.status || '—'})` : 'Sem zona Cloudflare Ibytera'; - const extraAuth = Boolean(d.purge_extra_auth_required); - const authCodeField = extraAuth - ? ` + function renderModalBody(d, domain) { + const accN = (d.accounts || []).length; + const steps = (d.infra_status?.steps || []) + .map((s) => `
  • ${esc(s.label)} — ${esc(s.message)}
  • `) + .join(''); + const accounts = (d.accounts || []) + .map((a) => `
  • ${esc(typeof a === 'string' ? a : a.email || a)}
  • `).join('') + || '
  • Nenhuma
  • '; + const cf = d.cloudflare_zone; + const cfTxt = cf?.name ? `Zona ${cf.name} (${cf.status || '—'})` : (cf?.error ? `Erro CF: ${cf.error}` : 'Sem zona Cloudflare Ibytera'); + const extraAuth = Boolean(d.purge_extra_auth_required); + const purgeBlocked = Boolean(d.purge_blocked); + const authCodeField = extraAuth + ? `

    Domínio protegido — exige código de autorização gerado em Infra (root).

    ` - : ''; - body.innerHTML = ` + : ''; + const purgeBtn = purgeBlocked + ? '

    Domínio na blocklist — purge desactivado.

    ' + : ``; + return ` `; - body.querySelector('#vm112-purge-btn')?.addEventListener('click', () => runPurge(domain)); + } + + function vm112DetailLoadingPanelHtml() { + return ` +
    + +

    A consultar VM112

    + +

    1.ª consulta ~10–15s · reabrir o mesmo domínio usa cache.

    +
    `; + } + + function vm112LoadingButtonHtml(label = 'A carregar') { + return ``; + } + + let _vm112LoadStepTimer = null; + + function startVm112LoadStepAnimation(root) { + stopVm112LoadStepAnimation(); + const steps = root?.querySelectorAll('#vm112-load-steps .vm112-load-step'); + if (!steps?.length) return; + let idx = 0; + _vm112LoadStepTimer = setInterval(() => { + steps.forEach((el, i) => { + el.classList.toggle('vm112-load-step--active', i === idx); + }); + idx = (idx + 1) % steps.length; + }, 2800); + } + + function stopVm112LoadStepAnimation() { + if (_vm112LoadStepTimer) { + clearInterval(_vm112LoadStepTimer); + _vm112LoadStepTimer = null; + } + } + + function setModalSubLoading() { + const sub = document.getElementById('vm112-domain-modal-sub'); + if (!sub) return; + sub.innerHTML = 'E-mail Tenant · a carregar'; + } + + function modalSkeletonHtml(raw, domain) { + return ` + + + `; + } + + function openModal(domain) { + const modal = document.getElementById('vm112-domain-modal'); + const body = document.getElementById('vm112-domain-modal-body'); + const title = document.getElementById('vm112-domain-modal-title'); + if (!modal || !body) return; + stopVm112LoadStepAnimation(); + modal.classList.remove('hidden'); + modal.setAttribute('aria-hidden', 'false'); + title.textContent = domain; + setModalSubLoading(); + const raw = selectedClient()?.raw; + body.innerHTML = raw + ? modalSkeletonHtml(raw, domain) + : ``; + body.setAttribute('aria-busy', 'true'); + startVm112LoadStepAnimation(body); + loadModal(domain); + modal.querySelectorAll('[data-close-vm112-modal]').forEach((el) => { + el.onclick = closeModal; + }); + } + + async function loadModal(domain) { + const body = document.getElementById('vm112-domain-modal-body'); + const sub = document.getElementById('vm112-domain-modal-sub'); + try { + const d = await apiFetch(`/v1/vm112/domains/${encodeURIComponent(domain)}`, {}, 45000); + stopVm112LoadStepAnimation(); + body.removeAttribute('aria-busy'); + const accN = (d.accounts || []).length; + if (sub) sub.textContent = `E-mail Tenant · ${accN} conta(s) · ${d.mail_host || ''}`; + body.innerHTML = renderModalBody(d, domain); + if (!d.purge_blocked) { + body.querySelector('#vm112-purge-btn')?.addEventListener('click', () => runPurge(domain)); + } } catch (e) { - body.innerHTML = `

    Erro: ${esc(e.message)}

    `; + stopVm112LoadStepAnimation(); + body.removeAttribute('aria-busy'); + if (sub) sub.textContent = 'E-mail Tenant · erro ao carregar'; + body.innerHTML = `

    Erro: ${esc(e.message)}

    `; + body.querySelector('#servicos-modal-retry')?.addEventListener('click', () => { + const rawRetry = selectedClient()?.raw; + setModalSubLoading(); + body.innerHTML = rawRetry + ? modalSkeletonHtml(rawRetry, domain) + : ``; + body.setAttribute('aria-busy', 'true'); + startVm112LoadStepAnimation(body); + loadModal(domain); + }); } } @@ -880,6 +1175,15 @@ const DeskServices = (() => { el.textContent = message; } + function renderPageShell(container, { staleHint = '' } = {}) { + if (_clients.length && !_selectedClientId) { + _selectedClientId = _clients[0].id; + } + container.innerHTML = pageHtml(); + bindPage(container); + if (staleHint) setPollStatus(container, staleHint); + } + async function renderPage(container, options = {}) { const { poll = false } = options; if (!container) return; @@ -888,33 +1192,47 @@ const DeskServices = (() => { return; } const hasPage = Boolean(container.querySelector('.servicos-page')); + const cached = readDomainsCache(); + if (poll && hasPage) { try { - await loadDomains(); + await loadDomains({ timeoutMs: DOMAINS_POLL_TIMEOUT_MS }); refreshPanels(container); setPollStatus(container, ''); } catch (e) { - setPollStatus(container, `Actualização falhou: ${e.message}`, true); + setPollStatus(container, `Actualização falhou (lista anterior mantida): ${e.message}`, true); } return; } + if (!hasPage) { - container.innerHTML = '

    A carregar clientes e serviços VM112…

    '; - } - try { - await loadDomains(); - if (_clients.length && !_selectedClientId) { - _selectedClientId = _clients[0].id; + const hydrated = hydrateFromCache(); + if (hydrated) { + renderPageShell(container, { + staleHint: hydrated.fresh + ? 'A actualizar lista VM112…' + : 'Lista em cache — a actualizar VM112…', + }); + } else { + container.innerHTML = '

    A carregar clientes e serviços VM112…

    '; } - container.innerHTML = pageHtml(); - bindPage(container); - setPollStatus(container, ''); - } catch (e) { + } + + try { + await loadDomains({ timeoutMs: DOMAINS_FETCH_TIMEOUT_MS }); if (hasPage) { - setPollStatus(container, `Erro ao carregar VM112: ${e.message}`, true); + refreshPanels(container); + setPollStatus(container, ''); + } else { + renderPageShell(container); + setPollStatus(container, ''); + } + } catch (e) { + if (hasPage || cached?.domains?.length) { + setPollStatus(container, `VM112 lento ou API Desk a reiniciar — ${e.message}`, true); return; } - container.innerHTML = `

    Erro ao carregar VM112: ${esc(e.message)}

    `; + container.innerHTML = `

    Erro ao carregar VM112: ${esc(e.message)}

    A VM112 demora ~10–15s por pedido. Se acabámos de actualizar a API Desk, aguarde 30s e tente de novo.

    `; container.querySelector('#servicos-retry')?.addEventListener('click', () => renderPage(container)); } } diff --git a/projects/ops-desk/frontend/assets/styles.css b/projects/ops-desk/frontend/assets/styles.css index 923a844..07fa9fe 100644 --- a/projects/ops-desk/frontend/assets/styles.css +++ b/projects/ops-desk/frontend/assets/styles.css @@ -519,7 +519,8 @@ body { flex: 1; } -.nav button { +.nav button, +.nav .nav-item { position: relative; width: 100%; text-align: left; @@ -537,6 +538,11 @@ body { margin-bottom: 0.02rem; border-radius: 14px; transition: background 0.15s, color 0.15s, box-shadow 0.15s, transform 0.15s; + text-decoration: none; +} + +.nav .nav-item:visited { + color: rgba(255,255,255,0.88); } .nav-icon-wrap { @@ -570,7 +576,8 @@ body { text-overflow: ellipsis; } -.nav button:hover { +.nav button:hover, +.nav .nav-item:hover { background: rgba(255,255,255,0.05); color: #fff; transform: translateX(1px); @@ -592,11 +599,13 @@ body { 0 8px 22px rgba(0,0,0,0.16); } -.nav button:hover .nav-icon-wrap { +.nav button:hover .nav-icon-wrap, +.nav .nav-item:hover .nav-icon-wrap { transform: translateY(-1px); } .nav-item-dashboard .nav-icon-svg { width: 52px; height: 52px; } +.nav-item-console .nav-icon-svg { width: 52px; height: 52px; } .nav-item-overview .nav-icon-svg { width: 50px; height: 50px; } .nav-item-tickets .nav-icon-svg { width: 52px; height: 52px; } .nav-item-leads .nav-icon-svg { width: 54px; height: 54px; } @@ -2511,12 +2520,53 @@ button.health-card { .modal-dns-highlight, .dns-records-section { + margin-top: 0.75rem; border: 1px solid #dbe4f4; border-radius: 10px; padding: 0.85rem 1rem; background: #f8fbff; } +.dns-viewer-section .dns-viewer-head { + display: flex; + flex-wrap: wrap; + gap: 0.35rem; + margin-bottom: 0.5rem; +} + +.dns-viewer-msg { + font-style: italic; + color: var(--muted, #5c6570); +} + +.dns-viewer-ns-warn { + color: #b5651d; +} + +.dns-viewer-checks { + margin: 0.35rem 0 0.65rem; +} + +.dns-viewer-actions { + display: flex; + flex-wrap: wrap; + gap: 0.35rem; + margin-top: 0.65rem; +} + +.dns-source-badge { + font-size: 0.72rem; + padding: 0.1rem 0.35rem; + border-radius: 4px; + background: #eef2f7; + color: #445; +} + +.dns-viewer-sub { + margin: 0.75rem 0 0.35rem; + font-size: 0.85rem; +} + /* ── Infra 2 — SOC Operations Console (dark, scoped) ── */ .soc-console { --soc-bg: #0b0f14; @@ -3345,6 +3395,104 @@ button.health-card { background: #fee2e2; } +/* Modal purge — loading VM112 detail (Spec 018) */ +.vm112-load-panel { + padding: 0.25rem 0; +} +.vm112-load-progress { + height: 4px; + background: #fecaca; + border-radius: 999px; + overflow: hidden; + margin-bottom: 0.85rem; +} +.vm112-load-progress-bar { + display: block; + height: 100%; + width: 38%; + background: linear-gradient(90deg, #b91c1c, #f87171, #b91c1c); + border-radius: 999px; + animation: vm112-load-bar 1.35s ease-in-out infinite; +} +@keyframes vm112-load-bar { + 0% { transform: translateX(-110%); } + 100% { transform: translateX(320%); } +} +.vm112-load-title { + margin: 0 0 0.65rem; + font-size: 0.92rem; + font-weight: 600; + color: #7f1d1d; +} +.vm112-load-steps { + margin: 0 0 0.5rem; + padding: 0; + list-style: none; +} +.vm112-load-step { + position: relative; + padding: 0.28rem 0 0.28rem 1.35rem; + font-size: 0.82rem; + color: var(--muted, #6b7280); +} +.vm112-load-step::before { + content: ''; + position: absolute; + left: 0.15rem; + top: 0.62rem; + width: 0.55rem; + height: 0.55rem; + border-radius: 50%; + background: #d1d5db; +} +.vm112-load-step--active { + color: #991b1b; + font-weight: 600; +} +.vm112-load-step--active::before { + background: #dc2626; + animation: vm112-purge-pulse 1s ease infinite; +} +.vm112-load-spinner { + display: inline-block; + width: 0.95em; + height: 0.95em; + border: 2px solid rgba(185, 28, 28, 0.22); + border-top-color: #b91c1c; + border-radius: 50%; + animation: vm112-load-spin 0.65s linear infinite; + vertical-align: -0.12em; + margin-right: 0.4rem; +} +@keyframes vm112-load-spin { + to { transform: rotate(360deg); } +} +.btn-loading { + display: inline-flex; + align-items: center; + justify-content: center; + gap: 0.15rem; + min-width: 9rem; + cursor: wait; + opacity: 0.92; +} +.vm112-load-dots::after { + display: inline-block; + width: 1.1em; + text-align: left; + animation: vm112-load-dots 1.4s steps(4, end) infinite; + content: ''; +} +@keyframes vm112-load-dots { + 0% { content: ''; } + 25% { content: '.'; } + 50% { content: '..'; } + 75%, 100% { content: '...'; } +} +.vm112-modal-sub-loading { + color: var(--muted, #6b7280); +} + /* Contas — Orquestração VM112 (Spec 017) */ .contas-page { max-width: 1200px; @@ -3667,7 +3815,7 @@ button.health-card { } .servicos-client-row { display: grid; - grid-template-columns: auto 1fr auto; + grid-template-columns: auto minmax(0, 1fr) auto; align-items: center; gap: 0.65rem; width: 100%; @@ -3680,6 +3828,17 @@ button.health-card { font: inherit; color: inherit; } +.servicos-client-main { + min-width: 0; + overflow: hidden; +} +.servicos-client-badges { + display: flex; + align-items: center; + justify-content: flex-end; + gap: 0.25rem; + flex-shrink: 0; +} .servicos-client-row:hover { background: #f3f6fb; } .servicos-client-row--selected { background: #e8f0fe; @@ -3695,7 +3854,9 @@ button.health-card { .servicos-client-main strong { display: block; font-size: 0.9rem; - word-break: break-word; + overflow: hidden; + text-overflow: ellipsis; + white-space: nowrap; } .servicos-client-main span { display: block; @@ -3826,8 +3987,103 @@ button.health-card { padding: 0.65rem 0.85rem; border-bottom: 1px solid #f0ebe3; } +.servicos-scope-item--clickable { + display: block; + width: 100%; + text-align: left; + border: none; + background: transparent; + cursor: pointer; + font: inherit; + transition: background 0.15s ease; +} +.servicos-scope-item--clickable:hover, +.servicos-scope-item--clickable:focus-visible { + background: #faf6f0; + outline: none; +} +.servicos-scope-item-head { + display: flex; + align-items: center; + justify-content: space-between; + gap: 0.35rem; + margin-bottom: 0.2rem; +} .servicos-scopes .servicos-scope-item:last-child { border-bottom: none; } .servicos-scopes strong { display: block; font-size: 0.85rem; } +.servicos-scope-desc { + display: block; + font-size: 0.75rem; + color: var(--muted, #6b7280); + margin-bottom: 0.35rem; +} +.servicos-scope-meta { + display: flex; + flex-wrap: wrap; + justify-content: space-between; + gap: 0.25rem 0.5rem; + font-size: 0.68rem; +} +.servicos-scope-spec { + color: #7c3aed; +} +.servicos-scope-goto { + color: #2f6fed; + font-weight: 600; +} +.servicos-scope-layer { + flex-shrink: 0; + font-size: 0.62rem; + font-weight: 700; + text-transform: uppercase; + letter-spacing: 0.03em; + padding: 0.12rem 0.4rem; + border-radius: 999px; +} +.servicos-scope-layer--tenant { background: #dbeafe; color: #1d4ed8; } +.servicos-scope-layer--app { background: #fce7f3; color: #9d174d; } +.servicos-scope-layer--infra { background: #e7e5e4; color: #44403c; } +.servicos-blocklist-banner { + margin: 0.65rem 0.85rem 0.5rem; + padding: 0.55rem 0.65rem; + background: #fff1f2; + border: 1px solid #fecaca; + border-radius: 8px; + font-size: 0.78rem; + color: #7f1d1d; +} +.servicos-blocklist-banner p { margin: 0.35rem 0 0; } +.servicos-blocklist-banner code { + font-size: 0.72rem; + background: #fee2e2; + padding: 0.05rem 0.25rem; + border-radius: 3px; +} +.servicos-client-badge--protected { + background: #fee2e2; + color: #991b1b; + font-size: 0.75rem; + line-height: 1; + padding: 0.2rem 0.35rem; +} +.servicos-tile-layer { + position: absolute; + top: 0.45rem; + left: 0.55rem; + font-size: 0.58rem; + font-weight: 700; + text-transform: uppercase; + letter-spacing: 0.02em; + padding: 0.1rem 0.35rem; + border-radius: 999px; + max-width: 46%; + overflow: hidden; + text-overflow: ellipsis; + white-space: nowrap; +} +.servicos-tile-layer--tenant { background: #dbeafe; color: #1d4ed8; } +.servicos-tile-layer--app { background: #fce7f3; color: #9d174d; } +.servicos-tile-layer--infra { background: #e7e5e4; color: #44403c; } .servicos-scopes span { font-size: 0.75rem; color: var(--muted, #6b7280); } .servicos-scope-hint { padding: 0.85rem; @@ -3865,6 +4121,12 @@ button.health-card { .purge-history-table tbody tr.purge-history-row:hover { background: rgba(47, 111, 237, 0.06); } +.event-auditor-row { + cursor: pointer; +} +.event-auditor-row:hover { + background: rgba(47, 111, 237, 0.06); +} .purge-history-link { color: var(--accent, #2f6fed); font-size: 0.85rem; @@ -4564,7 +4826,7 @@ button.health-card { .badge.billing { background: #f59e0b33; color: #fbbf24; font-size: 0.65rem; } .kpi-card--billing-pending .kpi-value { color: #f59e0b; } .kpi-card--billing-active .kpi-value { color: #22c55e; } -.servicos-billing-badge { cursor: pointer; margin-left: 0.35rem; font-size: 0.85rem; opacity: 0.45; } +.servicos-billing-badge { cursor: pointer; font-size: 0.85rem; opacity: 0.45; } .servicos-billing-badge--active { opacity: 1; filter: drop-shadow(0 0 4px #22c55e88); } .migration-job-row { display: flex; justify-content: space-between; padding: 0.6rem 0; border-bottom: 1px solid var(--border); } .migration-gate-ready { color: #22c55e; } diff --git a/projects/ops-desk/frontend/index.html b/projects/ops-desk/frontend/index.html index f274e3e..5e43cdc 100644 --- a/projects/ops-desk/frontend/index.html +++ b/projects/ops-desk/frontend/index.html @@ -4,7 +4,7 @@ Ligbox Ops — Support Desk - + @@ -215,10 +215,10 @@ Eventos - - + +

    Carregando…

    @@ -431,6 +432,19 @@ + - + - + + - + diff --git a/specs/017-vm112-domain-orchestration/spec.md b/specs/017-vm112-domain-orchestration/spec.md index b2207f7..ae7d739 100644 --- a/specs/017-vm112-domain-orchestration/spec.md +++ b/specs/017-vm112-domain-orchestration/spec.md @@ -1,7 +1,7 @@ # Feature Specification: Domínios VM112 — Purge & Histórico (017) **Criado:** 2026-06-16 -**Actualizado:** 2026-06-19 (extensões Spec 026 · Spec 032) +**Actualizado:** 2026-06-25 (performance modal + UX Escopo OPS + blocklist UI) **Solicitado por:** Roger **Status:** v1 + v2 concluídos · Fase 3 VM112 pendente **Prioridade:** P1 (testes E2E + padrão de limpeza) @@ -89,7 +89,7 @@ Técnicos `technician` e `noc` **não** acedem. 4. Domínio ∉ blocklist (`ligbox.com.br`, etc.) 5. Proxy VM112 `POST /api/admin/domains/{domain}/purge` com `X-Api-Key` -**Pós-purge Desk:** apagar `audit_domains`, `webhook_events`, `tickets`, `assist_sessions`, `audit_checks` com referência ao domínio. +**Pós-purge Desk:** apagar `audit_domains`, `tickets`, `assist_sessions`, `audit_checks`, `domain_console_scenarios` com referência ao domínio. **Preservar** `webhook_events` e inserir marcador `domain.purged` (`source=desk.purge`). --- @@ -97,12 +97,35 @@ Técnicos `technician` e `noc` **não** acedem. | Método | Path | Auth | |--------|------|------| -| GET | `/api/admin/domains` | `X-Api-Key` | +| GET | `/api/admin/domains` | `X-Api-Key` — resposta inclui `cached`, `cache_age_sec` (2026-06-25) | | GET | `/api/admin/domains/{domain}` | `X-Api-Key` | | POST | `/api/admin/domains/{domain}/purge` | `X-Api-Key` | | GET | `/api/admin/domains/purge-jobs/{job_id}` | `X-Api-Key` _(memória, efémero)_ | +### Cache listagem domínios (2026-06-25) + +**Problema:** `list_orchestrated_domains()` chamava `zmprov gad` em **cada** `GET /api/admin/domains` (~5s) — gargalo da página Serviços IaaS no Desk (Spec 018). + +**Implementação VM112** (`deploy/vm112-wizard/perf-domains-list-20260625/`): + +| Cache | TTL | Chave | Invalidação | +|-------|-----|-------|-------------| +| Lista Carbonio (`gad`) | 90s | `gad_all_domains` | `zmprov cd`, `zmprov dd`, purge concluído | +| Lista orquestrada montada | 60s | `orchestrated_domains_list` | idem | +| Detalhe domínio (purge modal) | 120s | `domain_detail:{domain}` | purge / dd / cd | +| Infra checks (`get_status`) | 90s | `infra_status:{domain}` | idem | +| Contas (`zmprov gaa`) | 45s | `accounts_list:{domain}` | purge contas / dd | + +**Optimizações modal purge (2026-06-25):** `get_domain_detail()` executa gaa + infra + Cloudflare **em paralelo** (`ThreadPoolExecutor`). 1.º hit ~16s → 2.º hit **~0,04s**. + +**Ficheiros:** `carbonio_cache.py`, `carbonio.list_all_domains()`, `domain_orchestration.get_domain_detail()`, `infrastructure.get_status()` cache. + +**Desk VM122** mantém cache proxy adicional 60s (`VM112_DOMAINS_CACHE_TTL`) — ver Spec 018 § Performance. + +--- + **Purge VM112 (ordem):** + 1. Apagar contas Carbonio (`zmprov da`) 2. Apagar domínio Carbonio (`zmprov dd`) 3. Remover portal users com `planned_corporate_email` no domínio @@ -272,3 +295,123 @@ A Spec 017 cobre o ciclo completo de purge de domínio VM112: **Purges registados (exemplo):** `myvexx.com`, `diarissima.com`, `ibytera.com` — visíveis em Eventos → Histórico de purges. **Próximo passo natural:** Fase 3 no wizard VM112; depois filtros/export no histórico se necessário. + +--- + +## Incidente 2026-06-25 — Auditor 404 + purge timeout (Roger) + +### Sintomas reportados + +| # | UI | Mensagem | Domínio / ref | +|---|-----|----------|----------------| +| 1 | Eventos → Auditor de Eventos | `Erro: Not Found` | webhook · **291** (`domain.purged`, `cenario-demo.ops.ligbox.com.br`) | +| 2 | Eventos → Histórico de purges | badge **ERRO**, VM112 **timed out** (33s), Desk **0** | `eplacebets.com` · job `84a78c85d6b6490c` | + +### Diagnóstico + +| Erro | Causa raiz | Impacto real | +|------|------------|--------------| +| Auditor 404 | Modal chamava detalhe inexistente (`GET /webhooks/events/{id}` ausente); evento **291** estava OK em SQLite | Purge `cenario-demo` **concluído** — falha só de visualização | +| Purge timed out | `httpx` poll VM112 com timeout **60s** → excepção `timed out`; job marcado `error` **antes** da fase Desk; `recover` não actuava em jobs `error` | `eplacebets.com` **pode** ter ficado parcial na VM112; Desk **não limpo** | + +### Correções (2026-06-25) + +| Camada | Alteração | +|--------|-----------| +| **API** | `GET /api/v1/webhooks/events/{event_id}` — detalhe para Auditor | +| **API purge** | `VM112_PURGE_HTTP_TIMEOUT=300` (env); poll tolerante a falhas transitórias; auto-recuperação se domínio já ausente na VM112 | +| **Jobs** | `recover_job` actua em status `error`; `_execute_job` tenta recuperação automática pós-timeout | +| **UI Eventos** | Linhas clicáveis → modal **Auditor de Eventos**; render dedicado `domain.purged`; filtro origem **Purge** | +| **UI Histórico** | Labels Desk actualizados (`domain.purged` preserva histórico); botão **Recuperar purge** em jobs com erro | + +### Ficheiros + +| Ficheiro | Alteração | +|----------|-----------| +| `api/app/main.py` | `GET /webhooks/events/{id}` | +| `api/app/vm112_domains.py` | timeout 300s, poll resiliente, recover pós-timeout | +| `api/app/vm112_purge_jobs.py` | recover em `error`, auto-recover | +| `frontend/index.html` | `event-auditor-modal`, filtro Purge | +| `frontend/assets/app.js` | Auditor, purge desk labels, botão recover | + +### Validação pós-deploy + +```bash +# Evento 291 — deve retornar domain.purged (HTTP 200) +curl -s -H "Authorization: Bearer $TOKEN" \ + "https://desk.ligbox.com.br/api/v1/webhooks/events/291" + +# Recuperar job eplacebets se VM112 já limpou +curl -s -X POST -H "Authorization: Bearer $TOKEN" \ + "https://desk.ligbox.com.br/api/v1/vm112/purge/jobs/84a78c85d6b6490c/recover" +``` + +### Resultado esperado + +1. Clicar evento **291** → modal mostra utilizador, ferramenta, domínio e payload (sem 404). +2. Job `84a78c85d6b6490c` → **Recuperar purge** conclui fase Desk se `eplacebets.com` já não existir na VM112; senão repetir purge após fix de timeout. +3. Novos purges longos (Carbonio) não falham aos ~33–60s por timeout HTTP do poll. + +### Resultado pós-deploy (2026-06-25, VM122) + +| Verificação | Resultado | +|-------------|-----------| +| `GET /webhooks/events/291` | ✅ `domain.purged` · `desk.purge` | +| `eplacebets.com` na VM112 | ✅ ausente (purge VM112 efectivo) | +| Job `84a78c85d6b6490c` recover | ✅ `status=done`, Desk limpo, marcador `domain.purged` inserido | +| Job posterior `e238c1efb7c24d92` | ✅ já estava `done` (retry manual anterior) | + +--- + +## UI Serviços — loading, blocklist, Escopo OPS (2026-06-25) + +Melhorias UX na página **Serviços IaaS** (Spec 018) sem alterar regras de purge. + +### Blocklist — apresentação + +| Domínio | API `purge_blocked` | UI lista clientes | UI modal purge | +|---------|---------------------|-------------------|----------------| +| `ligbox.com.br` | `true` | badge 🔒 (tooltip) | formulário desactivado | +| `itecnologys.com` | `true` | idem | idem | +| restantes | `false` | — | senha Root + confirmação | + +**Constante Desk:** `PURGE_BLOCKLIST` em `vm112_domains.py` — espelhada no frontend (`servicos.js`) para badges imediatos antes do detalhe API. + +**Painel Escopo OPS:** banner vermelho quando cliente seleccionado ∈ blocklist, listando domínios protegidos. + +### Modal «Gerir» — loading animado + +Enquanto `GET /api/v1/vm112/domains/{domain}` corre (~10–15s cold): + +| Elemento | Comportamento | +|----------|----------------| +| Skeleton | Dados da lista clientes (admin portal, Carbonio, site) | +| Barra progresso | Indeterminada (CSS `vm112-load-bar`) | +| Etapas | Rotação Carbonio → Infra CT114 → Cloudflare | +| Botão purge | Spinner + «A carregar…» desactivado | +| A11y | `aria-busy`, `aria-live="polite"` | + +### Escopo OPS — cards clicáveis + +Ver Spec **018** § Coluna Escopo OPS (mapa completo camada → Spec → destino). + +**Navegação cross-módulo:** `window.DeskNavigate.go(view, opts)` em `app.js`. + +### Ficheiros (2026-06-25 UX) + +| Ficheiro | Alteração | +|----------|-----------| +| `frontend/assets/servicos.js` | `OPS_SCOPES` enriquecido, `navigateScope`, blocklist UI, loading modal | +| `frontend/assets/styles.css` | `.vm112-load-*`, `.servicos-scope-*`, alinhamento lista clientes | +| `frontend/assets/app.js` | `DeskNavigate` | +| `frontend/index.html` | cache bust `?v=20260625align1` | + +### Versão deploy VM122 (2026-06-25) + +```bash +# Frontend (sem rebuild — docker cp) +docker cp servicos.js ligbox-ops-platform_frontend_1:/usr/share/nginx/html/assets/ +docker cp styles.css ligbox-ops-platform_frontend_1:/usr/share/nginx/html/assets/ +docker cp app.js ligbox-ops-platform_frontend_1:/usr/share/nginx/html/assets/ +docker cp index.html ligbox-ops-platform_frontend_1:/usr/share/nginx/html/ +``` diff --git a/specs/018-service-orchestration/spec.md b/specs/018-service-orchestration/spec.md index d491b22..801d00a 100644 --- a/specs/018-service-orchestration/spec.md +++ b/specs/018-service-orchestration/spec.md @@ -1,6 +1,7 @@ # Feature Specification: Orquestração de Serviços — Cliente & Catálogo (018) **Criado:** 2026-06-16 +**Actualizado:** 2026-06-25 (performance + Escopo OPS clicável + camadas tiles) **Solicitado por:** Roger **Status:** Fase 1 concluída (Desk VM122) **Wizard cliente:** inalterado na VM112 — só e-mail (`vm112-mail`) @@ -19,6 +20,85 @@ A página **Serviços** substitui a visão estreita «Contas / lista de domínio **Fase 2:** API Desk `clients` + `service_instances` em SQLite. **Fase 3:** Registry de wizards por `service_catalog.code` e purge por instância. +### Performance — Serviços IaaS (2026-06-25) + +**Cadeia:** Desk `servicos.js` → VM122 `GET /api/v1/vm112/domains` → VM112 `GET /api/admin/domains` → **`zmprov gad`** (~5s sem cache). + +| Camada | Problema | Mitigação | Deploy | +|--------|----------|-----------|--------| +| **VM112** | `zmprov gad` a cada pedido | Cache 90s `gad_all_domains` + lista orquestrada 60s; invalidação em `cd`/`dd`/purge | `deploy/vm112-wizard/perf-domains-list-20260625/` ✅ 2026-06-25 | +| **VM122 API** | Proxy repetia VM112 lenta | Cache 60s `VM112_DOMAINS_CACHE_TTL` | `vm112_domains.py` ✅ | +| **Desk frontend** | Ecrã preso «A carregar…» | `sessionStorage` stale-while-revalidate + dedup fetch | `servicos.js` ✅ | +| **Poll** | 30s sobrecarregava VM112 | Poll **90s** em `overview-home` | `app.js` ✅ | +| **Sessão** | API restart expulsava user | Rede ≠ logout (`ensureValidSession`) | `auth.js` ✅ | + +**Resultados medidos (2026-06-25):** + +| Pedido | Antes | Depois | +|--------|-------|--------| +| VM112 `/api/admin/domains` (2.º hit) | ~5–11s | **~0,03s** | +| VM112 `/api/admin/domains/{domain}` (2.º hit) | ~15–20s | **~0,04s** | +| VM112 detalhe (1.º hit, cold) | ~20s serial | **~16s** paralelo (infra+gaa+CF) | +| Desk `/api/v1/vm112/domains` (cache quente) | ~16s | **~0,08s** | +| Desk `/api/v1/vm112/domains/{domain}` (cache quente) | ~15s+ | **~3s** (1.º após lista) / **~0,1s** (2.º) | +| 1.º hit pós-restart wizard | — | ~6s lista / ~16s modal purge | + +### Modal Purge / «Gerir» (2026-06-25) + +**Endpoint:** `GET /api/v1/vm112/domains/{domain}` → VM112 `get_domain_detail()`. + +| Gargalo | Tempo | Mitigação | +|---------|-------|-----------| +| `get_status()` — SSH CT114, DNS, HTTPS | ~9s | Cache `infra_status:{domain}` 90s | +| `zmprov gaa` (contas) | ~3s | Cache `accounts_list:{domain}` 45s | +| Cloudflare API | ~1–2s | Paralelo + cache `domain_detail:{domain}` 120s | +| UI ecrã branco | UX | **Skeleton** imediato com dados da lista clientes | + +**Frontend:** `servicos.js` — modal abre com dados do cliente seleccionado; purge form activa após load. + +**Loading UX (2026-06-25):** barra indeterminada, etapas VM112 rotativas, botão spinner — ver Spec **017** § UI Serviços loading. + +**Desk API:** cache proxy detalhe 45s (`VM112_DOMAIN_DETAIL_TTL`). + +### Escopo OPS — cards clicáveis (2026-06-25) + +Coluna direita: cada escopo é **card botão** com **camada**, **Spec** e **destino**. + +| ID | Label | Camada | Spec | Destino ao clicar | +|----|-------|--------|------|-------------------| +| `carbonio` | Carbonio | Aplicação MOSP | 017 | Modal Serviços (detalhe/purge) | +| `site` | ligbox-sites | Tenant utilizador | 017 | Modal Serviços | +| `portal` | Portal users | Tenant utilizador | 028 | Console › `/admin/dominio?domain=` | +| `cloudflare` | Cloudflare | Infraestrutura | 037 | Console › DNS Viewer | +| `traefik` | Traefik / SNI | Infraestrutura | 026 | Desk › Infra CODE | +| `desk` | Desk OPS | Aplicação MOSP | 017 | Desk › Eventos › tab Purges | + +**Blocklist UI:** clientes `ligbox.com.br` / `itecnologys.com` — badge 🔒 na lista; banner no painel Escopo; purge modal desactivado (`purge_blocked` API). + +**Tiles centro:** badge camada (`Tenant utilizador` · `Aplicação MOSP` · `Infraestrutura`) + badge comercial (`SaaS` / `IaaS`). + +**Navegação:** `window.DeskNavigate.go('events', { eventsTab: 'purges' })` — helper em `app.js`. + +**Cache bust frontend (VM122):** `servicos.js`, `styles.css`, `app.js` → `?v=20260625align1`. + +--- + +```json +{ + "domains": [ "..." ], + "cached": true, + "cache_age_sec": 12 +} +``` + +**Regra deploy Roger:** + +1. VM112: `systemctl restart ligbox-wizard` invalida cache wizard (~6s no 1.º Serviços IaaS). +2. VM122: preferir `docker cp` sem restart; se restart API, Desk mostra cache browser. +3. Purge/create domain invalida cache VM112 automaticamente. + +Ver também Spec **017** § cache listagem domínios. + --- ## Problema @@ -384,8 +464,10 @@ A página Serviços no Desk é o **painel do técnico**; os wizards são **um po ### Coluna Escopo OPS +- **Cards clicáveis** (2026-06-25) — camada (tenant / app / infra), Spec, destino OPS - Lista dos escopos purge / operação quando serviço seleccionado -- **E-mail Tenant:** 6 escopos Spec 017 (Carbonio → Desk) +- **E-mail Tenant:** 6 escopos Spec 017 (Carbonio → Desk) — mapa na tabela § Escopo OPS cards +- **Blocklist:** banner + badge 🔒 para `ligbox.com.br`, `itecnologys.com` - **Futuro:** escopo dinâmico por `service_catalog.purge_scopes_json` - Indicador visual **quem gere** cada camada (matriz pizza — cliente vs Ligbox) - Nota: purge requer senha Root no modal (serviços SaaS provisionados) @@ -504,6 +586,10 @@ Um mesmo cliente pode misturar modos por instância de serviço (ex.: e-mail Saa - [x] Purge remove domínio e actualiza lista (Spec 017) - [x] Tiles futuros visíveis como «Em breve» - [x] Escopo OPS visível na coluna direita +- [x] Escopo OPS **cards clicáveis** com camada + Spec + navegação (2026-06-25) +- [x] Blocklist visível na UI (lista + banner Escopo + modal) +- [x] Modal purge com **loading animado** durante consulta VM112 +- [x] Tiles com badge **camada** (tenant / app / infra) - [x] RBAC inalterado --- diff --git a/specs/032-purge-domain-extra-auth/spec.md b/specs/032-purge-domain-extra-auth/spec.md index ac325d8..b338b89 100644 --- a/specs/032-purge-domain-extra-auth/spec.md +++ b/specs/032-purge-domain-extra-auth/spec.md @@ -27,6 +27,8 @@ Exigem um **código de autorização** gerado pelo **super_admin (root)** em **I | **Extra auth** | `myvexx.com` (+ lista configurável) | Senha Root + **código autorização** (uso único) | | **Normal** | restantes | Senha Root + confirmação domínio (Spec 017) | +**UI blocklist (2026-06-25):** badge 🔒 na lista clientes Serviços; banner no painel Escopo OPS; modal purge com `purge_blocked: true` — ver Spec **017** § UI Serviços blocklist. + --- ## Fluxo operacional (conferência)