engine_id: architecture_review template_id: base-review-report tier: deep coverage_mode: comprehensive focus_profiles:

• architecture_quality
• tech_debt
• security
• reliability
• scalability
• operations
• cost
• proof_strength status: final risk_lane: product-critical AI workflow review_trigger: Fresh Deep Tier comprehensive workspace review (session-first; prior artifacts ignored) staleness_condition: Material when LangGraph flags, Docker Compose services, CI gates, or design-spec runtime flags change

Architecture Review — Scopelytics AI Workspace (Deep)

TL;DR

• The product has a coherent FastAPI + ARQ + PostgreSQL + Redis + Next.js proxy shape with strong contract discipline (PS-001) and extensive local test coverage, but production proof is thinner than local quality because PR CI does not run backend pytest or check:api-contract.
• The highest structural risk is dual orchestration for design-spec (9k+ line DesignAutomationService vs LangGraph under backend/src/app/graphs/design_spec/) with shadow mode disabled, so cutover decisions lack safe parity evidence.
• P1 default wave: tighten production observability auth (metrics), align design-spec runtime flags across environments, add CI proof gates, and decide LangGraph cutover via ADR/tradeoff before large refactors of the monolith.

Project Context Summary

• Knowledge base / standards / profile: _context/design/architecture/project-knowledge-base.md, project-standards/ps-001 through ps-004, architect-project.profile.json — all present and active.
• Conflicts / gaps: _context/decisions/ has no durable ADRs (profile known_review_risks confirms). Prior review/action-summary artifacts in _context/design/architecture/ were excluded per session instruction.

Architecture Context

• Stakeholders: Product/engineering (design-spec pipeline), platform/ops (Docker Compose, metrics, workers), security (auth/admin, provider keys), finance (AI cost attribution).
• Concerns: Contract truth (PS-001), AI provider governance (PS-002), async artifact ops (PS-003), session/proxy boundaries (PS-004), LangGraph migration safety, CI proof strength.
• Viewpoints / views used: Container (Compose services), runtime (API/worker/frontend proxy), deployment (local Compose baseline), decision (missing ADRs), threat (proxy + admin + metrics), capacity (ARQ concurrency, rate limits).
• Evidence strength: direct for code paths cited; inferred for production deployment posture where only Compose defaults were checked; missing for live SLOs, penetration tests, and production incident history.
• Confidence: Medium — codebase and standards are well documented; production runtime configuration and operator runbooks were not executed in this review.

Scope And Lifecycle

• Status: final (advisory)
• Owning workflow: architecture-review.md via th-architect
• Owner or responsible group: Principal architecture advisory (session)
• In scope: Entire workspace runtime architecture — backend API/workers, design-spec and analysis pipelines, frontend proxy/auth, persistence, observability hooks, CI proof surface, project standards alignment.
• Out of scope: Implementing fixes, writing implementation plans, product UX brainstorm, QA execution, penetration testing, live production metrics/incidents, prior ADRs/specs/plans from earlier sessions.
• Risk lane: product-critical AI workflow
• Review trigger: User-requested Deep Tier comprehensive architecture_review (fresh session)
• Staleness condition: Re-run when LANGGRAPH_* flags, WIREFRAME_PREGENERATION_* / Stitch flags, .github/workflows/check-pr.yml, or admin/auth middleware defaults change materially

Review Coverage Matrix

Strengths

• Clear runtime boundaries: FastAPI /api/v1 composition (backend/src/app/api/v1/__init__.py), ARQ workers (backend/src/app/workers/tasks.py), Next.js same-origin proxy with prefix allowlist and CSRF (frontend/app/api/proxy/[...path]/route.ts, frontend/lib/server/csrf.ts).
• Backend-owned lifecycle truth for design-spec aligned with PS-001 (backend/src/app/models/design_spec.py, frontend/lib/api/design-spec.ts, frontend/PRD.md).
• AuthZ layering: JWT access/refresh with token version checks and explicit admin gate (backend/src/app/api/v1/dependencies.py:123-133).
• Health model: Liveness vs readiness separation with cached dependency probes (backend/src/app/main.py:200-228).
• Idempotency and queue dedup for uploads and design-spec auto-trigger (backend/src/app/api/v1/endpoints/analysis.py, backend/src/app/services/design_spec_auto_trigger.py).
• Rich automated tests locally (e.g. backend/tests/test_design_spec_wireframe_generator.py, test_design_automation_service_repairs.py, test_auth.py) and AI runtime guardrails (backend/src/app/core/guardrails/).
• LangGraph rollout discipline documented — never cut analysis and design-spec to graph-primary in one deploy (backend/src/app/graphs/README.md:67-72).

Findings

F1 — Design-spec logic concentrated in one service

Evidence: backend/src/app/services/design_automation_service.py · Proof: module split + adjacent test files green · Owner: Backend tech lead

• Problem: Design-spec generation, validation, prompts, and repair logic live in a single ~9k-line module while LangGraph adapters still delegate into it (backend/src/app/graphs/design_spec/adapters.py), creating a change blast radius on the product's most complex subsystem.
• Why: Reviews, onboarding, and safe refactors slow down; graph migration cannot shrink complexity until the service boundary is decomposed.
• Assumptions: No equivalent logic is hidden only in notebooks or scripts outside backend/src/app/services/.
• Severity / Lenses: P1 (high) · ISO/IEC 25010 (quality model) · maintainability · tech debt
• criterion_id: MAINT-001 (architecture quality / tech debt)
• Recommendation: Define capability-based modules (inventory, definitions, wireframe/export, validation) behind stable ports; keep LangGraph nodes as thin orchestration only.

F2 — LangGraph cutover lacks safe shadow evidence

Evidence: backend/src/app/graphs/README.md:67-72 · workers/tasks.py:61-75 · design_spec_auto_trigger.py:37-50 · Proof: dry-run shadow with diff metrics · Owner: Principal architect

• Problem: Design-spec LangGraph shadow hooks explicitly skip execution because graph persistence shares live CRUD, so operators cannot compare graph vs legacy outputs before flipping LANGGRAPH_DESIGN_SPEC_ENABLED.
• Why: Enabling graph-primary without parity evidence risks silent contract or readiness regressions on DesignSpecRun states (PS-001).
• Assumptions: Production still runs legacy-primary today (flags default false in config fields reviewed).
• Severity / Lenses: P1 (high) · architecture quality · reliability · proof strength
• criterion_id: reliability
• Recommendation: Prioritize a dry-run persistence adapter, then run documented rollout order (analysis shadow → analysis primary → design-spec shadow → design-spec primary) never combined in one deploy.

F3 — PR CI omits backend tests and API contract check

Evidence: .github/workflows/check-pr.yml (lint, mypy, frontend lint/typecheck/build only) · frontend/package.json scripts · Proof: new CI jobs + branch protection · Owner: Platform / QA

• Problem: Merge gates do not execute pytest or npm run check:api-contract, so PS-001 contract drift and backend regressions can reach main despite extensive local tests under backend/tests/.
• Why: Proof strength for architecture claims rests on developer discipline, not enforced pipeline evidence.
• Assumptions: No alternate CI workflow outside .github/workflows/ was found.
• Severity / Lenses: P1 (high) · proof strength · ISO/IEC 25010 (quality model) · portability (CI)
• criterion_id: proof_strength
• Recommendation: Add path-filtered pytest (smoke + design-spec contract subset minimum) and frontend check:api-contract to PR checks; expand to fuller suites on nightly if runtime is prohibitive.

F4 — No durable architecture decisions on disk

Evidence: _context/decisions/ empty · architect-project.profile.json:147 · Proof: merged ADR for graph cutover · Owner: Principal architect

• Problem: Material forks (LangGraph vs legacy, provider fallback policy, export identity) are encoded in code and KB but not as durable decision records, increasing governance drift risk.
• Why: Future agents and teams cannot see rejected options or approval boundaries without re-reading large services.
• Assumptions: No ADRs live outside _context/decisions/ under another naming scheme.
• Severity / Lenses: P2 (medium) · proof strength · governance
• criterion_id: governance drift (proof)
• Recommendation: Record XS/S decisions as Y-statements and M/L graph cutover as ADR/tradeoff matrix under _context/decisions/ or _context/design/architecture/.

F5 — Global rate limit excludes heavy job paths

Evidence: backend/src/app/core/config.py:111-120 · Proof: authenticated abuse simulation · Owner: Security + platform

• Problem: /api/v1/analyses, /api/v1/analysis-results, and /api/v1/design-spec-runs bypass the global rate limiter while still being expensive AI-backed surfaces.
• Why: Compromised or buggy clients can enqueue unbounded work unless other limits (DAILY_ANALYSIS_LIMIT, auth) bind them.
• Assumptions: Per-user analysis limits are enforced on all entry paths (spot-checked in config, not every endpoint).
• Severity / Lenses: P2 (medium) · security · scalability
• criterion_id: SEC-002 (rate limiting / abuse)
• Recommendation: Add per-user/per-route limits on enqueue endpoints or enforce at API gateway; keep health exclusions only.

F6 — Metrics scrape may be unauthenticated in prod

Evidence: backend/src/app/main.py:248-250 · METRICS_TOKEN default empty in config.py:135-137 · docker-compose.yml passes ${METRICS_TOKEN:-} · Proof: prod Compose requires token · Owner: Platform

• Problem: When METRICS_TOKEN is unset, /metrics is intentionally open; Compose defaults leave it empty, exposing Prometheus text format on the API port unless network-restricted.
• Why: Metrics can leak route labels, job IDs, and dependency health signals useful for targeted attacks.
• Assumptions: Production does not always front metrics with an allowlisted reverse proxy (not verified live).
• Severity / Lenses: P1 (high) · security · operations
• criterion_id: SEC-001 (secrets / exposure)
• Recommendation: Require non-empty METRICS_TOKEN in non-dev environments or bind metrics to internal-only listener; mirror worker metrics (0.0.0.0:9101) behind same policy.

F7 — Design-spec runtime flags can diverge by environment

Evidence: main.py:58-73 · workers/settings.py:40-58 · _context/lessons.md (Docker preview cache keys) · Proof: deploy manifest pins flags · Owner: Ops + backend

• Problem: WIREFRAME_PREGENERATION_ENABLED, Stitch preference, and degraded-verification flags are logged at startup but easy to mismatch between local API, worker container, and developer .env vs Compose — changing preview/workbook readiness semantics (PS-001, PS-003).
• Why: Operators interpret "ready" differently per environment; incidents resemble frontend bugs but are configuration drift.
• Assumptions: No centralized feature-flag service exists yet.
• Severity / Lenses: P1 (high) · reliability · operations · PS-003 Async Artifact And Operations
• criterion_id: reliability
• Recommendation: Treat the three flags as a versioned runtime profile document checked into ops config; add a single /api/v1/config or admin diagnostic surfacing effective worker+API values.

F8 — No documented SLOs or error budgets

Evidence: Metrics and health endpoints exist; no SLO definitions in repo docs · Proof: SLO doc + alert rules · Owner: SRE / product

• Problem: Reliability claims (queue backlog, design-spec latency, export success) lack user-facing SLOs, error budgets, or alert thresholds despite Google SRE-style instrumentation hooks.
• Why: Prioritization between P1 fixes and feature work lacks measurable failure budgets.
• Assumptions: External monitoring (Grafana folder under backend/ops/grafana/) is not wired to on-call in this repo snapshot.
• Severity / Lenses: P2 (medium) · Google SRE principles · reliability
• criterion_id: reliability
• Recommendation: Define SLIs for analysis completion time, design-spec run success rate, and export latency; tie /ready degradation and queue depth to paging policies.

F9 — Worker throughput capped at six concurrent jobs

Evidence: backend/src/app/workers/settings.py:33 (WORKER_MAX_CONCURRENT_JOBS = 6) · Proof: load test at target concurrency · Owner: Platform

• Problem: ARQ worker concurrency is hard-coded, not env-tunable, while analysis and design-spec jobs are long-running LLM workloads — a single worker fleet can become the system bottleneck under concurrent users.
• Why: Horizontal scale requires more worker containers, but each still caps parallel jobs at six regardless of CPU/memory.
• Assumptions: No second queue consumer pattern is configured in Compose by default.
• Severity / Lenses: P2 (medium) · scalability · cost (idle capacity)
• criterion_id: scalability
• Recommendation: Expose concurrency via settings with safe defaults; scale workers horizontally and monitor Redis queue depth + job age percentiles.

F10 — OpenAPI/docs exposed by default

Evidence: create_application() uses FastAPI defaults (main.py:146-153); root JSON lists /docs · Proof: production build disables docs · Owner: Security

• Problem: Interactive OpenAPI and ReDoc remain available unless deployment middleware blocks them; rate limit excludes /docs paths.
• Why: Attack surface expansion and schema leakage in production environments.
• Assumptions: ENVIRONMENT=production does not automatically disable docs (no docs_url=None found).
• Severity / Lenses: P2 (medium) · OWASP ASVS · security
• criterion_id: SEC-003 (exposure)
• Recommendation: Set docs_url=None, redoc_url=None, openapi_url=None when ENVIRONMENT=production (or behind admin auth).

Recommended Next Artifact

• Next: action-summary companion artifact
• Why: Multiple P1 themes (monolith, LangGraph proof, CI gates, metrics auth, runtime flags) overlap on remediation sequencing and need deduplicated master issues before roadmap or implementation handoff.
• Default scope: F2, F3, F6, F7 (decision + proof + ops hardening wave)
• Not next: roadmap_shaping — remediation direction for graph cutover and monolith decomposition is not pre-authorized; kd-handoff-spec — no approved spec scope yet.
• Requires approval: yes — user must confirm default scope and whether LangGraph primary cutover is in or out of the next wave.
• Proof gate: CI runs pytest smoke + check:api-contract on PR; production metrics require METRICS_TOKEN or documented network deny.
• Related findings: F1–F10; blocked auto companion tradeoff_decision_support until F2 scope is approved.
• Backlog vs default scope: Full table in ## Remediation Backlog And Handoff; only Default scope IDs are committed for the current wave.
• Plan path note: n/a until kd-handoff-spec after scope approval.

Remediation Backlog And Handoff

Engine-Specific Sections

Tech debt (profile)

• Concentration in design_automation_service.py (F1) dominates maintainability risk; LangGraph adapters currently wrap rather than replace that debt.
• Positive: extensive targeted tests under backend/tests/test_design_automation_service_repairs.py and wireframe/export suites reduce regression risk during decomposition.

Security (profile)

• Trust boundaries: Browser → Next.js proxy (CSRF + cookie auth) → FastAPI → PostgreSQL/Redis → OpenAI/Stitch providers.
• Admin routes consistently use get_current_admin_user (sample: backend/src/app/api/v1/endpoints/admin.py).
• Gaps: metrics exposure (F6), docs surface (F10), rate-limit exclusions (F5). LOGIN_ACCOUNT_REQUESTS_PER_15_MINUTES: 100000 in config.py:129 reads as non-production-hardened — verify before prod.

Reliability (profile)

• Worker gating for wireframe pre-generation on READY_FOR_REVIEW only (workers/tasks.py:41-58) prevents misleading progress — good reliability pattern.
• Weakness: environment flag drift (F7) and absent SLOs (F8).

Scalability (profile)

• Separate Redis URLs for cache vs queue (config.py:88-90) supports scaling patterns.
• Bottleneck: fixed WORKER_MAX_CONCURRENT_JOBS = 6 (F9) and long Next.js proxy timeouts up to 900s for design-spec export (frontend/app/api/proxy/[...path]/route.ts:47-48) — validate hosting model (Node server vs serverless) before scale-out.

Cost / value signal

• Per-job cost attribution exists on Analysis and DesignSpecRun (total_cost_usd, token_usage fields) — good cost observability foundation.
• Cost Estimate Section Gate: not passed (no material infra pricing change proposed); environment-specific cost tables omitted.
• Risk: unconstrained parallel LLM work under excluded rate limits (F5) can raise spend without user-visible throttling.

Risks And Follow-Up

• Risks: Enabling LangGraph-primary without shadow proof (F2); merging PRs without automated pytest (F3); operators misreading readiness due to flag drift (F7).
• Required decisions: Approve default remediation wave (F2, F3, F6, F7); decide whether F1 monolith split waits for graph cutover ADR.
• Recommended Harness handoff: After action-summary + scope approval → tradeoff_decision_support for LangGraph cutover; then kd-handoff-spec for CI and metrics hardening — never kd-dev directly from this review (PRC-1).
• Deep tier auxiliary: Consider templates/risk-matrix.md companion if leadership needs ranked cross-profile risk scoring beyond this report.

Decisions, Questions, And Assumptions

Blocking Decisions

Blocking Open Questions

Working Assumptions

• Workspace under review is Scopelytics AI (architect-project.profile.json), not a separate transport/logistics product; filename slug transport-system is session artifact naming only.
• Legacy design-spec path remains primary in typical deployments until LANGGRAPH_DESIGN_SPEC_ENABLED is explicitly enabled.
• User excluded prior review/ADR/spec/plan artifacts from evidence for this session.

Non-Blocking Questions

• Should nightly CI run full backend/tests suite vs PR smoke subset?

Evidence Ledger

Evidence ledger (scope, method, commands)

Scope

• Repository root D:/tryhand-project/scopelytics-ai-powered
• Focus: runtime architecture, CI, design-spec/analysis pipelines, security middleware, project standards

Method

• Read project KB, standards PS-001–004, architect profile
• Inspected entrypoints: main.py, api/v1/__init__.py, proxy route, worker settings/tasks
• Grepped config/security patterns; counted design_automation_service.py lines (~9265)
• Reviewed .github/workflows/check-pr.yml; confirmed _context/decisions/ empty
• Did not run pytest, docker compose, or live API calls in this session

Commands (advisory — not executed as proof)

• Backend quality: ruff check, mypy, pytest (per architect-project.profile.json)
• Frontend: npm run lint, npm run build, npm run check:api-contract

Skipped checks

• Live penetration test, production metrics scrape, load test, full pytest run

Self-Check

• Evidence Packet and Normalize Pass applied; material claims cite paths or are labeled assumption/inference.
• Eight focus profiles recorded in coverage matrix.
• Findings use table mode (10 findings) with detail blocks per schema.
• No implementation or plan authored from this review.

Artifact Self-Check

• Declared engine/template used: architecture_review / base-review-report.md / Deep comprehensive variant.
• Required H2 sections present per templates/base-review-report.md.
• Findings include criterion_id where checklist applied.
• ## Recommended Next Artifact and ## Remediation Backlog And Handoff included (material findings).
• Durable path: _context/design/architecture/2026-06-04-000005-transport-system-deep-review.md
• Post-report chat brief emitted separately in session close.