Files

Chelsea 6cf35ca034 Phase 1-3 + 6: pluggable filter system, app factory, test harness

Implements the parallel.md workstreams (Agents A-E) toward REFACTOR_GOAL.md.

Phase 1 — app factory + blueprints + /api/v1:
- app.py -> create_app() factory (no module-level app); entrypoints updated
- routes/ (auth, pages, settings, admin, assets) + blueprints/api.py at /api/v1
- config.py / extensions.py / security.py extracted; services/ layer added
- endpoint names preserved so template url_for() calls keep resolving
  (static check: all 27 template url_for endpoints are defined routes)

Phase 2 — one pluggable filter system:
- filter_pipeline/registry.py: @register_stage / @register_plugin + discover_modules
- engine._init_stages() instantiates registered stages (no hardcoded dict);
  process_batch is AI-aware: only short-circuits to the AI-disabled path when
  a filterset's stages declare requires_ai, so offline filtersets run with AI off
- BaseFilterPlugin gets a consumer (stages/plugins.py); Keyword/Quality
  re-enabled via filter_config.json plugins config
- comment tree modes ported to stages/comment_filter.py + shared rules.py;
  wired into /api/v1/posts/<uuid> and /api/v1/comments/<uuid> via
  FilterEngine.filter_comments() (fails open)
- offline quality_filter filterset exercises plugins+ranker without AI
- legacy filter_lib / comment_lib / html_generation_lib / generate_html /
  active_html path deleted

Phase 3 prep — pluggable fetchers + Postgres models:
- Post / Comment SQLAlchemy models added to models.py
- migrate_content_to_db.py backfill (idempotent by uuid, batched, --dry-run)
- platforms/ fetcher registry (extension point)
- live reads/writes still go through PostService (disk JSON); cutover deferred

Phase 6 — test harness:
- pytest.ini + tests/ (conftest with in-memory SQLite fixture, no Postgres;
  stubbed polling/filter singletons)
- test_app_factory.py (route registration, no module-level app),
  test_api_contracts.py (posts/post_detail/comments/filters shape with
  monkeypatched post_service + get_filter_engine),
  test_filter_pipeline.py + test_plugin_contract.py (Flask-free; validated
  locally 12/12 incl. drop-in stage/plugin discovered with zero core edits)

Other: .gitignore added (__pycache__, data/, secrets); pytest in requirements.

Verification: py_compile clean across the project; the Flask-free filter-pipeline
and plugin-contract tests pass locally. App-factory / API-contract tests need
deps+docker to run; runtime flask routes / Auth0-repeated-create_app also gated
on docker.

Co-Authored-By: Claude <noreply@anthropic.com>

2026-07-03 02:29:46 -05:00

2.9 KiB

Raw Blame History

Parallel Work Handoff

Current status: Phase 1 is mostly complete. app.py is now a create_app() factory, /api/v1 lives in blueprints/api.py, and legacy Jinja route groups are split under routes/. Runtime verification is still pending because the local environment does not have the full Flask/Postgres stack available.

Good Parallel Workstreams

Agent A: Phase 1 Verification + Cleanup

Owns: app.py, run_app.py, start_server.py, Dockerfile, README/DEPLOYMENT.
Verify flask --app app:create_app routes and startup in a deps-installed environment.
Check Auth0 registration under repeated create_app() calls.
Confirm url_for(...) endpoints still exist for templates.
Remove stale compatibility notes from progress.md once verified.
Avoid changing filter pipeline internals.

Agent B: Phase 2 Filter Registry

Owns: filter_pipeline/, filter_config.json, filtersets.json.
Add stage/plugin registry and auto-discovery.
Replace hardcoded stage dicts in filter_pipeline/engine.py.
Bridge BaseFilterPlugin to live FilterResult handling.
Avoid editing route modules except for minimal API integration points.

Agent C: Comment Filtering Consolidation

Owns: comment_lib.py, filter_lib.py, comment-related pipeline stages.
Port comment tree modes into filter_pipeline/stages/comment_filter.py.
Wire filtered comments into /api/v1/posts/<uuid> and /api/v1/comments/<uuid>.
Add small sample-data tests if a test harness is added.
Coordinate with Agent B on registry names and stage contracts.

Agent D: Phase 3 Data Model Prep

Owns: models.py, new migration scripts, DB query services.
Design Post and Comment SQLAlchemy models matching current JSON schema.
Draft migrate_content_to_db.py backfill from data/posts and data/comments.
Do not switch live reads/writes until Phase 2 route/filter behavior is stable.

Agent E: Test Harness

Owns: tests/, pytest config, lightweight fixtures.
Add app-factory tests that assert route registration and endpoint names.
Add /api/v1 contract tests with monkeypatched post_service and get_filter_engine.
Keep tests independent of a live Postgres where possible.

Serialization Points

Do Phase 1 runtime verification before deleting any more legacy UI behavior.
Phase 2 registry work and comment filtering can happen together, but merge the registry contract first.
Phase 3 database cutover should wait until /api/v1/posts and comment filtering behavior is stable.
Phase 4 SPA work can scaffold independently, but feature parity work depends on stable /api/v1 contracts.

Shared Cautions

Preserve endpoint names used by templates until the SPA replaces them.
Do not delete Jinja templates in this pass.
Do not remove JSON file reads until Postgres backfill and DB query paths are verified.
Run at least python -m py_compile on touched Python files.
Update progress.md after each substantial change.

2.9 KiB Raw Blame History