Phase 1-3 + 6: pluggable filter system, app factory, test harness

Implements the parallel.md workstreams (Agents A-E) toward REFACTOR_GOAL.md.

Phase 1 — app factory + blueprints + /api/v1:
- app.py -> create_app() factory (no module-level app); entrypoints updated
- routes/ (auth, pages, settings, admin, assets) + blueprints/api.py at /api/v1
- config.py / extensions.py / security.py extracted; services/ layer added
- endpoint names preserved so template url_for() calls keep resolving
  (static check: all 27 template url_for endpoints are defined routes)

Phase 2 — one pluggable filter system:
- filter_pipeline/registry.py: @register_stage / @register_plugin + discover_modules
- engine._init_stages() instantiates registered stages (no hardcoded dict);
  process_batch is AI-aware: only short-circuits to the AI-disabled path when
  a filterset's stages declare requires_ai, so offline filtersets run with AI off
- BaseFilterPlugin gets a consumer (stages/plugins.py); Keyword/Quality
  re-enabled via filter_config.json plugins config
- comment tree modes ported to stages/comment_filter.py + shared rules.py;
  wired into /api/v1/posts/<uuid> and /api/v1/comments/<uuid> via
  FilterEngine.filter_comments() (fails open)
- offline quality_filter filterset exercises plugins+ranker without AI
- legacy filter_lib / comment_lib / html_generation_lib / generate_html /
  active_html path deleted

Phase 3 prep — pluggable fetchers + Postgres models:
- Post / Comment SQLAlchemy models added to models.py
- migrate_content_to_db.py backfill (idempotent by uuid, batched, --dry-run)
- platforms/ fetcher registry (extension point)
- live reads/writes still go through PostService (disk JSON); cutover deferred

Phase 6 — test harness:
- pytest.ini + tests/ (conftest with in-memory SQLite fixture, no Postgres;
  stubbed polling/filter singletons)
- test_app_factory.py (route registration, no module-level app),
  test_api_contracts.py (posts/post_detail/comments/filters shape with
  monkeypatched post_service + get_filter_engine),
  test_filter_pipeline.py + test_plugin_contract.py (Flask-free; validated
  locally 12/12 incl. drop-in stage/plugin discovered with zero core edits)

Other: .gitignore added (__pycache__, data/, secrets); pytest in requirements.

Verification: py_compile clean across the project; the Flask-free filter-pipeline
and plugin-contract tests pass locally. App-factory / API-contract tests need
deps+docker to run; runtime flask routes / Auth0-repeated-create_app also gated
on docker.

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
2026-07-03 02:29:46 -05:00
parent cdba720a1c
commit 6cf35ca034
64 changed files with 3902 additions and 3776 deletions

View File

@@ -0,0 +1,90 @@
"""Plugin/stage contract test (Phase 6).
A drop-in stage and plugin, defined ONLY in this test module via the public
``@register_stage`` / ``@register_plugin`` decorators, are picked up by the
engine with zero edits to core files. This is the pluggability guarantee: a
new filter behavior is a new module + a config entry, never an edit to
``engine.py``.
"""
from filter_pipeline.engine import FilterEngine
from filter_pipeline.models import FilterResult
from filter_pipeline.plugins.base import BaseFilterPlugin
from filter_pipeline.registry import (
get_plugin_class,
get_stage_class,
register_plugin,
register_stage,
)
from filter_pipeline.stages.base_stage import BaseStage
@register_stage("sentinel_dropin_stage")
class SentinelStage(BaseStage):
"""Drop-in stage that tags any result it sees."""
def get_name(self):
return "Sentinel"
def process(self, post, result):
result.tags.append("sentinel_ran")
return result
@register_plugin("sentinel_dropin_plugin")
class SentinelPlugin(BaseFilterPlugin):
"""Drop-in plugin: never rejects, returns a fixed score."""
def get_name(self):
return "SentinelPlugin"
def should_filter(self, post, context=None):
return False
def score(self, post, context=None):
return 0.9
def test_dropin_stage_is_registered():
assert get_stage_class("sentinel_dropin_stage") is SentinelStage
def test_dropin_plugin_is_registered():
assert get_plugin_class("sentinel_dropin_plugin") is SentinelPlugin
def test_engine_instantiates_dropin_stage():
eng = FilterEngine("filter_config.json", "filtersets.json")
eng._init_stages()
assert "sentinel_dropin_stage" in eng._stages
stage = eng._stages["sentinel_dropin_stage"]
# Running the stage through the contract it claims to implement works.
result = FilterResult(post_uuid="x", passed=True, score=0.5)
out = stage.process({"uuid": "x"}, result)
assert "sentinel_ran" in out.tags
def test_dropin_stage_can_be_selected_in_a_filterset(tmp_path):
"""A filterset that lists the drop-in stage actually runs it.
Builds a throwaway config + filterset on disk so no core file is edited.
"""
import json
cfg = tmp_path / "cfg.json"
cfg.write_text(json.dumps({
"ai": {"enabled": False},
"cache": {"enabled": False},
"pipeline": {"default_stages": ["sentinel_dropin_stage"], "enable_parallel": False},
"plugins": {"enabled": [], "configs": {}},
}))
fs = tmp_path / "fs.json"
fs.write_text(json.dumps({"custom": {"post_rules": {}, "comment_rules": {}}}))
eng = FilterEngine(str(cfg), str(fs))
eng._init_stages()
results = eng.process_batch(
[{"uuid": "p1", "title": "t", "content": "", "score": 0, "timestamp": 0}],
"custom",
)
assert "sentinel_ran" in results[0].tags