Phase 1-3 + 6: pluggable filter system, app factory, test harness
Implements the parallel.md workstreams (Agents A-E) toward REFACTOR_GOAL.md. Phase 1 — app factory + blueprints + /api/v1: - app.py -> create_app() factory (no module-level app); entrypoints updated - routes/ (auth, pages, settings, admin, assets) + blueprints/api.py at /api/v1 - config.py / extensions.py / security.py extracted; services/ layer added - endpoint names preserved so template url_for() calls keep resolving (static check: all 27 template url_for endpoints are defined routes) Phase 2 — one pluggable filter system: - filter_pipeline/registry.py: @register_stage / @register_plugin + discover_modules - engine._init_stages() instantiates registered stages (no hardcoded dict); process_batch is AI-aware: only short-circuits to the AI-disabled path when a filterset's stages declare requires_ai, so offline filtersets run with AI off - BaseFilterPlugin gets a consumer (stages/plugins.py); Keyword/Quality re-enabled via filter_config.json plugins config - comment tree modes ported to stages/comment_filter.py + shared rules.py; wired into /api/v1/posts/<uuid> and /api/v1/comments/<uuid> via FilterEngine.filter_comments() (fails open) - offline quality_filter filterset exercises plugins+ranker without AI - legacy filter_lib / comment_lib / html_generation_lib / generate_html / active_html path deleted Phase 3 prep — pluggable fetchers + Postgres models: - Post / Comment SQLAlchemy models added to models.py - migrate_content_to_db.py backfill (idempotent by uuid, batched, --dry-run) - platforms/ fetcher registry (extension point) - live reads/writes still go through PostService (disk JSON); cutover deferred Phase 6 — test harness: - pytest.ini + tests/ (conftest with in-memory SQLite fixture, no Postgres; stubbed polling/filter singletons) - test_app_factory.py (route registration, no module-level app), test_api_contracts.py (posts/post_detail/comments/filters shape with monkeypatched post_service + get_filter_engine), test_filter_pipeline.py + test_plugin_contract.py (Flask-free; validated locally 12/12 incl. drop-in stage/plugin discovered with zero core edits) Other: .gitignore added (__pycache__, data/, secrets); pytest in requirements. Verification: py_compile clean across the project; the Flask-free filter-pipeline and plugin-contract tests pass locally. App-factory / API-contract tests need deps+docker to run; runtime flask routes / Auth0-repeated-create_app also gated on docker. Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
58
models.py
58
models.py
@@ -246,3 +246,61 @@ class Bookmark(db.Model):
|
||||
|
||||
def __repr__(self):
|
||||
return f'<Bookmark {self.post_uuid} by user {self.user_id}>'
|
||||
|
||||
|
||||
class Post(db.Model):
|
||||
"""A collected post/item, mirroring the on-disk ``data/posts/*.json`` schema.
|
||||
|
||||
Phase 3 prep: the model and ``migrate_content_to_db.py`` backfill exist, but
|
||||
live reads/writes still go through ``PostService`` (disk JSON). The cutover
|
||||
is gated on Phase 2 filter behavior being stable — see ``progress.md`` and
|
||||
``parallel.md`` (Agent D). Do not point the API at this model yet.
|
||||
"""
|
||||
|
||||
__tablename__ = 'posts'
|
||||
|
||||
uuid = db.Column(db.String(64), primary_key=True)
|
||||
external_id = db.Column(db.String(255), nullable=True, index=True)
|
||||
platform = db.Column(db.String(50), nullable=False, index=True)
|
||||
source = db.Column(db.String(500), nullable=False, default='', index=True)
|
||||
title = db.Column(db.String(500), nullable=False, default='')
|
||||
author = db.Column(db.String(255), nullable=True)
|
||||
url = db.Column(db.Text, nullable=True)
|
||||
content = db.Column(db.Text, nullable=True)
|
||||
score = db.Column(db.Integer, nullable=False, default=0)
|
||||
# Unix epoch seconds; BigInteger so far-future/large values fit.
|
||||
timestamp = db.Column(db.BigInteger, nullable=False, default=0, index=True)
|
||||
tags = db.Column(db.JSON, nullable=True)
|
||||
moderation_uuid = db.Column(db.String(64), nullable=True, index=True)
|
||||
created_at = db.Column(db.DateTime, nullable=False, default=datetime.utcnow)
|
||||
|
||||
comments = db.relationship('Comment', backref='post', lazy='dynamic')
|
||||
|
||||
def __repr__(self):
|
||||
return f'<Post {self.uuid} [{self.platform}:{self.source}]>'
|
||||
|
||||
|
||||
class Comment(db.Model):
|
||||
"""A comment on a post, mirroring ``data/comments/*.json``.
|
||||
|
||||
``parent_comment_uuid`` is a self-reference implementing the comment tree;
|
||||
null marks a top-level comment. ``comment_id`` is the platform's own id.
|
||||
"""
|
||||
|
||||
__tablename__ = 'comments'
|
||||
|
||||
uuid = db.Column(db.String(64), primary_key=True)
|
||||
post_uuid = db.Column(db.String(64), db.ForeignKey('posts.uuid'), nullable=False, index=True)
|
||||
platform = db.Column(db.String(50), nullable=True, index=True)
|
||||
parent_comment_uuid = db.Column(db.String(64), nullable=True, index=True)
|
||||
comment_id = db.Column(db.String(100), nullable=True)
|
||||
author = db.Column(db.String(255), nullable=True)
|
||||
content = db.Column(db.Text, nullable=True)
|
||||
score = db.Column(db.Integer, nullable=False, default=0)
|
||||
timestamp = db.Column(db.BigInteger, nullable=False, default=0, index=True)
|
||||
depth = db.Column(db.Integer, nullable=False, default=0)
|
||||
moderation_uuid = db.Column(db.String(64), nullable=True, index=True)
|
||||
created_at = db.Column(db.DateTime, nullable=False, default=datetime.utcnow)
|
||||
|
||||
def __repr__(self):
|
||||
return f'<Comment {self.uuid} on {self.post_uuid}>'
|
||||
|
||||
Reference in New Issue
Block a user