Implements the parallel.md workstreams (Agents A-E) toward REFACTOR_GOAL.md. Phase 1 — app factory + blueprints + /api/v1: - app.py -> create_app() factory (no module-level app); entrypoints updated - routes/ (auth, pages, settings, admin, assets) + blueprints/api.py at /api/v1 - config.py / extensions.py / security.py extracted; services/ layer added - endpoint names preserved so template url_for() calls keep resolving (static check: all 27 template url_for endpoints are defined routes) Phase 2 — one pluggable filter system: - filter_pipeline/registry.py: @register_stage / @register_plugin + discover_modules - engine._init_stages() instantiates registered stages (no hardcoded dict); process_batch is AI-aware: only short-circuits to the AI-disabled path when a filterset's stages declare requires_ai, so offline filtersets run with AI off - BaseFilterPlugin gets a consumer (stages/plugins.py); Keyword/Quality re-enabled via filter_config.json plugins config - comment tree modes ported to stages/comment_filter.py + shared rules.py; wired into /api/v1/posts/<uuid> and /api/v1/comments/<uuid> via FilterEngine.filter_comments() (fails open) - offline quality_filter filterset exercises plugins+ranker without AI - legacy filter_lib / comment_lib / html_generation_lib / generate_html / active_html path deleted Phase 3 prep — pluggable fetchers + Postgres models: - Post / Comment SQLAlchemy models added to models.py - migrate_content_to_db.py backfill (idempotent by uuid, batched, --dry-run) - platforms/ fetcher registry (extension point) - live reads/writes still go through PostService (disk JSON); cutover deferred Phase 6 — test harness: - pytest.ini + tests/ (conftest with in-memory SQLite fixture, no Postgres; stubbed polling/filter singletons) - test_app_factory.py (route registration, no module-level app), test_api_contracts.py (posts/post_detail/comments/filters shape with monkeypatched post_service + get_filter_engine), test_filter_pipeline.py + test_plugin_contract.py (Flask-free; validated locally 12/12 incl. drop-in stage/plugin discovered with zero core edits) Other: .gitignore added (__pycache__, data/, secrets); pytest in requirements. Verification: py_compile clean across the project; the Flask-free filter-pipeline and plugin-contract tests pass locally. App-factory / API-contract tests need deps+docker to run; runtime flask routes / Auth0-repeated-create_app also gated on docker. Co-Authored-By: Claude <noreply@anthropic.com>
230 lines
6.8 KiB
Markdown
230 lines
6.8 KiB
Markdown
# BalanceBoard
|
|
|
|
A Reddit-style content aggregator that collects posts from multiple platforms (Reddit, Hacker News, RSS feeds) and presents them in a unified, customizable feed.
|
|
|
|
## Features
|
|
|
|
- **Multi-Platform Support**: Collect content from Reddit, Hacker News, and RSS feeds
|
|
- **Automated Polling**: Background service polls sources at configurable intervals
|
|
- **User Authentication**: Local accounts with bcrypt password hashing and Auth0 OAuth support
|
|
- **Anonymous Browsing**: Browse public feed without creating an account
|
|
- **Password Reset**: Secure token-based password reset mechanism
|
|
- **Customizable Feeds**: Filter and customize content based on your preferences
|
|
- **Admin Panel**: Manage polling sources, view logs, and configure the system
|
|
- **Modern UI**: Card-based interface with clean, responsive design
|
|
|
|
## Quick Start
|
|
|
|
### Prerequisites
|
|
|
|
- Python 3.12+
|
|
- PostgreSQL database
|
|
- Docker (for containerized deployment)
|
|
|
|
### Local Development
|
|
|
|
1. **Clone the repository**
|
|
```bash
|
|
git clone https://git.scorpi.us/chelsea/balanceboard.git
|
|
cd balanceboard
|
|
```
|
|
|
|
2. **Set up environment**
|
|
```bash
|
|
python3 -m venv venv
|
|
source venv/bin/activate
|
|
pip install -r requirements.txt
|
|
```
|
|
|
|
3. **Configure environment variables**
|
|
```bash
|
|
cp .env.example .env
|
|
# Edit .env with your database credentials and settings
|
|
```
|
|
|
|
4. **Initialize the database**
|
|
```bash
|
|
python3 -c "from app import create_app; from database import db; app = create_app(); app.app_context().push(); db.create_all()"
|
|
```
|
|
|
|
5. **Run migrations (if needed)**
|
|
```bash
|
|
python3 migrate_password_reset.py
|
|
```
|
|
|
|
6. **Start the application**
|
|
```bash
|
|
python3 app.py
|
|
```
|
|
|
|
7. **Access the application**
|
|
- Open browser to `http://localhost:5000`
|
|
- Create an account or browse anonymously
|
|
|
|
### Docker Deployment
|
|
|
|
1. **Build the image**
|
|
```bash
|
|
docker build -t git.scorpi.us/chelsea/balanceboard:latest .
|
|
```
|
|
|
|
2. **Push to registry**
|
|
```bash
|
|
docker push git.scorpi.us/chelsea/balanceboard:latest
|
|
```
|
|
|
|
3. **Deploy with docker-compose**
|
|
```bash
|
|
docker-compose up -d
|
|
```
|
|
|
|
See [DEPLOYMENT.md](DEPLOYMENT.md) for detailed deployment instructions.
|
|
|
|
## Configuration
|
|
|
|
### Platform Sources
|
|
|
|
Configure available platforms and communities in `platform_config.json`:
|
|
|
|
```json
|
|
{
|
|
"reddit": {
|
|
"name": "Reddit",
|
|
"communities": [
|
|
{
|
|
"id": "programming",
|
|
"name": "r/programming",
|
|
"description": "Computer programming"
|
|
}
|
|
]
|
|
}
|
|
}
|
|
```
|
|
|
|
### Polling Configuration
|
|
|
|
Admins can configure polling sources via the Admin Panel:
|
|
- **Platform**: reddit, hackernews, or rss
|
|
- **Source ID**: Subreddit name, or RSS feed URL
|
|
- **Poll Interval**: How often to check for new content (in minutes)
|
|
- **Max Posts**: Maximum posts to collect per poll
|
|
- **Fetch Comments**: Whether to collect comments
|
|
- **Priority**: low, medium, or high
|
|
|
|
### Environment Variables
|
|
|
|
Key environment variables (see `.env.example`):
|
|
- `DATABASE_URL`: PostgreSQL connection string
|
|
- `SECRET_KEY`: Flask secret key for sessions
|
|
- `AUTH0_DOMAIN`: Auth0 domain (if using OAuth)
|
|
- `AUTH0_CLIENT_ID`: Auth0 client ID
|
|
- `AUTH0_CLIENT_SECRET`: Auth0 client secret
|
|
|
|
## Architecture
|
|
|
|
### Components
|
|
|
|
- **Flask Web Server** (`app.py`): Main application server
|
|
- **Polling Service** (`polling_service.py`): Background scheduler for data collection
|
|
- **Data Collection** (`data_collection.py`, `data_collection_lib.py`): Platform-specific data fetchers
|
|
- **Database Models** (`models.py`): SQLAlchemy ORM models
|
|
- **User Service** (`user_service.py`): User authentication and management
|
|
|
|
### Database Schema
|
|
|
|
- **users**: User accounts with authentication
|
|
- **poll_sources**: Configured polling sources
|
|
- **poll_logs**: History of polling activities
|
|
- **user_sessions**: Active user sessions
|
|
|
|
### Data Flow
|
|
|
|
1. Polling service checks enabled sources at configured intervals
|
|
2. Data collection fetchers retrieve posts from platforms
|
|
3. Posts are normalized to a common schema and stored in `data/posts/`
|
|
4. Web interface displays posts from the feed
|
|
5. Users can filter, customize, and interact with content
|
|
|
|
## API Endpoints
|
|
|
|
### Public Routes
|
|
- `GET /`: Main feed (anonymous or authenticated)
|
|
- `GET /login`: Login page
|
|
- `POST /login`: Authenticate user
|
|
- `GET /register`: Registration page
|
|
- `POST /register`: Create new account
|
|
- `GET /password-reset-request`: Request password reset
|
|
- `POST /password-reset-request`: Send reset link
|
|
- `GET /password-reset/<token>`: Reset password form
|
|
- `POST /password-reset/<token>`: Update password
|
|
|
|
### Authenticated Routes
|
|
- `GET /settings`: User settings
|
|
- `GET /logout`: Log out
|
|
|
|
### Admin Routes
|
|
- `GET /admin`: Admin panel
|
|
- `GET /admin/polling`: Manage polling sources
|
|
- `POST /admin/polling/add`: Add new source
|
|
- `POST /admin/polling/update`: Update source settings
|
|
- `POST /admin/polling/poll`: Manually trigger poll
|
|
|
|
## Development
|
|
|
|
### Project Structure
|
|
|
|
```
|
|
balanceboard/
|
|
├── app.py # Main Flask application
|
|
├── polling_service.py # Background polling service
|
|
├── data_collection.py # Data collection orchestration
|
|
├── data_collection_lib.py # Platform-specific fetchers
|
|
├── models.py # Database models
|
|
├── user_service.py # User management
|
|
├── database.py # Database setup
|
|
├── platform_config.json # Platform configurations
|
|
├── filtersets.json # Content filter definitions
|
|
├── templates/ # Jinja2 templates
|
|
├── static/ # Static assets (CSS, JS)
|
|
├── themes/ # UI themes
|
|
├── data/ # Data storage
|
|
│ ├── posts/ # Collected posts
|
|
│ ├── comments/ # Collected comments
|
|
│ └── moderation/ # Moderation data
|
|
├── requirements.txt # Python dependencies
|
|
├── Dockerfile # Docker image definition
|
|
├── docker-compose.yml # Docker composition
|
|
├── README.md # This file
|
|
└── DEPLOYMENT.md # Deployment instructions
|
|
```
|
|
|
|
### Adding a New Platform
|
|
|
|
1. Add platform config to `platform_config.json`
|
|
2. Implement fetcher in `data_collection_lib.py`:
|
|
- `fetchers.getPlatformData()`
|
|
- `converters.platform_to_schema()`
|
|
- `builders.build_platform_url()`
|
|
3. Update routing in `getData()` function
|
|
4. Test data collection
|
|
5. Add to available sources in admin panel
|
|
|
|
## Contributing
|
|
|
|
1. Fork the repository
|
|
2. Create a feature branch
|
|
3. Make your changes
|
|
4. Test thoroughly
|
|
5. Submit a pull request
|
|
|
|
See [DEPLOYMENT.md](DEPLOYMENT.md) for commit and issue management guidelines.
|
|
|
|
## License
|
|
|
|
This is a personal project by Chelsea. All rights reserved.
|
|
|
|
## Support
|
|
|
|
For issues and feature requests, please use the issue tracker at:
|
|
https://git.scorpi.us/chelsea/balanceboard/issues
|