Lamont/README.md

# LLM Bot Framework

A small, cloneable Python 3.11 framework for Discord bots that turn natural
language into validated feature commands. Flask is the application boundary,
PostgreSQL stores users and durable work, and feature packages keep their bot
handler, API routes, prompt, migrations, and scheduled jobs together.

The included reminders package is both usable and a reference implementation.

## What is included

- Discord DM adapter with passwordless provider identities
- Two-stage, OpenAI-compatible command routing and parsing
- Deterministic feature discovery through `modules/*/register(registry)`
- Flask API with short-lived JWTs, user API keys, and scoped service keys
- Transactional PostgreSQL migrations and a small parameterized SQL layer
- Lease-based scheduled jobs and a durable outbound-message queue
- One-time, daily, and weekly reminders with IANA timezone support
- Docker Compose, focused Ruff checks, pytest coverage, and CI

## Configuration and credential safety

On a fresh checkout, copy the root template only if `.env` does not already
exist. All processes load this root file.

```powershell
if (-not (Test-Path .env)) { Copy-Item .env.example .env }
```

For an existing installation, do not overwrite either environment file.
Manually merge the needed values from a legacy `config/.env` into the root
`.env`, verify the application with the root file, rotate any live or
previously shared credentials, and only then retire the legacy copy.

Replace every placeholder before exposing the service. The real `.env` is
ignored by Git and Docker and must stay local. Never commit it or bake it into
an image.

If a populated environment file or image has ever been shared, assume its
values are exposed. Rotate the Discord token, database password, JWT secret,
bot service key, and model-provider key before deployment. Updating `.gitignore`
does not remove secrets from existing history.

Important settings:

| Variable | Purpose |
| --- | --- |
| `DB_HOST`, `DB_PORT`, `DB_NAME`, `DB_USER`, `DB_PASS` | PostgreSQL connection |
| `JWT_SECRET` | Signs short-lived user sessions |
| `BOT_API_KEY` | Discord adapter service key; use at least 32 random characters |
| `BOT_API_KEY_SCOPES` | Normally `discord:session,outbox:claim,outbox:deliver` |
| `DISCORD_BOT_TOKEN` | Discord application token |
| `DISCORD_ENROLLMENT_MODE` | `allowlist` by default, or explicit `open` enrollment |
| `DISCORD_ALLOWLIST` | Comma- or whitespace-separated Discord user IDs |
| `API_URL` | API URL used by the bot; Compose default is `http://app:5000` |
| `OPENROUTER_API_KEY`, `OPENROUTER_BASE_URL` | OpenAI-compatible provider access |
| `AI_MODEL`, `AI_CONFIG_PATH` | Optional model and parser-config overrides |
| `DEFAULT_TIMEZONE` | IANA timezone for new users; defaults to `UTC` |
| `JOB_*`, `OUTBOX_*` | Worker polling, batch, and lease settings |

Generate secrets with a password manager or a cryptographic random generator.
The API registers `BOT_API_KEY` as a hashed service key on its first non-live
request; the Discord process must use the same raw value.

## Run with Docker

After configuring `.env`, start the complete stack:

```bash
docker compose up --build
```

The API is available at `http://localhost:8080`. The `migrate` one-shot service
must finish before the API, scheduler, and bot start. To work without Discord or
a live model provider, start only the database, migrations, API, and scheduler:

```bash
docker compose up --build db migrate app scheduler
```

Useful checks:

```bash
curl http://localhost:8080/health/live
curl http://localhost:8080/health/ready
docker compose logs -f app scheduler bot
```

`/health/live` only confirms that the API process is responding.
`/health/ready` also checks PostgreSQL.

## Run locally

Create a Python 3.11 virtual environment and install development dependencies:

```powershell
python -m venv .venv
.\.venv\Scripts\Activate.ps1
python -m pip install -r requirements-dev.txt
```

Install or connect to PostgreSQL 16, create the database/user named in `.env`,
and make sure it accepts connections at `DB_HOST` and `DB_PORT`. The example
uses `DB_HOST=127.0.0.1` for local development; Compose overrides that value
inside its containers. On PowerShell, check that the configured port responds:

```powershell
Test-NetConnection 127.0.0.1 -Port 5432
```

The Windows PostgreSQL installer includes pgAdmin, which can create the `app`
login role and `app` database from the example configuration. If `psql` is on
PATH, the equivalent commands are:

```powershell
psql -U postgres -c "CREATE ROLE app LOGIN PASSWORD 'replace-with-a-database-password';"
psql -U postgres -c "CREATE DATABASE app OWNER app;"
```

Use a different password in both `.env` and the role command. Then run each
process in its own terminal:

```bash
python -m core.migrations upgrade
python -m api.main
python -m scheduler.daemon
python -m bot.bot
```

The local API command uses Flask's development server. The Docker image uses
Gunicorn.

## Database migrations

Core migrations live in `config/migrations`; each feature may add numbered SQL
files under `modules/<name>/migrations`. Core migrations run first, followed by
feature namespaces in alphabetical and numeric order.

```bash
python -m core.migrations status
python -m core.migrations upgrade
```

Upgrades run transactionally under a PostgreSQL advisory lock. Applied
migrations are recorded with checksums. Do not edit an applied migration: a
changed checksum stops startup. Add the next numbered migration instead.

Back up an existing database before the first upgrade. Legacy password users
remain supported. To link one safely to Discord after migrating:

```bash
python -m core.manage link-discord --discord-id 123456789012345678 --username alice
python -m core.manage link-discord --discord-id 123456789012345678 --user-uuid 2d36fc2b-5145-4fca-a677-858ca6a36e2d
```

Choose exactly one of `--username` or `--user-uuid`. The command refuses to
move a Discord identity that is already linked to a different user.

## Authentication

The HTTP examples below use Compose's `http://localhost:8080`. For a locally
started Flask process, use `http://localhost:5000` instead.

There are three authentication paths:

- A password user registers and logs in for a short-lived JWT.
- That JWT can create, list, and revoke user API keys. The full key is returned
  only at creation; only its prefix and hash are stored.
- The Discord adapter uses the scoped `BOT_API_KEY` to exchange a stable
  Discord ID for a short-lived user JWT. It never receives a user's password.

Register and log in:

```bash
curl -X POST http://localhost:8080/api/register \
  -H "Content-Type: application/json" \
  -d '{"username":"alice","password":"replace-this-password","timezone":"America/Chicago"}'

curl -X POST http://localhost:8080/api/login \
  -H "Content-Type: application/json" \
  -d '{"username":"alice","password":"replace-this-password"}'
```

Use the returned JWT as `JWT_TOKEN` to create a long-lived user key:

```bash
curl -X POST http://localhost:8080/api/keys \
  -H "Authorization: Bearer JWT_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"name":"local-cli"}'
```

Save the response's `key` immediately; it is shown once. A user API key acts as
its owner for user routes, but cannot manage API keys or delete the account.

```bash
curl http://localhost:8080/api/reminders \
  -H "Authorization: Bearer USER_API_KEY"

curl http://localhost:8080/api/keys \
  -H "Authorization: Bearer JWT_TOKEN"

curl -X DELETE http://localhost:8080/api/keys/KEY_UUID \
  -H "Authorization: Bearer JWT_TOKEN"
```

Revocation takes effect on the next request. Optional `expires_at` values use
ISO-8601 timestamps.

### Discord enrollment

Enrollment is closed by default. Add stable Discord user IDs to the allowlist:

```dotenv
DISCORD_ENROLLMENT_MODE=allowlist
DISCORD_ALLOWLIST=123456789012345678,234567890123456789
```

To let any Discord user create a provider-backed account explicitly set:

```dotenv
DISCORD_ENROLLMENT_MODE=open
```

Display names are metadata only. Existing accounts are never matched
automatically by a mutable Discord display name; use `core.manage link-discord`
when joining a legacy password account.

## Feature modules

Every immediate, non-private package under `modules/` must export
`register(registry)`. Packages load alphabetically. Startup fails on malformed
registration, duplicate module names, duplicate command types, or duplicate job
types, so configuration errors surface early.

A typical package looks like this:

```text
modules/tasks/
  __init__.py
  commands.py
  prompts.py
  routes.py
  service.py
  migrations/
    0001_tasks.sql
```

Its `__init__.py` owns registration:

```python
from modules.tasks.commands import handleTask, validateTask
from modules.tasks.prompts import TASK_PROMPT
from modules.tasks.routes import registerRoutes
from modules.tasks.service import runFollowUp


def register(registry):
    registry.describe("Create and manage tasks")
    registry.register_command(
        "task",
        handleTask,
        prompt=TASK_PROMPT,
        validator=validateTask,
        description="Create, list, and complete tasks",
        help_text=["add buy milk to my tasks", "list my tasks"],
    )
    registry.register_routes(registerRoutes)
    registry.register_job("tasks.follow_up", runFollowUp)
```

`register_routes` and `register_job` are optional. A command handler keeps a
small async interface:

```python
async def handleTask(context, parsed):
    result, status = await context.api.request("get", "/api/tasks")
    if status == 200:
        await context.reply(f"You have {len(result['tasks'])} tasks.")
```

`CommandContext` exposes `user_uuid`, `discord_user_id`, `timezone`, an async
authenticated `api` client, and `await reply(content)`. The API client's
`request(method, endpoint, data=None, params=None)` returns `(JSON body, HTTP
status)` and refreshes the Discord-issued JWT once after a 401. This keeps
feature handlers independent of discord.py message objects.

The parser first routes against registered command descriptions, then invokes
the selected module's focused prompt and Python validator. Prompt templates may
use `{user_input}`, `{history_context}`, `{current_time}`, and `{timezone}`;
literal JSON braces are preserved. Validators return a list of human-readable
errors, allowing the parser to retry malformed model output.

### Extension walkthrough

1. Create a package under `modules/`; no central import list is needed.
2. Add the feature's tables as `migrations/0001_<name>.sql`. Use the next number
   for later schema changes.
3. Put business rules and owner-scoped database operations in `service.py`.
4. Add authenticated Flask routes with `api.security.requireUser()` and derive
   ownership from `flask.g.user_uuid`, never request JSON.
5. Add a focused prompt, validator, and `async handler(context, parsed)`.
6. Register the command, optional routes, and optional job types in
   `register(registry)`.
7. Run migration status, tests, and Ruff before starting the services.

Use the reminders package as the concrete example for transactions, ownership,
recurrence, jobs, and outbound delivery.

## Reminders

In a Discord DM, users can speak naturally:

```text
remind me tomorrow at 9 AM to call the dentist
remind me every day at 8 AM to take my medication
remind me every Friday at 5 PM to submit my timesheet
list my reminders
cancel reminder <id from the list>
set my timezone to America/Chicago
```

The REST API supports the same data:

```bash
curl -X POST http://localhost:8080/api/reminders \
  -H "Authorization: Bearer USER_TOKEN_OR_KEY" \
  -H "Content-Type: application/json" \
  -d '{"message":"check the oven","run_at":"2099-01-02T18:00:00-06:00","recurrence":{"frequency":"daily","interval":1}}'

curl http://localhost:8080/api/reminders \
  -H "Authorization: Bearer USER_TOKEN_OR_KEY"

curl -X DELETE http://localhost:8080/api/reminders/REMINDER_UUID \
  -H "Authorization: Bearer USER_TOKEN_OR_KEY"
```

Omit `recurrence` for a one-time reminder. Supported frequencies are `daily`
and `weekly`, with an optional positive `interval`. Times are stored in UTC,
while recurring reminders preserve the user's local wall-clock time across
daylight-saving changes. User timezones must be IANA names such as `UTC` or
`America/Chicago`.

After downtime, a due occurrence is delivered once by the scheduler's normal
processing path and the next future occurrence is scheduled; missed intervals
are not replayed. Cancelling marks the reminder and its pending job/outbound
records together where possible.

## Jobs and outbound delivery

The scheduler atomically claims due jobs with PostgreSQL row locks and
`SKIP LOCKED`. Jobs and outbound messages use leases so another worker can
recover work after a crash. Defaults are a five-second poll, five-minute lease,
three attempts, and exponential retries beginning at 30 seconds and capped at
15 minutes.

Delivery is **at least once**. Idempotency keys prevent ordinary duplicate job
and outbox creation, but a process can crash after Discord accepts a message and
before the API records success. Consumers and message wording should tolerate a
rare duplicate.

Discord claims only `discord_dm` outbox records through scoped internal routes,
sends them, and reports `sent` or `retry`. The existing webhook and ntfy helpers
remain available for custom features but are not the reminder delivery path.

## Tests and lint

```bash
python -m pip install -r requirements-dev.txt
ruff check .
pytest
```

Local pytest reports coverage without failing when PostgreSQL-dependent tests
are skipped. CI enforces at least 80% coverage across runtime framework code
with those integration tests enabled; only the thin administrative CLI wrappers
are excluded from that metric. Tests mock Discord and model-provider calls;
GitHub Actions runs both the Python 3.11 suite against PostgreSQL 16 and a
Compose startup smoke test.

## License

MIT