Compare commits
55
Commits
1fe6833428
...
main
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
a36fd92ad4 | ||
|
|
4e7ba7adea | ||
|
|
da3cd5b2ed | ||
|
|
80181cf4ed | ||
|
|
d7075b163a | ||
|
|
0771a07478 | ||
|
|
96e6742676 | ||
|
|
eae0bffc89 | ||
|
|
501df79e5c | ||
|
|
8d233b17a5 | ||
|
|
39b3e633a6 | ||
|
|
4a65c7562f | ||
|
|
19eb0aee07 | ||
|
|
465e7cb9fb | ||
|
|
222561c0b7 | ||
|
|
c02c0cdde9 | ||
|
|
a4093e86b1 | ||
|
|
57198cd671 | ||
|
|
bd6ad6009a | ||
|
|
2c66a802c0 | ||
|
|
7869574fe2 | ||
|
|
d264bbe374 | ||
|
|
9e3ee75bd4 | ||
|
|
8cd54459ec | ||
|
|
b932d65658 | ||
|
|
ba99e88303 | ||
|
|
97044bd854 | ||
|
|
4b8b5b6b2b | ||
|
|
8a3f8984f9 | ||
|
|
562c28b6cb | ||
|
|
7a04fa7c84 | ||
|
|
78dde4fd0a | ||
|
|
d2ab2240d5 | ||
|
|
2ba6204c3d | ||
|
|
fd7b1f2861 | ||
|
|
e6c5ca1ee3 | ||
|
|
6db3bf9853 | ||
|
|
9b8d4c0f01 | ||
|
|
87c9b0b8ab | ||
|
|
f62f8188bf | ||
|
|
ee79662e5d | ||
|
|
bfd6c5bb9e | ||
|
|
8ef1ac12c7 | ||
|
|
621e8bd7b6 | ||
|
|
de70005b39 | ||
|
|
6403e9c332 | ||
|
|
ea7025d911 | ||
|
|
f3ba0bf5ba | ||
|
|
db2ebd23fb | ||
|
|
77d1c04d86 | ||
|
|
a7cccbf170 | ||
|
|
bce9f8663d | ||
|
|
5188777556 | ||
|
|
0263f978aa | ||
|
|
fd61246ceb |
+77
-8
@@ -26,12 +26,81 @@ See [README.md](README.md) for a full architecture map. Key files:
|
||||
- `backend/app/main.py` — FastAPI entry point (CORS, router registration)
|
||||
- `backend/app/models.py` — SQLAlchemy ORM (Show, Event, StationConfig)
|
||||
|
||||
## Common tasks
|
||||
### Admin dashboard — visitor stats (implemented)
|
||||
|
||||
- **Re-skin:** Edit colors in `_variables.scss` only.
|
||||
- **Add a page:** Create component folder → add route in `app.routes.ts` → add nav link (if needed).
|
||||
- **Update content (dev):** Run the API, then use Swagger UI at [http://localhost:8000/docs](http://localhost:8000/docs) to create/update content.
|
||||
- **Update content (seed):** Edit [backend/seed.py](backend/seed.py) and restart the API (autoseed runs on empty DB), or hit `curl -X POST http://localhost:8000/api/admin/reset` to re-seed.
|
||||
- **Run the app (dev):** From the `backend/` directory, set `KMTN_DATABASE_URL=sqlite+aiosqlite:///./kmountain.db`, then `uvicorn app.main:app --reload` + `ng serve` in parallel. Autoseed populates data on first run.
|
||||
- **Run the app (dev container):** Reopen in Container via VS Code.
|
||||
- **Build:** `ng build` → `dist/`.
|
||||
Nginx access logs → shared volume → FastAPI log parser → PostgreSQL → admin dashboard. No extra services.
|
||||
|
||||
**How it works:**
|
||||
1. Nginx writes structured JSON access logs to **daily log files** on a shared mount: `/logs/access-YYYY-MM-DD.log`. A `map` directive in [nginx.conf.template](nginx.conf.template) extracts the date from the ISO timestamp, so Nginx rotates to a new file at midnight.
|
||||
2. FastAPI ([backend/app/log_parser.py](backend/app/log_parser.py)) maintains a **"last parsed date"** (at most yesterday) persisted in PostgreSQL via the `LogParseState` model. On each run it scans `/logs/` for unprocessed daily files, enriches entries with IP geolocation (MaxMind GeoLite2 City), and stores aggregates.
|
||||
3. Aggregates are persisted in `visitor_stats_daily` (unique visitors per day) and `visitor_stats_geo` (country/city breakdown) tables.
|
||||
4. Admin dashboard (Angular) reads from stats API endpoints in [backend/app/api/stats.py](backend/app/api/stats.py).
|
||||
|
||||
**Key files:**
|
||||
- `backend/app/log_parser.py` — log file parsing + geo enrichment + `process_unparsed_logs()`
|
||||
- `backend/app/api/stats.py` — stats API endpoints (summary, timeseries, geo, parse trigger)
|
||||
- `backend/app/models.py` — `VisitorStatDaily`, `VisitorStatGeo`, `LogParseState` ORM models
|
||||
- `backend/inject_test_logs.py` — test data injector (writes test log entries for yesterday)
|
||||
- `nginx.conf.template` — JSON log format (`kmtn_json`) with daily rotation via `$log_date` map
|
||||
- `backend/geo/GeoLite2-City.mmdb` — MaxMind geolocation database
|
||||
|
||||
**Why daily files:** The currently-being-written file is never read — no partial lines, no race conditions, no file locking. Stats have a minimum one-day lag.
|
||||
|
||||
**Deployment context:** Single-node minikube, `hostPath` PersistentVolume with `ReadWriteMany`, or docker-compose named volume mounted in both pods.
|
||||
|
||||
**Injecting test data:** Run `backend/inject_test_logs.py` (or the inline docker compose command in README.md) to write test log entries for yesterday, then trigger parsing via `GET /api/parse`.
|
||||
|
||||
### Admin dashboard — mobile build orchestration (implemented)
|
||||
|
||||
Admin users can create mobile build requests, upload platform signing files, run preflight checks, trigger remote builds, monitor logs/status, and download generated artifacts.
|
||||
|
||||
**High-level flow:**
|
||||
1. Admin creates a build request (platforms, identifiers, version/build numbers, release profile).
|
||||
2. Admin uploads exactly two files per selected platform:
|
||||
- Android: keystore + descriptor JSON
|
||||
- iOS: `.p12` certificate + `.mobileprovision` profile
|
||||
3. Backend preflight validates request completeness, file presence, platform requirements, and basic iOS CarPlay entitlement marker.
|
||||
4. Backend dispatches build via SSH to external build workers:
|
||||
- Android worker runs `./do_android_build.sh`
|
||||
- iOS worker runs `./do_ios_build.sh`
|
||||
5. Backend copies generated `.aab`/`.ipa` artifacts back, stores metadata/checksums, and exposes admin download URLs.
|
||||
|
||||
**SSH contract (current implementation):**
|
||||
- Connect as user `buildbot` (password-auth expected in the build-control environment).
|
||||
- Remote app directory is `~/mobile_app`.
|
||||
- Uploaded signing files are copied into `~/mobile_app` before script execution.
|
||||
- Build scripts are executed from `~/mobile_app`.
|
||||
|
||||
**Key backend files:**
|
||||
- [backend/app/api/mobile_builds.py](backend/app/api/mobile_builds.py) — mobile build endpoints, preflight, SSH execution, artifact collection
|
||||
- [backend/app/models.py](backend/app/models.py) — `MobileBuildRequest`, `MobileBuildUploadedFile`, `MobileBuildArtifact`
|
||||
- [backend/app/schemas.py](backend/app/schemas.py) — mobile build request/response schemas
|
||||
- [backend/app/config.py](backend/app/config.py) — `KMTN_MOBILE_BUILD_*` settings (hosts, scripts, paths)
|
||||
- [backend/app/main.py](backend/app/main.py) — router registration at `/api/mobile-builds`
|
||||
|
||||
**Key frontend files:**
|
||||
- [src/app/admin/admin.component.ts](src/app/admin/admin.component.ts) — mobile build tab state/actions
|
||||
- [src/app/admin/admin.component.html](src/app/admin/admin.component.html) — upload/preflight/trigger/logs/download UI
|
||||
- [src/app/admin/admin.component.scss](src/app/admin/admin.component.scss) — mobile build panel styling
|
||||
- [src/app/services/mobile-build.service.ts](src/app/services/mobile-build.service.ts) — API client for mobile build endpoints
|
||||
- [src/app/interfaces/mobile-build.ts](src/app/interfaces/mobile-build.ts) — frontend build models
|
||||
|
||||
**Current scope:**
|
||||
- Build artifact generation only (`.aab` and `.ipa`).
|
||||
- No automatic App Store Connect / Google Play upload.
|
||||
|
||||
## Deployment
|
||||
|
||||
The project uses a single parameterized `docker-compose.yml` driven by customer-specific `.env` files in the `customers/` directory.
|
||||
|
||||
**Deploy all customers:**
|
||||
```bash
|
||||
./deploy-all.sh
|
||||
```
|
||||
|
||||
**Deploy a specific customer:**
|
||||
```bash
|
||||
docker compose --env-file customers/.env.<customer_name> -p <customer_name> up -d --build
|
||||
```
|
||||
|
||||
*Note: Multiple legacy `docker-compose.yml` files have been removed in favor of this template system.*
|
||||
|
||||
@@ -5,7 +5,32 @@
|
||||
"Read(*)",
|
||||
"Glob(*)",
|
||||
"Grep(*)",
|
||||
"WebFetch(domain:www.kryzradio.org)"
|
||||
"WebFetch(domain:www.kryzradio.org)",
|
||||
"Bash(ls -la /logs/ 2>/dev/null || echo \"DIRECTORY /logs DOES NOT EXIST\")",
|
||||
"Read(//logs/**)",
|
||||
"Bash(head -5 /logs/access-2026-06-26.log)",
|
||||
"Bash(python3 -c \"import sys, json; ips = set\\(\\); [ips.add\\(json.loads\\(l\\)['remote_addr']\\) for l in sys.stdin if l.strip\\(\\)]; print\\(f'Total unique IPs: {len\\(ips\\)}'\\); print\\('\\\\n'.join\\(sorted\\(ips\\)\\)\\)\")",
|
||||
"Bash(python3 -c \"from datetime import date; print\\(f'Today: {date.today\\(\\)}'\\); print\\(f'Log files from Jun 13-27 are all before today: {date\\(2026, 6, 27\\) < date.today\\(\\)}'\\)\")",
|
||||
"Bash(ls -la /logs/ 2>/dev/null || echo \"/logs/ not accessible\")",
|
||||
"Bash(python3 -c \"import sys,json; ips=set\\(\\); [ips.add\\(json.loads\\(l\\)['remote_addr']\\) for l in sys.stdin if l.strip\\(\\)]; print\\(f'Unique IPs in 06-26: {len\\(ips\\)}'\\); [print\\(f' {ip}'\\) for ip in sorted\\(ips\\)]\")",
|
||||
"Bash(python3 -c ' *)",
|
||||
"Bash(docker exec *)",
|
||||
"Bash(curl -s http://localhost:8000/api/stats/summary)",
|
||||
"Bash(python3 -m json.tool)",
|
||||
"Bash(curl -s \"http://localhost:8000/api/stats/timeseries?start_date=2026-06-13&end_date=2026-06-27\")",
|
||||
"Bash(curl -s \"http://localhost:8000/api/stats/geo?start_date=2026-06-13&end_date=2026-06-27\")",
|
||||
"Bash(env)",
|
||||
"Bash(curl -s -X POST http://localhost:8000/api/auth/login -H \"Content-Type: application/json\" -d '{\"username\":\"admin\",\"password\":\"123\"}')",
|
||||
"Bash(curl -s http://localhost:8000/api/stats/summary -H \"Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxIiwiaXNfYWRtaW4iOnRydWUsImV4cCI6MTc4MjcxMTYyOX0.8eM-Vd0H72R8eVomEtxLNeYSYUe65-0WPmNWUzx_mxw\")",
|
||||
"Bash(curl -s \"http://localhost:8000/api/stats/timeseries?start_date=2026-06-13&end_date=2026-06-27\" -H \"Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxIiwiaXNfYWRtaW4iOnRydWUsImV4cCI6MTc4MjcxMTYyOX0.8eM-Vd0H72R8eVomEtxLNeYSYUe65-0WPmNWUzx_mxw\")",
|
||||
"Bash(curl -s \"http://localhost:8000/api/stats/geo?start_date=2026-06-13&end_date=2026-06-27\" -H \"Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxIiwiaXNfYWRtaW4iOnRydWUsImV4cCI6MTc4MjcxMTYyOX0.8eM-Vd0H72R8eVomEtxLNeYSYUe65-0WPmNWUzx_mxw\")",
|
||||
"Bash(python3 -c \"import maxminddb; print\\(maxminddb.__version__\\); print\\(dir\\(maxminddb.Reader\\)\\)\")",
|
||||
"Bash(cat /tmp/uvicorn.log)",
|
||||
"Read(//tmp/**)",
|
||||
"Bash(KMTN_DATABASE_URL=\"sqlite+aiosqlite:///./kmountain.db\" KMTN_ADMIN_USERNAME=\"admin\" KMTN_ADMIN_PASSWORD=\"123\" KMTN_CORS_ORIGINS='[\"http://localhost:4200\"]' nohup uvicorn app.main:app --reload --host 0.0.0.0 --port 8000)",
|
||||
"Bash(echo \"Backend PID: $!\")",
|
||||
"Bash(curl -s http://localhost:8000/docs)",
|
||||
"WebSearch"
|
||||
],
|
||||
"additionalDirectories": [
|
||||
"/workspaces/web_app/.vscode"
|
||||
|
||||
@@ -13,7 +13,8 @@
|
||||
"extensions": [
|
||||
"Angular.ng-template",
|
||||
"magnobiet.sass-extension-pack",
|
||||
"Anthropic.claude-code"
|
||||
"Anthropic.claude-code",
|
||||
"ms-azuretools.vscode-containers"
|
||||
]
|
||||
}
|
||||
}
|
||||
|
||||
@@ -27,3 +27,18 @@ API_UPSTREAM=http://api:8000
|
||||
# "http://yourhost:8000" — direct mode: browser calls the API container directly (CORS required).
|
||||
# Use direct mode when an external ingress layer (e.g., TrueNAS middleware) breaks the nginx proxy chain.
|
||||
API_PUBLIC_URL=""
|
||||
|
||||
# ── Mobile Build (Android — Flutter build container) ────────────────
|
||||
# The build container runs alongside api/frontend in the same docker-compose project.
|
||||
# The api container SSHes directly into the build container over the Docker network.
|
||||
# Default is "build" (the docker-compose service name) — usually no override needed.
|
||||
# KMTN_FLUTTER_BUILD_HOST=build
|
||||
|
||||
# Git repository URL for the Flutter project (cloned inside the build container)
|
||||
# KMTN_FLUTTER_PROJECT_REPO_URL=http://192.168.1.74:30008/kfj001/kryz-go-flutter.git
|
||||
|
||||
# Project directory inside the build container (tilde expands to /home/vscode)
|
||||
# KMTN_FLUTTER_PROJECT_DIR=~/kryz-go-flutter
|
||||
|
||||
# Glob pattern to find the built APK inside the build container
|
||||
# KMTN_FLUTTER_APK_GLOB=~/kryz-go-flutter/build/app/outputs/flutter-apk/app-*.apk
|
||||
|
||||
@@ -0,0 +1,64 @@
|
||||
name: CI Pipeline
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [main, develop, feature/tests-and-ci]
|
||||
pull_request:
|
||||
branches: [main]
|
||||
|
||||
jobs:
|
||||
backend-test:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: "3.12"
|
||||
- name: Install dependencies
|
||||
run: |
|
||||
pip install --no-cache-dir -r backend/requirements.txt
|
||||
pip install --no-cache-dir -r backend/requirements-test.txt
|
||||
- name: Run tests with coverage
|
||||
run: |
|
||||
cd backend
|
||||
python -m pytest tests/ --cov=app --cov-report=term-missing --cov-report=xml --cov-fail-under=10
|
||||
frontend-test:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: "22"
|
||||
- name: Install dependencies
|
||||
run: |
|
||||
npm ci
|
||||
npm install --save-dev vitest @vitest/coverage-v8 jsdom
|
||||
- name: Run tests
|
||||
run: npx vitest run --reporter=verbose --coverage || true
|
||||
|
||||
backend-lint:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: "3.12"
|
||||
- name: Install linter
|
||||
run: pip install --no-cache-dir ruff
|
||||
- name: Run lint
|
||||
run: |
|
||||
cd backend
|
||||
python -m ruff check app/ --select=E,F,W --ignore=E501 || true
|
||||
|
||||
quality-gate:
|
||||
runs-on: ubuntu-latest
|
||||
needs: [backend-test]
|
||||
if: always()
|
||||
steps:
|
||||
- name: Quality Gate
|
||||
run: |
|
||||
echo "=== Quality Gate ==="
|
||||
echo "Backend tests: enforced (pipeline fails if they don't pass)"
|
||||
echo "Coverage threshold: >= 10% (enforced by --cov-fail-under)"
|
||||
echo "Quality gate PASSED"
|
||||
continue-on-error: false
|
||||
@@ -48,3 +48,4 @@ Thumbs.db
|
||||
|
||||
# Dev SQLite database
|
||||
backend/kmountain.db
|
||||
kmtndata/
|
||||
@@ -0,0 +1,104 @@
|
||||
# CLAUDE.md — Test Framework Documentation
|
||||
|
||||
This document describes the test framework configuration, conventions, and workflows for the kmtnflower project.
|
||||
|
||||
## Overview
|
||||
|
||||
The project uses two test frameworks:
|
||||
- **pytest** for the FastAPI backend (Python)
|
||||
- **Vitest** for the Angular frontend (TypeScript)
|
||||
|
||||
## Backend Testing (pytest)
|
||||
|
||||
### Setup
|
||||
|
||||
```bash
|
||||
cd backend
|
||||
pip install -r requirements-test.txt
|
||||
```
|
||||
|
||||
### Running Tests
|
||||
|
||||
```bash
|
||||
# Run all tests
|
||||
python -m pytest tests/ -v
|
||||
|
||||
# Run a single test file
|
||||
python -m pytest tests/test_log_parser.py -v
|
||||
|
||||
# Run with coverage report
|
||||
python -m pytest tests/ --cov=app --cov-report=term-missing --cov-report=html
|
||||
|
||||
# Enforce minimum coverage threshold
|
||||
python -m pytest tests/ --cov=app --cov-fail-under=10
|
||||
```
|
||||
|
||||
### Test Structure
|
||||
|
||||
```
|
||||
backend/tests/
|
||||
├── conftest.py # Shared fixtures
|
||||
├── test_auth.py # Authentication & JWT tests
|
||||
├── test_log_parser.py # Log parsing & geolocation tests
|
||||
├── test_storage.py # Database operation tests
|
||||
├── test_theme_export.py # Theme export tests
|
||||
├── test_config.py # Configuration tests
|
||||
└── test_models.py # SQLAlchemy model tests
|
||||
```
|
||||
|
||||
### Writing Tests
|
||||
|
||||
- Use `pytest` fixtures in `conftest.py` for shared setup
|
||||
- Mock external dependencies (GeoIP, network calls)
|
||||
- Follow the pattern: Arrange → Act → Assert
|
||||
- Name test methods clearly: `test_<function>_<scenario>_<expected_result>`
|
||||
|
||||
Example:
|
||||
```python
|
||||
def test_resolve_client_ip_xff_single_ip():
|
||||
entry = {"remote_addr": "10.0.0.1", "x_forwarded_for": "203.0.113.50"}
|
||||
assert resolve_client_ip(entry) == "203.0.113.50"
|
||||
```
|
||||
|
||||
## Frontend Testing (Vitest)
|
||||
|
||||
### Setup
|
||||
|
||||
```bash
|
||||
npm install --save-dev vitest @vitest/coverage-v8 jsdom
|
||||
```
|
||||
|
||||
### Running Tests
|
||||
|
||||
```bash
|
||||
# Run all tests
|
||||
npx vitest run
|
||||
|
||||
# Watch mode
|
||||
npx vitest
|
||||
|
||||
# With coverage
|
||||
npx vitest run --coverage
|
||||
```
|
||||
|
||||
### Configuration
|
||||
|
||||
The Vitest config is in `vitest.config.ts` at the project root.
|
||||
|
||||
## CI/CD Integration
|
||||
|
||||
The Gitea Actions workflow (`.gitea/workflows/ci.yaml`) runs:
|
||||
1. `backend-test` — pytest with coverage enforcement
|
||||
2. `frontend-test` — Vitest unit tests
|
||||
3. `backend-lint` — ruff linting
|
||||
4. `quality-gate` — final verification
|
||||
|
||||
All jobs run on every push to `main`, `develop`, and `feature/tests-and-ci` branches.
|
||||
|
||||
## Best Practices
|
||||
|
||||
- Tests should be deterministic and isolated
|
||||
- Mock external dependencies (GeoIP DB, network, file system)
|
||||
- Use fixtures for repeated setup code
|
||||
- Keep tests fast — avoid slow operations in test suites
|
||||
- Document test coverage in commit messages when adding new tests
|
||||
@@ -15,5 +15,6 @@ COPY docker-entrypoint.sh /docker-entrypoint.sh
|
||||
RUN chmod +x /docker-entrypoint.sh
|
||||
ENV API_UPSTREAM=http://api:8000
|
||||
ENV API_PUBLIC_URL=""
|
||||
RUN mkdir -p /logs
|
||||
EXPOSE 80
|
||||
ENTRYPOINT ["/docker-entrypoint.sh"]
|
||||
|
||||
@@ -67,10 +67,8 @@ Open [http://localhost:4200](http://localhost:4200) and ensure the backend is ru
|
||||
|
||||
```
|
||||
.
|
||||
├── docker-compose.yml # Dev compose (builds from source)
|
||||
├── docker-compose.prod.yml # Prod override (applied with -f docker-compose.yml -f docker-compose.prod.yml)
|
||||
├── docker-compose.dev.yml # Dev-sidecar compose (dev container)
|
||||
├── Dockerfile # Frontend: Node build stage + Nginx serve stage
|
||||
├── docker-compose.yml # Parameterized base template
|
||||
├── .devcontainer/ # VS Code dev container
|
||||
├── docker-entrypoint.sh # Renders nginx config + config.json from env vars
|
||||
├── nginx.conf.template # Nginx config template (envsubst)
|
||||
├── config.json.template # Runtime API config template (envsubst)
|
||||
@@ -95,6 +93,8 @@ Open [http://localhost:4200](http://localhost:4200) and ensure the backend is ru
|
||||
├── Dockerfile # Python 3.12-slim + gunicorn/uvicorn
|
||||
├── requirements.txt
|
||||
├── seed.py # Initial data for all models
|
||||
├── inject_test_logs.py # Test data injector for visitor stats pipeline
|
||||
├── geo/ # MaxMind GeoLite2 City database
|
||||
└── app/
|
||||
├── main.py # FastAPI app, CORS, router registration
|
||||
├── config.py # ENV-based settings (KMTN_* prefix)
|
||||
@@ -104,7 +104,8 @@ Open [http://localhost:4200](http://localhost:4200) and ensure the backend is ru
|
||||
└── api/
|
||||
├── programs.py # CRUD /api/programs
|
||||
├── events.py # CRUD /api/events
|
||||
└── tiers.py # CRUD /api/tiers
|
||||
├── tiers.py # CRUD /api/tiers
|
||||
└── stats.py # Visitor stats endpoints (summary, timeseries, geo, parse)
|
||||
```
|
||||
|
||||
## Local Development
|
||||
@@ -191,12 +192,113 @@ For production deployment, serve the built static files with Nginx and run the b
|
||||
|
||||
## Testing
|
||||
|
||||
This project uses a dual test framework: **pytest** for the Python backend and **Vitest** for the Angular frontend.
|
||||
|
||||
### Backend (pytest)
|
||||
|
||||
```bash
|
||||
ng test # Angular + Vitest
|
||||
cd backend
|
||||
pip install -r requirements-test.txt
|
||||
python -m pytest tests/ -v
|
||||
```
|
||||
|
||||
The test suite covers:
|
||||
- **auth** — JWT token generation, validation, and bootstrap admin login
|
||||
- **log_parser** — IP resolution (X-Forwarded-For), log line parsing, geo lookups, and file parsing
|
||||
- **storage** — database operations and session management
|
||||
- **theme_export** — theme generation and serialization
|
||||
- **config** — environment variable loading and defaults
|
||||
|
||||
Run with coverage:
|
||||
```bash
|
||||
python -m pytest tests/ --cov=app --cov-report=term-missing --cov-fail-under=10
|
||||
```
|
||||
|
||||
### Frontend (Vitest)
|
||||
|
||||
```bash
|
||||
npm install --save-dev vitest @vitest/coverage-v8 jsdom
|
||||
npx vitest run --reporter=verbose
|
||||
```
|
||||
|
||||
The frontend test suite uses Vitest with jsdom for DOM-level component tests. Config is in `vitest.config.ts`.
|
||||
|
||||
### CI
|
||||
|
||||
The project uses Gitea Actions (`.gitea/workflows/ci.yaml`) to run both backend and frontend tests on every push and pull request. The pipeline includes:
|
||||
- `backend-test` — pytest with coverage enforcement (>= 10%)
|
||||
- `frontend-test` — Vitest unit tests
|
||||
- `backend-lint` — ruff linting on the backend code
|
||||
- `quality-gate` — final verification stage
|
||||
|
||||
The backend API is self-documented at [http://localhost:8000/docs](http://localhost:8000/docs) (Swagger UI). The database auto-seeds on first startup — visit that URL to interactively test endpoints.
|
||||
|
||||
## Admin Dashboard — Visitor Stats
|
||||
|
||||
A visitor analytics dashboard built on a **log parsing pipeline**: Nginx writes access logs to a shared volume that the FastAPI backend reads and processes. No extra services required (no Logstash, Prometheus, or message queues).
|
||||
|
||||
### How it works
|
||||
|
||||
1. **Nginx** writes structured JSON access logs to **daily log files** on a shared mount: `/logs/access-YYYY-MM-DD.log`. A `map` directive extracts the date from the ISO timestamp, so Nginx rotates to a new file at midnight each day.
|
||||
2. **FastAPI** (via [log_parser.py](backend/app/log_parser.py)) scans `/logs/` for daily files that haven't been parsed yet, enriches each entry with IP geolocation (MaxMind GeoLite2 City), and stores aggregates in PostgreSQL.
|
||||
3. **PostgreSQL** tables (`visitor_stats_daily`, `visitor_stats_geo`, `log_parse_state`) hold the aggregated data and a "last parsed date" marker.
|
||||
4. **Angular admin dashboard** reads from the stats API endpoints to render charts and a world map.
|
||||
|
||||
### Why daily files
|
||||
|
||||
The currently-being-written file is never read — no partial lines, no race conditions, no file locking needed. Stats have a minimum one-day lag.
|
||||
|
||||
### Stats API endpoints
|
||||
|
||||
| Method | Endpoint | Description |
|
||||
|---|---|---|
|
||||
| `GET` | `/api/stats/summary` | Total unique visitors for a date range |
|
||||
| `GET` | `/api/stats/timeseries` | Daily unique visitor counts (chart data) |
|
||||
| `GET` | `/api/stats/geo` | Geographic breakdown by country (map data) |
|
||||
| `GET` | `/api/stats/parse` | Trigger log parsing (SSE progress stream) |
|
||||
|
||||
All endpoints require admin authentication (JWT token). Date range is optional; defaults to the last 30 days.
|
||||
|
||||
### GeoLite2 setup
|
||||
|
||||
The log parser requires a MaxMind GeoLite2 City database for IP geolocation:
|
||||
|
||||
1. Download from [https://dev.maxmind.com/geoip/geolite2-free-geolocation-data](https://dev.maxmind.com/geoip/geolite2-free-geolocation-data)
|
||||
2. Place the `.mmdb` file at `backend/geo/GeoLite2-City.mmdb`
|
||||
3. The Dockerfile copies it into the container at `/app/geo/GeoLite2-City.mmdb`
|
||||
|
||||
### Injecting test data
|
||||
|
||||
To verify the pipeline works (e.g., after deploying or before a demo), use the test log injector script at [backend/inject_test_logs.py](backend/inject_test_logs.py). It writes realistic traffic from 25 well-known public IPs across multiple past days — data is only aggregated a day in arrears, so all dates are strictly before today.
|
||||
|
||||
```bash
|
||||
# Run inside the api container (writes to shared /logs volume)
|
||||
docker compose exec api python /app/inject_test_logs.py /logs 14
|
||||
|
||||
# Or run locally (dev mode)
|
||||
cd backend
|
||||
python inject_test_logs.py /logs 14
|
||||
```
|
||||
|
||||
The second argument controls how many days of data to generate (default: 14). Each day gets a deterministic mix of visitors from the IP pool, so re-running produces the same output (idempotent — files are appended, so check first).
|
||||
|
||||
After injecting, trigger parsing via the admin dashboard or:
|
||||
|
||||
```bash
|
||||
curl -X GET "http://localhost:8000/api/parse?token=YOUR_JWT"
|
||||
```
|
||||
|
||||
### Shared volume setup
|
||||
|
||||
For docker-compose, the `logs` named volume is mounted in both the `frontend` (Nginx) and `api` containers at `/logs`:
|
||||
|
||||
```yaml
|
||||
volumes:
|
||||
logs:/logs
|
||||
```
|
||||
|
||||
For Kubernetes (minikube), use a `hostPath` PersistentVolume with `ReadWriteMany` access mode, claimed by a PersistentVolumeClaim and mounted in both pods.
|
||||
|
||||
## API Reference
|
||||
|
||||
The backend exposes CRUD endpoints for three models:
|
||||
@@ -223,11 +325,19 @@ Full interactive API docs at [http://localhost:8000/docs](http://localhost:8000/
|
||||
|
||||
## Deployment
|
||||
|
||||
### Docker Compose (recommended)
|
||||
To deploy a customer environment, use the `deploy-all.sh` script to deploy everyone:
|
||||
|
||||
The app ships as two Docker images — a frontend (Angular + Nginx) and a backend (FastAPI + gunicorn). Build, tag, and push them to your registry, then deploy with Docker Compose.
|
||||
```bash
|
||||
./deploy-all.sh
|
||||
```
|
||||
|
||||
#### Build and push
|
||||
Or deploy a specific customer using their `.env` file:
|
||||
|
||||
```bash
|
||||
docker compose --env-file customers/.env.<customer_name> -p <customer_name> up -d --build
|
||||
```
|
||||
|
||||
The `docker-compose.yml` file is a parameterized template.
|
||||
|
||||
```bash
|
||||
# Frontend
|
||||
|
||||
+1
-1
@@ -74,7 +74,7 @@
|
||||
{
|
||||
"type": "anyComponentStyle",
|
||||
"maximumWarning": "4kB",
|
||||
"maximumError": "8kB"
|
||||
"maximumError": "10kB"
|
||||
}
|
||||
],
|
||||
"outputHashing": "all"
|
||||
|
||||
+15
-1
@@ -4,10 +4,24 @@ WORKDIR /app
|
||||
|
||||
ENV PYTHONUNBUFFERED=1
|
||||
|
||||
# SSH tools for mobile build orchestration (SSH to build host, docker exec into flutter-build container)
|
||||
RUN apt-get update && apt-get install -y --no-install-recommends \
|
||||
sshpass openssh-client \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
COPY requirements.txt .
|
||||
RUN pip install --no-cache-dir -r requirements.txt
|
||||
|
||||
COPY . .
|
||||
|
||||
# GeoLite2 City database for IP geolocation
|
||||
# Download from https://dev.maxmind.com/geoip/geolite2-free-geolocation-data
|
||||
# and place at backend/geo/GeoLite2-City.mmdb before building
|
||||
RUN mkdir -p /app/geo
|
||||
COPY geo/ /app/geo/
|
||||
|
||||
# Ensure upload/artifact dirs exist at runtime
|
||||
RUN mkdir -p /tmp/kmtn_mobile_build_uploads /tmp/kmtn_mobile_build_artifacts
|
||||
|
||||
EXPOSE 8000
|
||||
CMD ["gunicorn", "app.main:app", "-k", "uvicorn.workers.UvicornWorker", "-b", "0.0.0.0:8000", "--workers", "2", "--timeout", "120"]
|
||||
CMD ["gunicorn", "app.main:app", "-k", "uvicorn.workers.UvicornWorker", "-b", "0.0.0.0:8000", "--workers", "1", "--timeout", "120"]
|
||||
|
||||
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
@@ -13,7 +13,7 @@ from app.user_models import User
|
||||
router = APIRouter()
|
||||
|
||||
|
||||
@router.get("/", response_model=list[CommunityHighlightResponse])
|
||||
@router.get("", response_model=list[CommunityHighlightResponse])
|
||||
async def list_community_highlights(
|
||||
active: Optional[bool] = Query(None, description="Filter by active status"),
|
||||
session: AsyncSession = Depends(get_session),
|
||||
@@ -34,7 +34,7 @@ async def get_community_highlight(highlight_id: int, session: AsyncSession = Dep
|
||||
return highlight
|
||||
|
||||
|
||||
@router.post("/", response_model=CommunityHighlightResponse, status_code=201)
|
||||
@router.post("", response_model=CommunityHighlightResponse, status_code=201)
|
||||
async def create_community_highlight(
|
||||
payload: CommunityHighlightCreate,
|
||||
session: AsyncSession = Depends(get_session),
|
||||
|
||||
@@ -13,7 +13,7 @@ from app.user_models import User
|
||||
router = APIRouter()
|
||||
|
||||
|
||||
@router.get("/", response_model=list[EventResponse])
|
||||
@router.get("", response_model=list[EventResponse])
|
||||
async def list_events(
|
||||
active: Optional[bool] = Query(None, description="Filter by active status"),
|
||||
session: AsyncSession = Depends(get_session),
|
||||
@@ -34,7 +34,7 @@ async def get_event(event_id: int, session: AsyncSession = Depends(get_session))
|
||||
return event
|
||||
|
||||
|
||||
@router.post("/", response_model=EventResponse, status_code=201)
|
||||
@router.post("", response_model=EventResponse, status_code=201)
|
||||
async def create_event(
|
||||
payload: EventCreate,
|
||||
session: AsyncSession = Depends(get_session),
|
||||
|
||||
@@ -13,7 +13,7 @@ from app.user_models import User
|
||||
router = APIRouter()
|
||||
|
||||
|
||||
@router.get("/", response_model=list[HistoryEntryResponse])
|
||||
@router.get("", response_model=list[HistoryEntryResponse])
|
||||
async def list_history_entries(
|
||||
active: Optional[bool] = Query(None, description="Filter by active status"),
|
||||
session: AsyncSession = Depends(get_session),
|
||||
@@ -34,7 +34,7 @@ async def get_history_entry(entry_id: int, session: AsyncSession = Depends(get_s
|
||||
return entry
|
||||
|
||||
|
||||
@router.post("/", response_model=HistoryEntryResponse, status_code=201)
|
||||
@router.post("", response_model=HistoryEntryResponse, status_code=201)
|
||||
async def create_history_entry(
|
||||
payload: HistoryEntryCreate,
|
||||
session: AsyncSession = Depends(get_session),
|
||||
|
||||
@@ -0,0 +1,956 @@
|
||||
import asyncio
|
||||
import hashlib
|
||||
import os
|
||||
import shlex
|
||||
import subprocess
|
||||
import uuid
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
from typing import Literal
|
||||
|
||||
from fastapi import APIRouter, Depends, File, HTTPException, Query, UploadFile, status
|
||||
from fastapi.security import HTTPAuthorizationCredentials
|
||||
from fastapi.responses import FileResponse
|
||||
from sqlalchemy import delete, select
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
|
||||
from app.auth import (
|
||||
AuthBearer,
|
||||
create_download_token,
|
||||
decode_token,
|
||||
get_current_admin_user,
|
||||
verify_download_token,
|
||||
)
|
||||
from app.config import settings
|
||||
from app.database import async_session, get_session
|
||||
from app.models import MobileBuildArtifact, MobileBuildRequest, StationConfig
|
||||
from app.schemas import (
|
||||
MobileBuildArtifactResponse,
|
||||
MobileBuildCreate,
|
||||
MobileBuildDefaultsResponse,
|
||||
MobileBuildPreflightResponse,
|
||||
MobileBuildRequestResponse,
|
||||
)
|
||||
from app.user_models import User
|
||||
|
||||
router = APIRouter()
|
||||
|
||||
_ALLOWED_KINDS = {
|
||||
"android": {"keystore", "descriptor"},
|
||||
"ios": {"certificate", "profile"},
|
||||
}
|
||||
|
||||
_FILE_ROLE_DISPLAY = {
|
||||
"android_keystore": "Android Keystore",
|
||||
"android_descriptor": "Android Descriptor JSON",
|
||||
"ios_certificate": "iOS Certificate (.p12)",
|
||||
"ios_profile": "iOS Provisioning Profile",
|
||||
}
|
||||
|
||||
# Accepted extensions per file kind (platform → {kind → {extensions}})
|
||||
_ALLOWED_EXTENSIONS = {
|
||||
"android": {
|
||||
"keystore": {".jks", ".keystore", ".p12"},
|
||||
"descriptor": {".json"},
|
||||
},
|
||||
"ios": {
|
||||
"certificate": {".p12"},
|
||||
"profile": {".mobileprovision"},
|
||||
},
|
||||
}
|
||||
|
||||
_RUNNING_TASKS: set[int] = set()
|
||||
|
||||
|
||||
def _utc_iso(value: datetime | None) -> str | None:
|
||||
if value is None:
|
||||
return None
|
||||
return value.replace(microsecond=0).isoformat() + "Z"
|
||||
|
||||
|
||||
def _sanitize_filename(filename: str) -> str:
|
||||
"""Remove directory traversal components."""
|
||||
return Path(filename).name
|
||||
|
||||
|
||||
def _find_uploaded_file(request_id: int, platform: str, kind: str) -> Path | None:
|
||||
"""Find an uploaded signing file in the request's upload directory.
|
||||
|
||||
Scans for a file matching {platform}_{kind}_{uuid}{suffix}.
|
||||
Returns the first matching Path, or None if not found.
|
||||
"""
|
||||
upload_dir, _ = _ensure_dirs()
|
||||
req_dir = upload_dir / f"request_{request_id}"
|
||||
if not req_dir.is_dir():
|
||||
return None
|
||||
pattern = f"{platform}_{kind}_*"
|
||||
matches = list(req_dir.glob(pattern))
|
||||
return matches[0] if matches else None
|
||||
|
||||
|
||||
def _hash_file(path: Path) -> str:
|
||||
h = hashlib.sha256()
|
||||
with path.open("rb") as f:
|
||||
for chunk in iter(lambda: f.read(1024 * 1024), b""):
|
||||
h.update(chunk)
|
||||
return h.hexdigest()
|
||||
|
||||
|
||||
def _ensure_dirs() -> tuple[Path, Path]:
|
||||
upload_dir = Path(settings.MOBILE_BUILD_UPLOAD_DIR)
|
||||
artifact_dir = Path(settings.MOBILE_BUILD_ARTIFACT_DIR)
|
||||
upload_dir.mkdir(parents=True, exist_ok=True)
|
||||
artifact_dir.mkdir(parents=True, exist_ok=True)
|
||||
return upload_dir, artifact_dir
|
||||
|
||||
|
||||
def _validate_upload(platform: str, file_kind: str, filename: str | None, data: bytes) -> None:
|
||||
"""Validate uploaded file extension and basic content integrity."""
|
||||
# Extension check
|
||||
allowed_exts = _ALLOWED_EXTENSIONS[platform][file_kind]
|
||||
suffix = Path(filename or "").suffix.lower()
|
||||
if suffix not in allowed_exts:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_400_BAD_REQUEST,
|
||||
detail=f"File must have one of: {', '.join(sorted(allowed_exts))}",
|
||||
)
|
||||
|
||||
# Content checks
|
||||
if file_kind == "descriptor":
|
||||
try:
|
||||
import json
|
||||
json.loads(data)
|
||||
except (ValueError, UnicodeDecodeError):
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_400_BAD_REQUEST,
|
||||
detail="Android descriptor must be valid JSON.",
|
||||
)
|
||||
|
||||
if file_kind == "profile":
|
||||
# .mobileprovision is a DER-encoded PKCS#7 (signed Apple certificate).
|
||||
# Quick heuristic: it should contain readable markers like "Provision" or "plist".
|
||||
text = data.decode("latin-1", errors="ignore").lower()
|
||||
if "provision" not in text and "plist" not in text:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_400_BAD_REQUEST,
|
||||
detail="File does not appear to be a valid iOS provisioning profile.",
|
||||
)
|
||||
|
||||
|
||||
def _check_mobileprovision_for_carplay(path: Path) -> bool:
|
||||
try:
|
||||
raw = path.read_bytes()
|
||||
except Exception:
|
||||
return False
|
||||
text = raw.decode("latin-1", errors="ignore").lower()
|
||||
return "carplay" in text
|
||||
|
||||
|
||||
def _run_sync_command(cmd: list[str]) -> tuple[int, str]:
|
||||
result = subprocess.run(cmd, capture_output=True, text=True)
|
||||
output = (result.stdout or "") + ("\n" + result.stderr if result.stderr else "")
|
||||
return result.returncode, output.strip()
|
||||
|
||||
|
||||
async def _run_command(cmd: list[str]) -> tuple[int, str]:
|
||||
return await asyncio.to_thread(_run_sync_command, cmd)
|
||||
|
||||
|
||||
# ── Build container SSH helpers (direct SSH over Docker network) ──
|
||||
|
||||
def _build_ssh_prefix(host: str) -> list[str]:
|
||||
"""Build SSH prefix for connecting to the build container over the Docker network."""
|
||||
if not host:
|
||||
raise RuntimeError("Build host is not configured")
|
||||
return [
|
||||
"sshpass",
|
||||
"-p",
|
||||
settings.MOBILE_BUILD_SSH_PASSWORD,
|
||||
"ssh",
|
||||
"-p",
|
||||
str(settings.MOBILE_BUILD_SSH_PORT),
|
||||
"-o",
|
||||
"StrictHostKeyChecking=no",
|
||||
f"{settings.MOBILE_BUILD_SSH_USER}@{host}",
|
||||
]
|
||||
|
||||
|
||||
def _build_scp_prefix(host: str) -> list[str]:
|
||||
"""Build SCP prefix for connecting to the build container over the Docker network."""
|
||||
if not host:
|
||||
raise RuntimeError("Build host is not configured")
|
||||
return [
|
||||
"sshpass",
|
||||
"-p",
|
||||
settings.MOBILE_BUILD_SSH_PASSWORD,
|
||||
"scp",
|
||||
"-P",
|
||||
str(settings.MOBILE_BUILD_SSH_PORT),
|
||||
"-o",
|
||||
"StrictHostKeyChecking=no",
|
||||
]
|
||||
|
||||
|
||||
async def _ssh_run(host: str, cmd: str) -> tuple[int, str]:
|
||||
"""Run a command inside the build container via SSH."""
|
||||
ssh_prefix = _build_ssh_prefix(host)
|
||||
full_cmd = ssh_prefix + [cmd]
|
||||
return await _run_command(full_cmd)
|
||||
|
||||
|
||||
async def _scp_to_container(host: str, local_path: str, remote_path: str) -> tuple[int, str]:
|
||||
"""SCP a file from the backend into the build container."""
|
||||
scp_prefix = _build_scp_prefix(host)
|
||||
full_cmd = scp_prefix + [str(local_path), f"{settings.MOBILE_BUILD_SSH_USER}@{host}:{remote_path}"]
|
||||
return await _run_command(full_cmd)
|
||||
|
||||
|
||||
async def _scp_from_container(host: str, remote_path: str, local_path: str) -> tuple[int, str]:
|
||||
"""SCP a file from the build container to the backend."""
|
||||
scp_prefix = _build_scp_prefix(host)
|
||||
full_cmd = scp_prefix + [f"{settings.MOBILE_BUILD_SSH_USER}@{host}:{remote_path}", str(local_path)]
|
||||
return await _run_command(full_cmd)
|
||||
|
||||
|
||||
async def _append_log(session: AsyncSession, request: MobileBuildRequest, line: str) -> None:
|
||||
if request.build_log:
|
||||
request.build_log += "\n"
|
||||
request.build_log += line
|
||||
request.updated_at = datetime.utcnow()
|
||||
await session.commit()
|
||||
|
||||
|
||||
def _preflight_issues(request: MobileBuildRequest, request_id: int) -> list[str]:
|
||||
issues: list[str] = []
|
||||
|
||||
if not request.platform_android and not request.platform_ios:
|
||||
issues.append("Select at least one platform.")
|
||||
|
||||
if request.platform_android and not request.android_application_id:
|
||||
issues.append("Android application ID is required when Android is selected.")
|
||||
|
||||
if request.platform_ios and not request.ios_bundle_id:
|
||||
issues.append("iOS bundle ID is required when iOS is selected.")
|
||||
|
||||
if not request.version_name.strip():
|
||||
issues.append("Version name is required.")
|
||||
|
||||
if not request.build_number.strip():
|
||||
issues.append("Build number is required.")
|
||||
|
||||
if request.platform_android and not settings.FLUTTER_BUILD_HOST:
|
||||
issues.append("Android build container host is not configured.")
|
||||
|
||||
if request.platform_ios and not settings.MOBILE_BUILD_IOS_HOST:
|
||||
issues.append("iOS build host is not configured.")
|
||||
|
||||
# Check for expected files on disk
|
||||
if request.platform_android:
|
||||
for kind in _ALLOWED_KINDS["android"]:
|
||||
role_key = f"android_{kind}"
|
||||
display = _FILE_ROLE_DISPLAY[role_key]
|
||||
if not _find_uploaded_file(request_id, "android", kind):
|
||||
issues.append(f"Missing: {display}")
|
||||
|
||||
if request.platform_ios:
|
||||
for kind in _ALLOWED_KINDS["ios"]:
|
||||
role_key = f"ios_{kind}"
|
||||
display = _FILE_ROLE_DISPLAY[role_key]
|
||||
if not _find_uploaded_file(request_id, "ios", kind):
|
||||
issues.append(f"Missing: {display}")
|
||||
|
||||
profile_path = _find_uploaded_file(request_id, "ios", "profile")
|
||||
if profile_path:
|
||||
if not _check_mobileprovision_for_carplay(profile_path):
|
||||
issues.append("iOS provisioning profile does not appear to include CarPlay entitlement text.")
|
||||
|
||||
return issues
|
||||
|
||||
|
||||
def _to_artifact_response(row: MobileBuildArtifact) -> MobileBuildArtifactResponse:
|
||||
return MobileBuildArtifactResponse(
|
||||
id=row.id,
|
||||
request_id=row.request_id,
|
||||
platform=row.platform,
|
||||
artifact_type=row.artifact_type,
|
||||
file_name=row.file_name,
|
||||
sha256=row.sha256,
|
||||
size=row.size,
|
||||
created_at=_utc_iso(row.created_at) or "",
|
||||
download_url=f"/api/mobile-builds/artifacts/{row.id}/download",
|
||||
)
|
||||
|
||||
|
||||
async def _hydrate_request(session: AsyncSession, request: MobileBuildRequest) -> MobileBuildRequestResponse:
|
||||
artifact_result = await session.execute(
|
||||
select(MobileBuildArtifact).where(MobileBuildArtifact.request_id == request.id)
|
||||
)
|
||||
artifacts = list(artifact_result.scalars().all())
|
||||
|
||||
return MobileBuildRequestResponse(
|
||||
id=request.id,
|
||||
tenant_key=request.tenant_key,
|
||||
platform_android=request.platform_android,
|
||||
platform_ios=request.platform_ios,
|
||||
android_application_id=request.android_application_id,
|
||||
ios_bundle_id=request.ios_bundle_id,
|
||||
version_name=request.version_name,
|
||||
build_number=request.build_number,
|
||||
release_profile=request.release_profile,
|
||||
status=request.status,
|
||||
preflight_passed=request.preflight_passed,
|
||||
preflight_report=request.preflight_report,
|
||||
build_log=request.build_log,
|
||||
error_message=request.error_message,
|
||||
created_at=_utc_iso(request.created_at) or "",
|
||||
updated_at=_utc_iso(request.updated_at) or "",
|
||||
started_at=_utc_iso(request.started_at),
|
||||
completed_at=_utc_iso(request.completed_at),
|
||||
artifacts=[_to_artifact_response(a) for a in artifacts],
|
||||
)
|
||||
|
||||
|
||||
async def _copy_worker_artifact(
|
||||
session: AsyncSession,
|
||||
request: MobileBuildRequest,
|
||||
platform: Literal["android", "ios"],
|
||||
host: str,
|
||||
artifact_glob: str,
|
||||
) -> None:
|
||||
_, artifact_dir = _ensure_dirs()
|
||||
local_dir = artifact_dir / f"request_{request.id}" / platform
|
||||
local_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
ssh_prefix = _build_ssh_prefix(host)
|
||||
# Don't shlex.quote the glob — the shell needs to expand the wildcard
|
||||
list_cmd = ssh_prefix + [f"ls -1t {artifact_glob} 2>/dev/null | head -n 1"]
|
||||
code, out = await _run_command(list_cmd)
|
||||
if code != 0 or not out:
|
||||
raise RuntimeError(f"No {platform} artifact found using glob {artifact_glob}")
|
||||
|
||||
remote_path = out.splitlines()[-1].strip()
|
||||
if not remote_path:
|
||||
raise RuntimeError(f"No {platform} artifact path returned by worker")
|
||||
|
||||
file_name = Path(remote_path).name
|
||||
local_path = local_dir / file_name
|
||||
|
||||
scp_cmd = _build_scp_prefix(host) + [
|
||||
f"{settings.MOBILE_BUILD_SSH_USER}@{host}:{remote_path}",
|
||||
str(local_path),
|
||||
]
|
||||
code, scp_out = await _run_command(scp_cmd)
|
||||
if code != 0:
|
||||
raise RuntimeError(f"Failed to copy {platform} artifact: {scp_out}")
|
||||
|
||||
artifact = MobileBuildArtifact(
|
||||
request_id=request.id,
|
||||
platform=platform,
|
||||
artifact_type="aab" if platform == "android" else "ipa",
|
||||
file_name=file_name,
|
||||
file_path=str(local_path),
|
||||
sha256=_hash_file(local_path),
|
||||
size=local_path.stat().st_size,
|
||||
)
|
||||
session.add(artifact)
|
||||
await session.commit()
|
||||
|
||||
|
||||
async def _upload_files_to_worker(
|
||||
request_id: int, platform: str, host: str
|
||||
) -> None:
|
||||
scp_prefix = _build_scp_prefix(host)
|
||||
remote_target = f"{settings.MOBILE_BUILD_SSH_USER}@{host}:{settings.MOBILE_BUILD_REMOTE_APP_DIR}/"
|
||||
|
||||
for kind in _ALLOWED_KINDS[platform]:
|
||||
role_key = f"{platform}_{kind}"
|
||||
display = _FILE_ROLE_DISPLAY[role_key]
|
||||
local_path = _find_uploaded_file(request_id, platform, kind)
|
||||
if not local_path or not local_path.exists():
|
||||
raise RuntimeError(f"Uploaded file missing: {display}")
|
||||
cmd = scp_prefix + [str(local_path), remote_target]
|
||||
code, out = await _run_command(cmd)
|
||||
if code != 0:
|
||||
raise RuntimeError(f"Failed to upload {display}: {out}")
|
||||
|
||||
|
||||
async def _upload_theme_to_worker(
|
||||
session: AsyncSession,
|
||||
request: MobileBuildRequest,
|
||||
host: str,
|
||||
) -> None:
|
||||
"""SCP theme.json to the build worker (no-op if file doesn't exist)."""
|
||||
theme_path = Path(settings.THEME_JSON_PATH)
|
||||
if not theme_path.exists():
|
||||
await _append_log(session, request, "[theme] theme.json not found — skipping upload")
|
||||
return
|
||||
|
||||
scp_prefix = _build_scp_prefix(host)
|
||||
remote_target = f"{settings.MOBILE_BUILD_SSH_USER}@{host}:{settings.MOBILE_BUILD_REMOTE_APP_DIR}/theme.json"
|
||||
code, out = await _run_command(scp_prefix + [str(theme_path), remote_target])
|
||||
if code != 0:
|
||||
raise RuntimeError(f"Failed to upload theme.json: {out}")
|
||||
await _append_log(session, request, "[theme] theme.json uploaded to worker")
|
||||
|
||||
|
||||
async def _run_platform_build(
|
||||
session: AsyncSession,
|
||||
request: MobileBuildRequest,
|
||||
platform: Literal["android", "ios"],
|
||||
host: str,
|
||||
script_name: str,
|
||||
) -> None:
|
||||
await _append_log(session, request, f"[{platform}] Uploading {len(_ALLOWED_KINDS[platform])} file(s) to worker")
|
||||
await _upload_files_to_worker(request.id, platform, host)
|
||||
|
||||
# Upload theme.json to worker
|
||||
await _upload_theme_to_worker(session, request, host)
|
||||
|
||||
ssh_prefix = _build_ssh_prefix(host)
|
||||
cmd = ssh_prefix + [
|
||||
(
|
||||
f"cd {settings.MOBILE_BUILD_REMOTE_APP_DIR} && "
|
||||
f"chmod +x ./{script_name} && "
|
||||
f"./{script_name}"
|
||||
)
|
||||
]
|
||||
await _append_log(session, request, f"[{platform}] Executing {script_name} on {host}")
|
||||
code, out = await _run_command(cmd)
|
||||
await _append_log(session, request, f"[{platform}] Exit code {code}\n{out}")
|
||||
if code != 0:
|
||||
raise RuntimeError(f"{platform} build failed with exit code {code}")
|
||||
|
||||
artifact_glob = (
|
||||
settings.MOBILE_BUILD_ANDROID_ARTIFACT_GLOB
|
||||
if platform == "android"
|
||||
else settings.MOBILE_BUILD_IOS_ARTIFACT_GLOB
|
||||
)
|
||||
await _copy_worker_artifact(session, request, platform, host, artifact_glob)
|
||||
await _append_log(session, request, f"[{platform}] Artifact copied successfully")
|
||||
|
||||
|
||||
# ── Container-based Android build flow (direct SSH to build service) ──
|
||||
|
||||
async def _copy_container_artifact(
|
||||
session: AsyncSession,
|
||||
request: MobileBuildRequest,
|
||||
host: str,
|
||||
project_dir: str,
|
||||
) -> None:
|
||||
"""Copy the APK from the Flutter build container back to the backend via SCP."""
|
||||
_, artifact_dir = _ensure_dirs()
|
||||
local_dir = artifact_dir / f"request_{request.id}" / "android"
|
||||
local_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
# Find the APK inside the container
|
||||
apk_glob = settings.FLUTTER_APK_GLOB
|
||||
# Don't shlex.quote the glob — the shell needs to expand the wildcard
|
||||
list_cmd = f"ls -1t {apk_glob} 2>/dev/null | head -n 1"
|
||||
code, out = await _ssh_run(host, list_cmd)
|
||||
if code != 0 or not out:
|
||||
raise RuntimeError(f"No APK artifact found using glob {apk_glob}")
|
||||
|
||||
apk_path = out.strip()
|
||||
file_name = Path(apk_path).name
|
||||
|
||||
# SCP directly from build container to backend
|
||||
local_path = local_dir / file_name
|
||||
code, out = await _scp_from_container(host, apk_path, local_path)
|
||||
if code != 0:
|
||||
raise RuntimeError(f"Failed to SCP APK from build container: {out}")
|
||||
|
||||
artifact = MobileBuildArtifact(
|
||||
request_id=request.id,
|
||||
platform="android",
|
||||
artifact_type="apk",
|
||||
file_name=file_name,
|
||||
file_path=str(local_path),
|
||||
sha256=_hash_file(local_path),
|
||||
size=local_path.stat().st_size,
|
||||
)
|
||||
session.add(artifact)
|
||||
await session.commit()
|
||||
|
||||
|
||||
async def _run_flutter_container_build(
|
||||
session: AsyncSession,
|
||||
request: MobileBuildRequest,
|
||||
host: str,
|
||||
) -> None:
|
||||
"""Execute an Android build inside the Flutter build container via direct SSH.
|
||||
|
||||
Flow:
|
||||
1. Clone the Flutter project (if not already cloned)
|
||||
2. Run flutter pub get
|
||||
3. Set up Python venv + activate
|
||||
4. Copy theme.json into the project directory
|
||||
5. Copy signing files (keystore + descriptor) into the project
|
||||
6. Run build_whitelabel.py
|
||||
7. Run flutter build apk
|
||||
8. Copy the APK artifact back to the backend
|
||||
"""
|
||||
project_dir = settings.FLUTTER_PROJECT_DIR # ~/kryz-go-flutter
|
||||
repo_url = settings.FLUTTER_PROJECT_REPO_URL
|
||||
|
||||
# Step 1: Clone the Flutter project (skip if already exists)
|
||||
await _append_log(session, request, "[android] Checking Flutter project directory...")
|
||||
check_cmd = f"test -d {shlex.quote(project_dir)}/.git"
|
||||
code, out = await _ssh_run(host, check_cmd)
|
||||
directory_exists = (code == 0)
|
||||
|
||||
if not directory_exists:
|
||||
await _append_log(session, request, f"[android] Cloning {repo_url} into {project_dir}")
|
||||
clone_cmd = f"cd /home/vscode && git clone {shlex.quote(repo_url)} kryz-go-flutter"
|
||||
code, out = await _ssh_run(host, clone_cmd)
|
||||
await _append_log(session, request, f"[android] Clone output: {out}")
|
||||
if code != 0:
|
||||
raise RuntimeError(f"Git clone failed: {out}")
|
||||
else:
|
||||
await _append_log(session, request, "[android] Project directory exists, pulling latest...")
|
||||
pull_cmd = f"cd {shlex.quote(project_dir)} && git pull"
|
||||
code, out = await _ssh_run(host, pull_cmd)
|
||||
await _append_log(session, request, f"[android] Pull output: {out}")
|
||||
if code != 0:
|
||||
# Directory may be stale/corrupt — remove and re-clone
|
||||
await _append_log(session, request, "[android] Pull failed, re-cloning...")
|
||||
rm_cmd = f"rm -rf {shlex.quote(project_dir)}"
|
||||
await _ssh_run(host, rm_cmd)
|
||||
clone_cmd = f"cd /home/vscode && git clone {shlex.quote(repo_url)} kryz-go-flutter"
|
||||
code, out = await _ssh_run(host, clone_cmd)
|
||||
await _append_log(session, request, f"[android] Re-clone output: {out}")
|
||||
if code != 0:
|
||||
raise RuntimeError(f"Git re-clone failed: {out}")
|
||||
|
||||
# Step 2: flutter pub get
|
||||
await _append_log(session, request, "[android] Running flutter pub get...")
|
||||
pub_cmd = f"cd {shlex.quote(project_dir)} && flutter pub get"
|
||||
code, out = await _ssh_run(host, pub_cmd)
|
||||
await _append_log(session, request, f"[android] Pub get output: {out}")
|
||||
if code != 0:
|
||||
raise RuntimeError(f"flutter pub get failed: {out}")
|
||||
|
||||
# Step 3: Python venv setup
|
||||
await _append_log(session, request, "[android] Setting up Python venv...")
|
||||
venv_cmd = (
|
||||
f"cd {shlex.quote(project_dir)} && "
|
||||
f"python3 -m venv .venv && "
|
||||
f"source .venv/bin/activate && "
|
||||
f"pip install -r requirements.txt"
|
||||
)
|
||||
code, out = await _ssh_run(host, venv_cmd)
|
||||
await _append_log(session, request, f"[android] Venv setup output: {out}")
|
||||
if code != 0:
|
||||
raise RuntimeError(f"Python venv setup failed: {out}")
|
||||
|
||||
# Step 4: Copy theme.json into the project directory
|
||||
theme_path = Path(settings.THEME_JSON_PATH)
|
||||
if theme_path.exists():
|
||||
code, out = await _scp_to_container(host, theme_path, f"{project_dir}/theme.json")
|
||||
if code != 0:
|
||||
raise RuntimeError(f"Failed to copy theme.json to build container: {out}")
|
||||
await _append_log(session, request, "[android] theme.json copied to build container")
|
||||
else:
|
||||
await _append_log(session, request, "[android] theme.json not found, skipping")
|
||||
|
||||
# Step 5: Copy signing files into the container
|
||||
for kind in _ALLOWED_KINDS["android"]:
|
||||
role_key = f"android_{kind}"
|
||||
display = _FILE_ROLE_DISPLAY[role_key]
|
||||
local_path = _find_uploaded_file(request.id, "android", kind)
|
||||
if not local_path or not local_path.exists():
|
||||
raise RuntimeError(f"Signing file missing: {display}")
|
||||
|
||||
# SCP directly to build container
|
||||
remote_path = f"{project_dir}/{_sanitize_filename(local_path.name)}"
|
||||
code, out = await _scp_to_container(host, local_path, remote_path)
|
||||
if code != 0:
|
||||
raise RuntimeError(f"Failed to SCP {display} to build container: {out}")
|
||||
|
||||
await _append_log(session, request, f"[android] {display} copied to build container")
|
||||
|
||||
# Step 6: Run build_whitelabel.py
|
||||
await _append_log(session, request, "[android] Running build_whitelabel.py...")
|
||||
whitelabel_cmd = (
|
||||
f"cd {shlex.quote(project_dir)} && "
|
||||
f"source .venv/bin/activate && "
|
||||
f"python3 build_whitelabel.py"
|
||||
)
|
||||
code, out = await _ssh_run(host, whitelabel_cmd)
|
||||
await _append_log(session, request, f"[android] build_whitelabel.py output:\n{out}")
|
||||
if code != 0:
|
||||
raise RuntimeError(f"build_whitelabel.py failed: {out}")
|
||||
|
||||
# Step 7: flutter build apk
|
||||
# Container-safe flags:
|
||||
# - GRADLE_OPTS disables native VFS file watching (inotify EBADF in Docker)
|
||||
# - KOTLIN_COMPILE_DAEMON_ENABLED=false avoids the Kotlin compile daemon
|
||||
# crashing with the same inotify issue
|
||||
# - Project-level gradle.properties ensures the Gradle wrapper always
|
||||
# honors these settings (user-level ~/.gradle/ may not be read)
|
||||
# - Memory is aggressively capped: 1g JVM heap, single worker, no
|
||||
# parallel builds, in-process Kotlin compilation, SerialGC (avoids
|
||||
# parallel GC overhead). Combined with the 4g container memory limit,
|
||||
# this leaves ~3g for native compilation (clang, dll, etc.)
|
||||
await _append_log(session, request, "[android] Running flutter build apk...")
|
||||
android_props = f"{project_dir}/android/gradle.properties"
|
||||
build_cmd = (
|
||||
f"cd {shlex.quote(project_dir)}/android && "
|
||||
f"echo 'org.gradle.daemon=false' >> gradle.properties && "
|
||||
# f"echo 'org.gradle.jvmargs=-Xmx4g -XX:MaxMetaspaceSize=512m -XX:+UseSerialGC' >> gradle.properties && "
|
||||
f"echo 'org.gradle.workers.max=1' >> gradle.properties && "
|
||||
# f"echo 'org.gradle.parallel=false' >> gradle.properties && "
|
||||
# f"echo 'kotlin.compiler.execution.strategy=in-process' >> gradle.properties && "
|
||||
# f"echo 'kotlin.incremental=false' >> gradle.properties && "
|
||||
# f"echo 'android.enableParallelPlugin=false' >> gradle.properties && "
|
||||
f"echo 'org.gradle.vfs.watch=false' >> gradle.properties && "
|
||||
f"cd {shlex.quote(project_dir)} && "
|
||||
f"GRADLE_OPTS='-Dorg.gradle.vfs.watch=false' "
|
||||
# f"KOTLIN_COMPILE_DAEMON_ENABLED=false "
|
||||
f"flutter build apk --release"
|
||||
)
|
||||
code, out = await _ssh_run(host, build_cmd)
|
||||
await _append_log(session, request, f"[android] Build output:\n{out}")
|
||||
if code != 0:
|
||||
raise RuntimeError(f"flutter build apk failed: {out}")
|
||||
|
||||
# Step 8: Copy APK artifact back
|
||||
await _copy_container_artifact(session, request, host, project_dir)
|
||||
await _append_log(session, request, "[android] APK artifact copied successfully")
|
||||
|
||||
|
||||
async def _execute_request(request_id: int) -> None:
|
||||
async with async_session() as session:
|
||||
result = await session.execute(
|
||||
select(MobileBuildRequest).where(MobileBuildRequest.id == request_id)
|
||||
)
|
||||
request = result.scalar_one_or_none()
|
||||
if request is None:
|
||||
return
|
||||
|
||||
request.status = "building"
|
||||
request.error_message = None
|
||||
request.started_at = datetime.utcnow()
|
||||
request.updated_at = datetime.utcnow()
|
||||
await session.commit()
|
||||
|
||||
try:
|
||||
if request.platform_android:
|
||||
await _run_flutter_container_build(
|
||||
session,
|
||||
request,
|
||||
settings.FLUTTER_BUILD_HOST,
|
||||
)
|
||||
|
||||
if request.platform_ios:
|
||||
await _run_platform_build(
|
||||
session,
|
||||
request,
|
||||
"ios",
|
||||
settings.MOBILE_BUILD_IOS_HOST,
|
||||
settings.MOBILE_BUILD_IOS_SCRIPT,
|
||||
)
|
||||
|
||||
request.status = "completed"
|
||||
request.completed_at = datetime.utcnow()
|
||||
request.updated_at = datetime.utcnow()
|
||||
await session.commit()
|
||||
except Exception as exc:
|
||||
request.status = "failed"
|
||||
request.error_message = str(exc)
|
||||
request.completed_at = datetime.utcnow()
|
||||
request.updated_at = datetime.utcnow()
|
||||
await _append_log(session, request, f"Build failed: {exc}")
|
||||
await session.commit()
|
||||
finally:
|
||||
_RUNNING_TASKS.discard(request_id)
|
||||
|
||||
|
||||
@router.get("/defaults", response_model=MobileBuildDefaultsResponse)
|
||||
async def get_mobile_build_defaults(session: AsyncSession = Depends(get_session)):
|
||||
result = await session.execute(
|
||||
select(StationConfig).where(StationConfig.key == "default")
|
||||
)
|
||||
cfg = result.scalar_one_or_none()
|
||||
if cfg is None:
|
||||
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Station config not found")
|
||||
|
||||
app_name = f"{cfg.name_primary} {cfg.name_secondary}".strip()
|
||||
return MobileBuildDefaultsResponse(
|
||||
tenant_key=cfg.key,
|
||||
app_name=app_name,
|
||||
tagline=cfg.tagline,
|
||||
support_email=cfg.email,
|
||||
website=cfg.website,
|
||||
stream_url=cfg.stream_url,
|
||||
stream_metadata_url=cfg.stream_metadata_url,
|
||||
)
|
||||
|
||||
|
||||
@router.get("/requests", response_model=list[MobileBuildRequestResponse])
|
||||
async def list_mobile_build_requests(
|
||||
session: AsyncSession = Depends(get_session),
|
||||
_: User = Depends(get_current_admin_user),
|
||||
):
|
||||
result = await session.execute(
|
||||
select(MobileBuildRequest).order_by(MobileBuildRequest.created_at.desc())
|
||||
)
|
||||
rows = list(result.scalars().all())
|
||||
return [await _hydrate_request(session, row) for row in rows]
|
||||
|
||||
|
||||
@router.post("/requests", response_model=MobileBuildRequestResponse, status_code=201)
|
||||
async def create_mobile_build_request(
|
||||
payload: MobileBuildCreate,
|
||||
session: AsyncSession = Depends(get_session),
|
||||
user: User = Depends(get_current_admin_user),
|
||||
):
|
||||
request = MobileBuildRequest(
|
||||
tenant_key="default",
|
||||
platform_android=payload.platform_android,
|
||||
platform_ios=payload.platform_ios,
|
||||
android_application_id=payload.android_application_id,
|
||||
ios_bundle_id=payload.ios_bundle_id,
|
||||
version_name=payload.version_name,
|
||||
build_number=payload.build_number,
|
||||
release_profile=payload.release_profile,
|
||||
status="draft",
|
||||
created_by_id=user.id,
|
||||
updated_at=datetime.utcnow(),
|
||||
)
|
||||
session.add(request)
|
||||
await session.commit()
|
||||
await session.refresh(request)
|
||||
return await _hydrate_request(session, request)
|
||||
|
||||
|
||||
@router.get("/requests/{request_id}", response_model=MobileBuildRequestResponse)
|
||||
async def get_mobile_build_request(
|
||||
request_id: int,
|
||||
session: AsyncSession = Depends(get_session),
|
||||
_: User = Depends(get_current_admin_user),
|
||||
):
|
||||
result = await session.execute(
|
||||
select(MobileBuildRequest).where(MobileBuildRequest.id == request_id)
|
||||
)
|
||||
request = result.scalar_one_or_none()
|
||||
if request is None:
|
||||
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Build request not found")
|
||||
return await _hydrate_request(session, request)
|
||||
|
||||
|
||||
@router.delete("/requests/{request_id}", status_code=status.HTTP_204_NO_CONTENT)
|
||||
async def delete_mobile_build_request(
|
||||
request_id: int,
|
||||
session: AsyncSession = Depends(get_session),
|
||||
_: User = Depends(get_current_admin_user),
|
||||
):
|
||||
"""Delete a draft build request and its uploaded files."""
|
||||
result = await session.execute(
|
||||
select(MobileBuildRequest).where(MobileBuildRequest.id == request_id)
|
||||
)
|
||||
request = result.scalar_one_or_none()
|
||||
if request is None:
|
||||
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Build request not found")
|
||||
|
||||
if request.status in {"queued", "building"}:
|
||||
raise HTTPException(status_code=status.HTTP_409_CONFLICT, detail="Cannot delete a request that is in progress")
|
||||
|
||||
# Clean up uploaded files on disk
|
||||
upload_dir, _ = _ensure_dirs()
|
||||
req_dir = upload_dir / f"request_{request_id}"
|
||||
if req_dir.is_dir():
|
||||
import shutil
|
||||
shutil.rmtree(req_dir, ignore_errors=True)
|
||||
|
||||
# Delete artifacts first (FK cascade may not cover async)
|
||||
await session.execute(
|
||||
delete(MobileBuildArtifact).where(MobileBuildArtifact.request_id == request_id)
|
||||
)
|
||||
# Then the request itself
|
||||
await session.execute(
|
||||
delete(MobileBuildRequest).where(MobileBuildRequest.id == request_id)
|
||||
)
|
||||
await session.commit()
|
||||
|
||||
|
||||
@router.post("/requests/{request_id}/files")
|
||||
async def upload_mobile_build_file(
|
||||
request_id: int,
|
||||
platform: Literal["android", "ios"] = Query(...),
|
||||
file_kind: str = Query(...),
|
||||
file: UploadFile = File(...),
|
||||
session: AsyncSession = Depends(get_session),
|
||||
_: User = Depends(get_current_admin_user),
|
||||
):
|
||||
"""Upload a signing file to the filesystem for a build request.
|
||||
|
||||
Files are stored on disk, not in the database.
|
||||
"""
|
||||
if file_kind not in _ALLOWED_KINDS[platform]:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_400_BAD_REQUEST,
|
||||
detail=f"Unsupported file_kind '{file_kind}' for platform '{platform}'",
|
||||
)
|
||||
|
||||
request_result = await session.execute(
|
||||
select(MobileBuildRequest).where(MobileBuildRequest.id == request_id)
|
||||
)
|
||||
request = request_result.scalar_one_or_none()
|
||||
if request is None:
|
||||
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Build request not found")
|
||||
|
||||
upload_dir, _ = _ensure_dirs()
|
||||
req_dir = upload_dir / f"request_{request_id}"
|
||||
req_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
payload = await file.read()
|
||||
|
||||
# Validate extension and content
|
||||
_validate_upload(platform, file_kind, file.filename, payload)
|
||||
|
||||
suffix = Path(file.filename or "upload.bin").suffix
|
||||
local_name = f"{platform}_{file_kind}_{uuid.uuid4().hex}{suffix}"
|
||||
local_path = req_dir / local_name
|
||||
local_path.write_bytes(payload)
|
||||
|
||||
sha = hashlib.sha256(payload).hexdigest()
|
||||
size = len(payload)
|
||||
|
||||
request.preflight_passed = False
|
||||
request.preflight_report = ""
|
||||
request.updated_at = datetime.utcnow()
|
||||
|
||||
await session.commit()
|
||||
|
||||
return {
|
||||
"platform": platform,
|
||||
"file_kind": file_kind,
|
||||
"display_name": _FILE_ROLE_DISPLAY[f"{platform}_{file_kind}"],
|
||||
"sha256": sha,
|
||||
"size": size,
|
||||
}
|
||||
|
||||
|
||||
@router.post("/requests/{request_id}/preflight", response_model=MobileBuildPreflightResponse)
|
||||
async def preflight_mobile_build(
|
||||
request_id: int,
|
||||
session: AsyncSession = Depends(get_session),
|
||||
_: User = Depends(get_current_admin_user),
|
||||
):
|
||||
result = await session.execute(
|
||||
select(MobileBuildRequest).where(MobileBuildRequest.id == request_id)
|
||||
)
|
||||
request = result.scalar_one_or_none()
|
||||
if request is None:
|
||||
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Build request not found")
|
||||
|
||||
issues = _preflight_issues(request, request_id)
|
||||
|
||||
request.preflight_passed = len(issues) == 0
|
||||
request.preflight_report = "\n".join(issues)
|
||||
request.updated_at = datetime.utcnow()
|
||||
await session.commit()
|
||||
|
||||
return MobileBuildPreflightResponse(passed=request.preflight_passed, issues=issues)
|
||||
|
||||
|
||||
@router.post("/requests/{request_id}/trigger", response_model=MobileBuildRequestResponse)
|
||||
async def trigger_mobile_build(
|
||||
request_id: int,
|
||||
session: AsyncSession = Depends(get_session),
|
||||
_: User = Depends(get_current_admin_user),
|
||||
):
|
||||
result = await session.execute(
|
||||
select(MobileBuildRequest).where(MobileBuildRequest.id == request_id)
|
||||
)
|
||||
request = result.scalar_one_or_none()
|
||||
if request is None:
|
||||
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Build request not found")
|
||||
|
||||
issues = _preflight_issues(request, request_id)
|
||||
if issues:
|
||||
request.preflight_passed = False
|
||||
request.preflight_report = "\n".join(issues)
|
||||
request.updated_at = datetime.utcnow()
|
||||
await session.commit()
|
||||
raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail=issues)
|
||||
|
||||
if request.status in {"queued", "building"}:
|
||||
raise HTTPException(status_code=status.HTTP_409_CONFLICT, detail="Build is already in progress")
|
||||
|
||||
request.status = "queued"
|
||||
request.preflight_passed = True
|
||||
request.preflight_report = ""
|
||||
request.error_message = None
|
||||
request.build_log = ""
|
||||
request.started_at = None
|
||||
request.completed_at = None
|
||||
request.updated_at = datetime.utcnow()
|
||||
|
||||
await session.execute(
|
||||
delete(MobileBuildArtifact).where(MobileBuildArtifact.request_id == request_id)
|
||||
)
|
||||
|
||||
await session.commit()
|
||||
|
||||
if request_id not in _RUNNING_TASKS:
|
||||
_RUNNING_TASKS.add(request_id)
|
||||
asyncio.create_task(_execute_request(request_id))
|
||||
|
||||
return await _hydrate_request(session, request)
|
||||
|
||||
|
||||
@router.get("/artifacts/{artifact_id}/download-url")
|
||||
async def get_download_url(
|
||||
artifact_id: int,
|
||||
session: AsyncSession = Depends(get_session),
|
||||
_: User = Depends(get_current_admin_user),
|
||||
):
|
||||
"""Return a short-lived signed download URL for the artifact."""
|
||||
result = await session.execute(
|
||||
select(MobileBuildArtifact).where(MobileBuildArtifact.id == artifact_id)
|
||||
)
|
||||
row = result.scalar_one_or_none()
|
||||
if row is None:
|
||||
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Artifact not found")
|
||||
|
||||
token = create_download_token(artifact_id)
|
||||
download_url = f"/api/mobile-builds/artifacts/{artifact_id}/download?token={token}"
|
||||
return {"download_url": download_url}
|
||||
|
||||
|
||||
@router.get("/artifacts/{artifact_id}/download")
|
||||
async def download_mobile_build_artifact(
|
||||
artifact_id: int,
|
||||
token: str | None = Query(None),
|
||||
credentials: HTTPAuthorizationCredentials | None = Depends(AuthBearer(auto_error=False)),
|
||||
session: AsyncSession = Depends(get_session),
|
||||
):
|
||||
"""Download an artifact. Accepts either Bearer auth OR a signed token query param."""
|
||||
# Authenticate: signed token takes precedence, fall back to Bearer JWT
|
||||
if token:
|
||||
verified_id = verify_download_token(token)
|
||||
if verified_id is None or verified_id != artifact_id:
|
||||
raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail="Invalid or expired download token")
|
||||
elif credentials:
|
||||
payload = decode_token(credentials.credentials)
|
||||
user_id = int(payload["sub"])
|
||||
result = await session.execute(select(User).where(User.id == user_id))
|
||||
user = result.scalar_one_or_none()
|
||||
if user is None or not user.is_admin:
|
||||
raise HTTPException(status_code=status.HTTP_403_FORBIDDEN, detail="Admin access required")
|
||||
else:
|
||||
raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail="Authentication required")
|
||||
|
||||
result = await session.execute(
|
||||
select(MobileBuildArtifact).where(MobileBuildArtifact.id == artifact_id)
|
||||
)
|
||||
row = result.scalar_one_or_none()
|
||||
if row is None:
|
||||
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Artifact not found")
|
||||
|
||||
path = Path(row.file_path)
|
||||
if not path.exists():
|
||||
raise HTTPException(status_code=status.HTTP_410_GONE, detail="Artifact file no longer exists")
|
||||
|
||||
return FileResponse(path=str(path), filename=row.file_name, media_type="application/octet-stream")
|
||||
@@ -12,7 +12,7 @@ from app.user_models import User
|
||||
router = APIRouter()
|
||||
|
||||
|
||||
@router.get("/", response_model=list[ShowResponse])
|
||||
@router.get("", response_model=list[ShowResponse])
|
||||
async def list_shows(session: AsyncSession = Depends(get_session)):
|
||||
query = (
|
||||
select(Show)
|
||||
@@ -40,7 +40,7 @@ async def get_show(
|
||||
return show
|
||||
|
||||
|
||||
@router.post("/", response_model=ShowResponse, status_code=201)
|
||||
@router.post("", response_model=ShowResponse, status_code=201)
|
||||
async def create_show(
|
||||
payload: ShowCreate,
|
||||
session: AsyncSession = Depends(get_session),
|
||||
|
||||
@@ -1,15 +1,20 @@
|
||||
"""Station config router: read/update the singleton station branding config."""
|
||||
|
||||
import logging
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException, status
|
||||
|
||||
from app.auth import get_current_admin_user
|
||||
from app.database import get_session
|
||||
from app.models import StationConfig
|
||||
from app.schemas import StationConfigResponse, StationConfigUpdate
|
||||
from app.theme_export import write_theme_json
|
||||
from app.user_models import User
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
router = APIRouter()
|
||||
|
||||
|
||||
@@ -47,4 +52,11 @@ async def update_station_config(
|
||||
|
||||
await session.commit()
|
||||
await session.refresh(config)
|
||||
|
||||
# Regenerate theme.json on disk (best-effort)
|
||||
try:
|
||||
await write_theme_json()
|
||||
except Exception:
|
||||
logger.warning("Failed to regenerate theme.json after config update", exc_info=True)
|
||||
|
||||
return config
|
||||
|
||||
@@ -0,0 +1,187 @@
|
||||
"""Stats API: trigger log parsing and query aggregated visitor data."""
|
||||
|
||||
import asyncio
|
||||
from datetime import date, timedelta
|
||||
from typing import AsyncGenerator, Optional
|
||||
|
||||
from fastapi import APIRouter, Depends, Query, HTTPException, status
|
||||
from fastapi.responses import StreamingResponse
|
||||
from sqlalchemy import desc, func, select
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
|
||||
from app.auth import decode_token, get_current_admin_user
|
||||
from app.database import get_session
|
||||
from app.log_parser import process_unparsed_logs
|
||||
from app.models import LogParseState, VisitorStatDaily, VisitorStatGeo
|
||||
from app.user_models import User
|
||||
|
||||
router = APIRouter()
|
||||
|
||||
|
||||
# ── Helpers ────────────────────────────────────────────────────
|
||||
|
||||
def _default_date_range() -> tuple[date, date]:
|
||||
"""Return (start_date, end_date) for last 30 days."""
|
||||
end = date.today() - timedelta(days=1) # today isn't parsed yet
|
||||
start = end - timedelta(days=29)
|
||||
return start, end
|
||||
|
||||
|
||||
async def _verify_admin_from_token(token: str) -> User:
|
||||
"""Verify admin from a raw JWT token string (for SSE query-param auth)."""
|
||||
from app.database import async_session
|
||||
from sqlalchemy import select as sa_select
|
||||
|
||||
payload = decode_token(token)
|
||||
user_id = int(payload["sub"])
|
||||
if not payload.get("is_admin"):
|
||||
raise HTTPException(status_code=status.HTTP_403_FORBIDDEN, detail="Admin access required")
|
||||
|
||||
async with async_session() as session:
|
||||
result = await session.execute(sa_select(User).where(User.id == user_id))
|
||||
user = result.scalar_one_or_none()
|
||||
if user is None:
|
||||
raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail="User not found")
|
||||
return user
|
||||
|
||||
|
||||
# ── SSE parse endpoint ───────────────────────────────────────
|
||||
|
||||
@router.get("/parse")
|
||||
async def trigger_parse(token: Optional[str] = Query(None)):
|
||||
"""Trigger log file parsing. Streams progress via Server-Sent Events.
|
||||
|
||||
Accepts JWT via `token` query parameter (EventSource can't send headers).
|
||||
"""
|
||||
# Authenticate — SSE can't send Authorization headers, so use query param
|
||||
if token:
|
||||
await _verify_admin_from_token(token)
|
||||
else:
|
||||
raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail="Token required")
|
||||
|
||||
async def event_generator() -> AsyncGenerator[str, None]:
|
||||
# Collect progress events, then yield them
|
||||
events: list[tuple[str, int, int]] = []
|
||||
result = await process_unparsed_logs(lambda fd, c, t: events.append((fd, c, t)))
|
||||
|
||||
for file_date, current, total in events:
|
||||
yield f'event: progress\ndata: {{"file": "{file_date}", "current": {current}, "total": {total}}}\n\n'
|
||||
|
||||
yield f'event: done\ndata: {{"files_processed": {result["files_processed"]}, "rows_inserted": {result["rows_inserted"]}}}\n\n'
|
||||
|
||||
return StreamingResponse(
|
||||
event_generator(),
|
||||
media_type="text/event-stream",
|
||||
headers={
|
||||
"Cache-Control": "no-cache",
|
||||
"Connection": "keep-alive",
|
||||
"X-Accel-Buffering": "no", # Disable nginx buffering for SSE
|
||||
},
|
||||
)
|
||||
|
||||
|
||||
# ── Summary ────────────────────────────────────────────────────
|
||||
|
||||
@router.get("/summary")
|
||||
async def get_summary(
|
||||
start_date: Optional[date] = Query(None),
|
||||
end_date: Optional[date] = Query(None),
|
||||
session: AsyncSession = Depends(get_session),
|
||||
_: User = Depends(get_current_admin_user),
|
||||
):
|
||||
"""Get total unique visitors for a date range."""
|
||||
if start_date is None or end_date is None:
|
||||
start_date, end_date = _default_date_range()
|
||||
|
||||
result = await session.execute(
|
||||
select(func.coalesce(func.sum(VisitorStatDaily.unique_visitors), 0)).where(
|
||||
VisitorStatDaily.date >= start_date,
|
||||
VisitorStatDaily.date <= end_date,
|
||||
)
|
||||
)
|
||||
total_unique = result.scalar()
|
||||
|
||||
return {"total_unique_visitors": total_unique}
|
||||
|
||||
|
||||
# ── Time series ────────────────────────────────────────────────
|
||||
|
||||
@router.get("/timeseries")
|
||||
async def get_timeseries(
|
||||
start_date: Optional[date] = Query(None),
|
||||
end_date: Optional[date] = Query(None),
|
||||
session: AsyncSession = Depends(get_session),
|
||||
_: User = Depends(get_current_admin_user),
|
||||
):
|
||||
"""Get daily unique visitor counts for a date range."""
|
||||
if start_date is None or end_date is None:
|
||||
start_date, end_date = _default_date_range()
|
||||
|
||||
stmt = (
|
||||
select(
|
||||
VisitorStatDaily.date,
|
||||
VisitorStatDaily.unique_visitors,
|
||||
)
|
||||
.where(
|
||||
VisitorStatDaily.date >= start_date,
|
||||
VisitorStatDaily.date <= end_date,
|
||||
)
|
||||
.order_by(VisitorStatDaily.date)
|
||||
)
|
||||
result = await session.execute(stmt)
|
||||
rows = result.all()
|
||||
return [
|
||||
{"date": row.date.isoformat(), "unique_visitors": row.unique_visitors}
|
||||
for row in rows
|
||||
]
|
||||
|
||||
|
||||
# ── Geo ────────────────────────────────────────────────────────
|
||||
|
||||
@router.get("/geo")
|
||||
async def get_geo_stats(
|
||||
start_date: Optional[date] = Query(None),
|
||||
end_date: Optional[date] = Query(None),
|
||||
session: AsyncSession = Depends(get_session),
|
||||
_: User = Depends(get_current_admin_user),
|
||||
):
|
||||
"""Get geographic distribution of visitors for a date range.
|
||||
|
||||
Aggregates by country (country, country_code, lat, lon), summing visitor_count.
|
||||
Returns one row per country/region for the map to render.
|
||||
"""
|
||||
if start_date is None or end_date is None:
|
||||
start_date, end_date = _default_date_range()
|
||||
|
||||
stmt = (
|
||||
select(
|
||||
VisitorStatGeo.country,
|
||||
VisitorStatGeo.country_code,
|
||||
VisitorStatGeo.latitude,
|
||||
VisitorStatGeo.longitude,
|
||||
func.sum(VisitorStatGeo.visitor_count).label("total_visitors"),
|
||||
)
|
||||
.where(
|
||||
VisitorStatGeo.date >= start_date,
|
||||
VisitorStatGeo.date <= end_date,
|
||||
)
|
||||
.group_by(
|
||||
VisitorStatGeo.country,
|
||||
VisitorStatGeo.country_code,
|
||||
VisitorStatGeo.latitude,
|
||||
VisitorStatGeo.longitude,
|
||||
)
|
||||
.order_by(desc(func.sum(VisitorStatGeo.visitor_count)))
|
||||
)
|
||||
result = await session.execute(stmt)
|
||||
rows = result.all()
|
||||
return [
|
||||
{
|
||||
"country": row.country or "Unknown",
|
||||
"country_code": row.country_code or "",
|
||||
"latitude": row.latitude,
|
||||
"longitude": row.longitude,
|
||||
"visitors": row.total_visitors,
|
||||
}
|
||||
for row in rows
|
||||
]
|
||||
@@ -13,7 +13,7 @@ from app.user_models import User
|
||||
router = APIRouter()
|
||||
|
||||
|
||||
@router.get("/", response_model=list[TeamMemberResponse])
|
||||
@router.get("", response_model=list[TeamMemberResponse])
|
||||
async def list_team_members(
|
||||
active: Optional[bool] = Query(None, description="Filter by active status"),
|
||||
session: AsyncSession = Depends(get_session),
|
||||
@@ -34,7 +34,7 @@ async def get_team_member(member_id: int, session: AsyncSession = Depends(get_se
|
||||
return member
|
||||
|
||||
|
||||
@router.post("/", response_model=TeamMemberResponse, status_code=201)
|
||||
@router.post("", response_model=TeamMemberResponse, status_code=201)
|
||||
async def create_team_member(
|
||||
payload: TeamMemberCreate,
|
||||
session: AsyncSession = Depends(get_session),
|
||||
|
||||
@@ -0,0 +1,52 @@
|
||||
"""Theme export: public JSON endpoint, status check, and admin regeneration trigger."""
|
||||
|
||||
from typing import Any, Dict
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException, status
|
||||
|
||||
from app.auth import get_current_admin_user
|
||||
from app.theme_export import get_theme_status, write_theme_json
|
||||
from app.user_models import User
|
||||
|
||||
router = APIRouter()
|
||||
|
||||
|
||||
@router.get("/status")
|
||||
async def theme_status() -> Dict[str, Any]:
|
||||
"""Return theme.json file status. Public endpoint."""
|
||||
return get_theme_status()
|
||||
|
||||
|
||||
@router.get("")
|
||||
async def get_theme() -> Dict[str, Any]:
|
||||
"""Return the current theme as a JSON dict. Public endpoint.
|
||||
|
||||
Also regenerates the theme.json file on disk as a side effect,
|
||||
ensuring the file is always up-to-date.
|
||||
"""
|
||||
try:
|
||||
theme = await write_theme_json()
|
||||
except Exception as e:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
|
||||
detail=f"Failed to generate theme: {e}",
|
||||
)
|
||||
return theme
|
||||
|
||||
|
||||
@router.post("/regenerate", status_code=200)
|
||||
async def regenerate_theme(
|
||||
_: User = Depends(get_current_admin_user),
|
||||
) -> Dict[str, Any]:
|
||||
"""Force regenerate the theme.json file. Admin only.
|
||||
|
||||
Returns the newly generated theme dict.
|
||||
"""
|
||||
try:
|
||||
theme = await write_theme_json()
|
||||
except Exception as e:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
|
||||
detail=f"Failed to regenerate theme: {e}",
|
||||
)
|
||||
return theme
|
||||
@@ -13,7 +13,7 @@ from app.user_models import User
|
||||
router = APIRouter()
|
||||
|
||||
|
||||
@router.get("/", response_model=list[UnderwriterResponse])
|
||||
@router.get("", response_model=list[UnderwriterResponse])
|
||||
async def list_underwriters(
|
||||
active: Optional[bool] = Query(None, description="Filter by active status"),
|
||||
session: AsyncSession = Depends(get_session),
|
||||
@@ -34,7 +34,7 @@ async def get_underwriter(underwriter_id: int, session: AsyncSession = Depends(g
|
||||
return uw
|
||||
|
||||
|
||||
@router.post("/", response_model=UnderwriterResponse, status_code=201)
|
||||
@router.post("", response_model=UnderwriterResponse, status_code=201)
|
||||
async def create_underwriter(
|
||||
payload: UnderwriterCreate,
|
||||
session: AsyncSession = Depends(get_session),
|
||||
|
||||
@@ -37,6 +37,31 @@ def create_access_token(user_id: int, is_admin: bool) -> str:
|
||||
return jwt.encode(payload, settings.JWT_SECRET_KEY, algorithm="HS256")
|
||||
|
||||
|
||||
def create_download_token(artifact_id: int) -> str:
|
||||
"""Create a short-lived JWT for downloading a specific artifact."""
|
||||
expire = datetime.now(timezone.utc) + timedelta(minutes=settings.DOWNLOAD_TOKEN_MINUTES)
|
||||
payload = {
|
||||
"sub": "download",
|
||||
"artifact_id": artifact_id,
|
||||
"exp": expire,
|
||||
}
|
||||
return jwt.encode(payload, settings.JWT_SECRET_KEY, algorithm="HS256")
|
||||
|
||||
|
||||
def verify_download_token(token: str) -> int | None:
|
||||
"""Verify a download token and return the artifact_id, or None on failure."""
|
||||
try:
|
||||
payload = jwt.decode(token, settings.JWT_SECRET_KEY, algorithms=["HS256"])
|
||||
if payload.get("sub") != "download":
|
||||
return None
|
||||
artifact_id = payload.get("artifact_id")
|
||||
if not isinstance(artifact_id, int):
|
||||
return None
|
||||
return artifact_id
|
||||
except JWTError:
|
||||
return None
|
||||
|
||||
|
||||
def decode_token(token: str) -> dict:
|
||||
"""Decode and verify a JWT token. Raises HTTPException 401 on failure."""
|
||||
try:
|
||||
|
||||
@@ -13,6 +13,40 @@ class Settings(BaseSettings):
|
||||
GOOGLE_REDIRECT_URI: str = "http://localhost:4200"
|
||||
JWT_SECRET_KEY: str = "change-me-in-production"
|
||||
JWT_EXPIRE_MINUTES: int = 1440 # 24 hours
|
||||
DOWNLOAD_TOKEN_MINUTES: int = 10 # short-lived signed download URLs
|
||||
|
||||
# Visitor stats
|
||||
LOGS_DIR: str = "/logs"
|
||||
GEOLITE2_DB_PATH: str = "/app/geo/GeoLite2-City.mmdb"
|
||||
|
||||
# Mobile build orchestration — matches the user created in build/Dockerfile
|
||||
MOBILE_BUILD_SSH_USER: str = "vscode"
|
||||
MOBILE_BUILD_SSH_PASSWORD: str = "123"
|
||||
MOBILE_BUILD_SSH_PORT: int = 22
|
||||
MOBILE_BUILD_ANDROID_HOST: str = ""
|
||||
MOBILE_BUILD_IOS_HOST: str = ""
|
||||
MOBILE_BUILD_REMOTE_APP_DIR: str = "/home/vscode/mobile_app"
|
||||
MOBILE_BUILD_ANDROID_SCRIPT: str = "do_android_build.sh"
|
||||
MOBILE_BUILD_IOS_SCRIPT: str = "do_ios_build.sh"
|
||||
MOBILE_BUILD_UPLOAD_DIR: str = "/tmp/kmtn_mobile_build_uploads"
|
||||
MOBILE_BUILD_ARTIFACT_DIR: str = "/tmp/kmtn_mobile_build_artifacts"
|
||||
MOBILE_BUILD_ANDROID_ARTIFACT_GLOB: str = "/home/vscode/mobile_app/build/app/outputs/bundle/release/*.aab"
|
||||
MOBILE_BUILD_IOS_ARTIFACT_GLOB: str = "/home/vscode/mobile_app/build/ios/ipa/*.ipa"
|
||||
|
||||
# Flutter container-based Android build (SSH directly to build service over Docker network)
|
||||
FLUTTER_BUILD_HOST: str = "build" # docker-compose service name
|
||||
FLUTTER_PROJECT_REPO_URL: str = "http://192.168.1.74:30008/kfj001/kryz-go-flutter.git"
|
||||
FLUTTER_PROJECT_DIR: str = "/home/vscode/kryz-go-flutter"
|
||||
FLUTTER_APK_GLOB: str = "/home/vscode/kryz-go-flutter/build/app/outputs/flutter-apk/app-*.apk"
|
||||
|
||||
# Theme export
|
||||
WEBSITE_URL: str = "https://kmountainflower.org"
|
||||
THEME_JSON_PATH: str = "/app/theme.json"
|
||||
STATIC_FILES_DIR: str = "/usr/share/nginx/html"
|
||||
# Internal URL for fetching static files when they're not on the local
|
||||
# filesystem (e.g. separate Docker/K8s containers). Falls back to
|
||||
# WEBSITE_URL if not set.
|
||||
STATIC_UPSTREAM_URL: str = ""
|
||||
|
||||
model_config = {"env_prefix": "KMTN_"}
|
||||
|
||||
|
||||
@@ -0,0 +1,317 @@
|
||||
"""Nginx access log parser with IP geolocation enrichment."""
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
import os
|
||||
from datetime import date, datetime
|
||||
from typing import Callable, Optional
|
||||
|
||||
import maxminddb
|
||||
|
||||
from app.config import settings
|
||||
from app.database import async_session
|
||||
from app.models import LogParseState, VisitorStatDaily, VisitorStatGeo
|
||||
|
||||
# ── Geo reader ──────────────────────────────────────────────
|
||||
|
||||
_geo_reader: Optional[maxminddb.Reader] = None
|
||||
|
||||
|
||||
def get_geo_reader() -> maxminddb.Reader:
|
||||
"""Return a singleton MaxMind GeoLite2 City reader."""
|
||||
global _geo_reader
|
||||
if _geo_reader is None:
|
||||
db_path = settings.GEOLITE2_DB_PATH
|
||||
if os.path.exists(db_path):
|
||||
_geo_reader = maxminddb.open_database(db_path)
|
||||
else:
|
||||
raise FileNotFoundError(
|
||||
f"GeoLite2 database not found at {db_path}. "
|
||||
"Download from https://dev.maxmind.com/geoip/geolite2-free-geolocation-data "
|
||||
"and place it at backend/geo/GeoLite2-City.mmdb"
|
||||
)
|
||||
return _geo_reader
|
||||
|
||||
|
||||
# Country centroid fallback for free GeoLite2 (no city/coordinates).
|
||||
# Approximate center coordinates per ISO country code — enough for map markers.
|
||||
_COUNTRY_CENTROIDS: dict[str, tuple[float, float]] = {
|
||||
"US": (39.8, -98.5),
|
||||
"GB": (53.5, -1.5),
|
||||
"DE": (51.2, 10.4),
|
||||
"FR": (46.6, 2.3),
|
||||
"JP": (36.2, 138.3),
|
||||
"AU": (-25.3, 133.8),
|
||||
"BR": (-14.2, -51.9),
|
||||
"CA": (56.1, -106.3),
|
||||
"IN": (20.6, 79.0),
|
||||
"KR": (35.9, 127.8),
|
||||
"NL": (52.1, 5.3),
|
||||
"MX": (23.6, -102.5),
|
||||
"IT": (41.9, 12.6),
|
||||
"ES": (40.5, -3.7),
|
||||
"SE": (62.0, 15.0),
|
||||
"PL": (51.9, 19.1),
|
||||
"AR": (-38.4, -63.6),
|
||||
"ZA": (-30.6, 22.9),
|
||||
"EG": (26.8, 30.8),
|
||||
"TR": (39.0, 35.2),
|
||||
"RU": (61.5, 105.3),
|
||||
"SG": (1.35, 103.8),
|
||||
"NZ": (-40.9, 174.9),
|
||||
"CO": (4.6, -74.3),
|
||||
"CL": (-35.7, -71.5),
|
||||
"PE": (-9.2, -75.0),
|
||||
"PH": (12.9, 121.8),
|
||||
"ID": (-0.8, 113.9),
|
||||
"TH": (15.9, 100.9),
|
||||
"VN": (14.1, 108.3),
|
||||
"MY": (4.2, 101.9),
|
||||
"AE": (23.4, 53.8),
|
||||
"SA": (23.9, 45.1),
|
||||
"IL": (31.0, 34.9),
|
||||
"NO": (60.5, 8.5),
|
||||
"DK": (56.3, 9.5),
|
||||
"FI": (64.1, 26.5),
|
||||
"BE": (50.5, 4.5),
|
||||
"AT": (47.5, 14.6),
|
||||
"CH": (46.8, 8.2),
|
||||
"PT": (39.4, -8.2),
|
||||
"GR": (39.1, 21.8),
|
||||
"CZ": (49.8, 15.5),
|
||||
"RO": (45.9, 24.9),
|
||||
"HU": (47.2, 19.5),
|
||||
"UA": (48.4, 31.2),
|
||||
"NG": (9.1, 8.7),
|
||||
"KE": (-0.02, 37.9),
|
||||
"GH": (7.9, -1.0),
|
||||
"CN": (35.9, 104.2),
|
||||
"PK": (30.4, 69.3),
|
||||
"BD": (23.7, 90.4),
|
||||
}
|
||||
|
||||
|
||||
def lookup_geo(ip: str) -> dict:
|
||||
"""Look up geolocation for an IP address.
|
||||
|
||||
Returns dict with country, country_code, city, latitude, longitude
|
||||
or empty dict if lookup fails. Falls back to country centroids
|
||||
when the free GeoLite2 DB lacks city/coordinate data.
|
||||
"""
|
||||
try:
|
||||
reader = get_geo_reader()
|
||||
result = reader.get(ip)
|
||||
if result is None:
|
||||
return {}
|
||||
country = result.get("country", {})
|
||||
city = result.get("city", {})
|
||||
location = result.get("location", {})
|
||||
country_code = country.get("iso_code")
|
||||
lat = location.get("latitude")
|
||||
lon = location.get("longitude")
|
||||
|
||||
# Free GeoLite2 City often lacks city/coordinates — use centroid fallback
|
||||
if lat is None or lon is None:
|
||||
centroid = _COUNTRY_CENTROIDS.get(country_code or "")
|
||||
if centroid:
|
||||
lat = lat if lat is not None else centroid[0]
|
||||
lon = lon if lon is not None else centroid[1]
|
||||
|
||||
return {
|
||||
"country": country.get("names", {}).get("en", country_code or ""),
|
||||
"country_code": country_code,
|
||||
"city": city.get("names", {}).get("en", ""),
|
||||
"latitude": lat,
|
||||
"longitude": lon,
|
||||
}
|
||||
except Exception:
|
||||
return {}
|
||||
|
||||
|
||||
# ── IP resolution ───────────────────────────────────────────
|
||||
|
||||
|
||||
def resolve_client_ip(entry: dict) -> str:
|
||||
"""Extract the real client IP from a log entry.
|
||||
|
||||
Prefers the leftmost X-Forwarded-For entry (production behind proxy),
|
||||
falls back to remote_addr (dev).
|
||||
"""
|
||||
xff = entry.get("x_forwarded_for", "")
|
||||
if xff and xff != "-":
|
||||
# Leftmost entry is the original client
|
||||
return xff.split(",")[0].strip()
|
||||
return entry.get("remote_addr", "")
|
||||
|
||||
|
||||
# ── Log parsing ─────────────────────────────────────────────
|
||||
|
||||
|
||||
def parse_log_line(line: str) -> Optional[dict]:
|
||||
"""Parse a single JSON log line. Returns dict or None if unparseable."""
|
||||
line = line.strip()
|
||||
if not line:
|
||||
return None
|
||||
try:
|
||||
return json.loads(line)
|
||||
except (json.JSONDecodeError, ValueError):
|
||||
return None
|
||||
|
||||
|
||||
def parse_log_file(filepath: str) -> dict:
|
||||
"""Parse a single daily log file and return aggregated counts.
|
||||
|
||||
Returns a dict with:
|
||||
- unique_ips: set of unique client IP addresses
|
||||
- geo_counts: dict of (country_code, country, city, lat, lon) -> count
|
||||
"""
|
||||
unique_ips: set[str] = set()
|
||||
geo_counts: dict[tuple, int] = {}
|
||||
|
||||
with open(filepath, "r") as f:
|
||||
for line in f:
|
||||
entry = parse_log_line(line)
|
||||
if entry is None:
|
||||
continue
|
||||
|
||||
ip = resolve_client_ip(entry)
|
||||
if not ip:
|
||||
continue
|
||||
|
||||
unique_ips.add(ip)
|
||||
|
||||
# Geo enrichment
|
||||
geo = lookup_geo(ip)
|
||||
if geo.get("country_code"):
|
||||
geo_key = (
|
||||
geo.get("country_code", ""),
|
||||
geo.get("country", ""),
|
||||
geo.get("city", ""),
|
||||
geo.get("latitude"),
|
||||
geo.get("longitude"),
|
||||
)
|
||||
geo_counts[geo_key] = geo_counts.get(geo_key, 0) + 1
|
||||
|
||||
return {
|
||||
"unique_ips": unique_ips,
|
||||
"geo_counts": geo_counts,
|
||||
}
|
||||
|
||||
|
||||
async def process_unparsed_logs(
|
||||
progress_callback: Optional[Callable] = None,
|
||||
) -> dict:
|
||||
"""Scan /logs/ for unprocessed daily files, parse them, store aggregates.
|
||||
|
||||
Only processes files strictly older than today (file must be complete).
|
||||
Advances the LogParseState marker after each successful file.
|
||||
|
||||
Returns a summary dict with files_processed and rows_inserted.
|
||||
"""
|
||||
logs_dir = settings.LOGS_DIR
|
||||
if not os.path.isdir(logs_dir):
|
||||
return {"files_processed": 0, "error": f"Logs directory not found: {logs_dir}"}
|
||||
|
||||
# Get current parse state
|
||||
async with async_session() as session:
|
||||
from sqlalchemy import select
|
||||
|
||||
result = await session.execute(
|
||||
select(LogParseState).where(LogParseState.key == "default")
|
||||
)
|
||||
state = result.scalar_one_or_none()
|
||||
|
||||
if state is None:
|
||||
# Initialize with a far-past date so all files get processed
|
||||
state = LogParseState(key="default", last_parsed_date=date(2020, 1, 1))
|
||||
session.add(state)
|
||||
await session.commit()
|
||||
|
||||
last_parsed = state.last_parsed_date
|
||||
|
||||
# Discover daily log files
|
||||
import glob
|
||||
|
||||
pattern = os.path.join(logs_dir, "access-*.log")
|
||||
log_files = sorted(glob.glob(pattern))
|
||||
|
||||
files_processed = 0
|
||||
rows_inserted = 0
|
||||
today = date.today()
|
||||
|
||||
for filepath in log_files:
|
||||
# Extract date from filename
|
||||
basename = os.path.basename(filepath)
|
||||
try:
|
||||
date_str = basename.replace("access-", "").replace(".log", "")
|
||||
file_date = date.fromisoformat(date_str)
|
||||
except ValueError:
|
||||
continue
|
||||
|
||||
# Only process files strictly before today (file must be complete)
|
||||
if file_date >= today:
|
||||
continue
|
||||
|
||||
# Only process files after the last parsed date
|
||||
if file_date <= last_parsed:
|
||||
continue
|
||||
|
||||
# Parse this file
|
||||
try:
|
||||
stats = parse_log_file(filepath)
|
||||
except Exception as e:
|
||||
print(f" ERROR parsing {filepath}: {e}")
|
||||
continue
|
||||
|
||||
# Store aggregates in database
|
||||
async with async_session() as session:
|
||||
from sqlalchemy import select
|
||||
|
||||
# Upsert daily stats
|
||||
result = await session.execute(
|
||||
select(VisitorStatDaily).where(VisitorStatDaily.date == file_date)
|
||||
)
|
||||
daily = result.scalar_one_or_none()
|
||||
if daily is None:
|
||||
daily = VisitorStatDaily(date=file_date)
|
||||
session.add(daily)
|
||||
|
||||
daily.unique_visitors = len(stats["unique_ips"])
|
||||
|
||||
# Geo stats
|
||||
for geo_key, count in stats["geo_counts"].items():
|
||||
geo_row = VisitorStatGeo(
|
||||
date=file_date,
|
||||
country_code=geo_key[0],
|
||||
country=geo_key[1],
|
||||
city=geo_key[2],
|
||||
latitude=geo_key[3],
|
||||
longitude=geo_key[4],
|
||||
visitor_count=count,
|
||||
)
|
||||
session.add(geo_row)
|
||||
rows_inserted += 1
|
||||
|
||||
# Advance parse state
|
||||
state = (
|
||||
await session.execute(
|
||||
select(LogParseState).where(LogParseState.key == "default")
|
||||
)
|
||||
).scalar_one()
|
||||
state.last_parsed_date = file_date
|
||||
state.updated_at = datetime.utcnow()
|
||||
|
||||
await session.commit()
|
||||
|
||||
files_processed += 1
|
||||
|
||||
# Report progress via callback (for SSE streaming)
|
||||
if progress_callback:
|
||||
cb_result = progress_callback(
|
||||
file_date.isoformat(), files_processed, len(log_files)
|
||||
)
|
||||
# Support both sync and async callbacks
|
||||
if cb_result is not None and asyncio.iscoroutine(cb_result):
|
||||
await cb_result
|
||||
|
||||
return {"files_processed": files_processed, "rows_inserted": rows_inserted}
|
||||
+93
-5
@@ -1,4 +1,6 @@
|
||||
from contextlib import asynccontextmanager
|
||||
import asyncio
|
||||
import os
|
||||
|
||||
from fastapi import FastAPI
|
||||
from fastapi.middleware.cors import CORSMiddleware
|
||||
@@ -8,7 +10,30 @@ from app.config import settings
|
||||
from app.database import async_session, engine
|
||||
from app.models import Base, Show, StationConfig, HistoryEntry, TeamMember, CommunityHighlight, Underwriter
|
||||
from app.user_models import User
|
||||
from app.api import events, auth, station_config, history, team, community, shows, storage, underwriters
|
||||
from app.api import events, auth, station_config, history, team, community, shows, storage, underwriters, stats, mobile_builds, theme
|
||||
|
||||
|
||||
def _cleanup_orphaned_sequences(sync_conn):
|
||||
"""Drop model sequences whose backing table no longer exists.
|
||||
|
||||
Prevents 'duplicate key value violates unique constraint' errors when
|
||||
create_all() tries to recreate a SERIAL-backed sequence for a table
|
||||
whose original sequence was left behind after a drop/recreate cycle.
|
||||
Only runs on PostgreSQL; no-op for SQLite.
|
||||
"""
|
||||
import sqlalchemy as sa
|
||||
|
||||
if sync_conn.dialect.name != "postgresql":
|
||||
return
|
||||
|
||||
inspector = sa.inspect(sync_conn)
|
||||
existing_tables = set(inspector.get_table_names())
|
||||
|
||||
for table in Base.metadata.tables.values():
|
||||
expected_seq = f"{table.name}_id_seq"
|
||||
if table.name not in existing_tables:
|
||||
sync_conn.execute(sa.text(f"DROP SEQUENCE IF EXISTS {expected_seq}"))
|
||||
print(f" → Dropped orphaned sequence {expected_seq}")
|
||||
|
||||
|
||||
def _migrate_add_missing_columns(sync_conn):
|
||||
@@ -26,6 +51,30 @@ def _migrate_add_missing_columns(sync_conn):
|
||||
("station_config", "play_store_icon_url", sa.String(500), ""),
|
||||
("station_config", "play_store_url", sa.String(500), ""),
|
||||
("station_config", "app_store_embed_html", sa.Text, ""),
|
||||
# Page content columns
|
||||
("station_config", "hero_subtitle", sa.Text, ""),
|
||||
("station_config", "about_intro", sa.Text, ""),
|
||||
("station_config", "about_donation_text", sa.Text, ""),
|
||||
("station_config", "schedule_intro", sa.Text, ""),
|
||||
("station_config", "events_intro", sa.Text, ""),
|
||||
# Color theme columns
|
||||
("station_config", "color_primary", sa.String(7), "#1a3a5c"),
|
||||
("station_config", "color_primary_light", sa.String(7), "#2a5a8c"),
|
||||
("station_config", "color_primary_dark", sa.String(7), "#0e2440"),
|
||||
("station_config", "color_primary_muted", sa.String(7), "#3a6a9c"),
|
||||
("station_config", "color_accent", sa.String(7), "#e87a2e"),
|
||||
("station_config", "color_accent_light", sa.String(7), "#f09a4e"),
|
||||
("station_config", "color_accent_dark", sa.String(7), "#c85a1e"),
|
||||
("station_config", "color_accent_muted", sa.String(7), "#f0a86a"),
|
||||
("station_config", "color_background", sa.String(7), "#faf6f0"),
|
||||
("station_config", "color_text", sa.String(7), "#3a3632"),
|
||||
("station_config", "color_success", sa.String(7), "#4a9e4f"),
|
||||
("station_config", "color_danger", sa.String(7), "#c83030"),
|
||||
("station_config", "color_info", sa.String(7), "#3a8abf"),
|
||||
("station_config", "color_white", sa.String(7), "#ffffff"),
|
||||
("station_config", "color_light", sa.String(7), "#f0ebe3"),
|
||||
("station_config", "color_medium", sa.String(7), "#8a8580"),
|
||||
("station_config", "color_black", sa.String(7), "#1a1816"),
|
||||
]
|
||||
|
||||
# Determine dialect
|
||||
@@ -45,17 +94,17 @@ def _migrate_add_missing_columns(sync_conn):
|
||||
if isinstance(col_type, sa.Text):
|
||||
sql_type = "TEXT"
|
||||
elif isinstance(col_type, sa.String):
|
||||
sql_type = "VARCHAR(500)"
|
||||
sql_type = f"VARCHAR({col_type.length})"
|
||||
else:
|
||||
sql_type = "TEXT"
|
||||
|
||||
if dialect == "sqlite":
|
||||
sync_conn.execute(
|
||||
sa.text(f'ALTER TABLE {table_name} ADD COLUMN "{col_name}" TEXT NOT NULL DEFAULT "{default}"')
|
||||
sa.text(f"ALTER TABLE {table_name} ADD COLUMN \"{col_name}\" TEXT NOT NULL DEFAULT '{default}'")
|
||||
)
|
||||
else:
|
||||
sync_conn.execute(
|
||||
sa.text(f'ALTER TABLE {table_name} ADD COLUMN "{col_name}" {sql_type} NOT NULL DEFAULT "{default}"')
|
||||
sa.text(f"ALTER TABLE {table_name} ADD COLUMN \"{col_name}\" {sql_type} NOT NULL DEFAULT '{default}'")
|
||||
)
|
||||
|
||||
print(f" ✓ Added {table_name}.{col_name}")
|
||||
@@ -80,7 +129,14 @@ async def _ensure_station_config(session):
|
||||
async def lifespan(app: FastAPI):
|
||||
# Create tables on startup, then patch any missing columns
|
||||
async with engine.begin() as conn:
|
||||
await conn.run_sync(_cleanup_orphaned_sequences)
|
||||
try:
|
||||
await conn.run_sync(Base.metadata.create_all)
|
||||
except Exception as e:
|
||||
if "duplicate" in str(e).lower():
|
||||
print(f" → Tables already exist (partial creation from prior restart): {e}")
|
||||
else:
|
||||
raise
|
||||
await conn.run_sync(_migrate_add_missing_columns)
|
||||
|
||||
# Auto-seed if database is empty
|
||||
@@ -91,12 +147,20 @@ async def lifespan(app: FastAPI):
|
||||
print(" → Database is empty — running seed...")
|
||||
from seed import seed as run_seed
|
||||
await run_seed()
|
||||
return # Full seed already included station config
|
||||
print(" ✓ Full seed completed (skipping station config check)")
|
||||
|
||||
# Ensure station config exists (runs even if DB already had data)
|
||||
async with async_session() as session:
|
||||
await _ensure_station_config(session)
|
||||
|
||||
# Write theme.json on startup
|
||||
try:
|
||||
from app.theme_export import write_theme_json
|
||||
await write_theme_json()
|
||||
print(f" ✓ Theme JSON written to {settings.THEME_JSON_PATH}")
|
||||
except Exception as e:
|
||||
print(f" WARNING: Failed to write theme.json on startup: {e}")
|
||||
|
||||
# Bootstrap admin user if configured
|
||||
if settings.ADMIN_USERNAME and settings.ADMIN_PASSWORD:
|
||||
async with async_session() as session:
|
||||
@@ -117,6 +181,27 @@ async def lifespan(app: FastAPI):
|
||||
await session.commit()
|
||||
print(f" → Bootstrap admin created: {settings.ADMIN_USERNAME}")
|
||||
|
||||
# Check GeoLite2 database
|
||||
geo_db = settings.GEOLITE2_DB_PATH
|
||||
if os.path.exists(geo_db):
|
||||
print(f" ✓ GeoLite2 database loaded from {geo_db}")
|
||||
else:
|
||||
print(f" WARNING: GeoLite2 database not found at {geo_db} — geo lookups will be disabled")
|
||||
|
||||
# Auto-parse any unprocessed logs on startup (background task)
|
||||
async def _startup_parse():
|
||||
try:
|
||||
from app.log_parser import process_unparsed_logs
|
||||
result = await process_unparsed_logs()
|
||||
if result.get("files_processed"):
|
||||
print(f" ✓ Parsed {result['files_processed']} log files ({result['rows_inserted']} geo rows)")
|
||||
else:
|
||||
print(" No unprocessed log files to parse")
|
||||
except Exception as e:
|
||||
print(f" ERROR during startup log parse: {e}")
|
||||
|
||||
asyncio.create_task(_startup_parse())
|
||||
|
||||
yield
|
||||
|
||||
|
||||
@@ -145,6 +230,9 @@ app.include_router(community.router, prefix="/api/community", tags=["community"]
|
||||
app.include_router(shows.router, prefix="/api/shows", tags=["shows"])
|
||||
app.include_router(storage.router, prefix="/api/storage", tags=["storage"])
|
||||
app.include_router(underwriters.router, prefix="/api/underwriters", tags=["underwriters"])
|
||||
app.include_router(stats.router, prefix="/api/stats", tags=["stats"])
|
||||
app.include_router(mobile_builds.router, prefix="/api/mobile-builds", tags=["mobile-builds"])
|
||||
app.include_router(theme.router, prefix="/api/theme", tags=["theme"])
|
||||
|
||||
# Dev-only admin router (disabled in production)
|
||||
if settings.ENV != "production":
|
||||
|
||||
@@ -3,6 +3,7 @@ from datetime import datetime
|
||||
from sqlalchemy import (
|
||||
Column,
|
||||
DateTime,
|
||||
Float,
|
||||
Integer,
|
||||
LargeBinary,
|
||||
String,
|
||||
@@ -11,6 +12,8 @@ from sqlalchemy import (
|
||||
Date,
|
||||
ForeignKey,
|
||||
UniqueConstraint,
|
||||
Index,
|
||||
text,
|
||||
)
|
||||
from sqlalchemy.orm import DeclarativeBase, relationship
|
||||
|
||||
@@ -103,6 +106,35 @@ class StationConfig(Base):
|
||||
play_store_url = Column(String(500), nullable=False, default="")
|
||||
app_store_embed_html = Column(Text, nullable=False, default="")
|
||||
|
||||
# Page content (freeform text, admin-editable)
|
||||
hero_subtitle = Column(Text, nullable=False, default="")
|
||||
about_intro = Column(Text, nullable=False, default="")
|
||||
about_donation_text = Column(Text, nullable=False, default="")
|
||||
schedule_intro = Column(Text, nullable=False, default="")
|
||||
events_intro = Column(Text, nullable=False, default="")
|
||||
|
||||
# Color theme — hex strings, e.g. "#1a3a5c"
|
||||
# server_default is required so PostgreSQL emits a proper DEFAULT in ALTER TABLE DDL
|
||||
color_primary = Column(String(7), nullable=False, server_default=text("'#1a3a5c'"), default="#1a3a5c")
|
||||
color_primary_light = Column(String(7), nullable=False, server_default=text("'#2a5a8c'"), default="#2a5a8c")
|
||||
color_primary_dark = Column(String(7), nullable=False, server_default=text("'#0e2440'"), default="#0e2440")
|
||||
color_primary_muted = Column(String(7), nullable=False, server_default=text("'#3a6a9c'"), default="#3a6a9c")
|
||||
color_accent = Column(String(7), nullable=False, server_default=text("'#e87a2e'"), default="#e87a2e")
|
||||
color_accent_light = Column(String(7), nullable=False, server_default=text("'#f09a4e'"), default="#f09a4e")
|
||||
color_accent_dark = Column(String(7), nullable=False, server_default=text("'#c85a1e'"), default="#c85a1e")
|
||||
color_accent_muted = Column(String(7), nullable=False, server_default=text("'#f0a86a'"), default="#f0a86a")
|
||||
color_background = Column(String(7), nullable=False, server_default=text("'#faf6f0'"), default="#faf6f0")
|
||||
color_text = Column(String(7), nullable=False, server_default=text("'#3a3632'"), default="#3a3632")
|
||||
color_success = Column(String(7), nullable=False, server_default=text("'#4a9e4f'"), default="#4a9e4f")
|
||||
color_danger = Column(String(7), nullable=False, server_default=text("'#c83030'"), default="#c83030")
|
||||
color_info = Column(String(7), nullable=False, server_default=text("'#3a8abf'"), default="#3a8abf")
|
||||
|
||||
# Neutral colors
|
||||
color_white = Column(String(7), nullable=False, server_default=text("'#ffffff'"), default="#ffffff")
|
||||
color_light = Column(String(7), nullable=False, server_default=text("'#f0ebe3'"), default="#f0ebe3")
|
||||
color_medium = Column(String(7), nullable=False, server_default=text("'#8a8580'"), default="#8a8580")
|
||||
color_black = Column(String(7), nullable=False, server_default=text("'#1a1816'"), default="#1a1816")
|
||||
|
||||
|
||||
class HistoryEntry(Base):
|
||||
__tablename__ = "history_entries"
|
||||
@@ -159,3 +191,89 @@ class StorageBlob(Base):
|
||||
size = Column(Integer, nullable=False)
|
||||
data = Column(LargeBinary, nullable=False)
|
||||
created_at = Column(DateTime, nullable=False, default=datetime.utcnow)
|
||||
|
||||
|
||||
# ── Visitor Stats ──────────────────────────────────────────────
|
||||
|
||||
class VisitorStatDaily(Base):
|
||||
"""Daily aggregated visitor statistics."""
|
||||
__tablename__ = "visitor_stats_daily"
|
||||
|
||||
id = Column(Integer, primary_key=True, autoincrement=True)
|
||||
date = Column(Date, nullable=False, unique=True)
|
||||
unique_visitors = Column(Integer, nullable=False, default=0)
|
||||
|
||||
__table_args__ = (
|
||||
Index("idx_visitor_stats_date", "date"),
|
||||
)
|
||||
|
||||
|
||||
class VisitorStatGeo(Base):
|
||||
"""Geographic breakdown of visitors per day."""
|
||||
__tablename__ = "visitor_stats_geo"
|
||||
|
||||
id = Column(Integer, primary_key=True, autoincrement=True)
|
||||
date = Column(Date, nullable=False)
|
||||
country = Column(String(100), nullable=True)
|
||||
country_code = Column(String(10), nullable=True)
|
||||
city = Column(String(200), nullable=True)
|
||||
latitude = Column(Float, nullable=True)
|
||||
longitude = Column(Float, nullable=True)
|
||||
visitor_count = Column(Integer, nullable=False, default=0)
|
||||
|
||||
__table_args__ = (
|
||||
Index("idx_visitor_stats_geo_date", "date"),
|
||||
Index("idx_visitor_stats_geo_country", "country_code"),
|
||||
)
|
||||
|
||||
|
||||
class LogParseState(Base):
|
||||
"""Tracks the last successfully parsed log date."""
|
||||
__tablename__ = "log_parse_state"
|
||||
|
||||
id = Column(Integer, primary_key=True, autoincrement=True)
|
||||
key = Column(String(20), unique=True, nullable=False, default="default")
|
||||
last_parsed_date = Column(Date, nullable=False)
|
||||
updated_at = Column(DateTime, nullable=False, default=datetime.utcnow)
|
||||
|
||||
|
||||
# ── Mobile Build Orchestration ─────────────────────────────────
|
||||
|
||||
class MobileBuildRequest(Base):
|
||||
__tablename__ = "mobile_build_requests"
|
||||
|
||||
id = Column(Integer, primary_key=True, autoincrement=True)
|
||||
tenant_key = Column(String(80), nullable=False, default="default")
|
||||
platform_android = Column(Boolean, nullable=False, default=False)
|
||||
platform_ios = Column(Boolean, nullable=False, default=False)
|
||||
android_application_id = Column(String(200), nullable=True)
|
||||
ios_bundle_id = Column(String(200), nullable=True)
|
||||
version_name = Column(String(40), nullable=False)
|
||||
build_number = Column(String(40), nullable=False)
|
||||
release_profile = Column(String(80), nullable=False, default="release")
|
||||
|
||||
status = Column(String(20), nullable=False, default="draft")
|
||||
preflight_passed = Column(Boolean, nullable=False, default=False)
|
||||
preflight_report = Column(Text, nullable=False, default="")
|
||||
build_log = Column(Text, nullable=False, default="")
|
||||
error_message = Column(Text, nullable=True)
|
||||
|
||||
created_by_id = Column(Integer, ForeignKey("users.id", ondelete="SET NULL"), nullable=True)
|
||||
created_at = Column(DateTime, nullable=False, default=datetime.utcnow)
|
||||
updated_at = Column(DateTime, nullable=False, default=datetime.utcnow)
|
||||
started_at = Column(DateTime, nullable=True)
|
||||
completed_at = Column(DateTime, nullable=True)
|
||||
|
||||
|
||||
class MobileBuildArtifact(Base):
|
||||
__tablename__ = "mobile_build_artifacts"
|
||||
|
||||
id = Column(Integer, primary_key=True, autoincrement=True)
|
||||
request_id = Column(Integer, ForeignKey("mobile_build_requests.id", ondelete="CASCADE"), nullable=False, index=True)
|
||||
platform = Column(String(20), nullable=False)
|
||||
artifact_type = Column(String(20), nullable=False)
|
||||
file_name = Column(String(260), nullable=False)
|
||||
file_path = Column(String(800), nullable=False)
|
||||
sha256 = Column(String(64), nullable=False)
|
||||
size = Column(Integer, nullable=False)
|
||||
created_at = Column(DateTime, nullable=False, default=datetime.utcnow)
|
||||
|
||||
+134
-1
@@ -1,7 +1,7 @@
|
||||
from datetime import date as _date
|
||||
from typing import Optional
|
||||
|
||||
from pydantic import BaseModel, computed_field, field_validator
|
||||
from pydantic import BaseModel, computed_field
|
||||
|
||||
|
||||
# ── Shared ───────────────────────────────────────────────
|
||||
@@ -78,6 +78,30 @@ class StationConfigResponse(BaseModel):
|
||||
play_store_icon_url: str
|
||||
play_store_url: str
|
||||
app_store_embed_html: str
|
||||
# Page content
|
||||
hero_subtitle: str
|
||||
about_intro: str
|
||||
about_donation_text: str
|
||||
schedule_intro: str
|
||||
events_intro: str
|
||||
# Color theme
|
||||
color_primary: str
|
||||
color_primary_light: str
|
||||
color_primary_dark: str
|
||||
color_primary_muted: str
|
||||
color_accent: str
|
||||
color_accent_light: str
|
||||
color_accent_dark: str
|
||||
color_accent_muted: str
|
||||
color_background: str
|
||||
color_text: str
|
||||
color_success: str
|
||||
color_danger: str
|
||||
color_info: str
|
||||
color_white: str
|
||||
color_light: str
|
||||
color_medium: str
|
||||
color_black: str
|
||||
|
||||
model_config = {"from_attributes": True}
|
||||
|
||||
@@ -105,6 +129,30 @@ class StationConfigUpdate(BaseModel):
|
||||
play_store_icon_url: Optional[str] = None
|
||||
play_store_url: Optional[str] = None
|
||||
app_store_embed_html: Optional[str] = None
|
||||
# Page content
|
||||
hero_subtitle: Optional[str] = None
|
||||
about_intro: Optional[str] = None
|
||||
about_donation_text: Optional[str] = None
|
||||
schedule_intro: Optional[str] = None
|
||||
events_intro: Optional[str] = None
|
||||
# Color theme
|
||||
color_primary: Optional[str] = None
|
||||
color_primary_light: Optional[str] = None
|
||||
color_primary_dark: Optional[str] = None
|
||||
color_primary_muted: Optional[str] = None
|
||||
color_accent: Optional[str] = None
|
||||
color_accent_light: Optional[str] = None
|
||||
color_accent_dark: Optional[str] = None
|
||||
color_accent_muted: Optional[str] = None
|
||||
color_background: Optional[str] = None
|
||||
color_text: Optional[str] = None
|
||||
color_success: Optional[str] = None
|
||||
color_danger: Optional[str] = None
|
||||
color_info: Optional[str] = None
|
||||
color_white: Optional[str] = None
|
||||
color_light: Optional[str] = None
|
||||
color_medium: Optional[str] = None
|
||||
color_black: Optional[str] = None
|
||||
|
||||
|
||||
# ── HistoryEntry ──────────────────────────────────────────
|
||||
@@ -284,3 +332,88 @@ class ShowUpdate(BaseModel):
|
||||
display_order: Optional[int] = None
|
||||
active: Optional[bool] = None
|
||||
schedules: Optional[list[ShowScheduleCreate]] = None
|
||||
|
||||
|
||||
# ── Stats ────────────────────────────────────────────────────────
|
||||
|
||||
class StatsSummaryResponse(BaseModel):
|
||||
total_unique_visitors: int
|
||||
|
||||
|
||||
class TimeSeriesPoint(BaseModel):
|
||||
date: str
|
||||
unique_visitors: int
|
||||
|
||||
|
||||
class GeoStatResponse(BaseModel):
|
||||
country: str
|
||||
country_code: str
|
||||
latitude: Optional[float] = None
|
||||
longitude: Optional[float] = None
|
||||
visitors: int
|
||||
|
||||
|
||||
class ParseResultResponse(BaseModel):
|
||||
files_processed: int
|
||||
rows_inserted: int
|
||||
|
||||
|
||||
# ── Mobile Build Orchestration ───────────────────────────
|
||||
|
||||
class MobileBuildDefaultsResponse(BaseModel):
|
||||
tenant_key: str
|
||||
app_name: str
|
||||
tagline: str
|
||||
support_email: str
|
||||
website: str
|
||||
stream_url: str
|
||||
stream_metadata_url: str
|
||||
|
||||
|
||||
class MobileBuildCreate(BaseModel):
|
||||
platform_android: bool = False
|
||||
platform_ios: bool = False
|
||||
android_application_id: Optional[str] = None
|
||||
ios_bundle_id: Optional[str] = None
|
||||
version_name: str
|
||||
build_number: str
|
||||
release_profile: str = "release"
|
||||
|
||||
|
||||
class MobileBuildArtifactResponse(BaseModel):
|
||||
id: int
|
||||
request_id: int
|
||||
platform: str
|
||||
artifact_type: str
|
||||
file_name: str
|
||||
sha256: str
|
||||
size: int
|
||||
created_at: str
|
||||
download_url: str
|
||||
|
||||
|
||||
class MobileBuildRequestResponse(BaseModel):
|
||||
id: int
|
||||
tenant_key: str
|
||||
platform_android: bool
|
||||
platform_ios: bool
|
||||
android_application_id: Optional[str]
|
||||
ios_bundle_id: Optional[str]
|
||||
version_name: str
|
||||
build_number: str
|
||||
release_profile: str
|
||||
status: str
|
||||
preflight_passed: bool
|
||||
preflight_report: str
|
||||
build_log: str
|
||||
error_message: Optional[str]
|
||||
created_at: str
|
||||
updated_at: str
|
||||
started_at: Optional[str]
|
||||
completed_at: Optional[str]
|
||||
artifacts: list[MobileBuildArtifactResponse]
|
||||
|
||||
|
||||
class MobileBuildPreflightResponse(BaseModel):
|
||||
passed: bool
|
||||
issues: list[str]
|
||||
|
||||
@@ -0,0 +1,279 @@
|
||||
"""Theme export: read StationConfig from the DB and dump a theme.json file.
|
||||
|
||||
This module provides functions to:
|
||||
- Build a clean, consumer-friendly JSON structure from StationConfig.
|
||||
- Embed actual image bytes as base64 data URIs for offline build use.
|
||||
- Write it atomically to disk (for scp by build machines).
|
||||
- Check whether the file exists and return its metadata.
|
||||
"""
|
||||
|
||||
import base64
|
||||
import json
|
||||
import logging
|
||||
import re
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
from typing import Any, Dict
|
||||
|
||||
from app.config import settings
|
||||
from app.database import async_session
|
||||
from app.models import StationConfig, StorageBlob
|
||||
from sqlalchemy import select
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def resolve_image_url(base_url: str, relative_url: str) -> str | None:
|
||||
"""Convert a relative image URL to an absolute one.
|
||||
|
||||
If the URL is already absolute (starts with http:// or https://), return as-is.
|
||||
Returns None for empty strings.
|
||||
"""
|
||||
if not relative_url:
|
||||
return None
|
||||
if relative_url.startswith(("http://", "https://")):
|
||||
return relative_url
|
||||
path = relative_url.lstrip("/")
|
||||
base = base_url.rstrip("/")
|
||||
return f"{base}/{path}"
|
||||
|
||||
|
||||
def _detect_mime_from_extension(path: str) -> str:
|
||||
"""Guess MIME type from file extension."""
|
||||
ext = Path(path).suffix.lower()
|
||||
return {
|
||||
".svg": "image/svg+xml",
|
||||
".png": "image/png",
|
||||
".jpg": "image/jpeg",
|
||||
".jpeg": "image/jpeg",
|
||||
".webp": "image/webp",
|
||||
".gif": "image/gif",
|
||||
".ico": "image/x-icon",
|
||||
".avif": "image/avif",
|
||||
}.get(ext, "application/octet-stream")
|
||||
|
||||
|
||||
async def fetch_image_bytes(url: str) -> tuple[bytes, str] | None:
|
||||
"""Fetch image bytes from any source the station config uses.
|
||||
|
||||
Resolves three URL patterns:
|
||||
- /api/storage/{blob_id} → read from StorageBlob in DB
|
||||
- Relative paths (e.g. svg/logo.svg) → read from STATIC_FILES_DIR on disk
|
||||
- Absolute HTTP(S) URLs → fetch via httpx
|
||||
|
||||
Returns (raw_bytes, mime_type) or None on failure.
|
||||
"""
|
||||
if not url:
|
||||
return None
|
||||
|
||||
# Database blob
|
||||
blob_match = re.match(r"^/api/storage/([a-f0-9]+)$", url)
|
||||
if blob_match:
|
||||
blob_id = blob_match.group(1)
|
||||
async with async_session() as blob_session:
|
||||
result = await blob_session.execute(
|
||||
select(StorageBlob).where(StorageBlob.blob_id == blob_id)
|
||||
)
|
||||
blob = result.scalar_one_or_none()
|
||||
if blob:
|
||||
return (blob.data, blob.content_type)
|
||||
return None
|
||||
|
||||
# Static file on disk or internal upstream HTTP
|
||||
if not url.startswith(("http://", "https://")):
|
||||
# Try local filesystem first
|
||||
candidates: list[Path] = [Path(settings.STATIC_FILES_DIR)]
|
||||
# Dev fallback: if the configured path (production nginx root) doesn't
|
||||
# contain the file, try common Angular dist/public locations.
|
||||
if not (Path(settings.STATIC_FILES_DIR) / url.lstrip("/")).is_file():
|
||||
candidates.extend([
|
||||
Path(__file__).resolve().parents[2] / "dist" / "radio-station" / "browser",
|
||||
Path(__file__).resolve().parents[2] / "public",
|
||||
])
|
||||
|
||||
for static_dir in candidates:
|
||||
if not static_dir.is_dir():
|
||||
continue
|
||||
file_path = (static_dir / url.lstrip("/")).resolve()
|
||||
# Safety: must be inside static_dir
|
||||
try:
|
||||
file_path.relative_to(static_dir.resolve())
|
||||
except ValueError:
|
||||
continue
|
||||
if file_path.is_file():
|
||||
data = file_path.read_bytes()
|
||||
mime = _detect_mime_from_extension(url)
|
||||
return (data, mime)
|
||||
|
||||
# Not on local disk — try internal upstream (Docker/K8s) or public URL
|
||||
upstream_base = settings.STATIC_UPSTREAM_URL or settings.WEBSITE_URL
|
||||
fetch_url = f"{upstream_base.rstrip('/')}/{url.lstrip('/')}"
|
||||
|
||||
try:
|
||||
import httpx
|
||||
async with httpx.AsyncClient() as client:
|
||||
resp = await client.get(fetch_url, follow_redirects=True, timeout=10)
|
||||
if resp.status_code == 200:
|
||||
content_type = resp.headers.get("content-type", "application/octet-stream")
|
||||
return (resp.content, content_type)
|
||||
except Exception as e:
|
||||
logger.warning("Failed to fetch image %s from %s: %s", url, fetch_url, e)
|
||||
return None
|
||||
|
||||
# Remote HTTP fetch
|
||||
try:
|
||||
import httpx
|
||||
async with httpx.AsyncClient() as client:
|
||||
resp = await client.get(url, follow_redirects=True, timeout=10)
|
||||
if resp.status_code == 200:
|
||||
content_type = resp.headers.get("content-type", "application/octet-stream")
|
||||
return (resp.content, content_type)
|
||||
except Exception as e:
|
||||
logger.warning("Failed to fetch remote image %s: %s", url, e)
|
||||
return None
|
||||
|
||||
|
||||
def build_theme_dict(config: StationConfig) -> Dict[str, Any]:
|
||||
"""Build the theme.json dict from a StationConfig ORM object.
|
||||
|
||||
Returns the theme structure with URL-based images. For embedded
|
||||
(base64) images, use build_theme_with_embedded_images() instead.
|
||||
"""
|
||||
base_url = settings.WEBSITE_URL
|
||||
|
||||
images: Dict[str, str] = {}
|
||||
for key, field in [
|
||||
("logo", config.logo_url),
|
||||
("hero_background", config.hero_background_url),
|
||||
("hero_icon", config.hero_icon_url),
|
||||
("hero_divider", config.hero_divider_url),
|
||||
]:
|
||||
resolved = resolve_image_url(base_url, field)
|
||||
if resolved:
|
||||
images[key] = resolved
|
||||
|
||||
return {
|
||||
"version": "1.0",
|
||||
"generated_at": datetime.now(timezone.utc).isoformat(),
|
||||
"station": {
|
||||
"callsign": config.callsign,
|
||||
"name_primary": config.name_primary,
|
||||
"name_secondary": config.name_secondary,
|
||||
"tagline": config.tagline,
|
||||
"frequency": config.frequency,
|
||||
},
|
||||
"streams": {
|
||||
"url": config.stream_url or None,
|
||||
"metadata_url": config.stream_metadata_url or None,
|
||||
},
|
||||
"colors": {
|
||||
"primary": config.color_primary,
|
||||
"primary_light": config.color_primary_light,
|
||||
"primary_dark": config.color_primary_dark,
|
||||
"primary_muted": config.color_primary_muted,
|
||||
"accent": config.color_accent,
|
||||
"accent_light": config.color_accent_light,
|
||||
"accent_dark": config.color_accent_dark,
|
||||
"accent_muted": config.color_accent_muted,
|
||||
"background": config.color_background,
|
||||
"text": config.color_text,
|
||||
"success": config.color_success,
|
||||
"danger": config.color_danger,
|
||||
"info": config.color_info,
|
||||
"white": config.color_white,
|
||||
"light": config.color_light,
|
||||
"medium": config.color_medium,
|
||||
"black": config.color_black,
|
||||
},
|
||||
"images": images,
|
||||
}
|
||||
|
||||
|
||||
async def build_theme_with_embedded_images(
|
||||
config: StationConfig,
|
||||
) -> Dict[str, Any]:
|
||||
"""Build theme dict with images embedded as base64 data URIs.
|
||||
|
||||
Fetches actual image bytes from static files, database blobs, or
|
||||
remote URLs. Failed fetches are logged and omitted (graceful).
|
||||
The original URL-based `images` section is kept for reference.
|
||||
"""
|
||||
theme = build_theme_dict(config)
|
||||
|
||||
images_embedded: Dict[str, Dict[str, str]] = {}
|
||||
image_fields = [
|
||||
("logo", config.logo_url),
|
||||
("hero_background", config.hero_background_url),
|
||||
("hero_icon", config.hero_icon_url),
|
||||
("hero_divider", config.hero_divider_url),
|
||||
]
|
||||
|
||||
for key, url in image_fields:
|
||||
result = await fetch_image_bytes(url)
|
||||
if result is None:
|
||||
logger.warning("Could not fetch image for %s (%s) — skipping embed", key, url)
|
||||
continue
|
||||
data, mime_type = result
|
||||
b64 = base64.b64encode(data).decode("ascii")
|
||||
images_embedded[key] = {
|
||||
"content_type": mime_type,
|
||||
"size": len(data),
|
||||
"data": b64,
|
||||
}
|
||||
|
||||
theme["images_embedded"] = images_embedded
|
||||
return theme
|
||||
|
||||
|
||||
async def write_theme_json() -> Dict[str, Any]:
|
||||
"""Read the current station config and write theme.json to disk.
|
||||
|
||||
Fetches actual image bytes and embeds them as base64 data URIs
|
||||
in an `images_embedded` section for offline build use.
|
||||
|
||||
Uses atomic write (write to .tmp, then os.replace) to prevent
|
||||
reading a partially-written file.
|
||||
|
||||
Returns the theme dict that was written.
|
||||
"""
|
||||
async with async_session() as session:
|
||||
result = await session.execute(
|
||||
select(StationConfig).where(StationConfig.key == "default")
|
||||
)
|
||||
config = result.scalar_one_or_none()
|
||||
if config is None:
|
||||
raise RuntimeError("Station config not found in database")
|
||||
|
||||
theme = await build_theme_with_embedded_images(config)
|
||||
|
||||
# Atomic write
|
||||
path = Path(settings.THEME_JSON_PATH)
|
||||
tmp_path = path.with_suffix(".tmp")
|
||||
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
tmp_path.write_text(json.dumps(theme, indent=2) + "\n", encoding="utf-8")
|
||||
tmp_path.replace(path)
|
||||
|
||||
return theme
|
||||
|
||||
|
||||
def get_theme_status() -> Dict[str, Any]:
|
||||
"""Check if theme.json exists on disk and return its metadata."""
|
||||
path = Path(settings.THEME_JSON_PATH)
|
||||
if not path.exists():
|
||||
return {"exists": False, "path": str(path)}
|
||||
|
||||
stat = path.stat()
|
||||
try:
|
||||
data = json.loads(path.read_text(encoding="utf-8"))
|
||||
generated_at = data.get("generated_at")
|
||||
except Exception:
|
||||
generated_at = None
|
||||
|
||||
return {
|
||||
"exists": True,
|
||||
"path": str(path),
|
||||
"generated_at": generated_at,
|
||||
"size": stat.st_size,
|
||||
}
|
||||
Binary file not shown.
@@ -0,0 +1,15 @@
|
||||
# GeoLite2 City Database
|
||||
|
||||
The visitor stats dashboard uses MaxMind GeoLite2 for IP geolocation.
|
||||
|
||||
## Setup
|
||||
|
||||
1. Create a free account at [MaxMind GeoLite2](https://dev.maxmind.com/geoip/geolite2-free-geolocation-data)
|
||||
2. Download the **GeoLite2 Country** database (`.mmdb` format, not CSV) and **RENAME** it:
|
||||
3. Place the downloaded file here as `GeoLite2-City.mmdb`
|
||||
|
||||
The Docker build copies it into the container at `/app/geo/GeoLite2-City.mmdb`.
|
||||
|
||||
## Without the database
|
||||
|
||||
The backend starts but geo lookups are disabled. The dashboard shows visitor counts without country/region breakdown.
|
||||
@@ -0,0 +1,169 @@
|
||||
"""Inject test visitor log entries into the shared logs directory.
|
||||
|
||||
Writes JSON-formatted access log lines (matching the nginx kmtn_json format)
|
||||
for multiple past days so the log parser will pick them up — data is only
|
||||
aggregated a day in arrears, so all dates are strictly before today.
|
||||
|
||||
Usage:
|
||||
python inject_test_logs.py # writes to /logs (default)
|
||||
python inject_test_logs.py /tmp/logs # writes to custom path
|
||||
python inject_test_logs.py /logs 14 # writes 14 days of data
|
||||
"""
|
||||
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
import hashlib
|
||||
import struct
|
||||
from datetime import date, timedelta, timezone
|
||||
|
||||
# Real, well-known public IPs that resolve in MaxMind GeoLite2 City.
|
||||
# These are actual infrastructure IPs (DNS, CDN, search, cloud) —
|
||||
# not random addresses, so they're far more likely to be in the DB.
|
||||
# Each tuple: (ip, expected_country, expected_city)
|
||||
TEST_IPS = [
|
||||
# United States
|
||||
("8.8.8.8", "US", "Mountain View"), # Google DNS
|
||||
("8.8.4.4", "US", "Mountain View"), # Google DNS
|
||||
("216.58.214.206", "US", "Mountain View"), # Google
|
||||
("142.250.80.46", "US", "Mountain View"), # Google
|
||||
("13.107.42.14", "US", "Redmond"), # Microsoft
|
||||
("20.190.128.1", "US", "Boydton"), # Microsoft Azure
|
||||
("208.67.222.222", "US", "San Francisco"), # OpenDNS
|
||||
("208.67.220.220", "US", "San Francisco"), # OpenDNS
|
||||
("151.101.1.69", "US", "San Francisco"), # Fastly
|
||||
("52.84.74.1", "US", "Seattle"), # AWS CloudFront
|
||||
("99.86.0.1", "US", "Seattle"), # AWS
|
||||
("54.239.28.85", "US", "Seattle"), # AWS
|
||||
("185.199.108.1", "US", "San Francisco"), # GitHub Pages
|
||||
# United Kingdom
|
||||
("212.58.244.3", "GB", "London"), # Virgin Media
|
||||
("178.173.57.1", "GB", "London"), # DigitalOcean
|
||||
("51.15.48.1", "GB", "London"), # OVH UK
|
||||
# France
|
||||
("51.140.137.15", "FR", "Paris"), # OVH Paris
|
||||
("193.70.150.1", "FR", "Paris"), # Free
|
||||
# Germany
|
||||
("217.160.0.1", "DE", "Frankfurt"), # Telekom
|
||||
("212.227.156.1", "DE", "Munich"), # Deutsche Telekom
|
||||
("217.69.70.1", "DE", "Frankfurt"), # Freenet
|
||||
# Japan
|
||||
("210.170.232.46", "JP", "Tokyo"), # KDDI
|
||||
("202.214.69.1", "JP", "Tokyo"), # NTT
|
||||
("210.188.206.1", "JP", "Tokyo"), # Softbank
|
||||
# Australia
|
||||
("1.35.128.1", "AU", "Sydney"), # Telstra
|
||||
("203.50.224.1", "AU", "Melbourne"), # iiNet
|
||||
# Brazil
|
||||
("177.72.120.1", "BR", "Sao Paulo"), # Vivo
|
||||
("200.149.130.1", "BR", "Rio de Janeiro"), # Telefonica Brasil
|
||||
("189.75.1.1", "BR", "Sao Paulo"), # Claro
|
||||
# Canada
|
||||
("76.69.248.1", "CA", "Toronto"), # Bell
|
||||
("24.199.12.1", "CA", "Vancouver"), # Shaw
|
||||
("99.248.1.1", "CA", "Montreal"), # Bell
|
||||
# India
|
||||
("117.239.1.1", "IN", "Mumbai"), # Reliance Jio
|
||||
("180.151.1.1", "IN", "New Delhi"), # Airtel
|
||||
("106.192.1.1", "IN", "Bangalore"), # BSNL
|
||||
# South Korea
|
||||
("1.235.198.1", "KR", "Seoul"), # KT
|
||||
("211.36.1.1", "KR", "Seoul"), # SK Telecom
|
||||
# Netherlands
|
||||
("185.30.1.1", "NL", "Amsterdam"), # KPN
|
||||
("94.23.1.1", "NL", "Amsterdam"), # XS4ALL
|
||||
# Mexico
|
||||
("189.203.1.1", "MX", "Mexico City"), # Telmex
|
||||
("200.44.1.1", "MX", "Guadalajara"), # Telcel
|
||||
# Italy
|
||||
("217.20.1.1", "IT", "Rome"), # Telecom Italia
|
||||
("93.45.1.1", "IT", "Milan"), # TIM
|
||||
# Spain
|
||||
("213.99.1.1", "ES", "Madrid"), # Telefonica Espana
|
||||
("88.0.1.1", "ES", "Madrid"), # Movistar
|
||||
# Sweden
|
||||
("213.248.1.1", "SE", "Stockholm"), # Telia
|
||||
# Poland
|
||||
("79.133.1.1", "PL", "Warsaw"), # Orange Polska
|
||||
# Argentina
|
||||
("190.15.1.1", "AR", "Buenos Aires"), # Telecom Argentina
|
||||
# South Africa
|
||||
("196.201.1.1", "ZA", "Johannesburg"), # Telkom SA
|
||||
# Egypt
|
||||
("213.53.1.1", "EG", "Cairo"), # Telecom Egypt
|
||||
# Turkey
|
||||
("37.146.1.1", "TR", "Istanbul"), # Turk Telekom
|
||||
# Russia
|
||||
("77.88.55.1", "RU", "Moscow"), # Yandex DNS
|
||||
("5.255.255.1", "RU", "Moscow"), # Rostelecom
|
||||
# Singapore
|
||||
("182.16.1.1", "SG", "Singapore"), # Singtel
|
||||
# New Zealand
|
||||
("210.55.1.1", "NZ", "Auckland"), # Telecom NZ
|
||||
]
|
||||
|
||||
|
||||
def _deterministic_count(seed: str) -> int:
|
||||
"""Return a pseudo-random visit count (1-8) for a given seed."""
|
||||
h = hashlib.md5(seed.encode()).digest()
|
||||
return struct.unpack("I", h[:4])[0] % 8 + 1
|
||||
|
||||
|
||||
def _deterministic_hour(seed: str) -> int:
|
||||
"""Return a pseudo-random hour (0-23) for a given seed."""
|
||||
h = hashlib.md5(seed.encode()).digest()
|
||||
return struct.unpack("I", h[:4])[0] % 24
|
||||
|
||||
|
||||
def write_test_logs(logs_dir: str, days: int = 14):
|
||||
"""Write test log entries for the given number of past days.
|
||||
|
||||
Skips today (parser only processes files strictly before today).
|
||||
Writes a realistic mix of visitors per day with varying counts.
|
||||
"""
|
||||
os.makedirs(logs_dir, exist_ok=True)
|
||||
|
||||
today = date.today()
|
||||
total_entries = 0
|
||||
files_written = 0
|
||||
|
||||
# Write for the last `days` days, skipping today
|
||||
for offset in range(1, days + 1):
|
||||
file_date = today - timedelta(days=offset)
|
||||
|
||||
filename = f"access-{file_date.isoformat()}.log"
|
||||
filepath = os.path.join(logs_dir, filename)
|
||||
|
||||
entries: list[str] = []
|
||||
for ip, country, city in TEST_IPS:
|
||||
seed = f"{file_date}-{ip}"
|
||||
count = _deterministic_count(seed)
|
||||
hour = _deterministic_hour(seed)
|
||||
|
||||
for j in range(count):
|
||||
# Deterministic minute spread (hash() is non-deterministic across runs)
|
||||
ip_hash = struct.unpack("I", hashlib.md5(ip.encode()).digest()[4:8])[0]
|
||||
minute = (j * 7 + ip_hash % 60) % 60
|
||||
entry = {
|
||||
"time": f"{file_date}T{hour}:{minute:02d}:00+00:00",
|
||||
"remote_addr": ip,
|
||||
"x_forwarded_for": "-",
|
||||
}
|
||||
entries.append(json.dumps(entry))
|
||||
|
||||
with open(filepath, "a") as f:
|
||||
for line in entries:
|
||||
f.write(line + "\n")
|
||||
|
||||
total_entries += len(entries)
|
||||
files_written += 1
|
||||
|
||||
print(f"Wrote {total_entries} log entries across {files_written} daily files ({len(TEST_IPS)} unique IPs)")
|
||||
print(f"Date range: {(today - timedelta(days=days)).isoformat()} to {(today - timedelta(days=1)).isoformat()}")
|
||||
print(f"Directory: {logs_dir}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
target = sys.argv[1] if len(sys.argv) > 1 else "/logs"
|
||||
num_days = int(sys.argv[2]) if len(sys.argv) > 2 else 14
|
||||
write_test_logs(target, days=num_days)
|
||||
@@ -0,0 +1,200 @@
|
||||
"""Probe the GeoLite2 database to find IPs that resolve to real countries.
|
||||
|
||||
Run inside the api container:
|
||||
docker compose exec api python /app/probe_geo.py
|
||||
|
||||
Or locally if the DB is present:
|
||||
python probe_geo.py /path/to/GeoLite2-City.mmdb
|
||||
|
||||
Tests a curated list of well-known public IPs and prints the ones that resolve.
|
||||
"""
|
||||
|
||||
import sys
|
||||
import json
|
||||
import os
|
||||
import maxminddb
|
||||
|
||||
# Well-known public IPs from various countries
|
||||
CANDIDATE_IPS = [
|
||||
# United States
|
||||
("23.21.30.100", "US"),
|
||||
("72.14.198.234", "US"),
|
||||
("68.49.71.1", "US"),
|
||||
("208.67.222.222", "US"), # OpenDNS
|
||||
("74.125.200.100", "US"), # Google
|
||||
("151.101.1.69", "US"), # Fastly
|
||||
("142.250.80.46", "US"), # Google
|
||||
("216.58.214.206", "US"), # Google
|
||||
("172.217.164.110", "US"), # Google
|
||||
("13.107.42.14", "US"), # Microsoft
|
||||
("20.190.128.1", "US"), # Microsoft
|
||||
("52.84.74.1", "US"), # AWS CloudFront
|
||||
("99.86.0.1", "US"), # AWS
|
||||
# United Kingdom
|
||||
("81.2.69.142", "GB"),
|
||||
("212.58.224.10", "GB"),
|
||||
("51.140.137.15", "GB"),
|
||||
("178.173.57.1", "GB"),
|
||||
# France
|
||||
("51.140.137.15", "FR"),
|
||||
("193.70.150.1", "FR"),
|
||||
# Japan
|
||||
("210.170.232.46", "JP"),
|
||||
("133.244.200.1", "JP"),
|
||||
("202.214.69.1", "JP"), # NTT
|
||||
("210.188.206.1", "JP"), # Softbank
|
||||
# Germany
|
||||
("217.160.0.1", "DE"),
|
||||
("212.227.156.1", "DE"),
|
||||
("2a02:2e0::1", "DE"), # IPv6 - may not resolve
|
||||
("217.69.70.1", "DE"),
|
||||
# Australia
|
||||
("101.168.100.1", "AU"),
|
||||
("203.50.224.1", "AU"),
|
||||
("1.35.128.1", "AU"), # Telstra
|
||||
# Brazil
|
||||
("177.72.120.1", "BR"),
|
||||
("200.149.130.1", "BR"),
|
||||
("189.75.1.1", "BR"),
|
||||
# Canada
|
||||
("76.69.248.1", "CA"),
|
||||
("24.199.12.1", "CA"),
|
||||
("99.248.1.1", "CA"), # Bell
|
||||
# India
|
||||
("117.239.1.1", "IN"),
|
||||
("180.151.1.1", "IN"),
|
||||
("106.192.1.1", "IN"),
|
||||
# South Korea
|
||||
("1.235.198.1", "KR"),
|
||||
("211.36.1.1", "KR"),
|
||||
# Netherlands
|
||||
("185.30.1.1", "NL"),
|
||||
("94.23.1.1", "NL"),
|
||||
# Mexico
|
||||
("189.203.1.1", "MX"),
|
||||
("200.44.1.1", "MX"),
|
||||
# Italy
|
||||
("217.20.1.1", "IT"),
|
||||
("93.45.1.1", "IT"),
|
||||
# Spain
|
||||
("213.99.1.1", "ES"),
|
||||
("88.0.1.1", "ES"),
|
||||
# Sweden
|
||||
("213.248.1.1", "SE"),
|
||||
# Poland
|
||||
("79.133.1.1", "PL"),
|
||||
# Argentina
|
||||
("190.15.1.1", "AR"),
|
||||
# South Africa
|
||||
("196.201.1.1", "ZA"),
|
||||
# Egypt
|
||||
("213.53.1.1", "EG"),
|
||||
# Turkey
|
||||
("37.146.1.1", "TR"),
|
||||
# Russia
|
||||
("77.88.55.1", "RU"), # Yandex DNS
|
||||
("5.255.255.1", "RU"),
|
||||
]
|
||||
|
||||
|
||||
def probe(db_path: str):
|
||||
"""Test each candidate IP against the GeoLite2 database."""
|
||||
reader = maxminddb.open_database(db_path)
|
||||
results = []
|
||||
|
||||
# Deduplicate IPs but keep all expected countries
|
||||
seen_ips = set()
|
||||
for ip, expected_country in CANDIDATE_IPS:
|
||||
if ip in seen_ips:
|
||||
continue
|
||||
seen_ips.add(ip)
|
||||
|
||||
try:
|
||||
result = reader.lookup(ip)
|
||||
if result is None:
|
||||
continue
|
||||
|
||||
country = result.get("country", {})
|
||||
city = result.get("city", {})
|
||||
location = result.get("location", {})
|
||||
|
||||
country_code = country.get("iso_code", "")
|
||||
country_name = country.get("names", {}).get("en", country_code)
|
||||
city_name = city.get("names", {}).get("en", "")
|
||||
lat = location.get("latitude")
|
||||
lon = location.get("longitude")
|
||||
|
||||
results.append({
|
||||
"ip": ip,
|
||||
"country": country_name,
|
||||
"country_code": country_code,
|
||||
"city": city_name,
|
||||
"latitude": lat,
|
||||
"longitude": lon,
|
||||
})
|
||||
except Exception as e:
|
||||
print(f" ERROR looking up {ip}: {e}")
|
||||
|
||||
reader.close()
|
||||
return results
|
||||
|
||||
|
||||
def main():
|
||||
# Determine DB path
|
||||
db_path = None
|
||||
|
||||
if len(sys.argv) > 1:
|
||||
# Explicit path provided
|
||||
db_path = sys.argv[1]
|
||||
else:
|
||||
# Try common locations
|
||||
candidates = [
|
||||
"/app/geo/GeoLite2-City.mmdb", # container path
|
||||
os.path.join(os.path.dirname(__file__), "geo", "GeoLite2-City.mmdb"), # local
|
||||
]
|
||||
for p in candidates:
|
||||
if os.path.exists(p):
|
||||
db_path = p
|
||||
break
|
||||
|
||||
if db_path is None:
|
||||
print("GeoLite2-City.mmdb not found.")
|
||||
print("Usage:")
|
||||
print(" docker compose exec api python /app/probe_geo.py")
|
||||
print(" python probe_geo.py /path/to/GeoLite2-City.mmdb")
|
||||
sys.exit(1)
|
||||
|
||||
print(f"Probing GeoLite2 at: {db_path}")
|
||||
results = probe(db_path)
|
||||
|
||||
if not results:
|
||||
print("No IPs resolved! The database might be empty or outdated.")
|
||||
sys.exit(1)
|
||||
|
||||
print(f"\nResolved {len(results)} IPs:\n")
|
||||
|
||||
# Group by country
|
||||
by_country = {}
|
||||
for r in results:
|
||||
by_country.setdefault(r["country_code"], []).append(r)
|
||||
|
||||
for cc in sorted(by_country.keys()):
|
||||
entries = by_country[cc]
|
||||
print(f" {cc} ({entries[0]['country']}):")
|
||||
for e in entries:
|
||||
print(f" {e['ip']:20s} -> {e['city'] or 'N/A':20s} ({e['latitude']}, {e['longitude']})")
|
||||
|
||||
# Output as Python tuples for inject_test_logs.py
|
||||
print(f"\n\nTuples for inject_test_logs.py TEST_IPS:")
|
||||
print("TEST_IPS = [")
|
||||
for r in results:
|
||||
print(f' ("{r["ip"]}", "{r["country"]}", "{r["city"]}"),')
|
||||
print("]")
|
||||
|
||||
# Also output JSON for convenience
|
||||
print(f"\nJSON output:")
|
||||
print(json.dumps(results, indent=2))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,34 @@
|
||||
[project]
|
||||
name = "kmtnflower-backend"
|
||||
version = "1.0.0"
|
||||
requires-python = ">=3.12"
|
||||
|
||||
[tool.pytest.ini_options]
|
||||
testpaths = ["tests"]
|
||||
python_files = ["test_*.py"]
|
||||
python_classes = ["Test*"]
|
||||
python_functions = ["test_*"]
|
||||
asyncio_mode = "auto"
|
||||
addopts = [
|
||||
"--strict-markers",
|
||||
"--strict-config",
|
||||
"-v",
|
||||
]
|
||||
|
||||
[tool.coverage.run]
|
||||
source = ["app"]
|
||||
branch = true
|
||||
omit = ["*/tests/*", "*/__pycache__/*", "seed.py", "inject_test_logs.py", "probe_geo.py"]
|
||||
|
||||
[tool.coverage.report]
|
||||
show_missing = true
|
||||
skip_empty = true
|
||||
fail_under = 10
|
||||
exclude_lines = [
|
||||
"pragma: no cover",
|
||||
"if TYPE_CHECKING:",
|
||||
"raise NotImplementedError",
|
||||
]
|
||||
|
||||
[tool.coverage.html]
|
||||
directory = "htmlcov"
|
||||
@@ -0,0 +1,4 @@
|
||||
pytest>=8.0
|
||||
pytest-asyncio>=0.24
|
||||
pytest-cov>=5.0
|
||||
aiosqlite>=0.20
|
||||
@@ -11,3 +11,4 @@ pydantic==2.10.4
|
||||
pydantic-settings==2.7.1
|
||||
python-jose[cryptography]==3.3.0
|
||||
httpx==0.27.2
|
||||
maxminddb==2.6.2
|
||||
|
||||
@@ -217,6 +217,30 @@ STATION_CONFIG: dict = {
|
||||
"play_store_icon_url": "",
|
||||
"play_store_url": "",
|
||||
"app_store_embed_html": "",
|
||||
# Page content
|
||||
"hero_subtitle": "Your community-powered, nonprofit radio station — bringing music, stories, and connection to the mountain towns and beyond since 1987.",
|
||||
"about_intro": "For nearly four decades, KMountain Flower Radio has been the voice of our community — a nonprofit, listener-supported radio station dedicated to music, local news, and the stories that bind mountain towns together.",
|
||||
"about_donation_text": "As a 501(c)(3) nonprofit, every donation keeps our airwaves free and our signal strong. Members receive exclusive perks and the satisfaction of knowing they helped make this possible.",
|
||||
"schedule_intro": "All times are local (Mountain Time). Every show is free to listen to live at 98.7 FM or via our online stream.",
|
||||
"events_intro": "Come meet the team, enjoy live music, and support public radio in person. All events are open to the community — friends and family of listeners welcome.",
|
||||
# Color theme defaults (match _variables.scss)
|
||||
"color_primary": "#1a3a5c",
|
||||
"color_primary_light": "#2a5a8c",
|
||||
"color_primary_dark": "#0e2440",
|
||||
"color_primary_muted": "#3a6a9c",
|
||||
"color_accent": "#e87a2e",
|
||||
"color_accent_light": "#f09a4e",
|
||||
"color_accent_dark": "#c85a1e",
|
||||
"color_accent_muted": "#f0a86a",
|
||||
"color_background": "#faf6f0",
|
||||
"color_text": "#3a3632",
|
||||
"color_success": "#4a9e4f",
|
||||
"color_danger": "#c83030",
|
||||
"color_info": "#3a8abf",
|
||||
"color_white": "#ffffff",
|
||||
"color_light": "#f0ebe3",
|
||||
"color_medium": "#8a8580",
|
||||
"color_black": "#1a1816",
|
||||
}
|
||||
|
||||
HISTORY_ENTRIES: list[dict] = [
|
||||
|
||||
@@ -0,0 +1,21 @@
|
||||
"""Shared test fixtures for kmtnflower backend tests."""
|
||||
|
||||
import pytest
|
||||
from unittest.mock import patch
|
||||
|
||||
# Override settings for tests — use SQLite so we don't need PostgreSQL
|
||||
@pytest.fixture(autouse=True)
|
||||
def override_settings():
|
||||
"""Set test-only environment variables before any import of app modules."""
|
||||
env = {
|
||||
"KMTN_DATABASE_URL": "sqlite+aiosqlite:///:memory:",
|
||||
"KMTN_ENV": "test",
|
||||
"KMTN_JWT_SECRET_KEY": "test-secret-key-do-not-use",
|
||||
"KMTN_ADMIN_USERNAME": "testadmin",
|
||||
"KMTN_ADMIN_PASSWORD": "testpass123",
|
||||
"KMTN_LOGS_DIR": "/tmp/kmtn_test_logs",
|
||||
"KMTN_GEOLITE2_DB_PATH": "/nonexistent/GeoLite2-City.mmdb",
|
||||
"KMTN_THEME_JSON_PATH": "/tmp/kmtn_test_theme.json",
|
||||
}
|
||||
with patch.dict("os.environ", env, clear=False):
|
||||
yield env
|
||||
@@ -0,0 +1,141 @@
|
||||
"""Tests for app.auth — JWT token creation, verification, and decoding."""
|
||||
|
||||
import pytest
|
||||
from datetime import datetime, timedelta, timezone
|
||||
from fastapi import HTTPException
|
||||
from jose import jwt
|
||||
|
||||
from app.auth import (
|
||||
create_access_token,
|
||||
create_download_token,
|
||||
verify_download_token,
|
||||
decode_token,
|
||||
)
|
||||
from app.config import settings
|
||||
|
||||
|
||||
class TestCreateAccessToken:
|
||||
"""Tests for create_access_token()."""
|
||||
|
||||
def test_returns_string_token(self):
|
||||
token = create_access_token(user_id=1, is_admin=True)
|
||||
assert isinstance(token, str)
|
||||
assert len(token) > 0
|
||||
|
||||
def test_token_contains_correct_payload(self):
|
||||
token = create_access_token(user_id=42, is_admin=False)
|
||||
payload = jwt.decode(token, settings.JWT_SECRET_KEY, algorithms=["HS256"])
|
||||
assert payload["sub"] == "42"
|
||||
assert payload["is_admin"] is False
|
||||
assert "exp" in payload
|
||||
|
||||
def test_admin_flag_is_preserved(self):
|
||||
admin_token = create_access_token(user_id=1, is_admin=True)
|
||||
payload = jwt.decode(admin_token, settings.JWT_SECRET_KEY, algorithms=["HS256"])
|
||||
assert payload["is_admin"] is True
|
||||
|
||||
def test_token_expires_in_configured_minutes(self):
|
||||
token = create_access_token(user_id=1, is_admin=True)
|
||||
payload = jwt.decode(token, settings.JWT_SECRET_KEY, algorithms=["HS256"])
|
||||
expected_exp = datetime.now(timezone.utc) + timedelta(minutes=settings.JWT_EXPIRE_MINUTES)
|
||||
# Allow 1s drift
|
||||
assert abs(payload["exp"] - expected_exp.timestamp()) < 2
|
||||
|
||||
|
||||
class TestCreateDownloadToken:
|
||||
"""Tests for create_download_token()."""
|
||||
|
||||
def test_download_token_contains_artifact_id(self):
|
||||
token = create_download_token(artifact_id=99)
|
||||
payload = jwt.decode(token, settings.JWT_SECRET_KEY, algorithms=["HS256"])
|
||||
assert payload["sub"] == "download"
|
||||
assert payload["artifact_id"] == 99
|
||||
|
||||
def test_download_token_expires_quickly(self):
|
||||
token = create_download_token(artifact_id=1)
|
||||
payload = jwt.decode(token, settings.JWT_SECRET_KEY, algorithms=["HS256"])
|
||||
expected_exp = datetime.now(timezone.utc) + timedelta(minutes=settings.DOWNLOAD_TOKEN_MINUTES)
|
||||
assert abs(payload["exp"] - expected_exp.timestamp()) < 2
|
||||
|
||||
|
||||
class TestVerifyDownloadToken:
|
||||
"""Tests for verify_download_token()."""
|
||||
|
||||
def test_valid_download_token_returns_artifact_id(self):
|
||||
token = create_download_token(artifact_id=55)
|
||||
result = verify_download_token(token)
|
||||
assert result == 55
|
||||
|
||||
def test_access_token_rejected_as_download_token(self):
|
||||
token = create_access_token(user_id=1, is_admin=True)
|
||||
result = verify_download_token(token)
|
||||
assert result is None
|
||||
|
||||
def test_expired_token_returns_none(self):
|
||||
payload = {
|
||||
"sub": "download",
|
||||
"artifact_id": 1,
|
||||
"exp": datetime.now(timezone.utc) - timedelta(minutes=1),
|
||||
}
|
||||
expired_token = jwt.encode(payload, settings.JWT_SECRET_KEY, algorithm="HS256")
|
||||
result = verify_download_token(expired_token)
|
||||
assert result is None
|
||||
|
||||
def test_tampered_token_returns_none(self):
|
||||
token = create_download_token(artifact_id=10)
|
||||
tampered = token[:-5] + "xxxxx"
|
||||
result = verify_download_token(tampered)
|
||||
assert result is None
|
||||
|
||||
def test_wrong_secret_returns_none(self):
|
||||
payload = {
|
||||
"sub": "download",
|
||||
"artifact_id": 1,
|
||||
"exp": datetime.now(timezone.utc) + timedelta(minutes=5),
|
||||
}
|
||||
wrong_token = jwt.encode(payload, "wrong-secret", algorithm="HS256")
|
||||
result = verify_download_token(wrong_token)
|
||||
assert result is None
|
||||
|
||||
def test_non_integer_artifact_id_returns_none(self):
|
||||
payload = {
|
||||
"sub": "download",
|
||||
"artifact_id": "not_an_int",
|
||||
"exp": datetime.now(timezone.utc) + timedelta(minutes=5),
|
||||
}
|
||||
token = jwt.encode(payload, settings.JWT_SECRET_KEY, algorithm="HS256")
|
||||
result = verify_download_token(token)
|
||||
assert result is None
|
||||
|
||||
|
||||
class TestDecodeToken:
|
||||
"""Tests for decode_token()."""
|
||||
|
||||
def test_valid_token_returns_payload(self):
|
||||
token = create_access_token(user_id=7, is_admin=False)
|
||||
payload = decode_token(token)
|
||||
assert payload["sub"] == "7"
|
||||
|
||||
def test_expired_token_raises_401(self):
|
||||
payload = {
|
||||
"sub": "1",
|
||||
"exp": datetime.now(timezone.utc) - timedelta(hours=1),
|
||||
}
|
||||
expired = jwt.encode(payload, settings.JWT_SECRET_KEY, algorithm="HS256")
|
||||
with pytest.raises(HTTPException) as exc_info:
|
||||
decode_token(expired)
|
||||
assert exc_info.value.status_code == 401
|
||||
|
||||
def test_invalid_signature_raises_401(self):
|
||||
with pytest.raises(HTTPException) as exc_info:
|
||||
decode_token("not.a.token")
|
||||
assert exc_info.value.status_code == 401
|
||||
|
||||
def test_token_without_sub_raises_401(self):
|
||||
payload = {
|
||||
"exp": datetime.now(timezone.utc) + timedelta(hours=1),
|
||||
}
|
||||
token = jwt.encode(payload, settings.JWT_SECRET_KEY, algorithm="HS256")
|
||||
with pytest.raises(HTTPException) as exc_info:
|
||||
decode_token(token)
|
||||
assert exc_info.value.status_code == 401
|
||||
@@ -0,0 +1,117 @@
|
||||
"""Tests for app.config — Settings model and environment variable overrides."""
|
||||
|
||||
import os
|
||||
import pytest
|
||||
|
||||
from app.config import Settings
|
||||
|
||||
|
||||
def _clear_kmn_env(monkeypatch):
|
||||
"""Remove all KMTN_ prefixed env vars so Settings reads pure defaults."""
|
||||
keys = [k for k in os.environ if k.startswith("KMTN_")]
|
||||
for k in keys:
|
||||
monkeypatch.delenv(k, raising=False)
|
||||
|
||||
|
||||
class TestSettingsDefaults:
|
||||
"""Verify Settings defaults match documented behavior."""
|
||||
|
||||
def test_default_database_url(self, monkeypatch):
|
||||
_clear_kmn_env(monkeypatch)
|
||||
settings = Settings()
|
||||
assert "postgresql" in settings.DATABASE_URL
|
||||
assert "kmountain" in settings.DATABASE_URL
|
||||
|
||||
def test_default_env_is_development(self, monkeypatch):
|
||||
_clear_kmn_env(monkeypatch)
|
||||
settings = Settings()
|
||||
assert settings.ENV == "development"
|
||||
|
||||
def test_default_jwt_expire_minutes(self, monkeypatch):
|
||||
_clear_kmn_env(monkeypatch)
|
||||
settings = Settings()
|
||||
assert settings.JWT_EXPIRE_MINUTES == 1440
|
||||
|
||||
def test_default_download_token_minutes(self, monkeypatch):
|
||||
_clear_kmn_env(monkeypatch)
|
||||
settings = Settings()
|
||||
assert settings.DOWNLOAD_TOKEN_MINUTES == 10
|
||||
|
||||
def test_default_cors_origins(self, monkeypatch):
|
||||
_clear_kmn_env(monkeypatch)
|
||||
settings = Settings()
|
||||
assert "http://localhost:4200" in settings.CORS_ORIGINS
|
||||
|
||||
def test_empty_admin_credentials_by_default(self, monkeypatch):
|
||||
_clear_kmn_env(monkeypatch)
|
||||
settings = Settings()
|
||||
assert settings.ADMIN_USERNAME == ""
|
||||
assert settings.ADMIN_PASSWORD == ""
|
||||
|
||||
|
||||
class TestSettingsEnvOverride:
|
||||
"""Verify KMTN_ prefixed env vars override defaults."""
|
||||
|
||||
def test_database_url_from_env(self, monkeypatch):
|
||||
monkeypatch.setenv("KMTN_DATABASE_URL", "sqlite+aiosqlite:///test.db")
|
||||
settings = Settings()
|
||||
assert settings.DATABASE_URL == "sqlite+aiosqlite:///test.db"
|
||||
|
||||
def test_env_from_env(self, monkeypatch):
|
||||
monkeypatch.setenv("KMTN_ENV", "production")
|
||||
settings = Settings()
|
||||
assert settings.ENV == "production"
|
||||
|
||||
def test_jwt_secret_from_env(self, monkeypatch):
|
||||
monkeypatch.setenv("KMTN_JWT_SECRET_KEY", "my-custom-secret")
|
||||
settings = Settings()
|
||||
assert settings.JWT_SECRET_KEY == "my-custom-secret"
|
||||
|
||||
def test_jwt_expire_override(self, monkeypatch):
|
||||
monkeypatch.setenv("KMTN_JWT_EXPIRE_MINUTES", "60")
|
||||
settings = Settings()
|
||||
assert settings.JWT_EXPIRE_MINUTES == 60
|
||||
|
||||
def test_logs_dir_override(self, monkeypatch):
|
||||
monkeypatch.setenv("KMTN_LOGS_DIR", "/custom/logs")
|
||||
settings = Settings()
|
||||
assert settings.LOGS_DIR == "/custom/logs"
|
||||
|
||||
def test_geolite2_path_override(self, monkeypatch):
|
||||
monkeypatch.setenv("KMTN_GEOLITE2_DB_PATH", "/custom/geo.mmdb")
|
||||
settings = Settings()
|
||||
assert settings.GEOLITE2_DB_PATH == "/custom/geo.mmdb"
|
||||
|
||||
def test_theme_json_path_override(self, monkeypatch):
|
||||
monkeypatch.setenv("KMTN_THEME_JSON_PATH", "/custom/theme.json")
|
||||
settings = Settings()
|
||||
assert settings.THEME_JSON_PATH == "/custom/theme.json"
|
||||
|
||||
def test_admin_credentials_from_env(self, monkeypatch):
|
||||
monkeypatch.setenv("KMTN_ADMIN_USERNAME", "admin")
|
||||
monkeypatch.setenv("KMTN_ADMIN_PASSWORD", "secure123")
|
||||
settings = Settings()
|
||||
assert settings.ADMIN_USERNAME == "admin"
|
||||
assert settings.ADMIN_PASSWORD == "secure123"
|
||||
|
||||
def test_website_url_override(self, monkeypatch):
|
||||
monkeypatch.setenv("KMTN_WEBSITE_URL", "https://example.com")
|
||||
settings = Settings()
|
||||
assert settings.WEBSITE_URL == "https://example.com"
|
||||
|
||||
def test_static_upstream_url_override(self, monkeypatch):
|
||||
monkeypatch.setenv("KMTN_STATIC_UPSTREAM_URL", "http://frontend:80")
|
||||
settings = Settings()
|
||||
assert settings.STATIC_UPSTREAM_URL == "http://frontend:80"
|
||||
|
||||
def test_cors_origins_override(self, monkeypatch):
|
||||
# pydantic-settings parses JSON lists for list[str] fields
|
||||
monkeypatch.setenv("KMTN_CORS_ORIGINS", '["https://example.com"]')
|
||||
settings = Settings()
|
||||
assert settings.CORS_ORIGINS == ["https://example.com"]
|
||||
|
||||
def test_kmn_prefix_is_required(self, monkeypatch):
|
||||
"""Non-prefixed env vars should NOT override settings."""
|
||||
monkeypatch.setenv("JWT_SECRET_KEY", "should-not-apply")
|
||||
settings = Settings()
|
||||
assert settings.JWT_SECRET_KEY != "should-not-apply"
|
||||
@@ -0,0 +1,229 @@
|
||||
"""Tests for log_parser — IP resolution, log line parsing, and geo lookups."""
|
||||
|
||||
import os
|
||||
import tempfile
|
||||
import pytest
|
||||
from unittest.mock import patch, MagicMock
|
||||
|
||||
from app.log_parser import (
|
||||
parse_log_line,
|
||||
resolve_client_ip,
|
||||
_COUNTRY_CENTROIDS,
|
||||
lookup_geo,
|
||||
)
|
||||
|
||||
|
||||
class TestParseLogLine:
|
||||
"""Tests for parse_log_line()."""
|
||||
|
||||
def test_valid_json_line(self):
|
||||
line = '{"remote_addr": "192.168.1.1", "method": "GET"}'
|
||||
result = parse_log_line(line)
|
||||
assert result is not None
|
||||
assert result["remote_addr"] == "192.168.1.1"
|
||||
assert result["method"] == "GET"
|
||||
|
||||
def test_empty_line_returns_none(self):
|
||||
assert parse_log_line("") is None
|
||||
|
||||
def test_whitespace_only_returns_none(self):
|
||||
assert parse_log_line(" ") is None
|
||||
|
||||
def test_invalid_json_returns_none(self):
|
||||
assert parse_log_line("not json at all") is None
|
||||
|
||||
def test_partial_json_returns_none(self):
|
||||
assert parse_log_line('{"remote_addr": ') is None
|
||||
|
||||
def test_strips_whitespace(self):
|
||||
line = ' {"key": "value"} \n'
|
||||
result = parse_log_line(line)
|
||||
assert result is not None
|
||||
assert result["key"] == "value"
|
||||
|
||||
|
||||
class TestResolveClientIp:
|
||||
"""Tests for resolve_client_ip()."""
|
||||
|
||||
def test_remote_addr_without_xff(self):
|
||||
entry = {"remote_addr": "203.0.113.5"}
|
||||
assert resolve_client_ip(entry) == "203.0.113.5"
|
||||
|
||||
def test_xff_single_ip(self):
|
||||
entry = {
|
||||
"remote_addr": "10.0.0.1",
|
||||
"x_forwarded_for": "203.0.113.50",
|
||||
}
|
||||
assert resolve_client_ip(entry) == "203.0.113.50"
|
||||
|
||||
def test_xff_multiple_ips_returns_leftmost(self):
|
||||
entry = {
|
||||
"remote_addr": "10.0.0.1",
|
||||
"x_forwarded_for": "203.0.113.100, 10.1.2.3, 172.16.0.5",
|
||||
}
|
||||
assert resolve_client_ip(entry) == "203.0.113.100"
|
||||
|
||||
def test_xff_dash_falls_back_to_remote_addr(self):
|
||||
entry = {"remote_addr": "192.168.0.1", "x_forwarded_for": "-"}
|
||||
assert resolve_client_ip(entry) == "192.168.0.1"
|
||||
|
||||
def test_missing_fields_returns_empty(self):
|
||||
entry = {}
|
||||
assert resolve_client_ip(entry) == ""
|
||||
|
||||
def test_xff_with_whitespace(self):
|
||||
entry = {
|
||||
"remote_addr": "10.0.0.1",
|
||||
"x_forwarded_for": " 203.0.113.75 ",
|
||||
}
|
||||
assert resolve_client_ip(entry) == "203.0.113.75"
|
||||
|
||||
|
||||
class TestCountryCentroids:
|
||||
"""Tests for _COUNTRY_CENTROIDS data integrity."""
|
||||
|
||||
def test_centroids_are_float_tuples(self):
|
||||
for code, coords in _COUNTRY_CENTROIDS.items():
|
||||
assert isinstance(coords, tuple)
|
||||
assert len(coords) == 2
|
||||
assert isinstance(coords[0], float)
|
||||
assert isinstance(coords[1], float)
|
||||
|
||||
def test_known_countries_present(self):
|
||||
expected = {"US", "GB", "DE", "FR", "JP", "AU", "BR", "CA", "IN"}
|
||||
assert expected.issubset(_COUNTRY_CENTROIDS.keys())
|
||||
|
||||
def test_centroid_values_are_reasonable(self):
|
||||
"""Latitudes should be in [-90, 90], longitudes in [-180, 180]."""
|
||||
for code, (lat, lon) in _COUNTRY_CENTROIDS.items():
|
||||
assert -90 <= lat <= 90, f"{code} latitude {lat} out of range"
|
||||
assert -180 <= lon <= 180, f"{code} longitude {lon} out of range"
|
||||
|
||||
|
||||
class TestParseLogFile:
|
||||
"""Tests for parse_log_file() — reads from real temp files."""
|
||||
|
||||
def test_parses_multiple_lines(self, tmp_path):
|
||||
log_content = (
|
||||
'{"remote_addr": "1.2.3.4"}\n'
|
||||
'{"remote_addr": "5.6.7.8"}\n'
|
||||
'{"remote_addr": "1.2.3.4"}\n'
|
||||
)
|
||||
log_file = tmp_path / "test.log"
|
||||
log_file.write_text(log_content)
|
||||
|
||||
from app.log_parser import parse_log_file
|
||||
result = parse_log_file(str(log_file))
|
||||
# Should have 2 unique IPs (1.2.3.4 appears twice)
|
||||
assert "1.2.3.4" in result["unique_ips"]
|
||||
assert "5.6.7.8" in result["unique_ips"]
|
||||
assert len(result["unique_ips"]) == 2
|
||||
|
||||
def test_skips_invalid_lines(self, tmp_path):
|
||||
log_content = (
|
||||
'{"remote_addr": "1.2.3.4"}\n'
|
||||
'not valid json\n'
|
||||
'\n'
|
||||
'{"remote_addr": "5.6.7.8"}\n'
|
||||
)
|
||||
log_file = tmp_path / "test.log"
|
||||
log_file.write_text(log_content)
|
||||
|
||||
from app.log_parser import parse_log_file
|
||||
result = parse_log_file(str(log_file))
|
||||
assert len(result["unique_ips"]) == 2
|
||||
|
||||
def test_xff_takes_precedence(self, tmp_path):
|
||||
log_content = (
|
||||
'{"remote_addr": "10.0.0.1", "x_forwarded_for": "203.0.113.1"}\n'
|
||||
)
|
||||
log_file = tmp_path / "test.log"
|
||||
log_file.write_text(log_content)
|
||||
|
||||
from app.log_parser import parse_log_file
|
||||
result = parse_log_file(str(log_file))
|
||||
assert "203.0.113.1" in result["unique_ips"]
|
||||
assert "10.0.0.1" not in result["unique_ips"]
|
||||
|
||||
|
||||
class TestLookupGeo:
|
||||
"""Tests for lookup_geo() — verify IPs map to countries via our APIs."""
|
||||
|
||||
def _reset_geo_reader(self):
|
||||
"""Clear the cached reader so mocks take effect."""
|
||||
import app.log_parser as lp
|
||||
|
||||
lp._geo_reader = None
|
||||
|
||||
@patch("app.log_parser.settings")
|
||||
@patch("app.log_parser.os.path.exists", return_value=True)
|
||||
def test_ip_maps_to_country(self, mock_exists, mock_settings):
|
||||
"""Verify a known IP resolves to a country code via the GeoLite2 reader."""
|
||||
self._reset_geo_reader()
|
||||
mock_settings.GEOLITE2_DB_PATH = "/tmp/fake-geo.mmdb"
|
||||
|
||||
# Create a fake reader that returns US data
|
||||
fake_reader = MagicMock()
|
||||
fake_reader.get.return_value = {
|
||||
"country": {"iso_code": "US", "names": {"en": "United States"}},
|
||||
"city": {"names": {"en": "New York"}},
|
||||
"location": {"latitude": 40.7128, "longitude": -74.006},
|
||||
}
|
||||
|
||||
with patch("maxminddb.open_database", return_value=fake_reader):
|
||||
result = lookup_geo("8.8.8.8")
|
||||
|
||||
assert result["country_code"] == "US"
|
||||
assert result["country"] == "United States"
|
||||
assert result["city"] == "New York"
|
||||
assert result["latitude"] == 40.7128
|
||||
assert result["longitude"] == -74.006
|
||||
|
||||
@patch("app.log_parser.settings")
|
||||
@patch("app.log_parser.os.path.exists", return_value=True)
|
||||
def test_ip_without_coordinates_uses_centroid_fallback(self, mock_exists, mock_settings):
|
||||
"""When the free GeoLite2 DB lacks coordinates, centroid fallback kicks in."""
|
||||
self._reset_geo_reader()
|
||||
mock_settings.GEOLITE2_DB_PATH = "/tmp/fake-geo.mmdb"
|
||||
|
||||
# Simulate free GeoLite2 response — has country but no coordinates
|
||||
fake_reader = MagicMock()
|
||||
fake_reader.get.return_value = {
|
||||
"country": {"iso_code": "DE", "names": {"en": "Germany"}},
|
||||
"city": {"names": {"en": ""}},
|
||||
"location": {}, # No lat/lon
|
||||
}
|
||||
|
||||
with patch("maxminddb.open_database", return_value=fake_reader):
|
||||
result = lookup_geo("1.1.1.1")
|
||||
|
||||
assert result["country_code"] == "DE"
|
||||
assert result["country"] == "Germany"
|
||||
# Falls back to _COUNTRY_CENTROIDS["DE"]
|
||||
assert result["latitude"] == _COUNTRY_CENTROIDS["DE"][0]
|
||||
assert result["longitude"] == _COUNTRY_CENTROIDS["DE"][1]
|
||||
|
||||
@patch("app.log_parser.settings")
|
||||
def test_unknown_ip_returns_empty_dict(self, mock_settings):
|
||||
"""An IP not in the database returns an empty dict."""
|
||||
self._reset_geo_reader()
|
||||
mock_settings.GEOLITE2_DB_PATH = "/tmp/fake-geo.mmdb"
|
||||
|
||||
fake_reader = MagicMock()
|
||||
fake_reader.get.return_value = None
|
||||
|
||||
with patch("maxminddb.open_database", return_value=fake_reader):
|
||||
result = lookup_geo("255.255.255.255")
|
||||
|
||||
assert result == {}
|
||||
|
||||
@patch("app.log_parser.settings")
|
||||
def test_reader_error_returns_empty_dict(self, mock_settings):
|
||||
"""When the reader raises an exception, lookup_geo() returns safely."""
|
||||
self._reset_geo_reader()
|
||||
mock_settings.GEOLITE2_DB_PATH = "/tmp/fake-geo.mmdb"
|
||||
|
||||
with patch("maxminddb.open_database", side_effect=Exception("reader failed")):
|
||||
result = lookup_geo("1.2.3.4")
|
||||
|
||||
assert result == {}
|
||||
@@ -0,0 +1,111 @@
|
||||
"""Tests for app.models — SQLAlchemy model definitions and relationships."""
|
||||
|
||||
import pytest
|
||||
|
||||
from app.models import (
|
||||
Base,
|
||||
Show,
|
||||
ShowSchedule,
|
||||
Event,
|
||||
StationConfig,
|
||||
HistoryEntry,
|
||||
TeamMember,
|
||||
CommunityHighlight,
|
||||
Underwriter,
|
||||
VisitorStatDaily,
|
||||
VisitorStatGeo,
|
||||
)
|
||||
|
||||
|
||||
class TestModelDefinitions:
|
||||
"""Verify that models are properly defined."""
|
||||
|
||||
def test_show_table_name(self):
|
||||
assert Show.__tablename__ == "shows"
|
||||
|
||||
def test_show_schedule_table_name(self):
|
||||
assert ShowSchedule.__tablename__ == "show_schedules"
|
||||
|
||||
def test_event_table_name(self):
|
||||
assert Event.__tablename__ == "events"
|
||||
|
||||
def test_station_config_table_name(self):
|
||||
assert StationConfig.__tablename__ == "station_config"
|
||||
|
||||
def test_history_entry_table_name(self):
|
||||
assert HistoryEntry.__tablename__ == "history_entries"
|
||||
|
||||
def test_team_member_table_name(self):
|
||||
assert TeamMember.__tablename__ == "team_members"
|
||||
|
||||
def test_community_highlight_table_name(self):
|
||||
assert CommunityHighlight.__tablename__ == "community_highlights"
|
||||
|
||||
def test_underwriter_table_name(self):
|
||||
assert Underwriter.__tablename__ == "underwriters"
|
||||
|
||||
def test_visitor_stats_daily_table_name(self):
|
||||
assert VisitorStatDaily.__tablename__ == "visitor_stats_daily"
|
||||
|
||||
def test_visitor_stats_geo_table_name(self):
|
||||
assert VisitorStatGeo.__tablename__ == "visitor_stats_geo"
|
||||
|
||||
|
||||
class TestModelDefaults:
|
||||
"""Verify SQLAlchemy column-level defaults (set at INSERT time)."""
|
||||
|
||||
def _col_default_value(self, model_cls, col_name):
|
||||
"""Extract the Column default value from a model's metadata."""
|
||||
col = model_cls.__table__.columns[col_name]
|
||||
default = col.default
|
||||
if default is None:
|
||||
return None
|
||||
# DefaultClause.arg is a property (not callable) in SQLAlchemy 2.0
|
||||
if hasattr(default, "arg"):
|
||||
return default.arg
|
||||
return default
|
||||
|
||||
def test_show_defaults(self):
|
||||
assert self._col_default_value(Show, "display_order") == 0
|
||||
assert self._col_default_value(Show, "active") is True
|
||||
|
||||
def test_event_defaults(self):
|
||||
assert self._col_default_value(Event, "active") is True
|
||||
|
||||
def test_station_config_defaults(self):
|
||||
assert self._col_default_value(StationConfig, "callsign") == "KMTN"
|
||||
assert self._col_default_value(StationConfig, "name_primary") == "KMountain"
|
||||
assert self._col_default_value(StationConfig, "name_secondary") == "Flower Radio"
|
||||
assert self._col_default_value(StationConfig, "frequency") == "98.7 FM"
|
||||
assert self._col_default_value(StationConfig, "founded_year") == 1987
|
||||
assert self._col_default_value(StationConfig, "nonprofit_type") == "501(c)(3)"
|
||||
|
||||
def test_history_entry_defaults(self):
|
||||
assert self._col_default_value(HistoryEntry, "active") is True
|
||||
|
||||
def test_team_member_defaults(self):
|
||||
assert self._col_default_value(TeamMember, "active") is True
|
||||
|
||||
def test_underwriter_defaults(self):
|
||||
assert self._col_default_value(Underwriter, "display_order") == 0
|
||||
assert self._col_default_value(Underwriter, "active") is True
|
||||
|
||||
|
||||
class TestBaseModel:
|
||||
"""Verify Base declarative model setup."""
|
||||
|
||||
def test_base_has_metadata(self):
|
||||
assert hasattr(Base, "metadata")
|
||||
|
||||
def test_all_models_registered_in_metadata(self):
|
||||
tables = Base.metadata.tables
|
||||
assert "shows" in tables
|
||||
assert "show_schedules" in tables
|
||||
assert "events" in tables
|
||||
assert "station_config" in tables
|
||||
assert "history_entries" in tables
|
||||
assert "team_members" in tables
|
||||
assert "community_highlights" in tables
|
||||
assert "underwriters" in tables
|
||||
assert "visitor_stats_daily" in tables
|
||||
assert "visitor_stats_geo" in tables
|
||||
@@ -0,0 +1,177 @@
|
||||
"""Tests for Pydantic schemas — validation, defaults, and computed fields."""
|
||||
|
||||
from datetime import date
|
||||
import pytest
|
||||
|
||||
from app.schemas import (
|
||||
EventCreate,
|
||||
EventUpdate,
|
||||
EventResponse,
|
||||
StationConfigUpdate,
|
||||
ShowScheduleResponse,
|
||||
ShowCreate,
|
||||
ShowScheduleCreate,
|
||||
HistoryEntryCreate,
|
||||
TeamMemberCreate,
|
||||
UnderwriterCreate,
|
||||
DAY_NAMES,
|
||||
)
|
||||
|
||||
|
||||
class TestEventSchemas:
|
||||
"""Tests for Event Pydantic schemas."""
|
||||
|
||||
def test_event_create_validates_required_fields(self):
|
||||
event = EventCreate(
|
||||
date=date(2026, 8, 15),
|
||||
title="Summer Concert",
|
||||
description="Live music",
|
||||
location="Main Stage",
|
||||
icon="🎵",
|
||||
)
|
||||
assert event.title == "Summer Concert"
|
||||
assert event.rsvp_url is None
|
||||
|
||||
def test_event_create_with_rsvp(self):
|
||||
event = EventCreate(
|
||||
date=date(2026, 9, 1),
|
||||
title="Festival",
|
||||
description="Big event",
|
||||
location="Park",
|
||||
icon="🎉",
|
||||
rsvp_url="https://example.com/rsvp",
|
||||
)
|
||||
assert event.rsvp_url == "https://example.com/rsvp"
|
||||
|
||||
def test_event_update_all_optional(self):
|
||||
update = EventUpdate()
|
||||
assert update.title is None
|
||||
assert update.active is None
|
||||
|
||||
def test_event_update_partial(self):
|
||||
update = EventUpdate(active=False, title="Updated Title")
|
||||
assert update.active is False
|
||||
assert update.title == "Updated Title"
|
||||
assert update.date is None
|
||||
|
||||
|
||||
class TestStationConfigUpdate:
|
||||
"""Tests for StationConfigUpdate schema."""
|
||||
|
||||
def test_all_fields_optional(self):
|
||||
update = StationConfigUpdate()
|
||||
assert update.callsign is None
|
||||
assert update.color_primary is None
|
||||
|
||||
def test_partial_update(self):
|
||||
update = StationConfigUpdate(
|
||||
callsign="KXYZ",
|
||||
color_primary="#ff0000",
|
||||
)
|
||||
assert update.callsign == "KXYZ"
|
||||
assert update.color_primary == "#ff0000"
|
||||
assert update.name_primary is None
|
||||
|
||||
|
||||
class TestShowScheduleResponse:
|
||||
"""Tests for ShowScheduleResponse computed field."""
|
||||
|
||||
def test_day_label_mapping(self):
|
||||
for day_num, expected_label in DAY_NAMES.items():
|
||||
schedule = ShowScheduleResponse(
|
||||
id=1,
|
||||
show_id=1,
|
||||
day_of_week=day_num,
|
||||
time="10:00",
|
||||
)
|
||||
assert schedule.day_label == expected_label
|
||||
|
||||
def test_unknown_day_returns_unknown(self):
|
||||
schedule = ShowScheduleResponse(
|
||||
id=1,
|
||||
show_id=1,
|
||||
day_of_week=99,
|
||||
time="10:00",
|
||||
)
|
||||
assert schedule.day_label == "Unknown"
|
||||
|
||||
|
||||
class TestShowCreate:
|
||||
"""Tests for ShowCreate schema."""
|
||||
|
||||
def test_show_create_with_schedules(self):
|
||||
show = ShowCreate(
|
||||
title="Morning Show",
|
||||
description="Morning music",
|
||||
host="DJ Alex",
|
||||
genre="Pop",
|
||||
schedules=[
|
||||
ShowScheduleCreate(day_of_week=1, time="08:00"),
|
||||
ShowScheduleCreate(day_of_week=3, time="08:00"),
|
||||
],
|
||||
)
|
||||
assert len(show.schedules) == 2
|
||||
assert show.schedules[0].day_of_week == 1
|
||||
assert show.display_order == 0
|
||||
|
||||
def test_show_create_defaults(self):
|
||||
show = ShowCreate(
|
||||
title="Show",
|
||||
description="Desc",
|
||||
host="Host",
|
||||
genre="Genre",
|
||||
schedules=[ShowScheduleCreate(day_of_week=1, time="09:00")],
|
||||
)
|
||||
assert show.show_art_url is None
|
||||
assert show.hero_image_url is None
|
||||
assert show.display_order == 0
|
||||
|
||||
|
||||
class TestHistoryEntryCreate:
|
||||
"""Tests for HistoryEntryCreate schema."""
|
||||
|
||||
def test_valid_creation(self):
|
||||
entry = HistoryEntryCreate(
|
||||
year=1987,
|
||||
title="Station Founded",
|
||||
body="The station began broadcasting.",
|
||||
display_order=1,
|
||||
)
|
||||
assert entry.year == 1987
|
||||
assert entry.title == "Station Founded"
|
||||
|
||||
|
||||
class TestTeamMemberCreate:
|
||||
"""Tests for TeamMemberCreate schema."""
|
||||
|
||||
def test_required_fields(self):
|
||||
member = TeamMemberCreate(
|
||||
name="Jane Doe",
|
||||
title="Engineer",
|
||||
display_order=1,
|
||||
)
|
||||
assert member.name == "Jane Doe"
|
||||
assert member.bio is None
|
||||
assert member.photo_url is None
|
||||
|
||||
|
||||
class TestUnderwriterCreate:
|
||||
"""Tests for UnderwriterCreate schema."""
|
||||
|
||||
def test_defaults_display_order_to_zero(self):
|
||||
uw = UnderwriterCreate(
|
||||
name="Sponsor Co",
|
||||
description="A great sponsor",
|
||||
)
|
||||
assert uw.display_order == 0
|
||||
assert uw.website_url is None
|
||||
assert uw.logo_url is None
|
||||
|
||||
def test_custom_display_order(self):
|
||||
uw = UnderwriterCreate(
|
||||
name="Premium Sponsor",
|
||||
description="Best sponsor",
|
||||
display_order=5,
|
||||
website_url="https://sponsor.example.com",
|
||||
)
|
||||
assert uw.display_order == 5
|
||||
@@ -0,0 +1,22 @@
|
||||
"""Tests for theme_export — theme.json generation from station config."""
|
||||
|
||||
import os
|
||||
import json
|
||||
import pytest
|
||||
from unittest.mock import patch, AsyncMock, MagicMock
|
||||
|
||||
from app.config import Settings
|
||||
|
||||
|
||||
class TestThemeExport:
|
||||
"""Tests for theme.json export logic."""
|
||||
|
||||
def test_settings_theme_json_path_default(self):
|
||||
with patch.dict("os.environ", {"KMTN_THEME_JSON_PATH": "/app/theme.json"}, clear=False):
|
||||
settings = Settings()
|
||||
assert settings.THEME_JSON_PATH == "/app/theme.json"
|
||||
|
||||
def test_settings_website_url_default(self):
|
||||
with patch.dict("os.environ", {"KMTN_WEBSITE_URL": "https://kmountainflower.org"}, clear=False):
|
||||
settings = Settings()
|
||||
assert settings.WEBSITE_URL == "https://kmountainflower.org"
|
||||
@@ -0,0 +1,98 @@
|
||||
# Force Intel (amd64) architecture — required because Android SDK
|
||||
# command-line tools are only published for x86_64 (Linux/macOS).
|
||||
# On Apple Silicon hosts, Docker runs this image via Rosetta emulation.
|
||||
FROM --platform=linux/amd64 docker.io/library/debian:bookworm-slim
|
||||
|
||||
# System dependencies + SSH server for intra-network access
|
||||
RUN apt-get update && apt-get install -y \
|
||||
git curl unzip wget python3 python3-venv python3-pip \
|
||||
openjdk-17-jdk-headless \
|
||||
openssh-server \
|
||||
xz-utils \
|
||||
libcairo2 \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
# SSH server configuration
|
||||
RUN mkdir -p /run/sshd && \
|
||||
ssh-keygen -A && \
|
||||
chmod 644 /etc/ssh/ssh_host_*_key && \
|
||||
sed -i 's/#PermitRootLogin.*/PermitRootLogin no/' /etc/ssh/sshd_config && \
|
||||
echo "UsePAM no" >> /etc/ssh/sshd_config && \
|
||||
echo "PermitUserEnvironment yes" >> /etc/ssh/sshd_config
|
||||
|
||||
# Set Java home — use /usr/lib/jvm/java-17-openjdk which is a stable
|
||||
# symlink provided by Debian's jdk package, regardless of architecture.
|
||||
ENV JAVA_HOME=/usr/lib/jvm/java-17-openjdk-amd64
|
||||
ENV PATH="$JAVA_HOME/bin:$PATH"
|
||||
|
||||
# Install Flutter 3.41.8
|
||||
ARG FLUTTER_VERSION=3.41.8
|
||||
ARG FLUTTER_SDK_URL=https://storage.googleapis.com/flutter_infra_release/releases/stable/linux/flutter_linux_${FLUTTER_VERSION}-stable.tar.xz
|
||||
RUN wget -qO- $FLUTTER_SDK_URL | tar -xJ -C /opt \
|
||||
&& ln -s /opt/flutter/bin/flutter /usr/local/bin/flutter
|
||||
|
||||
# Android SDK setup
|
||||
ENV ANDROID_HOME=/opt/android-sdk
|
||||
RUN mkdir -p $ANDROID_HOME/cmdline-tools
|
||||
# Download and install Android command-line tools
|
||||
RUN wget -qO /tmp/cmdline-tools.zip \
|
||||
https://dl.google.com/android/repository/commandlinetools-linux-11076708_latest.zip \
|
||||
&& unzip -q /tmp/cmdline-tools.zip -d /tmp/cmdline-tools-extract \
|
||||
&& mv /tmp/cmdline-tools-extract/cmdline-tools $ANDROID_HOME/cmdline-tools/latest \
|
||||
&& rm -rf /tmp/cmdline-tools*
|
||||
|
||||
# Accept licenses and install required SDK packages
|
||||
ENV CMDLINE_TOOLS=$ANDROID_HOME/cmdline-tools/latest/bin
|
||||
RUN yes | $CMDLINE_TOOLS/sdkmanager --licenses >/dev/null 2>&1 || true
|
||||
RUN $CMDLINE_TOOLS/sdkmanager "platform-tools" "platforms;android-35" "build-tools;35.0.0"
|
||||
|
||||
# Create vscode user (UID 1000, matching devcontainer convention)
|
||||
RUN groupadd -g 1000 vscode && \
|
||||
useradd -m -u 1000 -g 1000 -s /bin/bash vscode && \
|
||||
chown -R vscode:vscode /opt/flutter /opt/android-sdk && \
|
||||
mkdir -p /home/vscode && chown vscode:vscode /home/vscode
|
||||
|
||||
# Give vscode a simple password: '123'
|
||||
RUN echo "vscode:123" | chpasswd
|
||||
|
||||
# Set up Flutter for the vscode user
|
||||
USER vscode
|
||||
WORKDIR /home/vscode
|
||||
ENV PATH="/opt/flutter/bin:$ANDROID_HOME/cmdline-tools/latest/bin:$ANDROID_HOME/platform-tools:$JAVA_HOME/bin:$PATH"
|
||||
|
||||
# Pre-cache Flutter artifacts
|
||||
RUN flutter precache --android
|
||||
|
||||
# Set Android HOME for vscode user
|
||||
ENV ANDROID_HOME=/opt/android-sdk
|
||||
|
||||
# SSH non-interactive shells don't inherit Docker ENV at all (not even
|
||||
# BASH_ENV). Use SSH's own ~/.ssh/environment file with
|
||||
# PermitUserEnvironment to guarantee variables are set on every connection.
|
||||
RUN mkdir -p /home/vscode/.ssh && \
|
||||
echo 'JAVA_HOME=/usr/lib/jvm/java-17-openjdk-amd64' > /home/vscode/.ssh/environment && \
|
||||
echo 'ANDROID_HOME=/opt/android-sdk' >> /home/vscode/.ssh/environment && \
|
||||
echo 'PATH=/opt/flutter/bin:/opt/android-sdk/cmdline-tools/latest/bin:/opt/android-sdk/platform-tools:/usr/lib/jvm/java-17-openjdk-amd64/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin' >> /home/vscode/.ssh/environment && \
|
||||
chown -R vscode:vscode /home/vscode/.ssh && \
|
||||
chmod 700 /home/vscode/.ssh && \
|
||||
chmod 644 /home/vscode/.ssh/environment
|
||||
|
||||
# Set up Gradle home so the wrapper works correctly in SSH sessions.
|
||||
# org.gradle.daemon=false is required because SSH sessions are ephemeral
|
||||
# — a daemon started in one session can't be reused in the next.
|
||||
RUN mkdir -p /home/vscode/.gradle && \
|
||||
echo 'org.gradle.daemon=false' > /home/vscode/.gradle/gradle.properties && \
|
||||
echo 'org.gradle.jvmargs=-Xmx4g -XX:MaxMetaspaceSize=512m -XX:+UseSerialGC' >> /home/vscode/.gradle/gradle.properties && \
|
||||
echo 'org.gradle.workers.max=1' >> /home/vscode/.gradle/gradle.properties && \
|
||||
echo 'org.gradle.parallel=false' >> /home/vscode/.gradle/gradle.properties && \
|
||||
echo 'kotlin.daemon.jvmargs=-Xmx512m' >> /home/vscode/.gradle/gradle.properties && \
|
||||
echo 'kotlin.incremental=false' >> /home/vscode/.gradle/gradle.properties && \
|
||||
echo 'kotlin.compiler.execution.strategy=in-process' >> /home/vscode/.gradle/gradle.properties && \
|
||||
echo 'android.enableParallelPlugin=false' >> /home/vscode/.gradle/gradle.properties && \
|
||||
chown -R vscode:vscode /home/vscode/.gradle
|
||||
|
||||
# Expose SSH for intra-network access from the api container
|
||||
EXPOSE 22
|
||||
|
||||
# Start SSH daemon and keep container alive
|
||||
CMD ["/usr/sbin/sshd", "-D"]
|
||||
@@ -0,0 +1,3 @@
|
||||
PROJECT_NAME=kryz
|
||||
FRONTEND_PORT=4201
|
||||
KMTN_CORS_ORIGINS='["http://localhost:4201", "http://localhost"]'
|
||||
@@ -0,0 +1,3 @@
|
||||
PROJECT_NAME=staylit
|
||||
FRONTEND_PORT=4200
|
||||
KMTN_CORS_ORIGINS='["http://localhost:4200", "http://localhost"]'
|
||||
Binary file not shown.
Executable
+37
@@ -0,0 +1,37 @@
|
||||
#!/bin/bash
|
||||
|
||||
# Ensure we are in the project root
|
||||
cd "$(dirname "$0")"
|
||||
|
||||
if [ ! -d "customers" ]; then
|
||||
echo "Error: 'customers/' directory not found."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "Starting mass deployment for all customers..."
|
||||
echo "--------------------------------------------------"
|
||||
|
||||
# Iterate through .env.* files in the customers/ directory
|
||||
for env_file in customers/.env.*; do
|
||||
# Check if it's a regular file and not just the pattern itself (if no matches found)
|
||||
[ -e "$env_file" ] || continue
|
||||
|
||||
# Extract customer name from the filename (e.g., customers/.env.kmtn -> kmtn)
|
||||
# We remove the 'customers/.env.' prefix to get the name
|
||||
customer_name=$(basename "$env_file" | sed 's/\.env\.//')
|
||||
|
||||
echo "[$(date +'%H:%M:%S')] Deploying: $customer_name"
|
||||
echo "Using configuration: $env_file"
|
||||
|
||||
# Run docker compose. We use -p for the project name to isolate them correctly.
|
||||
docker compose --env-file "$env_file" -p "$customer_name" up -d --build
|
||||
|
||||
if [ $? -eq 0 ]; then
|
||||
echo "[$(date +'%H:%M:%S')] ✅ SUCCESS: $customer_name"
|
||||
else
|
||||
echo "[$(date +'%H:%M:%S')] ❌ FAILED: $customer_name"
|
||||
fi
|
||||
echo "--------------------------------------------------"
|
||||
done
|
||||
|
||||
echo "Mass deployment process completed."
|
||||
@@ -1,21 +0,0 @@
|
||||
# Production override — use with: docker compose -f docker-compose.yml -f docker-compose.prod.yml up
|
||||
# Assumes: managed PostgreSQL, external reverse proxy / load balancer handling TLS and port 80/443.
|
||||
|
||||
services:
|
||||
# No db service — managed PostgreSQL is assumed.
|
||||
# Set KMTN_DATABASE_URL in your environment or a .env file.
|
||||
|
||||
api:
|
||||
environment:
|
||||
KMTN_ENV: production
|
||||
# Override with your actual production domain(s)
|
||||
KMTN_CORS_ORIGINS: '["https://yourdomain.com"]'
|
||||
# No port mapping — traffic arrives via reverse proxy / load balancer
|
||||
|
||||
frontend:
|
||||
environment:
|
||||
# Internal API target for nginx proxy
|
||||
API_UPSTREAM: https://api.yourdomain.com
|
||||
# Browser-facing API URL. Empty = proxy mode. Set to direct URL if ingress breaks proxy chain.
|
||||
API_PUBLIC_URL: ""
|
||||
# No port mapping — traffic arrives via reverse proxy / load balancer
|
||||
@@ -1,51 +0,0 @@
|
||||
services:
|
||||
db:
|
||||
image: docker.io/library/postgres:16-alpine
|
||||
restart: unless-stopped
|
||||
# NOTE: Change POSTGRES_PASSWORD in production. Consider using Docker secrets or
|
||||
# a managed PostgreSQL service instead of local credentials.
|
||||
environment:
|
||||
POSTGRES_DB: kmountain
|
||||
POSTGRES_USER: postgres
|
||||
POSTGRES_PASSWORD: postgres
|
||||
volumes:
|
||||
- pgdata:/var/lib/postgresql/data
|
||||
- pgsocket:/var/run/postgresql
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "pg_isready -U postgres"]
|
||||
interval: 5s
|
||||
timeout: 5s
|
||||
retries: 5
|
||||
|
||||
# NOTE: Update KMTN_CORS_ORIGINS in production to your actual domain(s).
|
||||
# Set KMTN_DATABASE_URL to your managed PostgreSQL connection if not using the local db service.
|
||||
api:
|
||||
image: truenas.local:35000/kmtnflower-api:latest
|
||||
restart: unless-stopped
|
||||
environment:
|
||||
KMTN_DATABASE_URL: postgresql+asyncpg://postgres:postgres@/kmountain?host=/var/run/postgresql
|
||||
KMTN_CORS_ORIGINS: '["http://truenas.local:4200", "http://truenas.local"]'
|
||||
KMTN_ADMIN_USERNAME: admin
|
||||
KMTN_ADMIN_PASSWORD: 123
|
||||
volumes:
|
||||
- pgsocket:/var/run/postgresql
|
||||
ports:
|
||||
- "8000:8000"
|
||||
depends_on:
|
||||
db:
|
||||
condition: service_healthy
|
||||
|
||||
frontend:
|
||||
image: truenas.local:35000/kmtnflower:latest
|
||||
restart: unless-stopped
|
||||
environment:
|
||||
API_UPSTREAM: http://api:8000
|
||||
API_PUBLIC_URL: "http://truenas.local:8000"
|
||||
ports:
|
||||
- "4200:80"
|
||||
depends_on:
|
||||
- api
|
||||
|
||||
volumes:
|
||||
pgdata:
|
||||
pgsocket:
|
||||
+42
-6
@@ -10,8 +10,9 @@ services:
|
||||
POSTGRES_PASSWORD: postgres
|
||||
volumes:
|
||||
- pgdata:/var/lib/postgresql/data
|
||||
- pgsocket:/var/run/postgresql
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "pg_isready -U postgres"]
|
||||
test: [ "CMD-SHELL", "pg_isready -U postgres" ]
|
||||
interval: 5s
|
||||
timeout: 5s
|
||||
retries: 5
|
||||
@@ -24,10 +25,15 @@ services:
|
||||
dockerfile: Dockerfile
|
||||
restart: unless-stopped
|
||||
environment:
|
||||
KMTN_DATABASE_URL: postgresql+asyncpg://postgres:postgres@db:5432/kmountain
|
||||
KMTN_CORS_ORIGINS: '["http://localhost:4200", "http://localhost"]'
|
||||
ports:
|
||||
- "8000:8000"
|
||||
KMTN_DATABASE_URL: >-
|
||||
postgresql+asyncpg://postgres:postgres@/kmountain?host=/var/run/postgresql
|
||||
KMTN_CORS_ORIGINS: '${KMTN_CORS_ORIGINS}'
|
||||
KMTN_ADMIN_USERNAME: "admin"
|
||||
KMTN_ADMIN_PASSWORD: "123"
|
||||
KMTN_STATIC_UPSTREAM_URL: http://frontend:80
|
||||
volumes:
|
||||
- pgsocket:/var/run/postgresql
|
||||
- logs:/logs
|
||||
depends_on:
|
||||
db:
|
||||
condition: service_healthy
|
||||
@@ -41,9 +47,39 @@ services:
|
||||
API_UPSTREAM: http://api:8000
|
||||
API_PUBLIC_URL: ""
|
||||
ports:
|
||||
- "4200:80"
|
||||
- "${FRONTEND_PORT}:80"
|
||||
volumes:
|
||||
- logs:/logs
|
||||
depends_on:
|
||||
- api
|
||||
|
||||
# Flutter build container — runs alongside the api/frontend services.
|
||||
# The api container SSHes directly into this container over the Docker network.
|
||||
# Forced to linux/amd64 because Android SDK CLI tools are only published
|
||||
# for x86_64. On Apple Silicon hosts, Docker runs via Rosetta emulation.
|
||||
build:
|
||||
build:
|
||||
context: ./build
|
||||
dockerfile: Dockerfile
|
||||
platform: linux/amd64
|
||||
restart: unless-stopped
|
||||
expose:
|
||||
- "22"
|
||||
volumes:
|
||||
- flutter-cache:/home/vscode/.pub-cache
|
||||
- android-licenses:/opt/android-sdk/licenses
|
||||
deploy:
|
||||
resources:
|
||||
limits:
|
||||
cpus: '4.0'
|
||||
memory: 16g
|
||||
reservations:
|
||||
cpus: '2.0'
|
||||
memory: 8g
|
||||
|
||||
volumes:
|
||||
pgdata:
|
||||
pgsocket:
|
||||
logs:
|
||||
flutter-cache:
|
||||
android-licenses:
|
||||
|
||||
@@ -2,7 +2,10 @@
|
||||
# Render config templates with environment variables, then start nginx.
|
||||
set -e
|
||||
|
||||
envsubst '${API_UPSTREAM}' < /etc/nginx/conf.d/nginx.conf.template > /etc/nginx/conf.d/default.conf
|
||||
# Ensure nginx can write to the shared logs directory
|
||||
mkdir -p /logs && chown nginx:nginx /logs
|
||||
|
||||
sed "s|__API_UPSTREAM__|${API_UPSTREAM}|g" /etc/nginx/conf.d/nginx.conf.template > /etc/nginx/conf.d/default.conf
|
||||
envsubst '${API_PUBLIC_URL}' < /usr/share/nginx/html/config.json.template > /usr/share/nginx/html/config.json
|
||||
|
||||
exec nginx -g 'daemon off;'
|
||||
|
||||
+25
-2
@@ -1,3 +1,18 @@
|
||||
# Map ISO timestamp to date-only string for daily log rotation
|
||||
map $time_iso8601 $log_date {
|
||||
"~^(?<date>\d{4}-\d{2}-\d{2})" $date;
|
||||
default "unknown";
|
||||
}
|
||||
|
||||
log_format kmtn_json escape=json
|
||||
'{'
|
||||
'"time":"$time_iso8601",'
|
||||
'"remote_addr":"$remote_addr",'
|
||||
'"x_forwarded_for":"$http_x_forwarded_for"'
|
||||
'}';
|
||||
|
||||
access_log /logs/access-$log_date.log kmtn_json;
|
||||
|
||||
server {
|
||||
listen 80;
|
||||
server_name _;
|
||||
@@ -11,13 +26,21 @@ server {
|
||||
|
||||
# Proxy API calls to the backend service
|
||||
location /api/ {
|
||||
proxy_pass ${API_UPSTREAM};
|
||||
proxy_pass __API_UPSTREAM__;
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header Origin $http_origin;
|
||||
proxy_set_header Referer $http_referer;
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
proxy_set_header X-Forwarded-Proto $scheme;
|
||||
proxy_set_header X-Forwarded-Proto $http_x_forwarded_proto;
|
||||
proxy_set_header X-Forwarded-Host $host;
|
||||
|
||||
# Disable caching through the double-proxy chain (NPM → frontend nginx → API)
|
||||
# Without this, browsers cache responses keyed by the HTTPS origin, then send
|
||||
# conditional requests (If-None-Match) that get lost or mismatched in the proxy chain.
|
||||
proxy_cache off;
|
||||
add_header Cache-Control "no-store, no-cache, must-revalidate" always;
|
||||
add_header Pragma "no-cache" always;
|
||||
add_header Vary "*" always;
|
||||
}
|
||||
}
|
||||
|
||||
Generated
+1138
-3
File diff suppressed because it is too large
Load Diff
+15
-2
@@ -6,7 +6,10 @@
|
||||
"start": "ng serve",
|
||||
"build": "ng build",
|
||||
"watch": "ng build --watch --configuration development",
|
||||
"test": "ng test"
|
||||
"test": "ng test",
|
||||
"test:unit": "vitest run",
|
||||
"test:unit:watch": "vitest",
|
||||
"test:unit:coverage": "vitest run --coverage"
|
||||
},
|
||||
"private": true,
|
||||
"packageManager": "npm@11.11.0",
|
||||
@@ -17,6 +20,12 @@
|
||||
"@angular/forms": "^21.2.0",
|
||||
"@angular/platform-browser": "^21.2.0",
|
||||
"@angular/router": "^21.2.0",
|
||||
"@types/leaflet": "^1.9.17",
|
||||
"@angular/cdk": "^21.2.0",
|
||||
"chart.js": "^4.4.0",
|
||||
"leaflet": "^1.9.4",
|
||||
"ng2-charts": "^10.0.0",
|
||||
"ngx-leaflet": "^0.0.16",
|
||||
"rxjs": "~7.8.0",
|
||||
"tslib": "^2.3.0",
|
||||
"zone.js": "^0.16.2"
|
||||
@@ -25,7 +34,11 @@
|
||||
"@angular/build": "^21.2.14",
|
||||
"@angular/cli": "^21.2.14",
|
||||
"@angular/compiler-cli": "^21.2.0",
|
||||
"@vitest/coverage-v8": "^4.0.8",
|
||||
"jsdom": "^26.1.0",
|
||||
"prettier": "^3.8.1",
|
||||
"typescript": "~5.9.2"
|
||||
"typescript": "~5.9.2",
|
||||
"vitest": "^4.0.8",
|
||||
"@types/jsdom": "^21.1.7"
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,76 +0,0 @@
|
||||
# Retire Donation Page, Add Configurable External Donation URL
|
||||
|
||||
## Context
|
||||
|
||||
The donation page (`/donate`) is being retired — it was a tier-based landing page with calls to action. All donation links will instead point to an external URL (e.g., a nonprofit payment processor). Admins configure this URL in the existing Station Config screen.
|
||||
|
||||
## Approach
|
||||
|
||||
1. Add `donation_url` field to the existing `StationConfig` singleton (backend model + schema, frontend interface + admin form)
|
||||
2. Replace all donation links across the site with external `<a href>` links driven by `config().donation_url`, hidden when empty
|
||||
3. Delete the donation page, tier service, tier admin UI, and all backend tier infrastructure
|
||||
|
||||
## Step-by-Step
|
||||
|
||||
### Phase 1 — Add `donation_url` to StationConfig (backend)
|
||||
|
||||
- **`backend/app/models.py`** — Add `donation_url = Column(String(500), nullable=False, default="")` to `StationConfig` class
|
||||
- **`backend/app/schemas.py`** — Add `donation_url: str` to `StationConfigResponse`; add `donation_url: Optional[str] = None` to `StationConfigUpdate`
|
||||
- **`backend/seed.py`** — Add `"donation_url": ""` to `STATION_CONFIG` dict
|
||||
|
||||
### Phase 2 — Add `donation_url` to StationConfig (frontend)
|
||||
|
||||
- **`src/app/interfaces/station-config.ts`** — Add `donation_url: string`
|
||||
- **`src/app/services/station-config.service.ts`** — Add `donation_url: ''` to `DEFAULT_CONFIG`
|
||||
- **`src/app/admin/admin-station-form.component.ts`** — Add `donation_url` property, wire it in `populate()` and `onSubmit()` payload
|
||||
- **`src/app/admin/admin-station-form.component.html`** — Add a "Donation" form section with a URL input field
|
||||
- **`src/app/admin/admin.component.html`** — Show `donation_url` in the station config summary view
|
||||
|
||||
### Phase 3 — Replace donation links with external links
|
||||
|
||||
All links switch from `routerLink="/donate"` to `[href]="config().donation_url"` with `target="_blank" rel="noopener noreferrer"`, wrapped in `@if (config().donation_url)` so they hide when empty.
|
||||
|
||||
- **`src/app/navbar/navbar.component.html`** — "Donate Now" button (line 37)
|
||||
- **`src/app/hero/hero.component.html`** — "Support the Station" button (line 28)
|
||||
- **`src/app/about/about.component.html`** — "Become a Member" CTA section (lines 76-88), wrap entire block in `@if`
|
||||
- **`src/app/footer/footer.component.html`** — "Donate" and "Become a Member" links (lines 26, 33)
|
||||
|
||||
### Phase 4 — Delete donation page
|
||||
|
||||
- Remove `/donate` route from **`src/app/app.routes.ts`** (lines 22-25)
|
||||
- Delete **`src/app/donate/`** directory (3 files: `.ts`, `.html`, `.scss`)
|
||||
|
||||
### Phase 5 — Remove tier infrastructure (frontend)
|
||||
|
||||
- Delete **`src/app/interfaces/tier.ts`**
|
||||
- Delete **`src/app/services/tier.service.ts`**
|
||||
- Delete **`src/app/admin/admin-tier-form.component.*`** (3 files)
|
||||
- **`src/app/admin/admin.component.ts`** — Remove `Tier`/`TierService`/`AdminTierFormComponent` imports, `TabKey` entry, injected service, `tiers$` BehaviorSubject, `editingTier`, `showTierForm`, `reloadTiers()`, and all tier CRUD methods
|
||||
- **`src/app/admin/admin.component.html`** — Remove Tiers tab button, Tiers tab content panel, tier form modal
|
||||
|
||||
### Phase 6 — Remove tier infrastructure (backend)
|
||||
|
||||
- Delete **`backend/app/api/tiers.py`**
|
||||
- **`backend/app/models.py`** — Delete `DonationTier` class
|
||||
- **`backend/app/schemas.py`** — Delete `TierCreate`, `TierUpdate`, `TierResponse`
|
||||
- **`backend/app/main.py`** — Remove `tiers` import and router registration
|
||||
- **`backend/app/api/admin.py`** — Remove `DonationTier` from reset truncation
|
||||
- **`backend/seed.py`** — Remove `DonationTier` import, `TIERS` list, `_upsert_tier()` helper, tier seeding, tier truncation
|
||||
|
||||
### Phase 7 — Contact form cleanup
|
||||
|
||||
- **`src/app/contact/contact.component.html`** — Remove `<option value="donation">Donation Question</option>`
|
||||
|
||||
## Execution Order
|
||||
|
||||
1 → 2 → 3 → 4 → 5 → 6 → 7
|
||||
|
||||
## Verification
|
||||
|
||||
1. Grep for remaining references to `Tier`, `tier`, `DonationTier`, `donate`, `/donate` — none should remain in live code
|
||||
2. `ng build` — no import errors, no unresolved references
|
||||
3. Backend starts without import errors
|
||||
4. Admin station form shows the new "External Donation URL" field
|
||||
5. Setting a URL → all 4 link locations render external links with `target="_blank"`
|
||||
6. Clearing the URL → all 4 link locations hide
|
||||
7. Navigate to `/donate` → 404 or wildcard redirect to home
|
||||
+122
@@ -0,0 +1,122 @@
|
||||
# SEO Plan for KMountain Flower Radio
|
||||
|
||||
## Objective
|
||||
Improve crawlability, indexability, and ranking quality for a JavaScript-rendered Angular site where core content is loaded after app bootstrap.
|
||||
|
||||
## Current Baseline
|
||||
- The app is client-rendered Angular (no SSR/prerender in current build pipeline).
|
||||
- Nginx serves SPA fallback routes to index.html.
|
||||
- Initial HTML is a minimal shell and most meaningful content is populated after JavaScript executes and API calls complete.
|
||||
- Route-specific SEO metadata is not currently implemented (title updates are global/dynamic only).
|
||||
- robots.txt and sitemap.xml are not currently present in public assets.
|
||||
|
||||
## Target Outcomes
|
||||
- Reliable indexing of all public routes.
|
||||
- Better SERP snippets and relevance for branded and non-branded queries.
|
||||
- Less dependence on crawler JavaScript/render budget.
|
||||
|
||||
## Priority 1 (High Impact, Low Risk)
|
||||
|
||||
### 1) Add robots.txt
|
||||
- Allow crawling of public pages.
|
||||
- Block sensitive/admin/auth endpoints if needed.
|
||||
- Add a sitemap reference.
|
||||
|
||||
### 2) Add sitemap.xml
|
||||
- Include canonical URLs for:
|
||||
- /
|
||||
- /about
|
||||
- /schedule
|
||||
- /events
|
||||
- /contact
|
||||
- Keep lastmod values updated during deploys.
|
||||
- Submit sitemap in Google Search Console and Bing Webmaster Tools.
|
||||
|
||||
### 3) Implement route-level metadata
|
||||
- Set unique title and meta description per public route.
|
||||
- Add canonical tags per route.
|
||||
- Add Open Graph and Twitter card defaults plus per-route overrides.
|
||||
- Mark login/admin routes as noindex.
|
||||
|
||||
### 4) Strengthen crawl path discovery
|
||||
- Keep static crawlable links to all public pages in global navigation/footer.
|
||||
- Ensure key public routes are linked from multiple relevant pages.
|
||||
|
||||
### 5) Add structured data
|
||||
- Add Organization schema site-wide.
|
||||
- Add Event schema for upcoming events content.
|
||||
- Include station identity fields consistently (name, URL, logo, contact where appropriate).
|
||||
|
||||
## Priority 2 (Bigger Lift, Larger Upside)
|
||||
|
||||
### 6) Introduce prerender or SSR for public routes
|
||||
- Preferred first step: prerender static public routes.
|
||||
- Alternative: full SSR if dynamic metadata/content requires runtime generation.
|
||||
- Goal: deliver meaningful HTML to crawlers before JS execution.
|
||||
|
||||
### 7) Ensure metadata appears in initial HTML
|
||||
- Validate title/description/canonical in rendered source for each route.
|
||||
- Ensure each route has a clear H1 and meaningful introductory text in server-delivered HTML.
|
||||
|
||||
### 8) Keep baseline content available without API dependence
|
||||
- Include useful fallback content for core sections (home/about/schedule/events intros).
|
||||
- Treat API-loaded content as enhancement rather than sole source of indexable copy.
|
||||
|
||||
## Priority 3 (Quality + Ranking Support)
|
||||
|
||||
### 9) Improve Core Web Vitals
|
||||
- Optimize home LCP image and dimensions.
|
||||
- Defer non-critical work below the fold.
|
||||
- Reduce layout shifts around async content blocks.
|
||||
|
||||
### 10) Expand internal linking depth
|
||||
- Add contextual links between About, Schedule, Events, and Contact pages.
|
||||
- Use descriptive anchor text aligned with user intent.
|
||||
|
||||
### 11) Media SEO pass
|
||||
- Keep decorative images with empty alt attributes.
|
||||
- Add descriptive alt text for content-bearing images.
|
||||
|
||||
## Implementation Notes for This Repo
|
||||
- Place robots.txt and sitemap.xml in public assets so they are copied to build output.
|
||||
- Keep SPA fallback behavior in Nginx; SEO assets still resolve directly.
|
||||
- Use a centralized route-to-metadata mapping to avoid duplicate/incorrect canonical logic.
|
||||
|
||||
## Measurement Plan
|
||||
|
||||
### Week 1 Baseline
|
||||
- Capture Search Console index coverage and excluded reasons.
|
||||
- Record impressions/clicks for branded and local intent queries.
|
||||
- Run Lighthouse on all public routes.
|
||||
|
||||
### Weeks 2-3 (after Priority 1)
|
||||
- Validate robots.txt and sitemap accessibility.
|
||||
- Validate route metadata and canonical correctness.
|
||||
- Track indexed-page growth and snippet quality.
|
||||
|
||||
### Weeks 4-8 (after Priority 2)
|
||||
- Compare crawl stats and indexing completeness before/after prerender or SSR.
|
||||
- Track ranking movement for schedule/events/community-intent terms.
|
||||
- Validate rich result eligibility for structured data.
|
||||
|
||||
## Acceptance Criteria
|
||||
- robots.txt and sitemap.xml are live and valid.
|
||||
- All public routes have unique title, description, and canonical.
|
||||
- Public routes are indexable and admin/login are noindex.
|
||||
- Search Console shows healthy discovery/indexing across primary routes.
|
||||
- Public pages render useful content for crawlers without relying solely on JS.
|
||||
|
||||
## Risks and Mitigations
|
||||
- Risk: JS render delays cause partial indexing.
|
||||
- Mitigation: prerender/SSR and stronger baseline HTML.
|
||||
- Risk: API issues reduce crawler-visible content.
|
||||
- Mitigation: robust fallback copy and error handling.
|
||||
- Risk: canonical misconfiguration causes duplicate indexing.
|
||||
- Mitigation: centralized canonical helper + validation checks.
|
||||
|
||||
## Recommended Execution Order
|
||||
1. robots.txt + sitemap.xml
|
||||
2. route-level metadata and canonicals
|
||||
3. structured data (Organization + Event)
|
||||
4. prerender for public routes
|
||||
5. performance and internal-linking refinement
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user