Adds visitor dashboard and related features

This commit is contained in:
2026-06-28 06:41:23 +00:00
parent a7cccbf170
commit 77d1c04d86
35 changed files with 1938 additions and 8 deletions
+24
View File
@@ -26,6 +26,30 @@ See [README.md](README.md) for a full architecture map. Key files:
- `backend/app/main.py` — FastAPI entry point (CORS, router registration)
- `backend/app/models.py` — SQLAlchemy ORM (Show, Event, StationConfig)
### Admin dashboard — visitor stats (implemented)
Nginx access logs → shared volume → FastAPI log parser → PostgreSQL → admin dashboard. No extra services.
**How it works:**
1. Nginx writes structured JSON access logs to **daily log files** on a shared mount: `/logs/access-YYYY-MM-DD.log`. A `map` directive in [nginx.conf.template](nginx.conf.template) extracts the date from the ISO timestamp, so Nginx rotates to a new file at midnight.
2. FastAPI ([backend/app/log_parser.py](backend/app/log_parser.py)) maintains a **"last parsed date"** (at most yesterday) persisted in PostgreSQL via the `LogParseState` model. On each run it scans `/logs/` for unprocessed daily files, enriches entries with IP geolocation (MaxMind GeoLite2 City), and stores aggregates.
3. Aggregates are persisted in `visitor_stats_daily` (unique visitors per day) and `visitor_stats_geo` (country/city breakdown) tables.
4. Admin dashboard (Angular) reads from stats API endpoints in [backend/app/api/stats.py](backend/app/api/stats.py).
**Key files:**
- `backend/app/log_parser.py` — log file parsing + geo enrichment + `process_unparsed_logs()`
- `backend/app/api/stats.py` — stats API endpoints (summary, timeseries, geo, parse trigger)
- `backend/app/models.py``VisitorStatDaily`, `VisitorStatGeo`, `LogParseState` ORM models
- `backend/inject_test_logs.py` — test data injector (writes test log entries for yesterday)
- `nginx.conf.template` — JSON log format (`kmtn_json`) with daily rotation via `$log_date` map
- `backend/geo/GeoLite2-City.mmdb` — MaxMind geolocation database
**Why daily files:** The currently-being-written file is never read — no partial lines, no race conditions, no file locking. Stats have a minimum one-day lag.
**Deployment context:** Single-node minikube, `hostPath` PersistentVolume with `ReadWriteMany`, or docker-compose named volume mounted in both pods.
**Injecting test data:** Run `backend/inject_test_logs.py` (or the inline docker compose command in README.md) to write test log entries for yesterday, then trigger parsing via `GET /api/parse`.
## Common tasks
- **Re-skin:** Edit colors in `_variables.scss` only.
+25 -1
View File
@@ -5,7 +5,31 @@
"Read(*)",
"Glob(*)",
"Grep(*)",
"WebFetch(domain:www.kryzradio.org)"
"WebFetch(domain:www.kryzradio.org)",
"Bash(ls -la /logs/ 2>/dev/null || echo \"DIRECTORY /logs DOES NOT EXIST\")",
"Read(//logs/**)",
"Bash(head -5 /logs/access-2026-06-26.log)",
"Bash(python3 -c \"import sys, json; ips = set\\(\\); [ips.add\\(json.loads\\(l\\)['remote_addr']\\) for l in sys.stdin if l.strip\\(\\)]; print\\(f'Total unique IPs: {len\\(ips\\)}'\\); print\\('\\\\n'.join\\(sorted\\(ips\\)\\)\\)\")",
"Bash(python3 -c \"from datetime import date; print\\(f'Today: {date.today\\(\\)}'\\); print\\(f'Log files from Jun 13-27 are all before today: {date\\(2026, 6, 27\\) < date.today\\(\\)}'\\)\")",
"Bash(ls -la /logs/ 2>/dev/null || echo \"/logs/ not accessible\")",
"Bash(python3 -c \"import sys,json; ips=set\\(\\); [ips.add\\(json.loads\\(l\\)['remote_addr']\\) for l in sys.stdin if l.strip\\(\\)]; print\\(f'Unique IPs in 06-26: {len\\(ips\\)}'\\); [print\\(f' {ip}'\\) for ip in sorted\\(ips\\)]\")",
"Bash(python3 -c ' *)",
"Bash(docker exec *)",
"Bash(curl -s http://localhost:8000/api/stats/summary)",
"Bash(python3 -m json.tool)",
"Bash(curl -s \"http://localhost:8000/api/stats/timeseries?start_date=2026-06-13&end_date=2026-06-27\")",
"Bash(curl -s \"http://localhost:8000/api/stats/geo?start_date=2026-06-13&end_date=2026-06-27\")",
"Bash(env)",
"Bash(curl -s -X POST http://localhost:8000/api/auth/login -H \"Content-Type: application/json\" -d '{\"username\":\"admin\",\"password\":\"123\"}')",
"Bash(curl -s http://localhost:8000/api/stats/summary -H \"Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxIiwiaXNfYWRtaW4iOnRydWUsImV4cCI6MTc4MjcxMTYyOX0.8eM-Vd0H72R8eVomEtxLNeYSYUe65-0WPmNWUzx_mxw\")",
"Bash(curl -s \"http://localhost:8000/api/stats/timeseries?start_date=2026-06-13&end_date=2026-06-27\" -H \"Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxIiwiaXNfYWRtaW4iOnRydWUsImV4cCI6MTc4MjcxMTYyOX0.8eM-Vd0H72R8eVomEtxLNeYSYUe65-0WPmNWUzx_mxw\")",
"Bash(curl -s \"http://localhost:8000/api/stats/geo?start_date=2026-06-13&end_date=2026-06-27\" -H \"Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxIiwiaXNfYWRtaW4iOnRydWUsImV4cCI6MTc4MjcxMTYyOX0.8eM-Vd0H72R8eVomEtxLNeYSYUe65-0WPmNWUzx_mxw\")",
"Bash(python3 -c \"import maxminddb; print\\(maxminddb.__version__\\); print\\(dir\\(maxminddb.Reader\\)\\)\")",
"Bash(cat /tmp/uvicorn.log)",
"Read(//tmp/**)",
"Bash(KMTN_DATABASE_URL=\"sqlite+aiosqlite:///./kmountain.db\" KMTN_ADMIN_USERNAME=\"admin\" KMTN_ADMIN_PASSWORD=\"123\" KMTN_CORS_ORIGINS='[\"http://localhost:4200\"]' nohup uvicorn app.main:app --reload --host 0.0.0.0 --port 8000)",
"Bash(echo \"Backend PID: $!\")",
"Bash(curl -s http://localhost:8000/docs)"
],
"additionalDirectories": [
"/workspaces/web_app/.vscode"