Adds visitor dashboard and related features
This commit is contained in:
@@ -26,6 +26,30 @@ See [README.md](README.md) for a full architecture map. Key files:
|
||||
- `backend/app/main.py` — FastAPI entry point (CORS, router registration)
|
||||
- `backend/app/models.py` — SQLAlchemy ORM (Show, Event, StationConfig)
|
||||
|
||||
### Admin dashboard — visitor stats (implemented)
|
||||
|
||||
Nginx access logs → shared volume → FastAPI log parser → PostgreSQL → admin dashboard. No extra services.
|
||||
|
||||
**How it works:**
|
||||
1. Nginx writes structured JSON access logs to **daily log files** on a shared mount: `/logs/access-YYYY-MM-DD.log`. A `map` directive in [nginx.conf.template](nginx.conf.template) extracts the date from the ISO timestamp, so Nginx rotates to a new file at midnight.
|
||||
2. FastAPI ([backend/app/log_parser.py](backend/app/log_parser.py)) maintains a **"last parsed date"** (at most yesterday) persisted in PostgreSQL via the `LogParseState` model. On each run it scans `/logs/` for unprocessed daily files, enriches entries with IP geolocation (MaxMind GeoLite2 City), and stores aggregates.
|
||||
3. Aggregates are persisted in `visitor_stats_daily` (unique visitors per day) and `visitor_stats_geo` (country/city breakdown) tables.
|
||||
4. Admin dashboard (Angular) reads from stats API endpoints in [backend/app/api/stats.py](backend/app/api/stats.py).
|
||||
|
||||
**Key files:**
|
||||
- `backend/app/log_parser.py` — log file parsing + geo enrichment + `process_unparsed_logs()`
|
||||
- `backend/app/api/stats.py` — stats API endpoints (summary, timeseries, geo, parse trigger)
|
||||
- `backend/app/models.py` — `VisitorStatDaily`, `VisitorStatGeo`, `LogParseState` ORM models
|
||||
- `backend/inject_test_logs.py` — test data injector (writes test log entries for yesterday)
|
||||
- `nginx.conf.template` — JSON log format (`kmtn_json`) with daily rotation via `$log_date` map
|
||||
- `backend/geo/GeoLite2-City.mmdb` — MaxMind geolocation database
|
||||
|
||||
**Why daily files:** The currently-being-written file is never read — no partial lines, no race conditions, no file locking. Stats have a minimum one-day lag.
|
||||
|
||||
**Deployment context:** Single-node minikube, `hostPath` PersistentVolume with `ReadWriteMany`, or docker-compose named volume mounted in both pods.
|
||||
|
||||
**Injecting test data:** Run `backend/inject_test_logs.py` (or the inline docker compose command in README.md) to write test log entries for yesterday, then trigger parsing via `GET /api/parse`.
|
||||
|
||||
## Common tasks
|
||||
|
||||
- **Re-skin:** Edit colors in `_variables.scss` only.
|
||||
|
||||
@@ -5,7 +5,31 @@
|
||||
"Read(*)",
|
||||
"Glob(*)",
|
||||
"Grep(*)",
|
||||
"WebFetch(domain:www.kryzradio.org)"
|
||||
"WebFetch(domain:www.kryzradio.org)",
|
||||
"Bash(ls -la /logs/ 2>/dev/null || echo \"DIRECTORY /logs DOES NOT EXIST\")",
|
||||
"Read(//logs/**)",
|
||||
"Bash(head -5 /logs/access-2026-06-26.log)",
|
||||
"Bash(python3 -c \"import sys, json; ips = set\\(\\); [ips.add\\(json.loads\\(l\\)['remote_addr']\\) for l in sys.stdin if l.strip\\(\\)]; print\\(f'Total unique IPs: {len\\(ips\\)}'\\); print\\('\\\\n'.join\\(sorted\\(ips\\)\\)\\)\")",
|
||||
"Bash(python3 -c \"from datetime import date; print\\(f'Today: {date.today\\(\\)}'\\); print\\(f'Log files from Jun 13-27 are all before today: {date\\(2026, 6, 27\\) < date.today\\(\\)}'\\)\")",
|
||||
"Bash(ls -la /logs/ 2>/dev/null || echo \"/logs/ not accessible\")",
|
||||
"Bash(python3 -c \"import sys,json; ips=set\\(\\); [ips.add\\(json.loads\\(l\\)['remote_addr']\\) for l in sys.stdin if l.strip\\(\\)]; print\\(f'Unique IPs in 06-26: {len\\(ips\\)}'\\); [print\\(f' {ip}'\\) for ip in sorted\\(ips\\)]\")",
|
||||
"Bash(python3 -c ' *)",
|
||||
"Bash(docker exec *)",
|
||||
"Bash(curl -s http://localhost:8000/api/stats/summary)",
|
||||
"Bash(python3 -m json.tool)",
|
||||
"Bash(curl -s \"http://localhost:8000/api/stats/timeseries?start_date=2026-06-13&end_date=2026-06-27\")",
|
||||
"Bash(curl -s \"http://localhost:8000/api/stats/geo?start_date=2026-06-13&end_date=2026-06-27\")",
|
||||
"Bash(env)",
|
||||
"Bash(curl -s -X POST http://localhost:8000/api/auth/login -H \"Content-Type: application/json\" -d '{\"username\":\"admin\",\"password\":\"123\"}')",
|
||||
"Bash(curl -s http://localhost:8000/api/stats/summary -H \"Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxIiwiaXNfYWRtaW4iOnRydWUsImV4cCI6MTc4MjcxMTYyOX0.8eM-Vd0H72R8eVomEtxLNeYSYUe65-0WPmNWUzx_mxw\")",
|
||||
"Bash(curl -s \"http://localhost:8000/api/stats/timeseries?start_date=2026-06-13&end_date=2026-06-27\" -H \"Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxIiwiaXNfYWRtaW4iOnRydWUsImV4cCI6MTc4MjcxMTYyOX0.8eM-Vd0H72R8eVomEtxLNeYSYUe65-0WPmNWUzx_mxw\")",
|
||||
"Bash(curl -s \"http://localhost:8000/api/stats/geo?start_date=2026-06-13&end_date=2026-06-27\" -H \"Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxIiwiaXNfYWRtaW4iOnRydWUsImV4cCI6MTc4MjcxMTYyOX0.8eM-Vd0H72R8eVomEtxLNeYSYUe65-0WPmNWUzx_mxw\")",
|
||||
"Bash(python3 -c \"import maxminddb; print\\(maxminddb.__version__\\); print\\(dir\\(maxminddb.Reader\\)\\)\")",
|
||||
"Bash(cat /tmp/uvicorn.log)",
|
||||
"Read(//tmp/**)",
|
||||
"Bash(KMTN_DATABASE_URL=\"sqlite+aiosqlite:///./kmountain.db\" KMTN_ADMIN_USERNAME=\"admin\" KMTN_ADMIN_PASSWORD=\"123\" KMTN_CORS_ORIGINS='[\"http://localhost:4200\"]' nohup uvicorn app.main:app --reload --host 0.0.0.0 --port 8000)",
|
||||
"Bash(echo \"Backend PID: $!\")",
|
||||
"Bash(curl -s http://localhost:8000/docs)"
|
||||
],
|
||||
"additionalDirectories": [
|
||||
"/workspaces/web_app/.vscode"
|
||||
|
||||
Reference in New Issue
Block a user