237 lines
9.9 KiB
Markdown
237 lines
9.9 KiB
Markdown
# Plan: Migrate File Uploads from Disk to Database Blob Storage
|
||
|
||
## Context
|
||
|
||
Uploaded images are currently saved to `backend/uploads/` on disk with UUID-based filenames, served via a `StaticFiles` mount in dev and (presumably) Nginx in prod. The upload logic is copy-pasted across two backend files. The goal is to:
|
||
|
||
1. Unify the duplicated upload code into a single endpoint
|
||
2. Store uploaded binary blobs in PostgreSQL instead of on disk
|
||
3. Serve them through a new `/api/storage/{blob_id}` endpoint
|
||
4. Update all frontend uploaders and display sites to work with the new URL format
|
||
|
||
Uploaded blob URLs will change from `uploads/{uuid}{ext}` (relative, disk-based) to `/api/storage/{blob_id}` (absolute API path, DB-backed). Static SVG defaults (`svg/small-flower.svg`) are untouched — they are Angular build assets, not uploaded files.
|
||
|
||
---
|
||
|
||
## Step 1 — Add `StorageBlob` model
|
||
|
||
**File:** `backend/app/models.py`
|
||
|
||
Add a new SQLAlchemy model at the bottom of the file (after `CommunityHighlight`):
|
||
|
||
```python
|
||
from datetime import datetime, timezone
|
||
|
||
class StorageBlob(Base):
|
||
__tablename__ = "storage_blobs"
|
||
|
||
id = Column(Integer, primary_key=True, autoincrement=True)
|
||
blob_id = Column(String(32), unique=True, nullable=False, index=True)
|
||
content_type = Column(String(100), nullable=False)
|
||
size = Column(Integer, nullable=False)
|
||
data = Column(LargeBinary, nullable=False)
|
||
created_at = Column(DateTime, nullable=False, default=lambda: datetime.now(timezone.utc))
|
||
```
|
||
|
||
- `blob_id`: 32-char hex string (from `uuid.uuid4().hex`), indexed for fast lookups
|
||
- `data`: `LargeBinary` → PostgreSQL `BYTEA`, holds the actual image bytes
|
||
- No FK relationships — blobs are referenced by URL string in existing models
|
||
|
||
No Alembic migration needed. `Base.metadata.create_all()` in `main.py` lifespan will create the table on next startup.
|
||
|
||
---
|
||
|
||
## Step 2 — Create the storage router
|
||
|
||
**New file:** `backend/app/api/storage.py`
|
||
|
||
Two endpoints in a single router:
|
||
|
||
### `POST /upload` (admin-only) — unified upload
|
||
|
||
```python
|
||
ALLOWED_MIME_TYPES = {"image/png", "image/jpeg", "image/webp", "image/avif"}
|
||
MAX_FILE_SIZE = 5 * 1024 * 1024 # 5 MB
|
||
|
||
# Magic byte signatures → MIME type
|
||
MAGIC_BYTES = {
|
||
b'\x89PNG\r\n\x1a\n': 'image/png',
|
||
b'\xff\xd8\xff': 'image/jpeg',
|
||
b'\x00\x00\x00\x1cftypavif': 'image/avif',
|
||
b'\x00\x00\x00\x20ftypwebp': 'image/webp',
|
||
}
|
||
```
|
||
|
||
Logic:
|
||
1. Read file content, enforce `MAX_FILE_SIZE`
|
||
2. Validate magic bytes against `MAGIC_BYTES` table — reject if no match (this also rejects SVG, which has no binary magic bytes)
|
||
3. Generate `blob_id = uuid.uuid4().hex`
|
||
4. Insert `StorageBlob` row into the database
|
||
5. Return `{"url": f"/api/storage/{blob_id}"}`
|
||
|
||
### `GET /{blob_id}` (public) — blob retrieval
|
||
|
||
1. Query `StorageBlob` by `blob_id`
|
||
2. Return via `Response(content=blob.data, media_type=blob.content_type, headers={"Content-Length": str(blob.size)})`
|
||
3. Return 404 if not found
|
||
|
||
---
|
||
|
||
## Step 3 — Register the storage router and remove the old upload router
|
||
|
||
**File:** `backend/app/main.py`
|
||
|
||
- Add `storage` to the import line (line 13): `from app.api import events, tiers, auth, station_config, history, team, community, shows, storage`
|
||
- Add router registration: `app.include_router(storage.router, prefix="/api/storage", tags=["storage"])`
|
||
- Remove `upload` from the import line
|
||
- Remove `app.include_router(upload.router, prefix="/api/upload", tags=["upload"])`
|
||
- Remove the `StaticFiles` mount block (lines 104–108) and its `from fastapi.staticfiles import StaticFiles` import (line 6)
|
||
- Remove the `os.makedirs(uploads_dir, exist_ok=True)` block in the lifespan (lines 71–73)
|
||
- Check if `os` import is still needed — it is not used elsewhere after removing uploads_dir, so remove it
|
||
|
||
---
|
||
|
||
## Step 4 — Delete the old disk-based upload file
|
||
|
||
**File:** `backend/app/api/upload.py` — **delete entirely**
|
||
|
||
This file is replaced by the new `storage.py` router.
|
||
|
||
---
|
||
|
||
## Step 5 — Remove duplicate upload code from station_config.py
|
||
|
||
**File:** `backend/app/api/station_config.py`
|
||
|
||
- Delete the `upload_image` endpoint function (lines 59–95)
|
||
- Delete `UPLOAD_DIR` and `MAX_FILE_SIZE` constants (lines 18–19)
|
||
- Remove unused imports: `os`, `uuid`, `UploadFile` from the `fastapi` import line (line 6)
|
||
|
||
The file will contain only the `GET /` and `PUT /` endpoints for station config CRUD.
|
||
|
||
---
|
||
|
||
## Step 6 — Add `resolveImageUrl()` helper to the frontend
|
||
|
||
**File:** `src/app/services/app-config.service.ts`
|
||
|
||
Add a utility function after `getAppConfig()`:
|
||
|
||
```typescript
|
||
/** Resolve an image URL for rendering.
|
||
* - Absolute HTTP(S) URLs are returned as-is.
|
||
* - Static SVG assets (svg/…) are served by the Angular build — returned as-is.
|
||
* - API paths (/api/storage/…) and legacy paths (uploads/…) are prefixed with apiBaseUrl.
|
||
*/
|
||
export function resolveImageUrl(url: string): string {
|
||
if (!url) return '';
|
||
if (url.startsWith('http://') || url.startsWith('https://')) return url;
|
||
if (url.startsWith('svg/')) return url;
|
||
if (url.startsWith('/')) return `${getAppConfig().apiBaseUrl}${url}`;
|
||
return `${getAppConfig().apiBaseUrl}/${url}`;
|
||
}
|
||
```
|
||
|
||
Key subtlety: new blob URLs start with `/` (`/api/storage/...`), so they concatenate directly with `apiBaseUrl`. Legacy relative paths (`uploads/...`) still get a `/` separator. Static SVG paths are left alone.
|
||
|
||
---
|
||
|
||
## Step 7 — Update frontend upload services
|
||
|
||
**File:** `src/app/services/upload.service.ts`
|
||
- Change `baseUrl` from `${getAppConfig().apiBaseUrl}/api/upload/image` to `${getAppConfig().apiBaseUrl}/api/storage/upload`
|
||
|
||
**File:** `src/app/services/station-config.service.ts`
|
||
- Change `uploadImage()` to call `${getAppConfig().apiBaseUrl}/api/storage/upload` instead of `${this.baseUrl}/upload`
|
||
- This unifies all uploads through the single storage endpoint
|
||
|
||
---
|
||
|
||
## Step 8 — Update admin form preview methods
|
||
|
||
Both admin forms have `getPreviewUrl()` methods that build the preview URL for uploaded images. Update them to use the shared `resolveImageUrl()`:
|
||
|
||
**File:** `src/app/admin/admin-team-form.component.ts`
|
||
- Import `resolveImageUrl` from `../services/app-config.service`
|
||
- Replace `getPreviewUrl()` body with `return resolveImageUrl(url);`
|
||
|
||
**File:** `src/app/admin/admin-show-form.component.ts`
|
||
- Same change as above
|
||
|
||
---
|
||
|
||
## Step 9 — Update the schedule component's `resolveAsset()`
|
||
|
||
**File:** `src/app/schedule/schedule.component.ts`
|
||
|
||
Replace `resolveAsset()` to use the shared helper:
|
||
|
||
```typescript
|
||
import { resolveImageUrl } from '../services/app-config.service';
|
||
|
||
resolveAsset(url: string | null): string {
|
||
return resolveImageUrl(url ?? '');
|
||
}
|
||
```
|
||
|
||
No template changes needed — the schedule template already calls `resolveAsset()` for `show_art_url` and `hero_image_url`.
|
||
|
||
---
|
||
|
||
## Step 10 — Add `resolveImageUrl()` to all display components
|
||
|
||
Currently, StationConfig image fields (`logo_url`, `hero_background_url`, `hero_icon_url`, `hero_divider_url`) and `member.photo_url` are bound directly in templates without any URL resolution. With the new `/api/storage/{blob_id}` format, they need resolution in dev mode (where the Angular app runs on `:4200` and the API on `:8000`).
|
||
|
||
For each component: import `resolveImageUrl` in the `.ts` file and expose it as a public method. Then wrap every image binding in the template.
|
||
|
||
### hero.component.ts / hero.component.html
|
||
- Add method: `resolveImageUrl(url: string): string { return resolveImageUrl(url); }`
|
||
- Template changes (3 bindings):
|
||
- Line 4: `[src]="config().hero_background_url"` → `[src]="resolveImageUrl(config().hero_background_url)"`
|
||
- Line 14: `[src]="config().hero_icon_url"` → `[src]="resolveImageUrl(config().hero_icon_url)"`
|
||
- Line 40: `[src]="config().hero_divider_url"` → `[src]="resolveImageUrl(config().hero_divider_url)"`
|
||
|
||
### navbar.component.ts / navbar.component.html
|
||
- Add method
|
||
- Template: Line 4: `[src]="config().logo_url"` → `[src]="resolveImageUrl(config().logo_url)"`
|
||
|
||
### footer.component.ts / footer.component.html
|
||
- Add method
|
||
- Template: Line 5: `[src]="config().logo_url"` → `[src]="resolveImageUrl(config().logo_url)"`
|
||
|
||
### about.component.ts / about.component.html
|
||
- Add method
|
||
- Template changes (3 bindings):
|
||
- Line 5: `[src]="config().hero_icon_url"` → `[src]="resolveImageUrl(config().hero_icon_url)"`
|
||
- Line 59: `[src]="member.photo_url"` → `[src]="resolveImageUrl(member.photo_url)"`
|
||
- Line 79: `[src]="config().logo_url"` → `[src]="resolveImageUrl(config().logo_url)"`
|
||
|
||
### donate.component.ts / donate.component.html
|
||
- Add method
|
||
- Template changes (2 bindings):
|
||
- Line 6: `[src]="config().hero_icon_url"` → `[src]="resolveImageUrl(config().hero_icon_url)"`
|
||
- Line 45: `[src]="config().logo_url"` → `[src]="resolveImageUrl(config().logo_url)"`
|
||
|
||
### login.component.ts / login.component.html
|
||
- Add method
|
||
- Template: Line 4: `[src]="config().logo_url"` → `[src]="resolveImageUrl(config().logo_url)"`
|
||
|
||
### schedule.component.ts / schedule.component.html
|
||
- Already has `resolveAsset()` — update it in Step 9
|
||
- Add a separate `resolveImageUrl()` method for the StationConfig logo
|
||
- Template: Line 4: `[src]="config().logo_url"` → `[src]="resolveImageUrl(config().logo_url)"`
|
||
|
||
---
|
||
|
||
## Verification
|
||
|
||
1. **Start the backend** — verify the `storage_blobs` table is created on startup
|
||
2. **Swagger UI** (`http://localhost:8000/docs`):
|
||
- `POST /api/storage/upload` — upload a PNG/JPEG/WebP/AVIF image → returns `{"url": "/api/storage/<hex>"}`
|
||
- Upload an SVG → should return 400 (rejected)
|
||
- Upload a non-image file with `Content-Type: image/png` → should return 400 (magic byte mismatch)
|
||
- `GET /api/storage/<hex>` — returns the binary image with correct Content-Type
|
||
3. **Admin UI** — log in, navigate to admin forms, upload images on team/show/station forms, verify the returned URL is stored in the correct field
|
||
4. **Public pages** — visit home, about, schedule, donate pages — verify all images render correctly
|
||
5. **Dev/prod parity** — verify that with `apiBaseUrl: ''` (prod), `/api/storage/{blob_id}` resolves correctly through the API proxy
|