Adds readme documentation, and startup script.

README.md

@@ -0,0 +1,81 @@
+# cali_scraper
+
+A scraper for the California Legislature's Daily File publications — the
+Assembly and Senate pages that list bills up for hearing — plus a
+relational store and a local web dashboard for browsing the results.
+
+## What this is
+
+The pipeline has three lanes:
+
+1. **Ingest (the scrape)** — `python -m scraper ingest`. Downloads and caches
+   the Assembly daily-file HTML page and the Senate daily-file PDF for a given
+   date, plus the chambers' "upcoming events" calendars. Raw pages are cached
+   on disk (`cache/raw/`), so re-running the same date hits the network zero
+   times. A legitimate "no publication that day" (weekend / recess) is treated
+   as a normal outcome, not an error.
+2. **Store & analyze** — `parse`, `migrate`, and `correlate` load the cached
+   pages into a SQLite database (`db/daily_file.sqlite`, schema in
+   `SCHEMA.md`); `analyze` runs subject-matter tagging on bills (built-in
+   keyword tagger by default; optionally a local Ollama model, see `llm:` in
+   `config.yaml`).
+3. **Dashboard (web server)** — `python -m scraper dashboard`. Serves an API
+   plus a static frontend on `http://127.0.0.1:8000/` so you can browse bills,
+   committees, members, and tags without touching the database directly.
+
+`start.sh` at the repo root runs the scrapes, then starts the dashboard.
+
+## How to kick it off
+
+### Prerequisites
+
+- Python 3.9+
+- Dependencies: `pip install -r requirements.txt`
+- Populate the database at least once so the dashboard has data to show (see
+  below — `start.sh` scrapes but does not build `db/daily_file.sqlite`).
+
+### Just run it
+
+```bash
+./start.sh
+```
+
+That scrapes **today's** daily files (plus the events calendars), then serves
+the dashboard at http://127.0.0.1:8000/. Press `Ctrl+C` to stop.
+
+### Useful variations
+
+```bash
+# scrape a specific date, then serve
+DATE=2026-09-02 ./start.sh
+
+# force a network re-fetch even if a date is already cached
+FORCE=1 ./start.sh
+
+# serve on a different port / bind address
+PORT=9000 HOST=0.0.0.0 ./start.sh
+
+# skip the scrape and just start the dashboard
+SKIP_SCRAPE=1 ./start.sh
+```
+
+### Build the database (one time)
+
+`start.sh` handles the scrape + server, but to see bills in the dashboard the
+SQLite database must be built from the cached pages:
+
+```bash
+python -m scraper ingest --date 2026-09-02                    # download + cache
+python -m scraper ingest --date 2026-09-02 --store-events \
+    --db db/daily_file.sqlite                                  # events calendar
+python -m scraper parse --input cache/raw/assembly_daily_file \
+    --house assembly --sqlite db/daily_file.sqlite             # bills table
+python -m scraper migrate --db db/daily_file.sqlite            # relational schema
+python -m scraper correlate --db db/daily_file.sqlite          # bill <-> committee
+python -m scraper analyze --db db/daily_file.sqlite            # subject tags
+```
+
+Then `./start.sh` (or `python -m scraper dashboard` directly) will serve it.
+
+See `INGESTION.md`, `DAILY_FILE_PARSE_SPEC.md`, `CORRELATE.md`, and
+`LLM_ANALYSIS.md` for the full command reference.