# LLM Analysis — Subject-Matter Tag Clouds (spec §4) **Task:** `t_950780e0` — Ollama bill analysis + subject-matter tag clouds. **Depends on:** `t_2736f40d` (correlation lane / relational DB schema). **Python:** 3.9.6. No new third-party dependencies (stdlib `urllib`). --- ## Architecture ``` config.yaml::llm ──Config.llm──▶ scraper/llm.py ──▶ bill_tags table │ (db/daily_file.sqlite) │ │ ▼ ▼ fallback tagger out/tag_cloud.json (zero-network, keyword / short_title extraction) ``` ### Modules | File | Purpose | |---|---| | `scraper/llm.py` | Ollama HTTP client, deterministic fallback tagger, `bill_tags` DDL, `build_tag_cloud()`, `analyze()` orchestrator. | | `scraper/analyze_cli.py` | `analyze` subcommand (guarded `register()` + `handle_analyze()`); wired additively into `scraper/cli.py`. | | `config.yaml` § `llm:` | Endpoint + model knobs (append; http/cache/alerts/sources untouched). | | `scraper/config.py` | `Config.llm` property; `_DEFAULTS["llm"]` fallbacks so the package survives a missing section. | --- ## Configuration Append to `config.yaml` (do NOT touch `http`, `cache`, `alerts`, `sources`): ```yaml llm: enabled: false # false → deterministic fallback tagger (no server) backend: ollama endpoint: "http://localhost:11434" # configurable; default local Ollama model: "llama3" # configurable; empty → fallback default_tags: 8 # max tags emitted per bill timeout_seconds: 30 ``` `--list-config` prints the resolved settings: ``` $ python -m scraper analyze --list-config llm: enabled: False backend: ollama endpoint: http://localhost:11434 model: llama3 default_tags: 8 timeout_seconds: 30 ``` --- ## Deterministic degradation The `analyze` path selects Ollama vs. fallback: | Trigger | Path | `generated_by` | reason | |---|---|---|---| | `enabled: false` | fallback | `fallback` | `disabled` | | `model` empty | fallback | `fallback` | `no model configured` | | endpoint reachable, model returns valid tags | ollama | `ollama:` | `enabled=true, model=` | | endpoint unreachable / connection refused | fallback | `fallback` | `unreachable: URLError` | | Ollama returns empty body | fallback | `fallback` | `ollama empty → fallback` | | Ollama returns unparseable response | fallback | `fallback` | `unreachable: ValueError` | Each bill tags row carries its own `generated_by` (`fallback` | `ollama:`), so a mixed run (some bills ollama, some degraded) is honestly recorded without fabrication. The `analyze()` summary's `path` field reports the dominant path (`fallback` wins on ties — conservative/honest). --- ## Storage: `bill_tags` ```sql CREATE TABLE IF NOT EXISTS bill_tags ( id INTEGER PRIMARY KEY, bill_id TEXT NOT NULL, tag TEXT NOT NULL, source TEXT, -- 'short_title' | 'title+taxonomy' | 'ollama' generated_by TEXT NOT NULL, -- 'fallback' | 'ollama:llama3' | … generated_at TEXT, UNIQUE (bill_id, tag, generated_by) ); ``` - Re-runs with the same `generated_by` add no rows (idempotent). - The flat `daily_file_records` table and its view are untouched. - `--rebuild` drops all `bill_tags` before rewriting. --- ## Usage ### Default run (fallback, zero network) ```bash $ python -m scraper analyze --rebuild === analyze === bills_tagged : 156 unique_tags : 118 path : fallback generated_by : fallback reason : disabled rows_written : 260 out : /Users/kyle/Documents/cali_scraper/out/tag_cloud.json top tags : 20 budget 14 bill related budget 9 health care 8 education 8 environment 7 child welfare 7 employment 7 energy 6 housing 5 insurance ``` ### Use live Ollama (flip enabled in config.yaml, or `--force-llm`) ```bash $ python -m scraper analyze --force-llm ``` When Ollama is unreachable (default when no server is running), the run silently falls back to the built-in tagger and logs a warning per bill. --- ## Tag-cloud output `out/tag_cloud.json` — deterministic, diff-stable: ```json { "tag_count": 118, "bill_count": 156, "generated_by": null, "tags": [ {"tag": "budget", "count": 20, "bills": 20}, {"tag": "health care", "count": 9, "bills": 9}, {"tag": "education", "count": 8, "bills": 8}, … ] } ``` `generated_by: null` aggregates all rows (both ollama and fallback); pass `--top N` to cap the list. --- ## Tests `tests/test_llm.py` — 12 tests, all offline (no real network): | Test | Coverage | |---|---| | `test_fallback_when_disabled_is_zero_network` | (a) `enabled=false`: `urlopen` is never called (`monkeypatch` asserts this). | | `test_tag_bill_fallback_never_invents` | Fallback always produces a signal; keyword tags are extracted correctly. | | `test_mocked_ollama_parsed_and_stored` | (b) Canned OpenAI-compatible dict fed via injectable `post`; `generated_by='ollama:llama3'` confirmed in DB. | | `test_ollama_parse_json_and_fenced` | JSON-in-fence parsing; bare JSON parsing; degraded fallback splitting. | | `test_unreachable_endpoint_falls_back_no_crash` | (c) Canned `post` raises `URLError`; all bills tagged, `generated_by='fallback'`, `reason='unreachable: …'`. | | `test_mixed_run_reports_fallback_path` | Ollama empty → per-bill fallback; `dominant='fallback'`. | | `test_build_tag_cloud_aggregates` | Cloud is sorted desc; JSON file shape correct. | | `test_bill_tags_table_is_additive_and_idempotent` | Second analyze adds 0 rows; flat table untouched. | | `test_ensure_tag_table_does_not_touch_flat` | `CREATE … IF NOT EXISTS` for `bill_tags` adds no column to `daily_file_records`. | | `test_analyze_registers_without_clobbering` | `analyze` subcommand doesn't remove sibling commands (`parse`/`migrate`/`ingest`). | | `test_config_llm_section_is_additive` | `config.llm` is loadable; `http`/`sources` still present. | | `test_handle_analyze_end_to_end` | CLI handler returns 0; `out/tag_cloud.json` written and parseable. | Full suite: 106 passed, 2 skipped, 1 xfailed. Coverage: 92.0% ≥ 85 gate. --- ## Contract compliance | Requirement | Where honoured | |---|---| | Python 3.9.6 | `from __future__ import annotations`; f-strings; no 3.10+ syntax. | | Configurable endpoint + model | `cfg["endpoint"]`, `cfg["model"]` read from `config.llm`; never hardcoded in the tagger. | | Stdlib `urllib` (no new dep) | `_urllib_post()` uses `urllib.request.Request` + `urlopen`. | | Deterministic fallback | `tag_bill_fallback()` runs when `enabled=false` / unreachable / empty model answer. | | Zero-neg degradation proven | `monkeypatch.setattr(urllib.request, "urlopen", _deny)` — asserts `urlopen` never called. | | `bill_tags(bill_id, tag, source, generated_by)` | DDL matches (plus `id`, `generated_at` for idempotency + audit trail). | | `out/tag_cloud.json` | Written by `write_tag_cloud()` via `analyze()`. | | CLI additive + guarded | `analyze_cli.register(sub)` inside `try/except` in `cli.build_parser()`; `handle_analyze` added to the handlers dict. | | pytest green, ≥ 85 coverage | 106 passed / 92.0% coverage. |