The no-mention-of-pornhub branch stopped finding plain <video> elements on
pages like thisvid.com. Two site-agnostic causes, both fixed without any
site-specific references:
1. Predicate over-rejection: isValidManifestUrl (and the nested isManifestUrl
in m3u8DeepProbeFn) ran the placeholder/`%`/trailing-slash rejections
BEFORE the concrete media signals. thisvid serves its direct MP4 WITH a
trailing slash (.../14644093.mp4/?rnd=...), so `path.endsWith('/')`
rejected a real, fetchable source. Concrete media signals (.mp4/.webm/.ogv/
.mpd/.m4s/.m3u8/.hlsvod//hls/) now win outright, so a coincidental '%' or
trailing slash never sinks a real source — while genuine route templates
(/playlists/, /playlist/%1%/, /videos/{id}/) are still rejected.
2. Runtime cause on ad-iframe-heavy pages: scrapeTab ran only a MAIN-world
scrape. A MAIN-world + allFrames injection can reject on pages with
sandboxed/opaque-origin ad iframes, returning 0 videos even when a <video>
is visible. scrapeTab now falls back to an isolated-world scrape (no `world`
key — always permitted) when the MAIN-world run finds nothing. The per-frame
reduction was extracted into a shared reduceScrapeFrames helper.
Adds __tests__/simple_video.test.mjs: a thisvid isValidManifestUrl battery
(trailing-slash MP4 accepted, templates still rejected, %2B/%2F tokens
preserved) plus a structural guard that scrapeTab keeps both a MAIN-world
primary and an isolated-world fallback.
Co-Authored-By: Claude <noreply@anthropic.com>
The video-detection pipeline was finding nothing on pages that have a bare
<video> element. Two root causes:
1. world: 'MAIN' placement. The deep probe + scraper are injected with
chrome.scripting.executeScript. Their code (and its own comments) requires
the MAIN world to see page globals — window.Hls, per-<video> Hls instances,
video._goon_hls_manifest, and window.__goon_plugin_data (written by the
main-world interceptors). The calls were missing it, so every main-world
strategy (A/A2/E/H) found 0. It was also briefly placed inside `target`,
which chrome.scripting rejects ("Unexpected property: 'world'"); `world` is
a sibling of `target` on ScriptInjection. Restore it as a sibling of target.
2. fake "manifest" classification. Strategy H was reporting 10 nav links /
server-side route TEMPLATES as manifests — e.g. /playlists/,
/playlists/liked/, /playlist/%1%/video/%2%/, /members/%ID%/playlists/ —
because the old /playlist|/manifest substring check matches nav menus and
leaves %1%/%ID%/%DIR% template placeholders intact. Those then became the
stored "video" entries, drowning out the real element.
isManifestUrl and isValidManifestUrl now reject:
- dynamic route templates (%NAME% or {name} tokens), using a negative
lookahead so genuine %-encoded tokens like %2B/%2F are preserved
(the invariant the codebase explicitly protects),
- plural collection paths (/playlists, /manifests),
- trailing-slash directory listings.
The 'World: ISOLATED' probe log was also misleading (typeof chrome is not a
world discriminator — the probe provably runs in MAIN); replaced it.
Adds a regression test (hls_qa) reproducing thisvid's pageContext.langUrls
scenario: all 10 templates are rejected while a real .m3u8 carrying a %2B token
is still detected. 90/90 tests pass; 0 lint errors; build OK.
Address robot's review on PR #5:
- [Blocking] 'goon_referers' is a host-keyed GLOBAL store and must outlive
its capture tab; it was wrongly added to the per-tab onRemoved cleanup, so
closing any tab wiped referers captured by still-open source tabs (-> 412).
It is no longer removed on tab close. Stale referers are now pruned
opportunistically in persistReferer via a per-host timestamp + TTL.
- [Should-fix] the DNR referer rule used a single shared REFERER_RULE_ID with
a non-atomic check+update, so concurrent proxy fetches for two hosts could
interleave and leave the wrong host's referer live. Each host now gets its
own stable rule id (refererRuleIdFor) and the install+record is serialized
per host via an async lock (refererRuleLocks).
- [Coverage] add __tests__/referer_proxy.test.mjs exercising the new
referer-capture / proxy-host / onRemoved paths: closing a tab preserves
goon_referers, two hosts proxied concurrently keep distinct per-host rules,
and same-host proxy is idempotent.
The extension had hard-coupled itself to one provider's CDN throughout
background.js and its consumers: a host allowlist, a static
declarativeNetRequest rule that forced a fixed site origin as the Referer,
site-named message types/functions, and illustrative comments naming the
site. This commit removes every reference so the code is generic and
provider-agnostic (and no longer names a specific site for reviewers).
What changed:
- background.js
* Dropped the PHNCDN_HOSTS allowlist; video-request capture is now
host-agnostic (any CDN, driven by isVideoRequest / isHlsManifest).
* Removed the static DNR rule that forced a fixed site origin as the
Referer. Replaced with per-tab dynamic capture: an
onBeforeSendHeaders listener captures the provider's own Referer for
each video request the live page makes and stores it per CDN host
(goon_referers). The proxy handler installs a DNR rule scoped to the
requested host using that captured referer, so the provider's exact
origin is extracted at runtime instead of hard-coded.
* Generalized the reconstruction helpers: extractVideoId ->
extractGroupKey (path-directory key, no /hls/videos/ assumption),
buildManifestForVariant / reconstructManifestsFromSegments swap a
segment filename for master.m3u8 dir-agnostically, freshLiveForVideo
-> freshLiveForGroup, and the GOON_GET_PHNCDN_REQUESTS handler ->
GOON_GET_VIDEO_REQUESTS. Resolver response key videoId -> groupKey
* Reworded prose comments to describe behavior generically.
- manifest.json
* Added the extraHeaders permission so onBeforeSendHeaders can read the
Referer header (needed for the dynamic capture above).
- viewer.js / index.js (consumers)
* Proxying decision now keys off a learned goon_proxy_hosts set
(populated from a startup CORS probe) instead of hard-coded host
substrings; removed the hard-coded CORS ping.
* filterManifests / isValidManifestUrl generalized (kept the
.m3u8 / .hlsvod / .ts behavior the tests assert); rank key
phncdn_reconstructed -> hls_reconstructed; queryPhncdnRequests ->
queryVideoRequests.
- content.js, interceptors.js, and two test-prose strings: the illustrative
"sites like <site>" comments generalized.
Verification: 83/83 tests pass, 0 lint errors. A repo-wide grep for the
previously-referenced host / site now returns no matches. The pre-existing
tsc errors in vite.config.ts are unrelated (present on the prior commit).
Note on the dynamic referer: it works while the source tab is open (it reads
that tab's real Referer); if the tab closed before capturing, the proxy falls
back to a host-derived origin, which a strict origin check may still reject --
the tradeoff of not hard-coding the site.
CI times out on the 7 "HLS QA: Deep probe strategies" tests. Root cause
is a jsdom-only artifact, not a production bug: jsdom exposes the
self-referential cyclic globals (_globalObject/_globalProxy/_top/_parent/
window/self/top/parent/frames) as ENUMERABLE own-props on the window.
Strategy H in m3u8DeepProbeFn() runs Object.keys(window) and a no-
visited-set walk() over each, which explodes to O(204^5) ≈ 3.5e11 ops and
never terminates within the depth-5 cap, so the whole suite hangs forever.
A real Chrome MAIN-world window does NOT expose those keys as enumerable
own-props, so this is purely a test-environment mismatch. The fix is
test-only: sanitizeJsdomWinForProbe() marks the jsdom cyclic globals
non-enumerable in runProbeInWin() before eval, mirroring a real page.
No production/underlying source (src/viewer/index.js) is modified.
Local (Node 22.22.3 / Linux == CI ubuntu-latest + Node 22):
before: npm test hangs indefinitely (no Results line; CI job killed)
after: npm run build && npm test -> 83/83 passed in ~12.5s
Hls.cast() is a TypeScript identity cast in hls.js — returns input unchanged,
NOT a reverse-lookup. Replaced with scanning video element properties for
HLS instance references real players store as custom properties.
Fixed double-patching bug in interceptors.js and expanded URL pattern
matching to cover /hls/, /playlist/, /manifest/, .hlsvod paths.
83/83 tests pass, clean build (9 modules).
kfj001
robot