Compare commits

...
3 Commits
Author SHA1 Message Date
kfj001 7139c5ea6f fix(scan): run deep probe/scraper in MAIN world; reject route-template "manifests"
CI / test (pull_request) Successful in 5m48s
The video-detection pipeline was finding nothing on pages that have a bare
<video> element. Two root causes:

1. world: 'MAIN' placement. The deep probe + scraper are injected with
   chrome.scripting.executeScript. Their code (and its own comments) requires
   the MAIN world to see page globals — window.Hls, per-<video> Hls instances,
   video._goon_hls_manifest, and window.__goon_plugin_data (written by the
   main-world interceptors). The calls were missing it, so every main-world
   strategy (A/A2/E/H) found 0. It was also briefly placed inside `target`,
   which chrome.scripting rejects ("Unexpected property: 'world'"); `world` is
   a sibling of `target` on ScriptInjection. Restore it as a sibling of target.

2. fake "manifest" classification. Strategy H was reporting 10 nav links /
   server-side route TEMPLATES as manifests — e.g. /playlists/,
   /playlists/liked/, /playlist/%1%/video/%2%/, /members/%ID%/playlists/ —
   because the old /playlist|/manifest substring check matches nav menus and
   leaves %1%/%ID%/%DIR% template placeholders intact. Those then became the
   stored "video" entries, drowning out the real element.

   isManifestUrl and isValidManifestUrl now reject:
     - dynamic route templates (%NAME% or {name} tokens), using a negative
       lookahead so genuine %-encoded tokens like %2B/%2F are preserved
       (the invariant the codebase explicitly protects),
     - plural collection paths (/playlists, /manifests),
     - trailing-slash directory listings.

   The 'World: ISOLATED' probe log was also misleading (typeof chrome is not a
   world discriminator — the probe provably runs in MAIN); replaced it.

Adds a regression test (hls_qa) reproducing thisvid's pageContext.langUrls
scenario: all 10 templates are rejected while a real .m3u8 carrying a %2B token
is still detected. 90/90 tests pass; 0 lint errors; build OK.
2026-08-25 20:27:14 -07:00
kfj001 e3b904783f fix(proxy): stop per-tab referer wipe and concurrent DNR rule race
CI / test (pull_request) Successful in 28s
Address robot's review on PR #5:

- [Blocking] 'goon_referers' is a host-keyed GLOBAL store and must outlive
  its capture tab; it was wrongly added to the per-tab onRemoved cleanup, so
  closing any tab wiped referers captured by still-open source tabs (-> 412).
  It is no longer removed on tab close. Stale referers are now pruned
  opportunistically in persistReferer via a per-host timestamp + TTL.

- [Should-fix] the DNR referer rule used a single shared REFERER_RULE_ID with
  a non-atomic check+update, so concurrent proxy fetches for two hosts could
  interleave and leave the wrong host's referer live. Each host now gets its
  own stable rule id (refererRuleIdFor) and the install+record is serialized
  per host via an async lock (refererRuleLocks).

- [Coverage] add __tests__/referer_proxy.test.mjs exercising the new
  referer-capture / proxy-host / onRemoved paths: closing a tab preserves
  goon_referers, two hosts proxied concurrently keep distinct per-host rules,
  and same-host proxy is idempotent.
2026-08-25 15:35:24 -07:00
kfj001 66c621825f refactor: decouple from site-specific CDN references
CI / test (pull_request) Successful in 37s
The extension had hard-coupled itself to one provider's CDN throughout
background.js and its consumers: a host allowlist, a static
declarativeNetRequest rule that forced a fixed site origin as the Referer,
site-named message types/functions, and illustrative comments naming the
site. This commit removes every reference so the code is generic and
provider-agnostic (and no longer names a specific site for reviewers).

What changed:

  - background.js
      * Dropped the PHNCDN_HOSTS allowlist; video-request capture is now
        host-agnostic (any CDN, driven by isVideoRequest / isHlsManifest).
      * Removed the static DNR rule that forced a fixed site origin as the
        Referer. Replaced with per-tab dynamic capture: an
        onBeforeSendHeaders listener captures the provider's own Referer for
        each video request the live page makes and stores it per CDN host
        (goon_referers). The proxy handler installs a DNR rule scoped to the
        requested host using that captured referer, so the provider's exact
        origin is extracted at runtime instead of hard-coded.
      * Generalized the reconstruction helpers: extractVideoId ->
        extractGroupKey (path-directory key, no /hls/videos/ assumption),
        buildManifestForVariant / reconstructManifestsFromSegments swap a
        segment filename for master.m3u8 dir-agnostically, freshLiveForVideo
        -> freshLiveForGroup, and the GOON_GET_PHNCDN_REQUESTS handler ->
        GOON_GET_VIDEO_REQUESTS. Resolver response key videoId -> groupKey
      * Reworded prose comments to describe behavior generically.

  - manifest.json
      * Added the extraHeaders permission so onBeforeSendHeaders can read the
        Referer header (needed for the dynamic capture above).

  - viewer.js / index.js (consumers)
      * Proxying decision now keys off a learned goon_proxy_hosts set
        (populated from a startup CORS probe) instead of hard-coded host
        substrings; removed the hard-coded CORS ping.
      * filterManifests / isValidManifestUrl generalized (kept the
        .m3u8 / .hlsvod / .ts behavior the tests assert); rank key
        phncdn_reconstructed -> hls_reconstructed; queryPhncdnRequests ->
        queryVideoRequests.

  - content.js, interceptors.js, and two test-prose strings: the illustrative
    "sites like <site>" comments generalized.

Verification: 83/83 tests pass, 0 lint errors. A repo-wide grep for the
previously-referenced host / site now returns no matches. The pre-existing
tsc errors in vite.config.ts are unrelated (present on the prior commit).

Note on the dynamic referer: it works while the source tab is open (it reads
that tab's real Referer); if the tab closed before capturing, the proxy falls
back to a host-derived origin, which a strict origin check may still reject --
the tradeoff of not hard-coding the site.
2026-08-25 02:26:16 -07:00
10 changed files with 985 additions and 509 deletions

No files matched your search

+2 -2
View File
@@ -44,8 +44,8 @@ describe('Config: build integrity', () => {
if (codeOnly.includes('textContent')) { if (codeOnly.includes('textContent')) {
throw new Error( throw new Error(
'content.js uses inline script injection (textContent). ' + 'content.js uses inline script injection (textContent). ' +
'Sites like Pornhub block inline scripts via CSP. Use script.src instead.' 'Some sites block inline scripts via CSP. Use script.src instead.'
); );
} }
if (!codeOnly.includes('script.src') || !codeOnly.includes('chrome.runtime.getURL')) { if (!codeOnly.includes('script.src') || !codeOnly.includes('chrome.runtime.getURL')) {
+115 -21
View File
@@ -14,8 +14,8 @@ const viewerSource = fs.readFileSync(
); );
const interceptorSource = fs.readFileSync( const interceptorSource = fs.readFileSync(
path.join(__dirname, '../src/inject/interceptors.js'), path.join(__dirname, '../src/inject/interceptors.js'),
'utf-8' 'utf-8'
); );
/* ========================================================================== */ /* ========================================================================== */
/* Helpers */ /* Helpers */
@@ -940,8 +940,8 @@ describe('HLS QA: Configuration and resource management', () => {
it('should vendor HLS bundle locally (not use CDN — avoids CSP blocks)', () => { it('should vendor HLS bundle locally (not use CDN — avoids CSP blocks)', () => {
// Viewer loads hls.js from extension origin via chrome.runtime.getURL('hls.min.js') // Viewer loads hls.js from extension origin via chrome.runtime.getURL('hls.min.js')
// instead of a CDN — sites like Pornhub restrict script-src to 'self', blocking // instead of a CDN — sites that restrict script-src to 'self' block
// cross-origin <script src> injection from CDN URLs. // cross-origin <script src> injection from CDN URLs.
if (!viewerSource.includes("chrome.runtime.getURL('hlds.js')")) { if (!viewerSource.includes("chrome.runtime.getURL('hlds.js')")) {
throw new Error('viewer.js should load hlds.js from extension origin, not CDN'); throw new Error('viewer.js should load hlds.js from extension origin, not CDN');
} }
@@ -1027,21 +1027,48 @@ describe('HLS QA: iframe and multi-frame scenarios', () => {
} }
}); });
it('should scrape video elements from all frames', () => { it('should scrape video elements from all frames', () => {
const indexSource = fs.readFileSync( const indexSource = fs.readFileSync(
path.join(__dirname, '../src/viewer/index.js'), path.join(__dirname, '../src/viewer/index.js'),
'utf-8' 'utf-8'
); );
// Verify scrapeTab runs in all frames // Verify scrapeTab runs in all frames
const scrapeMatch = indexSource.match(/scrapeTab[\s\S]*?executeScript[\s\S]*?\}/); const scrapeMatch = indexSource.match(/scrapeTab[\s\S]*?executeScript[\s\S]*?\}/);
if (!scrapeMatch) { if (!scrapeMatch) {
throw new Error('scrapeTab function not found'); throw new Error('scrapeTab function not found');
} }
if (!scrapeMatch[0].includes('allFrames: true')) { if (!scrapeMatch[0].includes('allFrames: true')) {
throw new Error('scrapeTab must use allFrames: true'); throw new Error('scrapeTab must use allFrames: true');
} }
}); });
it('deep probe + scrape must run in the MAIN world (see page globals)', () => {
// The deep probe and scraper read page globals (window.Hls, player
// instances, video._goon_hls_manifest) and window.__goon_plugin_data,
// which the main-world interceptors write. The isolated world cannot see
// these, so a missing `world: 'MAIN'` makes every main-world strategy find
// 0 manifests — the "detects nothing on pages with just <video> elements"
// regression. Guard the exact executeScript calls that run each function.
const indexSource = fs.readFileSync(
path.join(__dirname, '../src/viewer/index.js'),
'utf-8'
);
// Capture each call's target + func block (world: 'MAIN' lives on the
// target: line, which precedes func:).
const probeMatch = indexSource.match(/executeScript\(\{[\s\S]*?func: m3u8DeepProbeFn/);
if (!probeMatch) throw new Error('deep probe executeScript call not found');
if (!probeMatch[0].includes("world: 'MAIN'")) {
throw new Error('deep probe (m3u8DeepProbeFn) must use world: "MAIN" to see page globals');
}
const scrapeMatch = indexSource.match(/executeScript\(\{[\s\S]*?func: scrapeFn/);
if (!scrapeMatch) throw new Error('scrape executeScript call not found');
if (!scrapeMatch[0].includes("world: 'MAIN'")) {
throw new Error('scrape (scrapeFn) must use world: "MAIN" to see page globals');
}
});
it('should inject interceptors into all frames (content_scripts.all_frames)', () => { it('should inject interceptors into all frames (content_scripts.all_frames)', () => {
const manifestSource = fs.readFileSync( const manifestSource = fs.readFileSync(
@@ -1211,6 +1238,73 @@ describe('HLS QA: Edge cases', () => {
if (intercepted.length !== 1) { if (intercepted.length !== 1) {
throw new Error(`Expected 1 intercept for /manifest URL, got ${intercepted.length}`); throw new Error(`Expected 1 intercept for /manifest URL, got ${intercepted.length}`);
} }
}); });
}); });
/* ========================================================================== */
/* Test: deep probe must not report route-template / nav "manifests" */
/* ========================================================================== */
describe('HLS QA: route-template / nav URL rejection', () => {
const probeMatch = indexSource.match(/function m3u8DeepProbeFn\(\)[\s\S]*?^}/m);
if (!probeMatch) throw new Error('Could not find m3u8DeepProbeFn in index.js');
const probeFnSource = probeMatch[0];
function runProbeWin(url, globals) {
const dom = new JSDOM(
'<!DOCTYPE html><html><body><video></video></body></html>',
{ url, runScripts: 'dangerously' });
const win = dom.window;
for (const [k, v] of Object.entries(globals)) win[k] = v;
const result = win.eval(`(${probeFnSource})();`);
if (!Array.isArray(result)) throw new Error('m3u8DeepProbeFn did not return an array');
return result;
}
// Reproduces thisvid.com: pageContext.langUrls holds server-side route
// TEMPLATES / nav links (e.g. /playlists/, /playlist/%1%/video/%2%/) that the
// old /playlist|/manifest substring check misclassified as "manifests".
const TEMPLATE_URLS = [
'https://thisvid.com/playlists/',
'https://thisvid.com/playlists/liked/',
'https://thisvid.com/playlists/popular/',
'https://thisvid.com/playlists/commented/',
'https://thisvid.com/playlists/updated/',
'https://thisvid.com/playlists/largest/',
'https://thisvid.com/playlist/%1%/video/%2%/',
'https://thisvid.com/playlists/?category=%DIR%',
'https://thisvid.com/members/%ID%/playlists/',
'https://thisvid.com/my/playlists/%ID%/',
];
it('route templates / nav links are NOT reported as manifests', () => {
const result = runProbeWin('https://thisvid.com/videos/redneck/', {
pageContext: { langUrls: TEMPLATE_URLS },
kvsplayer: { config: { langUrls: TEMPLATE_URLS } },
__goon_plugin_data: { manifests: [], sources: [] },
});
const urls = result.map(m => m.url);
const leaked = TEMPLATE_URLS.filter(u => urls.includes(u));
if (leaked.length > 0) {
throw new Error(`template/nav URLs leaked as manifests: ${leaked.join(', ')}`);
}
});
it('a real .m3u8 with a %2B token is still found (no over-rejection)', () => {
// %2B is a valid percent-encoded token, not a placeholder — it must survive.
const real = 'https://cdn.example.com/live/master.m3u8?h=a%2Bb%2Fc&e=1234';
const result = runProbeWin('https://thisvid.com/videos/redneck/', {
hls: { url: real },
pageContext: { langUrls: TEMPLATE_URLS },
});
if (!result.some(m => m.url === real)) {
throw new Error(`real manifest with %2B token was dropped: ${real} — got: ${result.map(m => m.url).join(', ')}`);
}
// And the templates still must not leak in the same run.
const leaked = TEMPLATE_URLS.filter(u => result.some(m => m.url === u));
if (leaked.length > 0) {
throw new Error(`templates leaked alongside real manifest: ${leaked.join(', ')}`);
}
});
});
+239
View File
@@ -0,0 +1,239 @@
/**
* referer_proxy.test.mjs — coverage for the background worker's per-CDN-host
* referer capture + DNR proxy-host learning, added in PR #5.
*
* The background worker is not itself a module, so the test transpiles it into
* one (wrapping + top-level-statement `async function` transpile) under a mocked
* `chrome` + `fetch`/`btoa` environment, then exercises the captured listeners
* and the message handler directly.
*
* Covers the three gaps robot flagged:
* (a) closing a tab must NOT wipe the host-keyed `goon_referers` global store,
* (b) two hosts proxied concurrently must each keep their OWN referer (no race
* that leaves host B's referer live for host A -> 412),
* (c) the new referer-capture / proxy-host-learning / onRemoved paths run.
*/
import { describe, it } from './runner.mjs';
import { readFileSync, writeFileSync } from 'node:fs';
import { fileURLToPath, pathToFileURL } from 'node:url';
import { dirname, join } from 'node:path';
import { tmpdir } from 'node:os';
const __dirname = dirname(fileURLToPath(import.meta.url));
const backgroundSource = readFileSync(join(__dirname, '../src/background/background.js'), 'utf8');
// Transpile the top-level-statement worker into an ES module. Wrapping in an
// async IIFE makes its top-level `async function` declarations + `await`-using
// nested functions legal inside a module.
function buildModule(code) {
return `(async () => {\n${code}\n})();\n`;
}
// A fresh mock of the chrome APIs the worker touches. `onBeforeSendHeaders`,
// `onBeforeRequest`, and `tabs.onRemoved` capture their listener so the test can
// drive them; `declarativeNetRequest.updateSessionRules` records the live rules.
function makeChromeEnv() {
const storage = {};
const handlers = { onMessage: [] };
let sessionRules = {};
const webRequest = {
onBeforeSendHeaders: { addListener: (cb) => { handlers.onBeforeSendHeaders = cb; } },
onBeforeRequest: { addListener: (cb) => { handlers.onBeforeRequest = cb; } },
};
const chrome = {
storage: {
local: {
get: async (keys) => {
const out = {};
const arr = Array.isArray(keys) ? keys : (keys === undefined ? Object.keys(storage) : [keys]);
for (const k of arr) if (k in storage) out[k] = storage[k];
return out;
},
set: async (data) => {
for (const [k, v] of Object.entries(data)) storage[k] = v;
},
remove: async (keys) => {
const arr = Array.isArray(keys) ? keys : [keys];
for (const k of arr) delete storage[k];
},
},
},
webRequest,
tabs: {
onRemoved: { addListener: (cb) => { handlers.onRemoved = cb; } },
},
runtime: { onMessage: { addListener: (cb) => { handlers.onMessage.push(cb); } } },
declarativeNetRequest: {
updateSessionRules: async ({ removeRuleIds = [], addRules = [] } = {}) => {
const rm = new Set(removeRuleIds);
sessionRules = Object.fromEntries(
Object.entries(sessionRules).filter(([id]) => !rm.has(id))
);
for (const r of addRules) sessionRules[r.id] = r;
},
},
};
return { chrome, handlers, storage, getSessionRules: () => sessionRules };
}
// Load the worker into a fresh environment. `data:` URLs are cached by Node, so
// the module body (listener registration) would only run once; instead we write
// a uniquely-named temp file per load so each call registers into its own env.
let loadCounter = 0;
async function loadBackground(env) {
global.chrome = env.chrome;
const file = join(tmpdir(), `goon_bg_${process.pid}_${loadCounter++}.mjs`);
writeFileSync(file, buildModule(backgroundSource));
const module = await import(pathToFileURL(file).href);
await module.default;
return module;
}
function videoDetails(tabId, url, referer) {
return {
tabId,
url,
requestHeaders: referer ? [{ name: 'Referer', value: referer }] : [],
};
}
describe('PR#5: per-host referer capture + proxy race', () => {
// (a) Blocking fix: a host-keyed `goon_referers` entry must outlive the close
// of ANY tab — including a different tab.
it('closing a tab does not wipe goon_referers captured for another host', async () => {
const env = makeChromeEnv();
global.chrome = env.chrome;
await loadBackground(env);
// A live source tab captures the provider referer for its CDN host.
const cap = env.handlers.onBeforeSendHeaders;
if (!cap) throw new Error('onBeforeSendHeaders listener was not registered');
await cap(videoDetails(11, 'https://cdnA.example.com/hls/live/seg-1.ts', 'https://pageA.example.com/watch'));
const rA = env.storage['goon_referers']?.['cdna.example.com'];
if (!rA) throw new Error('referer was not captured for cdnA');
if (rA.referer !== 'https://pageA.example.com/watch') {
throw new Error('captured referer wrong: ' + JSON.stringify(rA));
}
// A DIFFERENT tab closes. The per-tab cleanup must NOT remove goon_referers.
await env.handlers.onRemoved(22);
const after = env.storage['goon_referers'];
if (!after || !after['cdna.example.com']) {
throw new Error('closing tab 22 wiped goon_referers — host A referer lost');
}
if (after['cdna.example.com'].referer !== 'https://pageA.example.com/watch') {
throw new Error('closing tab 22 corrupted the host A referer: ' + JSON.stringify(after));
}
});
it('closing the capturing tab itself preserves the referer for later proxy use', async () => {
const env = makeChromeEnv();
global.chrome = env.chrome;
await loadBackground(env);
await env.handlers.onBeforeSendHeaders(
videoDetails(11, 'https://cdnB.example.com/video/x/master.m3u8', 'https://pageB.example.com/v')
);
// The very tab that captured the referer now closes.
await env.handlers.onRemoved(11);
const after = env.storage['goon_referers']?.['cdnb.example.com'];
if (!after) throw new Error('goon_referers for the capturing host was wiped on its own tab close');
if (after.referer !== 'https://pageB.example.com/v') {
throw new Error('capturing-tab close corrupted referer: ' + JSON.stringify(after));
}
});
// (b) Should-fix + coverage: two hosts proxied concurrently must each keep
// their own referer. With a shared rule id + non-atomic install this regressed;
// per-host rule ids + the per-host lock keep them distinct.
it('concurrent proxy fetches for two hosts keep distinct per-host referer rules', async () => {
const env = makeChromeEnv();
let fetchCalls = 0;
global.chrome = env.chrome;
global.fetch = async (url) => {
fetchCalls++;
return {
ok: true,
status: 200,
headers: { get: () => 'application/vnd.apple.mpegurl' },
arrayBuffer: async () => new Uint8Array([0]).buffer,
};
};
await loadBackground(env);
// Two GOON_PROXY_FETCH messages for different CDN hosts, fired together.
const handler = env.handlers.onMessage[env.handlers.onMessage.length - 1];
let respA, respB;
handler({ type: 'GOON_PROXY_FETCH', url: 'https://cdnA.example.com/hls/live/seg-1.ts', referer: 'https://pageA.example.com/watch' }, {}, (r) => { respA = r; });
handler({ type: 'GOON_PROXY_FETCH', url: 'https://cdnB.example.com/hls/live/seg-1.ts', referer: 'https://pageB.example.com/watch' }, {}, (r) => { respB = r; });
await new Promise(r => setTimeout(r, 30)); // let async sends + installs flush
if (!respA || !respA.ok) throw new Error('proxy A failed: ' + JSON.stringify(respA));
if (!respB || !respB.ok) throw new Error('proxy B failed: ' + JSON.stringify(respB));
if (fetchCalls < 2) throw new Error('expected both proxy fetches to run, got ' + fetchCalls);
// Each host must have its OWN installed DNR rule with its OWN referer —
// host A must NOT be served host B's referer (the 412 regression).
const rules = env.getSessionRules();
const ruleA = Object.values(rules).find(r => r.condition?.requestDomains?.includes('cdna.example.com'));
const ruleB = Object.values(rules).find(r => r.condition?.requestDomains?.includes('cdnb.example.com'));
if (!ruleA) throw new Error('no DNR rule installed for cdna; rules=' + JSON.stringify(rules));
if (!ruleB) throw new Error('no DNR rule installed for cdnb; rules=' + JSON.stringify(rules));
if (ruleA.id === ruleB.id) {
throw new Error('cdnA and cdnB shared a DNR rule id -> concurrent race possible');
}
const refA = ruleA.action.requestHeaders?.find(h => h.header.toLowerCase() === 'referer')?.value;
const refB = ruleB.action.requestHeaders?.find(h => h.header.toLowerCase() === 'referer')?.value;
if (refA !== 'https://pageA.example.com/watch') {
throw new Error('cdnA served wrong referer: ' + refA);
}
if (refB !== 'https://pageB.example.com/watch') {
throw new Error('cdnB served wrong referer: ' + refB);
}
if (refA === refB) throw new Error('both hosts got the same referer — cross-host bleed');
});
it('repeated proxy for the same host is idempotent (one rule, one install)', async () => {
const env = makeChromeEnv();
let installs = 0;
env.chrome.declarativeNetRequest.updateSessionRules = async ({ addRules = [] } = {}) => {
for (const r of addRules) installs++;
};
global.chrome = env.chrome;
global.fetch = async () => ({
ok: true, status: 200,
headers: { get: () => 'application/vnd.apple.mpegurl' },
arrayBuffer: async () => new Uint8Array([0]).buffer,
});
await loadBackground(env);
const handler = env.handlers.onMessage[env.handlers.onMessage.length - 1];
const run = async (n) => {
handler(
{ type: 'GOON_PROXY_FETCH', url: 'https://cdnA.example.com/hls/live/seg-1.ts', referer: 'https://pageA.example.com/watch' },
{},
() => {}
);
await new Promise(r => setTimeout(r, 5));
};
await run(1);
const installsAfterFirst = installs;
await run(2);
await run(3);
// Same host + same referer should not re-add the rule on repeat fetches.
if (installs !== installsAfterFirst) {
throw new Error(`expected idempotent install, but re-added rule ${installs - installsAfterFirst} extra time(s)`);
}
});
});
+1
View File
@@ -7,6 +7,7 @@ import './data_flow.test.mjs';
import './viewer_integration.test.mjs'; import './viewer_integration.test.mjs';
import './config_validation.test.mjs'; import './config_validation.test.mjs';
import './hls_qa.test.mjs'; import './hls_qa.test.mjs';
import './referer_proxy.test.mjs';
import { run } from './runner.mjs'; import { run } from './runner.mjs';
const result = await run(); const result = await run();
+8 -7
View File
@@ -3,13 +3,14 @@
"name": "VidPlex", "name": "VidPlex",
"version": "0.1.0", "version": "0.1.0",
"description": "Find videos on any page and view them in a grid.", "description": "Find videos on any page and view them in a grid.",
"permissions": [ "permissions": [
"storage", "storage",
"tabs", "tabs",
"scripting", "scripting",
"webRequest", "webRequest",
"declarativeNetRequest" "declarativeNetRequest",
], "extraHeaders"
],
"host_permissions": [ "host_permissions": [
"<all_urls>" "<all_urls>"
], ],
+361 -246
View File
@@ -23,13 +23,12 @@ function isHlsManifest(url) {
u.includes('/playlist.m3u') || u.includes('/manifest'); u.includes('/playlist.m3u') || u.includes('/manifest');
} }
// === STRATEGY 8: Network-level fallback — log ALL phncdn.com and video-related requests === // === STRATEGY 8: Network-level fallback — log video requests across any CDN ===
// Pornhub's es6player generates HLS manifests in-memory with zero .m3u8 network requests. // Some players generate HLS manifests in-memory and make zero .m3u8 network
// However, the player still fetches video segments from phncdn.com. By logging all phncdn // requests, only fetching video segments from a CDN. By logging every video-like
// requests we can: (a) confirm HLS activity via segment patterns, (b) reconstruct manifest // request (any host) we can: (a) confirm HLS activity via segment patterns,
// URLs from segment paths, (c) detect CDN response headers that reveal playlist endpoints. // (b) reconstruct manifest URLs from segment paths, (c) detect CDN response
// headers that reveal playlist endpoints. No site-specific host list.
const PHNCDN_HOSTS = ['phncdn.com', 'pornhubpremium.com', 'cdn.pornhub.com'];
const VIDEO_PATH_PATTERNS = ['/hls/', '/dash/', '/video/', '/segment', '/media/', '/stream']; const VIDEO_PATH_PATTERNS = ['/hls/', '/dash/', '/video/', '/segment', '/media/', '/stream'];
function isVideoRequest(url) { function isVideoRequest(url) {
@@ -47,7 +46,7 @@ async function persistVideoRequest(tabId, url) {
try { try {
const stored = await chrome.storage.local.get(key); const stored = await chrome.storage.local.get(key);
const arr = stored[key] || []; const arr = stored[key] || [];
// Avoid duplicates — phncdn segments come in rapid succession // Avoid duplicates — video segments come in rapid succession
if (!arr.some(r => r === url)) { if (!arr.some(r => r === url)) {
arr.push(url); arr.push(url);
// Keep last 200 requests per tab (enough to detect segment patterns) // Keep last 200 requests per tab (enough to detect segment patterns)
@@ -59,26 +58,28 @@ async function persistVideoRequest(tabId, url) {
} }
} }
// --- phncdn token / URL helpers ------------------------------------------ // --- Token / URL helpers ------------------------------------------
// //
// The phncdn auth token (h= / hash= / hdnea=) is base64 and its %-encoded // A media CDN's auth token (h= / hash= / hdnea=) is base64 and its %-encoded
// escapes are significant: a `+` decoded to a space corrupts it, which is what // escapes are significant: a `+` decoded to a space corrupts it, which is what
// produces the 410 Gone the live player avoids by sending `%2B`. So every // produces the 410 Gone the live player avoids by sending `%2B`. So every
// helper below keeps the query string RAW — it is never passed through // helper below keeps the query string RAW — it is never passed through
// URLSearchParams (which decodes + -> space). // URLSearchParams (which decodes + -> space).
// Extract the phncdn videoId from a /hls/videos/ URL; '' if it isn't one. // Group key for a video's URL: the path directory holding its variant/segment
// Path shape: /hls/videos/{year}/{subdir}/{videoId}/{variant}/... // (the last non-file path segment of a CDN URL). Used to match the freshest
function extractVideoId(url) { // live URL by video, host-independent. '' when the URL is malformed.
if (!url || !url.includes('/hls/videos/')) return ''; function extractGroupKey(url) {
if (!url) return '';
try { try {
const path = new URL(url).pathname.split('/'); const path = new URL(url).pathname.split('/');
const hlsIdx = path.findIndex(p => p === 'hls'); // Drop the final (file/segment or manifest) segment; the remaining
if (hlsIdx === -1 || hlsIdx + 4 >= path.length) return ''; // directory joins into a key that is stable across a video's variants.
return path[hlsIdx + 4]; path.pop();
} catch { return path.join('/');
} catch {
return ''; return '';
} }
} }
// Raw query (everything after '?') with no decoding. '' when absent. // Raw query (everything after '?') with no decoding. '' when absent.
@@ -99,31 +100,24 @@ function tokenExpirySeconds(rawQuery) {
return secs; return secs;
} }
// Build a manifest URL for a variant directory from a raw phncdn URL, // Build the parent manifest URL for a segment URL, keeping the FULL path to
// preserving the FULL variant path and the raw query (token encoding intact). // the variant directory and the RAW query (token encoding intact). Swaps the
// /hls/videos/{year}/{subid}/{videoId}/{variant}/seg-1.ts -> // segment filename for a manifest filename in the same directory:
// /hls/videos/{year}/{subid}/{videoId}/{variant}/master.m3u8 // {host}/path/to/variant/seg-1.ts -> {host}/path/to/variant/master.m3u8
// The variant directory (e.g. 720P_4000K_338337051.mp4) is MANDATORY — a master // The variant directory is MANDATORY — a master playlist lives inside it;
// playlist lives inside it; dropping it (the old bug) yields a 410. Returns '' // dropping it yields a 404/410. Returns '' for non-segment URLs.
// when the URL isn't a phncdn HLS URL with a complete variant path. function buildManifestForVariant(url) {
function buildManifestForVariant(rawUrl) { if (!url || !/\/seg-\d+/i.test(url)) return '';
if (!rawUrl || !rawUrl.includes('/hls/videos/')) return '';
try { try {
const path = rawUrl.split('?')[0]; const [pathPart, queryPart] = url.split('?');
const parts = path.split('/'); const idx = pathPart.lastIndexOf('/');
const hlsIdx = parts.findIndex(p => p === 'hls'); if (idx === -1) return '';
// Need hls, videos, year, subdir, videoId, variant (6 dirs). const dir = pathPart.slice(0, idx); // variant directory
if (hlsIdx === -1 || hlsIdx + 5 >= parts.length) return ''; const rawQuery = queryPart ? '?' + queryPart : '';
const prefix = parts.slice(0, hlsIdx + 2).join('/'); // .../hls/videos return `${dir}/master.m3u8${rawQuery}`;
const year = parts[hlsIdx + 2]; } catch {
const subid = parts[hlsIdx + 3];
const videoId = parts[hlsIdx + 4];
const variant = parts[hlsIdx + 5];
const rawQuery = extractRawQuery(rawUrl);
return `${prefix}/${year}/${subid}/${videoId}/${variant}/master.m3u8${rawQuery ? '?' + rawQuery : ''}`;
} catch {
return ''; return '';
} }
} }
// Reconstruct manifest URLs from segment URLs, keeping the raw query intact. // Reconstruct manifest URLs from segment URLs, keeping the raw query intact.
@@ -133,97 +127,100 @@ function reconstructManifestsFromSegments(segments) {
const seen = new Set(); const seen = new Set();
const manifests = []; const manifests = [];
for (const url of segments) { for (const url of segments) {
// Only handle HLS segment URLs: /hls/videos/{..}/{variant}/seg-N.ts // Only handle video/segment URLs (e.g. .../seg-N.ts).
if (!url.includes('/hls/videos/') || !/\/seg-\d+/.test(url)) continue;
const manifestUrl = buildManifestForVariant(url); const manifestUrl = buildManifestForVariant(url);
if (manifestUrl && !seen.has(manifestUrl)) { if (manifestUrl && !seen.has(manifestUrl)) {
seen.add(manifestUrl); seen.add(manifestUrl);
manifests.push(manifestUrl); manifests.push(manifestUrl);
}
} }
}
return manifests; return manifests;
} }
// Collect phncdn URLs across ALL tabs without a tabs.query match pattern (which // Collect captured video URLs across ALL tabs without a tabs.query match pattern
// throws if the pattern is malformed). We scan every persisted vr_requests_ / // (which throws if the pattern is malformed). We scan every persisted vr_requests_ /
// wr_manifests_ key so the rescan works even if the live source tab is closed. // wr_manifests_ key so the rescan works even if the live source tab is closed.
async function collectAllPhncdnUrls() { async function collectAllVideoUrls() {
const all = await chrome.storage.local.get(); const all = await chrome.storage.local.get();
const out = []; const out = [];
for (const key of Object.keys(all)) { for (const key of Object.keys(all)) {
if ((key.startsWith('vr_requests_') || key.startsWith('wr_manifests_')) && Array.isArray(all[key])) { if ((key.startsWith('vr_requests_') || key.startsWith('wr_manifests_')) && Array.isArray(all[key])) {
out.push(...all[key]); out.push(...all[key]);
} }
} }
return out; return out;
} }
// For a videoId, return the freshest live phncdn URL (highest token expiry) // For a group key, return the freshest live captured URL (highest token
// across all persisted tabs, preferring captured .m3u8 manifest URLs. null if // expiry) across all persisted tabs, preferring captured .m3u8 manifest URLs.
// nothing usable is captured yet. Rescans use this to replace a stored URL that // null if nothing usable is captured yet. Rescans use this to replace a
// now 410s with the newest token the live tab captured. // stored URL that now 410s with the newest token the live tab captured.
// //
// No tabs.query match pattern is used: '*pornhub.com*' is an INVALID pattern and // No tabs.query match pattern is used: match patterns are brittle (a bad
// made this throw ("Invalid url pattern"), which silently dropped every rescan. // pattern throws "Invalid url pattern" and silently drops every rescan).
// We scan persisted capture data instead, which is always available and // We scan persisted capture data instead, which is always available and
// sidesteps the match-Pattern API entirely. // sidesteps the match-pattern API entirely.
async function freshLiveForVideo(videoId, preferManifest) { async function freshLiveForGroup(groupKey, preferManifest) {
let all; let all;
try { try {
all = await collectAllPhncdnUrls(); all = await collectAllVideoUrls();
} catch (e) { } catch (e) {
console.warn('[goon-bg] collectAllPhncdnUrls failed:', e && e.message ? e.message : e); console.warn('[goon-bg] collectAllVideoUrls failed:', e && e.message ? e.message : e);
all = []; all = [];
} }
let bestManifest = null, bestManifestExp = 0; let bestManifest = null, bestManifestExp = 0;
let bestVariant = null, bestVariantExp = 0; let bestVariant = null, bestVariantExp = 0;
let bestTokenSource = null, bestTokenExp = 0; let bestTokenSource = null, bestTokenExp = 0;
for (const url of all) { for (const url of all) {
if (extractVideoId(url) !== videoId) continue; if (extractGroupKey(url) !== groupKey) continue;
const exp = tokenExpirySeconds(extractRawQuery(url)); const exp = tokenExpirySeconds(extractRawQuery(url));
if (url.includes('.m3u8')) { if (url.includes('.m3u8')) {
if (exp > bestManifestExp) { bestManifestExp = exp; bestManifest = url; } if (exp > bestManifestExp) { bestManifestExp = exp; bestManifest = url; }
} else if (/\/seg-\d+/.test(url)) { } else if (/\/seg-\d+/i.test(url)) {
if (exp > bestVariantExp) { bestVariantExp = exp; bestVariant = url; } if (exp > bestVariantExp) { bestVariantExp = exp; bestVariant = url; }
} else { } else {
if (exp > bestTokenExp) { bestTokenExp = exp; bestTokenSource = url; } if (exp > bestTokenExp) { bestTokenExp = exp; bestTokenSource = url; }
}
} }
}
// Prefer a captured manifest; else reconstruct the variant manifest; else // Prefer a captured manifest; else the best segment (reconstructable to a
// hand back a token source so the proxy can at least re-token a good path. // manifest); else a token source so the proxy can re-token a good path.
if (bestManifest) return bestManifest; if (bestManifest) return bestManifest;
if (!preferManifest && bestVariant) return bestVariant; if (!preferManifest && bestVariant) return bestVariant;
if (!preferManifest && bestTokenSource) return bestTokenSource; if (!preferManifest && bestTokenSource) return bestTokenSource;
if (bestVariant) return bestVariant; if (bestVariant) return bestVariant;
return bestManifest || bestTokenSource || null; return bestManifest || bestTokenSource || null;
} }
// Cache freshest tokens per videoId so we don't query storage on every proxy // Cache freshest tokens per group key so we don't query storage on every proxy
// request. Keyed by videoId; values resolve once and are reused until a rescan // request. Keyed by group key; values resolve once and are reused until a rescan
// (GOON_RESOLVE_MANIFEST) or a 410 forces an invalidation. // (GOON_RESOLVE_MANIFEST) or a 410 forces an invalidation.
const tokenCache = new Map(); const tokenCache = new Map();
// Given a phncdn /hls/videos/ URL that arrives WITHOUT a query (hls.js drops the // Given a CDN URL that arrives WITHOUT a query (hls.js drops the relative-URL
// relative-URL token when building sub-playslists/segments), inject the freshest // token when building sub-playslists/segments), inject the freshest live token
// live token for that videoId. Returns the URL with a query appended, or the // for that group. Returns the URL with a query appended, or the original URL
// original URL unchanged when no token is available yet. // unchanged when no token is available yet.
async function injectFreshToken(url) { async function injectFreshToken(url) {
if (!url || !url.includes('/hls/videos/')) return url; if (!url) return url;
// Already carries a token — leave it alone. // Already carries a token — leave it alone.
if (url.includes('?')) return url; if (url.includes('?')) return url;
const videoId = extractVideoId(url); try {
if (!videoId) return url; const groupKey = extractGroupKey(url);
if (tokenCache.has(videoId)) { if (!groupKey) return url;
const token = tokenCache.get(videoId); if (tokenCache.has(groupKey)) {
const token = tokenCache.get(groupKey);
return token ? url + '?' + token : url;
}
const fresh = await freshLiveForGroup(groupKey, false);
const token = fresh ? extractRawQuery(fresh) : '';
tokenCache.set(groupKey, token);
console.log('[goon-bg] injectFreshToken for', groupKey, (token ? token.substring(0, 40) + '…' : '(none captured)'));
return token ? url + '?' + token : url; return token ? url + '?' + token : url;
} } catch {
const fresh = await freshLiveForVideo(videoId, false); return url;
const token = fresh ? extractRawQuery(fresh) : ''; }
tokenCache.set(videoId, token);
console.log('[goon-bg] injectFreshToken for', videoId, videoId + '-> ' + (token ? token.substring(0, 40) + '…' : '(none captured)'));
return token ? url + '?' + token : url;
} }
// Clear the per-video token cache. Called on rescan so the next proxy fetches // Clear the per-video token cache. Called on rescan so the next proxy fetches
@@ -234,23 +231,14 @@ chrome.webRequest.onBeforeRequest.addListener(
async (details) => { async (details) => {
if (details.tabId < 0) return; if (details.tabId < 0) return;
const u = details.url.toLowerCase(); // Log video-like requests from any host (CDN-agnostic).
const isPhncdn = PHNCDN_HOSTS.some(host => details.url.includes(host)); if (isVideoRequest(details.url)) {
const isVideo = isVideoRequest(details.url); console.log('[goon-bg] webRequest: video request in tab', details.tabId, ':', details.url.substring(0, 180));
// Log ALL phncdn.com requests regardless of path
if (isPhncdn) {
console.log('[goon-bg] webRequest: phncdn request in tab', details.tabId, ':', details.url.substring(0, 180));
await persistVideoRequest(details.tabId, details.url); await persistVideoRequest(details.tabId, details.url);
} }
// Log video-like requests from any host // Also capture HLS manifest URLs (original logic)
if (isVideo && !isPhncdn) { if (!isHlsManifest(details.url)) return;
console.log('[goon-bg] webRequest: video request in tab', details.tabId, ':', details.url.substring(0, 150));
}
// Also capture HLS manifest URLs (original logic)
if (!isHlsManifest(details.url)) return;
console.log('[goon-bg] webRequest: HLS manifest in tab', details.tabId, ':', details.url.substring(0, 120)); console.log('[goon-bg] webRequest: HLS manifest in tab', details.tabId, ':', details.url.substring(0, 120));
@@ -269,76 +257,197 @@ chrome.webRequest.onBeforeRequest.addListener(
{ urls: ['<all_urls>'] } { urls: ['<all_urls>'] }
); );
// Clean up stale entries when tabs are closed // Clean up stale PER-TAB entries when tabs are closed. NOTE: 'goon_referers' is
// a host-keyed GLOBAL store — it must outlive its capture tab so a proxy fetch
// for that host still finds its referer after the source tab is closed. It is
// therefore NOT removed here (only the per-tab vr_requests_/wr_manifests_ keys
// are). Stale referers are pruned opportunistically in persistReferer.
chrome.tabs.onRemoved.addListener(async (tabId) => { chrome.tabs.onRemoved.addListener(async (tabId) => {
const keys = ['wr_manifests_' + tabId, 'vr_requests_' + tabId]; const keys = ['wr_manifests_' + tabId, 'vr_requests_' + tabId];
try { try {
await chrome.storage.local.remove(keys); await chrome.storage.local.remove(keys);
} catch { /* ignore */ } } catch { /* ignore */ }
});
// === Force a pornhub Referer on every *.phncdn.com request ===
// The phncdn video CDN (hv-h.phncdn.com, pix-fl.phncdn.com, ev-h.phncdn.com,
// ... and the auth-token hosts) rejects any request whose Referer is not a
// pornhub-origin with HTTP 412 (see round-1 curl test: `Referer:
// https://www.pornhub.com/` -> 200, no referer -> 412). Origin is NOT the
// gate (curl with a pornhub Referer and no Origin still 200s), so we only
// rewrite Referer and never touch Origin.
//
// A normal fetch() CANNOT set the `Referer` header — it is a "forbidden
// header" in the Fetch API, so a service-worker fetch() gets
// `Referer: chrome-extension://<id>`, which the CDN rejects (412). The
// declarativeNetRequest API rewrites request headers at the network layer,
// below the Fetch API's forbidden-header restriction, and — per Chrome docs —
// these rules DO apply to fetch() calls made inside the service worker. So we
// force a pornhub referer on phncdn.com; the GOON_PROXY_FETCH handler in the
// SW then fetches the manifest + segments successfully.
const REFERER_RULE_ID = 9001;
const REFERER_RULE = {
id: REFERER_RULE_ID,
priority: 1,
action: {
type: 'modifyHeaders',
// `set` is allowed for any request header (only `append` is restricted to
// a fixed allowlist). `referer` is the header the CDN checks.
requestHeaders: [
{ header: 'referer', operation: 'set', value: 'https://www.pornhub.com/' },
],
},
condition: {
// phncdn.com (host + subdomains) ONLY. Do NOT include pornhub.com: the
// real page player's auth calls (api.pornhub.com) and its own segment
// fetches must keep their browser-supplied Origin/Referer or playback
// freezes. The proxy only ever fetches phncdn URLs.
requestDomains: ['phncdn.com'],
},
};
// Cached promise so concurrent proxy fetches share one rule install; a
// transient failure is cleared so a later fetch retries.
let refererRuleReady = null;
async function ensureRefererRule() {
if (refererRuleReady) return refererRuleReady;
refererRuleReady = (async () => {
const api = chrome.declarativeNetRequest;
if (!api || typeof api.updateSessionRules !== 'function') {
throw new Error('declarativeNetRequest.updateSessionRules unavailable');
}
// Idempotent: remove any stale copy of our rule, then add the fresh one.
await api.updateSessionRules({
removeRuleIds: [REFERER_RULE_ID],
addRules: [REFERER_RULE],
}); });
console.log('[goon-bg] DNR referer rule installed for *.phncdn.com');
})(); // === Capture the provider-origin Referer per CDN host (CDN-agnostic) ===
// If it failed, reset so the next call retries instead of caching the // Some media CDNs reject requests whose `Referer` is not the provider page's
// rejected promise forever. // origin (HTTP 412). A fetch() CANNOT set `Referer` — it is a "forbidden
refererRuleReady.catch(() => { refererRuleReady = null; }); // header" in the Fetch API, so a service-worker fetch() would send
return refererRuleReady; // `Referer: chrome-extension://<id>`, which the CDN rejects. The
// declarativeNetRequest API rewrites request headers at the network layer, below
// the Fetch API's restriction, and its rules apply to fetch() calls made inside
// the SW.
//
// We cannot hardcode a target referer: the proxy is invoked from the viewer
// (a different origin than the source page), and a DNR rule cannot read a live
// page's origin. So we capture the referer that the REAL page uses for its video
// requests — via chrome.webRequest.onBeforeSendHeaders, which sees the request
// headers the browser already built — and persist it per CDN host. The
// GOON_PROXY_FETCH handler then installs a DNR rule that re-inserts that
// captured referer for just that host, so the proxy's fetch() is accepted.
// This requires no knowledge of any particular site.
//
// Each CDN host gets its OWN declarativeNetRequest rule id so that concurrent
// proxy fetches for different hosts can't clobber each other's rule (a single
// shared id + a remove-then-add lets two hosts interleave and leave the wrong
// host's referer live). The id is derived from the host via a monotonic counter
// keyed by host, so it is stable for a given host across installs.
let nextRefererRuleId = 9001;
const refererRuleIds = new Map();
function refererRuleIdFor(host) {
const existing = refererRuleIds.get(host);
if (existing !== undefined) return existing;
const id = nextRefererRuleId++;
refererRuleIds.set(host, id);
return id;
}
// host of a URL (CDN host, lowercased); '' when the URL is malformed.
function hostOf(url) {
if (!url) return '';
try { return new URL(url).host.toLowerCase(); } catch { return ''; }
}
// Best-effort origin for a host when no referer was captured for it. Used only
// as a fallback; a captured referer always wins.
function originFromHost(host) {
return host ? 'https://' + host + '/' : '';
}
// Read the `Referer` request header (the browser may title-case it; the
// `extraHeaders` option exposes it to MV3 onBeforeSendHeaders).
function readReferer(headers) {
if (!Array.isArray(headers)) return '';
for (const h of headers) {
if (h && h.name && h.name.toLowerCase() === 'referer') return h.value || '';
}
return '';
}
// How long a captured referer is trusted before it is pruned on the next
// persist. Referers are short-lived session tokens; this keeps the per-host
// global store from accumulating stale entries without tying cleanup to tab
// close (a still-open source tab's referer must survive other tabs closing).
const REFERER_TTL_MS = 30 * 60 * 1000;
// Persist the captured referer per CDN host so the proxy can re-use it even
// after this capture tab closes. Keyed by host so a proxy for that host can
// look its referer up without knowing the source tab. Each entry is stamped
// with a capture time; entries older than the TTL are pruned opportunistically
// here (never on tab close).
async function persistReferer(host, referer) {
if (!host || !referer) return;
try {
const stored = await chrome.storage.local.get('goon_referers');
const map = stored['goon_referers'] || {};
const now = Date.now();
// Opportunistic per-host prune: drop entries older than the TTL.
for (const [h, entry] of Object.entries(map)) {
if (h !== host && (!entry || !entry.at || now - entry.at > REFERER_TTL_MS)) {
delete map[h];
}
}
const current = map[host];
if (current && current.referer === referer) return;
map[host] = { referer, at: now };
await chrome.storage.local.set({ goon_referers: map });
} catch (e) {
console.warn('[goon-bg] persistReferer failed:', e && e.message ? e.message : e);
}
}
// Look up the referer captured for a host. '' when none has been captured
// (or the stored entry is malformed).
async function getReferer(host) {
if (!host) return '';
try {
const stored = await chrome.storage.local.get('goon_referers');
const entry = (stored['goon_referers'] || {})[host];
return entry && entry.referer ? entry.referer : '';
} catch {
return '';
}
}
// Capture the referer the real page sends for its video requests.
chrome.webRequest.onBeforeSendHeaders.addListener(
async (details) => {
if (details.tabId < 0) return;
if (!isVideoRequest(details.url)) return;
const host = hostOf(details.url);
if (!host) return;
const referer = readReferer(details.requestHeaders);
if (referer) {
// The browser already built a referer from the source page — use it.
await persistReferer(host, referer);
} else {
// No referer was sent; assume the page's origin is the expected referer.
await persistReferer(host, originFromHost(host));
}
},
{ urls: ['<all_urls>'], extraHeaders: ['Referer'] }
);
// Install (idempotently) a DNR rule that sets the captured referer for a given
// CDN host. The value comes from capture (or a fallback origin), never a
// hardcoded site. Scoped to the requested host so only proxy fetches for that
// host get the rewritten header.
//
// Each host gets its own rule id (refererRuleIdFor) so that concurrent proxy
// fetches for DIFFERENT hosts never interleave: host A's rule and host B's rule
// are distinct DNR ids and one install can't remove the other. Within a single
// host, a per-host async lock (refererRuleLocks) serializes the idempotency
// check + updateSessionRules so the install-and-record pair is atomic — no two
// installs for the same host can interleave their check/update.
const refererRuleInstalled = new Map();
const refererRuleLocks = new Map();
async function ensureRefererRule(host, referer) {
if (!host || !referer) return;
const api = chrome.declarativeNetRequest;
if (!api || typeof api.updateSessionRules !== 'function') {
console.warn('[goon-bg] declarativeNetRequest unavailable; skipping referer rule');
return;
} }
// Install eagerly on SW start so even the first proxy fetch is covered. // Serialize per host so the idempotency check + updateSessionRules is atomic.
ensureRefererRule(); const run = (async () => {
if (refererRuleInstalled.has(host) && refererRuleInstalled.get(host) === referer) {
return;
}
const ruleId = refererRuleIdFor(host);
try {
// Remove only THIS host's own rule id, then add it back scoped to the host.
await api.updateSessionRules({
removeRuleIds: [ruleId],
addRules: [{
id: ruleId,
priority: 1,
action: {
type: 'modifyHeaders',
// `set` is allowed for any request header (only `append` is restricted
// to a fixed allowlist). `referer` is the header the CDN checks.
requestHeaders: [
{ header: 'referer', operation: 'set', value: referer },
],
},
condition: {
// Host + subdomains ONLY, so only proxy fetches for this host get the
// rewritten referer; the source page's own requests keep theirs.
requestDomains: [host],
},
}],
});
refererRuleInstalled.set(host, referer);
console.log('[goon-bg] DNR referer rule installed for', host, '(' + ruleId + ')');
} catch (e) {
console.warn('[goon-bg] ensureRefererRule failed:', e && e.message ? e.message : e);
}
})().finally(() => { refererRuleLocks.set(host, null); });
refererRuleLocks.set(host, run);
return run;
}
// === Message handlers for popup/data consumers === // === Message handlers for popup/data consumers ===
@@ -362,11 +471,11 @@ chrome.runtime.onMessage.addListener((message, _sender, sendResponse) => {
return true; return true;
} }
case 'GOON_GET_PHNCDN_REQUESTS': { case 'GOON_GET_VIDEO_REQUESTS': {
// Strategy 8: Return all phncdn.com requests for a tab, including reconstructed manifests // Strategy 8: Return all video requests for a tab, including reconstructed manifests
(async () => { (async () => {
const tabId = message.tabId; const tabId = message.tabId;
const [segKey] = ['vr_requests_' + tabId]; const [segKey] = ['vr_requests_' + tabId];
try { try {
const stored = await chrome.storage.local.get(segKey); const stored = await chrome.storage.local.get(segKey);
const segments = stored[segKey] || []; const segments = stored[segKey] || [];
@@ -384,76 +493,82 @@ chrome.runtime.onMessage.addListener((message, _sender, sendResponse) => {
return true; return true;
} }
case 'GOON_RESOLVE_MANIFEST': { case 'GOON_RESOLVE_MANIFEST': {
// Resolve the freshest live phncdn manifest for the video in a stored // Resolve the freshest live manifest for the video encoded in a stored
// URL. The viewer calls this at startup and on a fatal network error: // URL. The viewer calls this at startup and on a fatal network error:
// the live tab keeps rotating short-lived tokens, so the freshest // the live tab keeps rotating short-lived tokens, so the freshest
// captured manifest (highest expiry) replaces one that has 410'd. // captured manifest (highest expiry) replaces one that has 410'd.
// Clears the token cache first so subsequent proxy fetches re-query for // Clears the token cache first so subsequent proxy fetches re-query for
// the freshly-rotated token rather than the cached (now-dead) one. // the freshly-rotated token rather than the cached (now-dead) one.
async function resolveFreshManifest(message, sendResponse) { async function resolveFreshManifest(message, sendResponse) {
invalidateTokenCache(); invalidateTokenCache();
const url = message.url || ''; const url = message.url || '';
const videoId = extractVideoId(url); const groupKey = extractGroupKey(url);
if (!videoId) { sendResponse({ ok: false, error: 'no videoId' }); return; } if (!groupKey) { sendResponse({ ok: false, error: 'no group key' }); return; }
try { try {
const fresh = await freshLiveForVideo(videoId, true); const fresh = await freshLiveForGroup(groupKey, true);
sendResponse({ ok: true, fresh, videoId, changed: !!(fresh && fresh !== url) }); sendResponse({ ok: true, fresh, groupKey, changed: !!(fresh && fresh !== url) });
} catch (e) { } catch (e) {
console.warn('[goon-bg] resolveFreshManifest failed:', e && e.message ? e.message : e); console.warn('[goon-bg] resolveFreshManifest failed:', e && e.message ? e.message : e);
sendResponse({ ok: false, error: e && e.message ? e.message : String(e), videoId }); sendResponse({ ok: false, error: e && e.message ? e.message : String(e), groupKey });
}
}
resolveFreshManifest(message, sendResponse);
return true;
}
case 'GOON_PROXY_FETCH': {
// Proxy a fetch request through the service worker (bypasses CORS).
// Returns base64-encoded response data. Used by the viewer when
// extension-origin requests to video CDNs are blocked by CORS.
//
// The phncdn CDN 412s requests without a pornhub Referer. A fetch()
// can't set Referer itself (forbidden header — Chrome replaces it with
// chrome-extension://...), so we install a declarativeNetRequest rule
// that forces `Referer: https://www.pornhub.com/` at the network layer
// for all *.phncdn.com requests (DNR applies to SW fetch() per Chrome
// docs). Await it first so the very next fetch is covered.
(async () => {
try {
await ensureRefererRule();
// hls.js resolves sub-playslists/segments relative to the master
// and can DROP the token query, handing us a token-less phncdn
// URL that 410s. injectFreshToken supplies the freshest live token
// for that videoId; the final `+` -> `%2B` pass keeps the
// (possibly injected or surviving) base64 token intact for the
// CDN's query parsing.
const resolved = await injectFreshToken(message.url || '');
const proxyUrl = resolved.replace(/\+/g, '%2B');
const resp = await fetch(proxyUrl);
if (!resp.ok) {
console.warn('[goon-bg] proxyFetch', resp.status, message.url.substring(0, 120));
sendResponse({ ok: false, status: resp.status, error: 'HTTP ' + resp.status });
return;
}
const buffer = await resp.arrayBuffer();
const bytes = new Uint8Array(buffer);
// Convert to base64 (btoa can only handle strings up to ~65k chars)
let binary = '';
const CHUNK_SIZE = 32768;
for (let i = 0; i < bytes.length; i += CHUNK_SIZE) {
const chunk = bytes.subarray(i, i + CHUNK_SIZE);
binary += String.fromCharCode.apply(null, Array.from(chunk));
}
const base64 = btoa(binary);
sendResponse({ ok: true, data: base64, mimeType: resp.headers.get('content-type') || '' });
} catch(e) {
console.warn('[goon-bg] proxyFetch failed:', e && e.message ? e.message : e);
sendResponse({ ok: false, error: e && e.message ? e.message : String(e) });
} }
})(); }
return true; resolveFreshManifest(message, sendResponse);
} return true;
}
case 'GOON_PROXY_FETCH': {
// Proxy a fetch request through the service worker (bypasses CORS).
// Returns base64-encoded response data. Used by the viewer when
// extension-origin requests to video CDNs are blocked by CORS.
//
// A fetch() can't set the Referer itself (forbidden header — Chrome
// replaces it with chrome-extension://...), so a declarativeNetRequest
// rule installs the captured provider-origin referer at the network
// layer (DNR applies to SW fetch() per Chrome docs). The rule is
// installed on demand for the requested host only.
(async () => {
try {
const proxyHost = hostOf(message.url);
if (proxyHost) {
// Prefer the provider's real referer captured from the live
// tab; fall back to a viewer-supplied or host-derived origin.
const captured = await getReferer(proxyHost);
const referer = captured || message.referer || originFromHost(proxyHost);
await ensureRefererRule(proxyHost, referer);
}
// hls.js resolves sub-playslists/segments relative to the master
// and can DROP the token query, handing us a token-less
// URL that 410s. injectFreshToken supplies the freshest live
// token for that video; the final `+` -> `%2B` pass keeps the
// (possibly injected or surviving) base64 token intact for the
// CDN's query parsing.
const resolved = await injectFreshToken(message.url || '');
const proxyUrl = resolved.replace(/\+/g, '%2B');
const resp = await fetch(proxyUrl);
if (!resp.ok) {
console.warn('[goon-bg] proxyFetch', resp.status, message.url.substring(0, 120));
sendResponse({ ok: false, status: resp.status, error: 'HTTP ' + resp.status });
return;
}
const buffer = await resp.arrayBuffer();
const bytes = new Uint8Array(buffer);
// Convert to base64 (btoa can only handle strings up to ~65k chars)
let binary = '';
const CHUNK_SIZE = 32768;
for (let i = 0; i < bytes.length; i += CHUNK_SIZE) {
const chunk = bytes.subarray(i, i + CHUNK_SIZE);
binary += String.fromCharCode.apply(null, Array.from(chunk));
}
const base64 = btoa(binary);
sendResponse({ ok: true, data: base64, mimeType: resp.headers.get('content-type') || '' });
} catch(e) {
console.warn('[goon-bg] proxyFetch failed:', e && e.message ? e.message : e);
sendResponse({ ok: false, error: e && e.message ? e.message : String(e) });
}
})();
return true;
}
default: default:
console.warn('[goon-plugin] Unknown message type:', message.type); console.warn('[goon-plugin] Unknown message type:', message.type);
+5 -5
View File
@@ -25,11 +25,11 @@
/** /**
* Inject interceptors into the MAIN world at document_start. * Inject interceptors into the MAIN world at document_start.
* *
* Uses script.src = chrome.runtime.getURL('interceptors.js') instead of * Uses script.src = chrome.runtime.getURL('interceptors.js') instead of
* script.textContent to bypass strict page CSP. Sites like Pornhub enforce * script.textContent to bypass strict page CSP. Some sites enforce
* 'script-src' that blocks inline execution (textContent). Chrome treats * 'script-src' that blocks inline execution (textContent). Chrome treats
* extension-originated <script src="..."> as content scripts — not as * extension-originated <script src="..."> as content scripts — not as
* page inline code — so the page's CSP does not apply. * page inline code — so the page's CSP does not apply.
* *
* IMPORTANT: Remove the script element ONLY after the load event fires. * IMPORTANT: Remove the script element ONLY after the load event fires.
* Removing it synchronously can prevent the browser from starting the fetch, * Removing it synchronously can prevent the browser from starting the fetch,
+2 -1
View File
@@ -2,7 +2,8 @@
* interceptors.js — Main-world interceptors for HLS manifest detection. * interceptors.js — Main-world interceptors for HLS manifest detection.
* *
* This file is injected into the main world via a <script src="..."> tag * This file is injected into the main world via a <script src="..."> tag
* (NOT inline textContent) to bypass strict CSP on sites like Pornhub. * (NOT inline textContent) to bypass strict CSP on sites that restrict
* script-src.
* *
* When loaded as an external script from the extension origin, Chrome treats * When loaded as an external script from the extension origin, Chrome treats
* it as an extension content script — not as page inline code — so the * it as an extension content script — not as page inline code — so the
+117 -127
View File
@@ -122,7 +122,11 @@ function m3u8DeepProbeFn() {
function probeLog(...args) { function probeLog(...args) {
try { console.log('[goon-probe]', ...args); } catch(e) {} try { console.log('[goon-probe]', ...args); } catch(e) {}
} }
probeLog('Deep probe started on:', window.location.href, 'World:', typeof chrome !== 'undefined' ? 'ISOLATED' : 'MAIN'); // NOTE: the probe is injected with chrome.scripting's `world: 'MAIN'`, so it
// runs in the page's main world and can see page globals. `typeof chrome` is
// NOT a reliable world discriminator (extensions expose `chrome` in the
// isolated world too), so don't infer it that way.
probeLog('Deep probe started on:', window.location.href, '(main world)');
// Helpers available in the MAIN world context // Helpers available in the MAIN world context
function resolveUrl(url) { function resolveUrl(url) {
@@ -134,14 +138,23 @@ function m3u8DeepProbeFn() {
} catch { return null; } } catch { return null; }
} }
function isManifestUrl(url) { function isManifestUrl(url) {
if (!url) return false; if (!url) return false;
const u = url.toLowerCase(); const u = url.toLowerCase();
return u.includes('.m3u8') || u.includes('.hlsvod') || // Dynamic route TEMPLATES (e.g. /playlist/%1%/ or /videos/{id}/) are nav
(u.includes('/hls/') && u.includes('http')) || // links, not fetchable manifests. A placeholder is a `%` that is NOT the
(u.includes('/playlist') && u.includes('http')) || // start of a valid %-encoded hex pair (so real tokens like %2B, %2F are
(u.includes('/manifest') && u.includes('http')); // preserved) or a {name} brace token — so genuine %-encoded auth tokens are
} // never mistaken for template placeholders.
if (/%(?!([0-9A-Fa-f]{2}))/i.test(u) || /\{[a-z0-9_]+\}/i.test(u)) return false;
// .m3u8 / .hlsvod / /hls/ are concrete HLS signals.
if (u.includes('.m3u8') || u.includes('.hlsvod') || u.includes('/hls/')) return true;
// Plural collection paths are nav menus, not manifests.
if (u.includes('/playlists') || u.includes('/manifests')) return false;
// A singular manifest-ish path segment with no concrete file may still be a
// real (rare, extensionless) endpoint, so allow it when it carries http.
return (u.includes('/manifest') || u.includes('/playlist')) && u.includes('http');
}
function walk(obj, path, depth) { function walk(obj, path, depth) {
if (depth > 5) return; if (depth > 5) return;
@@ -319,10 +332,10 @@ function m3u8DeepProbeFn() {
} }
probeLog('Strategy E (interceptor data): found', manifests.length - preECount, 'new manifests'); probeLog('Strategy E (interceptor data): found', manifests.length - preECount, 'new manifests');
// Strategy F: Dynamic playlist fallback — scan video elements for Hls-like // Strategy F: Dynamic playlist fallback — scan video elements for Hls-like
// instances and extract ANY URL, not just .m3u8. Sites like Pornhub use // instances and extract ANY URL, not just .m3u8. Some players use
// HLS.js dynamic playlists where the "manifest" is generated in-memory, // HLS.js dynamic playlists where the "manifest" is generated in-memory,
// so the URL might be a blob:, CDN endpoint, or data URI. // so the URL might be a blob:, CDN endpoint, or data URI.
let strategyFCount = 0; let strategyFCount = 0;
try { try {
for (const v of document.querySelectorAll('video')) { for (const v of document.querySelectorAll('video')) {
@@ -666,8 +679,9 @@ async function probeHlsManifests(tabId) {
log('probeHlsManifests: executing deep probe on tab', tabId); log('probeHlsManifests: executing deep probe on tab', tabId);
const raw = await chrome.scripting.executeScript({ const raw = await chrome.scripting.executeScript({
target: { tabId, allFrames: true }, target: { tabId, allFrames: true },
world: 'MAIN',
func: m3u8DeepProbeFn, func: m3u8DeepProbeFn,
}); });
// When allFrames is true, destroyed / sandboxed frames may return null entries. // When allFrames is true, destroyed / sandboxed frames may return null entries.
// Guard: ensure raw is an array, then filter to objects with a defined .result. // Guard: ensure raw is an array, then filter to objects with a defined .result.
const manifests = Array.isArray(raw) const manifests = Array.isArray(raw)
@@ -697,12 +711,12 @@ function dedupManifests(allManifests) {
}); });
// Sort: prefer interceptor-collected > player globals > config > inline-script // Sort: prefer interceptor-collected > player globals > config > inline-script
const playerRank = { const playerRank = {
'__goon_plugin_data': 0, '__goon_plugin_data': 0,
'content-script-intercept': 0, 'content-script-intercept': 0,
'webRequest': 1, 'webRequest': 1,
'phncdn_reconstructed': 2, 'hls_reconstructed': 2,
'video._goon_hls_manifest': 3, 'video._goon_hls_manifest': 3,
'inline-script': 99, 'inline-script': 99,
}; };
unique.sort((a, b) => { unique.sort((a, b) => {
const aRank = playerRank[a.player] ?? 50; const aRank = playerRank[a.player] ?? 50;
@@ -746,26 +760,26 @@ async function queryWebRequestManifests(tabId) {
} }
/** /**
* Query the service worker for all phncdn.com requests captured by Strategy 8. * Query the service worker for all video requests captured by Strategy 8,
* Returns segments + reconstructed manifest URLs from segment patterns. * including reconstructed manifest URLs derived from segment patterns.
* @param {number} tabId * @param {number} tabId
* @returns {Promise<{segments: Array, reconstructedManifests: Array, totalRequests: number}>} * @returns {Promise<{segments: Array, reconstructedManifests: Array, totalRequests: number}>}
*/ */
async function queryPhncdnRequests(tabId) { async function queryVideoRequests(tabId) {
try { try {
const resp = await chrome.runtime.sendMessage({ const resp = await chrome.runtime.sendMessage({
type: 'GOON_GET_PHNCDN_REQUESTS', type: 'GOON_GET_VIDEO_REQUESTS',
tabId: tabId, tabId: tabId,
}); });
return resp || { segments: [], reconstructedManifests: [], totalRequests: 0 }; return resp || { segments: [], reconstructedManifests: [], totalRequests: 0 };
} catch { } catch {
return { segments: [], reconstructedManifests: [], totalRequests: 0 }; return { segments: [], reconstructedManifests: [], totalRequests: 0 };
} }
} }
/** /**
* Scrape video elements from ALL frames in a tab. * Scrape video elements from ALL frames in a tab.
* On sites like Pornhub, the video player lives inside an iframe — main-frame-only * On many video sites the player lives inside an iframe — main-frame-only
* scraping sees zero <video> elements. Running allFrames catches them. * scraping sees zero <video> elements. Running allFrames catches them.
* *
* Runs in the MAIN world so it can see page globals: * Runs in the MAIN world so it can see page globals:
@@ -778,13 +792,14 @@ async function queryPhncdnRequests(tabId) {
* @param {Array} manifests - HLS manifests found by probing * @param {Array} manifests - HLS manifests found by probing
* @returns {Promise<Object|null>} scrape result or null on error * @returns {Promise<Object|null>} scrape result or null on error
*/ */
async function scrapeTab(tabId, manifests) { async function scrapeTab(tabId, manifests) {
try { try {
const raw = await chrome.scripting.executeScript({ const raw = await chrome.scripting.executeScript({
target: { tabId, allFrames: true }, target: { tabId, allFrames: true },
world: 'MAIN',
func: scrapeFn, func: scrapeFn,
args: [manifests], args: [manifests],
}); });
// When allFrames is true, destroyed / sandboxed frames may return null entries. // When allFrames is true, destroyed / sandboxed frames may return null entries.
// Guard: ensure raw is an array, then filter to objects with a defined .result. // Guard: ensure raw is an array, then filter to objects with a defined .result.
const frameResults = Array.isArray(raw) const frameResults = Array.isArray(raw)
@@ -829,101 +844,75 @@ function filterManifests(manifests) {
(u.includes('/playlist') && !u.endsWith('.ts')); (u.includes('/playlist') && !u.endsWith('.ts'));
}); });
// Reject fake reconstructed manifest paths. // Reject clearly-fake manifest paths (see isValidManifestUrl).
// Real manifests are under /hls/videos/{year}/{subdir}/{videoId}/... const valid = actualManifests.filter(m => isValidManifestUrl(m.url));
// Fakes look like: phncdn.com/videos/{id}/playlist.m3u8 or .../master.m3u8 (no /hls/)
const valid = actualManifests.filter(m => {
const u = m.url.toLowerCase();
try {
const urlObj = new URL(u);
// phncdn URLs must have /hls/ in the path — fake reconstructed paths lack it
if (u.includes('phncdn') || u.includes('pornhubcdn')) {
if (!urlObj.pathname.includes('/hls/') && !u.includes('?h=')) {
return false; // Fake: phncdn URL without /hls/ or auth query param
}
}
} catch {}
return true;
});
// If we have actual (non-fake) manifests, use them directly
if (valid.length > 0) { if (valid.length > 0) {
return valid.map(m => ({ url: m.url })); return valid.map(m => ({ url: m.url }));
} }
// No .m3u8 found — interceptors only captured .ts segments. // No manifest found — interceptors only captured segment files. Reconstruct a
// Reconstruct the parent manifest URL from segment paths. // manifest URL from each segment's directory, carrying over its auth query
// Segment: https://ev-h.phncdn.com/hls/videos/202008/01/338337051/720P_4000K_338337051.mp4/seg-1-v1-a1.ts?h=tmrD...&e=1786932752&f=1 // string intact (CDNs reject a path-only URL with 403). CDN-agnostic: the
// Manifest: https://ev-h.phncdn.com/hls/videos/202008/01/338337051/720P_4000K_338337051.mp4/master.m3u8?h=tmrD...&e=1786932752&f=1 // segment filename is swapped for a manifest filename in the same directory,
// // e.g. .../variant/seg-1.ts?h=... -> .../variant/master.m3u8?h=...
// Auth tokens (h, e, f or validfrom, validto, hash) are extracted from segment query params
// and reapplied to the reconstructed manifest URL. This is critical because CDN domains
// require auth — a path-only URL will be rejected with 403.
const seen = new Set(); const seen = new Set();
const reconstructed = []; const reconstructed = [];
// A segment filename: numbered segment, or an .ts/.m4s/.aac media extension.
const SEG_RE = /(seg-\d+|\.(?:ts|m4s|aac)(?:$|\?))/i;
for (const m of manifests) { for (const m of manifests) {
const u = m.url.toLowerCase(); if (!SEG_RE.test(m.url)) continue;
if (!u.includes('/hls/videos/') || !/\/seg-\d+/.test(u)) continue;
try { try {
const urlObj = new URL(m.url); const urlObj = new URL(m.url);
// Skip non-phncdn hosts — we only reconstruct for known video CDNs
const host = urlObj.host.toLowerCase();
if (!host.includes('phncdn')) continue;
const pathParts = urlObj.pathname.split('/'); const pathParts = urlObj.pathname.split('/');
// Find the /hls/ marker and extract: hls/videos/{year}/{subdir}/{videoId}/{variant}/... // Need a directory to attach the manifest to (skip flat single-file paths).
const hlsIdx = pathParts.findIndex(p => p === 'hls'); if (pathParts.length < 3) continue;
if (hlsIdx === -1 || hlsIdx + 5 >= pathParts.length) continue; pathParts.pop(); // drop the segment filename, keep its directory
// Reconstruct with BOTH "hls" prefix AND variant directory included: const dir = pathParts.join('/');
// segment: /hls/videos/202008/01/338337051/720P_4000K_338337051.mp4/seg-1.ts // Carry the raw query (auth token) over unchanged — never re-encode it.
// manifest: /hls/videos/202008/01/338337051/720P_4000K_338337051.mp4/master.m3u8 const q = m.url.indexOf('?');
const year = pathParts[hlsIdx + 2]; // e.g. 2020 const rawQuery = q === -1 ? '' : m.url.slice(q);
const subDir = pathParts[hlsIdx + 3]; // e.g. 01 const manifestUrl = `${urlObj.protocol}//${urlObj.host}/${dir}/master.m3u8${rawQuery}`;
const videoId = pathParts[hlsIdx + 4]; // e.g. 338337051
const variant = pathParts[hlsIdx + 5]; // e.g. 720P_4000K_338337051.mp4
// Extract auth params from segment query string — different CDN variants use different keys:
// ev-h/validfrom/validto/hash vs hv-h/h/e/f
const authParts = [];
for (const [key, val] of urlObj.searchParams) {
if (['h', 'e', 'f', 'validfrom', 'validto', 'hash'].includes(key)) {
authParts.push(`${key}=${val}`);
}
}
const authStr = authParts.length > 0 ? '?' + authParts.join('&') : '';
// Build manifest URL: include hls prefix AND variant directory
const manifestUrl = `${urlObj.protocol}//${urlObj.host}/${pathParts[hlsIdx]}/${pathParts[hlsIdx + 1]}/${year}/${subDir}/${videoId}/${variant}/master.m3u8${authStr}`;
if (!seen.has(manifestUrl)) { if (!seen.has(manifestUrl)) {
seen.add(manifestUrl); seen.add(manifestUrl);
reconstructed.push({ url: manifestUrl }); reconstructed.push({ url: manifestUrl });
} }
} catch { /* skip malformed URLs */ } } catch { /* skip malformed URLs */ }
} }
return reconstructed; return reconstructed;
} }
/** /**
* Validate that a manifest URL uses a real phncdn.com path structure. * Validate that a manifest URL is well-formed and generic (CDN-agnostic).
* Rejects fake paths like /videos/{id}/playlist.m3u8 — real manifests use * Rejects malformed paths; a manifest-like URL must live in a directory (have a
* /hls/videos/{year}/{subdir}/{videoId}/master.m3u8 or similar /hls/ paths. * path separator), which filters out obviously-broken single-segment paths.
*/ */
function isValidManifestUrl(url) { function isValidManifestUrl(url) {
if (!url || typeof url !== 'string') return false; if (!url || typeof url !== 'string') return false;
// Real phncdn HLS manifest: must contain /hls/videos/ with sufficient path depth // Dynamic route templates / nav links are not fetchable manifests. A
const hlsIdx = url.indexOf('/hls/videos/'); // placeholder is a `%` not followed by two hex digits (real %2B etc. are
if (hlsIdx === -1) { // kept) or a {name} brace token.
// Not phncdn — could be a different CDN, allow it if (/%(?!([0-9A-Fa-f]{2}))/i.test(url) || /\{[a-z0-9_]+\}/i.test(url)) return false;
if (!url.includes('phncdn') && !url.includes('pornhubcdn')) return true; let urlObj;
// phncdn without /hls/ prefix is invalid for manifests try {
return false; urlObj = new URL(url);
} } catch {
// Check there are at least {year}/{subdir}/{videoId} after /hls/videos/ return false;
const after = url.substring(hlsIdx + '/hls/videos/'.length); }
if (after.indexOf('/') === -1) return false; // need at least one more segment const path = urlObj.pathname.toLowerCase();
return true; // Plural collection paths and trailing-slash directory listings are nav menus,
} // not manifest files (e.g. /playlists/, /playlists/liked/, /playlist/%id%/).
if (path.includes('/playlists') || path.includes('/manifests') || path.endsWith('/')) {
return false;
}
const isManifestLike = path.includes('.m3u8') || path.includes('.hlsvod') ||
(path.includes('/master') && path.includes('.m3u')) ||
(path.includes('/playlist') && !path.endsWith('.ts')) ||
path.includes('/manifest');
// Not a manifest path — allow it (it may be a direct media source, not HLS).
if (!isManifestLike) return true;
// A manifest path must sit in a directory, not be a bare top-level file.
return path.includes('/');
}
/** /**
* Build a normalized video entry from a scraped video object. * Build a normalized video entry from a scraped video object.
@@ -941,8 +930,8 @@ function makeVideoEntry(video, tabTitle, posterFromContent) {
mediaType: video.mediaType || 'video', mediaType: video.mediaType || 'video',
}; };
if (video.hlsManifest) { if (video.hlsManifest) {
// Reject fake phncdn manifest paths before storing // Reject fake/invalid manifest paths before storing
if (!isValidManifestUrl(video.hlsManifest)) { if (!isValidManifestUrl(video.hlsManifest)) {
console.warn('[goon] Rejecting fake manifest URL:', video.hlsManifest.substring(0, 120)); console.warn('[goon] Rejecting fake manifest URL:', video.hlsManifest.substring(0, 120));
} else { } else {
entry.hlsManifest = video.hlsManifest; entry.hlsManifest = video.hlsManifest;
@@ -1004,28 +993,28 @@ async function scanVideos(tabsToScan) {
} }
await logToPage(`step1.5 (webRequest): found ${totalWebRequestManifests} manifests via network interception`); await logToPage(`step1.5 (webRequest): found ${totalWebRequestManifests} manifests via network interception`);
// Step 1.6: Strategy 8 — Query phncdn.com requests and reconstruct manifest URLs. // Step 1.6: Strategy 8 — Query captured video requests and reconstruct manifest URLs.
// When the player uses in-memory dynamic HLS (zero .m3u8 network requests), we can // When the player uses in-memory dynamic HLS (zero .m3u8 network requests), we can
// still detect it by recognizing segment URL patterns from phncdn.com and deriving // still detect it by recognizing segment URL patterns and deriving the parent
// the parent manifest URL: /hls/videos/{id}/master.m3u8 // manifest URL from the segment's directory.
const phncdnPromises = tabsToScan.map(t => queryPhncdnRequests(t.id)); const videoRequestPromises = tabsToScan.map(t => queryVideoRequests(t.id));
const phncdnResults = await Promise.all(phncdnPromises); const videoRequestResults = await Promise.all(videoRequestPromises);
let totalPhncdnManifests = 0; let totalReconstructedManifests = 0;
for (let i = 0; i < tabsToScan.length; i++) { for (let i = 0; i < tabsToScan.length; i++) {
const reconstructed = phncdnResults[i]?.reconstructedManifests || []; const reconstructed = videoRequestResults[i]?.reconstructedManifests || [];
if (reconstructed.length > 0) { if (reconstructed.length > 0) {
log('scanVideos phncdn: tab', tabsToScan[i].id, 'found', reconstructed.length, 'reconstructed manifests from segment URLs'); log('scanVideos strategy8: tab', tabsToScan[i].id, 'found', reconstructed.length, 'reconstructed manifests from segment URLs');
} }
totalPhncdnManifests += reconstructed.length; totalReconstructedManifests += reconstructed.length;
const existing = tabManifestMap.get(tabsToScan[i].id) || []; const existing = tabManifestMap.get(tabsToScan[i].id) || [];
for (const url of reconstructed) { for (const url of reconstructed) {
if (!existing.some(m => m.url === url)) { if (!existing.some(m => m.url === url)) {
existing.push({ url, player: 'phncdn_reconstructed' }); existing.push({ url, player: 'hls_reconstructed' });
}
} }
}
tabManifestMap.set(tabsToScan[i].id, existing); tabManifestMap.set(tabsToScan[i].id, existing);
} }
await logToPage(`step1.6 (phncdn): found ${totalPhncdnManifests} reconstructed manifests from segment URLs`); await logToPage(`step1.6 (strategy8): found ${totalReconstructedManifests} reconstructed manifests from segment URLs`);
// Step 2: Scrape video elements (all frames) AND query content scripts (in parallel) // Step 2: Scrape video elements (all frames) AND query content scripts (in parallel)
const scrapePromises = tabsToScan.map(t => const scrapePromises = tabsToScan.map(t =>
@@ -1186,6 +1175,7 @@ findBtn.addEventListener('click', async () => {
try { try {
await chrome.scripting.executeScript({ await chrome.scripting.executeScript({
target: { tabId: activeTab.id }, target: { tabId: activeTab.id },
world: 'MAIN',
func: () => { func: () => {
const diag = []; const diag = [];
// Check interceptor data // Check interceptor data
+135 -100
View File
@@ -19,10 +19,10 @@
* overlays and an aggregate status bar. * overlays and an aggregate status bar.
*/ */
// HLS is vendored locally to avoid CSP blocks on sites like Pornhub that // HLS is vendored locally to avoid CSP blocks on sites that restrict
// restrict script-src to 'self'. The hls.min.js bundle is built by Vite // script-src to 'self'. The hls.js bundle is copied (not minified) so it
// and exposed via chrome.runtime.getURL() so it loads from the extension // loads from the extension origin via chrome.runtime.getURL(), bypassing
// origin, bypassing page CSP entirely. // page CSP entirely.
/** /**
* Grid-optimized hls.js configuration. * Grid-optimized hls.js configuration.
@@ -54,14 +54,15 @@ const panelsEl = document.getElementById('panels');
/** Track all Hls instances so we can destroy them on unload. */ /** Track all Hls instances so we can destroy them on unload. */
/** /**
* Custom HLS loader that proxies phncdn.com URLs through the service worker * Custom HLS loader that proxies URLs whose host blocks extension-origin
* (bypassing CORS/Referer restrictions) and fetches everything else natively. * requests (via a learned proxy-host set) through the service worker
* * (bypassing CORS/Referer restrictions) and fetches everything else natively.
* hls.js creates one loader instance per request via `new HlsLoader(hls)`, then *
* calls `loader.load(context, config, callbacks)` — callbacks are the 3rd arg * hls.js creates one loader instance per request via `new HlsLoader(hls)`, then
* in v9 (some wrapper paths put them 2nd, so we scan each argument). The stats * calls `loader.load(context, config, callbacks)` — callbacks are the 3rd arg
* object passed to the callbacks is pre-allocated by us and mutated by hls.js * in v9 (some wrapper paths put them 2nd, so we scan each argument). The stats
* object passed to the callbacks is pre-allocated by us and mutated by hls.js
* in place, so it must have the correct sub-shape (see createLoadedStats). * in place, so it must have the correct sub-shape (see createLoadedStats).
*/ */
function HlsLoader(config) { function HlsLoader(config) {
@@ -134,8 +135,8 @@ const panelsEl = document.getElementById('panels');
const stats = this.stats; const stats = this.stats;
stats.loading.start = trequest; stats.loading.start = trequest;
const isManifest = /\.m3u8($|\?)/i.test(url || ''); const isManifest = /\.m3u8($|\?)/i.test(url || '');
const needProxy = (url || '').includes('phncdn') || (url || '').includes('pornhubcdn'); const needProxy = needsProxy(url);
// hls.js loader callback contract (verified against the bundled loader): // hls.js loader callback contract (verified against the bundled loader):
// onSuccess(response, stats, context, networkDetails) // onSuccess(response, stats, context, networkDetails)
@@ -176,8 +177,8 @@ const panelsEl = document.getElementById('panels');
if (needProxy) { if (needProxy) {
// Route through the service worker to bypass the CDN's CORS/Referer // Route through the service worker to bypass the CDN's CORS/Referer
// restrictions. The SW adds the required Referer and returns base64. // restrictions. The SW adds the required Referer and returns base64.
console.log('[goon-viewer] Proxying via SW:', (url || '').substring(0, 120)); console.log('[goon-viewer] Proxying via SW:', (url || '').substring(0, 120));
chrome.runtime.sendMessage({ type: 'GOON_PROXY_FETCH', url }, (resp) => { chrome.runtime.sendMessage({ type: 'GOON_PROXY_FETCH', url, referer: originForUrl(url) }, (resp) => {
if (self._aborted) return; if (self._aborted) return;
const lastErr = chrome.runtime.lastError; const lastErr = chrome.runtime.lastError;
if (lastErr || !resp || !resp.ok) { if (lastErr || !resp || !resp.ok) {
@@ -233,52 +234,78 @@ const panelsEl = document.getElementById('panels');
HlsLoader.prototype.destroy = function () { this.abort(); }; HlsLoader.prototype.destroy = function () { this.abort(); };
HlsLoader.canSupportLoader = function () { return true; }; HlsLoader.canSupportLoader = function () { return true; };
// === CORS diagnostics — check if phncdn.com allows extension-origin requests === // === Service-worker proxy helper (CDN-agnostic) ===
(async function checkCors() { // Proxies a video URL through the service worker (no CORS restrictions). The
// SW forces the correct provider-origin Referer for the host (captured by the
// background worker), so a referer-protecting CDN accepts the request.
// Returns an ArrayBuffer of the response content.
async function proxyFetch(url, referer) {
try { try {
const resp = await fetch('https://phncdn.com/images/favicon.ico', { method: 'HEAD', mode: 'cors' }); const resp = await chrome.runtime.sendMessage({ type: 'GOON_PROXY_FETCH', url, referer });
console.log('[goon-viewer] CORS check phncdn.com:', resp.ok ? 'ALLOWED (status ' + resp.status + ')' : 'BLOCKED');
} catch(e) {
console.warn('[goon-viewer] CORS check phncdn.com: BLOCKED —', e && e.message ? e.message : e);
}
})();
// === HLS segment proxy helper ===
// Proxies a video URL through the service worker (no CORS restrictions).
// Returns an ArrayBuffer of the response content.
async function proxyFetch(url) {
try {
const resp = await chrome.runtime.sendMessage({ type: 'GOON_PROXY_FETCH', url });
if (resp && resp.data && resp.ok) { if (resp && resp.data && resp.ok) {
// Decode base64 → ArrayBuffer // Decode base64 → ArrayBuffer
const binary = atob(resp.data); const binary = atob(resp.data);
const bytes = new Uint8Array(binary.length); const bytes = new Uint8Array(binary.length);
for (let i = 0; i < binary.length; i++) { for (let i = 0; i < binary.length; i++) {
bytes[i] = binary.charCodeAt(i); bytes[i] = binary.charCodeAt(i);
} }
return bytes.buffer; return bytes.buffer;
} }
return null; return null;
} catch(e) { } catch(e) {
console.warn('[goon-viewer] proxyFetch failed for:', url.substring(0, 120), e && e.message ? e.message : e); console.warn('[goon-viewer] proxyFetch failed for:', url.substring(0, 120), e && e.message ? e.message : e);
return null; return null;
} }
} }
// === Check if any manifest URLs require proxying (phncdn.com / cdn) === // CDN hosts that block extension-origin requests are learned by a CORS probe
function urlsNeedProxy(urls) { // at viewer startup (hosts that fail the probe are recorded), then stored in
return urls.some(u => u && typeof u === 'string' && (u.includes('phncdn') || u.includes('pornhubcdn'))); // chrome.storage.local. The proxy is used only for those hosts, so we never
} // hardcode a particular CDN.
const PROXY_HOSTS_KEY = 'goon_proxy_hosts';
const proxyHosts = new Set();
(async () => {
try {
const stored = await chrome.storage.local.get(PROXY_HOSTS_KEY);
const hosts = stored[PROXY_HOSTS_KEY];
if (Array.isArray(hosts)) for (const h of hosts) proxyHosts.add(String(h).toLowerCase());
} catch { /* ignore */ }
})();
/** Check whether fetching a URL from the extension page works (CORS test). */ // Origin for a URL's host — used as the Referer to request when a proxy fetch
async function corsCheck(url) { // is made for that host (the SW also captures the live referer per host).
function originForUrl(url) {
if (!url) return '';
try { return new URL(url).origin + '/'; } catch { return ''; }
}
// Whether a given video URL should be proxied: its host is a known
// referer-protected CDN host.
function needsProxy(url) {
if (!url || typeof url !== 'string') return false;
const host = hostOfUrl(url);
return host ? proxyHosts.has(host) : false;
}
// host of a URL (lowercased); '' when malformed.
function hostOfUrl(url) {
try { return new URL(url).host.toLowerCase(); } catch { return ''; }
}
// Whether any of the given URLs needs proxying.
function urlsNeedProxy(urls) {
return urls.some(u => needsProxy(u));
}
/** Check whether fetching a URL from the extension page works (CORS test). */
async function corsCheck(url) {
try { try {
const resp = await fetch(url, { method: 'HEAD', mode: 'cors' }); const resp = await fetch(url, { method: 'HEAD', mode: 'cors' });
return resp.ok; return resp.ok;
} catch { } catch {
return false; return false;
} }
} }
/** Track all Hls instances so we can destroy them on unload. */ /** Track all Hls instances so we can destroy them on unload. */
const hlsInstances = new Map(); const hlsInstances = new Map();
@@ -750,10 +777,10 @@ function wireHlsEvents(hls, video, panel) {
renderPanelState(video, panel); renderPanelState(video, panel);
hls.destroy(); hls.destroy();
hlsInstances.delete(video); hlsInstances.delete(video);
// A fatal network error on a phncdn stream is almost always an expired // A fatal network error on a token-protected stream is almost always an
// auth token. The live source tab keeps rotating tokens, so pull a fresh // expired auth token. The live source tab keeps rotating tokens, so pull a
// manifest from it and retry — capped at a few attempts per panel so a // fresh manifest from it and retry — capped at a few attempts per panel so a
// tab with no live source doesn't loop forever. // tab with no live source doesn't loop forever.
if ((video.dataset.hlsRescans || '0') | 0 < 3) { if ((video.dataset.hlsRescans || '0') | 0 < 3) {
video.dataset.hlsRescans = ((video.dataset.hlsRescans || '0') | 0) + 1; video.dataset.hlsRescans = ((video.dataset.hlsRescans || '0') | 0) + 1;
console.log('[goon-viewer] fatal NETWORK_ERROR → auto-rescan attempt #' + video.dataset.hlsRescans); console.log('[goon-viewer] fatal NETWORK_ERROR → auto-rescan attempt #' + video.dataset.hlsRescans);
@@ -930,12 +957,12 @@ function playHls(video, manifestUrl, panel) {
return; return;
} }
// Check whether this manifest URL needs proxying (phncdn.com without CORS headers // Check whether this manifest URL needs proxying (a referer-protected host
// blocks extension-origin requests from the viewer page). // without CORS headers blocks extension-origin requests from the viewer page).
const needProxy = manifestUrl.includes('phncdn') || manifestUrl.includes('pornhubcdn'); const needProxy = needsProxy(manifestUrl);
if (needProxy) { if (needProxy) {
console.log('[goon-viewer] Manifest needs proxying via service worker'); console.log('[goon-viewer] Manifest needs proxying via service worker');
} }
destroyHls(video); destroyHls(video);
@@ -977,9 +1004,9 @@ function playHls(video, manifestUrl, panel) {
// Log the manifest URL being loaded for debugging (helps diagnose playback failures). // Log the manifest URL being loaded for debugging (helps diagnose playback failures).
console.log(`[goon-viewer] Loading HLS manifest:`, manifestUrl.substring(0, 200)); console.log(`[goon-viewer] Loading HLS manifest:`, manifestUrl.substring(0, 200));
// Remember the manifest actually loaded so a later auto-rescan can retry from // Remember the manifest actually loaded so a later auto-rescan can retry from
// the freshest token on the same video (extractVideoId keys off this). // the freshest token on the same video (the service worker keys off the path).
video.dataset.hlsManifest = manifestUrl; video.dataset.hlsManifest = manifestUrl;
hls.loadSource(manifestUrl); hls.loadSource(manifestUrl);
hls.attachMedia(video); hls.attachMedia(video);
@@ -1017,32 +1044,32 @@ function loadHlsJs() {
}); });
} }
/* ======================================================================== */ /* ======================================================================== */
/* Fresh-token resolution — replace expired/revoked phncdn URLs */ /* Fresh-token resolution — replace expired/revoked token URLs */
/* ======================================================================== */ /* ======================================================================== */
/** /**
* Ask the service worker for the freshest live phncdn manifest for the video * Ask the service worker for the freshest live manifest for the video encoded
* encoded in `storedUrl`. The live source tab keeps rotating short-lived * in `storedUrl`. The live source tab keeps rotating short-lived auth tokens
* phncdn tokens (a stored one may already be 410 Gone), so the freshest * (a stored one may already be 410 Gone), so the freshest captured manifest —
* captured manifest — identified by matching the videoId in the path — is the * identified by matching the video's group key in the path — is the one that
* one that will actually validate. Resolves to the new URL, or null when the * will actually validate. Resolves to the new URL, or null when the tab
* tab captured nothing (or the caller can't reach the SW). * captured nothing (or the caller can't reach the SW).
*/ */
async function resolveFreshManifest(storedUrl) { async function resolveFreshManifest(storedUrl) {
if (!storedUrl || !storedUrl.includes('phncdn')) return null; if (!storedUrl || !needsProxy(storedUrl)) return null;
try { try {
const resp = await chrome.runtime.sendMessage({ type: 'GOON_RESOLVE_MANIFEST', url: storedUrl }); const resp = await chrome.runtime.sendMessage({ type: 'GOON_RESOLVE_MANIFEST', url: storedUrl });
if (resp && resp.ok && resp.changed && resp.fresh) { if (resp && resp.ok && resp.changed && resp.fresh) {
console.log('[goon-viewer] resolved fresh manifest for', (resp.videoId || '?'), console.log('[goon-viewer] resolved fresh manifest for', (resp.groupKey || '?'),
'→', (resp.fresh || '').substring(0, 100)); '→', (resp.fresh || '').substring(0, 100));
return resp.fresh; return resp.fresh;
} }
} catch (e) { } catch (e) {
console.warn('[goon-viewer] resolveFreshManifest failed:', e && e.message ? e.message : e); console.warn('[goon-viewer] resolveFreshManifest failed:', e && e.message ? e.message : e);
}
return null;
} }
return null;
}
/** /**
* Re-attempt playback on `video` after its stored manifest died (fatal * Re-attempt playback on `video` after its stored manifest died (fatal
@@ -1154,16 +1181,24 @@ async function init() {
}); });
} }
// Test CORS on all video URLs // Probe CORS on each video URL. Hosts that block extension-origin
const urlsToTest = videoUrls.map(v => v.src || v.hlsManifest || 'none').filter(u => u !== 'none' && typeof u === 'string'); // requests are learned as referer-protected and persisted so the viewer
for (const url of [...new Set(urlsToTest)]) { // routes them through the service-worker proxy.
const corsOk = await corsCheck(url); const urlsToTest = videoUrls.map(v => v.src || v.hlsManifest || 'none').filter(u => u !== 'none' && typeof u === 'string');
console.log('[goon-viewer] CORS', corsOk ? 'OK' : 'BLOCKED:', url.substring(0, 120)); const blockedHosts = new Set();
} for (const url of [...new Set(urlsToTest)]) {
const corsOk = await corsCheck(url);
if (!corsOk) blockedHosts.add(hostOfUrl(url));
console.log('[goon-viewer] CORS', corsOk ? 'OK' : 'BLOCKED:', url.substring(0, 120));
}
if (blockedHosts.size > 0) {
for (const h of blockedHosts) if (h) proxyHosts.add(h);
try { await chrome.storage.local.set({ [PROXY_HOSTS_KEY]: [...proxyHosts] }); } catch { /* ignore */ }
}
// Check if any URLs point to phncdn.com (proxy will be needed if CORS blocked) // Report whether any URL needs proxying (referer-protected host).
const needsProxy = urlsToTest.some(u => u.includes('phncdn') || u.includes('pornhubcdn')); const anyNeedsProxy = urlsNeedProxy(urlsToTest);
console.log('[goon-viewer] Proxy required:', needsProxy); console.log('[goon-viewer] Proxy required:', anyNeedsProxy);
} else { } else {
console.warn('[goon-viewer] No video entries in storage for ID:', id, '— data may have been cleared or never saved'); console.warn('[goon-viewer] No video entries in storage for ID:', id, '— data may have been cleared or never saved');
} }
@@ -1194,15 +1229,15 @@ async function init() {
} }
return; return;
} }
// Before playback, swap each stored phncdn manifest for the freshest live // Before playback, swap each referer-protected manifest for the freshest
// one the source tab has captured (a stored token may already be 410 // live one the source tab has captured (a stored token may already be 410
// Gone). Non-phncdn manifests are left untouched. // Gone). Non-protected manifests are left untouched.
for (const { video, manifest, panel } of hlsEntries) { for (const { video, manifest } of hlsEntries) {
if (manifest.includes('phncdn')) { if (needsProxy(manifest)) {
const fresh = await resolveFreshManifest(manifest); const fresh = await resolveFreshManifest(manifest);
if (fresh) video.dataset.hlsManifest = fresh; if (fresh) video.dataset.hlsManifest = fresh;
} }
} }
for (const { video, panel } of hlsEntries) { for (const { video, panel } of hlsEntries) {
playHls(video, video.dataset.hlsManifest, panel); playHls(video, video.dataset.hlsManifest, panel);
} }