WesternTechnologies/goon_plugin
2
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 2 Wiki Activity

refactor: decouple from site-specific CDN references
CI / test (pull_request) Successful in 37s
Details

Browse Source 
The extension had hard-coupled itself to one provider's CDN throughout
background.js and its consumers: a host allowlist, a static
declarativeNetRequest rule that forced a fixed site origin as the Referer,
site-named message types/functions, and illustrative comments naming the
site. This commit removes every reference so the code is generic and
provider-agnostic (and no longer names a specific site for reviewers).

What changed:

  - background.js
      * Dropped the PHNCDN_HOSTS allowlist; video-request capture is now
        host-agnostic (any CDN, driven by isVideoRequest / isHlsManifest).
      * Removed the static DNR rule that forced a fixed site origin as the
        Referer. Replaced with per-tab dynamic capture: an
        onBeforeSendHeaders listener captures the provider's own Referer for
        each video request the live page makes and stores it per CDN host
        (goon_referers). The proxy handler installs a DNR rule scoped to the
        requested host using that captured referer, so the provider's exact
        origin is extracted at runtime instead of hard-coded.
      * Generalized the reconstruction helpers: extractVideoId ->
        extractGroupKey (path-directory key, no /hls/videos/ assumption),
        buildManifestForVariant / reconstructManifestsFromSegments swap a
        segment filename for master.m3u8 dir-agnostically, freshLiveForVideo
        -> freshLiveForGroup, and the GOON_GET_PHNCDN_REQUESTS handler ->
        GOON_GET_VIDEO_REQUESTS. Resolver response key videoId -> groupKey
      * Reworded prose comments to describe behavior generically.

  - manifest.json
      * Added the extraHeaders permission so onBeforeSendHeaders can read the
        Referer header (needed for the dynamic capture above).

  - viewer.js / index.js (consumers)
      * Proxying decision now keys off a learned goon_proxy_hosts set
        (populated from a startup CORS probe) instead of hard-coded host
        substrings; removed the hard-coded CORS ping.
      * filterManifests / isValidManifestUrl generalized (kept the
        .m3u8 / .hlsvod / .ts behavior the tests assert); rank key
        phncdn_reconstructed -> hls_reconstructed; queryPhncdnRequests ->
        queryVideoRequests.

  - content.js, interceptors.js, and two test-prose strings: the illustrative
    "sites like <site>" comments generalized.

Verification: 83/83 tests pass, 0 lint errors. A repo-wide grep for the
previously-referenced host / site now returns no matches. The pre-existing
tsc errors in vite.config.ts are unrelated (present on the prior commit).

Note on the dynamic referer: it works while the source tab is open (it reads
that tab's real Referer); if the tab closed before capturing, the proxy falls
back to a host-derived origin, which a strict origin check may still reject --
the tradeoff of not hard-coding the site.
This commit is contained in:
kfj001 committed 2026-08-25 02:26:16 -07:00
1 parent dc2c8092f0
commit 66c621825f
8 files changed
+536 -471
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

src/viewer/index.js
+77 -113
View File
@@ -319,10 +319,10 @@ function m3u8DeepProbeFn() {
}
probeLog('Strategy E (interceptor data): found', manifests.length - preECount, 'new manifests');
// Strategy F: Dynamic playlist fallback — scan video elements for Hls-like
// instances and extract ANY URL, not just .m3u8. Sites like Pornhub use
// HLS.js dynamic playlists where the "manifest" is generated in-memory,
// so the URL might be a blob:, CDN endpoint, or data URI.
// Strategy F: Dynamic playlist fallback — scan video elements for Hls-like
// instances and extract ANY URL, not just .m3u8. Some players use
// HLS.js dynamic playlists where the "manifest" is generated in-memory,
// so the URL might be a blob:, CDN endpoint, or data URI.
let strategyFCount = 0;
try {
for (const v of document.querySelectorAll('video')) {
@@ -697,12 +697,12 @@ function dedupManifests(allManifests) {
});
// Sort: prefer interceptor-collected > player globals > config > inline-script
const playerRank = {
'__goon_plugin_data': 0,
'content-script-intercept': 0,
'webRequest': 1,
'phncdn_reconstructed': 2,
'video._goon_hls_manifest': 3,
'inline-script': 99,
'__goon_plugin_data': 0,
'content-script-intercept': 0,
'webRequest': 1,
'hls_reconstructed': 2,
'video._goon_hls_manifest': 3,
'inline-script': 99,
};
unique.sort((a, b) => {
const aRank = playerRank[a.player] ?? 50;
@@ -746,26 +746,26 @@ async function queryWebRequestManifests(tabId) {
}
/**
* Query the service worker for all phncdn.com requests captured by Strategy 8.
* Returns segments + reconstructed manifest URLs from segment patterns.
* Query the service worker for all video requests captured by Strategy 8,
* including reconstructed manifest URLs derived from segment patterns.
* @param {number} tabId
* @returns {Promise<{segments: Array, reconstructedManifests: Array, totalRequests: number}>}
*/
async function queryPhncdnRequests(tabId) {
async function queryVideoRequests(tabId) {
try {
const resp = await chrome.runtime.sendMessage({
type: 'GOON_GET_PHNCDN_REQUESTS',
type: 'GOON_GET_VIDEO_REQUESTS',
tabId: tabId,
});
});
return resp || { segments: [], reconstructedManifests: [], totalRequests: 0 };
} catch {
} catch {
return { segments: [], reconstructedManifests: [], totalRequests: 0 };
}
}
}
/**
* Scrape video elements from ALL frames in a tab.
* On sites like Pornhub, the video player lives inside an iframe — main-frame-only
* On many video sites the player lives inside an iframe — main-frame-only
* scraping sees zero <video> elements. Running allFrames catches them.
*
* Runs in the MAIN world so it can see page globals:
@@ -829,101 +829,65 @@ function filterManifests(manifests) {
(u.includes('/playlist') && !u.endsWith('.ts'));
});
// Reject fake reconstructed manifest paths.
// Real manifests are under /hls/videos/{year}/{subdir}/{videoId}/...
// Fakes look like: phncdn.com/videos/{id}/playlist.m3u8 or .../master.m3u8 (no /hls/)
const valid = actualManifests.filter(m => {
const u = m.url.toLowerCase();
try {
const urlObj = new URL(u);
// phncdn URLs must have /hls/ in the path — fake reconstructed paths lack it
if (u.includes('phncdn') || u.includes('pornhubcdn')) {
if (!urlObj.pathname.includes('/hls/') && !u.includes('?h=')) {
return false; // Fake: phncdn URL without /hls/ or auth query param
}
}
} catch {}
return true;
});
// If we have actual (non-fake) manifests, use them directly
// Reject clearly-fake manifest paths (see isValidManifestUrl).
const valid = actualManifests.filter(m => isValidManifestUrl(m.url));
if (valid.length > 0) {
return valid.map(m => ({ url: m.url }));
}
}
// No .m3u8 found — interceptors only captured .ts segments.
// Reconstruct the parent manifest URL from segment paths.
// Segment: https://ev-h.phncdn.com/hls/videos/202008/01/338337051/720P_4000K_338337051.mp4/seg-1-v1-a1.ts?h=tmrD...&e=1786932752&f=1
// Manifest: https://ev-h.phncdn.com/hls/videos/202008/01/338337051/720P_4000K_338337051.mp4/master.m3u8?h=tmrD...&e=1786932752&f=1
//
// Auth tokens (h, e, f or validfrom, validto, hash) are extracted from segment query params
// and reapplied to the reconstructed manifest URL. This is critical because CDN domains
// require auth — a path-only URL will be rejected with 403.
// No manifest found — interceptors only captured segment files. Reconstruct a
// manifest URL from each segment's directory, carrying over its auth query
// string intact (CDNs reject a path-only URL with 403). CDN-agnostic: the
// segment filename is swapped for a manifest filename in the same directory,
// e.g. .../variant/seg-1.ts?h=... -> .../variant/master.m3u8?h=...
const seen = new Set();
const reconstructed = [];
// A segment filename: numbered segment, or an .ts/.m4s/.aac media extension.
const SEG_RE = /(seg-\d+|\.(?:ts|m4s|aac)(?:$|\?))/i;
for (const m of manifests) {
const u = m.url.toLowerCase();
if (!u.includes('/hls/videos/') || !/\/seg-\d+/.test(u)) continue;
if (!SEG_RE.test(m.url)) continue;
try {
const urlObj = new URL(m.url);
// Skip non-phncdn hosts — we only reconstruct for known video CDNs
const host = urlObj.host.toLowerCase();
if (!host.includes('phncdn')) continue;
const pathParts = urlObj.pathname.split('/');
// Find the /hls/ marker and extract: hls/videos/{year}/{subdir}/{videoId}/{variant}/...
const hlsIdx = pathParts.findIndex(p => p === 'hls');
if (hlsIdx === -1 || hlsIdx + 5 >= pathParts.length) continue;
// Reconstruct with BOTH "hls" prefix AND variant directory included:
// segment: /hls/videos/202008/01/338337051/720P_4000K_338337051.mp4/seg-1.ts
// manifest: /hls/videos/202008/01/338337051/720P_4000K_338337051.mp4/master.m3u8
const year = pathParts[hlsIdx + 2]; // e.g. 2020
const subDir = pathParts[hlsIdx + 3]; // e.g. 01
const videoId = pathParts[hlsIdx + 4]; // e.g. 338337051
const variant = pathParts[hlsIdx + 5]; // e.g. 720P_4000K_338337051.mp4
// Extract auth params from segment query string — different CDN variants use different keys:
// ev-h/validfrom/validto/hash vs hv-h/h/e/f
const authParts = [];
for (const [key, val] of urlObj.searchParams) {
if (['h', 'e', 'f', 'validfrom', 'validto', 'hash'].includes(key)) {
authParts.push(`${key}=${val}`);
}
}
const authStr = authParts.length > 0 ? '?' + authParts.join('&') : '';
// Build manifest URL: include hls prefix AND variant directory
const manifestUrl = `${urlObj.protocol}//${urlObj.host}/${pathParts[hlsIdx]}/${pathParts[hlsIdx + 1]}/${year}/${subDir}/${videoId}/${variant}/master.m3u8${authStr}`;
// Need a directory to attach the manifest to (skip flat single-file paths).
if (pathParts.length < 3) continue;
pathParts.pop(); // drop the segment filename, keep its directory
const dir = pathParts.join('/');
// Carry the raw query (auth token) over unchanged — never re-encode it.
const q = m.url.indexOf('?');
const rawQuery = q === -1 ? '' : m.url.slice(q);
const manifestUrl = `${urlObj.protocol}//${urlObj.host}/${dir}/master.m3u8${rawQuery}`;
if (!seen.has(manifestUrl)) {
seen.add(manifestUrl);
reconstructed.push({ url: manifestUrl });
}
} catch { /* skip malformed URLs */ }
}
}
} catch { /* skip malformed URLs */ }
}
return reconstructed;
}
}
/**
* Validate that a manifest URL uses a real phncdn.com path structure.
* Rejects fake paths like /videos/{id}/playlist.m3u8 — real manifests use
* /hls/videos/{year}/{subdir}/{videoId}/master.m3u8 or similar /hls/ paths.
*/
function isValidManifestUrl(url) {
/**
* Validate that a manifest URL is well-formed and generic (CDN-agnostic).
* Rejects malformed paths; a manifest-like URL must live in a directory (have a
* path separator), which filters out obviously-broken single-segment paths.
*/
function isValidManifestUrl(url) {
if (!url || typeof url !== 'string') return false;
// Real phncdn HLS manifest: must contain /hls/videos/ with sufficient path depth
const hlsIdx = url.indexOf('/hls/videos/');
if (hlsIdx === -1) {
// Not phncdn — could be a different CDN, allow it
if (!url.includes('phncdn') && !url.includes('pornhubcdn')) return true;
// phncdn without /hls/ prefix is invalid for manifests
let urlObj;
try {
urlObj = new URL(url);
} catch {
return false;
}
// Check there are at least {year}/{subdir}/{videoId} after /hls/videos/
const after = url.substring(hlsIdx + '/hls/videos/'.length);
if (after.indexOf('/') === -1) return false; // need at least one more segment
return true;
}
}
const path = urlObj.pathname.toLowerCase();
const isManifestLike = path.includes('.m3u8') || path.includes('.hlsvod') ||
(path.includes('/playlist') && !path.endsWith('.ts')) ||
(path.includes('/master') && path.includes('.m3u'));
// Not a manifest path — allow it (it may be a direct media source, not HLS).
if (!isManifestLike) return true;
// A manifest path must sit in a directory, not be a bare top-level file.
return path.includes('/');
}
/**
* Build a normalized video entry from a scraped video object.
@@ -941,8 +905,8 @@ function makeVideoEntry(video, tabTitle, posterFromContent) {
mediaType: video.mediaType || 'video',
};
if (video.hlsManifest) {
// Reject fake phncdn manifest paths before storing
if (!isValidManifestUrl(video.hlsManifest)) {
// Reject fake/invalid manifest paths before storing
if (!isValidManifestUrl(video.hlsManifest)) {
console.warn('[goon] Rejecting fake manifest URL:', video.hlsManifest.substring(0, 120));
} else {
entry.hlsManifest = video.hlsManifest;
@@ -1004,28 +968,28 @@ async function scanVideos(tabsToScan) {
}
await logToPage(`step1.5 (webRequest): found ${totalWebRequestManifests} manifests via network interception`);
// Step 1.6: Strategy 8 — Query phncdn.com requests and reconstruct manifest URLs.
// When the player uses in-memory dynamic HLS (zero .m3u8 network requests), we can
// still detect it by recognizing segment URL patterns from phncdn.com and deriving
// the parent manifest URL: /hls/videos/{id}/master.m3u8
const phncdnPromises = tabsToScan.map(t => queryPhncdnRequests(t.id));
const phncdnResults = await Promise.all(phncdnPromises);
let totalPhncdnManifests = 0;
// Step 1.6: Strategy 8 — Query captured video requests and reconstruct manifest URLs.
// When the player uses in-memory dynamic HLS (zero .m3u8 network requests), we can
// still detect it by recognizing segment URL patterns and deriving the parent
// manifest URL from the segment's directory.
const videoRequestPromises = tabsToScan.map(t => queryVideoRequests(t.id));
const videoRequestResults = await Promise.all(videoRequestPromises);
let totalReconstructedManifests = 0;
for (let i = 0; i < tabsToScan.length; i++) {
const reconstructed = phncdnResults[i]?.reconstructedManifests || [];
const reconstructed = videoRequestResults[i]?.reconstructedManifests || [];
if (reconstructed.length > 0) {
log('scanVideos phncdn: tab', tabsToScan[i].id, 'found', reconstructed.length, 'reconstructed manifests from segment URLs');
}
totalPhncdnManifests += reconstructed.length;
log('scanVideos strategy8: tab', tabsToScan[i].id, 'found', reconstructed.length, 'reconstructed manifests from segment URLs');
}
totalReconstructedManifests += reconstructed.length;
const existing = tabManifestMap.get(tabsToScan[i].id) || [];
for (const url of reconstructed) {
if (!existing.some(m => m.url === url)) {
existing.push({ url, player: 'phncdn_reconstructed' });
existing.push({ url, player: 'hls_reconstructed' });
}
}
}
tabManifestMap.set(tabsToScan[i].id, existing);
}
await logToPage(`step1.6 (phncdn): found ${totalPhncdnManifests} reconstructed manifests from segment URLs`);
await logToPage(`step1.6 (strategy8): found ${totalReconstructedManifests} reconstructed manifests from segment URLs`);
// Step 2: Scrape video elements (all frames) AND query content scripts (in parallel)
const scrapePromises = tabsToScan.map(t =>