A Puppeteer-based tool for scanning websites to find third-party (or optionally first-party) network requests matching specified patterns, and generate Adblock-formatted rules.

• Scan websites and detect matching third-party or first-party resources
• Output Adblock-formatted blocking rules
• Support for multiple filters per site
• Grouped titles (! ) before site matches, including redirect source and matching regex
• Ignore unwanted domains (global and per-site)
• Block unwanted domains during scan (simulate adblock)
• Support Chrome, Firefox, Safari user agents (desktop or mobile)
• Advanced fingerprint spoofing and referrer header simulation
• Delay, timeout, reload options per site
• Verbose and debug modes
• Dump matched full URLs into matched_urls.log
• Save output in normal Adblock format or localhost (127.0.0.1/0.0.0.0)
• Subdomain handling (collapse to root or full subdomain)
• Optionally match only first-party, third-party, or both
• Enhanced redirect handling with JavaScript and meta refresh detection
• Capture and drive popup/popunder chains (capture_popups + interact_popups) so domains reachable only via a clicked popup still match
• Per-site proxy routing (SOCKS5, SOCKS4, HTTP, HTTPS) with pre-flight health checks

--block-ads makes the scanner block matching requests during the scan (separate from capturing rules) — to keep ad/tracker noise out of the page, speed up loads, or test that a list catches what it should.

Adding lists. Pass one or more EasyList-format filter lists (same syntax as uBlock Origin / EasyList):

# Single list
node nwss.js --block-ads=easylist.txt

# Multiple lists — comma-separated, no spaces
node nwss.js --block-ads=easylist.txt,easyprivacy.txt,mylist.txt

Lists are plain-text network rules — ||doubleclick.net^, /ads/*, ||example.com^$script, etc. Element-hiding/cosmetic rules (##…) don't apply to request blocking and are ignored. The scanned page's own top-level document is never blocked (only sub-resources), so a site whose own domain is in a list still loads.

Engine — js vs rust (--adblock-engine, default js):

The two engines are interchangeable — same rule format, same blocking result; rust is purely a speed option for big lists. If you pass --adblock-engine=rust without adblock-rs installed, install it (npm i adblock-rs) or drop the flag to use js.

# Fast matching over big lists with the Rust engine
npm install adblock-rs
node nwss.js --block-ads=easylist.txt,easyprivacy.txt --adblock-engine=rust

Example:

{
  "ignoreDomains": [
    "googleapis.com",
    "googletagmanager.com"
  ],
  "ignoreDomainsByUrl": [
    "\\/jwplayer\\/"
  ],
  "blockDomainsByUrl": [
    "\\/tracker\\/"
  ],
  "sites": [
    {
      "url": "https://example.com/",
      "userAgent": "chrome",
      "filterRegex": "ads|analytics",
      "resourceTypes": ["script", "xhr", "image"],
      "reload": 2,
      "delay": 5000,
      "timeout": 30000,
      "verbose": 1,
      "debug": 1,
      "interact": true,
      "fingerprint_protection": "random",
      "referrer_headers": {
        "mode": "random_search",
        "search_terms": ["example reviews", "best deals"]
      },
      "custom_headers": {
        "X-Custom-Header": "value"
      },
      "firstParty": 0,
      "thirdParty": 1,
      "subDomains": 0,
      "blocked": [
        "googletagmanager.com",
        ".*tracking.*"
      ]
    }
  ]
}

Window cleanup modes: false (disabled), true (conservative - closes obvious leftovers), "realtime" (continuously cleanup oldest pages when threshold exceeded), "all" (aggressive - closes all content pages after group). Both active modes preserve the main Puppeteer window and wait 16 seconds before cleanup to avoid interfering with active operations.

When a page redirects to a new domain, first-party/third-party detection is based on the final redirected domain, and all intermediate redirect domains (like bit.ly, t.co) are automatically excluded from the generated rules.

Simple formats:

"referrer_headers": "https://google.com/search?q=example"
"referrer_headers": ["url1", "url2"]

Smart modes:

"referrer_headers": {"mode": "random_search", "search_terms": ["reviews"]}
"referrer_headers": {"mode": "social_media"}
"referrer_headers": {"mode": "direct_navigation"}
"referrer_headers": {"mode": "custom", "custom": ["https://news.ycombinator.com/"]}

Disable referrer for specific URLs:

"referrer_disable": ["https://example.com/no-ref", "sensitive-site.com"]

Capture (and optionally drive) the popup/popunder windows that ad and redirect scripts open, so domains reachable only via a popup chain still match filterRegex. The same filterRegex applies to the whole chain — it must contain every pattern you expect along it. Popup capture only fires when the main page is actually clicking, so set interact: true and interact_clicks: true as well.

Route traffic through a VPN for specific sites. Requires sudo privileges. The VPN connection is established before scanning and torn down after the site completes.

Note: VPN modifies system-level routing. During concurrent scanning, all traffic routes through the active tunnel — not just the site that requested it. For isolated per-site VPN, run sites sequentially or use the same VPN config for all concurrent sites.

Authentication: If the .ovpn file already contains credentials (via auth-user-pass /path/to/file or an inline <auth-user-pass> block), no additional config is needed — just provide the config path. The username/password fields are only needed when the .ovpn file has a bare auth-user-pass directive that expects interactive input.

Route traffic through a proxy for specific sites. Supports SOCKS5, SOCKS4, HTTP, and HTTPS proxies. Unlike VPN, proxy routing is per-site-group — only URLs in the same config block use the proxy; other sites connect directly.

Note: Chromium's --proxy-server flag is browser-wide. Sites requiring different proxies (or direct vs proxied) are automatically separated into different browser instances. Tasks are sorted so proxy groups are contiguous to minimise restarts.

Legacy aliases (socks5_proxy, socks5_bypass, socks5_remote_dns, socks5_debug) are supported for backwards compatibility.

SOCKS5 — no auth:

{
  "url": ["https://blocked-site.com/", "https://another-blocked.com/"],
  "proxy": "socks5://127.0.0.1:1080",
  "search_string": ["tracking.js"]
}

HTTP proxy with credentials:

{
  "url": ["https://geo-restricted.com/"],
  "proxy": "http://user:pass@proxy.corp.com:3128",
  "search_string": ["analytics"]
}

SOCKS5 with bypass list and debug:

{
  "url": ["https://target-site.com/"],
  "proxy": "socks5://user:pass@proxy.example.com:9050",
  "proxy_bypass": ["localhost", "127.0.0.1", "*.internal.corp"],
  "proxy_remote_dns": true,
  "proxy_debug": true,
  "search_string": ["tracker"]
}

Mixed direct + proxied in one config:

[
  {
    "url": ["https://direct-site.com/"],
    "search_string": ["ads"]
  },
  {
    "url": ["https://blocked-site.com/"],
    "proxy": "socks5://127.0.0.1:1080",
    "search_string": ["ads"]
  }
]

If a proxy is unreachable, the batch is skipped with a clear error before any navigation is attempted:

[error] [proxy] Unreachable: socks5://127.0.0.1:1080 — Connection refused
[error] [proxy] Skipping 5 URL(s) in this batch

If a proxy fails mid-scan, Chromium's error code is detected and diagnosed:

[error] [proxy] ERR_SOCKS_CONNECTION_FAILED — proxy: socks5://127.0.0.1:1080 — URL: https://example.com/
[error] [proxy] Check: is the proxy running? Are credentials correct? Is the target reachable from the proxy?

Detected error codes: ERR_PROXY_CONNECTION_FAILED, ERR_SOCKS_CONNECTION_FAILED, ERR_TUNNEL_CONNECTION_FAILED, ERR_PROXY_AUTH_UNSUPPORTED, ERR_PROXY_AUTH_REQUESTED, ERR_SOCKS_CONNECTION_HOST_UNREACHABLE, ERR_PROXY_CERTIFICATE_INVALID, ERR_NO_SUPPORTED_PROXIES.

Create a .nwssconfig file in the project root to define CLI settings per config file. When a config filename matches a key, those settings are automatically applied. CLI flags merge with and override .nwssconfig settings.

{
  "configs": {
    "config-clean1.json": {
      "output": "outputfile.txt",
      "max_concurrent": 30,
      "dns_cache": true,
      "cache_requests": true,
      "dumpurls": true,
      "remove_tempfiles": true,
      "color": true
    },
    "config-clean2.json": {
      "output": "outputfile.txt",
      "max_concurrent": 15,
      "dns_cache": true,
      "cache_requests": true,
      "dumpurls": true,
      "remove_tempfiles": true,
      "color": true,
      "debug": true,
      "block_ads": "easylist.txt,easyprivacy.txt"
    }
  }
}

Usage:

node nwss.js config-clean1.json                    # uses .nwssconfig settings
node nwss.js config-clean2.json --debug             # .nwssconfig + debug override
node nwss.js config-other.json --max-concurrent 5   # no match in .nwssconfig, uses CLI flags

Supported settings: output, max_concurrent, dns_cache, cache_requests, dumpurls, remove_tempfiles, color, remove_dupes, compress_logs, debug, silent, verbose, headful, keep_open, dry_run, titles, sub_domains, no_interact, ghost_cursor, plain, cdp, dnsmasq, unbound, privoxy, pihole, eval_on_doc, use_puppeteer_core, use_obscura, ignore_cache, clear_cache, block_ads, compare, localhost, append.

Priority: CLI flags > .nwssconfig > hardcoded defaults.

These options go at the root level of your config.json:

The searchstring parameter supports all characters including special symbols. Only double quotes need JSON escaping:

{
  "searchstring": [
    ")}return n}function N(n,e,r){try{\"function\"==typeof",
    "addEventListener(\"click\",function(){",
    "{\"status\":\"success\",\"data\":[",
    "console.log('Debug: ' + value);",
    "`API endpoint: ${baseUrl}/users`",
    "@media screen and (max-width: 768px)",
    "if(e&&e.preventDefault){e.preventDefault()}",
    "__webpack_require__(/*! ./module */ \"./src/module.js\")",
    "console.log('Hello world')",
    "#header { background-color: #ff0000; }",
    "$(document).ready(function() {",
    "completion: 85% @ $1,500 budget",
    "SELECT * FROM users WHERE id = *",
    "regex: ^[a-z0-9._%+-]+@[a-z0-9.-]+\\.[a-z]{2,}$",
    "typeof window !== 'undefined'"
  ]
}

Character escaping rules:

• " becomes \" (required in JSON)
• \ becomes \\ (if searching for literal backslashes)
• All other characters are used literally: ' ` @ # $ % * ^ [ ] { } ( ) ; = ! ? :

# Scan with default config and output to console
node nwss.js

# Scan and save rules to file
node nwss.js -o blocklist.txt

# Append new rules to existing file
node nwss.js --append -o blocklist.txt

# Clean existing rules and append new ones
node nwss.js --clean-rules --append -o blocklist.txt

# Debug mode with URL dumping and colored output
node nwss.js --debug --dumpurls --color -o rules.txt

# Dry run to see what would be matched
node nwss.js --dry-run --debug

# Validate configuration before running
node nwss.js --validate-config

# Clean rule files
node nwss.js --clean-rules existing_rules.txt

# Maximum stealth scanning
node nwss.js --debug --color -o stealth_rules.txt

# High-performance scanning with custom concurrency
node nwss.js --max-concurrent 12 --cleanup-interval 300 -o rules.txt

{
  "url": [
    "https://popup-heavy-site1.com",
    "https://popup-heavy-site2.com", 
    "https://popup-heavy-site3.com"
  ],
  "filterRegex": "\\.(space|website|tech)\\b",
  "window_cleanup": "all",
  "interact": true,
  "reload": 2,
  "resourceTypes": ["script", "fetch"],
  "comments": "Aggressive cleanup for sites that open many popups"
}

{
  "url": "https://complex-site.com",
  "filterRegex": "analytics|tracking",
  "window_cleanup": true,
  "interact": true,
  "delay": 8000,
  "reload": 3,
  "comments": [
    "Conservative cleanup preserves potentially active content",
    "Good for sites with complex iframe structures"
  ]
}

{
  "url": "https://anti-bot-site.com",
  "interact": true,
  "interact_clicks": true,
  "cursor_mode": "ghost",
  "realistic_click": true,
  "interact_click_count": 3,
  "ghost_cursor_duration": 5000,
  "ghost_cursor_speed": 1.2,
  "fingerprint_protection": "random",
  "filterRegex": "tracking|analytics",
  "comments": "ghost-cursor uses Bezier curves with overshoot for realistic mouse paths"
}

Or enable globally via CLI:

node nwss.js --ghost-cursor --debug -o rules.txt

Ghost-cursor clicks. The cursor moves with cursor_mode: "ghost", but it only clicks when both interact: true and interact_clicks: true are set (same rule as the built-in path). Click behavior:

• realistic_click: true — each press adds hand-tremor during the hold and a mouseup drift, so mousedown ≠ mouseup coordinates (the press is routed through the same humanClick the built-in content clicks use).
• interact_click_count — number of clicks per load (default 3, capped at 20). The default of 3 matters because some ad SDKs swallow the 1st/2nd click as warmup.
• Duration vs. clicks: realistic clicks take ~600–700ms each, and the bezier movement loop reserves up to half of ghost_cursor_duration for them. So the default ghost_cursor_duration: 2000 only fits ~1 click — raise it to roughly interact_click_count × 700 + movement (e.g. 5000–8000) to fit all of them.

Note: ghost-cursor is an optional dependency. Install with npm install ghost-cursor. If not installed, the scanner falls back to the built-in mouse simulation automatically.

{
  "url": "https://shopping-site.com",
  "userAgent": "chrome",
  "fingerprint_protection": "random",
  "referrer_headers": {
    "mode": "random_search",
    "search_terms": ["product reviews", "best deals", "price comparison"]
  },
  "interact": true,
  "delay": 6000,
  "filterRegex": "analytics|tracking|ads"
}

{
  "url": "https://news-site.com",
  "userAgent": "firefox",
  "fingerprint_protection": true,
  "referrer_headers": {"mode": "social_media"},
  "custom_headers": {
    "Accept-Language": "en-US,en;q=0.9"
  },
  "filterRegex": "doubleclick|googletagmanager"
}

{
  "url": "https://tech-blog.com",
  "fingerprint_protection": "random",
  "referrer_headers": {
    "mode": "custom",
    "custom": [
      "https://news.ycombinator.com/",
      "https://www.reddit.com/r/programming/",
      "https://lobste.rs/"
    ]
  }
}

{
  "url": "https://geo-restricted-site.com",
  "openvpn": "/etc/openvpn/us-server.ovpn",
  "filterRegex": "tracking|analytics",
  "userAgent": "chrome",
  "fingerprint_protection": "random"
}

{
  "url": "https://region-locked-site.com",
  "openvpn": {
    "config": "/etc/openvpn/eu-server.ovpn",
    "username": "vpn_user",
    "password": "vpn_pass",
    "connect_timeout": 45000
  },
  "filterRegex": "ads|tracking"
}

{
  "url": "https://another-site.com",
  "vpn": "/etc/wireguard/wg-us.conf",
  "filterRegex": "analytics",
  "userAgent": "firefox"
}

The scanner includes intelligent window management to prevent memory accumulation during long scans:

• Conservative cleanup (window_cleanup: true): Selectively closes pages that appear to be leftovers from previous scans
• Aggressive cleanup (window_cleanup: "all"): Closes all content pages from previous operations for maximum memory recovery
• Main window preservation: Both modes always preserve the main Puppeteer browser window to maintain stability
• Popup window handling: Automatically detects and closes popup windows created by previous site scans
• Timing protection: 16-second delay ensures no active operations are interrupted during cleanup
• Active page protection: Never affects pages currently being processed by concurrent scanning operations
• Memory reporting: Reports estimated memory freed from closed windows for performance monitoring

Use aggressive cleanup for sites that open many popups or when processing large numbers of URLs. Use conservative cleanup when you want to preserve potentially active content but still free obvious leftovers.

wget -q -O - https://dl.google.com/linux/linux_signing_key.pub | sudo gpg --dearmor -o /usr/share/keyrings/googlechrome-linux-keyring.gpg

echo "deb [arch=amd64 signed-by=/usr/share/keyrings/googlechrome-linux-keyring.gpg] http://dl.google.com/linux/chrome/deb/ stable main" | sudo tee /etc/apt/sources.list.d/google-chrome.list

sudo apt update
sudo apt install google-chrome-stable

sudo apt install bind9-dnsutils whois

sudo apt install openvpn

Grant passwordless sudo for OpenVPN operations:

sudo visudo -f /etc/sudoers.d/openvpn-nwss

Add:

your_username ALL=(root) NOPASSWD: /usr/sbin/openvpn, /usr/bin/kill, /usr/bin/pgrep, /usr/bin/pkill

On WSL2, load the TUN module (required each reboot):

sudo modprobe tun

sudo apt install wireguard

Grant passwordless sudo for WireGuard operations:

sudo visudo -f /etc/sudoers.d/wg-nwss

Add:

your_username ALL=(root) NOPASSWD: /usr/bin/wg-quick, /usr/bin/wg

• If both firstParty: 0 and thirdParty: 0 are set for a site, it will be skipped.
• ignoreDomains applies globally across all sites.
• ignoreDomains supports wildcards (e.g., *.ads.com matches tracker.ads.com)
• Blocking (blocked) can match full domains or regex.
• If a site's blocked field is missing, no extra blocking is applied.
• --clean-rules with --append will clean existing files first, then append new rules
• --remove-dupes works with all output modes and removes duplicates from final output
• Validation tools help ensure rule files are properly formatted before use
• --remove-tempfiles removes Chrome/Puppeteer temporary files before exiting, avoids disk space issues
• For maximum stealth, combine fingerprint_protection: "random" with appropriate referrer_headers modes
• User agents are automatically updated to latest versions (Chrome 131, Firefox 133, Safari 18.2)
• Referrer headers work independently from fingerprint protection - use both for best results
• VPN connections (vpn/openvpn) are established before scanning and torn down after the site completes
• If an .ovpn file contains embedded credentials, no additional auth config is needed in the JSON
• VPN affects system-level routing — all concurrent scans will route through the active tunnel
• Both vpn (WireGuard) and openvpn can be set, but vpn takes precedence
• Ghost-cursor (cursor_mode: "ghost") is optional — install with npm i ghost-cursor. Falls back to built-in mouse if not installed
• Ghost-cursor duration defaults to interact_duration (or 2000ms), capped by the 15s hard timeout

scripts/test-stealth.js is a developer-facing smoke test for the fingerprint spoofing stack in lib/fingerprint.js. It launches Puppeteer with the same applyAllFingerprintSpoofing call that nwss.js uses, navigates to public bot-detection pages, and reports what they concluded — pass/warn/fail counts from sannysoft, trust score from creepjs, raw navigator values from browserleaks.

Use it to A/B a stealth change: run before the edit, run after, diff the output. Not a unit test — it doesn't assert; it reports.

node scripts/test-stealth.js                  # all targets, human-readable
node scripts/test-stealth.js sannysoft        # one target only
node scripts/test-stealth.js --no-spoof       # baseline (spoof disabled)
node scripts/test-stealth.js --ua=firefox     # spoof a different UA family
node scripts/test-stealth.js --format=json    # machine-readable for diff/jq
node scripts/test-stealth.js --help           # full flag list

Set PUPPETEER_NO_SANDBOX=1 when running as root (CI containers, some Docker setups). Off by default so local dev doesn't silently drop the Chromium sandbox.