Web Monitoring & Scraping
Redefined for Speed
& Simplicity.

A zero-dependency, hyper-fast engine to track anything on the web. CSS in, Structured Data out. No heavy runtime required.

~5MB

Idle RAM

0

External Deps

Lua

Logic Hooks

spyweb - bash

Deploy Anywhere.

Dual binary. Zero dependencies. Total control.

Linux

x86_64 / Server

Ideal for headless servers and 24/7 VPS monitoring. Includes systemd-ready binary.

Windows

x86_64 / Desktop

Includes the System Tray app for silent background monitoring and instant notifications.

macOS

Universal / M1 / M2

Native builds optimized for Apple Silicon and Intel-based Macs.

Onboarding

Get Started in Minutes.

1

Unzip & Launch

Download the binary for your system and extract. Zero installers, zero runtimes, just a single, high-performance executable.

2

Configure

Define your targets in jobs.toml. Use simple selectors for basics, or drop in a Lua script for complex extraction logic.

3

Deploy

Fire up the engine. SpyWeb monitors in the background, dedupes data, and alerts you via Slack, Discord, or Desktop notifications.

Right Tool for the Right Job.

Puppeteer and Playwright are the gold standards for End-to-End Testing and QA. SpyWeb is built for the other side of the coin: Continuous Monitoring and Scraping.

TESTING & QA

PUPPETEER

The industry standard for Chromium automation. Built for developers who need to simulate user behavior in a browser environment.

~300MB Package

MONITORING & SCRAPING

SPYWEB

~7MB Static Binary
< 5MB Idle RAM
Raw JSON-RPC CDP Access
Built-in State Engine

SpyWeb doesn't restrict functionality. Execute any raw CDP method directly for total control.

CROSS-BROWSER QA

PLAYWRIGHT

Powerful cross-browser engine for modern web apps. Built to ensure your site works perfectly across WebKit, Firefox, and Chromium.

~1GB Dependency

When you need to test your UI, use Playwright. When you need to monitor the web at scale without melting your servers, use SpyWeb.

Built for Real-World Reliability.

Hyper-Efficient

Stays under 5MB idle RAM, so you can run many jobs on small VPS instances without wasting memory on overhead.

Lua Scripting

Use Lua to handle edge cases, normalize data, and keep complex scraping logic close to the job that needs it.

Hot-Reload

Edit config.toml or hooks.lua and the engine reloads jobs immediately, without stopping the service.

Built-in Intelligence

Persistent storage handles deduplication, state, and history without any extra service to manage.

JS Rendering

Swap HTTP for a real browser via CDP; Chromium, Chrome, Edge, or any CDP-compatible runtime. Spyweb doesn't bundle one. ~7MB binary either way.

Pipeline Telemetry

Track stage timings, memory usage, and browser state so failures and slow steps are easy to isolate.

Alert Anywhere

Send alerts through desktop notifications, webhooks, or raw JSON export for the systems you already use.

Zero Friction

Ship as a CLI or tray app with no runtime dependency, deployment stays simple on servers and desktops alike.

Fault Tolerant

Hook failures stay contained, one bad job does not take down the engine or block the rest of the run.

Lua Testing

Co-located test_* functions run in fresh Lua VMs with temporary databases, keeping validation close to the hooks it covers.

Operational Confidence

Fresh VMs, isolated databases, stage telemetry, and non-fatal errors make production behavior predictable and testable.

Bring Your Own UI

Every SpyWeb instance exposes a local REST API, you can build your own dashboards or integrate with the tools you already run. Read API Guide →

Simple or Scriptable.

config/default
# simple monitoring job that loops on default interval ( runs every 10 minutes )
[[jobs]]
name = "Job Board Scraper"
url = "https://example.com/jobs"
selector = ".job-card"
fields = ["title:h2", "salary:.salary-badge"]
-- Render JS-heavy pages by swapping the HTTP client for a headless Chromium browser
local browser = cdp.launch({
    headless = true,
    keep_alive = true,
})

function override_fetch(request)
    local page = browser:attach()
    defer(function() page:close() end)

    local ok, err = page:open(request.url)
    if not ok then
        return { error = "Navigation failed: " .. err }
    end

    if page:wait_for_selector(".item", 8000) then
        return { status = 200, body = page:content() }
    end

    notify({ title = "Layout Changed", body = request.url })
    return { error = "Selector timeout" }
end

Check out GitHub examples or our documentation for more detailed examples.

Endless Possibilities.

E-Commerce

Track price drops, restocks, and competitor discounts on any shop without an API.

Job Boards

Be the first to apply. Monitor specialized job boards and get instant notifications.

Real Estate

Watch for new listings in specific neighborhoods and skip the aggregator delays.

Crypto & Finance

Monitor whale movements, new token listings, or regulatory updates in real-time.

OSINT & News

Track government sites, court records, or niche news portals for breaking information.

AI Training

Automate the collection of clean, structured datasets for LLM fine-tuning or RAG.

~7MB

Self-Contained

<5MB

Idle Memory

Rust

Core Engine

Luau

Scripting