Can an AI agent use ScriptMasterLabs for web scraping or extraction without a monthly subscription?

Yes. Current ScriptMasterLabs web access includes narrow pay-per-use jobs for one public page: structured web extraction and URL-to-Markdown. The caller reads fresh x402 payment requirements immediately before authorizing a paid call.

CRAWLTOLL is ScriptMasterLabs' x402 crawler-paywall project for monetizing automated access while also serving as a discovery bridge to current SML web extraction jobs for AI agents.

Does the public x402 Gateway expose crawl_paid_fetch?

No. The canonical public x402 Gateway is intentionally a narrow discovery and payment-requirement inspection product. crawl_paid_fetch remains part of the explicit full compatibility profile in source and is not advertised as a current public Gateway tool.

← SCRIPTMASTERLABS

WEB SCRAPING API FOR AI AGENTS · X402 CRAWLER PAYWALL

CRAWLTOLL

Two-way infrastructure for the agentic web.

For AI agents: discover narrow pay-per-use web extraction jobs that turn a known public URL into structured data or clean LLM-ready Markdown. For publishers: use the CRAWLTOLL x402 pattern to put machine-readable payment terms in front of automated crawler access while human access can remain free.

web scraping APIAI agentsweb extractionURL to Markdownx402MCPBase USDC

SIDE A · AI AGENTS / CONSUMERS

Scrape or extract one public page on demand

When an agent already knows the source URL, use the smallest current SML job that matches the required output.

Structured extraction: one public URL → machine-readable structured output.

Page: https://scriptmasterlabs.com/web-scraping-api-for-ai-agents
Contract: https://squeezeos-api.onrender.com/api/web/once-info
Free preview: https://squeezeos-api.onrender.com/api/web/demo

LLM-ready Markdown: one public URL → compact Markdown suitable for agent context and RAG ingestion.

Page: https://scriptmasterlabs.com/url-to-markdown-api
Contract: https://squeezeos-api.onrender.com/api/web/reader-info
Execute: GET /api/web/read-markdown?url={public_https_url}

Web Scraping API URL to Markdown Web Research

SIDE B · PUBLISHERS

Put an x402 toll booth in front of crawler traffic

CRAWLTOLL is the publisher-side pattern: automated clients can receive an HTTP 402 challenge with machine-readable payment requirements while publisher policy determines which machine traffic is gated.

protocol = x402 v2
network = eip155:8453
asset = USDC
payTo = 0xc29185fa176357612f3194735753e520e91adc46
authority = https://squeezeos-api.onrender.com/.well-known/x402

Fresh target payment requirements are authoritative. A 402 response is a payment requirement, not settlement proof.

View CRAWLTOLL package Buyer evidence

MCP + SOURCE TRUTH

Where `crawl_paid_fetch` fits

GitLab source contains a real crawl_paid_fetch implementation with server-side fetch/extraction and SSRF guarding. It belongs to the explicit full compatibility profile.

It is not part of the canonical public x402 Gateway profile. The public Gateway intentionally stays narrow: discover x402-paid APIs and inspect fresh payment requirements while the caller controls spending policy. That distinction prevents old broad-tool identity from leaking back into current machine metadata.

Canonical x402 Gateway: https://mcp-x402.onrender.com/mcp
SqueezeOS MCP: https://squeezeos-api.onrender.com/mcp
Live web extraction: https://squeezeos-api.onrender.com/api/web/once-info
Live URL-to-Markdown: https://squeezeos-api.onrender.com/api/web/reader-info

WHICH JOB SHOULD AN AGENT CHOOSE?

Known URL, unknown source, or whole-site crawl?

Known URL + structured fields: use Web Extract One-Shot.

Known URL + LLM-ready context: use URL-to-Markdown / Agent Web Access.

Unknown source + evidence needed: use Web Research.

Whole-site crawling, heavy JavaScript interaction, massive indexing, browser automation, or reserved enterprise throughput: use a conventional crawl/browser platform. SML does not disguise a narrow pay-per-call primitive as a bulk crawler.

PAYMENT TRUTH

Base mainnet only for current SML authority

The current ScriptMasterLabs payment authority is x402 v2 · USDC · Base mainnet (eip155:8453). Historical XRPL/RLUSD or multi-rail material must not override the fresh target challenge.

SCRIPTMASTERLABS · Truth First. Proof Always. Pay Only for Accepted Delivery.