urlrecon

Async multi-module URL / domain reconnaissance

v1.7.0
Linux

Quick Start

Install via jcli (recommended)

jcli install urlrecon

If you don't have jcli yet, install it first with curl -fsSL https://cli.johlem.net/tools/jcli/install.sh | bash.

Scan a target

urlrecon --target example.com                                 # default modules, terminal output
urlrecon --target example.com --output json                   # pipe-clean JSON
urlrecon --target example.com --modules headers,tls,waf       # subset
urlrecon --target example.com --modules jsendpoints           # opt-in JS endpoint + secret scan
urlrecon --target example.com --output json --out-file r.json # write to file
urlrecon --file targets.txt --output json                     # bulk: one report per target

Absorbed subcommands

# Certificate Transparency / chain validation (former certchain tool)
urlrecon certchain query example.com
urlrecon certchain validate example.com:443
urlrecon certchain inspect ./leaf.pem

# Historical subdomain inventory + takeover detection (former domaindrift tool)
urlrecon domaindrift history example.com
urlrecon domaindrift takeover example.com

What it does

urlrecon is a passive recon CLI: feed it a URL or domain and it runs a fleet of 19 analysis modules concurrently, returning a structured report. Built in Rust, single binary, zero runtime dependencies. Re-implements + extends the capabilities of the retired headerscan, certchain, and domaindrift tools (security headers, WAF, TLS, redirect-chain, CT-log query, chain validation, and subdomain-takeover detection are now part of urlrecon).

Modules

Modules marked opt-in are skipped by --modules all; pass the name explicitly (e.g. --modules jsendpoints) to run them.

ModuleWhat it does
corsSends a probe with a synthetic Origin and classifies Access-Control-Allow-Origin / -Credentials: reflective origin or *+creds → Critical, null → High, * alone → Low
dnsA / AAAA / MX / NS / TXT / SOA records via hickory-resolver (system config → Cloudflare fallback)
dnssecDirect DNSKEY + DS queries with raw-bytes RDATA parsing. Reports key count, KSK/ZSK split, algorithms, and chain posture (signed+delegated, DS-only → bogus, etc.)
domain_history (opt-in)Historical control signal: crt.sh issuer eras (flags CA churn) + Wayback CDX first/last/max-gap. No API key required
emailauthSPF + DMARC + DKIM posture over DNS TXT. SPF +all Critical, DMARC p=none Medium, multi-string TXT concatenation per RFC 7208 §3.3
geoIP geolocation + ASN / ISP via ipwho.is. Pre-resolves the domain to an A record first for reliability
headersHTTP security header analysis: HSTS, CSP, X-Content-Type-Options, X-Frame-Options, Referrer-Policy. 0–10 risk score
inventoryRoot-page link + media + file-download census. Buckets file links (pdf, doc, spreadsheet, archive, …) so --download can pick them up
jsendpoints (opt-in)Fetches every <script src> from the root (cap 20, 2 MB each) and regex-scans bodies for API paths plus AWS / GitHub / Slack / Stripe / Google / JWT secret shapes. Secrets are redacted in the report
portsCommon-port TCP probe (26 ports). Telnet → Critical, exposed databases → High, RDP/VNC → Medium
redirectsHop-by-hop redirect-chain walk. HTTPS → HTTP downgrade flagged Critical; loops flagged High
robots/robots.txt fetch + parse. Disallow paths matching admin/api/.env/etc. escalate to Medium
shodanOptional API lookup (~/.urlrecon/keys.toml: shodan_key). Open ports + banners + ASN + CVE/CVSS. Degrades to an Info finding when no key is set
sitemap/sitemap.xml fetch with /sitemap_index.xml fallback. Same interesting-path flagging
subdomainscrt.sh certificate-transparency log enumeration. 500-subdomain emit cap with truncation note
techPassive Wappalyzer-style fingerprinting against 33 technologies (CMS, frameworks, JS libs, servers)
tlsTLS handshake via tokio-rustls. Protocol + cipher + cert (subject/issuer/SAN/expiry) with strength tiers
wafPassive WAF / CDN fingerprinting against 10 products (Cloudflare, Akamai, Imperva, Sucuri, Fastly, etc.)
whoisDomain registration via RDAP (modern WHOIS replacement). Registrar, expiry, DNSSEC, EPP status flags

Absorbed subcommands

urlrecon ships the former certchain and domaindrift tools as subcommands rather than the flat recon path. Output formats terminal / json / csv / markdown apply uniformly via --format.

SubcommandWhat it does
certchain query <domain>Query crt.sh CT logs for certificates issued for a domain. --include-expired opt-in
certchain validate <host[:port]>Fetch the live TLS chain and walk it: trust path, expiry, key strength, SAN coverage
certchain inspect <path.pem>Parse a local PEM file (DER follow-up): subject, issuer, validity, SANs, key type
certchain batch <file>Newline-delimited domain list — one CT query per domain
domaindrift history <domain>Historical subdomain inventory via CT logs + Wayback Machine. --include-expired, --limit
domaindrift takeover <domain>Subdomain takeover detection against ~45 service fingerprints (S3, GitHub Pages, Heroku, …)

CLI

FlagWhat it does
-t, --target <URL>Target host, URL, or IP
-f, --file <PATH>Bulk mode — newline-delimited target list (# comments, blank lines ignored)
-M, --modules <LIST>Comma-separated module subset, or all (default; opt-in modules excluded)
-o, --output <FMT>terminal / json / csv / markdown
-O, --out-file <PATH>Write structured output to file (banner suppressed)
--timeout <SECS>Per-request timeout (default 10s)
--concurrency <N>Max concurrent async workers (default 10)
--rate-limit <MS>Minimum milliseconds between requests (default 0)
--download <BUCKETS>Post-recon: fetch inventory.file.* URLs for the listed buckets (pdf,doc,spreadsheet,…)
--download-dir <PATH>Download root (default ./urlrecon-downloads/)
-T, --tuiFull-screen ratatui dashboard
-m, --minimalOne finding per line, pipe-friendly
-v, --verbosePer-module timings on stderr, raw data dump
--no-colorDisable ANSI colors
--self-checkCheck cli.johlem.net for an available update
--self-updateDownload + SHA-256-verify + atomic-rename the latest signed binary (previous → .bak)

Design choices

Authorisation: urlrecon's ports module performs full TCP handshakes against the target's host. Run only against systems you own or have explicit written authorisation to scan. The subdomains and geo modules use third-party APIs (crt.sh and ipwho.is) — review their ToS before automating against client targets.