这是indexloc提供的服务,不要输入任何密码
Skip to content

Add urlscan.io #1643

@deehyeon

Description

@deehyeon

Description

This PR adds a new passive source named urlscan for discovering subdomains via the urlscan.io
Search API.

Changes

  • Integrated urlscan into the passive sources registry
  • Implemented the Source interface with API request, result parsing, and error handling
  • Added support for API key configuration (URLSCAN_API_KEY) from environment/config
  • Tested and verified output using the subfinder tool with multiple target domains

Motivation

  • urlscan.io provides archived scanning data that often contains unique subdomains not present in CT logs, DNS datasets, or other existing sources
  • This improves subdomain coverage and complements current providers like Shodan, Censys, SecurityTrails, etc.

Notes

  • Free API tier supports up to 1,000 results per query
  • Rate limits are handled with basic retry/backoff logic (429 responses)
  • Only public/unlisted scans are considered to respect visibility flags

Looking forward to feedback and suggestions from the maintainers 🙏

Metadata

Metadata

Assignees

Labels

Type: EnhancementMost issues will probably ask for additions or changes.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions