-
Notifications
You must be signed in to change notification settings - Fork 1.5k
Closed
Labels
Type: EnhancementMost issues will probably ask for additions or changes.Most issues will probably ask for additions or changes.
Description
Description
This PR adds a new passive source named urlscan for discovering subdomains via the urlscan.io
Search API.
Changes
- Integrated urlscan into the passive sources registry
- Implemented the Source interface with API request, result parsing, and error handling
- Added support for API key configuration (URLSCAN_API_KEY) from environment/config
- Tested and verified output using the subfinder tool with multiple target domains
Motivation
urlscan.ioprovides archived scanning data that often contains unique subdomains not present in CT logs, DNS datasets, or other existing sources- This improves subdomain coverage and complements current providers like Shodan, Censys, SecurityTrails, etc.
Notes
- Free API tier supports up to 1,000 results per query
- Rate limits are handled with basic retry/backoff logic (429 responses)
- Only public/unlisted scans are considered to respect visibility flags
Looking forward to feedback and suggestions from the maintainers 🙏
Metadata
Metadata
Assignees
Labels
Type: EnhancementMost issues will probably ask for additions or changes.Most issues will probably ask for additions or changes.