-
Notifications
You must be signed in to change notification settings - Fork 1.5k
Description
Describe the bug
Ive gotten some errors when running subfinder for some hours on 4 hosts which have around 10k subs each (run it on the first host, then after its finished run it on the second, then the third and then the fourth. and after that, which is around 5 minutes from start, you start all over again in the loop).
The errors:
could not run source anubis: Get "https://jldc.me/anubis/subdomains/redbull.com": di al tcp 172.67.210.148:443: socket: too many open files
could not run source securitytrails: Get "https://api.securitytrails.com/v1/domain/r edbull.com/subdomains": dial tcp: lookup api.securitytrails.com on 10.6.30.8:53: no such host
could not run source certspotterold: Get "https://certspotter.com/api/v0/certs?doma in=redbull.com": dial tcp [2600:1f18:2469:d900:203b:729e:2e54:8c43]:443: socket: too many open files
could not run source crtsh: dial tcp [2a0e:ac00:c7:d449::5bc7:d449]:5432: socket: t oo many open files
The source isnt only securitytrails and Anubis. I got these errors multiple times in 1 run with various sources like alienvault, crt, certspotter, ipv4info, hackertarget, riddler, passivetotal, baidu etc.
Subfinder version
2.4.5
Complete command you used to reproduce this
I used the following code: https://gist.github.com/003random/c7fa23b12c83f136f254f63fe25e688c which imports
"github.com/projectdiscovery/subfinder/v2/pkg/passive"
"github.com/projectdiscovery/subfinder/v2/pkg/runner"
"github.com/projectdiscovery/subfinder/v2/pkg/subscraping"