这是indexloc提供的服务,不要输入任何密码
Skip to content

[Bug] too many open files error errors after continues subfinder run #345

@003random

Description

@003random

Describe the bug
Ive gotten some errors when running subfinder for some hours on 4 hosts which have around 10k subs each (run it on the first host, then after its finished run it on the second, then the third and then the fourth. and after that, which is around 5 minutes from start, you start all over again in the loop).

The errors:
could not run source anubis: Get "https://jldc.me/anubis/subdomains/redbull.com": di al tcp 172.67.210.148:443: socket: too many open files
could not run source securitytrails: Get "https://api.securitytrails.com/v1/domain/r edbull.com/subdomains": dial tcp: lookup api.securitytrails.com on 10.6.30.8:53: no such host
could not run source certspotterold: Get "https://certspotter.com/api/v0/certs?doma in=redbull.com": dial tcp [2600:1f18:2469:d900:203b:729e:2e54:8c43]:443: socket: too many open files
could not run source crtsh: dial tcp [2a0e:ac00:c7:d449::5bc7:d449]:5432: socket: t oo many open files

The source isnt only securitytrails and Anubis. I got these errors multiple times in 1 run with various sources like alienvault, crt, certspotter, ipv4info, hackertarget, riddler, passivetotal, baidu etc.

Subfinder version
2.4.5

Complete command you used to reproduce this
I used the following code: https://gist.github.com/003random/c7fa23b12c83f136f254f63fe25e688c which imports

        "github.com/projectdiscovery/subfinder/v2/pkg/passive"
	"github.com/projectdiscovery/subfinder/v2/pkg/runner"
	"github.com/projectdiscovery/subfinder/v2/pkg/subscraping"

Metadata

Metadata

Assignees

Labels

Priority: MediumThis issue may be useful, and needs some attention.Type: BugInconsistencies or issues which will cause an issue or problem for users or implementors.

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions