+
Skip to content

Conversation

WGH-
Copy link
Collaborator

@WGH- WGH- commented Jan 12, 2022

Go net/http cannot send HTTP requests containing invalid URL encoding in path (e.g. bare percent)
at all[1]. Browsers send a bare percent in such scenario, and do not implicitly autoencode it.

Until the upstream issue is resolved somehow, we have only two alternatives: either fail to fetch such URLs, or at least attempt the autoencoded variant. Lots of webservers handle them the same way, so it's worth trying.

There aren't too many websites with invalid URL encoding in path component, though.

[1] golang/go#29808

Go net/http cannot send HTTP requests containing
invalid URL encoding in path (e.g. bare percent)
at all[1]. Browsers send a bare percent in
such scenario, and do not implicitly autoencode it.

Until the upstream issue is resolved somehow,
we have only two alternatives: either fail
to fetch such URLs, or at least attempt the
autoencoded variant. Lots of webservers
handle them the same way, so it's worth trying.

There aren't too many websites with invalid
URL encoding in path component, though.

[1] golang/go#29808
Copy link
Member

@asciimoo asciimoo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@asciimoo asciimoo merged commit f661193 into gocolly:master Mar 8, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

点击 这是indexloc提供的php浏览器服务,不要输入任何密码和下载