Surf - Advanced HTTP Client for Go

Surf is a powerful, feature-rich HTTP client library for Go that makes working with HTTP requests intuitive and enjoyable. With advanced features like browser impersonation, JA3/JA4 fingerprinting, and comprehensive middleware support, Surf provides everything you need for modern web interactions.

✨ Key Features

🎭 Browser Impersonation

Chrome & Firefox Support: Accurately mimic Chrome v131 and Firefox v131 browser fingerprints
Platform Diversity: Impersonate Windows, macOS, Linux, Android, and iOS devices
TLS Fingerprinting: Full JA3/JA4 fingerprint customization for enhanced privacy
Automatic Headers: Proper header ordering and browser-specific values
WebKit Form Boundaries: Accurate multipart form boundary generation matching real browsers

🔒 Advanced TLS & Security

Custom JA3/JA4: Configure precise TLS fingerprints with HelloID and HelloSpec
HTTP/2 & HTTP/3: Full HTTP/2 support with customizable settings (SETTINGS frame, window size, priority)
Certificate Pinning: Custom TLS certificate validation
DNS-over-TLS: Enhanced privacy with DoT support
Proxy Support: HTTP, HTTPS, and SOCKS5 proxy configurations with rotation support

🚀 Performance & Reliability

Connection Pooling: Efficient connection reuse with singleton pattern
Automatic Retries: Configurable retry logic with custom status codes
Response Caching: Built-in body caching for repeated access
Streaming Support: Efficient handling of large responses and SSE
Compression: Automatic handling of gzip, deflate, brotli, and zstd
Keep-Alive: Persistent connections with configurable parameters

🛠️ Developer Experience

Standard Library Compatible: Convert to net/http.Client for third-party library integration
Fluent API: Chainable methods for elegant code
Middleware System: Extensible request/response/client middleware with priority support
Type Safety: Strong typing with generics support via enetx/g
Debug Mode: Comprehensive request/response debugging
Error Handling: Result type pattern for better error management
Context Support: Full context.Context integration for cancellation and timeouts

📦 Installation

go get -u github.com/enetx/surf

Required Go version: 1.25+

🔄 Standard Library Compatibility

Surf provides seamless integration with Go's standard net/http package, allowing you to use Surf's advanced features with any library that expects a standard *http.Client.

// Create a Surf client with advanced features
surfClient := surf.NewClient().
    Builder().
    Impersonate().Chrome().
    Session().
    Build()

// Convert to standard net/http.Client
stdClient := surfClient.Std()

// Use with any third-party library
// Example: AWS SDK, Google APIs, OpenAI client, etc.
resp, err := stdClient.Get("https://api.example.com")

Preserved Features When Using Std():

✅ JA3/TLS fingerprinting
✅ HTTP/2 settings
✅ Browser impersonation headers
✅ Cookies and sessions
✅ Proxy configuration
✅ Custom headers and User-Agent
✅ Timeout settings
✅ Redirect policies
✅ Request/Response middleware

Limitations with Std():

❌ Retry logic (implement at application level)
❌ Response body caching
❌ Remote address tracking
❌ Request timing information

🚀 Quick Start

Basic GET Request

package main

import (
    "fmt"
    "log"
    "github.com/enetx/surf"
)

func main() {
    resp := surf.NewClient().Get("https://api.github.com/users/github").Do()
    if resp.IsErr() {
        log.Fatal(resp.Err())
    }

    fmt.Println(resp.Ok().Body.String())
}

JSON Response Handling

type User struct {
    Name     string `json:"name"`
    Company  string `json:"company"`
    Location string `json:"location"`
}

resp := surf.NewClient().Get("https://api.github.com/users/github").Do()
if resp.IsOk() {
    var user User
    resp.Ok().Body.JSON(&user)
    fmt.Printf("User: %+v\n", user)
}

🎭 Browser Impersonation

Chrome Impersonation

client := surf.NewClient().
    Builder().
    Impersonate().
    Chrome().        // Latest Chrome v131
    Build()

resp := client.Get("https://example.com").Do()

Firefox with Random OS

client := surf.NewClient().
    Builder().
    Impersonate().
    RandomOS().      // Randomly selects Windows, macOS, Linux, Android, or iOS
    FireFox().       // Latest Firefox v131
    Build()

Platform-Specific Impersonation

// iOS Chrome
client := surf.NewClient().
    Builder().
    Impersonate().
    IOS().
    Chrome().
    Build()

// Android Chrome
client := surf.NewClient().
    Builder().
    Impersonate().
    Android().
    Chrome().
    Build()

🔧 Advanced Configuration

Custom JA3 Fingerprint

// Use specific browser versions
client := surf.NewClient().
    Builder().
    JA().
    Chrome().     // Latest Chrome
    Build()

// Available browser fingerprints:
// Chrome: Chrome(), Chrome58(), Chrome70(), Chrome83(), Chrome96(), Chrome100(), Chrome102(), Chrome106(), Chrome120(), Chrome131()
// Firefox: Firefox(), Firefox55(), Firefox63(), Firefox99(), Firefox102(), Firefox105(), Firefox120(), Firefox131()
// Edge: Edge(), Edge85(), Edge106()
// Safari: Safari()
// Mobile: IOS(), IOS11(), IOS12(), IOS13(), IOS14(), Android()

// Randomized fingerprints for evasion
client := surf.NewClient().
    Builder().
    JA().
    Randomized().    // Random TLS fingerprint
    Build()

// With custom HelloID
client := surf.NewClient().
    Builder().
    JA().
    SetHelloID(utls.HelloChrome_Auto).
    Build()

// With custom HelloSpec
client := surf.NewClient().
    Builder().
    JA().
    SetHelloSpec(customSpec).
    Build()

HTTP/2 Configuration

client := surf.NewClient().
    Builder().
    HTTP2Settings().
    HeaderTableSize(65536).
    EnablePush(0).
    InitialWindowSize(6291456).
    MaxHeaderListSize(262144).
    ConnectionFlow(15663105).
    Set().
    Build()

Proxy Configuration

// Single proxy
client := surf.NewClient().
    Builder().
    Proxy("http://proxy.example.com:8080").
    Build()

// Rotating proxies
proxies := []string{
    "http://proxy1.example.com:8080",
    "http://proxy2.example.com:8080",
    "socks5://proxy3.example.com:1080",
}

client := surf.NewClient().
    Builder().
    Proxy(proxies).  // Randomly selects from list
    Build()

🔌 Middleware System

Request Middleware

client := surf.NewClient().
    Builder().
    With(func(req *surf.Request) error {
        req.AddHeaders("X-Custom-Header", "value")
        fmt.Printf("Request to: %s\n", req.GetRequest().URL)
        return nil
    }).
    Build()

Response Middleware

client := surf.NewClient().
    Builder().
    With(func(resp *surf.Response) error {
        fmt.Printf("Response status: %d\n", resp.StatusCode)
        fmt.Printf("Response time: %v\n", resp.Time)
        return nil
    }).
    Build()

Client Middleware

client := surf.NewClient().
    Builder().
    With(func(client *surf.Client) {
        // Modify client configuration
        client.GetClient().Timeout = 30 * time.Second
    }).
    Build()

📤 Request Types

POST with JSON

user := map[string]string{
    "name": "John Doe",
    "email": "john@example.com",
}

resp := surf.NewClient().
    Post("https://api.example.com/users", user).
    Do()

Form Data

formData := map[string]string{
    "username": "john",
    "password": "secret",
}

resp := surf.NewClient().
    Post("https://example.com/login", formData).
    Do()

File Upload

// Single file upload
resp := surf.NewClient().
    FileUpload(
        "https://api.example.com/upload",
        "file",                    // field name
        "/path/to/file.pdf",       // file path
    ).Do()

// With additional form fields
extraData := g.MapOrd[string, string]{
    "description": "Important document",
    "category": "reports",
}

resp := surf.NewClient().
    FileUpload(
        "https://api.example.com/upload",
        "file",
        "/path/to/file.pdf",
        extraData,
    ).Do()

Multipart Form

fields := g.NewMapOrd[g.String, g.String]()
fields.Set("field1", "value1")
fields.Set("field2", "value2")

resp := surf.NewClient().
    Multipart("https://api.example.com/form", fields).
    Do()

🔄 Session Management

Persistent Sessions

client := surf.NewClient().
    Builder().
    Session().        // Enable cookie jar
    Build()

// Login
client.Post("https://example.com/login", credentials).Do()

// Subsequent requests will include session cookies
resp := client.Get("https://example.com/dashboard").Do()

Manual Cookie Management

// Set cookies
cookies := []*http.Cookie{
    {Name: "session", Value: "abc123"},
    {Name: "preference", Value: "dark_mode"},
}

resp := surf.NewClient().
    Get("https://example.com").
    AddCookies(cookies...).
    Do()

// Get cookies from response
if resp.IsOk() {
    for _, cookie := range resp.Ok().Cookies {
        fmt.Printf("Cookie: %s = %s\n", cookie.Name, cookie.Value)
    }
}

📊 Response Handling

Status Code Checking

resp := surf.NewClient().Get("https://api.example.com/data").Do()

if resp.IsOk() {
    switch {
    case resp.Ok().StatusCode.IsSuccess():
        fmt.Println("Success!")
    case resp.Ok().StatusCode.IsRedirect():
        fmt.Println("Redirected to:", resp.Ok().Location())
    case resp.Ok().StatusCode.IsClientError():
        fmt.Println("Client error:", resp.Ok().StatusCode)
    case resp.Ok().StatusCode.IsServerError():
        fmt.Println("Server error:", resp.Ok().StatusCode)
    }
}

Body Processing

resp := surf.NewClient().Get("https://example.com/data").Do()
if resp.IsOk() {
    body := resp.Ok().Body

    // As string
    content := body.String()

    // As bytes
    data := body.Bytes()

    // MD5 hash
    hash := body.MD5()

    // UTF-8 conversion
    utf8Content := body.UTF8()

    // Check content
    if body.Contains("success") {
        fmt.Println("Request succeeded!")
    }

    // Save to file
    err := body.Dump("response.html")
}

Streaming Large Responses

resp := surf.NewClient().Get("https://example.com/large-file").Do()
if resp.IsOk() {
    reader := resp.Ok().Body.Stream()

    scanner := bufio.NewScanner(reader)
    for scanner.Scan() {
        fmt.Println(scanner.Text())
    }
}

Server-Sent Events (SSE)

resp := surf.NewClient().Get("https://example.com/events").Do()
if resp.IsOk() {
    resp.Ok().Body.SSE(func(event *sse.Event) bool {
        fmt.Printf("Event: %s, Data: %s\n", event.Name, event.Data)
        return true  // Continue reading (false to stop)
    })
}

🔍 Debugging

Request/Response Debugging

resp := surf.NewClient().
    Get("https://api.example.com").
    Do()

if resp.IsOk() {
    resp.Ok().Debug().
        Request().      // Show request details
        Response(true). // Show response with body
        Print()
}

TLS Information

resp := surf.NewClient().Get("https://example.com").Do()
if resp.IsOk() {
    if tlsInfo := resp.Ok().TLSGrabber(); tlsInfo != nil {
        fmt.Printf("TLS Version: %s\n", tlsInfo.Version)
        fmt.Printf("Cipher Suite: %s\n", tlsInfo.CipherSuite)
        fmt.Printf("Server Name: %s\n", tlsInfo.ServerName)

        for _, cert := range tlsInfo.PeerCertificates {
            fmt.Printf("Certificate CN: %s\n", cert.Subject.CommonName)
        }
    }
}

⚡ Performance Optimization

Connection Reuse with Singleton

// Create a reusable client
client := surf.NewClient().
    Builder().
    Singleton().      // Enable connection pooling
    Impersonate().
    Chrome().
    Build()

// Reuse for multiple requests
for i := 0; i < 100; i++ {
    resp := client.Get("https://api.example.com/data").Do()
    // Process response
}

// Clean up when done
defer client.CloseIdleConnections()

Response Caching

client := surf.NewClient().
    Builder().
    CacheBody().      // Enable body caching
    Build()

resp := client.Get("https://api.example.com/data").Do()
if resp.IsOk() {
    // First access reads from network
    data1 := resp.Ok().Body.Bytes()

    // Subsequent accesses use cache
    data2 := resp.Ok().Body.Bytes()  // No network I/O
}

Retry Configuration

client := surf.NewClient().
    Builder().
    Retry(3, 2*time.Second).           // Max 3 retries, 2 second wait
    RetryCodes(http.StatusTooManyRequests, http.StatusServiceUnavailable).
    Build()

🌐 Advanced Features

H2C (HTTP/2 Cleartext)

// Enable HTTP/2 without TLS
client := surf.NewClient().
    Builder().
    H2C().
    Build()

resp := client.Get("http://localhost:8080/h2c-endpoint").Do()

Custom Headers Order

// Control exact header order for fingerprinting evasion
headers := g.NewMapOrd[g.String, g.String]()
headers.Set("User-Agent", "Custom/1.0")
headers.Set("Accept", "*/*")
headers.Set("Accept-Language", "en-US")
headers.Set("Accept-Encoding", "gzip, deflate")

client := surf.NewClient().
    Builder().
    SetHeaders(headers).  // Headers will be sent in this exact order
    Build()

Custom DNS Resolver

client := surf.NewClient().
    Builder().
    Resolver("8.8.8.8:53").  // Use Google DNS
    Build()

DNS-over-TLS

client := surf.NewClient().
    Builder().
    DNSOverTLS("1.1.1.1:853").  // Cloudflare DoT
    Build()

Unix Domain Sockets

client := surf.NewClient().
    Builder().
    UnixDomainSocket("/var/run/docker.sock").
    Build()

resp := client.Get("http://localhost/v1.24/containers/json").Do()

Network Interface Binding

client := surf.NewClient().
    Builder().
    InterfaceAddr("192.168.1.100").  // Bind to specific IP
    Build()

Raw HTTP Requests

rawRequest := `GET /api/data HTTP/1.1
Host: example.com
User-Agent: Custom/1.0
Accept: application/json

`

resp := surf.NewClient().
    Raw(g.String(rawRequest), "https").
    Do()

📚 API Reference

Client Methods

Method	Description
`NewClient()`	Creates a new HTTP client with defaults
`Get(url, params...)`	Creates a GET request
`Post(url, data)`	Creates a POST request
`Put(url, data)`	Creates a PUT request
`Patch(url, data)`	Creates a PATCH request
`Delete(url, data...)`	Creates a DELETE request
`Head(url)`	Creates a HEAD request
`FileUpload(url, field, path, data...)`	Creates a multipart file upload
`Multipart(url, fields)`	Creates a multipart form request
`Raw(raw, scheme)`	Creates a request from raw HTTP

Builder Methods

Method	Description
`Impersonate()`	Enable browser impersonation
`JA()`	Configure JA3/JA4 fingerprinting
`HTTP2Settings()`	Configure HTTP/2 parameters
`H2C()`	Enable HTTP/2 cleartext
`Proxy(proxy)`	Set proxy configuration (string, []string for rotation)
`DNS(dns)`	Set custom DNS resolver
`DNSOverTLS()`	Configure DNS-over-TLS
`Session()`	Enable cookie jar for sessions
`Singleton()`	Enable connection pooling (reuse client)
`Timeout(duration)`	Set request timeout
`MaxRedirects(n)`	Set maximum redirects
`NotFollowRedirects()`	Disable redirect following
`FollowOnlyHostRedirects()`	Only follow same-host redirects
`ForwardHeadersOnRedirect()`	Forward headers on redirects
`RedirectPolicy(fn)`	Custom redirect policy function
`Retry(max, wait, codes...)`	Configure retry logic
`CacheBody()`	Enable response body caching
`With(middleware, priority...)`	Add middleware
`BasicAuth(auth)`	Set basic authentication
`BearerAuth(token)`	Set bearer token authentication
`UserAgent(ua)`	Set custom user agent
`SetHeaders(headers...)`	Set request headers
`AddHeaders(headers...)`	Add request headers
`AddCookies(cookies...)`	Add cookies
`WithContext(ctx)`	Add context
`ContentType(type)`	Set content type
`GetRemoteAddress()`	Track remote address
`DisableKeepAlive()`	Disable keep-alive
`DisableCompression()`	Disable compression
`ForceHTTP1()`	Force HTTP/1.1
`UnixDomainSocket(path)`	Use Unix socket
`InterfaceAddr(addr)`	Bind to network interface
`Boundary(fn)`	Custom multipart boundary generator
`Std()`	Convert to standard net/http.Client

Request Methods

Method	Description
`Do()`	Execute the request
`WithContext(ctx)`	Add context to request
`SetHeaders(headers...)`	Set request headers
`AddHeaders(headers...)`	Add request headers
`AddCookies(cookies...)`	Add cookies to request

Response Properties

Property	Type	Description
`StatusCode`	`StatusCode`	HTTP status code
`Headers`	`Headers`	Response headers
`Cookies`	`Cookies`	Response cookies
`Body`	`*Body`	Response body
`URL`	`*url.URL`	Final URL after redirects
`Time`	`time.Duration`	Request duration
`ContentLength`	`int64`	Content length
`Proto`	`string`	HTTP protocol version
`Attempts`	`int`	Number of retry attempts

Body Methods

Method	Description
`String()`	Get body as string
`Bytes()`	Get body as bytes
`JSON(v)`	Decode JSON into struct
`XML(v)`	Decode XML into struct
`MD5()`	Calculate MD5 hash
`UTF8()`	Convert to UTF-8
`Stream()`	Get buffered reader
`SSE(fn)`	Process Server-Sent Events
`Dump(file)`	Save to file
`Contains(pattern)`	Check if contains pattern
`Limit(n)`	Limit body size
`Close()`	Close body reader

🤝 Contributing

Contributions are welcome! Please feel free to submit a Pull Request. For major changes, please open an issue first to discuss what you would like to change.

Fork the repository
Create your feature branch (git checkout -b feature/AmazingFeature)
Commit your changes (git commit -m 'Add some AmazingFeature')
Push to the branch (git push origin feature/AmazingFeature)
Open a Pull Request

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

🙏 Acknowledgments

Built with enetx/http for enhanced HTTP functionality
TLS fingerprinting powered by uTLS
Generic utilities from enetx/g

📞 Support

Issues: GitHub Issues
Discussions: GitHub Discussions
Documentation: pkg.go.dev

Made with ❤️ by the Surf contributors

Name		Name	Last commit message	Last commit date
Latest commit History 1 Commit
.github/workflows		.github/workflows
examples		examples
header		header
internal/drainbody		internal/drainbody
pkg		pkg
tests		tests
LICENSE		LICENSE
README.md		README.md
adapter.go		adapter.go
body.go		body.go
builder.go		builder.go
client.go		client.go
cookies.go		cookies.go
debug.go		debug.go
defaults.go		defaults.go
dnsovertls.go		dnsovertls.go
errors.go		errors.go
go.mod		go.mod
go.sum		go.sum
headers.go		headers.go
http2s.go		http2s.go
impersonate.go		impersonate.go
impersonate_os.go		impersonate_os.go
ja.go		ja.go
ja_roundtripper.go		ja_roundtripper.go
middleware_client.go		middleware_client.go
middleware_request.go		middleware_request.go
middleware_response.go		middleware_response.go
request.go		request.go
response.go		response.go
status.go		status.go
tls.go		tls.go

License

tejzpr/surf

Folders and files

Latest commit

History

Repository files navigation

Surf - Advanced HTTP Client for Go

✨ Key Features

🎭 Browser Impersonation

🔒 Advanced TLS & Security

🚀 Performance & Reliability

🛠️ Developer Experience

📦 Installation

🔄 Standard Library Compatibility

🚀 Quick Start

Basic GET Request

JSON Response Handling

🎭 Browser Impersonation

Chrome Impersonation

Firefox with Random OS

Platform-Specific Impersonation

🔧 Advanced Configuration

Custom JA3 Fingerprint

HTTP/2 Configuration

Proxy Configuration

🔌 Middleware System

Request Middleware

Response Middleware

Client Middleware

📤 Request Types

POST with JSON

Form Data

File Upload

Multipart Form

🔄 Session Management

Persistent Sessions

Manual Cookie Management

📊 Response Handling

Status Code Checking

Body Processing

Streaming Large Responses

Server-Sent Events (SSE)

🔍 Debugging

Request/Response Debugging

TLS Information

⚡ Performance Optimization

Connection Reuse with Singleton

Response Caching

Retry Configuration

🌐 Advanced Features

H2C (HTTP/2 Cleartext)

Custom Headers Order

Custom DNS Resolver

DNS-over-TLS

Unix Domain Sockets

Network Interface Binding

Raw HTTP Requests

📚 API Reference

Client Methods

Builder Methods

Request Methods

Response Properties

Body Methods

🤝 Contributing

📄 License

🙏 Acknowledgments

📞 Support

About

Resources

License

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Languages

Packages