Why is Golang reverse proxy blocked by Cloudflare while Nginx works alright?

小能豆

Why is Golang reverse proxy blocked by Cloudflare while Nginx works alright?

For example, I have this simplest nginx.conf for the Nginx reverse proxy:

worker_processes  1;

events {
    worker_connections  1024;
}


http {
    include       mime.types;
    default_type  application/octet-stream;

    sendfile        on;

    keepalive_timeout  65;

    server {
        listen       8081;

        location / {
          proxy_pass https://icanhazip.com;
        }

        error_page   500 502 503 504  /50x.html;
        location = /50x.html {
            root   html;
        }
    }
}

And this simple reverse proxy program implemented in Go:

package main

import (
    "fmt"
    "log"
    "net/http"
    "net/http/httputil"
    "net/url"
)

func main() {
    testUrl, err := url.Parse("https://icanhazip.com")
    if err != nil {
        log.Fatal(err)
    }

    proxy := httputil.NewSingleHostReverseProxy(testUrl)

    server := http.Server{
        Addr:    fmt.Sprintf(":%d", 8082),
        Handler: http.HandlerFunc(proxy.ServeHTTP),
    }

    if err := server.ListenAndServe(); err != http.ErrServerClosed {
        log.Fatal(err)
    }
}

After I got both services running on my local computer, access to the Nginx reverse proxy service is working without problem, returning my ip address correctly:

# curl http://127.0.0.1:8081
92.**7.1**.112

But access to the Go reverse proxy program result in a Cloudflare error:

# curl http://127.0.0.1:8082
<html>
<head><title>403 Forbidden</title></head>
<body>
<center><h1>403 Forbidden</h1></center>
<hr><center>cloudflare</center>
</body>
</html>

Since they are both running on the same computer and trying to access the same destination URL, it got to be something wrong with this Go reverse proxy program? Anyway I can make the Go reverse proxy program working the same way as Nginx?

So far I have tried the following:

Overriding the Director function to set some headers, for example

    proxy := httputil.NewSingleHostReverseProxy(testUrl)

    originalDirector := proxy.Director
    proxy.Director = func(req *http.Request) {
        originalDirector(req)
        req.Header.Set("Host", req.Host)
    }

Making a new ReverseProxy struct with custom Rewrite with SetXForward and things, for example

    proxy := &httputil.ReverseProxy{
                Rewrite: func(r *httputil.ProxyRequest) {
                    r.SetURL(testUrl)
                    r.Out.Host = r.In.Host
                    r.SetXForwarded()
                    r.Out.Header.Set("Host", r.In.Host)
                },
            }

So far nothing works, the Go reverse proxy program is still blocked by Cloudflare while Nginx works alright.

On another note this Go reverse proxy program does work with local URLs, for example if I change the URL to:

testUrl, err := url.Parse("http://127.0.0.1:8081")

Then it can chain with the Nginx reverse proxy and work as expected. So any idea how to make it work with remote URLs like Nginx does?

阅读 73

2023-12-18

共1个答案

小能豆

The issue you’re encountering is likely related to the fact that Cloudflare is blocking requests made by your Go reverse proxy. Cloudflare and other similar services use various security mechanisms to prevent abuse, and sometimes normal behavior from certain types of applications (like reverse proxies) can trigger these security mechanisms.

Here are a few suggestions you can try to make your Go reverse proxy work with Cloudflare:

1. Set User-Agent Header

Cloudflare might be blocking requests without a proper User-Agent header. Set a User-Agent header in your Go reverse proxy to mimic a standard web browser.

proxy := httputil.NewSingleHostReverseProxy(testUrl)
proxy.Director = func(req *http.Request) {
    req.Header.Set("User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.3")
}

2. Set X-Forwarded-For Header

Some services, including Cloudflare, use the X-Forwarded-For header to determine the original IP address of the client. Set this header in your Go reverse proxy.

proxy := httputil.NewSingleHostReverseProxy(testUrl)
proxy.Director = func(req *http.Request) {
    req.Header.Set("X-Forwarded-For", req.RemoteAddr)
}

3. Adjust the Rewrite Function

You might need to adjust the Rewrite function in your custom ReverseProxy to handle headers more effectively.

proxy := &httputil.ReverseProxy{
    Rewrite: func(r *http.Request) {
        r.Host = testUrl.Host
        r.URL.Scheme = testUrl.Scheme
        r.URL.Host = testUrl.Host
        r.Header.Set("Host", testUrl.Host)
        r.Header.Set("X-Forwarded-For", r.RemoteAddr)
    },
}

4. Set Transport Settings

Adjust the Transport settings to ensure that the connection is reused.

proxy := httputil.NewSingleHostReverseProxy(testUrl)
proxy.Transport = &http.Transport{
    DisableKeepAlives: false,
}

5. Handle Cookies

If the target server relies on cookies, you may need to handle them appropriately.

proxy := httputil.NewSingleHostReverseProxy(testUrl)
proxy.ModifyResponse = func(resp *http.Response) error {
    // Handle and modify cookies if needed
    return nil
}

Try a combination of these suggestions and see if it resolves the issue. Keep in mind that Cloudflare may have additional security mechanisms, and you might need to adjust your Go reverse proxy accordingly. If these suggestions do not work, you may need to consult Cloudflare’s documentation or support for further assistance.

2023-12-18