HTTP proxies: A weird network behavior

2026-06-10T00:00:00+00:00

The other day I encountered an interesting network issue when using busybox wget.

Setup</h2>
My setup consists of a minimal Alpine Linux container and a proxy. The container runs a shell script which uses wget to download a tarball from a remote server. Internet access is not allowed directly from the container and requires the use of HTTP proxy.

Busybox wget</h2>
Busybox wget is a lightweight implementation of wget and it's targeted at embedded devices. It's built with minimal features compared with GNU wget. You can find it available in OpenWRT routers and Alpine Linux containers.

The issue</h2>
One of my shell scripts downloads a tarball using the following command:
`wget -qO file.tgz https://fastdl.mongodb.org/tools/db/mongodb-database-tools-amazon-x86_64-100.6.1.tgz </code></pre> wget</code> is Busybox wget and the -q</code> is the quiet mode which suppresses any output messages except errors. It's important to mention that the script had been working normally for the past months. On this day, the script exited with a non-zero exit code and wget printed the error: wget: too many redirections</code>.`Investigation</h2> When files or pages are moved, HTTP servers can reply with a 301 Moved Permanently</code> along with a new location where the page can be fetched. Usually, clients by default follow-up on these responses by sending a new HTTP request to the new location provided in the servers' response. The error message implies that wget received too many 301 HTTP responses. I removed the quiet flag, essentially enabling the output to be shown and added the -S</code> option to include the HTTP response from the server in the output. wget -SO file.tgz https://fastdl.mongodb.org/tools/db/mongodb-database-tools-amazon-x86_64-100.6.1.tgz Connecting to localhost:8888 (127.0.0.1:8888) HTTP/1.1 301 Moved Permanently Via: 1.1 e32df02fcdc72c0a9541c48adde0e3dc.cloudfront.net (CloudFront), 1.1 tinyproxy (tinyproxy) Server: CloudFront Date: Wed, 10 Jun 2026 17:53:40 GMT Content-Type: text/html Content-Length: 167 Location: https://fastdl.mongodb.org/tools/db/mongodb-database-tools-amazon-x86_64-100.6.1.tgz HTTP/1.1 301 Moved Permanently Via: 1.1 e32df02fcdc72c0a9541c48adde0e3dc.cloudfront.net (CloudFront), 1.1 tinyproxy (tinyproxy) Server: CloudFront Date: Wed, 10 Jun 2026 17:53:40 GMT Content-Type: text/html Content-Length: 167 Location: https://fastdl.mongodb.org/tools/db/mongodb-database-tools-amazon-x86_64-100.6.1.tgz HTTP/1.1 301 Moved Permanently Via: 1.1 491eafc4794fd895b854d7ee08e54d4e.cloudfront.net (CloudFront), 1.1 tinyproxy (tinyproxy) Server: CloudFront Date: Wed, 10 Jun 2026 17:53:40 GMT Content-Type: text/html Content-Length: 167 Location: https://fastdl.mongodb.org/tools/db/mongodb-database-tools-amazon-x86_64-100.6.1.tgz ... wget: too many redirections </code></pre> I trimmed the output presented above since it was too long and I got 16 301 Moved Permanently</code> responses before wget gave up. Weirdly, the Location</code> URL is the same for all 16 responses. My first assumption was a server misconfiguration. Could a server reply with a 301 Moved Permanently</code> and point the new location to the same URL essentially looping the request back to itself? Possible? Yes but unlikely. No one implements HTTP servers anymore since projects like Apache httpd and nginx already provide a good and flexible implementations that covers most of the needs. I opened a shell locally in my laptop and curled the endpoint: curl -O /tmp/file.tgz https://fastdl.mongodb.org/tools/db/mongodb-database-tools-amazon-x86_64-100.6.1.tgz </code></pre> I forgot to add the -L</code> parameter</a> to curl. (-L</code> instructs curl to follow any HTTP redirects) curl</code> is one of the tools that won't follow the 301</code> requests unless you explicit ask for it. To my surprise, curl didn't receive any 301</code> and downloaded the file without any issues. At this point I knew that the debug session needs to switch to the container where this behavior started. I SSHed into the container and installed curl</code> and GNU wget</code> from Alpine Linux repository to replicate the test. Surprinsgly, both curl</code> and GNU wget</code> downloaded the file without issues. Keep in mind that the container is configured to use an http and https proxy through the environment variables http_proxy</code> and https_proxy</code>. All three applications (curl</code>, GNU wget</code> and busybox wget</code>) support proxies by looking those variables. I decided to try a different remote server with the tool that had this weird behavior: busybox wget</code>. My first attempt was to ifconfig.io</code> which replies with the IP address from which your request is arriving: wget -O- https://ifconfig.io </code></pre> What was the outcome? wget</code> outputed my IP address correctly. I'm almost certain that I'm chaising a remote server misconfiguration. One last test to download busybox tarball from their official page and I'm done with this "issue": wget -O /tmp/busybox-1.36.1.tar.bz2 https://busybox.net/downloads/busybox-1.36.1.tar.bz2 Connecting to localhost:8888 (127.0.0.1:8888) HTTP/1.1 301 Moved Permanently Via: 1.1 tinyproxy (tinyproxy) Date: Wed, 10 Jun 2026 20:46:42 GMT Server: Apache Location: https://busybox.net/downloads/busybox-1.36.1.tar.bz2 HTTP/1.1 301 Moved Permanently Via: 1.1 tinyproxy (tinyproxy) Date: Wed, 10 Jun 2026 20:46:42 GMT Server: Apache Location: https://busybox.net/downloads/busybox-1.36.1.tar.bz2 ... wget: too many redirections </code></pre> Ohoo! Same issue! Either I'm unlucky and picked two different misconfigured servers or it's a bug. I have to check the wget</code> source code and understand how wget</code> is sending the requests and handling the 301</code> replies. Before jumping to the code we need to understand how proxies work. How proxies work</h2> (Skip this section if you know how HTTP proxies work.) A proxy acts as a middle layer between clients and destination servers. Proxies can be used for security filtering, content caching and network isolation. POSIX applications look http_proxy</code> and https_proxy</code> environment variables in an attempt to check if a proxy should be used when establishing connections to remote servers. GNU wget</code>, curl</code> and busybox wget</code> have support for proxies by checking these variables. Client-proxy interaction differs depending on the protocol used by the remote server. GET request with proxy</th> </tr> GET http://example.com/page HTTP/1.1 Host: example.com </pre> </td> </tr> </table> </div> GET request without proxy</th> </tr> GET /page HTTP/1.1 Host: example.com </pre> </td> </tr> </table> </div> If the remote server is using HTTP, then a client can issue a GET request to the proxy like the one shown left. HTTP reference</a> state that GET requests should use an absolute path in requests to an server and an absolute URL in requests to proxies. If no proxy is used, then the client issues a GET request directly to the server as shown right. If the remote server is using HTTPS, then the proxy request (left) won't work. HTTPS uses TLS which requires the client to initiate a handshake with the server to establish a secure channel. The client has to initiate the handshake directly to the server otherwise it cannot verify the server's authenticity. Proxies, by their nature are effectively man-in-the-middle the connection between both ends. TLS is going to be disrupted and the connection is going to be terminated. To address this problem the HTTP reference</a> has a CONNECT</code> method. The CONNECT</code> can only be used by clients to request proxies the establish of a blind tunnel to remote destinations. When accessing a HTTPS server over a proxy, the client starts by issuing a CONNECT</code> request to the proxy: CONNECT example.com:443 HTTP/1.1 </code></pre> And the server usually replies with: HTTP/1.1 200 Connection established </code></pre> From this point, the proxy expects to receive raw TCP data from the client. The traffic is sent unmodified to the server. Likewise, any traffic sent by the server is forwarded unmodified back to the client until one side closes the connection to the proxy. Scouting wget source code</h2> While getting used to the wget source code, I was puzzled that I couldn't find the CONNECT request. wget supports two SSL implementations. Either through the OpenSSL dependency or, if unavailable, their own SSL implementation. /* openssl (and maybe internal TLS) support is configured / if (server.protocol == P_HTTPS) { / openssl-based helper * Inconvenient API since we can't give it an open fd / int fd = spawn_https_helper_openssl(server.host, server.port); # if ENABLE_FEATURE_WGET_HTTPS if (fd < 0) { / no openssl? try internal / sfp = open_socket(lsa); spawn_ssl_client(server.host, fileno(sfp), /flags/ 0); goto socket_opened; } </code></pre> spawn_https_helper_openssl</code> is a helper function that wraps and delegates the SSL implementation to OpenSSL. Interestingly, the function creates a socket pair and forks the process into a child process. static int spawn_https_helper_openssl(const char host, unsigned port) { char allocated = NULL; char servername; int sp[2]; int pid; IF_FEATURE_WGET_HTTPS(volatile int child_failed = 0;) if (socketpair(AF_UNIX, SOCK_STREAM, 0, sp) != 0) ... fflush_all(); pid = xvfork(); if (pid == 0) { /* Child / char argv[13]; char **argp; close(sp[0]); xmove_fd(sp[1], 0); xdup2(0, 1); xmove_fd(2, 3); xopen("/dev/null", O_RDWR); memset(&argv, 0, sizeof(argv)); argv[0] = (char)"openssl"; argv[1] = (char)"s_client"; argv[2] = (char)"-quiet"; argv[3] = (char)"-connect"; argv[4] = (char*)host; ... BB_EXECVP(argv[0], argv); ... </code></pre> The child process redirects stdin</code> and stdout</code> to the socket pair and invokes the OpenSSL binary through execve</code></a>. Any data sent from the parent process to the child will end up as input for the OpenSSL process. Likewise, any data outputted by OpenSSL will be read by the parent process. I'm still unsure why this implementation was prefered over using an OpenSSL library (libssl). If you know why, let me know. wget is built by default with ENABLE_FEATURE_WGET_OPENSSL</code> enabled and OpenSSL is used if available on the device. wget simply sends HTTP requests on top of a connection established by OpenSSL and receives the raw HTTP responses back. I don't feel like messing with the OpenSSL code (just yet) but I shouldn't be the first person having this use case. A quick look into the openssl s_client</code> documentation</a> reveals the -proxy</code> argument: When used with the -connect</code> flag, the program uses the host and port specified with this flag and issues an HTTP CONNECT</code> command to connect to the desired server. </blockquote> A quick test to the -proxy</code> confirms that it solves the bug. We are delegating the CONNECT</code> request to the OpenSSL binary. Moving on, I needed to see what happens when OpenSSL is not available or if wget was built without OpenSSL support thus ENABLE_FEATURE_WGET_OPENSSL=0</code>. In that case, spawn_ssl_client</code> function is used instead and follows the same inter-process communication as the OpenSSL helper. Instead of OpenSSL binary, a busybox ssl_client</code> application is spawned instead. It felt so easy having a convenient -proxy</code> OpenSSL argument but the ssl_client</code> doesn't have any equivalent argument. Cool! A proper challenge this time. After inspecting the code, CONNECT</code> needs to be send before establishing the TLS connection. The main challenge was to send the CONNECT</code>, check for the proxy's reply which should be 200 Connection established</code> and only then start the spawn_ssl_client</code>. Patch submission</h2> I submitted a patch to the Busybox mailing list</a>. Was my first time using the Git over email workflow via git send-email</code>. The concept wasn't new to me, but I never actually used it in the past. Maybe one day I'll understand why some folkes prefer Git email over the usual Issue/Pull request approach. To me, the latter is just more convenient. This article is a part one of two articles in the "HTTP proxies" series. Follow up on part two</a>. </div> Welcome 2026-05-12T00:00:00+00:00 Welcome to my blog! This blog will be about whatever comes to my mind. It can be about my life or any topics of my interest. You can subscribe to the Atom feed using your favorite article reader. The page's design is minimal with no major frameworks and no JavaScript. Why? Since this is a personal page, it should reflect me on both the content and the layout. Both were designed by me from scratch, following the principle of keeping it simple. I have nothing against JavaScript, or any other tech stack :). I might use it in the future, if/when I need additional functionality. For the moment I don't need anything more that HTML and CSS. The font used is the "Sensation" by Bernd Montag</a>. All icons used are Bootstrap Icons by the Bootstrap Authors</a>.

Welcome

Vítor S. Nunes - Blog

HTTP proxies: A tiny mistake in tinyproxy

HTTP proxies: A weird network behavior

Setup</h2>
My setup consists of a minimal Alpine Linux container and a proxy. The container runs a shell script which uses wget to download a tarball from a remote server. Internet access is not allowed directly from the container and requires the use of HTTP proxy.</p>

Busybox wget</h2>
Busybox wget is a lightweight implementation of wget and it's targeted at embedded devices. It's built with minimal features compared with GNU wget. You can find it available in OpenWRT routers and Alpine Linux containers.</p>

Vítor S. Nunes - Blog

HTTP proxies: A tiny mistake in tinyproxy

HTTP proxies: A weird network behavior

Setup</h2> My setup consists of a minimal Alpine Linux container and a proxy. The container runs a shell script which uses wget to download a tarball from a remote server. Internet access is not allowed directly from the container and requires the use of HTTP proxy.</p>

Busybox wget</h2> Busybox wget is a lightweight implementation of wget and it's targeted at embedded devices. It's built with minimal features compared with GNU wget. You can find it available in OpenWRT routers and Alpine Linux containers.</p>

The issue</h2> One of my shell scripts downloads a tarball using the following command:</p>

Scouting wget source code</h2> While getting used to the wget source code, I was puzzled that I couldn't find the CONNECT request.</p> wget supports two SSL implementations. Either through the OpenSSL dependency or, if unavailable, their own SSL implementation.</p>

Welcome

Setup</h2>
My setup consists of a minimal Alpine Linux container and a proxy. The container runs a shell script which uses wget to download a tarball from a remote server. Internet access is not allowed directly from the container and requires the use of HTTP proxy.</p>

Busybox wget</h2>
Busybox wget is a lightweight implementation of wget and it's targeted at embedded devices. It's built with minimal features compared with GNU wget. You can find it available in OpenWRT routers and Alpine Linux containers.</p>