Vítor S. Nunes
‹ Blog

HTTP proxies: A weird network behavior

8 mins read 2026-06-10

The other day I encountered an interesting network issue when using busybox wget.

Setup

My setup consists of a minimal Alpine Linux container and a proxy. The container runs a shell script which uses wget to download a tarball from a remote server. Internet access is not allowed directly from the container and requires the use of HTTP proxy.

Busybox wget

Busybox wget is a lightweight implementation of wget and it's targeted at embedded devices. It's built with minimal features compared with GNU wget. You can find it available in OpenWRT routers and Alpine Linux containers.

The issue

One of my shell scripts downloads a tarball using the following command:

wget -qO file.tgz https://fastdl.mongodb.org/tools/db/mongodb-database-tools-amazon-x86_64-100.6.1.tgz

wget is Busybox wget and the -q is the quiet mode which suppresses any output messages except errors.

It's important to mention that the script had been working normally for the past months. On this day, the script exited with a non-zero exit code and wget printed the error: wget: too many redirections.

Investigation

When files or pages are moved, HTTP servers can reply with a 301 Moved Permanently along with a new location where the page can be fetched. Usually, clients by default follow-up on these responses by sending a new HTTP request to the new location provided in the servers' response.

The error message implies that wget received too many 301 HTTP responses. I removed the quiet flag, essentially enabling the output to be shown and added the -S option to include the HTTP response from the server in the output.

wget -SO file.tgz https://fastdl.mongodb.org/tools/db/mongodb-database-tools-amazon-x86_64-100.6.1.tgz
Connecting to localhost:8888 (127.0.0.1:8888)
  HTTP/1.1 301 Moved Permanently
  Via: 1.1 e32df02fcdc72c0a9541c48adde0e3dc.cloudfront.net (CloudFront), 1.1 tinyproxy (tinyproxy)
  Server: CloudFront
  Date: Wed, 10 Jun 2026 17:53:40 GMT
  Content-Type: text/html
  Content-Length: 167
  Location: https://fastdl.mongodb.org/tools/db/mongodb-database-tools-amazon-x86_64-100.6.1.tgz
  HTTP/1.1 301 Moved Permanently
  Via: 1.1 e32df02fcdc72c0a9541c48adde0e3dc.cloudfront.net (CloudFront), 1.1 tinyproxy (tinyproxy)
  Server: CloudFront
  Date: Wed, 10 Jun 2026 17:53:40 GMT
  Content-Type: text/html
  Content-Length: 167
  Location: https://fastdl.mongodb.org/tools/db/mongodb-database-tools-amazon-x86_64-100.6.1.tgz
  HTTP/1.1 301 Moved Permanently
  Via: 1.1 491eafc4794fd895b854d7ee08e54d4e.cloudfront.net (CloudFront), 1.1 tinyproxy (tinyproxy)
  Server: CloudFront
  Date: Wed, 10 Jun 2026 17:53:40 GMT
  Content-Type: text/html
  Content-Length: 167
  Location: https://fastdl.mongodb.org/tools/db/mongodb-database-tools-amazon-x86_64-100.6.1.tgz
...
wget: too many redirections

I trimmed the output presented above since it was too long and I got 16 301 Moved Permanently responses before wget gave up. Weirdly, the Location URL is the same for all 16 responses. My first assumption was a server misconfiguration. Could a server reply with a 301 Moved Permanently and point the new location to the same URL essentially looping the request back to itself? Possible? Yes but unlikely. No one implements HTTP servers anymore since projects like Apache httpd and nginx already provide a good and flexible implementations that covers most of the needs.

I opened a shell locally in my laptop and curled the endpoint:

curl -O /tmp/file.tgz https://fastdl.mongodb.org/tools/db/mongodb-database-tools-amazon-x86_64-100.6.1.tgz

I forgot to add the -L parameter to curl. (-L instructs curl to follow any HTTP redirects) curl is one of the tools that won't follow the 301 requests unless you explicit ask for it. To my surprise, curl didn't receive any 301 and downloaded the file without any issues.

At this point I knew that the debug session needs to switch to the container where this behavior started. I SSHed into the container and installed curl and GNU wget from Alpine Linux repository to replicate the test. Surprinsgly, both curl and GNU wget downloaded the file without issues. Keep in mind that the container is configured to use an http and https proxy through the environment variables http_proxy and https_proxy. All three applications (curl, GNU wget and busybox wget) support proxies by looking those variables.

I decided to try a different remote server with the tool that had this weird behavior: busybox wget. My first attempt was to ifconfig.io which replies with the IP address from which your request is arriving:

wget -O- https://ifconfig.io

What was the outcome? wget outputed my IP address correctly. I'm almost certain that I'm chaising a remote server misconfiguration. One last test to download busybox tarball from their official page and I'm done with this "issue":

wget -O /tmp/busybox-1.36.1.tar.bz2 https://busybox.net/downloads/busybox-1.36.1.tar.bz2
Connecting to localhost:8888 (127.0.0.1:8888)
  HTTP/1.1 301 Moved Permanently
  Via: 1.1 tinyproxy (tinyproxy)
  Date: Wed, 10 Jun 2026 20:46:42 GMT
  Server: Apache
  Location: https://busybox.net/downloads/busybox-1.36.1.tar.bz2
  HTTP/1.1 301 Moved Permanently
  Via: 1.1 tinyproxy (tinyproxy)
  Date: Wed, 10 Jun 2026 20:46:42 GMT
  Server: Apache
  Location: https://busybox.net/downloads/busybox-1.36.1.tar.bz2
...
wget: too many redirections

Ohoo! Same issue! Either I'm unlucky and picked two different misconfigured servers or it's a bug. I have to check the wget source code and understand how wget is sending the requests and handling the 301 replies. Before jumping to the code we need to understand how proxies work.

How proxies work

(Skip this section if you know how HTTP proxies work.)

A proxy acts as a middle layer between clients and destination servers. Proxies can be used for security filtering, content caching and network isolation. POSIX applications look http_proxy and https_proxy environment variables in an attempt to check if a proxy should be used when establishing connections to remote servers. GNU wget, curl and busybox wget have support for proxies by checking these variables.

Client-proxy interaction differs depending on the protocol used by the remote server.

GET request with proxy 
    GET http://example.com/page HTTP/1.1
    Host: example.com
GET request without proxy 
    GET /page HTTP/1.1
    Host: example.com

If the remote server is using HTTP, then a client can issue a GET request to the proxy like the one shown left. HTTP reference state that GET requests should use an absolute path in requests to an server and an absolute URL in requests to proxies. If no proxy is used, then the client issues a GET request directly to the server as shown right.

If the remote server is using HTTPS, then the proxy request (left) won't work. HTTPS uses TLS which requires the client to initiate a handshake with the server to establish a secure channel. The client has to initiate the handshake directly to the server otherwise it cannot verify the server's authenticity. Proxies, by their nature are effectively man-in-the-middle the connection between both ends. TLS is going to be disrupted and the connection is going to be terminated. To address this problem the HTTP reference has a CONNECT method. The CONNECT can only be used by clients to request proxies the establish of a blind tunnel to remote destinations. When accessing a HTTPS server over a proxy, the client starts by issuing a CONNECT request to the proxy:

CONNECT example.com:443 HTTP/1.1

And the server usually replies with:

HTTP/1.1 200 Connection established

From this point, the proxy expects to receive raw TCP data from the client. The traffic is sent unmodified to the server. Likewise, any traffic sent by the server is forwarded unmodified back to the client until one side closes the connection to the proxy.

Scouting wget source code

While getting used to the wget source code, I was puzzled that I couldn't find the CONNECT request.

wget supports two SSL implementations. Either through the OpenSSL dependency or, if unavailable, their own SSL implementation.

    /* openssl (and maybe internal TLS) support is configured */
        if (server.protocol == P_HTTPS) {
          /* openssl-based helper
          * Inconvenient API since we can't give it an open fd
          */
          int fd = spawn_https_helper_openssl(server.host, server.port);
    # if ENABLE_FEATURE_WGET_HTTPS
          if (fd < 0) { /* no openssl? try internal */
            sfp = open_socket(lsa);
            spawn_ssl_client(server.host, fileno(sfp), /*flags*/ 0);
            goto socket_opened;
          }

spawn_https_helper_openssl is a helper function that wraps and delegates the SSL implementation to OpenSSL. Interestingly, the function creates a socket pair and forks the process into a child process.

static int spawn_https_helper_openssl(const char *host, unsigned port)
{
	char *allocated = NULL;
	char *servername;
	int sp[2];
	int pid;
	IF_FEATURE_WGET_HTTPS(volatile int child_failed = 0;)

	if (socketpair(AF_UNIX, SOCK_STREAM, 0, sp) != 0)
  ...

	fflush_all();
	pid = xvfork();
	if (pid == 0) {
		/* Child */
		char *argv[13];
		char **argp;

		close(sp[0]);
		xmove_fd(sp[1], 0);
		xdup2(0, 1);

		xmove_fd(2, 3);
		xopen("/dev/null", O_RDWR);
		memset(&argv, 0, sizeof(argv));
		argv[0] = (char*)"openssl";
		argv[1] = (char*)"s_client";
		argv[2] = (char*)"-quiet";
		argv[3] = (char*)"-connect";
		argv[4] = (char*)host;
  ...
  BB_EXECVP(argv[0], argv);
  ...

The child process redirects stdin and stdout to the socket pair and invokes the OpenSSL binary through execve. Any data sent from the parent process to the child will end up as input for the OpenSSL process. Likewise, any data outputted by OpenSSL will be read by the parent process. I'm still unsure why this implementation was prefered over using an OpenSSL library (libssl). If you know why, let me know.

wget is built by default with ENABLE_FEATURE_WGET_OPENSSL enabled and OpenSSL is used if available on the device. wget simply sends HTTP requests on top of a connection established by OpenSSL and receives the raw HTTP responses back.

I don't feel like messing with the OpenSSL code (just yet) but I shouldn't be the first person having this use case. A quick look into the openssl s_client documentation reveals the -proxy argument:

When used with the -connect flag, the program uses the host and port specified with this flag and issues an HTTP CONNECT command to connect to the desired server.

A quick test to the -proxy confirms that it solves the bug. We are delegating the CONNECT request to the OpenSSL binary.

Moving on, I needed to see what happens when OpenSSL is not available or if wget was built without OpenSSL support thus ENABLE_FEATURE_WGET_OPENSSL=0. In that case, spawn_ssl_client function is used instead and follows the same inter-process communication as the OpenSSL helper. Instead of OpenSSL binary, a busybox ssl_client application is spawned instead. It felt so easy having a convenient -proxy OpenSSL argument but the ssl_client doesn't have any equivalent argument. Cool! A proper challenge this time.

After inspecting the code, CONNECT needs to be send before establishing the TLS connection. The main challenge was to send the CONNECT, check for the proxy's reply which should be 200 Connection established and only then start the spawn_ssl_client.

Patch submission

I submitted a patch to the Busybox mailing list. Was my first time using the Git over email workflow via git send-email. The concept wasn't new to me, but I never actually used it in the past. Maybe one day I'll understand why some folkes prefer Git email over the usual Issue/Pull request approach. To me, the latter is just more convenient.

This article is a part one of two articles in the "HTTP proxies" series. Follow up on part two.
© 2026 Vítor S. Nunes