Low CPU Usage, Slow Responses: Investigating Gunicorn's Write Waits

The main configuration in this article uses two Gunicorn synchronous workers, each writing one response at a time. We used a small Python application written without a framework to measure how a slow receiver affects responses to other requests.

A synchronous worker does not handle its next request until it finishes writing the current response. A worker is a program responsible for processing requests. What happens to another request when both workers cannot finish writing because the receivers read slowly?

In this configuration, two simultaneous slow downloads increased the response time of another endpoint from about 1.6ms to about 5.9 seconds. The metric is p95: the value that contains 95% of response times when they are sorted from shortest to longest. Representative values below are the median of the p95 values from three runs of each condition.

With just one download, the other endpoint's p95 was about 1.7ms. Adding NGINX between the application and receivers, and enabling it to read ahead and buffer responses, brought p95 to about 2.3ms with the same two synchronous workers. The download client's completion time was about eight seconds, as set by its read-rate control.

These results do not establish a property of all web applications or rank web frameworks. In a supplementary condition using the same application with four threads, p95 was about 1.5ms even without NGINX. We did not measure an asynchronous server.

First, the conditions we actually measured

Response time for the other endpoint runs from the start of its TCP connection attempt to receipt of its 12-byte body. Each condition has three runs of 72 requests. CPU and memory were also capped for the experiment; the limits are listed in the configuration section at the end.

This experiment cannot tell us that another framework or an asynchronous server would slow down at the same request count, or that NGINX is always necessary. Below, we examine the differences in the main configuration in this table and the mechanisms that help explain them.

Run downloads alongside a response that is quick to produce

We provided two endpoints in the same application.

/fast is an API: an endpoint that a program calls to receive a result. We refer to these calls as the “other requests.” There are no database calls or heavy computations, so the response is quick to produce. MiB is a data-size unit: 1MiB is 1,048,576 bytes.

A client is the program that sends a request to a server and receives its response. A browser is one example. Here, experimental programs stand in for a person downloading data and another person making requests.

Gunicorn's sync workers handle one request at a time in the main configuration. The design documentation for the version used describes this behavior. Even with the same framework, the server's execution model and worker count affect how many requests can progress concurrently. We used a short application without framework-specific functionality to observe the server's behavior.

We varied the number of downloads between zero, one, and two, while sending /fast requests at eight per second for nine seconds. A slow response did not delay the next scheduled request, which was sent 125ms later. Each run measured 72 requests, and each condition was run three times.

We also measured requests routed through NGINX. A server that forwards requests and responses between an application and its clients is acting as a reverse proxy.

NGINX can read the application's response ahead of its delivery to the client, storing data in memory or temporary files. This is response buffering. The tables and figures call this “read-ahead on/off.” Enabling it does not mean NGINX waits for the entire response to accumulate before sending anything to the client.

Two downloads occupied both response workers

All tables and figures in this section keep the two Gunicorn sync workers fixed. We show the p95 of the other requests—the value containing 95% of their response times—as the median of three runs. One millisecond is one thousandth of a second.

The unit is milliseconds. For example, 5,945ms is about 5.9 seconds.

With direct access, one download left the other worker available for /fast. With two downloads, both workers were sending large responses, and /fast requests waited their turn.

The synchronous worker implementation provides a mechanism for understanding this. If receivers read slowly enough to fill the memory used for sending and receiving, a socket write waits for space to become available. A synchronous worker that cannot finish writing its response cannot advance to the next request.

Gunicorn's sync worker implementation writes response chunks in sequence. Its write implementation calls a socket, the interface a program uses to exchange data over a network. Until that write completes, the worker cannot move on to another request.

We observed the application's write duration and the other requests' response times. We did not measure the occupancy of each TCP buffer or inspect worker stacks to locate their waits. The observations are consistent with the behavior above, but do not establish how many bytes occupied each buffer.

Application CPU usage in these runs averaged at most about 0.8% of one core. Two workers can remain occupied by write waits even when there is computational capacity left.

Turning off NGINX read-ahead does not remove OS buffers

With NGINX read-ahead disabled and two downloads, p95 was about 1.1 seconds. This was shorter than direct access, but did not return to the roughly 2–3ms seen without downloads.

Before choosing the response size, we ran a pilot with 8MiB read at 1MiB/s. In that pilot, the other endpoint's p95 was about 2ms even with NGINX read-ahead disabled. We then changed the response to 16MiB and reading to 2MiB/s, keeping the paced download near eight seconds while increasing the data volume. Pilot values are not included in the tables.

Both response size and read rate changed between the pilot and main measurements. This does not isolate an effect of response size alone. The conclusions below apply to the main 16MiB, 2MiB/s condition.

The configuration that disables read-ahead is proxy_buffering off. OS send and receive memory still exists between the application and NGINX, and between NGINX and the receiving program. NGINX also uses a small buffer to read from the application. The NGINX documentation explains that proxy_buffer_size controls the maximum amount it can receive at a time even when buffering is off.

If the application can hand all response data to these buffers, it may move on to the next request before the receiver finishes. We did not measure individual socket occupancy, so these results do not define a universal boundary between 8MiB and 16MiB. The existence of buffering with read-ahead off follows from the specifications; we did not independently measure its capacity or contribution to the difference here.

Finishing a write is different from finishing a download

For the two-download condition, we measured two intervals separately.

• Application write duration: from the start of the response generator to handing the last response data to the OS send buffer.
• Client completion time: from sending the download request to receiving 16MiB and completing the pacing waits. This includes the rate-control wait after the final read.

For each column, the table takes the median of the two responses in each run, then the median across the three runs.

With read-ahead on, NGINX reads the application's response quickly. Data that does not fit in its memory buffers can also be saved to a temporary file. We observed behavior corresponding to this feature in the official NGINX specification.

In this configuration, the application can finish writing toward NGINX quickly and proceed to other requests. NGINX can continue sending to the receiver, separating application-write completion from client completion. The short API response times are consistent with this behavior. They do not show that enabling response buffering gives the same improvement in every application.

With read-ahead on, the total logical size of open temporary files for two downloads peaked at about 25.2MiB. Reducing the time application workers spend waiting on writes uses NGINX memory and files to store response data instead. File contents may also reside in OS memory, so saving data to a file does not eliminate memory consumption.

A supplementary configuration with more concurrent request capacity

For another check, we changed Gunicorn to gthread, one process with four threads. Threads let multiple activities progress within one process. This configuration provides four request-handling threads.

With direct access and two simultaneous downloads, the other endpoint's p95 was about 1.5ms. Two threads could be waiting on writes while others remained available for requests.

This supplementary condition changes both execution model and concurrent request capacity. It does not isolate either the model or capacity effect. It demonstrates that the same application can behave differently under another server configuration; it does not establish a performance advantage of threads or an optimal thread count.

What these measurements establish—and what they do not

In the main configuration, two concurrent slow receivers increased application write duration and the other endpoint's latency. Together with the Gunicorn sync specification, the fact that a worker cannot handle its next request until it finishes the current response helps interpret the result. The main comparison of NGINX response buffering was made with this worker configuration fixed.

We did not compare frameworks such as Flask, Django, or FastAPI, other servers or runtimes such as Uvicorn or Node.js, or asynchronous execution. These measurements cannot tell us that those environments would exhibit the same behavior or timings. The supplementary configuration also achieved about 1.5ms without NGINX, so NGINX is not established as a general requirement for web applications.

What to check in your own application

First, separate the time spent producing response data from the time spent writing it. If producing the data is quick but writing takes a long time, investigate client reads and the sending and receiving along the path.

Next, check whether a worker waiting on a write can handle another request. For a one-request-at-a-time configuration such as the sync workers used here, examine both the number of write waits and the worker count.

If a PaaS or another hosting setup places a proxy in front of the application, check how far that proxy reads responses ahead of client delivery. After changing a setting, compare the other requests' latency along with proxy memory and temporary-file usage.

Enabling read-ahead may also affect when data arrives for responses intended to deliver small updates immediately. This experiment covers ordinary HTTP responses with a known size. SSE, which can display text incrementally, and WebSockets, which keep a connection for bidirectional communication, need separate checks.

Before applying these results, check the server execution model, worker or thread count, and proxy settings in your application. Do not infer behavior from the framework's name alone. The practical takeaway is to determine whether other requests can progress while a response waits to be written.

Measurements are available as condition summaries, individual-run summaries, per-request response times, large-response write and completion times, and resource time series. The data README explains aggregation and the reproduction code.

KamuiDash is a PaaS for deploying web applications from GitHub.