Description of problem:

In our organization, we have begun experimenting with inline malware scanning in our enterprise network proxies.

When inline malware scanning is enabled, when a client requests a download through the proxy, before the proxy returns any portion of the file to the client, the proxy first downloads the file and performs a malware scan. If the scan finds no malware, then the file is returned to the client.

The proxy's actions produce behavior that is very non-intuitive to both humans and automated programs: an attempt to download a large file will appear to "hang" with no progress for many seconds, and then the entire file is downloaded at essentially LAN network speeds (>100 Mbit/s).

Because this behavior can be non-intuitive, the proxy supports a feature it calls "trickle first" mode: instead of serving no data to the client until the scan is complete, the proxy starts sending the file to the client at the (non-configurable) rate of 1 byte per second. The idea is to convince both humans and automated programs that the download is successfully in progress. Once the proxy completes the malware scan, the rest of the file is sent to the client at LAN network speeds.

Unfortunately, the logic yum uses to detect dead download servers interacts badly with the proxy's behavior.

If the proxy does not use trickle mode, yum will wait for the number of seconds defined by timeout (default: 30), give up, and attempt to use another server.

But the proxy's "trickle first" mode isn't enough to satisfy yum. That is because yum has a non-configurable download rate threshold of 1000 bytes per second. If the download falls below that rate for more than 5 seconds, yum assumes the server is dead.

Thus, the ONLY way to make yum work properly with our enterprise network proxy is to set yum's timeout to a value greater than the amount of time it will take for yum to download any package; that is, to completely disable yum's dead server detection algorithms. That's a work-around, but it's an undesirable one.

A slightly better work-around would be for yum to make the minimal download rate threshold configurable.  E.g.:

# The number of bytes per second at which a server must send data to prevent
# yum from assuming the server is too slow to use.  Default: 1000.
minrate=1000

This still isn't ideal, as it won't detect the case where a server sends part of the file before dying. But it will detect the case when a server is still answering pings, but isn't serving up any data. And it shouldn't be too difficult to implement. (If I could figure out where yum's logic is implemented, I'd take a crack at it myself.)

Version-Release number of selected component (if applicable):

0:python-urlgrabber-3.9.1-24.fc18.noarch
0:yum-3.4.3-54.fc18.noarch