Description of problem: Customers might have constraints when it comes to bandwidth and latency to their edge locations where they might adopt and need to run a lightweight registry to provide the containers to the edge devices at the those edge locations. The connection back to the datacenters and/or internet might also share bandwidth for other site services like VOIP or other data requirements. Hence there is a need to have any tool that potentially can sync/transfer/copy containers to the edge registry be able to limit the amount of bandwidth used during that transfer even if the transfer is happening in the off hours of the sites operations. As a customer I am using Skopeo to copy those images to my edge registry but require it to transfer the data at a certain rate instead of just consuming all the available bandwidth possible. Version-Release number of selected component (if applicable): 9.2 How reproducible: NA - Feature Steps to Reproduce: 1. 2. 3. Actual results: Today Skopeo would use all available bandwidth during a sync/transfer/copy Expected results: In the future I want to be able to pass a switch telling Skopeo how much bandwidth it can consume for a container sync/transfer/copy Additional info:
I see the desire for some sort of declarative knob, but note there is e.g. tc which can be used to generically shape traffic independently of any specific application. https://access.redhat.com/solutions/3067231 https://joshrosso.com/c/tc/
Assigning to Miloslav. I've also cc'd @dwalsh as I think he'll be interested in this too.
I do appreciate that limiting bandwidth is very important on some systems. That said, I don’t think it’s all that user-friendly or productive for every single binary that could ever reach the internet to grow its own (necessarily tool-specific) bandwidth limiting option; and I’m not very convinced that even doing that for the major users, like (skopeo sync) might well be, is the best approach. Enforcing some kind of more global limit, either based on a route or possibly on a cgroup, seems to me to be a far more general and safe approach, notably because it can be set up by the administrator and it does not rely on individual users’ discretion to use the bandwidth limiting option; and because the bandwidth limiting implementation would be consistent for all affected programs, regardless of implementation details like the language and libraries used. It seems that cgroups provides facilities to do that (net_cls in cgroupsv1, or iptables’ xt_cgroup in cgroupsv2) — but I’m not really familiar with either of these.
I am familiar with tc since I used it as an example here: https://cloud.redhat.com/blog/openshift-latency-bandwidth-testing-for-edge although others from Red Hat have had mixed success when using it. I am open for different approaches that could be applied but I think we need to make it easy to consume for customers as the edge.
One thing to consider how am I using tc if this is running in an OCP cluster?
Now that is a good question, and it seems to have a pretty satisfactory answer: https://access.redhat.com/solutions/5018951 / https://docs.openshift.com/container-platform/4.13/nodes/pods/nodes-pods-configuring.html#nodes-pods-configuring-bandwidth_nodes-pods-configuring .