Bug 2224114

Summary:	[RFE] Skopeo to Provide Bandwidth Limiting
Product:	Red Hat Enterprise Linux 9	Reporter:	Benjamin Schmaus <bschmaus>
Component:	skopeo	Assignee:	Miloslav Trmač <mitr>
Status:	CLOSED WONTFIX	QA Contact:	atomic-bugs <atomic-bugs>
Severity:	low	Docs Contact:
Priority:	unspecified
Version:	9.2	CC:	dwalsh, jnovy, lsm5, mboddu, pthomas, tsweeney, umohnani, walters
Target Milestone:	rc	Keywords:	FutureFeature
Target Release:	---	Flags:	tsweeney: needinfo? (dwalsh)
Hardware:	All
OS:	Linux
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2023-07-28 15:14:48 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Benjamin Schmaus 2023-07-19 19:58:13 UTC

Description of problem:

Customers might have constraints when it comes to bandwidth and latency to their edge locations where they might adopt and need to run a lightweight registry to provide the containers to the edge devices at the those edge locations.  The connection back to the datacenters and/or internet might also share bandwidth for other site services like VOIP or other data requirements.  Hence there is a need to have any tool that potentially can sync/transfer/copy containers to the edge registry be able to limit the amount of bandwidth used during that transfer even if the transfer is happening in the off hours of the sites operations.

As a customer I am using Skopeo to copy those images to my edge registry but require it to transfer the data at a certain rate instead of just consuming all the available bandwidth possible.

Version-Release number of selected component (if applicable):
9.2

How reproducible:
NA - Feature

Steps to Reproduce:
1.
2.
3.

Actual results:
Today Skopeo would use all available bandwidth during a sync/transfer/copy

Expected results:
In the future I want to be able to pass a switch telling Skopeo how much bandwidth it can consume for a container sync/transfer/copy

Additional info:

Comment 2 Colin Walters 2023-07-19 20:39:31 UTC

I see the desire for some sort of declarative knob, but note there is e.g. tc which can be used to generically shape traffic independently of any specific application.

https://access.redhat.com/solutions/3067231
https://joshrosso.com/c/tc/

Comment 3 Tom Sweeney 2023-07-21 18:17:34 UTC

Assigning to Miloslav.  I've also cc'd @dwalsh as I think he'll be interested in this too.

Comment 4 Miloslav Trmač 2023-07-24 17:52:47 UTC

I do appreciate that limiting bandwidth is very important on some systems.

That said, I don’t think it’s all that user-friendly or productive for every single binary that could ever reach the internet to grow its own (necessarily tool-specific) bandwidth limiting option; and I’m not very convinced that even doing that for the major users, like (skopeo sync) might well be, is the best approach.

Enforcing some kind of more global limit, either based on a route or possibly on a cgroup, seems to me to be a far more general and safe approach, notably because it can be set up by the administrator and it does not rely on individual users’ discretion to use the bandwidth limiting option; and because the bandwidth limiting implementation would be consistent for all affected programs, regardless of implementation details like the language and libraries used.

It seems that cgroups provides facilities to do that (net_cls in cgroupsv1, or iptables’ xt_cgroup in cgroupsv2) — but I’m not really familiar with either of these.

Comment 5 Benjamin Schmaus 2023-07-24 21:04:46 UTC

I am familiar with tc since I used it as an example here: https://cloud.redhat.com/blog/openshift-latency-bandwidth-testing-for-edge  although others from Red Hat have had mixed success when using it.

I am open for different approaches that could be applied but I think we need to make it easy to consume for customers as the edge.

Comment 6 Benjamin Schmaus 2023-07-26 13:55:35 UTC

One thing to consider how am I using tc if this is running in an OCP cluster?

Comment 7 Miloslav Trmač 2023-07-26 14:59:32 UTC

Now that is a good question, and it seems to have a pretty satisfactory answer: https://access.redhat.com/solutions/5018951 / https://docs.openshift.com/container-platform/4.13/nodes/pods/nodes-pods-configuring.html#nodes-pods-configuring-bandwidth_nodes-pods-configuring .