Bug 2161229 - Add rate-limiting to metadata agents
Summary: Add rate-limiting to metadata agents
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-neutron
Version: 17.0 (Wallaby)
Hardware: Unspecified
OS: Unspecified
urgent
medium
Target Milestone: z1
: 17.1
Assignee: Miguel Lavalle
QA Contact: Eran Kuris
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2023-01-16 10:50 UTC by Bernard Cafarelli
Modified: 2023-09-20 00:30 UTC (History)
13 users (show)

Fixed In Version: openstack-neutron-18.6.1-1.20230518200974.el9ost
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2023-09-20 00:29:44 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Launchpad 1989199 0 None None None 2023-01-16 10:50:20 UTC
OpenStack gerrit 858879 0 None MERGED Add rate-limiting to metadata agents 2023-09-05 10:24:06 UTC
Red Hat Issue Tracker OSP-21441 0 None None None 2023-01-16 10:53:06 UTC
Red Hat Issue Tracker RHOSPDOC-1386 0 None None None 2023-08-29 13:54:14 UTC
Red Hat Product Errata RHBA-2023:5138 0 None None None 2023-09-20 00:30:21 UTC

Description Bernard Cafarelli 2023-01-16 10:50:05 UTC
This bugzilla goal is to complete upstream launchpad bug [0] and include it in OSP:

At the moment, there is no limit on how many requests the metadata-agent will handle. Some users may sometimes run scripts in their instances that
try to query the metadata endpoint at high rate (for example a bugged k8s cloud controller manager), causing an increased load on some or all the
component above the metadata-agent.

Spec was accepted and merged [1], but proposed code change was not fully completed [2]

[0] https://bugs.launchpad.net/neutron/+bug/1989199
[1] https://review.opendev.org/c/openstack/neutron-specs/+/856831
[2] https://review.opendev.org/c/openstack/neutron/+/858879

Comment 19 Bracha Frenkel 2023-09-13 14:41:47 UTC
verified on RHOS-17.1-RHEL-9-20230907.n.1
in compute node I edited the file: /var/lib/config-data/puppet-generated/neutron/etc/neutron/neutron.conf
```
with the following values:
[metadata_rate_limiting]
rate_limit_enabled = true
base_query_rate_limit = 1
burst_window_duration = 10
burst_query_rate_limit = 10
```

I restarted the service `tripleo_ovn_metadata_agent.service`
from the vm 
```
[cloud-user@vm1 ~]$ curl http://169.254.169.254
1.0
2007-01-19
2007-03-01
2007-08-29
2007-10-10
2007-12-15
2008-02-01
2008-09-01
2009-04-04
latest[cloud-user@vm1 ~]$ curl http://169.254.169.254
<html><body><h1>429 Too Many Requests</h1>
You have sent too many requests in a given amount of time.
</body></html>  
```

Comment 20 Vadim Khitrin 2023-09-13 14:47:36 UTC
Bracha has no permissions to move this but to `VERIFIED`.

Comment 24 errata-xmlrpc 2023-09-20 00:29:44 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Release of components for Red Hat OpenStack Platform 17.1.1 (Wallaby)), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2023:5138


Note You need to log in before you can comment on or make changes to this bug.