Bug 2075727

Summary: The memory usage of the squid process keeps increasing.
Product: Red Hat Enterprise Linux 9 Reporter: hariki <tsuyoshi.hariki>
Component: squidAssignee: Luboš Uhliarik <luhliari>
Status: CLOSED ERRATA QA Contact: Branislav Náter <bnater>
Severity: high Docs Contact:
Priority: unspecified    
Version: CentOS StreamCC: bstinson, chorn, jwboyer, robo
Target Milestone: rcKeywords: Rebase, Triaged
Target Release: ---Flags: pm-rhel: mirror+
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: squid-5.5-1.el9 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2022-11-15 10:26:24 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
squid.conf none

Description hariki 2022-04-15 01:45:15 UTC 
Created attachment 1872683 [details]
squid.conf

Description of problem:
The memory usage of the squid process keeps increasing on HTTP forward proxy service.

Version-Release number of selected component (if applicable):
squid-5.2-1.el9 on CentOS Stream 9

How reproducible:
At 10 to 40 requests/second environment

Steps to Reproduce:
1. edit /etc/squid/squid.conf(attached)
2. % sudo systemctl start squid.service

Actual results:
The actual memory usage after 24 hours of operation is as follows
% free -h
               total        used        free      shared  buff/cache   available
Mem:           7.8Gi       6.9Gi       135Mi        59Mi       719Mi       543Mi
Swap:          8.0Gi       5.7Gi       2.3Gi
% ps auxw |grep ^squid
squid     221306  3.8 82.2 14533656 6685856 ?    S    Apr12  54:36 (squid-1) --kid squid-1 --foreground -f /etc/squid/squid.conf

Expected results:
Restarting squid process will restore it.
% sudo systemctl restart squid.service
% free -h
               total        used        free      shared  buff/cache   available
Mem:           7.8Gi       549Mi       6.5Gi        59Mi       709Mi       6.9Gi
% ps auxw |grep ^squid
squid     263768  2.6  0.8 185720 67360 ?        S    16:42   0:00 (squid-1) --kid squid-1 --foreground -f /etc/squid/squid.conf

Additional info:
This is similar to the thread of following site.
https://www.spinics.net/lists/squid/msg94816.html

So, we created rpm using squid-5.4.1 and updated it.
Similarly, memory usage after 24 hours of operation improved as follows
% free -h
               total        used        free      shared  buff/cache   available
Mem:           7.8Gi       1.2Gi       5.9Gi        73Mi       700Mi       6.3Gi
Swap:          8.0Gi          0B       8.0Gi
% ps auxw |grep ^squid
squid       9002  4.1  9.5 798632 773548 ?       S    Apr12  58:24 (squid-1) --kid squid-1 --foreground -f /etc/squid/squid.conf

Could you please update the base version of squid on CentOS Stream 9/ Red Hat Enterprise Linux 9 to the 5.4.1?
Comment 1 Luboš Uhliarik 2022-04-18 16:29:05 UTC 
Hi Hariki,

thanks for reporting this bug. I think we will rebase squid directly to 5.5, but let's see. I will work on this ASAP.


Upstream bz: https://bugs.squid-cache.org/show_bug.cgi?id=5132
Upstream commit: https://github.com/squid-cache/squid/commit/54ad10efe146c863bdadee0d1161299ea89f5419
Comment 4 Luboš Uhliarik 2022-04-20 11:09:59 UTC 
New squid 5.5 is present in Fedora Rawhide/f36 now. Let's wait few days if the update in Fedora is OK and I will update squid in c9s as well.

https://bodhi.fedoraproject.org/updates/FEDORA-2022-7cc2088335
Comment 8 Robert Hanzel 2022-06-10 08:16:49 UTC 
Hi Lubos, we are experiencing this bug in RHEL9 in production env. Do you already have a build for rhel9/c9s?
Thanks
Comment 12 Branislav Náter 2022-06-13 13:24:43 UTC 
(In reply to Robert Hanzel from comment #8)
> Hi Lubos, we are experiencing this bug in RHEL9 in production env. Do you
> already have a build for rhel9/c9s?
> Thanks

Hi Robert,

yes, build has been already created in koji [1] and should be available in c9s in a few days.

1. https://kojihub.stream.centos.org/koji/buildinfo?buildID=21544
Comment 13 Branislav Náter 2022-06-15 13:38:47 UTC 
(In reply to Branislav Náter from comment #12)
> (In reply to Robert Hanzel from comment #8)
> > Hi Lubos, we are experiencing this bug in RHEL9 in production env. Do you
> > already have a build for rhel9/c9s?
> > Thanks
> 
> Hi Robert,
> 
> yes, build has been already created in koji [1] and should be available in
> c9s in a few days.
> 
> 1. https://kojihub.stream.centos.org/koji/buildinfo?buildID=21544

I've found new build in following c9s repo:

https://composes.stream.centos.org/production/CentOS-Stream-9-20220613.0/compose/AppStream/x86_64/os/Packages/squid-5.5-1.el9.x86_64.rpm

@robo we'll appreciate any feedback regarding new squid build. Thanks in advance.
Comment 14 Robert Hanzel 2022-06-20 07:46:12 UTC 
(In reply to Branislav Náter from comment #13)
> (In reply to Branislav Náter from comment #12)
> > (In reply to Robert Hanzel from comment #8)
> > > Hi Lubos, we are experiencing this bug in RHEL9 in production env. Do you
> > > already have a build for rhel9/c9s?
> > > Thanks
> > 
> > Hi Robert,
> > 
> > yes, build has been already created in koji [1] and should be available in
> > c9s in a few days.
> > 
> > 1. https://kojihub.stream.centos.org/koji/buildinfo?buildID=21544
> 
> I've found new build in following c9s repo:
> 
> https://composes.stream.centos.org/production/CentOS-Stream-9-20220613.0/
> compose/AppStream/x86_64/os/Packages/squid-5.5-1.el9.x86_64.rpm
> 
> @robo we'll appreciate any feedback regarding new squid build. Thanks
> in advance.

Hi Brano,
after one week of testing, everything looks good. SQUID looks stable without inappropriate memory consumption.
Comment 15 Branislav Náter 2022-06-20 08:08:53 UTC 
(In reply to Robert Hanzel from comment #14)
> (In reply to Branislav Náter from comment #13)
> > (In reply to Branislav Náter from comment #12)
> > > (In reply to Robert Hanzel from comment #8)
> > > > Hi Lubos, we are experiencing this bug in RHEL9 in production env. Do you
> > > > already have a build for rhel9/c9s?
> > > > Thanks
> > > 
> > > Hi Robert,
> > > 
> > > yes, build has been already created in koji [1] and should be available in
> > > c9s in a few days.
> > > 
> > > 1. https://kojihub.stream.centos.org/koji/buildinfo?buildID=21544
> > 
> > I've found new build in following c9s repo:
> > 
> > https://composes.stream.centos.org/production/CentOS-Stream-9-20220613.0/
> > compose/AppStream/x86_64/os/Packages/squid-5.5-1.el9.x86_64.rpm
> > 
> > @robo we'll appreciate any feedback regarding new squid build. Thanks
> > in advance.
> 
> Hi Brano,
> after one week of testing, everything looks good. SQUID looks stable without
> inappropriate memory consumption.

Thanks for the feedback!
Comment 17 hariki 2022-06-21 00:11:29 UTC 
Thank you for the new package squid-5.5. I have confirmed that there is no problem in our environment as well. Please close this issue.
Comment 18 Branislav Náter 2022-06-21 07:59:24 UTC 
(In reply to hariki from comment #17)
> Thank you for the new package squid-5.5. I have confirmed that there is no
> problem in our environment as well. Please close this issue.

Thanks for letting us know! FYI, bug will be closed on rhel-9.1 GA (when we release new package in RHEL).
Comment 20 errata-xmlrpc 2022-11-15 10:26:24 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (squid bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2022:8136
Comment 21 Christian Horn 2025-01-23 04:35:54 UTC 
We might see a new occurrance of this on rhel8 and 9:
https://issues.redhat.com/browse/RHEL-50261