Bug 1263521 - [undercloud] swift proxy-server keep logging "Invalid user token - deferring reject downstream"
[undercloud] swift proxy-server keep logging "Invalid user token - deferring ...
Status: POST
Product: Red Hat OpenStack
Classification: Red Hat
Component: python-keystonemiddleware (Show other bugs)
7.0 (Kilo)
All Linux
high Severity medium
: ---
: 11.0 (Ocata)
Assigned To: John Dennis
Mike Abrams
: Triaged, ZStream
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2015-09-16 00:32 EDT by Jaison Raju
Modified: 2017-09-18 11:40 EDT (History)
17 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed:
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
Config (1.51 KB, text/plain)
2015-09-16 00:32 EDT, Jaison Raju
no flags Details


External Trackers
Tracker ID Priority Status Summary Last Updated
Red Hat Knowledge Base (Solution) 2483401 None None None 2016-08-03 05:28 EDT
OpenStack gerrit 441549 None None None 2017-03-03 23:38 EST

  None (edit)
Description Jaison Raju 2015-09-16 00:32:54 EDT
Created attachment 1073892 [details]
Config

Description of problem:
After initial deployment of undercloud, swift proxy-server keeps logging the
following error :
Sep 16 05:21:07 instack.localdomain proxy-server[27467]: Invalid user token - deferring reject downstream
Sep 16 05:21:07 instack.localdomain proxy-server[27467]: 192.0.2.1 192.0.2.1 16/Sep/2015/09/21/07 GET /info HTTP/1.0 200 - - - - 738 - tx76694f16d3c74c27bf7d4-0055f93483 - 0.0006 - - 1442395267.932071924 1442395
Sep 16 05:21:09 instack.localdomain proxy-server[27467]: Invalid user token - deferring reject downstream

This is noticed in swift.log , messages or from systemd commands .
This is logged almost every second , hence log file increases by +500M in a day or two .
This error is notice on overcloud controllers & undercloud node .
openstack overcloud deploy --templates --ntp-server xx.xx.xx.xx --control-flavor control --compute-flavor compute --swift-storage-flavor control --ceph-storage-flavor control --block-storage-flavor control --control-scale 1 --compute-scale 1 --neutron-tunnel-types vxlan --neutron-network-type vxlan --neutron-public-interface eth4 --libvirt-type kvm


Version-Release number of selected component (if applicable):
OSP7 Director

How reproducible:
Yes

Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:
# date
Wed Sep 16 05:25:52 EDT 2015
[root@instack ~]# journalctl -u openstack-swift-proxy --since 05:25:52
-- Logs begin at Mon 2015-09-14 12:53:35 EDT, end at Wed 2015-09-16 05:25:57 EDT. --
Sep 16 05:25:52 instack.localdomain proxy-server[27467]: Invalid user token - deferring reject downstream
Sep 16 05:25:52 instack.localdomain proxy-server[27467]: 192.0.2.1 192.0.2.1 16/Sep/2015/09/25/52 GET /info HTTP/1.0 200 - - - - 738 - tx2f7370777c5b4d9fa5a71-0055f935a0 - 0.0007 - - 1442395552.575284004 1442395
Sep 16 05:25:54 instack.localdomain proxy-server[27467]: Invalid user token - deferring reject downstream
Sep 16 05:25:54 instack.localdomain proxy-server[27467]: 192.0.2.1 192.0.2.1 16/Sep/2015/09/25/54 GET /info HTTP/1.0 200 - - - - 738 - tx071e3d2d1c8f4067ad5ff-0055f935a2 - 0.0006 - - 1442395554.579209089 1442395
Sep 16 05:25:56 instack.localdomain proxy-server[27467]: Invalid user token - deferring reject downstream
Sep 16 05:25:56 instack.localdomain proxy-server[27467]: 192.0.2.1 192.0.2.1 16/Sep/2015/09/25/56 GET /info HTTP/1.0 200 - - - - 738 - tx0c098e5cd5d34383817a1-0055f935a4 - 0.0009 - - 1442395556.584398031 1442395
[root@instack ~]#  ls -lh /var/log/messages /var/log/swift/swift.log 
-rw-------. 1 root root 629M Sep 16 05:26 /var/log/messages
-rw-r--r--. 1 root root  31M Sep 16 05:26 /var/log/swift/swift.log
Comment 8 Mike Burns 2016-04-07 16:50:54 EDT
This bug did not make the OSP 8.0 release.  It is being deferred to OSP 10.
Comment 15 Paul Grist 2016-10-13 23:21:10 EDT
Can we take a look at whether this is just a case of excessive logging swift can't control or whether any log options need to be considered?
Comment 25 Pete Zaitcev 2017-04-28 13:07:06 EDT
The upstream review 441549 was approved today. We need to put this in
urgently to make GA, or at least 1-day fixes. I'm changing the component
to keystonemiddleware so that relevant people can act upon it.

Note You need to log in before you can comment on or make changes to this bug.