Bug 1563783 - [OSP13] openstack undercloud backup fails over: /undercloud-backups/UC-backup-20180222101308.tar 413 Request Entity Too Large [first 60 chars of response]
Summary: [OSP13] openstack undercloud backup fails over: /undercloud-backups/UC-backup...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: instack-undercloud
Version: 13.0 (Queens)
Hardware: x86_64
OS: Linux
high
high
Target Milestone: beta
: 13.0 (Queens)
Assignee: Carlos Camacho
QA Contact: Omri Hochman
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-04-04 17:04 UTC by Omri Hochman
Modified: 2023-09-14 04:26 UTC (History)
8 users (show)

Fixed In Version: instack-undercloud-8.4.0-4.el7ost openstack-tripleo-common-8.6.1-6.el7ost
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-06-27 13:50:03 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Launchpad 1761412 0 None None None 2018-04-05 08:44:35 UTC
OpenStack gerrit 559024 0 None MERGED Allow uploading big files to swift (5GB) 2020-02-17 09:01:22 UTC
OpenStack gerrit 559349 0 None MERGED Enable missing support for large objects in Swift 2020-02-17 09:01:22 UTC
OpenStack gerrit 559927 0 None MERGED Enable missing support for large objects in Swift 2020-02-17 09:01:22 UTC
OpenStack gerrit 563977 0 None MERGED Allow uploading big files to swift (5GB) 2020-02-17 09:01:22 UTC
Red Hat Product Errata RHEA-2018:2086 0 None None None 2018-06-27 13:50:53 UTC

Description Omri Hochman 2018-04-04 17:04:10 UTC
[OSP13] openstack undercloud backup fails over: /undercloud-backups/UC-backup-20180222101308.tar 413 Request Entity Too Large  [first 60 chars of response] 


Environment: 
------------
instack-8.1.1-0.20180313084439.0d768a3.el7ost.noarch
instack-undercloud-8.4.0-2.el7ost.noarch
openstack-tripleo-heat-templates-8.0.2-0.20180327213843.f25e2d8.el7ost.noarch
python-swift-2.17.1-0.20180314165245.caeeb54.el7ost.noarch
openstack-swift-container-2.17.1-0.20180314165245.caeeb54.el7ost.noarch
openstack-swift-proxy-2.17.1-0.20180314165245.caeeb54.el7ost.noarch
openstack-swift-object-2.17.1-0.20180314165245.caeeb54.el7ost.noarch
python2-swiftclient-3.5.0-1.el7ost.noarch
openstack-swift-account-2.17.1-0.20180314165245.caeeb54.el7ost.noarch
puppet-swift-12.3.1-0.20180221111701.5bdce0a.el7ost.noarch


Steps:
------

(1) Attempt to perform OSP13 Undercloud backup,
 according to instructions: 

https://review.openstack.org/#/c/558429/1/doc/source/install/controlplane_backup_restore/01_undercloud_backup.rst 

results command exist with the following failure:
--------------------------------------------------
(undercloud) [stack@undercloud75 ~]$
(undercloud) [stack@undercloud75 ~]$ openstack undercloud backup --add-path /etc/hosts \
>                               --add-path /var/log/ \
>                               --add-path /var/lib/glance/images/ \
>                               --add-path /srv/node/ \
>                               --add-path /etc/
Started Mistral Workflow tripleo.undercloud_backup.v1.backup. Execution ID: 3711cf24-716f-4686-ad80-93956852cc33
Waiting for messages on queue 'tripleo' with no timeout.




Undercloud backup finished with errors
Output: {u'execution': {u'created_at': u'2018-02-22 15:08:57',
                u'id': u'3711cf24-716f-4686-ad80-93956852cc33',
                u'input': {u'queue_name': u'tripleo',
                           u'sources_path': u'/etc/,/etc/hosts,/home/stack/,/srv/node/,/var/lib/glance/images/,/var/log/'},
                u'name': u'tripleo.undercloud_backup.v1.backup',
                u'params': {u'namespace': u''},
                u'spec': {u'description': u'This workflow will launch the Undercloud backup',
                          u'input': [{u'sources_path': u'/home/stack/'},
                                     {u'queue_name': u'tripleo'}],
                          u'name': u'backup',
                          u'tags': [u'tripleo-common-managed'],
                          u'tasks': {u'cleanup_backup': {u'action': u'tripleo.undercloud.remove_temp_dir',
..
..
..
..
..
                                                       u'retry': u'count=5 delay=1',
                                                       u'type': u'direct',
                                                       u'version': u'2.0'},
                                     u'upload_backup': {u'action': u'tripleo.undercloud.upload_backup_to_swift',
                                                        u'input': {u'backup_path': u'<% $.backup_path.path %>'},
                                                        u'name': u'upload_backup',
                                                        u'on-error': u'send_message',
                                                        u'on-success': u'cleanup_backup',
                                                        u'publish': {u'message': u'<% task().result %>',
                                                                     u'status': u'SUCCESS'},
                                                        u'publish-on-error': {u'message': u'<% task().result %>',
                                                                              u'status': u'FAILED'},
                                                        u'type': u'direct',
                                                        u'version': u'2.0'}},
                          u'version': u'2.0'},
                u'updated_at': u'2018-02-22 15:08:57'},
 u'message': {u'msg': u'Object PUT failed: https://192.168.0.2:13808/v1/AUTH_5b0b3efc458a4fc4af8e21487556aeca/undercloud-backups/UC-backup-20180222101308.tar 413 Request Entity Too Large  [first 60 chars of response] <html><h1>Request Entity Too Large</h1><p>The body of your r'},
 u'status': u'FAILED'}
(undercloud) [stack@undercloud75 ~]$
(undercloud) [stack@undercloud75 ~]$
(undercloud) [stack@undercloud75 ~]$
(undercloud) [stack@undercloud75 ~]$


Checking the size of file and HardDisk ( seems there's enough space on HD):
---------------------------------------------------------------------------

[root@undercloud75 undercloud-backup-JjgPDr]# df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/vda1       104G   31G   74G  30% /
devtmpfs        7.8G     0  7.8G   0% /dev
tmpfs           7.8G  4.0K  7.8G   1% /dev/shm
tmpfs           7.8G  788K  7.8G   1% /run
tmpfs           7.8G     0  7.8G   0% /sys/fs/cgroup
tmpfs           1.6G     0  1.6G   0% /run/user/1000
tmpfs           1.6G     0  1.6G   0% /run/user/0
[root@undercloud75 undercloud-backup-JjgPDr]# ls -lah
total 8.2G
drwx------.  2 mistral mistral   86 Feb 22 10:09 .
drwxrwxrwt. 14 root    root    4.0K Feb 22 10:08 ..
-rw-r--r--.  1 mistral mistral  35M Feb 22 10:09 all-databases-20180222100900.sql.gz
-rw-r--r--.  1 mistral mistral 8.2G Feb 22 10:13 filesystem-20180222100929.tar

Comment 1 Carlos Camacho 2018-04-05 12:42:58 UTC
Both fixes are merged/backported

Comment 4 Omri Hochman 2018-04-20 18:54:54 UTC
Adding blocker flag -  As the backup/restore undercloud is a regression test for DFG:DF

Comment 10 Omri Hochman 2018-05-08 12:53:17 UTC
Verified : 
[stack@undercloud75 ~]$ source stackrc
(undercloud) [stack@undercloud75 ~]$ openstack undercloud backup --add-path /etc/hosts \
>                               --add-path /var/log/ \
>                               --add-path /var/lib/glance/images/ \
>                               --add-path /srv/node/ \
>                               --add-path /etc/

Started Mistral Workflow tripleo.undercloud_backup.v1.backup. Execution ID: 78716244-f8f4-49c5-b1e7-872bb3b4cb11
Waiting for messages on queue 'tripleo' with no timeout.
Undercloud Backup succeed
(undercloud) [stack@undercloud75 ~]$
(undercloud) [stack@undercloud75 ~]$ rpm -qa | grep instack
instack-undercloud-8.4.1-4.el7ost.noarch

Comment 13 errata-xmlrpc 2018-06-27 13:50:03 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2018:2086

Comment 14 Red Hat Bugzilla 2023-09-14 04:26:23 UTC
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 1000 days


Note You need to log in before you can comment on or make changes to this bug.