Workaround:

After a failed upgrade:

yum remove -y openstack-dashboard
systemctl daemon-reload
systemctl restart httpd
openstack undercloud upgrade

[root@undercloud-0 openstack-dashboard]# systemctl status httpd
● httpd.service - The Apache HTTP Server
   Loaded: loaded (/usr/lib/systemd/system/httpd.service; enabled; vendor preset: disabled)
  Drop-In: /usr/lib/systemd/system/httpd.service.d
           └─openstack-dashboard.conf
   Active: failed (Result: exit-code) since Wed 2017-09-13 18:13:14 EDT; 8h ago
     Docs: man:httpd(8)
           man:apachectl(8)
  Process: 17279 ExecStop=/bin/kill -WINCH ${MAINPID} (code=exited, status=0/SUCCESS)
  Process: 17295 ExecStartPre=/usr/bin/python /usr/share/openstack-dashboard/manage.py collectstatic --noinput --clear (code=exited, status=1/FAILURE)
 Main PID: 32185 (code=exited, status=0/SUCCESS)

Sep 13 18:13:14 undercloud-0.redhat.local python[17295]: app_config = AppConfig.create(entry)
Sep 13 18:13:14 undercloud-0.redhat.local python[17295]: File "/usr/lib/python2.7/site-packages/django/apps/config.py", line 86, in create
Sep 13 18:13:14 undercloud-0.redhat.local python[17295]: module = import_module(entry)
Sep 13 18:13:14 undercloud-0.redhat.local python[17295]: File "/usr/lib64/python2.7/importlib/__init__.py", line 37, in import_module
Sep 13 18:13:14 undercloud-0.redhat.local python[17295]: __import__(name)
Sep 13 18:13:14 undercloud-0.redhat.local python[17295]: ImportError: No module named tuskar_boxes
Sep 13 18:13:14 undercloud-0.redhat.local systemd[1]: httpd.service: control process exited, code=exited status=1
Sep 13 18:13:14 undercloud-0.redhat.local systemd[1]: Failed to start The Apache HTTP Server.
Sep 13 18:13:14 undercloud-0.redhat.local systemd[1]: Unit httpd.service entered failed state.
Sep 13 18:13:14 undercloud-0.redhat.local systemd[1]: httpd.service failed.


[root@undercloud-0 ~]# cat /usr/lib/systemd/system/httpd.service.d/openstack-dashboard.conf 
[Service]
ExecStartPre=/usr/bin/python /usr/share/openstack-dashboard/manage.py collectstatic --noinput --clear
ExecStartPre=/usr/bin/python /usr/share/openstack-dashboard/manage.py compress --force
TimeoutStartSec=2min

[Unit]
After=memcached.service


[root@undercloud-0 ~]# ls -ls /usr/share/openstack-dashboard/static/
total 0
0 drwxr-xr-x.  4 root root  98 Sep 13 18:07 app
0 drwxr-xr-x.  3 root root  58 Sep 13 18:07 auth
0 drwxr-xr-x.  5 root root  41 Jun  1 12:50 bootstrap
0 drwxr-xr-x.  4 root root  81 Sep 13 18:07 custom
0 drwxr-xr-x. 10 root root 127 Sep 13 18:07 dashboard
0 drwxr-xr-x.  5 root root 143 Sep 13 18:07 framework
0 drwxr-xr-x.  5 root root  57 Sep 13 18:07 horizon
0 drwxr-xr-x.  5 root root  40 Sep 13 16:59 redhat_access_plugin_openstack
0 drwxr-xr-x.  3 root root  20 Jun  1 12:50 scss
0 drwxr-xr-x.  4 root root  28 Sep 13 17:02 tuskar_boxes

[root@undercloud-0 ~]# grep -Ri tuskar_boxes /usr/share/openstack-dashboard/
Binary file /usr/share/openstack-dashboard/openstack_dashboard/local/enabled/_60_tuskar_boxes.pyc matches

[root@undercloud-0 ~]# ls -l  /usr/share/openstack-dashboard/openstack_dashboard/local/enabled
total 36
-rw-r--r--. 1 root root 224 Sep 13 17:02 _10_admin.pyc
-rw-r--r--. 1 root root 228 Sep 13 17:02 _20_project.pyc
-rw-r--r--. 1 root root 230 Sep 13 17:02 _30_identity.pyc
-rw-r--r--. 1 root root 185 Jan 27  2016 _50_settings.py.example
-rw-r--r--. 1 root root 515 Sep 13 17:02 _50_tuskar.pyc
-rw-r--r--. 1 root root 361 Sep 13 17:02 _60_tuskar_boxes.pyc
-rw-r--r--. 1 root root 355 Sep 13 17:02 _60_tuskar_sat_ui.pyc
-rw-r--r--. 1 root root   0 Jan 27  2016 __init__.py
-rw-r--r--. 2 root root 163 Jun  1 12:51 __init__.pyc
-rw-r--r--. 2 root root 163 Jun  1 12:51 __init__.pyo

We have BZ 1324894 where we track the removal of tuskar-* packages. It seems we also need to explicitly remove some of these configuration files. 

There is some relevant info in the doctext for BZ 1353346 

        sudo yum remove tuskar*;
        tuskarpyc_files=(/usr/share/openstack-dashboard/openstack_dashboard/local/enabled/_10_admin.pyc  /usr/share/openstack-dashboard/openstack_dashboard/local/enabled/_20_project.pyc  /usr/share/openstack-dashboard/openstack_dashboard/local/enabled/_30_identity.pyc /usr/share/openstack-dashboard/openstack_dashboard/local/enabled/_50_tuskar.pyc /usr/share/openstack-dashboard/openstack_dashboard/local/enabled/_60_tuskar_boxes.pyc  /usr/share/openstack-dashboard/openstack_dashboard/local/enabled/_60_tuskar_sat_ui.pyc); 
        for file in ${tuskarpyc_files[@]}; do sudo rm -f $file; done

I am not sure why you have the tuaskar_boxes directory in /static/... generally not sure if we can safely remove openstack-dashboard as per your workaround in comment #1, or if we should be more explicit about it and remove the .pyc files and that tuskar_boxes directory only.

The package removal for tuskar in BZ 1324894 essentially does
  sudo', 'yum', 'remove', '-y', '*tuskar*'

I wonder if it is missing some 'tuskar-boxes' package and that is why we have these files left? I went looking and found this https://pypi.python.org/pypi/tuskar-ui-extras not sure if that's it.

I will revisit this Monday - whatever the case it should be a simple enough change - into instack_undercloud/undercloud.py (like in 'die_tuskar_die' @ https://review.openstack.org/#/c/466797/1/instack_undercloud/undercloud.py@1199 )

just posted a patch to https://code.engineering.redhat.com/gerrit/#/c/120011/ - unfortunately it has to be d/stream gerrit since there is no upstream Liberty accepting branches any more :)

It just adds a few lines to the existing 'die_tuskar_die function https://github.com/openstack/instack-undercloud/blob/99319f9ec3299c894ab20ccd5ffd8a237b428c55/instack_undercloud/undercloud.py#L1762-L1771

808     # Also deal with any left-over tuskar related .pyc files causing problems.$
809     # Downstream only OSP8 https://bugzilla.redhat.com/show_bug.cgi?id=1483233$
810     pyc_path = ("/usr/share/openstack-dashboard/openstack_dashboard/"$         
811                 "local/enabled/")$                                             
812     if os.path.isdir(pyc_path):$                                               
813         for pyc_file in os.listdir(pyc_path):$                                 
814             if fnmatch.fnmatchcase(pyc_file, "*.pyc"):$                        
815                 remove_path = os.path.join(pyc_path, pyc_file)$                
816                 if os.path.isfile(remove_path):$                               
817                     os.remove(remove_path)$                                    

Adding the change to trackers - mcornea can we test if removing the pyc files is enough please :)

(In reply to Marios Andreou from comment #5)
> just posted a patch to
> https://code.engineering.redhat.com/gerrit/#/c/120011/ - unfortunately it
> has to be d/stream gerrit since there is no upstream Liberty accepting
> branches any more :)
> 
> It just adds a few lines to the existing 'die_tuskar_die function
> https://github.com/openstack/instack-undercloud/blob/
> 99319f9ec3299c894ab20ccd5ffd8a237b428c55/instack_undercloud/undercloud.
> py#L1762-L1771
> 
> 808     # Also deal with any left-over tuskar related .pyc files causing
> problems.$
> 809     # Downstream only OSP8
> https://bugzilla.redhat.com/show_bug.cgi?id=1483233$
> 810     pyc_path = ("/usr/share/openstack-dashboard/openstack_dashboard/"$  
> 
> 811                 "local/enabled/")$                                      
> 
> 812     if os.path.isdir(pyc_path):$                                        
> 
> 813         for pyc_file in os.listdir(pyc_path):$                          
> 
> 814             if fnmatch.fnmatchcase(pyc_file, "*.pyc"):$                 
> 
> 815                 remove_path = os.path.join(pyc_path, pyc_file)$         
> 
> 816                 if os.path.isfile(remove_path):$                        
> 
> 817                     os.remove(remove_path)$                             
> 
> 
> Adding the change to trackers - mcornea can we test if removing the pyc
> files is enough please :)

Running undercloud upgrade fails with permission denied:
[stack@undercloud-0 ~]$ openstack undercloud upgrade
/usr/lib/python2.7/site-packages/keyring/backends/Gnome.py:6: PyGIWarning: GnomeKeyring was imported without specifying a version first. Use gi.require_version('GnomeKeyring', '1.0') before import to ensure that the right version gets loaded.
  from gi.repository import GnomeKeyring
Loaded plugins: search-disabled-repos
No packages marked for update
Logging to /home/stack/.instack/install-undercloud.log
Checking for a FQDN hostname...
Static hostname detected as undercloud-0.redhat.local
Transient hostname detected as undercloud-0.redhat.local
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib/python2.7/site-packages/instack_undercloud/undercloud.py", line 833, in install
    _die_tuskar_die()
  File "/usr/lib/python2.7/site-packages/instack_undercloud/undercloud.py", line 817, in _die_tuskar_die
    os.remove(remove_path)
OSError: [Errno 13] Permission denied: '/usr/share/openstack-dashboard/openstack_dashboard/local/enabled/__init__.pyc'
Command 'instack-install-undercloud' returned non-zero exit status 1

thanks very much mcornea will look into it and get you an update on the review. I think we can just guard the file removal and make a best effort like we do for the package removal (ie if it fails, don't fail the upgrade try to continue)

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:0543