Bug 1285023 - [ceph-deploy] : permission errors while running ceph-deploy after following the doc for repo based installation on ubuntu
[ceph-deploy] : permission errors while running ceph-deploy after following t...
Product: Red Hat Ceph Storage
Classification: Red Hat
Component: Documentation (Show other bugs)
x86_64 Linux
unspecified Severity medium
: rc
: 1.3.2
Assigned To: ceph-docs@redhat.com
Depends On:
  Show dependency treegraph
Reported: 2015-11-24 12:04 EST by shylesh
Modified: 2015-12-18 05:07 EST (History)
4 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2015-12-18 05:07:09 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)

  None (edit)
Description shylesh 2015-11-24 12:04:58 EST
For repo based installation on ubuntu I followed the doc https://gitlab.cee.redhat.com/ngoswami/red-hat-ceph-storage-installation-guide-ubuntu/blob/devel/calamari.adoc. In the "set online repositories" section we have to run 

sudo bash -c 'umask 0077; echo deb https://customername:customerpasswd@rhcs.download.redhat.com/ubuntu/1.3-updates/Installer $(lsb_release -sc) main | tee /etc/apt/sources.list.d/Installer.list'
sudo bash -c 'wget -O - https://www.redhat.com/security/fd431d51.txt | apt-key add -'
sudo apt-get update
sudo apt-get install ceph-deploy
sudo ceph-deploy repo --repo-url 'https://customername:customerpasswd@rhcs.download.redhat.com/ubuntu/1.3-updates/Calamari' Calamari `hostname -f`
sudo ceph-deploy repo --repo-url 'https://customername:customerpasswd@rhcs.download.redhat.com/ubuntu/1.3-updates/Tools' Tools `hostname -f`

after running ceph-deploy command as sudo 2 file will be generated
 .cephdeploy.conf and ceph.log with root as owner and without write permission for others.

so in the next step if we run 

ceph-deploy repo --repo-url 'https://customername:customerpasswd@rhcs.download.redhat.com/ubuntu/1.3-updates/MON' --gpg-url https://www.redhat.com/security/fd431d51.txt Monitor <MONHOST1> [<MONHOST2> [<MONHOST3> ...]]

This command fails with 

ceph@magna043:~/new$ ceph-deploy repo --repo-url 'https://redhat:OgYZNpkj6jZAIF20XFZW0gnnwYBjYcmt7PeY76bLHec9@rhcs.download.redhat.com/ubuntu/1.3-updates/MON' --gpg-url https://www.redhat.com/security/fd431d51.txt Monitor magna044
[ceph_deploy][ERROR ] Traceback (most recent call last):
[ceph_deploy][ERROR ]   File "/usr/lib/python2.7/dist-packages/ceph_deploy/util/decorators.py", line 69, in newfunc
[ceph_deploy][ERROR ]     return f(*a, **kw)
[ceph_deploy][ERROR ]   File "/usr/lib/python2.7/dist-packages/ceph_deploy/cli.py", line 150, in _main
[ceph_deploy][ERROR ]     fh = logging.FileHandler('{cluster}.log'.format(cluster=args.cluster))
[ceph_deploy][ERROR ]   File "/usr/lib/python2.7/logging/__init__.py", line 903, in __init__
[ceph_deploy][ERROR ]     StreamHandler.__init__(self, self._open())
[ceph_deploy][ERROR ]   File "/usr/lib/python2.7/logging/__init__.py", line 928, in _open
[ceph_deploy][ERROR ]     stream = open(self.baseFilename, self.mode)
[ceph_deploy][ERROR ] IOError: [Errno 13] Permission denied: '/home/ceph/new/ceph.log'
[ceph_deploy][ERROR ]

I did a workaround of changing the permission of the files and it works fine.
Comment 1 Nilamdyuti 2015-11-24 13:00:07 EST
Shylesh discussed this with me over IRC for a probable doc change so I felt I should have a look at it too.

I logged in to magna043 and magna044 as "ceph" user and repeated the steps from a new directory "ceph-test" in magna043. magna043 is admin node and magna044 is monitor node. Before repeating the steps I removed the gpg keys, repos, disabled a mod_fastcgi repo, removed ~/.cephdeploy.conf from the admin node and uninstalled ceph-deploy. I also removed gpg key, Monitor repo from magna044 and disabled mod_fastcgi repo.

I then repeated all the steps and found the same error. If ceph-deploy is run as sudo in the admin node while adding Calamari and Tools repos, the generated ceph.log file gets root as it's owner and the subsequent command to add Monitor repo in Monitor node fails with "Permission denied" error as mentioned by Shylesh.

Changing the permission of ceph.log allowed the ceph-deploy command to add Monitor repo in monitor node to run successfully.

So, there is definitely a issue with running ceph-deploy commands with sudo in admin node for adding Calamari and Tools online repos and it needs to be investigated.

Alfredo, can you please have a look at this?
Comment 3 Alfredo Deza 2015-11-25 09:25:09 EST
Unfortunately, this is correct and I am not sure there is something ceph-deploy can do to workaround this. This is a consequence of running with sudo and in order to be able to continue the logs will need to be changed with proper permissions.

The only thing I can think of that would be transparent for a user is if they were using root directly or if we change the code to always use a special type of connection that will prefix commands with sudo regardless if they are root or not.

This is a side-effect of bending ceph-deploy to perform actions in situations that weren't planned (e.g. create files that require super user privileges in an admin server)
Comment 4 Nilamdyuti 2015-11-26 04:42:50 EST
As our docs have sudo mentioned in the ceph-deploy commands for admin node, our customers might face this issue while trying Install/Upgrade using online repos. So, as a immediate solution, I propose either one of the following two doc changes:

1. Change the permission of ceph.log in admin node that has root ownership as a result of ceph-deploy commands being run with sudo. I just have to add a single command after the "sudo ceph-deploy..." commands to add Calamari and Tools online repos.

The command would be: sudo chown ceph:ceph ceph.log

2. Remove "sudo" from the prefix of ceph-deploy commands for admin node. This way the generated ceph.log file will have the ceph-deploy user (e.g 'ceph') as the file owner. As a result when subsequent ceph-deploy commands are run to add Monitor or OSD online repos in remote nodes, there won't be any permission issue and the log file will get appended easily.

I actually don't get the logic behind adding sudo only for the ceph-deploy commands of admin node. The ceph-deploy user that we create (e.g 'ceph'), we enable passwordless sudo access for it. So, when ceph-deploy is run to add a online repo to /etc/apt/sources.list.d/ it does it without any restriction as ceph user already has sudo access. It works perfectly when online Calamari and Tools repo is added using "ceph-deploy..." commands without sudo. I tested that too in magna043.ceph.redhat.com (admin node) by creating another ceph working directory 'ceph-test2' for ceph-deploy user 'ceph' and executed the ceph-deploy commands without sudo. The repos were successfully added to /etc/apt/sources.list.d/ and there were no permission errors. The generated 'ceph.log' file in 'ceph-test2' directory had 'ceph' as it's owner.

However, if the source code for ceph-deploy is coded to have sudo for ceph-deploy in admin node for superuser permissions required for some other files then maybe removing sudo from the docs won't be the right thing to do.

Let me know which of the above doc changes sounds good to you as an immediate solution.
Comment 5 Alfredo Deza 2015-11-30 08:44:34 EST
I went through the code again and tried replicating the issue and Nilamdyuti is right: there is no need to prefix ceph-deploy usage with `sudo`.

When ceph-deploy connects to remote hosts (or in this case locally) it has a "detection for sudo need" that will automatically use sudo for system calls when the user that is calling ceph-deploy is not the root user or it is not using sudo.

The fix here should be to remove the explicit use of 'sudo' from the docs.

Nilamdyuti thanks for taking the time to investigate this a bit further and ascertain that there is no need to use sudo with ceph-deploy.
Comment 7 shylesh 2015-12-03 12:37:45 EST
I tested the doc changes for installation and hemanth tested the same for upgrade and confirmed that it works. Hence marking this as verified.
Comment 8 Anjana Suparna Sriram 2015-12-18 05:07:09 EST
Fixed in 1.3.2 release.

Note You need to log in before you can comment on or make changes to this bug.