Bug 1295604

This is the most urgent Doc bug for 1.3.2 from PM's point of view.

Hi Bara,
  let's start with apt-cacher. I do not have a preference between the two, as long as we get to a working setup.

  1.I believe apt-cacher is older and -ng is a more recent project, but unclear where the effort is going these days.  Barring setting up a full Landscape server (probably too much work for our use), either will do.

  2.Use case: assume cluster not connected to the internet, will get OS and Ceph upgrades via cache. As you said, install cache first, then use it to install RHCS. Assume OS already installed.

  3. Access to online repo: contact Harish, ask for credentials QE uses in testing.

  4. Landscape is great, but if we can solve this just with a cache it is probably simpler for us and for the end-user. Setting up a dedicated landscape server is a bit more complex, and I do not want to see users that get stuck in doing that have to deal with both our support and Canonicals... it gets complicated.

Hi Federico, 

Could you please provide me more background why apt-cacher is needed? What exactly is going to change with 1.3.2? 

My understanding is:

ice-setup will not be there any more in 1.3.2, so we need to find out some other way how to install Ceph off-line (ISO-based installation [1]). This change does not affect online installation using the online repositories. 

So we need to set up the caching server as described here: https://help.ubuntu.com/community/Apt-Cacher-Server (sections Server Installation, Server Configuration) and import package from the ISO to the cache (section Load Packages Server from CD). 

Then we should be able to install ceph-deploy from that loaded packages and using ceph-deploy install Ceph without the need to execute ice-setup first.

Is my understanding correct? 

This is what I tried to do and, unfortunately, failed:

1) I set up the cache based on the information here: https://help.ubuntu.com/community/Apt-Cacher-Server. 

2) I imported packages an Ceph ISO (rhceph-1.3.1-ubuntu-x86_64-dvd.iso). 

3) Set up another machine to use the caching server. 

4) But when I tried to install the ice-setup I got "E: Unable to locate package ice-setup". The same with ceph-deploy. (I originally thought that ceph-deploy was installed correctly, but then I found out that some other repo, which was installed by default, includes ceph-deploy. When I disabled half of the repos, I got the same error as with ice-setup)

I honestly don't know what to do next, since I tried to troubleshoot this issue, but without any luck. Whom should I contact? QA? I have a WIP version of the procedure how it should work based on the information provided in the https://help.ubuntu.com/community/Apt-Cacher-Server:

http://etherpad.corp.redhat.com/ubuntu-caching

[1] https://access.redhat.com/documentation/en/red-hat-ceph-storage/version-1.3/installation-guide-for-ubuntu-x86-64/#installation_via_iso )

I'm adding Nilam's insights:

I setup 4 Ubuntu Trusty VMs and attempted the RHCS v1.3.1 installation from cache. First I prepared each node with pre-installation requirements. Then I configured the admin node as the apt-cacher server. I followed all the steps properly and configured the server's apt process to run through cache but still apt seems to pick up the default deb url for apt i.e, archive.ubuntu.com. I used a proxy to apt and even manually changed /etc/apt/sources.list to point to the apt-cacher server (another way of pointing to apt-cacher server) but the best it did was install ceph-deploy version 1.4 (an older version) which won't support the new options such as "repo" which are supported by the new version 1.5.27.3. That's the reason your ceph-deploy commands were failing when you were trying to install Ceph with cache installed ceph-deploy. Also, I could not install calamari-server, calamari-clients as apt failed to locate the packages.

Although the "apt-cacher-import.pl" script claimed to have successfully imported 50 odd packages from the RHCS v1.3.1 ISO in one of the previous steps but I don't think it was entirely successful, otherwise it would have found the latest ceph-deploy package from the ISO, Calamari and other ceph packages.

I think to install Ceph from cache, the approach would be a bit different as in https://help.ubuntu.com/community/Apt-Cacher-Server, in the step "Load Packages on Server from CD" the apt-cacher-import.pl script is used to import the packages from Ubuntu ISO i.e, the OS and here we are trying to import packages from Ceph ISO that will be installed on top of the OS.

Hi Bara,
  Nothing will change with 1.3.2, ice-setup is still there.

  However, with 2.0 ice-setup will be gone. For this reason, we have introduced support for Satellite 6 (which covers RHEL), and we will introduce Satellite 5 and Apt-cacher.

  The procedure should conceptually mirror the steps we have to work with Satellite 6 in RHEL, so you can take inspiration there.

  I think it is time we bring in someone from QE and engineering to help you draft this.  Contact Harish for QE, and Gregory Meno for Engineering, they will assign you more eyes to look at this.

Thanks!

It sounds like you've tried to setup a package cache and are testing it by installing ceph-deploy -- which failed. Is this the current status?

Hi Gregory, 

yes, and I also tried to install ice-setup and I got "E: Unable to locate package ...".

Nilam thinks that the problem is when you try to import packages from an ISO which is not ISO with Ubuntu. The packages got imported to the /var/cache/apt-cacher/packages directory but the apt-get install could find them.. 

Otherwise, the cache works with normal Ubuntu repos (for example I was able to install vim using that)

The steps here - https://help.ubuntu.com/community/Apt-Cacher-Server - works, but only with Ubuntu repos not with Ceph ISO.

Bara

I too tried to setup the Cache Server and Install ceph using that but was unable to proceed further., As Nilam mentioned, it is installing ceph-deploy 1.4
which is very old and could not do much. I just followed the procedure to set up the cache server from here :"https://help.ubuntu.com/community/Apt-Cacher-Server",

I tried apt-cacher today, and the apt-cacher-import.pl script will not work for our use-case. The script just dumps all the .deb files into a cache directory without importing any of the repo metadata. In other words, the .deb files are "cached" on the apt-cacher server, but Apt clients have no way to discover them.

I tried mounting the ISO and copying the files to /opt, but that failed because apt-cacher errors on the file:// URL scheme.

apt-cacher will work in scenarios where the apt-cacher server itself has access to the internet and would serve as a "gateway" of sorts. It would not work in cases where the cluster's apt-cacher server is fully disconnected from the internet, and the customer wants to install from an ISO.

By contrast, apt-cacher-ng is much easier to set up. There is only a single line change needed in its configuration file ("LocalDirs"). I've documented the walkthru here: http://etherpad.corp.redhat.com/ubuntu-caching

Ken, thank you very much for clarifying this. 

Federico, do we want to create a separate article on the Customer Portal (as for the Satellite and Ceph) or should I add this information to the pre-installation section of the Ubuntu Installation guide? 

Thank you, 

Bara

Hi Bara,
 Anjana also asked me in person yesterday, here is the answer :)

 We want to do the same thing for the Ubuntu Caching approach as we did for Satellite —  so KB article sounds good. We do not want to modify the installation plan.  

Ken — thanks for solving this for us, your outline rocks. Moving to apt-cacher-ng is a-ok by me.

 We will want to figure out a good title, since folks will know to search for Satellite but finding this one for Ubuntu will be harder. I suggest "Package Caching for Red Hat Ceph storage on Ubuntu". If there is a keywords section, we can help the likelihood a search will find this with a bit of SEO.

verified

By mistake I had moved this bz to verified state. Moving it back to ON_QA.

Here are my observation from link: https://access.redhat.com/articles/2148241


Setting Up a Caching Server
---------------------------
Step:-
2 . If you have a firewall enabled on the caching server, open the TCP 3142 port.

--- Lets add the command to open port 3142
sudo iptables -I INPUT 1 -i <iface> -p tcp -s <ip-address>/<netmask> --dport 3142 -j ACCEPT
sudo apt-get install iptables-persistent

3. Download the Red Hat Ceph Storage ISO.

--- From where to download the ISO ?
 We need to provide the doc link to get the ISO - Lets add this link here - https://access.redhat.com/articles/1554343

7. Copy the ISO files to the directory in /opt/:
sudo cp -a /mnt/rhcs-1.3/* /opt/rhcs-1.3

--- Lets use the same mount path as mentioned in step 4 & step 5. Change the example to :- sudo cp -a /mnt/rhcs-iso/* /opt/rhcs-1.3

8. Open the acng.conf file located in the /etc/apt-cacher-ng/ directory and modify it to point to the directory created in step 5. Edit the LocalDirs setting as follows:

--- There already exists a line with "LocalDirs: acng-doc /usr/share/doc/apt-cacher-ng" - We should ask the customer to comment out that line or replace with "LocalDirs: [name] [directory]"


Configuring the Ceph Cluster to Use the Caching Server
------------------------------------------------------
Before moving to configuration, we should first provide the link to complete the per-requisites for Installing a Ceph Cluster. 
Then test the connection from any of the cluster node and then proceed with Installing the Admin Node, then MON ,OSD and RGW

If someone who has never done Installation using CDN or ISO will have to struggle, Let's follow the similar procedure as we have for CDN Installation here and replace the CDN repo location with apt-cache location.

1. On the Calamari node, verify that the node can reach the cache stored on the caching server:

"curl -l http://[caching_server]/[name]"  -- This command fails. Correct command is 

"curl -I http://[caching_server]/[name]"

Verified the Doc. Looks okay to me ..
Moving to Verified..