Bug 1295604 - [Doc] Ubuntu Package cache
[Doc] Ubuntu Package cache
Product: Red Hat Ceph Storage
Classification: Red Hat
Component: Documentation (Show other bugs)
Unspecified Unspecified
unspecified Severity unspecified
: rc
: 1.3.2
Assigned To: Bara Ancincova
: Documentation, ZStream
Depends On:
Blocks: 1213028 1299303
  Show dependency treegraph
Reported: 2016-01-04 20:34 EST by Federico Lucifredi
Modified: 2016-03-01 03:22 EST (History)
9 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
.Package caching for Ubuntu is now supported With this release, a caching server can be set up to provide Red Hat Ceph Storage repositories for offline Ceph clusters. See the https://access.redhat.com/articles/2148241[Package Caching for Red Hat Ceph Storage on Ubuntu] article on the Red Hat Customer Portal to learn more.
Story Points: ---
Clone Of:
Last Closed: 2016-03-01 03:22:19 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)

  None (edit)
Description Federico Lucifredi 2016-01-04 20:34:14 EST
Description of problem:

Obsoleting the package cache provided by ICE Setup in Ceph 2.0 will require customers to use alternative package caching solutions. For RHEL, we introduced Satellite 6 support in RHCS 1.3.1, while for Ubuntu users we will provide an Apt-Caher and/or Landscape workflow in this release.

Documentation needs to be provided on how the customer can set up a disconnected package cache. 

Trello card: https://trello.com/c/5prPVhAL
Comment 2 Federico Lucifredi 2016-01-04 20:35:44 EST
This is the most urgent Doc bug for 1.3.2 from PM's point of view.
Comment 4 Federico Lucifredi 2016-01-11 10:36:28 EST
Hi Bara,
  let's start with apt-cacher. I do not have a preference between the two, as long as we get to a working setup.

  1.I believe apt-cacher is older and -ng is a more recent project, but unclear where the effort is going these days.  Barring setting up a full Landscape server (probably too much work for our use), either will do.

  2.Use case: assume cluster not connected to the internet, will get OS and Ceph upgrades via cache. As you said, install cache first, then use it to install RHCS. Assume OS already installed.

  3. Access to online repo: contact Harish, ask for credentials QE uses in testing.

  4. Landscape is great, but if we can solve this just with a cache it is probably simpler for us and for the end-user. Setting up a dedicated landscape server is a bit more complex, and I do not want to see users that get stuck in doing that have to deal with both our support and Canonicals... it gets complicated.
Comment 5 Bara Ancincova 2016-01-13 10:20:27 EST
Hi Federico, 

Could you please provide me more background why apt-cacher is needed? What exactly is going to change with 1.3.2? 

My understanding is:

ice-setup will not be there any more in 1.3.2, so we need to find out some other way how to install Ceph off-line (ISO-based installation [1]). This change does not affect online installation using the online repositories. 

So we need to set up the caching server as described here: https://help.ubuntu.com/community/Apt-Cacher-Server (sections Server Installation, Server Configuration) and import package from the ISO to the cache (section Load Packages Server from CD). 

Then we should be able to install ceph-deploy from that loaded packages and using ceph-deploy install Ceph without the need to execute ice-setup first.

Is my understanding correct? 

This is what I tried to do and, unfortunately, failed:

1) I set up the cache based on the information here: https://help.ubuntu.com/community/Apt-Cacher-Server. 

2) I imported packages an Ceph ISO (rhceph-1.3.1-ubuntu-x86_64-dvd.iso). 

3) Set up another machine to use the caching server. 

4) But when I tried to install the ice-setup I got "E: Unable to locate package ice-setup". The same with ceph-deploy. (I originally thought that ceph-deploy was installed correctly, but then I found out that some other repo, which was installed by default, includes ceph-deploy. When I disabled half of the repos, I got the same error as with ice-setup)

I honestly don't know what to do next, since I tried to troubleshoot this issue, but without any luck. Whom should I contact? QA? I have a WIP version of the procedure how it should work based on the information provided in the https://help.ubuntu.com/community/Apt-Cacher-Server:


[1] https://access.redhat.com/documentation/en/red-hat-ceph-storage/version-1.3/installation-guide-for-ubuntu-x86-64/#installation_via_iso )
Comment 6 Bara Ancincova 2016-01-14 11:57:44 EST
I'm adding Nilam's insights:

I setup 4 Ubuntu Trusty VMs and attempted the RHCS v1.3.1 installation from cache. First I prepared each node with pre-installation requirements. Then I configured the admin node as the apt-cacher server. I followed all the steps properly and configured the server's apt process to run through cache but still apt seems to pick up the default deb url for apt i.e, archive.ubuntu.com. I used a proxy to apt and even manually changed /etc/apt/sources.list to point to the apt-cacher server (another way of pointing to apt-cacher server) but the best it did was install ceph-deploy version 1.4 (an older version) which won't support the new options such as "repo" which are supported by the new version That's the reason your ceph-deploy commands were failing when you were trying to install Ceph with cache installed ceph-deploy. Also, I could not install calamari-server, calamari-clients as apt failed to locate the packages.

Although the "apt-cacher-import.pl" script claimed to have successfully imported 50 odd packages from the RHCS v1.3.1 ISO in one of the previous steps but I don't think it was entirely successful, otherwise it would have found the latest ceph-deploy package from the ISO, Calamari and other ceph packages.

I think to install Ceph from cache, the approach would be a bit different as in https://help.ubuntu.com/community/Apt-Cacher-Server, in the step "Load Packages on Server from CD" the apt-cacher-import.pl script is used to import the packages from Ubuntu ISO i.e, the OS and here we are trying to import packages from Ceph ISO that will be installed on top of the OS.
Comment 7 Federico Lucifredi 2016-01-14 21:04:06 EST
Hi Bara,
  Nothing will change with 1.3.2, ice-setup is still there.

  However, with 2.0 ice-setup will be gone. For this reason, we have introduced support for Satellite 6 (which covers RHEL), and we will introduce Satellite 5 and Apt-cacher.

  The procedure should conceptually mirror the steps we have to work with Satellite 6 in RHEL, so you can take inspiration there.

  I think it is time we bring in someone from QE and engineering to help you draft this.  Contact Harish for QE, and Gregory Meno for Engineering, they will assign you more eyes to look at this.

Comment 9 Gregory Meno 2016-01-18 12:37:36 EST
It sounds like you've tried to setup a package cache and are testing it by installing ceph-deploy -- which failed. Is this the current status?
Comment 10 Bara Ancincova 2016-01-19 06:40:11 EST
Hi Gregory, 

yes, and I also tried to install ice-setup and I got "E: Unable to locate package ...".

Nilam thinks that the problem is when you try to import packages from an ISO which is not ISO with Ubuntu. The packages got imported to the /var/cache/apt-cacher/packages directory but the apt-get install could find them.. 

Otherwise, the cache works with normal Ubuntu repos (for example I was able to install vim using that)

The steps here - https://help.ubuntu.com/community/Apt-Cacher-Server - works, but only with Ubuntu repos not with Ceph ISO.

Comment 11 Hemanth Kumar 2016-01-21 07:30:34 EST
I too tried to setup the Cache Server and Install ceph using that but was unable to proceed further., As Nilam mentioned, it is installing ceph-deploy 1.4
which is very old and could not do much. I just followed the procedure to set up the cache server from here :"https://help.ubuntu.com/community/Apt-Cacher-Server",
Comment 12 Ken Dreyer (Red Hat) 2016-02-01 17:25:01 EST
I tried apt-cacher today, and the apt-cacher-import.pl script will not work for our use-case. The script just dumps all the .deb files into a cache directory without importing any of the repo metadata. In other words, the .deb files are "cached" on the apt-cacher server, but Apt clients have no way to discover them.

I tried mounting the ISO and copying the files to /opt, but that failed because apt-cacher errors on the file:// URL scheme.

apt-cacher will work in scenarios where the apt-cacher server itself has access to the internet and would serve as a "gateway" of sorts. It would not work in cases where the cluster's apt-cacher server is fully disconnected from the internet, and the customer wants to install from an ISO.

By contrast, apt-cacher-ng is much easier to set up. There is only a single line change needed in its configuration file ("LocalDirs"). I've documented the walkthru here: http://etherpad.corp.redhat.com/ubuntu-caching
Comment 13 Bara Ancincova 2016-02-02 04:44:21 EST
Ken, thank you very much for clarifying this. 

Federico, do we want to create a separate article on the Customer Portal (as for the Satellite and Ceph) or should I add this information to the pre-installation section of the Ubuntu Installation guide? 

Thank you, 

Comment 14 Federico Lucifredi 2016-02-03 01:46:19 EST
Hi Bara,
 Anjana also asked me in person yesterday, here is the answer :)

 We want to do the same thing for the Ubuntu Caching approach as we did for Satellite —  so KB article sounds good. We do not want to modify the installation plan.  

Ken — thanks for solving this for us, your outline rocks. Moving to apt-cacher-ng is a-ok by me.

 We will want to figure out a good title, since folks will know to search for Satellite but finding this one for Ubuntu will be harder. I suggest "Package Caching for Red Hat Ceph storage on Ubuntu". If there is a keywords section, we can help the likelihood a search will find this with a bit of SEO.
Comment 19 Harish NV Rao 2016-02-11 11:04:40 EST
Comment 20 Harish NV Rao 2016-02-15 09:29:55 EST
By mistake I had moved this bz to verified state. Moving it back to ON_QA.
Comment 21 Hemanth Kumar 2016-02-16 06:55:06 EST
Here are my observation from link: https://access.redhat.com/articles/2148241

Setting Up a Caching Server
2 . If you have a firewall enabled on the caching server, open the TCP 3142 port.

--- Lets add the command to open port 3142
sudo iptables -I INPUT 1 -i <iface> -p tcp -s <ip-address>/<netmask> --dport 3142 -j ACCEPT
sudo apt-get install iptables-persistent

3. Download the Red Hat Ceph Storage ISO.

--- From where to download the ISO ?
 We need to provide the doc link to get the ISO - Lets add this link here - https://access.redhat.com/articles/1554343

7. Copy the ISO files to the directory in /opt/:
sudo cp -a /mnt/rhcs-1.3/* /opt/rhcs-1.3

--- Lets use the same mount path as mentioned in step 4 & step 5. Change the example to :- sudo cp -a /mnt/rhcs-iso/* /opt/rhcs-1.3

8. Open the acng.conf file located in the /etc/apt-cacher-ng/ directory and modify it to point to the directory created in step 5. Edit the LocalDirs setting as follows:

--- There already exists a line with "LocalDirs: acng-doc /usr/share/doc/apt-cacher-ng" - We should ask the customer to comment out that line or replace with "LocalDirs: [name] [directory]"

Configuring the Ceph Cluster to Use the Caching Server
Before moving to configuration, we should first provide the link to complete the per-requisites for Installing a Ceph Cluster. 
Then test the connection from any of the cluster node and then proceed with Installing the Admin Node, then MON ,OSD and RGW

If someone who has never done Installation using CDN or ISO will have to struggle, Let's follow the similar procedure as we have for CDN Installation here and replace the CDN repo location with apt-cache location.

1. On the Calamari node, verify that the node can reach the cache stored on the caching server:

"curl -l http://[caching_server]/[name]"  -- This command fails. Correct command is 

"curl -I http://[caching_server]/[name]"
Comment 25 Hemanth Kumar 2016-02-17 08:18:56 EST
Verified the Doc. Looks okay to me ..
Moving to Verified..

Note You need to log in before you can comment on or make changes to this bug.