Bug 921116 - [EC2] Client code needs to be updated to handle new AMI copy feature when selecting yum repo hostname
Summary: [EC2] Client code needs to be updated to handle new AMI copy feature when sel...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Red Hat Update Infrastructure for Cloud Providers
Classification: Red Hat
Component: RHUA
Version: 2.1.1
Hardware: All
OS: Linux
unspecified
high
Target Milestone: ---
: ---
Assignee: James Slagle
QA Contact: mkovacik
URL:
Whiteboard:
Depends On: 956531 956681 957179 957180 1068715
Blocks:
TreeView+ depends on / blocked
 
Reported: 2013-03-13 13:54 UTC by Todd Sanders
Modified: 2016-10-03 13:24 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-08-04 13:11:28 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
Screen log, RHEL 5.5, rh-amazon-rhui-client, build 83-1 (19.30 KB, text/plain)
2013-04-25 09:41 UTC, mkovacik
no flags Details
Screen log, RHEL 5.5, us-east-1 i.e. oldest possible, round-trip, rh-amazon-rhui-client, build 83-1 (17.44 KB, text/plain)
2013-04-25 14:07 UTC, mkovacik
no flags Details
jbeap-ami-c9d600a0.log (26.84 KB, text/plain)
2013-05-07 09:39 UTC, mkovacik
no flags Details
mrg-ami-fcdc4195.log (25.71 KB, text/plain)
2013-05-07 09:39 UTC, mkovacik
no flags Details
vsa-ami-c9d600a0.log (29.71 KB, text/plain)
2013-05-07 09:40 UTC, mkovacik
no flags Details
jbews-ami-c61ebdaf.log (33.34 KB, text/plain)
2013-05-07 11:56 UTC, mkovacik
no flags Details

Description Todd Sanders 2013-03-13 13:54:47 UTC
Description of problem:

EC2 has introduced a new AMI copy feature that allows for copying of AMIs between regions:

"We are excited to announce the immediate availability of a new feature: Amazon Machine Image (AMI) Copy. AMI Copy enables you to copy your AMIs across AWS regions, thus making it easier for you to leverage multiple AWS regions and accelerate your geographical expansion and help increase application performance and availability.

To use AMI Copy, simply select the AMI to be copied from within the AWS Management Console, choose the destination region, and start the copy. AMI Copy can also be accessed via the EC2 Command Line Interface or EC2 API as described in the EC2 User Guide. Once the copy is complete, the new AMI can be used to launch new EC2 instances in the destination region."  

If you boot an instance and then rebundle it to a new AMI, that new AMI is broken if you copy it to a new region, because it has the old
region written in the yum repo config.

Comment 1 wes hayutin 2013-04-18 13:23:36 UTC
commit 20b326119a0d9b7721f2603301ca3af7b6fb470e
Author: Wes Hayutin <whayutin>
Date:   Thu Apr 18 09:21:53 2013 -0400

    bugzilla 921116 partial fix for ami copy issue, sets the region dynamically. Need to update the names of repos in .repo file

Comment 2 wes hayutin 2013-04-19 16:37:10 UTC
commit 8a725be1a8e965a9e0fbf2a628f38f9bbf5222e1
Author: wes hayutin <whayutin>
Date:   Fri Apr 19 12:17:36 2013 -0400

    bugzilla 921116, had to add a fix to dynamically set the load balancer

diff --git a/rel-eng/rhui-2.0/amazon/client/usr/lib/yum-plugins/amazon-id.py b/rel-eng/rhui-2.0/amazon/client/usr/lib/yum-plug
index ecb6d37..8180e5d 100644
--- a/rel-eng/rhui-2.0/amazon/client/usr/lib/yum-plugins/amazon-id.py
+++ b/rel-eng/rhui-2.0/amazon/client/usr/lib/yum-plugins/amazon-id.py
@@ -15,7 +15,7 @@
 
 import base64
 import urllib2
-
+import json
 from urllib2 import ProxyHandler
 from yum.plugins import TYPE_CORE
 
@@ -63,11 +63,12 @@ def init_hook(conduit):
         # bz#921116, copy ami to another region
         # Setting repo.mirrorlist causes traceback, setting repo.url instead
         for repo in rhui_repos:
+            original = repo.mirrorlist
             region_new = json.loads(_load_id())["region"]
-            start = repo.mirrorlist.find(".") + 1
-            end = repo.mirrorlist.find(".", start)
-            region_old = repo.mirrorlist[start:end]
-            repo.url = repo.mirrorlist.replace(region_old, region_new)
+            start = original.find(".") + 1
+            end = original.find(".", start)
+            region_old = original[start:end]
+            repo.mirrorlist = original.replace(region_old, region_new).encode('ascii')
 
 def _load_id():
     '''
diff --git a/rel-eng/rhui-2.0/amazon/client/usr/lib/yum-plugins/rhui-lb.py b/rel-eng/rhui-2.0/amazon/client/usr/lib/yum-plugin
index 14834bd..ab3ad00 100644
--- a/rel-eng/rhui-2.0/amazon/client/usr/lib/yum-plugins/rhui-lb.py
+++ b/rel-eng/rhui-2.0/amazon/client/usr/lib/yum-plugins/rhui-lb.py
@@ -12,6 +12,8 @@
 # http://www.gnu.org/licenses/old-licenses/gpl-2.0.txt.
 
 import httplib
+import json
+import urllib2
 import urlparse
 
 from urlgrabber import grabber
@@ -21,6 +23,8 @@ from yum.plugins import TYPE_CORE, PluginYumExit
 requires_api_version = '2.5'
 plugin_type = (TYPE_CORE,)
 
+ID_DOC_URL = "http://169.254.169.254/latest/dynamic/instance-identity/document"
+
 
 def postreposetup_hook(conduit):
     """
@@ -58,6 +62,18 @@ def postreposetup_hook(conduit):
     up_lb = None
     balancers_tried = 0
     for lb in cds_balancers:
+
+        # Dynamically set the cds load balancer to the appropriate region 
+        # bz#921116, copy ami to another region
+        original = lb
+        region_new = json.loads(_load_id())["region"]
+        start = original.find(".") + 1
+        end = original.find(".", start)
+        region_old = original[start:end]
+        new = original.replace(region_old, region_new).encode('ascii')
+        lb = new
+
+

Comment 3 wes hayutin 2013-04-19 16:38:26 UTC
To change the repo names.. a customer could do the following..

[root@ip-10-197-31-139 ~]# yum search rh-amazon-rhui-client
Loaded plugins: amazon-id, rhui-lb, security
rhui-ap-southeast-2-client-config-server-6                                                             | 2.6 kB     00:00     
rhui-ap-southeast-2-client-config-server-6/primary_db                                                  | 2.5 kB     00:00     
rhui-ap-southeast-2-rhel-server-releases                                                               | 3.7 kB     00:00     
rhui-ap-southeast-2-rhel-server-releases/primary_db                                                    |  20 MB     00:04     
rhui-ap-southeast-2-rhel-server-releases-optional                                                      | 3.5 kB     00:00     
rhui-ap-southeast-2-rhel-server-releases-optional/primary_db                                           | 2.4 MB     00:02     
============================================= N/S Matched: rh-amazon-rhui-client =============================================
rh-amazon-rhui-client.noarch : Yum repository and entitlement certificate configuration

  Name and summary matches only, use "search all" for everything.
[root@ip-10-197-31-139 ~]# yumdownloader rh-amazon-rhui-client.noarch
Loaded plugins: amazon-id, rhui-lb
rh-amazon-rhui-client-2.2.79-1.el6_3.noarch.rpm                                                        |  41 kB     00:00     
[root@ip-10-197-31-139 ~]# rpm -e rh-amazon-rhui-client --nodeps
warning: /etc/yum/pluginconf.d/rhui-lb.conf saved as /etc/yum/pluginconf.d/rhui-lb.conf.rpmsave
warning: /etc/yum.repos.d/rhui-load-balancers.conf saved as /etc/yum.repos.d/rhui-load-balancers.conf.rpmsave
warning: /etc/yum.repos.d/redhat-rhui.repo saved as /etc/yum.repos.d/redhat-rhui.repo.rpmsave
warning: /etc/yum.repos.d/redhat-rhui-client-config.repo saved as /etc/yum.repos.d/redhat-rhui-client-config.repo.rpmsave
[root@ip-10-197-31-139 ~]# rpm -Uvh rh-amazon-rhui-client-2.2.79-1.el6_3.noarch.rpm 
Preparing...                ########################################### [100%]
   1:rh-amazon-rhui-client  ########################################### [100%]
[INFO:choose_repo] choose_repo:36 2013-04-19 12:30:20,241: Zone [us-west-1c]
[INFO:choose_repo] choose_repo:47 2013-04-19 12:30:20,241: Setting region in redhat-rhui.repo
[INFO:choose_repo] choose_repo:49 2013-04-19 12:30:20,241: Executing [sed -i 's/REGION/us-west-1/' /etc/yum.repos.d/redhat-rhui.repo]
[INFO:choose_repo] choose_repo:53 2013-04-19 12:30:20,263: Enabling binary repos in redhat-rhui.repo
[INFO:choose_repo] choose_repo:72 2013-04-19 12:30:20,265: Enabling load balancer plugin
[INFO:choose_repo] choose_repo:74 2013-04-19 12:30:20,265: Executing [sed -i 's/enabled=0/enabled=1/' /etc/yum/pluginconf.d/rhui-lb.conf]
[INFO:choose_repo] choose_repo:78 2013-04-19 12:30:20,273: Setting region in load balancer config
[INFO:choose_repo] choose_repo:80 2013-04-19 12:30:20,274: Executing [sed -i 's/REGION/us-west-1/' /etc/yum.repos.d/rhui-load-balancers.conf]
[INFO:choose_repo] choose_repo:84 2013-04-19 12:30:20,282: Enabling client config repo
[INFO:choose_repo] choose_repo:87 2013-04-19 12:30:20,283: Executing [sed -i 's/enabled=0/enabled=1/' /etc/yum.repos.d/redhat-rhui-client-config.repo]
[INFO:choose_repo] choose_repo:91 2013-04-19 12:30:20,290: Setting region in redhat-rhui-client-config.repo
[INFO:choose_repo] choose_repo:93 2013-04-19 12:30:20,291: Executing [sed -i 's/REGION/us-west-1/' /etc/yum.repos.d/redhat-rhui-client-config.repo]
[root@ip-10-197-31-139 ~]# yum repolist
Loaded plugins: amazon-id, rhui-lb, security
rhui-us-west-1-client-config-server-6                                                                  | 2.6 kB     00:00     
rhui-us-west-1-client-config-server-6/primary_db                                                       | 2.5 kB     00:00     
rhui-us-west-1-rhel-server-releases                                                                    | 3.7 kB     00:00     
rhui-us-west-1-rhel-server-releases/primary_db                                                         |  20 MB     00:00     
rhui-us-west-1-rhel-server-releases-optional                                                           | 3.5 kB     00:00     
rhui-us-west-1-rhel-server-releases-optional/primary_db                                                | 2.4 MB     00:00     
repo id                                           repo name                                                             status
rhui-us-west-1-client-config-server-6             Red Hat Update Infrastructure 2.0 Client Configuration Server 6            1
rhui-us-west-1-rhel-server-releases               Red Hat Enterprise Linux Server 6 (RPMs)                              10,365
rhui-us-west-1-rhel-server-releases-optional      Red Hat Enterprise Linux Server 6 Optional (RPMs)                      6,074
repolist: 16,440
[root@ip-10-197-31-139 ~]#

Comment 4 wes hayutin 2013-04-22 13:39:25 UTC
set of changes for this issue

commit b3244ab25430787b2162c6045e5a78e4c7269888
Author: Wes Hayutin <whayutin>
Date:   Mon Apr 22 09:04:39 2013 -0400

    move region change into postreposetup_hook

commit a45ce378c3daafa74c7bfc83491e2d9f7b7e9f3c
Author: wes hayutin <whayutin>
Date:   Sat Apr 20 13:28:33 2013 -0400

    bz#921116, added explanation in each repo file for 'REGION' appearing in the repo name and url

commit 12cd4f2727c6ac471db9ddb9abb0d62f8e98e2b6
Author: wes hayutin <whayutin>
Date:   Sat Apr 20 13:26:34 2013 -0400

    add level 5 debug logging to dynamic region setting

commit 0800f18969a9b9ef6bbdb5f59cbe2179960dfb76
Author: wes hayutin <whayutin>
Date:   Sat Apr 20 12:36:44 2013 -0400

    bz#921116, do not set the region name in the repo files, leave it as REGION so it is consistent across regions.

commit 371a6009a62c7cab4266e5b3d3a06dd923a39ee1
Author: wes hayutin <whayutin>
Date:   Sat Apr 20 12:35:41 2013 -0400

    only change the region if it doesn't match the region in the id document

commit 8a725be1a8e965a9e0fbf2a628f38f9bbf5222e1
Author: wes hayutin <whayutin>
Date:   Fri Apr 19 12:17:36 2013 -0400

    bugzilla 921116, had to add a fix to dynamically set the load balancer

commit 20b326119a0d9b7721f2603301ca3af7b6fb470e
Author: Wes Hayutin <whayutin>
Date:   Thu Apr 18 09:21:53 2013 -0400

    bugzilla 921116 partial fix for ami copy issue, sets the region dynamically. Need to update the names of repos in .repo files

Comment 5 mkovacik 2013-04-24 14:15:13 UTC
Hey, applying the patch on RHEL 5.6 ami-2eb1825a I've got quite an unforeseen stack; rhui plugins can't be loaded...


[mkovacik@localhost rhui-client]$ ssh -i ~/.pem/mkovacik-eu-west-1.pem root.compute.amazonaws.com                                                                                                               
[root@ip-10-59-142-183 ~]# rpm -Uvh /tmp/rh-amazon-rhui-client-2.2.81-1.el5.noarch.rpm 
Preparing...                ########################################### [100%]
   1:rh-amazon-rhui-client  warning: /etc/yum/pluginconf.d/rhui-lb.conf saved as /etc/yum/pluginconf.d/rhui-lb.conf.rpmsave
########################################### [100%]
[INFO:choose_repo] choose_repo:36 2013-04-24 09:56:06,835: Zone [eu-west-1c]
[INFO:choose_repo] choose_repo:57 2013-04-24 09:56:06,836: Enabling binary repos in redhat-rhui.repo
[INFO:choose_repo] choose_repo:76 2013-04-24 09:56:06,836: Enabling load balancer plugin
[INFO:choose_repo] choose_repo:78 2013-04-24 09:56:06,837: Executing [sed -i 's/enabled=0/enabled=1/' /etc/yum/pluginconf.d/rhui-lb.conf]
[INFO:choose_repo] choose_repo:82 2013-04-24 09:56:06,843: Setting region in load balancer config
[INFO:choose_repo] choose_repo:84 2013-04-24 09:56:06,843: Executing [sed -i 's/REGION/eu-west-1/' /etc/yum.repos.d/rhui-load-balancers.conf]
[INFO:choose_repo] choose_repo:88 2013-04-24 09:56:06,848: Enabling client config repo
[INFO:choose_repo] choose_repo:91 2013-04-24 09:56:06,849: Executing [sed -i 's/enabled=0/enabled=1/' /etc/yum.repos.d/redhat-rhui-client-config.repo]
warning: /etc/yum.repos.d/redhat-eu-west.repo saved as /etc/yum.repos.d/redhat-eu-west.repo.rpmsave
warning: /etc/yum.repos.d/redhat-eu-west-client-config.repo saved as /etc/yum.repos.d/redhat-eu-west-client-config.repo.rpmsave
[root@ip-10-59-142-183 ~]# yum clean all ; yum -d 5 repolist -v 
Plugin "amazon-id" can't be imported
Plugin "rhui-lb" can't be imported
Loaded plugins: fastestmirror, security
Cleaning up Everything
Plugin "amazon-id" can't be imported
Loading "fastestmirror" plugin
Not loading "rhnplugin" plugin, as it is disabled
Plugin "rhui-lb" can't be imported
Loading "security" plugin
Config time: 0.170
Yum Version: 3.2.22
Determining fastest mirrors
Could not retrieve mirrorlist https://rhui2-cds01.REGION.aws.ce.redhat.com/pulp/mirror/rhui-client-config/rhel/server/5/x86_64/os error was
[Errno 4] IOError: <urlopen error (-2, 'Name or service not known')>
Error: Cannot find a valid baseurl for repo: rhui-REGION-client-config-server-5
[root@ip-10-59-142-183 ~]# cat /etc/yum.repos.d/rhui-load-balancers.conf 
rhui2-cds01.eu-west-1.aws.ce.redhat.com
rhui2-cds02.eu-west-1.aws.ce.redhat.com

Comment 6 mkovacik 2013-04-25 09:41:21 UTC
Created attachment 739757 [details]
Screen log, RHEL 5.5, rh-amazon-rhui-client, build 83-1

Works OK for RHEL 5.5

Comment 7 mkovacik 2013-04-25 14:07:18 UTC
Created attachment 739920 [details]
Screen log, RHEL 5.5, us-east-1 i.e. oldest possible, round-trip, rh-amazon-rhui-client, build 83-1

Works OK

Comment 8 mkovacik 2013-05-07 09:39:06 UTC
Created attachment 744645 [details]
jbeap-ami-c9d600a0.log

Comment 9 mkovacik 2013-05-07 09:39:51 UTC
Created attachment 744646 [details]
mrg-ami-fcdc4195.log

Comment 10 mkovacik 2013-05-07 09:40:18 UTC
Created attachment 744647 [details]
vsa-ami-c9d600a0.log

Comment 11 mkovacik 2013-05-07 11:56:03 UTC
Created attachment 744695 [details]
jbews-ami-c61ebdaf.log

Comment 14 Bryan Kearney 2016-08-04 13:11:28 UTC
We believe this is fixed. If you are still seeing issues, please feel free to re-open.


Note You need to log in before you can comment on or make changes to this bug.