Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1825084

Summary: Support web-console initiated updates in restricted network environments
Product: OpenShift Container Platform Reporter: Hideshi Fukumoto <hfukumot>
Component: OpenShift Update ServiceAssignee: Lalatendu Mohanty <lmohanty>
OpenShift Update Service sub component: operator QA Contact: liujia <jiajliu>
Status: CLOSED ERRATA Docs Contact:
Severity: medium    
Priority: medium CC: ableisch, aos-bugs, aos-team-ota, bleanhar, jack.ottofaro, jiajliu, jokerman, lmohanty, mfuruta, wking, yanyang
Version: 4.6   
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2021-06-02 05:39:45 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Comment 1 Scott Dodson 2020-04-17 12:09:20 UTC
This is expected if the cluster cannot talk to the Internet, is this cluster configured with a proxy?

Comment 4 Scott Dodson 2020-04-21 17:11:49 UTC
Need to document that clusterversions/version upstream needs to be emptied.

Lowering priority back to medium because a cluster that cannot connect to the Internet isn't going to receive updates anyway so there's no functional impact here. This is purely cosmetic.

Comment 5 Hideshi Fukumoto 2020-04-22 06:34:16 UTC
Hi,

I think OCP 4.3 is supported in restricted networks environments / disconnected environments [1].

In fact, our customer installed OCP 4.3 under the disconnected network environment, and he confirmed
that the following CLI based upgrade procedure (with "--force" option) was successful from OCP 4.3.9 to
OCP 4.3.10. (based on the github[2])

  a. As the customer's OCP cluster version was 4.3.9, he could upgrade it to 4.3.10 or 4.3.12 using Stable-4.3 channel.
     So, first he updated the contents in the private registry based on doc [3].

  b. Then (after mirroring OCP private repository), he run "oc adm upgrade" command with the following options:

      $ oc adm upgrade --to-image {local_registry}/{local_repository}:{ocp_release} --allow-explicit-upgrade --force
      
      ex) 
        oc adm upgrade --to-image local-ocp-registry:50000/ocp4/openshift4:4.3.10-x86_64 --allow-explicit-upgrade --force

        --to-image: Specify an image to upgrade
        --allow-explicit-upgrade: Upgrade even if the upgrade target is not listed in the available versions list
        --force: Do upgrade if signature cannot be gotten from internet

  I think this is a workaround using CLI for this GUI issue, hoowever the end user customer would like to upgrade
  the cluster safely & easily via Web Console GUI.

[1] https://docs.openshift.com/container-platform/4.3/installing/install_config/installing-restricted-networks-preparations.html
[2] https://github.com/openshift/openshift-docs/blob/5de3002649cf4bb9b96b170f78bdd514cc322c24/modules/update-restricted-network-cli.adoc
[3] https://access.redhat.com/documentation/en-us/openshift_container_platform/4.3/html-single/installing/index#installation-mirror-repository_installing-restricted-networks-preparations

Best Regards,
Hideshi

Comment 6 W. Trevor King 2020-05-04 20:55:49 UTC
> Need to document that clusterversions/version upstream needs to be emptied.

Turns out it's the channel that needs to be emptied [1].

> ... the end user customer would like to upgrade the cluster safely & easily via Web Console GUI.

Web console is probably not going to happen (at least via some officially-documented procedure) 4.3, although we're working on enabling it in a future 4.y.

The safety side of things was the signature issue, and that should be better since bug 1783054 (although we still need openshift-docs around that :/).  Or did you have additional safety concerns beyond that?

It's hard to handle multiple issues (some of which seem like feature requests) in a single bug.  Are there particular aspects remaining that do not have existing, more-focused bugs that you still need addressed?

[1]: https://bugzilla.redhat.com/show_bug.cgi?id=1827378#c4

Comment 7 Hideshi Fukumoto 2020-05-07 03:00:25 UTC
(In reply to W. Trevor King from comment #6)

> Web console is probably not going to happen (at least via some
> officially-documented procedure) 4.3, although we're working on enabling it
> in a future 4.y.
> 
> The safety side of things was the signature issue, and that should be better
> since bug 1783054 (although we still need openshift-docs around that :/). 
> Or did you have additional safety concerns beyond that?
> 
> It's hard to handle multiple issues (some of which seem like feature
> requests) in a single bug.  Are there particular aspects remaining that do
> not have existing, more-focused bugs that you still need addressed?
> 
> [1]: https://bugzilla.redhat.com/show_bug.cgi?id=1827378#c4

  Thanks for your comment.

  In this bug, I'd like to focus on the issue on the Web Console GUI.
  And, for the safety side of things, I'd like to follow it via the other bug#1828214.

Comment 8 W. Trevor King 2020-05-19 04:18:30 UTC
Lots of work still to do before we have official docs around running a local Cincinnati service to support updates from the web console in restricted-network environments.  We're actively working on it, but it's going to land in 4.6 first.

Comment 10 Lalatendu Mohanty 2020-06-18 15:24:10 UTC
>   In this bug, I'd like to focus on the issue on the Web Console GUI.

Can you please explain little about your expectation here? I am assuming you do not want to see the error on the web console in the restricted-network environment. But not sure if you meant the same.

Comment 11 Hideshi Fukumoto 2020-06-18 22:08:36 UTC
(In reply to Lalatendu Mohanty from comment #10)
> >   In this bug, I'd like to focus on the issue on the Web Console GUI.
> 
> Can you please explain little about your expectation here? I am assuming you
> do not want to see the error on the web console in the restricted-network
> environment. But not sure if you meant the same.

  Our expectation is to be able to update the cluster via the Web Console
  without issuing any error messages.

Comment 12 W. Trevor King 2020-07-06 22:13:56 UTC
Restoring UpcomingSprint.  Comment 8 is still current.

Comment 13 Jack Ottofaro 2020-07-30 20:04:23 UTC
Restoring UpcomingSprint.  Comment 8 is still current.

Comment 14 Lalatendu Mohanty 2020-08-20 18:49:28 UTC
Adding UpcomingSprint.  https://bugzilla.redhat.com/show_bug.cgi?id=1825084#c8 is still current.

Comment 15 W. Trevor King 2020-08-25 17:18:32 UTC
[1] is the Tech Preview flow for setting up a local update service, which you can use to get web-console-initiated updates in restricted network environments.  Not sure if that's sufficient to close out this bug, or if we should wait for it to GA, which, as the blog post points out, should happen alongside OCP 4.6.

[1]: https://www.openshift.com/blog/openshift-update-service-update-manager-for-your-cluster

Comment 18 Lalatendu Mohanty 2020-08-31 14:32:22 UTC
Moving this bug to 4.6.z as running Cincinnati i.e. OpenShift Update service not possible at this point of time.

Comment 20 W. Trevor King 2020-08-31 17:12:00 UTC
Setting back to 4.6.0 to keep Eric's bot happy, because we aren't punting this to 4.7 and it doesn't seem worth moving this to 4.7 just so we can clone it back to 4.6.z.

Comment 21 W. Trevor King 2020-08-31 18:26:35 UTC
I didn't realize we had an update-service product now.  Moving this bug over there.

Comment 29 W. Trevor King 2020-09-16 23:29:00 UTC
I'm pretty sure that the GA releases of the OpenShift Update Service will be sufficient to close this bug, because a local Update Service is all you need to have web-console initiated updates.  That release will be somewhere after OCP's 4.6 goes GA, and hopefully before OCP's 4.7 goes GA.  But exact timing for the Update Service's first GA release is hard to predict.  Getting the Update Service out in a timely GA is currently one of our top priorities on the updates team.

Comment 30 W. Trevor King 2020-09-16 23:29:28 UTC
... but it's not going to happen this sprint ;).

Comment 32 W. Trevor King 2020-11-20 17:16:33 UTC
Moving under OCP, now that we are back to being a sub-component instead of a parallel (in Bugzilla) project.

Comment 34 W. Trevor King 2020-12-02 05:45:17 UTC
4.6 has been out for a bit now.  We're closing in on a GA release of the OpenShift Update Service operator.  Remaining bits in flight include [1,2].

[1]: https://github.com/openshift/cincinnati-operator/pull/93
[2]: https://github.com/openshift/openshift-docs/pull/26219

Comment 38 liujia 2021-01-25 02:28:56 UTC
Hi Hideshi Fukumoto

From what I have known, developers are keep working on it, but also together with other workloads. Maybe @Lala can give more update here.

Comment 44 liujia 2021-05-19 09:05:08 UTC
OCP version:4.6.0-0.nightly-2021-05-15-131411
osus version: 
cincinnati-container-v4.6.0-10
cincinnati-operator-container-v4.6.0-7

1. Install osus on OCP to build update graph against local registry.
# ./oc get pod -n openshift-update-service 
NAME                                     READY   STATUS    RESTARTS   AGE
update-686949dcb9-zwzdn                  2/2     Running   0          5m40s
updateservice-operator-9d4bc5b79-j4gb7   1/1     Running   0          44m
# ./oc get route -n openshift-update-service 
NAME                         HOST/PORT                                                                                          PATH   SERVICES               PORT            TERMINATION   WILDCARD
update-policy-engine-route   update-policy-engine-route-openshift-update-service.apps.jliu-46.qe.gcp.devcluster.openshift.com          update-policy-engine   policy-engine   edge/None     None

2. Check current message with default upstream(before pointing the upstream to osus service).
# ./oc adm upgrade
Cluster version is 4.6.0-0.nightly-2021-05-15-131411

warning: Cannot display available updates:
  Reason: RemoteFailed
  Message: Unable to retrieve available updates: Get "https://api.openshift.com/api/upgrades_info/v1/graph?arch=amd64&channel=stable-4.6&id=4d134a56-fcea-43c6-8beb-411c7580bad5&version=4.6.0-0.nightly-2021-05-15-131411": dial tcp 54.243.181.251:443: connect: connection timed out

3. Update upstream to use osus's policyEngineURI in CV
# ./oc get clusterversion -o json|jq .items[].spec
{
  "channel": "stable-4.6",
  "clusterID": "4d134a56-fcea-43c6-8beb-411c7580bad5",
  "upstream": "https://update-policy-engine-route-openshift-update-service.apps.jliu-46.qe.gcp.devcluster.openshift.com/api/upgrades_info/v1/graph"
}

4. Check the error message dissapeared from both cli and web-console.
# ./oc adm upgrade
Cluster version is 4.6.0-0.nightly-2021-05-15-131411

No updates available. You may force an upgrade to a specific release image, but doing so may not be supported and result in downtime or data loss.

So according to above, the issue should be supported through osus now.

Comment 45 liujia 2021-05-19 09:12:20 UTC
We already designed test cases for epic OTA-269, which has covered this scenario. So move NeedsTestCase

Comment 48 errata-xmlrpc 2021-06-02 05:39:45 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (RHEA: OSUS enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2021:2196