Bug 2034319 - Negation constraint is not validating packages
Summary: Negation constraint is not validating packages
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: OLM
Version: 4.10
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: ---
: 4.10.0
Assignee: Tyler Slaton
QA Contact: Bruno Andrade
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2021-12-20 16:31 UTC by Bruno Andrade
Modified: 2022-03-10 16:35 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2022-03-10 16:35:23 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github openshift operator-framework-olm pull 248 0 None open Bug 2034319: Sync constraint changes 2022-01-27 21:59:42 UTC
Red Hat Product Errata RHSA-2022:0056 0 None None None 2022-03-10 16:35:35 UTC

Description Bruno Andrade 2021-12-20 16:31:32 UTC
Description of problem:

A bundle with negation olm.constraint including a package was able to be installed even with the package is installed in the namespace.

Version-Release number of selected component (if applicable):
4.10.0-0.ci-2021-12-19-122218
OLM version: 0.19.0
git commit: 13f7723ba6098a17c7c44da17d844bb0d87d30a7


How reproducible:
Always

Steps to Reproduce:
1. Have an index with etcd bundle with the following constraints:

image: quay.io/openshift-community-operators/etcd@sha256:94346b5ee6149d1411b2f37f815526db3b86e62a03879337f6194428d52c336e
name: etcdoperator.v0.9.4
package: etcd
properties:
- type: olm.constraint
  value:
    message: All are denied for etcd because..
    none:
      constraints:
      - message: Package prometheus is needed for...
        package:
         packageName: prometheus
         versionRange: '>=0.27.0'
index: quay.io/bandrade/etcd1-index:11.0 was used

2. Create a namespace, og, and subscription for prometheus

 oc get csv -n test-1                                                                                                                                                                                                                130 ↵
NAME                              DISPLAY                            VERSION    REPLACES                    PHASE
elasticsearch-operator.5.3.1-16   OpenShift Elasticsearch Operator   5.3.1-16                               Succeeded
prometheusoperator.0.47.0         Prometheus Operator                0.47.0     prometheusoperator.0.37.0   Succeeded

3. Install etcd package from the mentioned catalog, it should fail but it's being successfully installed:

oc get csv -n test-1                                                                                                                                                                                                                
NAME                              DISPLAY                            VERSION    REPLACES                    PHASE
elasticsearch-operator.5.3.1-16   OpenShift Elasticsearch Operator   5.3.1-16                               Succeeded
etcdoperator.v0.9.4               etcd                               0.9.4      etcdoperator.v0.9.2         Succeeded
prometheusoperator.0.47.0         Prometheus Operator                0.47.0     prometheusoperator.0.37.0   Succeeded



Expected results:
Should restrict the installation of the bundle since a restricted package is installed

Additional info:

Comment 1 Nick Hale 2021-12-21 20:16:46 UTC
Thanks for the report Bruno!

Could you provide a must-gather for the cluster?

Thanks

Comment 2 Bruno Andrade 2021-12-27 16:38:51 UTC
Hi Nick,

I tried again on 4.10.0-0.nightly-2021-12-23-153012 and it worked, I'll keep testing in another version and let you know if I find any issues again.

oc get csv -n test-1           
NAME                             DISPLAY                            VERSION   REPLACES              PHASE
elasticsearch-operator.5.3.2-9   OpenShift Elasticsearch Operator   5.3.2-9                         Succeeded
etcdoperator.v0.9.4              etcd                               0.9.4     etcdoperator.v0.9.2   Succee


Closing as NOT_A_BUG for now, thanks.

Comment 3 Bruno Andrade 2022-01-05 15:29:48 UTC
Hi Nick,

I tested again and I'm seeing this issue again, I'll attach the must-gather as you requested.

Reopening for investigation.

Comment 5 Nick Hale 2022-01-05 20:13:37 UTC
It looks like we might have already run into an issue with the compound "constraints" as predicates implementation. For my fellow engineers on the OLM team, I suspect that this is happening because predicates act as package cache filters rather than being pulled into the actual SAT solving algorithm. This means that operators which are already installed can have properties/constraints that SHOULD conflict with installation candidates but DON'T; i.e. since they were prematurely filtered from the SAT problem, instead of included w/ "must exist" constraints (as is the norm for already installed operators).

If we verify that this is indeed the root cause, we'll need to reimplement compound constraints as SAT solver constraints (instead of predicates).

Raising severity.

Comment 6 Tyler Slaton 2022-01-26 18:40:51 UTC
Hi, giving an update here for those watching. 

While this BZ initially looks incorrect, the behavior here is actually working as expected (not a bug). The main issue here seems to be with the documentation which does not properly explain how negation (and really compound constraints in general) work. To explain, a negation in the root of an `olm.constraint` does not do much by itself. What "none" is essentially saying is "for this constraint, it cannot be solved by the defined package". Because of this, negation is best used in a nested constraint, like so:

---
type: olm.constraint
value:
  message: All are required because...
  all:
    constraints:
    - all:
        constraints:
        - failureMessage: GVK Buf/v1 required for...
          gvk:
            group: bufs.example.com
            version: v1
            kind: Buf
    - not:
        constraints:
        - failureMessage: Package foo version >=1.0.0 cannot be required for...
          package:
            name: foo
            versionRange: '>=1.0.0'
---


This statement essentially says that "Package foo with version >= v1.0.0 cannot be used a solution for the requirement of the Buf GVK".

** Negation does not mean that the given package cannot be installed on the cluster in order for the constraint to be satisfied. ** 

To make these points very clear, we are updating the EP (https://github.com/operator-framework/enhancements/pull/108) with new clarifications and examples. We are also going to update the `None` type to be `Not` which should make the UX a bit clearer (https://github.com/operator-framework/api/pull/221).

Comment 9 Bruno Andrade 2022-01-31 17:35:49 UTC
Hi Tyler,


Thanks for your explanation, in order to verify this bug, I created a bundle with the following constraints:

image: quay.io/openshift-community-operators/etcd@sha256:94346b5ee6149d1411b2f37f815526db3b86e62a03879337f6194428d52c336e
name: etcdoperator.v0.9.4
package: etcd
properties:
- type: olm.constraint
  value:
    all:
     constraints:
      - all:
         constraints:
         - failureMessage: PodMonitor monitoring.coreos.com/v1 required for...
           gvk:
             group: monitoring.coreos.com
             version: v1
             kind: PodMonitor
      - not:
         constraints:
         - failureMessage: Package prometheus version >=0.37.0 cannot be required for...
           package:
             packageName: prometheus
             versionRange: '>0.37.0'


Image: quay.io/bandrade/etcd1-index:25.0

Considering your explanation package prometheus with version >0.37.0 can't be used to solve PodMonitor GVK. When I create the subscription, it avoids csv install due to not having any package to meet bundle requirements, however, the message is not shown in the subscription object:

oc get clusterversion                                                                                                                                                                                                               
NAME      VERSION                              AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.10.0-0.nightly-2022-01-29-015515   True        False         111m    Cluster version is 4.10.0-0.nightly-2022-01-29-0155

oc get csv -n test-1              
NAME                              DISPLAY                            VERSION    REPLACES                          PHASE
elasticsearch-operator.5.3.4-12   OpenShift Elasticsearch Operator   5.3.4-12   elasticsearch-operator.5.3.4-11   Succeeded


oc get subs etcd -n test-1 -o yaml
apiVersion: operators.coreos.com/v1alpha1
kind: Subscription
metadata:
  creationTimestamp: "2022-01-31T17:21:36Z"
  generation: 1
  labels:
    operators.coreos.com/etcd.test-1: ""
  name: etcd
  namespace: test-1
  resourceVersion: "69447"
  uid: 40b08429-55c4-409e-8a2e-8ee077c30265
spec:
  channel: singlenamespace-alpha
  installPlanApproval: Automatic
  name: etcd
  source: ocp-47181
  sourceNamespace: openshift-marketplace
status:
  catalogHealth:
  - catalogSourceRef:
      apiVersion: operators.coreos.com/v1alpha1
      kind: CatalogSource
      name: certified-operators
      namespace: openshift-marketplace
      resourceVersion: "69277"
      uid: 8dd011cc-f20f-48a2-aa19-87cfdee46d11
    healthy: true
    lastUpdated: "2022-01-31T17:21:36Z"
  - catalogSourceRef:
      apiVersion: operators.coreos.com/v1alpha1
      kind: CatalogSource
      name: community-operators
      namespace: openshift-marketplace
      resourceVersion: "69284"
      uid: d67f39b9-74c6-4bee-9aed-01f29580a80b
    healthy: true
    lastUpdated: "2022-01-31T17:21:36Z"
  - catalogSourceRef:
      apiVersion: operators.coreos.com/v1alpha1
      kind: CatalogSource
      name: ocp-47181
      namespace: openshift-marketplace
      resourceVersion: "69280"
      uid: 32824fba-12c0-4114-bc87-5319def9d782
    healthy: true
    lastUpdated: "2022-01-31T17:21:36Z"
  - catalogSourceRef:
      apiVersion: operators.coreos.com/v1alpha1
      kind: CatalogSource
      name: qe-app-registry
      namespace: openshift-marketplace
      resourceVersion: "69281"
      uid: 7dba8898-0640-4b84-b8be-cd2329c25953
    healthy: true
    lastUpdated: "2022-01-31T17:21:36Z"
  - catalogSourceRef:
      apiVersion: operators.coreos.com/v1alpha1
      kind: CatalogSource
      name: redhat-marketplace
      namespace: openshift-marketplace
      resourceVersion: "69278"
      uid: b4577d58-741b-43c9-946e-6647031fcdc9
    healthy: true
    lastUpdated: "2022-01-31T17:21:36Z"
  - catalogSourceRef:
      apiVersion: operators.coreos.com/v1alpha1
      kind: CatalogSource
      name: redhat-operators
      namespace: openshift-marketplace
      resourceVersion: "69286"
      uid: f713319c-a30e-4def-8a6f-4624b6d7ede2
    healthy: true
    lastUpdated: "2022-01-31T17:21:36Z"
  conditions:
  - lastTransitionTime: "2022-01-31T17:21:36Z"
    message: all available catalogsources are healthy
    reason: AllCatalogSourcesHealthy
    status: "False"
    type: CatalogSourcesUnhealthy
  lastUpdated: "2022-01-31T17:21:36Z"


In catalog log, I just see "install plan not found" message:


E0131 17:21:36.588450       1 queueinformer_operator.go:290] sync "test-1" failed: installplans.operators.coreos.com "install-lt7mr" not found
time="2022-01-31T17:21:36Z" level=info msg=syncing event=update reconciling="*v1alpha1.Subscription" selflink=
time="2022-01-31T17:21:36Z" level=warning msg="an error was encountered during reconciliation" error="Operation cannot be fulfilled on subscriptions.operators.coreos.com \"etcd\": the object has been modified; please apply your changes to the latest version and try again" event=update reconciling="*v1alpha1.Subscription" selflink=
E0131 17:21:36.600547       1 queueinformer_operator.go:290] sync {"update" "test-1/etcd"} failed: Operation cannot be fulfilled on subscriptions.operators.coreos.com "etcd": the object has been modified; please apply your changes to the latest version and try again
time="2022-01-31T17:21:36Z" level=info msg=syncing event=update reconciling="*v1alpha1.Subscription" selflink=
time="2022-01-31T17:21:36Z" level=warning msg="unable to get installplan from cache" channel=beta id=jk1iw installplan=install-lt7mr namespace=test-1 pkg=prometheus source=community-operators sub=prometheus-beta-community-operators-openshift-marketplace
E0131 17:21:36.606165       1 queueinformer_operator.go:290] sync "test-1" failed: installplans.operators.coreos.com "install-lt7mr" not found
time="2022-01-31T17:21:36Z" level=info msg=syncing event=update reconciling="*v1alpha1.Subscription" selflink=
time="2022-01-31T17:21:36Z" level=warning msg="unable to get installplan from cache" channel=beta id=THugR installplan=install-lt7mr namespace=test-1 pkg=prometheus source=community-operators sub=prometheus-beta-community-operators-openshift-marketplace
E0131 17:21:36.762324       1 queueinformer_operator.go:290] sync "test-1" failed: installplans.operators.coreos.com "install-lt7mr" not found
time="2022-01-31T17:21:37Z" level=warning msg="unable to get installplan from cache" channel=beta id=zBCWh installplan=install-lt7mr namespace=test-1 pkg=prometheus source=community-operators sub=prometheus-beta-community-operators-openshift-marketplace
E0131 17:21:37.161687       1 queueinformer_operator.go:290] sync "test-1" failed: installplans.operators.coreos.com "install-lt7mr" not found
time="2022-01-31T17:21:37Z" level=warning msg="unable to get installplan from cache" channel=beta id=mxg/r installplan=install-lt7mr namespace=test-1 pkg=prometheus source=community-operators sub=prometheus-beta-community-operators-openshift-marketplace
E0131 17:21:37.561683       1 queueinformer_operator.go:290] sync "test-1" failed: installplans.operators.coreos.com "install-lt7mr" not found
time="2022-01-31T17:21:37Z" level=warning msg="unable to get installplan from cache" channel=beta id=KsemU installplan=install-lt7mr namespace=test-1 pkg=prometheus source=community-operators sub=prometheus-beta-community-operators-openshift-marketplace
E0131 17:21:37.962017       1 queueinformer_operator.go:290] sync "test-1" failed: installplans.operators.coreos.com "install-lt7mr" not found
time="2022-01-31T17:21:38Z" level=warning msg="unable to get installplan from cache" channel=beta id=/FfLE installplan=install-lt7mr namespace=test-1 pkg=prometheus source=community-operators sub=prometheus-beta-community-operators-openshift-marketplace
E0131 17:21:38.362247       1 queueinformer_operator.go:290] sync "test-1" failed: installplans.operators.coreos.com "install-lt7mr" not found
time="2022-01-31T17:21:38Z" level=warning msg="unable to get installplan from cache" channel=beta id=AZUep installplan=install-lt7mr namespace=test-1 pkg=prometheus source=community-operators sub=prometheus-beta-community-operators-openshift-marketplace
E0131 17:21:38.762377       1 queueinformer_operator.go:290] sync "test-1" failed: installplans.operators.coreos.com "install-lt7mr" not found
time="2022-01-31T17:21:39Z" level=warning msg="unable to get installplan from cache" channel=beta id=MyeC1 installplan=install-lt7mr namespace=test-1 pkg=prometheus source=community-operators sub=prometheus-beta-community-operators-openshift-marketplace
time="2022-01-31T17:21:39Z" level=warning msg="unable to get installplan from cache" channel=beta id=+XWew installplan=install-lt7mr namespace=test-1 pkg=prometheus source=community-operators sub=prometheus-beta-community-operators-openshift-marketplace
E0131 17:21:39.561368       1 queueinformer_operator.go:290] sync "test-1" failed: installplans.operators.coreos.com "install-lt7mr" not found
time="2022-01-31T17:21:39Z" level=warning msg="unable to get installplan from cache" channel=beta id=1ovru installplan=install-lt7mr namespace=test-1 pkg=prometheus source=community-operators sub=prometheus-beta-community-operators-openshift-marketplace
E0131 17:21:39.962612       1 queueinformer_operator.go:290] sync "test-1" failed: installplans.operators.coreos.com "install-lt7mr" not found
time="2022-01-31T17:21:40Z" level=warning msg="unable to get installplan from cache" channel=beta id=rs5dU installplan=install-lt7mr namespace=test-1 pkg=prometheus source=community-operators sub=prometheus-beta-community-operators-openshift-marketplace
E0131 17:21:40.362681       1 queueinformer_operator.go:290] sync "test-1" failed: installplans.operators.coreos.com "install-lt7mr" not found
time="2022-01-31T17:21:40Z" level=warning msg="unable to get installplan from cache" channel=beta id=Ug20D installplan=install-lt7mr namespace=test-1 pkg=prometheus source=community-operators sub=prometheus-beta-community-operators-openshift-marketplace
E0131 17:21:40.762998       1 queueinformer_operator.go:290] sync "test-1" failed: installplans.operators.coreos.com "install-lt7mr" not found
time="2022-01-31T17:21:41Z" level=warning msg="unable to get installplan from cache" channel=beta id=43sla installplan=install-lt7mr namespace=test-1 pkg=prometheus source=community-operators sub=prometheus-beta-community-operators-openshift-marketplace
E0131 17:21:41.161747       1 queueinformer_operator.go:290] sync "test-1" failed: installplans.operators.coreos.com "install-lt7mr" not found
time="2022-01-31T17:21:41Z" level=warning msg="unable to get installplan from cache" channel=beta id=+Es5/ installplan=install-lt7mr namespace=test-1 pkg=prometheus source=community-operators sub=prometheus-beta-community-operators-openshift-marketplace
E0131 17:21:41.562852       1 queueinformer_operator.go:290] sync "test-1" failed: installplans.operators.coreos.com "install-lt7mr" not found
time="2022-01-31T17:21:41Z" level=warning msg="unable to get installplan from cache" channel=beta id=d5U3S installplan=install-lt7mr namespace=test-1 pkg=prometheus source=community-operators sub=prometheus-beta-community-operators-openshift-marketplace
E0131 17:21:41.961977       1 queueinformer_operator.go:290] sync "test-1" failed: installplans.operators.coreos.com "install-lt7mr" not found
time="2022-01-31T17:21:42Z" level=warning msg="unable to get installplan from cache" channel=beta id=uG/YT installplan=install-lt7mr namespace=test-1 pkg=prometheus source=community-operators sub=prometheus-beta-community-operators-openshift-marketplace
E0131 17:21:42.362163       1 queueinformer_operator.go:290] sync "test-1" failed: installplans.operators.coreos.com "install-lt7mr" not found
time="2022-01-31T17:21:43Z" level=warning msg="unable to get installplan from cache" channel=beta id=R6KYw installplan=install-lt7mr namespace=test-1 pkg=prometheus source=community-operators sub=prometheus-beta-community-operators-openshift-marketplace

I'll attach the must-gather for verification.

Comment 11 Tyler Slaton 2022-01-31 21:10:20 UTC
Hi again, Bruno.

Thanks for looking into this. When you ran this test, was it on a fresh namespace/cluster? If not, would you mind doing these same steps on either a fresh cluster or a fresh namespace? Looking at the must-gather you attached, it would seem that there is a failed prometheus subscription in that `test-1` namespace so it would be good to confirm how that got there as it may be causing issues.

Comment 16 Bruno Andrade 2022-02-04 07:55:07 UTC
Hi Tyler,

Thanks for your explanation, marking as VERIFIED considering https://bugzilla.redhat.com/show_bug.cgi?id=2034319#c14

Cluster version: 4.10.0-0.nightly-2022-01-29-015515

Comment 18 errata-xmlrpc 2022-03-10 16:35:23 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 4.10.3 security update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:0056


Note You need to log in before you can comment on or make changes to this bug.