What happened? When I try to add a dependency to the installer, or execute any other dependency related action I see the next error: ❯ go mod tidy go: github.com/openshift/machine-api-operator.1-0.20200429102619-d36974451290 requires github.com/openshift/library-go.0-20200324092245-db2a8546af81 requires bitbucket.org/ww/goautoneg.0-20120707110453-75cd24fc2f2c: reading https://api.bitbucket.org/2.0/repositories/ww/goautoneg?fields=scm: 404 Not Found What did you expect to happen? `go mod tidy` should work How to reproduce it (as minimally and precisely as possible)? Try to add/update a dependency, or just execute `go mod tidy`. If these commands work, then try to remove "api.bitbucket.org/2.0/repositories/ww/goautoneg" from the local cache.
I've set the Severity to urgent as it blocks go operations critical to development.
Initial investigation: 1. machine-api-operator, cluster-api-provider-{aws,azure,gcp,openstack} have a broken dependency (https://api.bitbucket.org/2.0/repositories/ww/goautoneg?fields=scm) 2. bumping them fixes the issue with the broken dependency, but... 3. bumping of cluster-api-provider-gcp fetches a new version of cloud.google.com/go 4. in the new version they broke google cloud api for dns, and installation fails because current terraform-provider-google can't work with new google api (https://prow.ci.openshift.org/view/gs/origin-ci-test/pr-logs/pull/openshift_installer/4349/pull-ci-openshift-installer-master-e2e-gcp/1325961889447940096) 5. bumping of terraform-provider-google causes a failure in terraform-plugin-sdk, which has different signatures for some public functions 6. since we don't use public terraform-plugin-sdk and have a downstream copy (https://github.com/openshift/hashicorp-terraform-plugin-sdk) we need to upgrade it first. 7. when I upgraded the downstream terraform-plugin-sdk and tried to use it in the installer several other terraform providers (aws, azurerm, ignition and vsphere) got broken. 8. because we use only downstream copies of the broken terraform providers, we need to upgrade them first (github.com/openshift/terraform-provider-aws, github.com/openshift/terraform-provider-azurerm, github.com/community-terraform-providers/terraform-provider-ignition/v2, github.com/openshift/terraform-provider-vsphere).
Basically now we can't add or upgrade installer dependencies.
For anybody else hitting this, I may have a workaround of sorts. There is obviously no way this workaround is complete and therefore some things must remain broken. In fact the workaround probably breaks new things I haven't discovered, yet. Therefore I wouldn't weight this workaround highly if deciding whether this bug should be a blocker or not (it should be a blocker). I see from strace that go is trying to access $GOPATH/pkg/mod/cache/download/bitbucket.org/ww/goautoneg/@v/v0.0.0-20120707110453-75cd24fc2f2c.mod, which doesn't exist. After looking at another .mod file I guessed that "module bitbucket.org/ww/goautoneg" as a single line of text might be valid contents for this file. I created the file with these contents, and now 'go get' doesn't complain that it can't access bitbucket.org and fetches the requested module, which I was also able to vendor.
Similar issue reported in bug 1876799. That workaround sounds nicer, but I'm not sure I understand it as we don't have a direct dep on autoneg. Is this some syntax in go.mod to specify an alternate for the indirect dep?
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Moderate: OpenShift Container Platform 4.7.0 security, bug fix, and enhancement update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2020:5633