Bug 1868765

Summary:	[vsphere][ci] could not reserve an IP address: no available addresses
Product:	OpenShift Container Platform	Reporter:	Joseph Callen <jcallen>
Component:	Installer	Assignee:	Joseph Callen <jcallen>
Installer sub component:	openshift-installer	QA Contact:	jima
Status:	CLOSED ERRATA	Docs Contact:
Severity:	low
Priority:	low	CC:	adahiya, bleanhar, ccoleman, jima, mstaeble
Version:	4.6	Keywords:	UpcomingSprint
Target Milestone:	---
Target Release:	4.7.0
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	No Doc Update
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2021-02-24 15:15:36 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Joseph Callen 2020-08-13 18:39:22 UTC

Description of problem:

1.) The current CI subnet is public ipv4 and limited. If teardown does not 
happen correctly addresses will be reserved and not in use.
2.) When all the available addresses are use phpIPAM provides a 0.0.0.0
address that causes additional issues.

This will be resolved in multiple steps:
1.) Moving to VMC, CI build/orch cluster with VPC access to VMC environment. Large subnet private ipv4 subnet (/22)
2.) Replace phpIPAM with netbox.

Without having a specific component for this to land I added it to installer. I plan on working on this.



Version-Release number of the following components:
rpm -q openshift-ansible
rpm -q ansible
ansible --version

How reproducible:

Steps to Reproduce:
1.
2.
3.

Actual results:
Please include the entire output from the last TASK line through the end of output if an error is generated

Expected results:

Additional info:
Please attach logs from ansible-playbook with the -vvv flag

Comment 1 Clayton Coleman 2020-08-13 18:45:17 UTC

Manifesting in job like https://prow.ci.openshift.org/view/gs/origin-ci-test/logs/release-openshift-ocp-installer-e2e-vsphere-upi-serial-4.6/1293945192834928640 with:

Error: Error running command 'echo '{"network":"139.178.94.128","hostname":"bootstrap-0.ci-op-lj68vtgi-5c9dd.origin-ci-int-aws.dev.rhcloud.com","ipam":"139.178.89.254","ipam_token":"EYmMFjaaQDNp7i1MlHRUZ0kPZC2hUHky"}' | ipam/cidr_to_ip.sh
': exit status 1. Output: could not reserve an IP address: no available addresses

Comment 4 Joseph Callen 2020-10-09 18:18:26 UTC

With IPI keepalived vrrp id collisions we have now created multiple nsx-t segments. Each segment has a 192.168.x.0/27 subnet where x corresponds to ci-segment-x

https://github.com/openshift/release/compare/master...jcpowermac:move_vsphere_to_networks

This is currently blocked because LEASED_RESOURCE is empty in multistep:
https://issues.redhat.com/browse/DPTP-1626

Comment 8 Joseph Callen 2020-11-30 19:46:42 UTC

We need this PR to merge first: https://github.com/kubernetes-sigs/boskos/pull/54

Comment 13 errata-xmlrpc 2021-02-24 15:15:36 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 4.7.0 security, bug fix, and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2020:5633