Bug 1888380

Summary: Azure Quota Limits do not list Spot VMs
Product: OpenShift Container Platform Reporter: Joel Speed <jspeed>
Component: DocumentationAssignee: Chinmayi Chandrasekhar <cchandra>
Status: CLOSED CURRENTRELEASE QA Contact: MayXu <maxu>
Severity: medium Docs Contact: Latha S <lmurthy>
Priority: medium    
Version: 4.6CC: aos-bugs, gpei, jialiu, lmurthy, wking
Target Milestone: ---   
Target Release: 4.6.z   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2022-01-20 06:36:35 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Joel Speed 2020-10-14 17:37:19 UTC
Document URL: https://docs.openshift.com/container-platform/4.6/installing/installing_azure/installing-azure-account.html#installation-azure-limits_installing-azure-account

Section Number and Name: Azure Account Limits

Describe the issue: The docs do not mention that a cluster may need Spot VM limits increasing in case the user wants to use spot VMs (see azure docs https://docs.microsoft.com/en-us/azure/azure-resource-manager/management/azure-subscription-service-limits#virtual-machines-limits---azure-resource-manager).

When a user first tries to create Spot VMs, if they exceed the default of 10 cores, they will have issues in Machine API that will tell them `LowPriorityCores` are required, which doesn't immediately match the name of the limit in the limits UI, so may want to be documented as an optional limit for admins to set.

Suggestions for improvement: Include a section suggesting spot VM limits are increased in case that it a feature that they are intending to leverage

Additional information:

Comment 3 Chinmayi Chandrasekhar 2021-11-29 09:41:47 UTC
PR: https://github.com/openshift/openshift-docs/pull/39292

Comment 4 MayXu 2021-11-29 14:52:14 UTC
changed the 05_masters.json and 06_worker6.json based on https://docs.microsoft.com/en-us/azure/virtual-machines/linux/spot-template

$ az vm list -g maxuspotvm-vjm99-rg  --query "[?priority!=null].{Name:name, Priority:priority, billing:billingProfile.maxPrice, Eviction:evictionPolicy}" -o table
Name                                 Priority    Billing    Eviction
-----------------------------------  ----------  ---------  ----------
maxuspotvm-vjm99-master-0            Spot        -1.0       Deallocate
maxuspotvm-vjm99-master-1            Spot        -1.0       Deallocate
maxuspotvm-vjm99-master-2            Spot        -1.0       Deallocate
maxuspotvm-vjm99-worker-centralus-1  Spot        -1.0       Deallocate
maxuspotvm-vjm99-worker-centralus-2  Spot        -1.0       Deallocate
maxuspotvm-vjm99-worker-centralus-3  Spot        -1.0       Deallocate

$ oc get clusterversion
NAME      VERSION                              AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.10.0-0.nightly-2021-11-28-164900   True        False         7s      Cluster version is 4.10.0-0.nightly-2021-11-28-164900

Comment 8 MayXu 2021-12-02 05:16:26 UTC
with IPI can create spot vm worker 
1. create install-config.yaml
openshift-install create install-config
2. openshift-install create manifests
3. change the openshift/99_openshift-cluster-api_worker-machineset-2.yaml and 99_openshift-cluster-api_worker-machineset-1.yaml
spec.template.spec.providerSpec.value.spotVMOptions (Add this field, and set it to {}.)
    spec:
      providerSpec:
        value:
          spotVMOptions: {}
4.openshift-install create cluster 

cluster install is OK 

$ oc get clusterVersion
NAME      VERSION                              AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.10.0-0.nightly-2021-12-01-164437   True        False         15m     Cluster version is 4.10.0-0.nightly-2021-12-01-164437

$ az vm list -g maxuspot-5794m-rg -o table --query "[].{Name:name, Priority:priority, billing:billingProfile.maxPrice, Eviction:evictionPolicy}"
Name                                    Priority    Billing    Eviction
--------------------------------------  ----------  ---------  ----------
maxuspot-5794m-master-0                 Regular
maxuspot-5794m-master-1                 Regular
maxuspot-5794m-master-2                 Regular
maxuspot-5794m-worker-centralus1-l6ptq  Spot        -1.0       Deallocate
maxuspot-5794m-worker-centralus2-9l6l4
maxuspot-5794m-worker-centralus3-szvsj  Spot        -1.0       Deallocate

Comment 9 MayXu 2022-01-06 14:08:25 UTC
for spot vCPU quotas, ref: 
https://docs.microsoft.com/en-us/azure/azure-portal/supportability/spot-quota