Description of problem:

os-release:
NAME="Red Hat Enterprise Linux Server"
VERSION="7.9 (Maipo)"
kernel: 3.10.0-1160.45.1.el7.x86_64
OpenShift Server Version: 4.9.5


This is for IBM cloud.  oc new-app is failing for OpenShift 4.9 when running on RHEL 7. Indicates invalid options ...

time="2021-11-15T19:14:40Z" level=info msg="metacopy option not supported on this kernelmetacopy=on"
time="2021-11-15T19:14:40Z" level=info msg="Not using native diff for overlay, this may cause degraded performance for building images: failed to mount overlay: invalid argument"
I1115 19:14:40.076127       1 defaults.go:102] Defaulting to storage driver "overlay" with options [mountopt=metacopy=on].


The  mountopt=metacopy=on   seems to be incompatible when running on RHEL 7

Was able to capture the yaml for the build job, hack it, re-submit the pod manually and have the build work by adding the --storage-driver=overlay option to the openshift-sti-build container

  - args:
    - openshift-sti-build
    - --storage-driver=overlay
    - --loglevel=0


Version-Release number of selected component (if applicable):
OpenShift 4.9

How reproducible:
Always fails

Steps to Reproduce:
1. oc new-app -l app=pvg-nodejs --name pvg-nodejs  https://github.com/openshift/nodejs-ex.git
2. build job errors
3. logs from build job yield:  

vagrant@verify-cluster:~/armada-ansible$ oc logs pvg-nodejs-1-build
time="2021-11-15T19:14:40Z" level=info msg="metacopy option not supported on this kernelmetacopy=on"
time="2021-11-15T19:14:40Z" level=info msg="Not using native diff for overlay, this may cause degraded performance for building images: failed to mount overlay: invalid argument"
I1115 19:14:40.076127       1 defaults.go:102] Defaulting to storage driver "overlay" with options [mountopt=metacopy=on].
Caching blobs under "/var/cache/blobs".
Getting image source signatures
Copying blob sha256:63f9f4c31162a6a5dacd999a0dc65007e15b2ca6b2d9360a1234c27de12e7f38
Copying blob sha256:d49355ce848e2c86326ad8d7c3e4905e2883c89e1f6b79553427f1acf21ea7d1
Copying blob sha256:ce3c6836540f978b55c511d236429e26b7a45f5a6f1204ab8d4378afaf77332f
Copying blob sha256:236ea3424c2d1a06de825b1bb1967351c30915621e0f650756f402d7cbd27216
Copying config sha256:e7672e73aa234fab0cbfbaf539abf6714c91827eb8923cdd1457f05d65939649
Writing manifest to image destination
Storing signatures
Generating dockerfile with builder image image-registry.openshift-image-registry.svc:5000/openshift/nodejs@sha256:1f5fca1a02b22d32e28abf8f5b966ef5ad4e915ab4e1610d7885b290300dafda
Adding transient rw bind mount for /run/secrets/rhsm
Adding transient rw bind mount for /run/secrets/etc-pki-entitlement
Adding transient rw bind mount for /run/secrets/redhat.repo
STEP 1: FROM image-registry.openshift-image-registry.svc:5000/openshift/nodejs@sha256:1f5fca1a02b22d32e28abf8f5b966ef5ad4e915ab4e1610d7885b290300dafda
time="2021-11-15T19:15:32Z" level=error msg="error unmounting /var/lib/containers/storage/overlay/cc89cf6dab21a44ba722f1753c5aec849d6e6b753b6b32792d887076efc54996/merged: invalid argument"
error: build error: error mounting new container: error mounting build container "a432f63ab46daef1ebddb837ff405c6ca7852ac2470d93336761a5f711a8f5a4": error creating overlay mount to /var/lib/containers/storage/overlay/cc89cf6dab21a44ba722f1753c5aec849d6e6b753b6b32792d887076efc54996/merged, mount_data="metacopy=on,lowerdir=/var/lib/containers/storage/overlay/l/SDXDLK7IGRBBZ7KBHJIPWKKMDB:/var/lib/containers/storage/overlay/l/EWN5Y4XLI6IRQEHXPRKZVWYGRZ:/var/lib/containers/storage/overlay/l/LHUU3LV6ILIKDNMIGNUQNDZ2CZ:/var/lib/containers/storage/overlay/l/SMFNH6KFJ3ERA6ZR73VJFJYWNU,upperdir=/var/lib/containers/storage/overlay/cc89cf6dab21a44ba722f1753c5aec849d6e6b753b6b32792d887076efc54996/diff,workdir=/var/lib/containers/storage/overlay/cc89cf6dab21a44ba722f1753c5aec849d6e6b753b6b32792d887076efc54996/work": invalid argument


Actual results:
Build fails

Expected results:
Build to be successful

Additional info: