Is this something we can implement in BuildDefaults?

https://github.com/csrwng/openshift-docs/blob/master/admin_guide/build_defaults_overrides.adoc

it's not likely something that would make sense as a build default.  you'd have to require everyone have a secret with the exact same name defined in their namespace, and that that was the secret they always wanted to use to pull source code.

*** Bug 1348054 has been marked as a duplicate of this bug. ***

(In reply to Ben Parees from comment #3)
> it's not likely something that would make sense as a build default.  you'd
> have to require everyone have a secret with the exact same name defined in
> their namespace, and that that was the secret they always wanted to use to
> pull source code.

Is there a way to do this as an annotation?

> Is there a way to do this as an annotation?

i'm not clear on what you're asking/suggesting.  Do it as an annotation on what?  What would the annotation value be?

(In reply to Ben Parees from comment #6)
> > Is there a way to do this as an annotation?
> 
> i'm not clear on what you're asking/suggesting.  Do it as an annotation on
> what?  What would the annotation value be?

Ben, We often use annotations (such as with the router) to define that this route is different from the defaults (example being, timeouts, etc).

Could the same idea be _used_  is simply the question I had? This would keep thing uniform with what we are doing with custom routes and thus _may_ translate well?

I'm still not sure i'm following your suggestion (and i'm not familiar w/ the route annotations you're referring to, so perhaps that's part of why i'm not following it), but the fundamental ask here is for new-app/new-build to inject a required secret when it constructs the buildconfig.

The suggested(and not useful imho) solution was to add it as a cluster-wide default that gets added to all buildconfigs when they are created (assuming the buildconfig doesn't explicitly provide a value).

If we did do that, then:
1) every user would have to use the same shared secret because all their buildconfigs will get that value

or

2) they would have to explicitly update their buildconfig with the value they want to use, which is not something they can do via new-app (they'd have to edit the buildconfig after new-app created it).

or

3) they would have to explicitly update their buildconfig to say "ignore the system wide default" (via your proposed annotation) which again would have to be done after the buildconfig was created because new-app/new-build don't currently give you the ability to specify annotations to be added to the buildconfig that is being created.

(In reply to Ben Parees from comment #8)
> I'm still not sure i'm following your suggestion (and i'm not familiar w/
> the route annotations you're referring to, so perhaps that's part of why i'm
> not following it), but the fundamental ask here is for new-app/new-build to
> inject a required secret when it constructs the buildconfig.
> 

Correct! My question is if that secrete can be built based on an annotation? 

https://github.com/openshift/origin/pull/9710 and example of some router based annotations.

i'm not sure why we'd want to build a secret from an annotation instead of just having the user reference a secret directly?

(In reply to Ben Parees from comment #10)
> i'm not sure why we'd want to build a secret from an annotation instead of
> just having the user reference a secret directly?

I think the issue is that the build and the secrete can not be creatd with new-app? 

> Eric can you confirm? 

Thus because new-app can't create the secrete, it can't be linked, and this creates a two step processes.

> I think the issue is that the build and the secrete can not be creatd with new-app? 

My impression was the request was not to create the secret with new-app, it was to be able to reference an existing secret when invoking new-app, such that the resulting buildconfig included a reference to that secret.

(In reply to Ben Parees from comment #12)
> > I think the issue is that the build and the secrete can not be creatd with new-app? 
> 
> My impression was the request was not to create the secret with new-app, it
> was to be able to reference an existing secret when invoking new-app, such
> that the resulting buildconfig included a reference to that secret.

So,  https://bugzilla.redhat.com/show_bug.cgi?id=1318968 ?

No, that looks significantly different.

To be clear about this buildconfig request:  we already support referencing secrets from a buildconfig in various ways (as a secret for pulling/pushing images, as a secret for fetching source, as a secret that can be used by the assemble script during the s2i process, etc).

This RFE is just asking that the user be able to pass the name of a secret to new-app and have new-app include the reference in the buildconfig it generates.  (as far as i understand the RFE).

secrets for the assemble script or other parts of the build process:
https://docs.openshift.org/latest/dev_guide/builds.html#using-secrets

secrets for fetching source for the build:
https://docs.openshift.org/latest/dev_guide/builds.html#using-private-repositories-for-builds

secrets for pushing/pulling images during the build:
https://docs.openshift.org/latest/dev_guide/builds.html#using-docker-credentials-for-pushing-and-pulling-images

Ben is correct.

This RFE was to run oc new-app and reference an existing secret that would allow the build to use the secret's info as _currently_ the only way to, say, deploy an app that needs to pull stuff from a private git repo is to start a new-app with the knowledge that it will fail and then attach the secret to the failed build-config and deploy again.

Assigning to Cesar since he's working on the trello card associated w/ this RFE.

I have a customer looking for this feature. Their internal gitlab is secured via client certificates. Currently, there is no way to pass these to new-app or otherwise construct all application artifacts (bc/dc/service/etc) via the cli under this condition. A buildconfig can be constructed using new-build and the git secrets and other artifacts constructed manually, or new-app can be used so long as the builder image is explicitly defined (via the image~repo syntax), saved to a file which is then modified to add the git secrets, then created. Needless to say this makes automation more difficult.

Hi,

One of the customers asking for this feature here.

Ben / Eric are correct in what we are asking for, I don't know about other customers.

A lot of the nicer features that openshift provide to get up and running in a short period time fall down in a heap when authentication has to be taken into a account.

At the moment we're having to instruct our end-users to use oc new-app followed by oc patch to splice in a build secret to the build config. Our beginner-to-intermediate users are not 100% across the detailed json/yaml structures, and just want to be able to type something in that 'works' in a way they can understand.  Having to splice in build secrets (with enough brackets involved that it takes me back to my lisp days), is a bit of a smell that should easily be fixed. 

It would definitely increase the first-look / out of the box experience for any users working in corporate environments that require authentication for interaction with corporate resources.

Regards,
Dale.

Thanks Dale, I think the oc set build-secret command that Cesar just added will help you out, you can see the syntax summarized here: https://github.com/openshift/origin/pull/10615

Thanks Ben,

I think "oc set build-secret" is a good start in simplifying things for users, but not quite what was asking for.  Right now our users need:

oc new-app <url of private github>
oc patch bc/<app-name> -p '{"spec":{"source":{"sourceSecret":{"name":"mysecert"}}}}'
oc start-build bc/<app-name>

With the set build-secret command we have:
oc new-app <url of private github>
oc set build-secret bc/<app-name> mysecret
oc start-build bc/<app-name>

What I'm chasing is:
oc new-app --build-secret mysecret <url of private github>

of course --docker-pull-secret as well would be useful as well.

Currently we are resistant to adding more options to new-app to customize the content it produces, in preference to a more unix-like philosophy of stringing multiple simpler commands together.

Otherwise new-app is going to have an unmanageable number of arguments for things like "add a github trigger, add a generic trigger, add an imagechange trigger, add a pull secret, add a push secret, add a git secret" etc.

we have some other thoughts around how to allow users to customize what new-app produces, and also provide other mechanisms for defaulting secrets consume by builds, but adding more arguments to new-app is not our most likely path.

Ben,
An I right in saying you would prefer:

oc new-app -o yaml <private git url> | oc set build-secret -f - <build-secret> | oc set pull-secret -f - <pull-secret> | oc set oc create -f -

Right now I can do the very same by chaining a bunch of "oc patch" commands together.

My understanding of new-app is that it is supposed to provide a quickstart way of getting something functional from openshift.  It providees no new functionality, but gives a way of creating a working application with a single command.

It achieves this, except for the case where you are in a corporate envionrment. new-app is fails pretty badly in a corporate environment at the moment (for both private docker search and private git repo pulling). 

My average user would be a lot more comfortable with 
"oc new-app --build-secret <secret> <git url>" 
then it would be 
"oc new-app <git url> -o yaml | oc set build-secret <secret> -f - | oc create -f -

My advanced user is already comfortable with "oc patch" and how to use it.  So what exactly are we gaining by all this extra syntax? I would argue that making oc new-app usable for the "average" corporate user to get up and running is far more valuable then a bit more syntacic sugar around the "oc patch" command.

Being able to automatically generate project specific build/pull/push default secrets and having some ability to take advantage of them may be usable by us as well though.. This would reduce createing an application to:
"oc new-app <git url>"
And still have it work in a secure envionrment - I would definitely consider that a win.

+1, IMO goal should be to have something working with one command. Adding a hook is fine to be in a separate command because lack of hooks doesn't render build nonfunctional.

wrt number of options, there are already a lot of them in some of the commands. I was thinking to file a bug about that but didn't yet give it enough of thought. In very short I think proper grouping and formatting would to a large extend make commands more useful even if we have a couple of more option.

@dale
> An I right in saying you would prefer:
> oc new-app -o yaml <private git url> | oc set build-secret -f - <build-secret> | oc set pull-secret -f - <pull-secret> | oc set oc create -f -

yes

> Right now I can do the very same by chaining a bunch of "oc patch" commands together.

except that you need to know the json syntax to do that, which was your complaint in your original comment, no?


@aleksandar
> +1, IMO goal should be to have something working with one command. Adding a hook is fine to be in a separate command because lack of hooks doesn't render build nonfunctional.

You say that now.  Someone else may not feel that way.  What about post-commit hooks that run tests?  This is exactly the debate i do not want to start having about what does/doesn't belong in new-app.  Having specific tools avoids that problem.

My lacmus test is errors. Starting to work with some not very well known system I'm very frustrated to see error messages because I don't understand why and when did they happen. I think that would be similar with other people.

If you create a build without proper secret, you end up with some errors in the project (because of failed builds). If you lack hooks, then no errors should show up and user could have a clean experience through the whole process. Or if errors occur, they would indicate some real unexpected behavior.

I believe we need to always look for a superior, clean and nice UX. Anything that look "ugly" IMO is a bug. I wonder if we have already somebody from UX team looking at oc tool.

(In reply to Ben Parees from comment #30)
> @dale
> > An I right in saying you would prefer:
> > oc new-app -o yaml <private git url> | oc set build-secret -f - <build-secret> | oc set pull-secret -f - <pull-secret> | oc set oc create -f -
> 
> yes
> 
> > Right now I can do the very same by chaining a bunch of "oc patch" commands together.
> 
> except that you need to know the json syntax to do that, which was your
> complaint in your original comment, no?
> 
> 
> @aleksandar
> > +1, IMO goal should be to have something working with one command. Adding a hook is fine to be in a separate command because lack of hooks doesn't render build nonfunctional.
> 
> You say that now.  Someone else may not feel that way.  What about
> post-commit hooks that run tests?  This is exactly the debate i do not want
> to start having about what does/doesn't belong in new-app.  Having specific
> tools avoids that problem.

Hi Ben,
Just to re-state things for clarity
We're trying to secure a clean, uncomplicated 'out of the box' experience for my new users who are experimenting. 

My understanding of 'new-app' is supposed to the quick way of getting an application up and running - a way of showing off the power of Openshift. I understand that it is *not* supposed to be a swiss-army knife and be all things to all people.

The only problem right now is that new-app fails badly in a corporate environment at the most basic level.  IT DOES NOT WORK!  As an introductory experience to new users in a corporate environment it is a pretty big failure.  

What I'm asking for is the minimum required to get new-app to work in a corporate environment, such that people 'dabbling' or giving it try will get some success first go.

I'm neither for nor against oc set-build-secret, as far as I can see it is for a different use case (a useful syntatic sugar for anybody with a bit of experience with Openshift) then what I'm trying to get happen here.

I'm trying to foster greater adoption of Openshift in my organisation.  For the non-hardcore developer (and in this category has a lot of people not familiar with unix pipes philosophy), The ease of use during their first experience goes a long way towards whether they will continue to use Openshift or not.

+1 to Dale's comments and this RFE

Another customer here and I also work for a large corporate deploying OpenShift with private docker registries and private github that is for all practical purposes disconnected from the internet.

The OpenShift documentation (which is excellent btw) refers to new-app as a quick start way to deploy an application, and when deploying on a cluster that has access to internet RedHat and Docker Hub images and github, it is a quick and easy way to deploy an application.

However, in our disconnected OpenShift environment that has no access to the internet, it fails badly as Dale said and is confusing to new developers and users of OpenShift. Having a failed build and then having to patch or edit the bc is clunky (and also doesn't look good when we're demoing OpenShift internally to try and drive OpenShift adoption within my organization)

Adding a --build-secret option to oc new-app to pass in a cert secret for the build for git https urls or a ssh key secret for git ssh urls would be very helpful to us.

@Mark our current thought on how to address this is described in this card:
https://trello.com/c/NoVpS2OS

We prefer that approach to adding more flags to new-app/new-build.  You will have to:

1) create the git secret
2) annotate it

once you've done that, new-build/new-app will work repeatedly.

What do you think?

@Ben - yes that approach would appear to work well for us.

https://github.com/openshift/origin/pull/12301

What about using annotated source secrets that can be added to a buildconfig automatically? [1]  If the secret is created before running the new-app command, then the annotation can be used to match a secret related to a secure git repository URL.  As long as the URLs follow the scheme described in [1].


kind: Secret
apiVersion: v1
metadata:
  name: secure-repo
  annotations:
    build.openshift.io/source-secret-match-uri-1: https://*github.com/acme/*
data:


oc new-app https://someuser@github.com/acme/project-abc.git ...





[1]
https://docs.openshift.org/latest/dev_guide/builds.html#automatic-addition-of-a-source-secret-to-a-build-configuration

Travis i'm not sure who your question/suggestion is directed at, but the feature you described is what we implemented in fulfillment of this RFE, as described in comment 34.

change status to verify since this feature already tested on 3.5

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:0884