Bug 1964128

David, if you have no objection, I propose that Major submit a PR to update this.

I'm happy to propose a PR, but I have some higher-level questions.

Question #1: Handling botocore
First off, in Tom's COPR repo[0] he has aws-cli-v2 built with a vendored version of botocore since v2's requirements ask for a very modern version of botocore. That seems fine, but do we want to just bump the version of python-botocore in Fedora to that newer version? I'm not up to speed with any breaking changes between botocore<2 and botocore>=2.

Question #2: New package?
Do we want to make a awscli2 package and obsolete the awscli package? Or, do we want to keep the current awscli package and simply update to v2 bits?

[0] https://copr-dist-git.fedorainfracloud.org/cgit/spot/aws-cli-2/aws-cli-2.git/tree/

The only botocore consumers I see are awscli, boto3, certbot-dns-route53, and s3transfer. Is the botocore version awscli2 needs a released version?

As of the most recent aws-cli-v2 release[0] it looks like an unreleased version of botocore is still required:

requires = [
    'botocore==2.0.0dev117',
    'colorama>=0.2.5,<0.4.4',
    'docutils>=0.10,<0.16',
    'cryptography>=3.3.2,<3.4.0',
    's3transfer>=0.4.2,<0.5.0',
    'ruamel.yaml>=0.15.0,<0.16.0',
    # wcwidth 0.2.0 and up do not get copied into
    # the pyinstaller executable.
    'wcwidth<0.2.0',
    'prompt-toolkit>=2.0.0,<3.0.0',
    'distro>=1.5.0,<1.6.0',
    'awscrt==0.11.13',
]

[0] https://github.com/aws/aws-cli/blob/e5c05ee99e5d8aa83c914370c6a2befcb001fdce/setup.py#L27

Since it's unreleased, I vote we use a vendored botocore until it's officially released, and update this package. I see no benefit to keeping the old version.

I'm attempting to package awscrt in the meantime since it's required for awscli-v2, but I'm getting some errors from the compiler: https://github.com/awslabs/aws-crt-python/issues/296

Opened a bug on aws-lc: https://github.com/awslabs/aws-lc/issues/172

Tom Calloway already packaged it. It was in review and has been approved.

Oh okay. Do we just want to copy/paste what he has in COPR over to the Fedora package?

(In reply to Gwyn Ciesla from comment #5)
> Since it's unreleased, I vote we use a vendored botocore until it's
> officially released, and update this package. I see no benefit to keeping
> the old version.

The big change in the new version of the cli for us is the pager. The pager setting will potentially break any interactive shell scripting that's built on the v1. The dev(vendored) botocore was exactly why I thought we should keep them both for the duration as there is not yet a guarantee that the V2 python delivered in github will be consistent with the multi-lang version released in S3. AWS considers what they have as released, but it's a little volatile in my opinion since there is no guarantee that another team won't drop different binaries into the plugins for release and we may face some variations in what's delivered in the base python that Tom is rolling. 

I would say that we need to be ready to extend what's deployed with supporting plugin builds as they become available or keep the bundling that Tom has done here.

(In reply to Gwyn Ciesla from comment #5)
> Since it's unreleased, I vote we use a vendored botocore until it's
> officially released, and update this package. I see no benefit to keeping
> the old version.

The big change in the new version of the cli for us is the pager. The pager setting will potentially break any interactive shell scripting that's built on the v1. The dev(vendored) botocore was exactly why I thought we should keep them both for the duration as there is not yet a guarantee that the V2 python delivered in github will be consistent with the multi-lang version released in S3. AWS considers what they have as released, but it's a little volatile in my opinion since there is no guarantee that another team won't drop different binaries into the plugins for release and we may face some variations in what's delivered in the base python that Tom is rolling. 

I would say that we need to be ready to extend what's deployed with supporting plugin builds as they become available or keep the bundling that Tom has done here.

(In reply to Major Hayden 🤠 from comment #9)
> Oh okay. Do we just want to copy/paste what he has in COPR over to the
> Fedora package?

All that  previously said in comment 10 (and 11 by accident), I am good with taking the vendored dependency in the package until it's released as well. I am optimistically cautious, but I think we will be able to provide the right support when it moves out of the dev branch.

Hmm, after taking a deeper look, I worry about the aws-lc component. It seems like we're introducing new cryptography components to Fedora and that might need some extra attention.

Hey Major, thanks for that added review. The aws-lc component should definitely get a review before we start to look at the updates. 

So here is what I am thinking - The work that @spotrh is doing is great, but it's bundling a lot of things that we haven't reviewed and should potentially manage as a dependencies, like the aws-lc component. I am starting to lean towards the better practice to be to remove the bundling for these requirements and start building with chained builds of the component parts. I know that this creates the same problem that @gwync found when she was originally managing boto3 and I was managing the botocore + awscli independently, but we managed to get that straight. I really am concerned that there will be other surprises like the aws-lc that will creep in over time and I think we should be managing them intentionally.

I, uh, backed away from this effort making the sign of the cross. aws-lc made this a lot harder. 

To be specific, I closed 1928967 by saying this:

"I'm abandoning this particular packaging effort. Botocore-2 isn't too hard to package, but aws-cli-2 keeps growing dependencies out of it, most notably, a newer version of s3transfer than found anywhere but rawhide and awscrt, which depends on most of the rest of the aws SDK. I chased that down for a while until I ran into aws-lc, which is a forked copy of BoringSSL. Packaging that would either require some really ugly hacks or a Conflicts: openssl. It was at this point I decided I no longer cared enough."

Thanks Spot. Appreciate the update. I don't think that what the AWS service team is packaging in the CLIv2 is in any way stable or predictable enough for pulling into the work we are doing here, even though they seem to believe that they are ready to abandon the cliv1. I am also concerned about the new plugins requiring additional. 

This is really unfortunate. At this point, I'll keep tracking on the changes and put some serious time into collaborating with the AWS internal service team. When it works with the current modules, we'll package the next gen.

This bug appears to have been reported against 'rawhide' during the Fedora 35 development cycle.
Changing version to 35.

https://github.com/aws/aws-cli/pull/6352 - is a dramatic change in the way that the Amazon SDK team is thinking about making the installer work. I am going to resubmit this for review using the new guidelines.

has there been any more progress on this? Is https://github.com/awslabs/aws-lc/issues/487 the only thing blocking it?

I have all of the blockers completed.

Good afternoon, any updates on this? I'm willing to help this going forward


Best

the awscli package on fedora repos is currently 1.27 whilst the latest download from Amazon is now at 2.11.17. Is this likely to get updated whilst Fedora 38 is the current release?

For anyone following this bug - the package review ticket is https://bugzilla.redhat.com/show_bug.cgi?id=2189420

Listed on Fedora Packages https://packages.fedoraproject.org/pkgs/awscli2/awscli2/