Description of problem: [zaitcev@suzdal ~]$ echo -n "Paste KEY: " && AWS_SECRET_ACCESS_KEY=$(head -1) AWS_ACCESS_KEY_ID=***** duplicity --allow-source-mismatch --tempdir $HOME/tmp --include-filelist backups/niphredil.duplicity.list --exclude '.*' --exclude '*' . boto3+s3://s3.amazonaws.com/zaitcev.lan/backups/niphredil Paste KEY: ******************** Reading globbing filelist backups/niphredil.duplicity.list Attempt 1 failed. ClientError: An error occurred (403) when calling the HeadBucket operation: Forbidden Attempt 2 failed. AttributeError: 'NoneType' object has no attribute 'objects' Attempt 3 failed. AttributeError: 'NoneType' object has no attribute 'objects' Attempt 4 failed. AttributeError: 'NoneType' object has no attribute 'objects' Giving up after 5 attempts. AttributeError: 'NoneType' object has no attribute 'objects' [zaitcev@suzdal ~]$ Version-Release number of selected component (if applicable): duplicity-0.8.19-1.fc34.x86_64 python3-botocore-1.20.77-1.fc34.noarch python3-boto3-1.17.77-1.fc34.noarch python3-boto-2.49.0-8.fc34.noarch How reproducible: Synchronous Steps to Reproduce: 1. Run duplicity to make a backup Actual results: Fails as reported Expected results: Runs like in Fedora 33 Additional info: This used to work.
Interesting, I'm using this successfully. Does it work if you remove the environment variables and let it use the credentials in ~/.aws/credentials?
(In reply to Gwyn Ciesla from comment #1) > Does it work if you remove the > environment variables and let it use the credentials in ~/.aws/credentials? I never had that file and there's nothing in duplicity(1) man page. Do you happen a syntax reference on hand?
https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-files.html
I wrote credentials into the ~/.aws/credentials, but same result: [zaitcev@suzdal ~]$ duplicity --allow-source-mismatch --tempdir $HOME/tmp --include-filelist backups/niphredil.duplicity.list --exclude '.*' --exclude '*' . boto3+s3://s3.amazonaws.com/zaitcev.lan/backups/niphredil Reading globbing filelist backups/niphredil.duplicity.list Attempt 1 failed. ClientError: An error occurred (403) when calling the HeadBucket operation: Forbidden ^C[zaitcev@suzdal ~]$ However, if I use the old boto library, then it seems to work, and it must be picking ~/.aws/credentials: [zaitcev@suzdal ~]$ duplicity --allow-source-mismatch --tempdir $HOME/tmp --include-filelist backups/niphredil.duplicity.list --exclude '.*' --exclude '*' . s3://s3.amazonaws.com/zaitcev.lan/backups/niphredil Reading globbing filelist backups/niphredil.duplicity.list Warning, found incomplete backup sets, probably left from aborted session Local and Remote metadata are synchronized, no sync needed. Last full backup date: Sat Nov 3 13:25:24 2012 GnuPG passphrase for decryption: Retype passphrase for decryption to confirm: --------------[ Backup Statistics ]-------------- StartTime 1623705487.91 (Mon Jun 14 16:18:07 2021) EndTime 1623705566.44 (Mon Jun 14 16:19:26 2021) ElapsedTime 78.53 (1 minute 18.53 seconds) SourceFiles 117522 SourceFileSize 34025696145 (31.7 GB) NewFiles 5210 NewFileSize 348785189 (333 MB) DeletedFiles 1004 ChangedFiles 115 ChangedFileSize 11818751 (11.3 MB) ChangedDeltaSize 0 (0 bytes) DeltaEntries 6329 RawDeltaSize 354310447 (338 MB) TotalDestinationSizeChange 279089258 (266 MB) Errors 0 ------------------------------------------------- [zaitcev@suzdal ~]$
Can you see the contents of your bucket with "aws s3 ls"?
I don't have AWS SDK installed, so I don't have the "aws" command. But "s3cmd ls s3://zaitcev.lan/backups/niphredil/ | more" works.
It's available in the repos, as awscli.
Thanks, seems like it's working [root@suzdal zaitcev]# dnf install awscli ............. [zaitcev@suzdal ~]$ aws s3 ls 2011-04-12 06:57:39 test-1235163301 2015-10-27 22:02:20 test-59fbb0254181129f 2011-04-12 07:11:35 zaitcev.lan [zaitcev@suzdal ~]$
Does the new boto method for duplicity work now?
Sorry, Gwyn, the "new boto method" does not tell me what I need to test. I have no clue which is old and which is new. Please name the URL scheme that I can configure.
I meant the boto3+s3 scheme you posted above, but I'm just now noticing an error. You posted: boto3+s3://s3.amazonaws.com/zaitcev.lan/backups/niphredil Try: boto3+s3:///zaitcev.lan/backups/niphredil Assuming that zaitcev.lan is the name of your s3 bucket.
Indeed, not specifying the endpoint appears to work: [zaitcev@suzdal ~]$ duplicity --allow-source-mismatch --tempdir $HOME/tmp --include-filelist backups/niphredil.duplicity.list --exclude '.*' --exclude '*' . boto3+s3:///zaitcev.lan/backups/niphredil Reading globbing filelist backups/niphredil.duplicity.list Warning, found incomplete backup sets, probably left from aborted session Synchronizing remote metadata to local cache... GnuPG passphrase for decryption: Copying duplicity-full-signatures.20121103T182524Z.sigtar.gpg to local cache. ........ lots and lots of re-downloading Seems like the duplicity in F34 lost or discarded its cache, but it pretends to work otherwise.