Description of problem: Unable to perform duplicity backups on Google Drive Version-Release number of selected component (if applicable): 0.8.12-1.fc32.x86_64 How reproducible: Run duplicity by using the pydrive backend and with a regular Google account. Steps to Reproduce: 1. sudo dnf install duplicity 2. Configure your Google account 3. duplicity incr --name home_gdrive --encrypt $key --full-if-older-than 2M --num-retries 5 --timeout 60 --allow-source-mismatch --verbosity 4 /path/to/my/backup/source pydrive+gdocs://$google_email/$bak_folder where $key, $google_email and $bak_folder should be replaced with suitable values. The encryption passphrase is read from the environment variable $PASSPHRASE. Actual results: Attempt 1 failed. RedirectMissingLocation: Redirected but the response is missing a Location: header. Attempt 2 failed. RedirectMissingLocation: Redirected but the response is missing a Location: header. Attempt 3 failed. RedirectMissingLocation: Redirected but the response is missing a Location: header. Attempt 4 failed. RedirectMissingLocation: Redirected but the response is missing a Location: header. Giving up after 5 attempts. RedirectMissingLocation: Redirected but the response is missing a Location: header. Expected results: No error. Additional info: I am experiencing this problem since Fedora 32. Instead, duplicity ran correctly until Fedora 31. I think the problem may be related to the following google-api-client issue: https://github.com/googleapis/google-api-python-client/issues/803 due to a recent change in httplib2 (since version 0.16). Indeed, I compared the version of some relevant packages installed in Fedora 31 and 32, and the only big difference is the version of httplib2: * Fedora 31 packages: duplicity-0.8.12-1.fc31.x86_64 python3-httplib2-0.13.1-1.fc31.noarch python3-google-api-client-1.6.7-8.fc31.noarch python3-PyDrive-1.3.1-9.fc31.noarch * Fedora 32 packages: duplicity-0.8.12-1.fc32.x86_64 python3-httplib2-0.17.3-1.fc32.noarch python3-google-api-client-1.6.7-11.fc32.noarch python3-PyDrive-1.3.1-12.fc32.noarch According to the above google-api-client issue, upgrading the google-api-client to the last version should fix this problem. An alternative is to downgrade httplib2 to version 0.15.
I'll work on this.
I am also having this issue on Fedora 32. "deja dup Giving up after 5 attempts. RedirectMissingLocation: Redirected but the response is missing a Location: header." deja-dup.x86_64 42.4-2.fc32 @updates duplicity.x86_64 0.8.15-2.fc32 @updates python2-httplib2.noarch 0.18.1-3.fc32 @updates python3-PyDrive.noarch 1.3.1-12.fc32 @fedora python3-google-api-client.noarch 1:1.6.7-11.fc32 @fedora python3-httplib2.noarch 0.18.1-3.fc32 @updates
Does duplicity 0.8.16 in updates-testing help?
(In reply to Gwyn Ciesla from comment #3) > Does duplicity 0.8.16 in updates-testing help? It doesn't appear to have fixed the issue. I used duplicity.x86_64 0.8.16-1.fc32 @updates-testing
Unfortunately, it looks that this problem affects Fedora 33 as well: Attempt 1 failed. RedirectMissingLocation: Redirected but the response is missing a Location: header. Attempt 2 failed. RedirectMissingLocation: Redirected but the response is missing a Location: header. Attempt 3 failed. RedirectMissingLocation: Redirected but the response is missing a Location: header. Attempt 4 failed. RedirectMissingLocation: Redirected but the response is missing a Location: header. Giving up after 5 attempts. RedirectMissingLocation: Redirected but the response is missing a Location: header. Here below are the installed rpms: duplicity-0.8.16-1.fc33.x86_64 python3-httplib2-0.18.1-6.fc33.noarch python3-google-api-client-1.6.7-13.fc33.noarch python3-PyDrive-1.3.1-13.fc33.noarch So, I updated the "Version" field of this bug report from 32 to 33.
I think this is the underlying issue: https://github.com/googleapis/google-api-python-client/issues/803. In summary, a new version of httplib2 > 0.16.0 breaks older google docs api code as it changed the way it handles redirects. It appears to be solved if you use the newest APIs (pip install --upgrade google-api-python-client google-auth-httplib2 google-auth-oauthlib). I also see that there is a new Google Drive backend (https://gitlab.com/duplicity/duplicity/-/blob/master/duplicity/backends/gdrivebackend.py) which might solve this issue, although I have not tested it. It needs to be packaged into our rpms.
Unfortunately, this bug is still present in Fedora 34. Did anyone find a way to get the Google Drive backend working? I noted that duplicity introduced a new GDrive backend (as also @
[continued from previous comment...] ... (as also Kim suggested). I tried to use it but without success.
I believe google-api-python-client can now be upgraded. Michele, do you have plans to do so soon?
According to https://stackoverflow.com/questions/59815620/gcloud-upload-httplib2-redirectmissinglocation-redirected-but-the-response-is-m it is really about the version of google-api-python-client And https://pypi.org/project/google-api-python-client/ states that "The 2.0 release of google-api-python-client includes a substantial reliability improvement, compared with 1.x, as discovery documents are now cached in the library rather than fetched dynamically. It is highly recommended to upgrade from v1.x to v2.x."
After some tests with a colleague, it seems that updating this library and google-auth-httplib2 (from 0.0.3 to 0.1.0) isn't sufficient, you also need to upgrade pydrive2 to 1.10.0 (or so) for duplicity/deja-dup to work with Google Drive. This is covered in https://bugzilla.redhat.com/show_bug.cgi?id=2015282 so both tickets are dependent on each other.
FEDORA-2021-4bbfcae462 has been submitted as an update to Fedora 35. https://bodhi.fedoraproject.org/updates/FEDORA-2021-4bbfcae462
FEDORA-2021-ed754a04da has been submitted as an update to Fedora 34. https://bodhi.fedoraproject.org/updates/FEDORA-2021-ed754a04da
FEDORA-2021-ed754a04da has been pushed to the Fedora 34 testing repository. Soon you'll be able to install the update with the following command: `sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2021-ed754a04da` You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2021-ed754a04da See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.
FEDORA-2021-4bbfcae462 has been pushed to the Fedora 35 testing repository. Soon you'll be able to install the update with the following command: `sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2021-4bbfcae462` You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2021-4bbfcae462 See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.
FEDORA-2021-4bbfcae462 has been pushed to the Fedora 35 stable repository. If problem still persists, please make note of it in this bug report.
In Fedora 35, I installed: sudo dnf upgrade --advisory=FEDORA-2021-4bbfcae462 Upgrade python3-google-api-client-2:2.31.0-2.fc35.noarch Upgrade python3-google-auth-httplib2-0.1.0-1.fc35.noarch However, with duplicity I still get the same error: Attempt of put Nr. 1 failed. RedirectMissingLocation: Redirected but the response is missing a Location: header. Attempt of put Nr. 2 failed. RedirectMissingLocation: Redirected but the response is missing a Location: header. Attempt of put Nr. 3 failed. RedirectMissingLocation: Redirected but the response is missing a Location: header. Attempt of put Nr. 4 failed. RedirectMissingLocation: Redirected but the response is missing a Location: header. Giving up after 5 attempts. RedirectMissingLocation: Redirected but the response is missing a Location: header.
Marco, would it be possible to delete PyDrive? Code shouldn't use PyDrive path, but just to ensure that is not breaking something.
Sorry. I did not understand. You mean removing the PyDrive package as follows: sudo dnf remove python3-PyDrive ? This would remove 18 packages. Not sure it would not break anything.
Are those packages from Fedora? Can you paste the output here?
Sure! $ sudo dnf remove python3-PyDrive Dependencies resolved. ============================================================================================================================================================================================================================================== Package Architecture Version Repository Size ============================================================================================================================================================================================================================================== Removing: python3-PyDrive noarch 1.3.1-18.fc35 @fedora 171 k Removing unused dependencies: abseil-cpp x86_64 20210324.2-2.fc35 @fedora 1.8 M grpc-data noarch 1.39.1-12.fc35 @updates 12 k python3-cachetools noarch 4.2.2-3.fc35 @fedora 77 k python3-google-api-client noarch 2:2.31.0-2.fc35 @updates 55 M python3-google-api-core noarch 2.1.1-1.fc35 @updates 564 k python3-google-auth noarch 1:2.3.3-1.fc35 @updates 900 k python3-google-auth-httplib2 noarch 0.1.0-1.fc35 @updates 44 k python3-googleapis-common-protos noarch 1.53.0-10.fc35 @fedora 988 k python3-grpcio x86_64 1.39.1-12.fc35 @updates 5.7 M python3-grpcio-gcp noarch 0.2.2-6.fc35 @fedora 77 k python3-oauth2client noarch 4.1.3-15.fc35 @fedora 610 k python3-protobuf noarch 3.14.0-6.fc35 @fedora 1.5 M python3-pyasn1-modules noarch 0.4.8-7.fc35 @fedora 1.0 M python3-pyyaml x86_64 5.4.1-4.fc35 @fedora 674 k python3-rsa noarch 4.7.2-1.fc35 @fedora 201 k python3-uritemplate noarch 3.0.1-6.fc35 @fedora 62 k re2 x86_64 1:20190801-9.fc35 @fedora 371 k Transaction Summary ============================================================================================================================================================================================================================================== Remove 18 Packages Freed space: 70 M
I think I get it. You're not using deja-dup, that has a dep on PyDrive2 and should keep python3-google-api-client and others. You're using duplicity directly, that has a recommendation on PyDrive that should be changed to PyDrive2 (I'll submit a PR): https://src.fedoraproject.org/rpms/duplicity/blob/rawhide/f/duplicity.spec#_19 Can you do the following: $ sudo dnf install PyDrive2 $ sudo dnf remove PyDrive That should remove PyDrive only and keep the other packages or at least some of them. Once you've PyDrive2 installed you shouldn't face the problem anymore.
Oh, I see. Thanks! OK, I run: sudo dnf install python3-PyDrive2 sudo dnf remove python3-PyDrive Indeed, only the PyDrive package has been removed. Now, it seems that duplicity is running fine. It will take some time to finish. I will post here how it has gone once done.
It worked! Thanks a lot.
FEDORA-2021-ed754a04da has been pushed to the Fedora 34 stable repository. If problem still persists, please make note of it in this bug report.