Bug 1492288
| Summary: | MemoryError during backup | ||||||
|---|---|---|---|---|---|---|---|
| Product: | [Fedora] Fedora | Reporter: | kartochka378 | ||||
| Component: | duplicity | Assignee: | Gwyn Ciesla <gwync> | ||||
| Status: | CLOSED CURRENTRELEASE | QA Contact: | Fedora Extras Quality Assurance <extras-qa> | ||||
| Severity: | unspecified | Docs Contact: | |||||
| Priority: | unspecified | ||||||
| Version: | 27 | CC: | agrover, alciregi, chrisa, daryll, giacecco, gwync, haliyo, james, j.hoffmann, liblit, mbrancaleoni, olivier.crete, steve.bz, twaugh, zaitcev | ||||
| Target Milestone: | --- | ||||||
| Target Release: | --- | ||||||
| Hardware: | Unspecified | ||||||
| OS: | Unspecified | ||||||
| Whiteboard: | |||||||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |||||
| Doc Text: | Story Points: | --- | |||||
| Clone Of: | Environment: | ||||||
| Last Closed: | 2018-03-05 17:42:54 UTC | Type: | Bug | ||||
| Regression: | --- | Mount Type: | --- | ||||
| Documentation: | --- | CRM: | |||||
| Verified Versions: | Category: | --- | |||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||
| Embargoed: | |||||||
| Bug Depends On: | 1512733 | ||||||
| Bug Blocks: | |||||||
| Attachments: |
|
||||||
Same for me backup file on external hdd over USB 3.0 Tried to stop auto backup for a day, then after 2 updates to testing and manually start backup it finished with success. No idea what package fixed it. I set to auto backup and wait for traceback. Interesting, it still traceback when auto backup is on, and succeed in case I disable auto backup and start manually. Weird. Same here.
duplicity-0.7.13.1-2.fc26.x86_64
Auto backup on external USB 3 HD
Encryption enabled.
Traceback (most recent call last):
File "/usr/bin/duplicity", line 1540, in <module>
with_tempdir(main)
File "/usr/bin/duplicity", line 1534, in with_tempdir
fn()
File "/usr/bin/duplicity", line 1385, in main
do_backup(action)
File "/usr/bin/duplicity", line 1410, in do_backup
globals.archive_dir).set_values()
File "/usr/lib64/python2.7/site-packages/duplicity/collections.py", line 710, in set_values
self.get_backup_chains(partials + backend_filename_list)
File "/usr/lib64/python2.7/site-packages/duplicity/collections.py", line 836, in get_backup_chains
add_to_sets(f)
File "/usr/lib64/python2.7/site-packages/duplicity/collections.py", line 824, in add_to_sets
if set.add_filename(filename):
File "/usr/lib64/python2.7/site-packages/duplicity/collections.py", line 101, in add_filename
self.set_manifest(filename)
File "/usr/lib64/python2.7/site-packages/duplicity/collections.py", line 148, in set_manifest
self.set_files_changed()
File "/usr/lib64/python2.7/site-packages/duplicity/collections.py", line 128, in set_files_changed
mf = self.get_manifest()
File "/usr/lib64/python2.7/site-packages/duplicity/collections.py", line 250, in get_manifest
return self.get_local_manifest()
File "/usr/lib64/python2.7/site-packages/duplicity/collections.py", line 224, in get_local_manifest
return manifest.Manifest().from_string(manifest_buffer)
File "/usr/lib64/python2.7/site-packages/duplicity/manifest.py", line 215, in from_string
vi = VolumeInfo().from_string(match.group(1))
MemoryError
Here is the upstream bug for this https://bugs.launchpad.net/ubuntu/+source/duplicity/+bug/1727653 This is fixed by 0.7.15, which I just tested, which is in updates-testing, see bug #1512733 Yesterday I've installed 0.7.15, the full backup succeded.
Today the incremental backup has stopped with
Traceback (innermost last):
File "/usr/bin/duplicity", line 1559, in <module>
with_tempdir(main)
File "/usr/bin/duplicity", line 1545, in with_tempdir
fn()
File "/usr/bin/duplicity", line 1394, in main
do_backup(action)
File "/usr/bin/duplicity", line 1526, in do_backup
check_last_manifest(col_stats) # not needed for full backup
File "/usr/bin/duplicity", line 1228, in check_last_manifest
last_backup_set.check_manifests()
File "/usr/lib64/python2.7/site-packages/duplicity/collections.py", line 208, in check_manifests
remote_manifest = self.get_remote_manifest()
File "/usr/lib64/python2.7/site-packages/duplicity/collections.py", line 249, in get_remote_manifest
return manifest.Manifest().from_string(manifest_buffer)
File "/usr/lib64/python2.7/site-packages/duplicity/manifest.py", line 202, in from_string
vi = VolumeInfo().from_string(match.group(1))
File "/usr/lib64/python2.7/site-packages/duplicity/manifest.py", line 390, in from_string
linelist = s.strip().split("
")
MemoryError
Still happens for me with Fedora 27 Duplicity 0.7.15 Same here for fedora 27 w/ duplicity-0.7.15-1.fc27.x86_64 on afp:// mount, was the same on smb:// mount. Also for me, see discussion with upstream at https://bugs.launchpad.net/ubuntu/+source/duplicity/+bug/1730451 A me too. F 27 with duplicity 0.7.15-1.fc27 Slightly different backtrace.
Traceback (innermost last):
File "/usr/bin/duplicity", line 1559, in <module>
with_tempdir(main)
File "/usr/bin/duplicity", line 1545, in with_tempdir
fn()
File "/usr/bin/duplicity", line 1394, in main
do_backup(action)
File "/usr/bin/duplicity", line 1420, in do_backup
action).set_values()
File "/usr/lib64/python2.7/site-packages/duplicity/collections.py", line 720, in set_values
self.get_backup_chains(partials + backend_filename_list)
File "/usr/lib64/python2.7/site-packages/duplicity/collections.py", line 846, in get_backup_chains
add_to_sets(f)
File "/usr/lib64/python2.7/site-packages/duplicity/collections.py", line 834, in add_to_sets
if set.add_filename(filename):
File "/usr/lib64/python2.7/site-packages/duplicity/collections.py", line 102, in add_filename
self.set_manifest(filename)
File "/usr/lib64/python2.7/site-packages/duplicity/collections.py", line 153, in set_manifest
self.set_files_changed()
File "/usr/lib64/python2.7/site-packages/duplicity/collections.py", line 129, in set_files_changed
mf = self.get_manifest()
File "/usr/lib64/python2.7/site-packages/duplicity/collections.py", line 256, in get_manifest
return self.get_local_manifest()
File "/usr/lib64/python2.7/site-packages/duplicity/collections.py", line 234, in get_local_manifest
return manifest.Manifest().from_string(manifest_buffer)
File "/usr/lib64/python2.7/site-packages/duplicity/manifest.py", line 202, in from_string
vi = VolumeInfo().from_string(match.group(1))
MemoryError
Another me too?
Different backtrace on automatic incremental backup
Fedora 27, duplicity 0.7.16.1.fc27, kernel 4.14.14-300.fc27.x86_64
Traceback (innermost last):
File "/usr/bin/duplicity", line 1559, in <module>
with_tempdir(main)
File "/usr/bin/duplicity", line 1545, in with_tempdir
fn()
File "/usr/bin/duplicity", line 1394, in main
do_backup(action)
File "/usr/bin/duplicity", line 1527, in do_backup
incremental_backup(sig_chain)
File "/usr/bin/duplicity", line 669, in incremental_backup
globals.backend)
File "/usr/bin/duplicity", line 430, in write_multivol
globals.volsize)
File "/usr/lib64/python2.7/site-packages/duplicity/gpg.py", line 385, in GPGWriteFile
data = block_iter.next().data
File "/usr/lib64/python2.7/site-packages/duplicity/diffdir.py", line 520, in next
result = self.process_continued()
File "/usr/lib64/python2.7/site-packages/duplicity/diffdir.py", line 700, in process_continued
data, last_block = self.get_data_block(self.process_fp)
File "/usr/lib64/python2.7/site-packages/duplicity/diffdir.py", line 684, in get_data_block
buf = fp.read(read_size)
File "/usr/lib64/python2.7/site-packages/duplicity/librsync.py", line 88, in read
self._add_to_outbuf_once()
File "/usr/lib64/python2.7/site-packages/duplicity/librsync.py", line 98, in _add_to_outbuf_once
self._add_to_inbuf()
File "/usr/lib64/python2.7/site-packages/duplicity/librsync.py", line 113, in _add_to_inbuf
assert not self.infile.close()
File "/usr/lib64/python2.7/site-packages/duplicity/diffdir.py", line 440, in close
self.callback(self.sig_gen.getsig(), *self.extra_args)
File "/usr/lib64/python2.7/site-packages/duplicity/librsync.py", line 232, in getsig
return ''.join(self.sigstring_list)
MemoryError
One more thing: backup is to Samba share, and broke only after upgrade to Fedora 27 (from 26) Another me too, same stacktrace as in comment 11 *** Bug 1544191 has been marked as a duplicate of this bug. *** This seems to be resolved for me (for now). Found that my backup configuration was including a very large sub-directory in my ~/Downloads folder that was explicitly excluded, yet was picked up by the backup. Re-did the configuration and now things are fine. Makes me suspect that at least in my case the volume to be backed up was an issue, or perhaps just re-doing the backup configuration. Excellent! |
Created attachment 1326676 [details] Traceback Description of problem: Traceback durung backup Version-Release number of selected component (if applicable): duplicity-0.7.13.1-2.fc26 How reproducible: every time Steps to Reproduce: 1. start backup 2. 3. Actual results: Traceback Expected results: success backup Additional info: testing repo used backup file in remote nfs share, rest is usual default