Bug 1751901

Summary:	Btrfs: fix unwritten extent buffers and hangs on future writeback attempts
Product:	[Fedora] Fedora	Reporter:	Chris Murphy <bugzilla>
Component:	kernel	Assignee:	Kernel Maintainer List <kernel-maint>
Status:	CLOSED ERRATA	QA Contact:	Fedora Extras Quality Assurance <extras-qa>
Severity:	unspecified	Docs Contact:
Priority:	unspecified
Version:	30	CC:	airlied, bskeggs, fedora-kernel-btrfs, hdegoede, ichavero, itamar, jarodwilson, jeremy, jglisse, john.j5live, jonathan, josef, kernel-maint, linville, masami256, mchehab, mjg59, steved
Target Milestone:	---
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	kernel-5.2.15-200.fc30 kernel-5.2.17-100.fc29	Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2019-09-19 01:31:12 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Chris Murphy 2019-09-13 00:11:41 UTC

This bug is in all 5.2+ kernels, fix is on the way to 5.3.0 and stable. While rare, if we can spare even one user from running into it, that'd be a good thing.

Upstream discussion and patch
https://lore.kernel.org/linux-btrfs/20190911145542.1125-1-fdmanana@kernel.org/T/#u

Arch has applied to 5.2.14. I separately tested that it applies cleanly.
https://git.archlinux.org/linux.git/commit/?h=v5.2.14-arch2&id=45fc8773f47b7cbe56caab0e14abf26d1e044e63


Call trace if this bug is hit looks like this:


  [49887.347053] INFO: task btrfs-transacti:1752 blocked for more than 122 seconds.
  [49887.347059]       Not tainted 5.2.13-gentoo #2
  [49887.347060] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
  [49887.347062] btrfs-transacti D    0  1752      2 0x80004000
  [49887.347064] Call Trace:
  [49887.347069]  ? __schedule+0x265/0x830
  [49887.347071]  ? bit_wait+0x50/0x50
  [49887.347072]  ? bit_wait+0x50/0x50
  [49887.347074]  schedule+0x24/0x90
  [49887.347075]  io_schedule+0x3c/0x60
  [49887.347077]  bit_wait_io+0x8/0x50
  [49887.347079]  __wait_on_bit+0x6c/0x80
  [49887.347081]  ? __lock_release.isra.29+0x155/0x2d0
  [49887.347083]  out_of_line_wait_on_bit+0x7b/0x80
  [49887.347084]  ? var_wake_function+0x20/0x20
  [49887.347087]  lock_extent_buffer_for_io+0x28c/0x390
  [49887.347089]  btree_write_cache_pages+0x18e/0x340
  [49887.347091]  do_writepages+0x29/0xb0
  [49887.347093]  ? kmem_cache_free+0x132/0x160
  [49887.347095]  ? convert_extent_bit+0x544/0x680
  [49887.347097]  filemap_fdatawrite_range+0x70/0x90
  [49887.347099]  btrfs_write_marked_extents+0x53/0x120
  [49887.347100]  btrfs_write_and_wait_transaction.isra.4+0x38/0xa0
  [49887.347102]  btrfs_commit_transaction+0x6bb/0x990
  [49887.347103]  ? start_transaction+0x33e/0x500
  [49887.347105]  transaction_kthread+0x139/0x15c

Comment 1 Chris Murphy 2019-09-13 00:57:52 UTC

Email with PR to Linus including fix for 5.3.0 is here; it also mentions they intend to push it to stable.
https://lore.kernel.org/linux-btrfs/cover.1568307806.git.dsterba@suse.com/T/#u

The commit referenced in that PR is here (it's the same as above, but maybe the trail of breadcrumbs with this is better)
https://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux.git/commit/?h=for-5.3-rc8&id=18dfa7117a3f379862dcd3f67cadd678013bb9dd

Comment 2 Chris Murphy 2019-09-13 01:11:46 UTC

I applied this on top of 5.3.0-rc8, compiles on x86_64 and boots fine.

Comment 3 Fedora Update System 2019-09-17 12:04:42 UTC

FEDORA-2019-38d2e07507 has been submitted as an update to Fedora 29. https://bodhi.fedoraproject.org/updates/FEDORA-2019-38d2e07507

Comment 4 Fedora Update System 2019-09-17 12:09:01 UTC

FEDORA-2019-e3010166bd has been submitted as an update to Fedora 30. https://bodhi.fedoraproject.org/updates/FEDORA-2019-e3010166bd

Comment 5 Fedora Update System 2019-09-18 01:20:29 UTC

kernel-5.2.15-200.fc30, kernel-headers-5.2.15-200.fc30 has been pushed to the Fedora 30 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2019-e3010166bd

Comment 6 Fedora Update System 2019-09-18 01:26:49 UTC

kernel-5.2.15-100.fc29, kernel-headers-5.2.15-100.fc29 has been pushed to the Fedora 29 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2019-38d2e07507

Comment 7 Fedora Update System 2019-09-19 01:31:12 UTC

kernel-5.2.15-200.fc30, kernel-headers-5.2.15-200.fc30 has been pushed to the Fedora 30 stable repository. If problems still persist, please make note of it in this bug report.

Comment 8 Fedora Update System 2019-09-20 01:38:32 UTC

FEDORA-2019-fa29e8d620 has been submitted as an update to Fedora 29. https://bodhi.fedoraproject.org/updates/FEDORA-2019-fa29e8d620

Comment 9 Fedora Update System 2019-09-21 03:04:23 UTC

kernel-5.2.16-100.fc29, kernel-headers-5.2.16-100.fc29 has been pushed to the Fedora 29 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2019-fa29e8d620

Comment 10 Fedora Update System 2019-09-24 12:34:18 UTC

FEDORA-2019-a570a92d5a has been submitted as an update to Fedora 29. https://bodhi.fedoraproject.org/updates/FEDORA-2019-a570a92d5a

Comment 11 Fedora Update System 2019-09-25 03:47:28 UTC

kernel-5.2.17-100.fc29, kernel-headers-5.2.17-100.fc29, kernel-tools-5.2.17-100.fc29 has been pushed to the Fedora 29 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2019-a570a92d5a

Comment 12 Fedora Update System 2019-10-02 01:41:15 UTC

kernel-5.2.17-100.fc29, kernel-headers-5.2.17-100.fc29, kernel-tools-5.2.17-100.fc29 has been pushed to the Fedora 29 stable repository. If problems still persist, please make note of it in this bug report.