Bug 1258753 - Munmap transparent huge page and VM_BUG_ON in set_vma_resv_flags functions
Munmap transparent huge page and VM_BUG_ON in set_vma_resv_flags functions
Product: Fedora
Classification: Fedora
Component: libhugetlbfs (Show other bugs)
x86_64 Linux
unspecified Severity high
: ---
: ---
Assigned To: emunson
Fedora Extras Quality Assurance
Depends On:
  Show dependency treegraph
Reported: 2015-09-01 04:11 EDT by SunDong
Modified: 2016-07-19 15:17 EDT (History)
2 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2016-07-19 15:17:54 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)
testcase and the bug information screenshots (141.89 KB, application/x-gzip)
2015-09-01 04:11 EDT, SunDong
no flags Details

  None (edit)
Description SunDong 2015-09-01 04:11:49 EDT
Created attachment 1068873 [details]
testcase and the bug information screenshots

Description of problem:
I think I find a linux bug, I have the test cases is constructed. I can stable recurring problems in fedora22(4.0.4-301.fc22.x86_64) kernel version, arch for x86_64.
I construct transparent huge page, when the parent and child process with MAP_SHARE, MAP_PRIVATE way to access the same huge page area, it has the opportunity to lead to huge page copy on write failure, and then it will munmap the child corresponding mmap area, but then the child mmap area with VM_MAYSHARE attributes, child process munmap this area can trigger VM_BUG_ON in set_vma_resv_flags functions (VM_BUG_ON(vma->vm_flags & VM_MAYSHARE)).

Version-Release number of selected component (if applicable):

How reproducible:
stable recurring problems

Steps to Reproduce:
Testcase and the information can refer to the attachment.

Actual results:

Expected results:

Additional info:
I think we can through the following two methods to modify the problem. The two methods are validated.

Method 1:
diff --git a/mm/mmap.c b/mm/mmap.c
index bb50cac..19c43ca 100644
--- a/mm/mmap.c
+++ b/mm/mmap.c
@@ -1347,6 +1347,7 @@ unsigned long do_mmap_pgoff(struct file *file, unsigned long addr,
                                return -ENODEV;
                        if (vm_flags & (VM_GROWSDOWN|VM_GROWSUP))
                                return -EINVAL;
+                       vm_flags &= ~(VM_SHARED | VM_MAYSHARE);
@@ -1368,6 +1369,7 @@ unsigned long do_mmap_pgoff(struct file *file, unsigned long addr,
                         * Set pgoff according to addr for anon_vma.
                        pgoff = addr >> PAGE_SHIFT;
+                       vm_flags &= ~(VM_SHARED | VM_MAYSHARE);
                        return -EINVAL;

method 2: 
diff --git a/mm/hugetlb.c b/mm/hugetlb.c
index 650ee57..153ceff 100644
--- a/mm/hugetlb.c
+++ b/mm/hugetlb.c
@@ -423,7 +423,6 @@ static void set_vma_resv_map(struct vm_area_struct *vma, struct resv_map *map)
 static void set_vma_resv_flags(struct vm_area_struct *vma, unsigned long flags)
-       VM_BUG_ON(vma->vm_flags & VM_MAYSHARE);
        set_vma_private_data(vma, get_vma_private_data(vma) | flags);

If you used low level kernel version, you may fix other bugs and you will find this problem.

commit 4998a6c0edce7fae9c0a5463f6ec3fa585258ee7
commit 66aebce747eaf9bc456bf1f1b217d8db843031d0
commit f12d5bfceb7e1f9051563381ec047f7f13956c3c
Comment 1 Fedora End Of Life 2016-07-19 15:17:54 EDT
Fedora 22 changed to end-of-life (EOL) status on 2016-07-19. Fedora 22 is
no longer maintained, which means that it will not receive any further
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version. If you
are unable to reopen this bug, please file a new report against the
current release. If you experience problems, please add a comment to this

Thank you for reporting this bug and we are sorry it could not be fixed.

Note You need to log in before you can comment on or make changes to this bug.