1327316 – [RHEL-6.7][selinux-policy] load_policy: page allocation failure. order:0, mode:0xd0

RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1327316 - [RHEL-6.7][selinux-policy] load_policy: page allocation failure. order:0, mode:0xd0

Summary: [RHEL-6.7][selinux-policy] load_policy: page allocation failure. order:0, mod...

Keywords:
Status:	CLOSED CANTFIX
Alias:	None
Product:	Red Hat Enterprise Linux 6
Classification:	Red Hat
Component:	selinux-policy
Sub Component:
Version:	6.9
Hardware:	Unspecified
OS:	Unspecified
Priority:	unspecified
Severity:	unspecified
Target Milestone:	rc
Target Release:	---
Assignee:	Lukas Vrabec
QA Contact:	BaseOS QE Security Team
Docs Contact:
URL:
Whiteboard:
Duplicates (1):	1301089 (view as bug list)
Depends On:
Blocks:	1359574 1366045
TreeView+	depends on / blocked

Reported:	2016-04-14 18:41 UTC by PaulB
Modified:	2016-10-19 17:22 UTC (History)
CC List:	13 users (show)
Fixed In Version:
Doc Type:	Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed:	2016-09-27 13:33:05 UTC
Target Upstream Version:
Embargoed:

Attachments	(Terms of Use)

Description PaulB 2016-04-14 18:41:09 UTC

Description of problem:
Target system failed kdump testing, as kdump kernel failed to boot
due to:
 load_policy: page allocation failure. order:0, mode:0xd0 

Version-Release number of selected component (if applicable):
distro: RHEL-6.7 Server x86_64
kernel: 2.6.32-573.26.1.el6

How reproducible:
 unknown

Steps to Reproduce:
1. Install target system system listed in comment#1 with RHEL-6.7 Server x86_64
2. Install kernel-2.6.32-573.26.1.el6
3. Trigger a crash
   echo c > /proc/sysrq-trigger

Actual results:
https://beaker.engineering.redhat.com/recipes/2645087
http://beaker-archive.app.eng.bos.redhat.com/beaker-logs/2016/04/12999/1299965/2645087/console.log
---<-snip->---
Out of memory: Kill process 2300 (load_selinux_po) score 1 or sacrifice child 
Killed process 2302, UID 0, (load_policy) total-vm:115612kB, anon-rss:132kB, file-rss:4kB 
 rport-6:0-13: blocked FC remote port time out: removing rport 
load_policy: page allocation failure. order:0, mode:0xd0 
Pid: 2302, comm: load_policy Not tainted 2.6.32-573.26.1.el6.x86_64 #1 
Call Trace: 
 [<ffffffff81137a8c>] ? __alloc_pages_nodemask+0x7dc/0x950 
 [<ffffffff811373d9>] ? __alloc_pages _nodemask+0x129/66666  vvV&��� � �k+k�;+�� ;+�[ ��&� ��v �
 [<ffffffff811781ca>] ? fallback_alloc+0x1ba/0x270 
 [<ffffffff81177c1f>] ? cache_grow+0x2cf/0x320 
 [<ffffffff81177f49>] ? ____cache_alloc_node+0x99/0x160 
 [<ffffffff81178d53>] ? kmem_cache_alloc+0x123/0x190 
 [<ffffffff8124873d>] ? avtab_insert_nonunique+0xcd/0x140 
 [<ffffffff8100bc0e>] ? apic_timer_interrupt+0xe/0x20 
 [<ffffffff81252726>] ? cond_insertf+0x116/0x200 
 [<ffffffff81248a5e>] ? avtab_read_item+0x15e/0x390 
 [<ffffffff8100bc0e>] ? apic_timer_interrupt+0xe/0x20 
 [<ffffffff81252610>] ? cond_insertf+0x0/0x200 
 [<ffffffff81252610>] ? cond_insertf+0x0/0x200 
 [<ffffffff81248901>] ? avtab_read_item+0x1/0x390 
 [<ffffffff81252bd1>] ? cond_read_av_list+0xb1/0xd0 
 [<ffffffff81252e82>] ? cond_read_list+0x162/0x2a0 
 [<ffffffff8124ca66>] ? policydb_read+0x576/0x1080 
 [<ffffffff8117b686>] ? kmem_cache_create+0x456/0x5a0 
 [<ffffffff812507dc>] ? security_load_policy+0x5c/0x410 
 [<ffffffff81184b47>] ? mem_cgroup_update_file_mapped+0x17/0x90 
 [<ffffffff81127a37>] ? unlock_page+0x27/0x30 
 [<ffffffff811526d9>] ? __do_fault+0x469/0x530 
 [<ffffffff81152897>] ? handle_pte_fault+0xf7/0xb20 
 [<ffffffff8153927e>] ? thread_return+0x4e/0x7d0 
 [<ffffffff8117097a>] ? alloc_pages_current+0xaa/0x110 
 [<ffffffff8100bc0e>] ? apic_timer_interrupt+0xe/0x20 
 [<ffffffff81153559>] ? handle_mm_fault+0x299/0x3d0 
 [<ffffffff8104f204>] ? __do_page_fault+0x1f4/0x500 
 [<ffffffff81134f4e>] ? __get_free_pages+0xe/0x50 
 [<ffffffff8115fa69>] ? vmap_page_range_noflush+0x279/0x370 
 [<ffffffff8153f90e>] ? do_page_fault+0x3e/0xa0 
 [<ffffffff8153cc55>] ? page_fault+0x25/0x30 
 [<ffffffff812443eb>] ? sel_write_load+0xdb/0x710 
 [<ffffffff8123f2bb>] ? selinux_file_permission+0xfb/0x150 
 [<ffffffff81232026>] ? security_file_permission+0x16/0x20 
 [<ffffffff81192208>] ? vfs_write+0xb8/0x1a0 
 [<ffffffff811936f6>] ? fget_light_pos+0x16/0x50 
 [<ffffffff81192d41>] ? sys_write+0x51/0xb0 
 [<ffffffff8100b0d2>] ? system_call_fastpath+0x16/0x1b 
---<-snip->---


Expected results:
successful boot of kdump kernel and capture of vmcore

Additional info:

Comment 3 Lukas Vrabec 2016-04-18 10:24:07 UTC

I don't think this is something for selinux-policy component. 

Paul, 
Can you look on this? 

Thank you.

Comment 4 Paul Moore 2016-04-18 16:57:43 UTC

It appears that the kernel is having a difficult time allocating kernel memory for the SELinux policy load, I'm guessing this is due to memory pressure caused by the fact that kernel is a kdump recovery kernel and a good portion of the physical memory is reserved for dumping to disk.

Is there any reason why we are loading the SELinux policy in the kdump kernel?  This kernel boot instance is single user and exists only to write the memory image to disk, yes?  If so, it seems like we could avoid loading the SELinux policy in this case and avoiding this problem completely.

Comment 5 PaulB 2016-05-16 12:05:29 UTC

(In reply to Paul Moore from comment #4)
> It appears that the kernel is having a difficult time allocating kernel
> memory for the SELinux policy load, I'm guessing this is due to memory
> pressure caused by the fact that kernel is a kdump recovery kernel and a
> good portion of the physical memory is reserved for dumping to disk.
> 
> Is there any reason why we are loading the SELinux policy in the kdump
> kernel?  This kernel boot instance is single user and exists only to write
> the memory image to disk, yes?  If so, it seems like we could avoid loading
> the SELinux policy in this case and avoiding this problem completely.


All,
Adding kdump team for response...
ruyang?

Best,
-pbunyan

Comment 6 Dave Young 2016-05-17 01:16:24 UTC

We load selinux policy when dumping to rootfs, the purpose is to labeling the dump files correctly.

For the oom, Paul, can you retest with kdump.conf option "debug_mem_level 3" so that we can get who is using most of the memory?

Thanks
Dave

Comment 8 Lukas Vrabec 2016-09-26 14:00:20 UTC

Could you look on comment#6 and retest it with changed kdump.conf? 

Thanks.

Comment 9 Lukas Vrabec 2016-09-26 14:02:00 UTC

*** Bug 1301089 has been marked as a duplicate of this bug. ***

Comment 10 Lukas Vrabec 2016-09-26 14:07:15 UTC

*** Bug 1323086 has been marked as a duplicate of this bug. ***

Comment 11 Lukas Vrabec 2016-09-27 13:33:05 UTC

This looks more kernel bug then selinux-policy issue. Closing as CANTFIX and this should be fixed in #1379260.

Comment 13 Pratyush Anand 2016-10-18 02:43:11 UTC

In reply to PaulB from comment #12)
> -----------------------------------------------------------
> However, looking at the console log I do see the following:
> -----------------------------------------------------------
> ---<-snip->---
> Loading SELINUX policy 
> load_policy invoked oom-killer: gfp_mask=0xd0, order=0, oom_adj=0,
> oom_score_adj=0 

Hi Paul,

You should not see these messages with "Release 2.0.0-304". 

Following patch removes selinux policy load from kdump kernel.

98f374eeff89 mkdumprd: Remove load_selinux_policy script

~Pratyush

Comment 14 PaulB 2016-10-19 17:22:00 UTC

(In reply to Pratyush Anand from comment #13)
> In reply to PaulB from comment #12)
> > -----------------------------------------------------------
> > However, looking at the console log I do see the following:
> > -----------------------------------------------------------
> > ---<-snip->---
> > Loading SELINUX policy 
> > load_policy invoked oom-killer: gfp_mask=0xd0, order=0, oom_adj=0,
> > oom_score_adj=0 
> 
> Hi Paul,
> 
> You should not see these messages with "Release 2.0.0-304". 
> 
> Following patch removes selinux policy load from kdump kernel.
> 
> 98f374eeff89 mkdumprd: Remove load_selinux_policy script
> 
> ~Pratyush

Pratyush,
You are correct. 
The Call Trace in comment #12 is no longer seen:
host: dell-per320-02.khw.lab.eng.bos.redhat.com
distro: RHEL-6.8
kernel: 2.6.32-642.el6
kexec-tools: 2.0.0-304.el6

See here:
 https://beaker.engineering.redhat.com/jobs/1558254

Best,
-pbunyan

Note You need to log in before you can comment on or make changes to this bug.