RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 985142 - audit freezes the system on disk error
Summary: audit freezes the system on disk error
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: selinux-policy
Version: 7.0
Hardware: All
OS: Linux
high
high
Target Milestone: beta
: ---
Assignee: Miroslav Grepl
QA Contact: Ondrej Moriš
URL:
Whiteboard:
Depends On:
Blocks: RHEL7CCC 978512
TreeView+ depends on / blocked
 
Reported: 2013-07-16 22:49 UTC by Ondrej Moriš
Modified: 2015-11-03 10:24 UTC (History)
6 users (show)

Fixed In Version: selinux-policy-3.12.1-69.el7
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2014-06-13 09:52:46 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)

Description Ondrej Moriš 2013-07-16 22:49:23 UTC 
Description of problem:

When audit is configured to perform SINGLE action on disk error, it will freezes the system instead when such an error happens. System became unusable and needs to be rebooted. Moreover, quite a kernel call trace is printed into the /v/l/m (see additional info).

Version-Release number of selected component (if applicable):

audit-2.3.1-4.el7.x86_64

How reproducible:

100%

Steps to Reproduce:

0. watch /var/log/messages

1. Configure auditd as follows (/etc/audit/auditd.conf):

log_file = /var/log/audit/audit.log
log_format = RAW
log_group = root
priority_boost = 3
flush = INCREMENTAL
freq = 20
num_logs = 4
max_log_file = 8 
max_log_file_action = KEEP_LOGS
space_left = 75
space_left_action = SYSLOG
action_mail_acct = root
admin_space_left = 50
admin_space_left_action = SINGLE
disk_full_action = SINGLE
disk_error_action = SINGLE
tcp_listen_queue = 5
tcp_max_per_addr = 1
tcp_client_ports = 60
tcp_client_max_idle = 0
name_format = none
enable_krb5 = no
krb5_principal = auditd
DISP_qos = lossy

2. restart auditd service 

3. simulate the dist error as follows:

chcon system_u:object_r:games_data_t:s0 /var/log/audit/audit.log

4. do some action, eg. auditctl -m "test" or something similar

Actual results:

System "freezes" with a huge kernel call trace (see additional info).

Expected results:

System should enter single user mode.

Additional info:


Jul 16 18:08:58 hp-dl360gen8-01 auditd[15791]: The audit daemon is now changing the system to single user mode due to previously mentioned write error
...<many of these>...
Jul 16 18:08:58 hp-dl360gen8-01 auditd[15791]: The audit daemon is now changing the system to single user mode due to previously mentioned write error
Jul 16 18:08:58 hp-dl360gen8-01 systemd[1]: SELinux policy denies access.
...<many of these>...
Jul 16 18:08:58 hp-dl360gen8-01 systemd[1]: SELinux policy denies access.
Jul 16 18:08:58 hp-dl360gen8-01 auditd[15791]: The audit daemon is now changing the system to single user mode due to previously mentioned write error
...<many of these>...
Jul 16 18:08:58 hp-dl360gen8-01 auditd[15791]: The audit daemon is now changing the system to single user mode due to previously mentioned write error
Jul 16 18:08:58 hp-dl360gen8-01 systemd[1]: SELinux policy denies access.
...<many of these>...
Jul 16 18:10:28 hp-dl360gen8-01 kernel: [ 5486.570262] BUG: soft lockup - CPU#8 stuck for 22s! [auditd:15796]
Jul 16 18:10:28 hp-dl360gen8-01 kernel: [ 5486.622240] BUG: soft lockup - CPU#12 stuck for 22s! [systemd:1]
Jul 16 18:10:28 hp-dl360gen8-01 kernel: [ 5486.602334] Modules linked in: nf_conntrack_netbios_ns
Jul 16 18:10:28 hp-dl360gen8-01 kernel: [ 5486.622241] Modules linked in: nf_conntrack_netbios_ns nf_conntrack_broadcast ipt_MASQUERADE ip6table_nat nf_nat_ipv
6 ip6table_mangle ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 iptable_nat nf_nat_ipv4 nf_nat iptable_mangle ipt_REJECT nf_conntrack_ipv4 nf_defrag_ipv4 xt_con
ntrack nf_conntrack ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter ip_tables sg mperf coretemp kvm_intel kvm tg3 iTCO_wdt crc32_pclmul ptp g
hash_clmulni_intel iTCO_vendor_support ioatdma hpwdt sb_edac pps_core hpilo lpc_ich microcode serio_raw pcspkr edac_core mfd_core dca acpi_power_meter zlib wp5
12 twofish_generic twofish_avx_x86_64 twofish_x86_64_3way twofish_x86_64 twofish_common tea sha512_ssse3 sha512_generic sha256_ssse3 sha1_ssse3 serpent_avx_x86
_64 serpent_sse2_x86_64 serpent_generic seed salsa20_generic salsa20_x86_64 rmd320 rmd256 rmd160 rmd128 michael_mic md4 lzo khazad ghash_generic gcm fcrypt dm_
crypt des_generic deflate zlib_deflate cts crypto_null ccm cast6_avx_x86_64 cast6_generic cast_common camellia_generic camellia_aesni_avx_x86_64 camellia_x86_6
4 blowfish_generic blowfish_x86_64 blowfish_common arc4 ansi_cprng xfs libcrc32c ata_generic pata_acpi mgag200 i2c_algo_bit drm_kms_helper sd_mod ttm crc_t10di
f ata_piix drm crc32c_intel libata i2c_core hpsa dm_mirror dm_region_hash dm_log dm_mod [last unloaded: tcrypt]
Jul 16 18:10:28 hp-dl360gen8-01 kernel: [ 5486.622313] CPU: 12 PID: 1 Comm: systemd Not tainted 3.10.0-0.rc7.64.el7.x86_64 #1
Jul 16 18:10:28 hp-dl360gen8-01 kernel: [ 5486.622314] Hardware name: HP ProLiant DL360p Gen8, BIOS P71 07/15/2012
Jul 16 18:10:28 hp-dl360gen8-01 kernel: [ 5486.622316] task: ffff8804298e0000 ti: ffff8804298a4000 task.ti: ffff8804298a4000
Jul 16 18:10:28 hp-dl360gen8-01 kernel: [ 5486.622316] RIP: 0010:[<ffffffff810d8dd2>]  [<ffffffff810d8dd2>] audit_log_start+0x92/0x430
Jul 16 18:10:28 hp-dl360gen8-01 kernel: [ 5486.622324] RSP: 0018:ffff8804298a5ae8  EFLAGS: 00000206
Jul 16 18:10:28 hp-dl360gen8-01 kernel: [ 5486.622324] RAX: 000000000000ea60 RBX: 00000001004ebc2a RCX: 0000000000000146
Jul 16 18:10:28 hp-dl360gen8-01 kernel: [ 5486.622325] RDX: 00000001004f1b57 RSI: 000000000000ea60 RDI: 0000000000000140
Jul 16 18:10:28 hp-dl360gen8-01 kernel: [ 5486.622326] RBP: ffff8804298a5b60 R08: ffff8804298a5b28 R09: 0000000000000001
Jul 16 18:10:28 hp-dl360gen8-01 kernel: [ 5486.622327] R10: fffffffeffb1cf09 R11: 000000000001e843 R12: 0000000000000000
Jul 16 18:10:28 hp-dl360gen8-01 kernel: [ 5486.622328] R13: 0000000000000286 R14: ffff8804298a5b60 R15: ffffffff8160133a
Jul 16 18:10:28 hp-dl360gen8-01 kernel: [ 5486.622329] FS:  00007ff7a0425880(0000) GS:ffff88083f880000(0000) knlGS:0000000000000000
Jul 16 18:10:28 hp-dl360gen8-01 kernel: [ 5486.622330] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Jul 16 18:10:28 hp-dl360gen8-01 kernel: [ 5486.622330] CR2: 00007f3e1f923ef0 CR3: 0000000827ad1000 CR4: 00000000000407e0
Jul 16 18:10:28 hp-dl360gen8-01 kernel: [ 5486.622331] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Jul 16 18:10:28 hp-dl360gen8-01 kernel: [ 5486.622332] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Jul 16 18:10:28 hp-dl360gen8-01 kernel: [ 5486.622332] Stack:
Jul 16 18:10:28 hp-dl360gen8-01 kernel: [ 5486.622333]  0000000000000000 000000d000000453 ffff8804298e0000 ffff8804291acb40
Jul 16 18:10:28 hp-dl360gen8-01 kernel: [ 5486.622336]  0000000000000000 0000000000000000 ffff8804298e0000 ffffffff81094310
Jul 16 18:10:28 hp-dl360gen8-01 kernel: [ 5486.622339]  dead000000100100 dead000000200200 ffff8804298a5ba8 ffff8804298e0000
Jul 16 18:10:28 hp-dl360gen8-01 kernel: [ 5486.622341] Call Trace:
Jul 16 18:10:28 hp-dl360gen8-01 kernel: [ 5486.622343]  [<ffffffff81094310>] ? wake_up_state+0x20/0x20
Jul 16 18:10:28 hp-dl360gen8-01 kernel: [ 5486.622348]  [<ffffffff810d998f>] audit_log_common_recv_msg+0x4f/0xa0
Jul 16 18:10:28 hp-dl360gen8-01 kernel: [ 5486.622350]  [<ffffffff810da05d>] audit_receive_msg+0x67d/0x8c0
Jul 16 18:10:28 hp-dl360gen8-01 kernel: [ 5486.622352]  [<ffffffff81288bc5>] ? sock_has_perm+0x75/0x90
Jul 16 18:10:28 hp-dl360gen8-01 kernel: [ 5486.622357]  [<ffffffff810da2f0>] audit_receive+0x50/0xa0
Jul 16 18:10:28 hp-dl360gen8-01 kernel: [ 5486.622359]  [<ffffffff8152148d>] netlink_unicast+0xdd/0x190
Jul 16 18:10:28 hp-dl360gen8-01 kernel: [ 5486.622365]  [<ffffffff81521806>] netlink_sendmsg+0x2c6/0x6c0
Jul 16 18:10:28 hp-dl360gen8-01 kernel: [ 5486.622367]  [<ffffffff814dee29>] sock_sendmsg+0x99/0xd0
Jul 16 18:10:28 hp-dl360gen8-01 kernel: [ 5486.622372]  [<ffffffff811a793b>] ? kern_path+0x4b/0x70
Jul 16 18:10:28 hp-dl360gen8-01 kernel: [ 5486.622376]  [<ffffffff814e6eb9>] ? skb_dequeue+0x59/0x70
Jul 16 18:10:28 hp-dl360gen8-01 kernel: [ 5486.622380]  [<ffffffff814e6eb9>] ? skb_dequeue+0x59/0x70
Jul 16 18:10:28 hp-dl360gen8-01 kernel: [ 5486.622383]  [<ffffffff814df381>] SYSC_sendto+0x121/0x1c0
Jul 16 18:10:28 hp-dl360gen8-01 kernel: [ 5486.622385]  [<ffffffff8128bc0e>] ? file_has_perm+0x8e/0xa0
Jul 16 18:10:28 hp-dl360gen8-01 kernel: [ 5486.622388]  [<ffffffff8119c71e>] ? ____fput+0xe/0x10
Jul 16 18:10:28 hp-dl360gen8-01 kernel: [ 5486.622392]  [<ffffffff814dfd6e>] SyS_sendto+0xe/0x10
Jul 16 18:10:28 hp-dl360gen8-01 kernel: [ 5486.622394]  [<ffffffff8160cd59>] system_call_fastpath+0x16/0x1b
Jul 16 18:10:28 hp-dl360gen8-01 kernel: [ 5486.622399] Code: 24 10 8b 15 9d ff 85 00 8b 05 93 ff 85 00 8b 0d 75 b0 be 00 85 d2 48 63 f0 0f 84 32 01 00 00 41 8d 3c 16 39 f9 0f 86 26 01 00 00 <45> 85 e4 0f 84 b5 00 00 00 85 c0 0f 84 ad 00 00 00 48 8b 15 16 
Jul 16 18:10:28 hp-dl360gen8-01 kernel: [ 5486.635235] BUG: soft lockup - CPU#13 stuck for 22s! [systemctl:15841]
...
Comment 1 Daniel Walsh 2013-07-17 14:02:25 UTC 
THis is an SELinux issue, can you run the test in permissive mode and gather the AVC's.

We particularly want to see what auditd_t is trying to do to systemd.
Comment 2 Steve Grubb 2013-07-17 14:17:06 UTC 
I'll also will look at auditd code to see if there is some room for improvement after trying to change the run level.
Comment 3 Ondrej Moriš 2013-07-18 09:01:14 UTC 
Gathering AVC might be a problem, since the situation is completely different in permissive mode. I see only the following AVC:

----
time->Wed Jul 17 10:56:14 2013
type=SYSCALL msg=audit(1374072974.945:29): arch=c000003e syscall=1 success=yes exit=209 a0=4 a1=7fc3772c0000 a2=d1 a3=d0 items=0 ppid=1 pid=2390 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 ses=4294967295 tty=(none) comm="auditd" exe="/usr/sbin/auditd" subj=system_u:system_r:auditd_t:s0 key=(null)
type=AVC msg=audit(1374072974.945:29): avc:  denied  { append } for  pid=2390 comm="auditd" path="/var/log/audit/audit.log" dev="dm-1" ino=2368001 scontext=system_u:system_r:auditd_t:s0 tcontext=system_u:object_r:games_data_t:s0 tclass=file

Which seems to be perfectly correct.

The problem appears only if audit.log cannot be opened for writing, which is the case only in enforcing mode I think. 

Any suggestions how to get more information you may find helpful?
Comment 4 Miroslav Grepl 2013-07-18 10:27:34 UTC 
Ondrej,
could you run systemd in DEBUG mode? We could see more info in

# journalctl -x
Comment 5 Ondrej Moriš 2013-07-18 22:38:21 UTC 
Well, I have booted with systemd in debug and when a disk_error is hit, systemd keeps printing tons of the following messages:

Jul 18 18:20:52 hp-dl360gen8-01.rhts.eng.bos.redhat.com systemd[1]: SELinux policy denies access.
Jul 18 18:20:52 hp-dl360gen8-01.rhts.eng.bos.redhat.com systemd[1]: SELinux access check scon=system_u:system_r:auditd_t:s0 tcon=system_u:object_r:systemd_unit_file_t:s0 tclass=service perm=start path=/usr/lib/systemd/system/rescue.target cmdline=(null): -13

and 

Jul 18 18:20:52 hp-dl360gen8-01.rhts.eng.bos.redhat.com auditd[1776]: The audit daemon is now changing the system to single user mode due to previously mention...

There are thousands of both these messages (after a few minutes /v/l/m is about 200 MB and keeps growing until the system goes down).

Then (later) there is:

Jul 18 18:20:52 hp-dl360gen8-01.rhts.eng.bos.redhat.com kernel: Out of memory: Kill process 1509 (dhclient) score 0 or sacrifice child

I am sorry, but I see nothing else.
Comment 6 Miroslav Grepl 2013-07-22 06:56:31 UTC 
commit 1f239dd385a2f13d36f3ac38698e8525a1c2d796
Author: Miroslav Grepl <mgrepl>
Date:   Mon Jul 22 08:55:23 2013 +0200

    Make auditd working if audit is configured to perform SINGLE action on disk error
Comment 7 Steve Grubb 2013-07-22 13:32:04 UTC 
Will it also work if auditd attempts to halt the system?
Comment 8 Ondrej Moriš 2013-08-07 13:53:30 UTC 
I have tried to reproduce the issues with selinux-policy-3.12.1-69.el7.noarch and it is still there:

Aug  7 15:00:57 cc-toe3 kernel: [ 1085.847453] Code: 62 85 00 8b 0d a5 00 be 00 85 d2 48 63 f0 0f 84 32 01 00 00 41 8d 3c 16 39 f9 0f 86 26 01 00 00 45 85 e4 0f 84 b5 00 00 00 85 c0 <0f> 84 ad 00 00 00 48 8b 15 46 b2 92 00 49 89 f2 49 29 d2 4f 8d 
Aug  7 15:00:57 cc-toe3 kernel: [ 1087.617085]  cmac rmd160 crypto_null camellia_generic camellia_x86_64 lzo cast6_generic cast5_generic cast_common deflate zlib_deflate cts gcm ccm serpent_sse2_x86_64 serpent_generic blowfish_generic blowfish_x86_64 blowfish_common twofish_generic ip6table_filter twofish_x86_64_3way twofish_x86_64 i
p6_tables twofish_common xcbc ebtable_nat ebtables sha512_generic des_generic ah6 ah4 esp6 esp4 xfrm4_mode_beet xfrm4_tunnel tunnel4 xfrm4_mode_tunnel xfrm4_mode_transport xfrm6_mode_transport xfrm6_mode_ro xfrm6_mode_beet xfrm6_mode_tunnel ipcomp ipcomp6 xfrm6_tunnel tunnel6 xfrm_ipcomp af_key ipt_MASQUERADE iptable_nat nf_nat_ipv4 
nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT xt_CHECKSUM iptable_mangle iptable_filter ip_tables bridge stp llc sg mperf kvm_amd kvm amd64_edac_mod e1000e edac_mce_amd ptp sp5100_tco microcode pcspkr serio_raw edac_core k10temp i2c_piix4 pps_core hpilo hpwdt bnx2 acpi_power_meter xfs libcrc32c sr_mod c
drom i2c_algo_bit sd_mod drm_kms_helper crc_t10dif ata_generic ttm ahci pata_acpi libahci pata_atiixp drm libata i2c_core hpsa dm_mirror dm_region_hash dm_log dm_mod
Aug  7 15:00:57 cc-toe3 kernel: [ 1088.188720] CPU: 1 PID: 3839 Comm: sshd Not tainted 3.10.0-5.el7.bz983949.1.x86_64 #1
Aug  7 15:00:57 cc-toe3 kernel: [ 1088.234226] Hardware name: HP ProLiant DL385 G7, BIOS A18 03/19/2012
Aug  7 15:00:57 cc-toe3 kernel: [ 1088.271683] task: ffff88023873e420 ti: ffff880238cae000 task.ti: ffff880238cae000
Aug  7 15:00:57 cc-toe3 kernel: [ 1088.315166] RIP: 0010:[<ffffffff810d8dad>]  [<ffffffff810d8dad>] audit_log_start+0x9d/0x430
Aug  7 15:00:57 cc-toe3 kernel: [ 1088.363961] RSP: 0018:ffff880238cafae8  EFLAGS: 00000206
Aug  7 15:00:57 cc-toe3 kernel: [ 1088.394658] RAX: 000000000000ea60 RBX: 00000001000aa880 RCX: 0000000000000141
Aug  7 15:00:57 cc-toe3 kernel: [ 1088.436683] RDX: 00000001000be735 RSI: 000000000000ea60 RDI: 0000000000000140
Aug  7 15:00:57 cc-toe3 kernel: [ 1088.477679] RBP: ffff880238cafb60 R08: ffff880238cafb28 R09: 0000000000000000
Aug  7 15:00:57 cc-toe3 kernel: [ 1088.519230] R10: fffffffefff5032b R11: 000000000000be7b R12: 0000000000000000
Aug  7 15:00:57 cc-toe3 kernel: [ 1088.562318] R13: 0000000000000286 R14: ffff88013bc97960 R15: ffffffff815f89da
Aug  7 15:00:57 cc-toe3 kernel: [ 1088.603742] FS:  00007fd00dd40840(0000) GS:ffff88013bc80000(0000) knlGS:0000000000000000
Aug  7 15:00:57 cc-toe3 kernel: [ 1088.650891] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
Aug  7 15:00:57 cc-toe3 kernel: [ 1088.683759] CR2: 00007fd00afffbe0 CR3: 0000000228f80000 CR4: 00000000000006e0
Aug  7 15:00:57 cc-toe3 kernel: [ 1088.725558] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Aug  7 15:00:57 cc-toe3 kernel: [ 1088.768518] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Aug  7 15:00:57 cc-toe3 kernel: [ 1088.809264] Stack:
Aug  7 15:00:57 cc-toe3 kernel: [ 1088.821342]  0000000000000000 000000d000000964 ffff88023873e420 ffff88013a3cd9c0
Aug  7 15:00:57 cc-toe3 kernel: [ 1088.865345]  0000000000000000 0000000000000000 ffff88023873e420 ffffffff810942f0
Aug  7 15:00:57 cc-toe3 kernel: [ 1088.908541]  dead000000100100 dead000000200200 ffff880238cafba8 ffff88023873e420
Aug  7 15:00:57 cc-toe3 kernel: [ 1088.949848] Call Trace:
Aug  7 15:00:57 cc-toe3 kernel: [ 1088.963537]  [<ffffffff810942f0>] ? wake_up_state+0x20/0x20
Aug  7 15:00:57 cc-toe3 kernel: [ 1088.994471]  [<ffffffff810d995f>] audit_log_common_recv_msg+0x4f/0xa0
Aug  7 15:00:57 cc-toe3 kernel: [ 1089.031300]  [<ffffffff810da02d>] audit_receive_msg+0x67d/0x8c0
Aug  7 15:00:57 cc-toe3 kernel: [ 1089.063664]  [<ffffffff81289235>] ? sock_has_perm+0x75/0x90
Aug  7 15:00:57 cc-toe3 kernel: [ 1089.095969]  [<ffffffff810da2c0>] audit_receive+0x50/0xa0
Aug  7 15:00:57 cc-toe3 kernel: [ 1089.127704]  [<ffffffff8151902d>] netlink_unicast+0xdd/0x190
Aug  7 15:00:57 cc-toe3 kernel: [ 1089.160423]  [<ffffffff815193a6>] netlink_sendmsg+0x2c6/0x6c0
Aug  7 15:00:57 cc-toe3 kernel: [ 1089.193433]  [<ffffffff814d6a49>] sock_sendmsg+0x99/0xd0
Aug  7 15:00:57 cc-toe3 kernel: [ 1089.224382]  [<ffffffff8115acd3>] ? handle_pte_fault+0x93/0xa40
Aug  7 15:00:57 cc-toe3 kernel: [ 1089.258859]  [<ffffffff811832ba>] ? kmem_cache_alloc+0x1ba/0x1d0
Aug  7 15:00:57 cc-toe3 kernel: [ 1089.293964]  [<ffffffff814d6fa1>] SYSC_sendto+0x121/0x1c0
Aug  7 15:00:57 cc-toe3 kernel: [ 1089.325636]  [<ffffffff811acac5>] ? do_vfs_ioctl+0x305/0x520
Aug  7 15:00:57 cc-toe3 kernel: [ 1089.356439]  [<ffffffff8128c27e>] ? file_has_perm+0x8e/0xa0
Aug  7 15:00:57 cc-toe3 kernel: [ 1089.387870]  [<ffffffff814d798e>] SyS_sendto+0xe/0x10
Aug  7 15:00:57 cc-toe3 kernel: [ 1089.415424]  [<ffffffff81604459>] system_call_fastpath+0x16/0x1b

However with this version I do not see any of the previous messages:

Jul 16 18:08:58 hp-dl360gen8-01 systemd[1]: SELinux policy denies access.
Jul 16 18:08:58 hp-dl360gen8-01 auditd[15791]: The audit daemon is now changing the system to single user mode due to previously mentioned write error
Comment 9 Miroslav Grepl 2013-08-08 06:11:11 UTC 
And do you have systemd in DEBUG mode?

We need to see additional info for


Jul 16 18:08:58 hp-dl360gen8-01 systemd[1]: SELinux policy denies access.
Comment 10 Daniel Walsh 2013-08-09 12:28:43 UTC 
Any other AVC messages?
Comment 11 Ondrej Moriš 2013-08-21 13:53:32 UTC 
(In reply to Miroslav Grepl from comment #9)
> And do you have systemd in DEBUG mode?
> 
> We need to see additional info for
> 
> 
> Jul 16 18:08:58 hp-dl360gen8-01 systemd[1]: SELinux policy denies access.

As I mentioned in Comment #8, I do not see those message anymore. 

(In reply to Daniel Walsh from comment #10)
> Any other AVC messages?

Unfortunately, I do not see any, as I mentioned earlier, in permissive mode there are no AVC messages at all and in enforcing mode I do not see them neither.

My understanding is that the original bug had two parts, the first part (selinux one) was fixed (comment #6) but another part (kernel?) remains broken. Maybe we should consider changing the component or opening another bug.

I can invite you to my testing machine so that you see the issue in more detail if needed.
Comment 12 Miroslav Grepl 2013-08-21 14:26:48 UTC 
Ondro,
the problem is you are still getting

Jul 16 18:08:58 hp-dl360gen8-01 systemd[1]: SELinux policy denies access.
Jul 16 18:08:58 hp-dl360gen8-01 auditd[15791]: The audit daemon is now changing the system to single user mode due to previously mentioned write error

right?
Comment 13 Ondrej Moriš 2013-08-21 14:31:50 UTC 
(In reply to Miroslav Grepl from comment #12)
> Ondro,
> the problem is you are still getting
> 
> Jul 16 18:08:58 hp-dl360gen8-01 systemd[1]: SELinux policy denies access.
> Jul 16 18:08:58 hp-dl360gen8-01 auditd[15791]: The audit daemon is now
> changing the system to single user mode due to previously mentioned write
> error
> 
> right?

No, this is fixed, I do not see them anymore. I saw these in the original bug description, but it was fixed with your fixes. The problem is that the system is still crashing with a call trace mentioned in comment #8.

By the way:

Please notice that there we are simulating a disk error by selinux (changing security context of /var/log/audit/audit.log). Therefore, in permissive mode such method to invoke disk error does not work and hence everything is fine and there are no AVC messages. That is the reason why we need enforcing mode to reproduce this issue.

What about aforementioned call trace? Isn't there anything useful for locating the problem?
Comment 14 Steve Grubb 2013-08-27 13:56:25 UTC 
So...why is this bug changed to audit? The audit daemon is definitely not the problem. It issues the command to go to runlevel 1 just as if you typed it on the command line. The trace above is a kernel call trace. Someone from the kernel should look at it, but function names are missing so its probably not very useful.
Comment 15 Steve Grubb 2013-08-27 14:14:05 UTC 
Is this a new test? Did it work on RHEL6?

The reason I ask is that in discussion with eric paris, we think that what's happening is the auditd has write error, issues command for systemd, goes back to listening, sees the avc, tries to write it, perm denied, issues new command to systemd, goes back to listening, sees new avc, and around we go over and over until the system hangs. Did this work in RHEL6?
Comment 16 Eric Paris 2013-08-27 17:12:41 UTC 
kernel-3.10.0-9.el7.x86_64
systemd-206-4.el7.x86_64
audit-2.3.2-1.el7.x86_64
selinux-policy-3.12.1-70.el7.noarch

Works just fine for me.  It goes to single user mode and asks for a pword....
Comment 20 Ludek Smid 2014-06-13 09:52:46 UTC 
This request was resolved in Red Hat Enterprise Linux 7.0.

Contact your manager or support representative in case you have further questions about the request.
Note You need to log in before you can comment on or make changes to this bug.