Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1381219

Summary: Remove udev rule for deadline elevator (from host deploy) - not needed in EL7 hosts.
Product: Red Hat Enterprise Virtualization Manager Reporter: Julio Entrena Perez <jentrena>
Component: ovirt-host-deployAssignee: Yedidyah Bar David <didi>
Status: CLOSED ERRATA QA Contact: Petr Kubica <pkubica>
Severity: medium Docs Contact:
Priority: medium    
Version: 3.6.9CC: bazulay, dougsland, fcami, jentrena, lsurette, nsoffer, pbrilla, pdwyer, rbalakri, Rhev-m-bugs, srevivo, tnisan, trichard, ykaul, ylavi
Target Milestone: ovirt-4.1.1Flags: gklein: testing_plan_complete-
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Previously, host-deploy added a udev rule to change the default I/O scheduler to 'deadline', because the then-default scheduler was not adequate. This rule used 'RUN' to run a command to set the default. Some storage vendors have recommendations to set other specific schedulers for their devices, some of which use 'ATTR', a sysfs attribute. However, udevadm scans all the files in /etc/udev/rules.d, then first sets all the ATTR configurations, and then runs the RUN commands, making it impossible to follow the storage vendors' recommendations successfully, as our rule would override them. Now, with RHEL 7, the default is 'deadline', so the udev rule is not needed anymore. host-deploy was changed to not add this rule, and to remove the conf file it added in the past, if found. It is now possible to successfully set I/O schedulers as needed using ATTR.
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-04-25 00:41:54 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Integration RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Julio Entrena Perez 2016-10-03 12:55:28 UTC
Description of problem:
File /usr/share/ovirt-host-deploy/plugins/ovirt-host-deploy/tune/ovirt-iosched.rules contains a udev rule used to switch the disk scheduler to deadline.

The rule does so using a "RUN+=" command:

ACTION=="add|change", SUBSYSTEM=="block", ENV{DEVTYPE}=="disk", RUN+="/bin/sh -c 'echo deadline > /sys/${DEVPATH}/queue/scheduler'"

We are requesting the rule to do so using the sysfs attribute instead:

ACTION=="add|change", SUBSYSTEM=="block", ENV{DEVTYPE}=="disk", ATTR{queue/scheduler}="noop"

The reason for this request is twofold:
- it's probably cheaper to request udev to set the sysfs attribute directly than it is to run a shell command.
- rules supplied by hardware vendors already use sysfs attributes, which introduces the problem described below.


Some vendors recommend different settings, for example:

  https://support.purestorage.com/Solutions/Operating_Systems/Linux/Linux_Recommended_Settings

  ACTION=="add|change", KERNEL=="sd*[!0-9]", SUBSYSTEM=="block", ENV{ID_VENDOR}=="PURE", ATTR{queue/scheduler}="noop"

Taking the above example, creating a "99-*.rules" file with the rule above should override the rule included in file "ovirt-iosched.rules" which ends up deployed in hosts as "12-ovirt-iosched.rules".

BUT while udev honours the numeric order determined by the filenames, udev seems to apply sysfs attributes before running commands.
As a result, the command RUN+="/bin/sh -c 'echo deadline > /sys/${DEVPATH}/queue/scheduler'" in file "12-ovirt-iosched.rules" is executed AFTER the sysfs ATTR{queue/scheduler}="noop" in file "99-*.rules":

# udevadm --debug test /block/sdi 2>&1 | grep scheduler
udev_rules_apply_to_event: RUN '/bin/sh -c 'echo deadline > /sys/${DEVPATH}/queue/scheduler'' /etc/udev/rules.d/12-ovirt-iosched.rules:5
udev_rules_apply_to_event: ATTR '/sys/devices/platform/host6/session1/target6:0:0/6:0:0:7/block/sdi/queue/scheduler' writing 'noop' /etc/udev/rules.d/99-pure-storage.rules:1
udevadm_test: run: '/bin/sh -c 'echo deadline > /sys/${DEVPATH}/queue/scheduler''


Version-Release number of selected component (if applicable):
ovirt-host-deploy-1.4.1-1.el6ev

How reproducible:
Always

Steps to Reproduce:
1. Create a udev rule file which sets the disk elevator to a value other than 'deadline'
2. Ensure that the rule in the new file uses ATTR{queue/scheduler}= to set the sysfs attribute.
3. Ensure that the number in the new rule file is higher than 12 to allow it to override "12-ovirt-iosched.rules".

Actual results:
'deadline' disk scheduler is applied to all devices, including those matched by the new custom rule file.
udev runs commands after sysfs attribute changes.

Expected results:
custom scheduler is applied to devices matched by the new rule file.
"12-ovirt-iosched.rules" uses sysfs attribute to set all devices to 'deadline' disk scheduler, thus allowing other rules that use sysfs attribute to override the disk scheduler as per the comments in the file:

"# Settings may be overridden by applying subsequent rules."

Comment 1 Julio Entrena Perez 2016-10-03 12:56:35 UTC
> We are requesting the rule to do so using the sysfs attribute instead:

> ACTION=="add|change", SUBSYSTEM=="block", ENV{DEVTYPE}=="disk", ATTR{queue/scheduler}="noop"

Sorry for the typo, the rule should read:

ACTION=="add|change", SUBSYSTEM=="block", ENV{DEVTYPE}=="disk", ATTR{queue/scheduler}="deadline"

Comment 2 Yaniv Kaul 2016-10-03 14:31:52 UTC
I think this is confusing two requests - change to sysfs; be able to configure th3 scheduler (and specifically, to noop if using an all flash array)

Comment 3 Julio Entrena Perez 2016-10-03 14:34:31 UTC
Sorry Yaniv, that was a typo (see comment 1).

Current rule is:

ACTION=="add|change", SUBSYSTEM=="block", ENV{DEVTYPE}=="disk", RUN+="/bin/sh -c 'echo deadline > /sys/${DEVPATH}/queue/scheduler'"

This request is for the rule to be:

ACTION=="add|change", SUBSYSTEM=="block", ENV{DEVTYPE}=="disk", ATTR{queue/scheduler}="deadline"

Nothing else is being requested, sorry for the confusion.

Comment 4 Yaniv Kaul 2016-10-03 19:47:55 UTC
(In reply to Julio Entrena Perez from comment #3)
> Sorry Yaniv, that was a typo (see comment 1).
> 
> Current rule is:
> 
> ACTION=="add|change", SUBSYSTEM=="block", ENV{DEVTYPE}=="disk",
> RUN+="/bin/sh -c 'echo deadline > /sys/${DEVPATH}/queue/scheduler'"
> 
> This request is for the rule to be:
> 
> ACTION=="add|change", SUBSYSTEM=="block", ENV{DEVTYPE}=="disk",
> ATTR{queue/scheduler}="deadline"
> 
> Nothing else is being requested, sorry for the confusion.

Why would you use a deadline scheduler against PURE? Or any other all-flash-aray?

Comment 5 Julio Entrena Perez 2016-10-04 08:33:08 UTC
(In reply to Yaniv Kaul from comment #4)

> Why would you use a deadline scheduler against PURE? Or any other
> all-flash-aray?

They don't.

They want to be able to override the scheduler to "noop" using the udev rule documented by the vendor at https://support.purestorage.com/Solutions/Operating_Systems/Linux/Linux_Recommended_Settings :

ACTION=="add|change", KERNEL=="sd*[!0-9]", SUBSYSTEM=="block", ENV{ID_VENDOR}=="PURE", ATTR{queue/scheduler}="noop"

Unfortunately, that rule does not override ovirt's rule unless ovirt's rule uses sysfs attribute ATTR{queue/scheduler}="deadline".
If the ovirt rule uses a RUN+= command then the rule documented by the vendor does not override ovirt's rule.

Comment 6 Sandro Bonazzola 2016-10-04 11:05:22 UTC
Moving to storage team for review.

Comment 13 Yaniv Kaul 2017-02-12 11:12:10 UTC
Can't we just drop this file altogether? Why are we even trying to change anything? From https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html/Performance_Tuning_Guide/chap-Red_Hat_Enterprise_Linux-Performance_Tuning_Guide-Storage_and_File_Systems.html#sect-Red_Hat_Enterprise_Linux-Performance_Tuning_Guide-Considerations-IO_Schedulers 
:
"The default I/O scheduler for all block devices, except for SATA disks."

Comment 14 Julio Entrena Perez 2017-02-12 11:41:13 UTC
This has changed in RHEL 7.
I've tested commenting out the line in file 12-ovirt-iosched.rules and after rebooting the host it still uses deadline for the storage domain disks, so for RHEL 7 that's not needed any more.

Comment 15 Yaniv Lavi 2017-02-12 15:22:26 UTC
See comment #14.

Comment 16 Nir Soffer 2017-02-12 15:53:59 UTC
This indeed seems useless on rhel 7. However this file is installed by host-deploy
so it should be handled by host-deploy maintainers.

Comment 18 Petr Kubica 2017-04-12 07:38:47 UTC
Verified in rhv-4.1.1-7