Bug 1295088

Summary: Inconsistencies regarding i/o scheduler, shmmax, vmstat output
Product: Red Hat Enterprise Linux 7 Reporter: giulioo
Component: doc-Performance_Tuning_GuideAssignee: Milan Navratil <mnavrati>
Status: CLOSED CURRENTRELEASE QA Contact: ecs-bugs
Severity: unspecified Docs Contact:
Priority: high    
Version: 7.2CC: jbainbri, jmoyer, lwoodman, mnavrati, rhel-docs
Target Milestone: rcKeywords: Documentation
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-02-01 13:34:51 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description giulioo 2016-01-02 08:52:21 UTC
1)
   Paragraph 1.2
   "..deadline replaces cfq as the default I/O scheduler in Red Hat Enterprise Linux 7. "

   Paragraph 5.1.2
   "..deadline
    The default I/O scheduler for all block devices except SATA disks"

Which is correct?  1.2 which says "deadline" is default, or 5.1.2 which says SATA disks are an exception?


2)
  Paragraph 4.3.2.3

  "shmmax
    Defines the maximum size of a single shared memory segment allowed by the kernel, in pages"

According to https://access.redhat.com/solutions/1437883, shmmax is expressed in bytes, not in pages. 
BTW, https://access.redhat.com/solutions/1437883 is confusing too, since it says
         # Controls the maximum number of shared memory segments, in pages
         kernel.shmall = 268435456
but shmall is not the "maximum number of shared segments", that would be SHMMNI


3)
   Paragraph 5.2.1
    "si    Swap in, or writes to swap space, in KB. "
⁠    "so    Swap out, or reads from swap space, in KB. "
    "⁠bi    Block in, or block write operations, in KB. "
    "⁠bo    Block out, or block read operations, in KB. "


   However, the man page for vmstat states:
     Swap
         si: Amount of memory swapped in from disk (/s).
         so: Amount of memory swapped to disk (/s).
     IO
         bi: Blocks received from a block device (blocks/s).
         bo: Blocks sent to a block device (blocks/s).


It seems to me the "tuning guide" is saying the opposite of the man page:
- the tuning guide says 
     "in"  means "to disk/to swap space"
     "out" means "from disk/from swap space"
- the man page says
     "in"  means "from disk/from swap space"
     "out" means "to disk/to swap space"
I think the man page is correct.

Comment 7 Milan Navratil 2016-02-08 22:14:55 UTC
(In reply to giulioo from comment #0)

Thank you for taking the time to report this bug against the Performance Tuning Guide. I'm with the documentation team, and I've made the following changes after verifying with SMEs.


> 1)
>    Paragraph 1.2
>    "..deadline replaces cfq as the default I/O scheduler in Red Hat
> Enterprise Linux 7. "
> 
>    Paragraph 5.1.2
>    "..deadline
>     The default I/O scheduler for all block devices except SATA disks"
> 
> Which is correct?  1.2 which says "deadline" is default, or 5.1.2 which says
> SATA disks are an exception?

Paragraph 1.2 now reads: 
-- deadline replaces cfq as the default I/O scheduler for all block devices, except for SATA disks, in Red Hat Enterprise Linux 7. 

> 
> 
> 2)
>   Paragraph 4.3.2.3
> 
>   "shmmax
>     Defines the maximum size of a single shared memory segment allowed by
> the kernel, in pages"
> 
> According to https://access.redhat.com/solutions/1437883, shmmax is
> expressed in bytes, not in pages. 
> BTW, https://access.redhat.com/solutions/1437883 is confusing too, since it
> says
>          # Controls the maximum number of shared memory segments, in pages
>          kernel.shmall = 268435456
> but shmall is not the "maximum number of shared segments", that would be
> SHMMNI
> 

I've updated the shmmax entry based on another Bugzilla; now, the entry says this: 

⁠shmmax
    Defines the maximum size (in bytes) of a single shared memory segment allowed by the kernel. Starting with Red Hat Enterprise Linux 7.1, the default value for shmmax set by the kernel at boot time is 18446744073692774399 KB. In Red Hat Enterprise Linux 7.1, this value was overwritten by the kernel.shmmax sysctl parameter set in the /usr/lib/sysctl.d/00-system.conf file. In Red Hat Enterprise Linux 7.2 and later, parameters overriding the kernel default have been removed, so the system-wide default value is set to 18446744073692774399 KB. To change the system-wide default value, set kernel.shmmax in /etc/sysctl.d/01-shm.conf to the intended value. 

> 
> 3)
>    Paragraph 5.2.1
>     "si    Swap in, or writes to swap space, in KB. "
> ⁠    "so    Swap out, or reads from swap space, in KB. "
>     "⁠bi    Block in, or block write operations, in KB. "
>     "⁠bo    Block out, or block read operations, in KB. "
> 
> 
>    However, the man page for vmstat states:
>      Swap
>          si: Amount of memory swapped in from disk (/s).
>          so: Amount of memory swapped to disk (/s).
>      IO
>          bi: Blocks received from a block device (blocks/s).
>          bo: Blocks sent to a block device (blocks/s).
> 
> 
> It seems to me the "tuning guide" is saying the opposite of the man page:
> - the tuning guide says 
>      "in"  means "to disk/to swap space"
>      "out" means "from disk/from swap space"
> - the man page says
>      "in"  means "from disk/from swap space"
>      "out" means "to disk/to swap space"
> I think the man page is correct.

The guide now says:

si
    Swap in, or reads from swap space, in KB. 
⁠so
    Swap out, or writes to swap space, in KB. 

As soon as the book is republished and available on the Red Hat Customer Portal, this bug will be closed.

Thanks again for filing this bug and helping as improve our documentation.

Comment 8 giulioo 2016-02-09 15:58:18 UTC
>⁠shmmax
>    Defines the maximum size (in bytes) of a single shared memory segment >allowed by the kernel. Starting with Red Hat Enterprise Linux 7.1, the default >value for shmmax set by the kernel at boot time is 18446744073692774399 KB.

RHEL7.2:
$ cat /proc/sys/kernel/shmmax
18446744073692774399
$

If /proc/sys/kernel/shmmax is in bytes, why will the new docs say "KB" instead of bytes:
    "... time is 18446744073692774399 KB."
?

Comment 9 Larry Woodman 2016-02-12 16:30:19 UTC

They are in bytes:


/usr/include/linux/shm.h

/*
 * SHMMAX, SHMMNI and SHMALL are upper limits are defaults which can
 * be increased by sysctl
 */

#define SHMMAX 0x2000000                 /* max shared seg size (bytes) */
#define SHMMIN 1                         /* min shared seg size (bytes) */
#define SHMMNI 4096                      /* max num of segs system wide */
#define SHMALL (SHMMAX/getpagesize()*(SHMMNI/16))
#define SHMSEG SHMMNI                    /* max shared segs per process */

Comment 10 Larry Woodman 2016-02-12 16:41:43 UTC
But SHMALL is in pages:

static int newseg(struct ipc_namespace *ns, struct ipc_params *params)
{
        key_t key = params->key;
        int shmflg = params->flg;
        size_t size = params->u.size;
        int error;
        struct shmid_kernel *shp;
        size_t numpages = (size + PAGE_SIZE - 1) >> PAGE_SHIFT;
        struct file * file;
        char name[13];
        int id;
        vm_flags_t acctflag = 0;

        if (size < SHMMIN || size > ns->shm_ctlmax)
                return -EINVAL;

        if (ns->shm_tot + numpages > ns->shm_ctlall)
                return -ENOSPC;