Bug 2048462

Hi Benoit,

the include and exclude rules are quite complicated and I can not yet call myself an expert on this area of the code. That said, I am afraid that BACKUP_PROG_EXCLUDE won't do what you intend as it only excludes paths from the backup generation (tar archive, rsync copy, ...) and not from the layout generation. I would use EXCLUDE_MOUNTPOINTS to achieve what you want. Note that the doc that you have quoted is directly above the EXCLUDE_* rules in /usr/share/rear/conf/default.conf, not above the BACKUP_PROG_EXCLUDE rules, so it must refer to the EXCLUDE rules. Also I suspect that '/mnt/*'  would not do what you want anyway, because it excludes everything under /mnt and not /mnt itself. Home that helps, if not, I can investigate further.

Concerning "a device should be excluded if all the content of it is excluded, right?", according to the comment in default.conf, it is the contrary. The mountpoints specified under EXCLUDE_MOUNTPOINTS "[w]ill be automatically added to the $BACKUP_PROG_EXCLUDE array during backup to prevent the excluded filesystems' data to be backed up", but there is nothing that says that it will be added in the opposite direction. So a device being excluded means that all its content is excluded, but not vice-versa, if I understand the doc correctly.

(Yes, the comments in default.conf are actually the documentation of the various settings, usually the best and often the only documentation that exists. It is maybe not ideal that comments in the config file serve as documentation, but on the other hand I have found these comments quite detailed and complete.)

Hi Pavel,

Thanks for the comment.
I agree with EXCLUDE_MOUTPOINTS: it will exclude everything form the backup. 
But you are saying "So a device being excluded means that all its content is excluded, but not vice-versa, if I understand the doc correctly."
The issue is that a device CAN'T be explicitly excluded:
---
# You cannot exclude a device (e.g. /dev/sdg) directly. Instead you have to exclude everything 
# ON that device and then the dependency tracker will automatically exclude the device from the
# recovery (because there won't be any recovery information for that "unnecessary" device)."
---

My point is why if I exclude everything from a PATH (which is a mount path), the corresponding device is not excluded (as it should be from the point above).

I tried with /mnt2 to exclude some specifics with /mnt but with the same result.

I'm still trying to understand how the tracker is working ...

Benoit

I think that the comment "you have to exclude everything ON that device" is ambiguous and confusing and the source of your problems, because it can be read to mean two different things: files on that device (you exclude them using BACKUP_PROG_EXCLUDE) and storage objects on that device, like filesystems and LVM stuff (you exclude them using the EXCLUDE_* variables). In reality it means the latter, but you have understood it as meaning the former. The problem is, "everything on that device" could refer either to files inside a filesystem, or to the filesystem itself and the underlying storage stuff, and is not clear which one is meant. Try adding
EXCLUDE_MOUNTPOINTS+=( '/mnt' )
and I think it will be fine. If so, I would call it a documentation issue.

Here's the difference in disklayout before and after I do that (vdb1 is mounted on /mnt):

diff -ur inclayout/disklayout.conf exclayout/disklayout.conf
--- inclayout/disklayout.conf   2022-01-31 13:50:31.141253489 -0500
+++ exclayout/disklayout.conf   2022-01-31 13:50:54.375253489 -0500
@@ -1,4 +1,4 @@
-# Disk layout dated 20220131135030 (YYYYmmddHHMMSS)
+# Disk layout dated 20220131135053 (YYYYmmddHHMMSS)
 # NAME        KNAME     PKNAME   TRAN TYPE FSTYPE SIZE MOUNTPOINT
 # /dev/vda    /dev/vda                disk         20G 
 # `-/dev/vda1 /dev/vda1 /dev/vda      part xfs     20G /
@@ -12,13 +12,13 @@
 part /dev/vda 21473771008 1048576 primary boot /dev/vda1
 # Disk /dev/vdb
 # Format: disk <devname> <size(bytes)> <partition label type>
-disk /dev/vdb 10737418240 msdos
+#disk /dev/vdb 10737418240 msdos
 # Partitions on /dev/vdb
 # Format: part <device> <partition size(bytes)> <partition start(bytes)> <partition type|name> <flags> /dev/<partition>
-part /dev/vdb 10736369664 1048576 primary none /dev/vdb1
+#part /dev/vdb 10736369664 1048576 primary none /dev/vdb1
 # Filesystems (only ext2,ext3,ext4,vfat,xfs,reiserfs,btrfs are supported).
 # Format: fs <device> <mountpoint> <fstype> [uuid=<uuid>] [label=<label>] [<attributes>]
 fs /dev/vda1 / xfs uuid=507725cd-1cf2-496e-ac97-dd0d9324ae53 label=  options=rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota
-fs /dev/vdb1 /mnt ext2 uuid=9dcb0185-98b0-46a1-8be1-ce0dea426a2e label= blocksize=4096 reserved_blocks=4% max_mounts=-1 check_interval=0d bytes_per_inode=16382 default_mount_options=user_xattr,acl options=rw,relatime,seclabel
+#fs /dev/vdb1 /mnt ext2 uuid=9dcb0185-98b0-46a1-8be1-ce0dea426a2e label= blocksize=4096 reserved_blocks=4% max_mounts=-1 check_interval=0d bytes_per_inode=16382 default_mount_options=user_xattr,acl options=rw,relatime,seclabel
 # Swap partitions or swap files
 # Format: swap <filename> uuid=<uuid> label=<label>

by the way by "So a device being excluded means that all its content is excluded" I meant a storage object (in this case, filesystem) being excluded, not exactly a device. While you can't exclude block devices (at least not disks), you can exclude filesystems on them (referred to by their mountpoints).

Thanks Pavel,

With EXCLUDE_MOUNTPOINTS, it effectively works.

But what is more confusing, is that cust has ONLY_INCLUDE_VG=(rootvg) and:
----
# Only include LVM2 volume groups - the opposite of EXCLUDE_VG (handy if you only want vg00 to be included)
# EXCLUDE_VG and EXCLUDE_MOUNTPOINTS will get populated automatically, if needed
# syntax : e.g. ONLY_INCLUDE_VG=( "vg00" "vg01" )
ONLY_INCLUDE_VG=()
---

=> EXCLUDE_MOUNTPOINTS will get populated automatically
But I think only mount points of other VG...

Will inform cust to use explicitly EXCLUDE_MOUNTPOINTS

Thanks again !

Benoit

Regarding ONLY_INCLUDE_VG and mountpoint exclusion, beware, it seems that it does not work properly in all cases, see bz1848927 - does it really populate EXCLUDE_MOUNTPOINTS of the excluded VG for the customer in their case (I don't really understand what was confusing, the documentation, or the actual behavior)? We can reopen that bug if needed.

Hello, 

My issue is different I think:

with:
ONLY_INCLUDE_VG=('rhel')

and vdb not in a VG:
---
sr0            11:0    1 1024M  0 rom  
vda           252:0    0   20G  0 disk 
|-vda1        252:1    0  600M  0 part /boot/efi
|-vda2        252:2    0    1G  0 part /boot
`-vda3        252:3    0 18.4G  0 part 
  |-rhel-root 253:0    0 16.4G  0 lvm  /
  `-rhel-swap 253:1    0    2G  0 lvm  [SWAP]
vdb           252:16   0  512M  0 disk /mnt2
---

=> vdb is not in a VG, and then is not excluded (disk /dev/vdb 536870912 loop) (not in opposite EXCLUDE_VG ...)

What is weird: I made a mistake with the name of the VG in my first attempt, and everything was excluded ... including vdb ...

(In reply to Welterlen Benoit from comment #8)
> Hello, 
> 
> My issue is different I think:
> 
> with:
> ONLY_INCLUDE_VG=('rhel')
> 
> and vdb not in a VG:
> ---
> sr0            11:0    1 1024M  0 rom  
> vda           252:0    0   20G  0 disk 
> |-vda1        252:1    0  600M  0 part /boot/efi
> |-vda2        252:2    0    1G  0 part /boot
> `-vda3        252:3    0 18.4G  0 part 
>   |-rhel-root 253:0    0 16.4G  0 lvm  /
>   `-rhel-swap 253:1    0    2G  0 lvm  [SWAP]
> vdb           252:16   0  512M  0 disk /mnt2
> ---
> 
> => vdb is not in a VG, and then is not excluded (disk /dev/vdb 536870912
> loop) (not in opposite EXCLUDE_VG ...)

I believe that ONLY_INCLUDE_VG means "exclude all VGs except this", not "exclude everything except this". May seem weird, but actually it is a good thing. Otherwise it would have excluded even your /boot and EFI System Partition.

> What is weird: I made a mistake with the name of the VG in my first attempt,
> and everything was excluded ... including vdb ...

Maybe because the mountpoint for vdb is in one of the VG's filesystems, and if this gets excluded, all mounts with mountpoints there get excluded as well? (There is a dependency solver in the layout code.)

Thanks Pavel for the details.
We really have to face issues with rear to understand the options :D

I think the BZ can be closed.

Thanks for your work !

Best regards

Benoit

(In reply to Welterlen Benoit from comment #10)
> Thanks Pavel for the details.
> We really have to face issues with rear to understand the options :D

the include / exclude rules are especially complicated. That's why I would hate to add even more such options, like https://github.com/rear/rear/pull/2597 / bz1946911.

> I think the BZ can be closed.

Don't you want to turn it into a doc issue? (Not sure how exactly this works.)

> Thanks for your work !

you are very welcome! I am glad to have had an opportunity to become more familiar with the include / exclude rules.

Regards, Pavel

Issue migration from Bugzilla to Jira is in process at this time. This will be the last message in Jira copied from the Bugzilla bug.

This BZ has been automatically migrated to the issues.redhat.com Red Hat Issue Tracker. All future work related to this report will be managed there.

Due to differences in account names between systems, some fields were not replicated.  Be sure to add yourself to Jira issue's "Watchers" field to continue receiving updates and add others to the "Need Info From" field to continue requesting information.

To find the migrated issue, look in the "Links" section for a direct link to the new issue location. The issue key will have an icon of 2 footprints next to it, and begin with "RHEL-" followed by an integer.  You can also find this issue by visiting https://issues.redhat.com/issues/?jql= and searching the "Bugzilla Bug" field for this BZ's number, e.g. a search like:

"Bugzilla Bug" = 1234567

In the event you have trouble locating or viewing this issue, you can file an issue by sending mail to rh-issues. You can also visit https://access.redhat.com/articles/7032570 for general account information.