RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1460077 - Executing db_hotbackup as user root instead of the user owning the db-files causes issues
Summary: Executing db_hotbackup as user root instead of the user owning the db-files c...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: libdb
Version: 7.4
Hardware: All
OS: Linux
unspecified
medium
Target Milestone: rc
: ---
Assignee: Matej Mužila
QA Contact: Vaclav Danek
Lenka Špačková
URL:
Whiteboard:
Depends On:
Blocks: 1420851 1465891 1466363
TreeView+ depends on / blocked
 
Reported: 2017-06-09 02:40 UTC by Christian Horn
Modified: 2020-12-14 08:50 UTC (History)
7 users (show)

Fixed In Version:
Doc Type: Known Issue
Doc Text:
"db_hotbackup -c" should be used with caution The "db_hotbackup" command with the "-c" option must be run by the user that owns the database. If the user is different and the log file reaches its maximal size, a new log file is created with an ownership of the user that ran the command, which consequently makes the database unusable for its owner. This note has been added to the `db_hotbackup(1)` manual page.
Clone Of:
Environment:
Last Closed: 2018-04-10 17:44:41 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Knowledge Base (Solution) 28918 0 None None None 2017-06-09 02:42:23 UTC
Red Hat Product Errata RHBA-2018:0952 0 None None None 2018-04-10 17:45:24 UTC

Description Christian Horn 2017-06-09 02:40:18 UTC
Description of problem:
openldap uses libdb as database backend.  When writes are done on the openldap (resulting in writes on the underlying libdb files), and db_hotbackup is executed as root instead of the user owning the db-files, this can result in db-files beeing owned by the root-user, and as a result the ldap-service going down.
This is not intuitive for users, they run db_hotbackup as root and then face corruptions.

Version-Release number of selected component (if applicable):
current

How reproducible:
always, after some minutes

Steps to Reproduce:
1. create a simple openldap setup
2. perform write operations, for example in a loop create/remove an ldap object
3. at the same time run db_hotbackup in a loop, as root user

Actual results:
After some time
db_hotbackup: BDB0060 PANIC: fatal region error detected; run recovery
db_hotbackup: DB_ENV->log_archive: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery
db_hotbackup: BDB1581 File handles still open at environment close
db_hotbackup: BDB1582 Open file handle: /var/lib/ldap/__db.001
db_hotbackup: BDB1582 Open file handle: /var/lib/ldap/__db.002
db_hotbackup: BDB1582 Open file handle: /var/lib/ldap/__db.003
db_hotbackup: BDB1582 Open file handle: /var/lib/ldap/./log.0000000130
db_hotbackup: BDB0060 PANIC: fatal region error detected; run recovery
db_hotbackup: dbenv->close: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery
db_hotbackup: BDB5043 HOT BACKUP FAILED!

Expected results:
I suggest that db_hotbackup verifies if it gets started as the user who owns the db-files.  

Additional info:

Comment 5 Christian Horn 2017-08-16 00:32:21 UTC
Hi,

(In reply to Matej Mužila from comment #4)
> Workaround to this problem is to run db_hotbackup not by user root but by
> user ldap.
Thank you, we in support are aware, and this is also documented in the linked kbase article.  
This is however not preventing the customer though to run the backup with a different user.  This bz is to get more bulletproof tools and lower chances that people have to experience the issue first while running as incorrect user, risking loss of data and waste of time, before they understand that they use the incorrect user.  Checking this in the tool should be simple, and the gains are huge.

Comment 8 Christian Horn 2017-09-05 02:03:32 UTC
(In reply to Matej Mužila from comment #7)
> I'm afraid that there is no bulletproof way to get the owner of database.
> For example when the database is not initialized, there are no files that
> could be used to determine owner of the database.
Yes, we are not worried about that case here as the files do not exist.  Whoever executes the backup then and has write permissions can write the first backup (excluding unlikely things like disk full etc.).
 
> Even if there are database files present, determining owner of the database
> from them is a bit tricky. 
Do you mean the database files?  If the filesystem is aware of ownerships, we should be able to see who owns the file?

> There may exist a situation, where database files
> are not owned by user that runs the database.
To my understanding, the requirement is that the db_hotbackup is executed as the same user who owns the files, correct?
If that is the case, shouldn't we just need 
a) the user-id under which db_hotbackup is started
b) the owner of the existing files
and then compare these?

> Determining the right user from a previous log file is insufficient, because
> previous log file may be nonexistent.
Yes, but I think it's ok to not consider that situation.

Comment 11 Christian Horn 2017-09-22 00:43:43 UTC
(In reply to Matej Mužila from comment #10)
> (In reply to Christian Horn from comment #8)
> > > Even if there are database files present, determining owner of
> > > the database
> > > from them is a bit tricky. 
> > Do you mean the database files?  If the filesystem is aware of ownerships,
> > we should be able to see who owns the file?
> 
> Yes, we are able to find out who owns those files, but imagine a situation
> when files are not owned by the user that runs database. They may be owned
> by different user and have set read/write permissions to a group or anybody.
> This is obviously not the case of ldap, however ldap is not the only thing
> that uses libdb.

I think you refer to the situation which we want to make more bulletproof with this bz.  Saying it in pseudo-code:

if (no databasefiles do yet exist) {
  just write the files with the permissions of the user
  executing the db_hotbackup script
}
else {
  if ( (user who started db_hotbackup) == (user who owns existing files) ) {
    just do the backup, permissions seem to be in order.
  }
  else {
    do some appropriate(tm) action.
    - For example, if started as root, write files as user who owns
      existing files.
    - or atleast output a big fat warning message if this situation
      is detected
    optional: blindly try to write the backup files anyway,
    this is the action we currently undertake.
  }
}


> > To my understanding, the requirement is that the db_hotbackup is executed as
> > the same user who owns the files, correct?
> > If that is the case, shouldn't we just need 
> > a) the user-id under which db_hotbackup is started
> > b) the owner of the existing files
> > and then compare these?
> 
> db_hotbackup should be ran by the same user, that runs database (in this
> case user ldap).

Agreed.  The request in this bz is to better deal with the situation
when that is not the case.  Even outputting "this looks fishy, the 
user who started the script does not own existing backup files" would
help.


> > > Determining the right user from a previous log file is
> > > insufficient, because
> > > previous log file may be nonexistent.
> > Yes, but I think it's ok to not consider that situation.
> 
> I don't think so. This situation is real and may occur. I think that making
> db_hotbackup to "guess" the right user is much worse than not doing so, if
> it is impossible to do so in all cases. It would make usage of db_hotbackup
> even less predictable and tricky than it is now.

I have not understood the situation here referred to well enough to comment, I can not imagine a situation where we have to guess.  Getting less predictable is not good of course.  How about just trying to detect the situation then and additionally to what we currently do (simply trying to write the files) additionally outputting a warning?

Comment 13 Christian Horn 2017-09-28 00:15:10 UTC
(In reply to Matej Mužila from comment #12)
> We may print warning when we detect potentially dangerous situation (user
> different to user owning files). Most of dangerous situations would be
> detected (and announced to admin) and behavior of db_hotbackup would not
> change.
> 
> What do you think about it?

Great, lets do that.

Comment 17 Christian Horn 2017-11-16 01:40:38 UTC
(In reply to Matej Mužila from comment #16)
> We think, it would be better to mention in man page, that if db_hotbackup
> is run with -c parameter, it is necessary to run it by the user that owns
> databse files.
> 
> The other thing is that the problem occurs quite rarely (1 of 40 000 cases)
> and it is not caused by the program itsefl, but by running the program
> incorrectly.
> 
> Do you think it is OK?

Customer will continue to run into this, and just then when having run into the issue investigate and hit the error description - so warning would be beneficial over just documenting, IMHO.
We have to take only documentation if we are not getting the warning, though.  We have this in kbase for customers, having it in the manpage will document it for all users.

Comment 30 errata-xmlrpc 2018-04-10 17:44:41 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:0952


Note You need to log in before you can comment on or make changes to this bug.