I have a request, can we assume that there will be something in the cpdb.log file appended we can check in our pre-upgrade tool to verify users finished this check step?

In pre-6.2.14 world, we will ask them to run this:

liquibase --classpath=/usr/share/java/postgresql-jdbc.jar:./cpvalidation --changeLogFile=./cpvalidation/changelog-validate.xml --driver=org.postgresql.Driver --url=jdbc:postgresql:candlepin --username=candlepin --password=$(grep org.quartz.dataSource.myDS.password /etc/candlepin/candlepin.conf | cut -d= -f2) --logLevel=debug migrate 2>&1 | tee -a /var/log/candlepin/cpdb.log

This is leaving:

DEBUG 12/4/17 4:07 AM:liquibase: ./cpvalidation/changelog-validate.xml:

DEBUG 12/4/17 4:07 AM:liquibase: ./cpvalidation/changelog-validate.xml:

which we can grep at the later stage. Can you please make sure this is left when running cpdb? Perhaps by increasing verbosity, or making new verbosity option or adding an extra INFO line because by default there is just "Liquibase Update Successful" which is not enough, thanks.

Please disregard comment 2, this is not relevant anymore. But we'd appreciate increased verbosity for sure, if that helps you to debug possible issues during calls.

Last thing - we do assume that cpdb returns non-zero shell code when validation fails. When looking into 6.2.13 codebase, I don't see explicit exit calls there, just want to make sure. Thanks.

Additional logging changes were also made as part of this patch. By default, cpdb now executes the liquibase command with --logLevel=SEVERE (up from none); and if you want all of the logging as requested above, cpdb now has a --verbose flag, which changes the liquibase command's log level to DEBUG.

When the liquibase command returns non-zero, cpdb propagates an exception up, causing it to also return a non-zero (but not necessarily the same) error code. With the way the validation bits are written at present, any validation failure causes liquibase to fail in this manner.

Ok please make sure the "--verbose" flag is in 6.2.x series so we can call it. Thanks.

Build: Satellite 6.2.14 snap1

rpm -q candlepin
candlepin-0.9.54.26-1.el7.noarch

[root@ibm-x3650-01 candlepin]# ./cpdb --help
Usage: cpdb [options]

Options:
  -h, --help            show this help message and exit
  --create              create the Candlepin database; cannot be used with
                        --update or --validate
  --schema-only         assumes database is already created by another tool
                        andapplies schema to database; used with --create
  --validate            validate the current state of the Candlepin database;
                        cannot be used with --create
  --update              update the Candlepin database; cannot be used with
                        --create
  --drop                drop the existing Candlepin database before creating
  -u DBUSER, --user=DBUSER
                        database user to use
  -d DB, --database=DB  database name to use
  -p DBPASSWORD, --password=DBPASSWORD
                        database password to use
  --dbhost=DBHOST       the database host to use (optional)
  --dbport=DBPORT       the database port to use (optional)
  --community           true if used in a community fashion
  --oracle              run against an Oracle installation
  --oracle-user=ORACLE_USER
                        Oracle DBA user. Defaults to 'sys'
  --oracle-password=ORACLE_PASSWORD
                        Oracle DBA password
  --verbose             enables verbose logging/output


Using verbose increases the log-level on liquibase to debug

[root@ibm-x3650-01 candlepin]# sudo -u postgres -H -- psql -d candlepin -c "update cp_pool set productid='Ohno' where id='0000000000000000000000000000000';"
UPDATE 1
[root@ibm-x3650-01 candlepin]# ./cpdb --validate -u candlepin -p 1111111111111111111111111111111 -d candlepin --verboseValidating Candlepin database
--driver=org.postgresql.Driver --classpath=/usr/share/java/postgresql-jdbc.jar:/var/lib/tomcat/webapps/candlepin/WEB-INF/classes/ --changeLogFile=db/changelog/changelog-validate.xml --url=jdbc:postgresql:candlepin --username=candlepin --password=1111111111111111111111111111111 --logLevel=debug

########## ERROR ############
Error running command: liquibase --driver=org.postgresql.Driver --classpath=/usr/share/java/postgresql-jdbc.jar:/var/lib/tomcat/webapps/candlepin/WEB-INF/classes/ --changeLogFile=db/changelog/changelog-validate.xml --url=jdbc:postgresql:candlepin --username=candlepin --password=1111111111111111111111111111111 --logLevel=debug migrate -Dcommunity=False
Status code: 65280
Command output: DEBUG 1/15/18 8:13 AM:liquibase: Connected to candlepin@jdbc:postgresql:candlepin
DEBUG 1/15/18 8:13 AM:liquibase: Setting auto commit to false from true
DEBUG 1/15/18 8:13 AM:liquibase: Executing QUERY database command: select count(*) from public.databasechangeloglock
.....
.....
org.candlepin.liquibase.PerOrgProductsMigrationValidationTask.execute(PerOrgProductsMigrationValidationTask.java:189)
	at org.candlepin.liquibase.LiquibaseCustomTaskWrapper.execute(LiquibaseCustomTaskWrapper.java:71)
	... 9 more


For more information, use the --logLevel flag
Traceback (most recent call last):
  File "./cpdb", line 297, in <module>
    dbsetup.validate()
  File "./cpdb", line 70, in validate
    self._run_liquibase("db/changelog/changelog-validate.xml")
  File "./cpdb", line 106, in _run_liquibase
    output = run_command("liquibase %s migrate -Dcommunity=%s" % (liquibase_options, self.community))
  File "./cpdb", line 38, in run_command
    error_out(command, status, output)
  File "./cpdb", line 46, in error_out
    raise Exception("Error running command")
Exception: Error running command
[root@ibm-x3650-01 candlepin]# echo $?
1

The return code when encountered with error
Also the default loglevel increase to severe
./cpdb --validate -u candlepin -p zzzzzzzzz -d candlepin
Validating Candlepin database
--driver=org.postgresql.Driver --classpath=/usr/share/java/postgresql-jdbc.jar:/var/lib/tomcat/webapps/candlepin/WEB-INF/classes/ --changeLogFile=db/changelog/changelog-validate.xml --url=jdbc:postgresql:candlepin --username=candlepin --password=zzzzzzz --logLevel=severe
Liquibase Update Successful

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2018:0273

Hey, I don't understand. There was an error and you flipped to VERIFIED. Can you explain please? This looks like an issue to me. It was the purpose of this BZ to actually introduce the new parameter.