Bug 1831787

Description of problem:
Composite Content View is failing to promote. The only error logged:

2020-05-05T08:21:19 [W|app|] Failed to calculate obsolete and new
2020-05-05T08:21:19 [E|bac|] PG::InFailedSqlTransaction: ERROR:  current transaction is aborted, commands ignored until end of transaction block: SELECT  "taxonomies".* FROM "taxonomies" WHERE "taxonomies"."type" IN ('Organization') AND "taxonomies"."id" = $1 LIMIT $2 (ActiveRecord::StatementInvalid)



Looking at /usr/share/foreman/app/services/puppet_class_importer.rb, we see the exception that hides the error, so we comment the rescue statement (lines 81..83 below) to get the actual exception on production.log:

 67   def obsolete_and_new(changes = { })
 68     return if changes.empty?
 69     changes.values.map(&:keys).flatten.uniq.each do |env_name|
 70       if changes['new'] && changes['new'][env_name].try(:>, '') # we got new classes
 71         add_classes_to_foreman(env_name, JSON.parse(changes['new'][env_name]))
 72       end
 73       if changes['obsolete'] && changes['obsolete'][env_name].try(:>, '') # we need to remove classes
 74         remove_classes_from_foreman(env_name, JSON.parse(changes['obsolete'][env_name]))
 75       end
 76       if changes['updated'] && changes['updated'][env_name].try(:>, '') # we need to update classes
 77         update_classes_in_foreman(env_name, JSON.parse(changes['updated'][env_name]))
 78       end
 79     end
 80     []
 81   #rescue => e
 82   #  Foreman::Logging.exception('Failed to calculate obsolete and new', e)
 83   #  [e.to_s]
 84   end


Result in /var/log/foreman/production.log:


2020-05-05T15:56:03 [E|bac|] PG::ProtocolViolation: ERROR:  invalid message format
: SELECT "taxonomies".* FROM "taxonomies" WHERE "taxonomies"."id" IN ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14, ...

NOTE: the list goes on to $69801.  The taxonomies table in this specific case has ~30 entries only. There are certainly many duplicates in this ID lookup list.


After applying the patch below to do a .uniq on the list of IDs being passed to this lookup:

diff --git a/app/models/concerns/audit_extensions.rb b/app/models/concerns/audit_extensions.rb
index 32690be48..432b490ea 100644
--- a/app/models/concerns/audit_extensions.rb
+++ b/app/models/concerns/audit_extensions.rb
@@ -241,6 +241,7 @@ module AuditExtensions
     elsif associated.respond_to?(:"#{key_name}_ids")
       ids_arr = associated.send("#{key_name}_ids")
     end
-    self.send("#{key_name}_ids=", ids_arr)
+    Rails.logger.error("IDs: #{ids_arr}")
+    self.send("#{key_name}_ids=", ids_arr.uniq)
   end
 end



...the CCV promote works successfully.
And this logs to production.log a very long list of IDs that are being uniq'd:

2020-05-05T16:31:47 [E|app|] IDs: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...


Each taxonomy ID appears _thousands_ of times on this list.



Version-Release number of selected component (if applicable):
foreman-1.22.0.39-2.el7sat.noarch


How reproducible:
Every time for certain individual Composite Content Views.

Steps to Reproduce:
1. Publish or promote one of a couple individual Composite Content Views

Actual results:
It fails and enters paused-error state with a not-so-evident error saying:
~~~
2020-05-05T08:21:19 [E|bac|] PG::InFailedSqlTransaction: ERROR:  current transaction is aborted, commands ignored until end of transaction block: SELECT  "taxonomies".* FROM "taxonomies" WHERE "taxonomies"."type" IN ('Organization') AND "taxonomies"."id" = $1 LIMIT $2 (ActiveRecord::StatementInvalid)
~~~

Expected results:
Success.


Additional info:
Thank you jsherrill in particular for the invaluable assistance in getting this issued sorted out.