Description of problem: Manifest upload failed Version-Release number of selected component (if applicable): How reproducible: Always Steps to Reproduce: 1.Install SAM 1.3 2.Upload manifest (attached in the bug) 3. Actual results: Manifest upload fails ==> /var/log/katello/delayed_job.log <== 2013-09-19T22:02:59-0400: [Worker(delayed_job host:cloud-qe-6.idm.lab.bos.redhat.com pid:21469)] Provider#queue_import_manifest failed with RestClient::InternalServerError: Resources::Candlepin::Owner: 500 Internal Server Error {"displayMessage":"Failed to import archive"} (POST /candlepin/owners/kumarcorp/imports) - 0 failed attempts 2013-09-19T22:02:59-0400: [Worker(delayed_job host:cloud-qe-6.idm.lab.bos.redhat.com pid:21469)] PERMANENTLY removing Provider#queue_import_manifest because of 1 consecutive failures. Resources::Candlepin::Owner: 500 Internal Server Error {"displayMessage":"Failed to import archive"} (POST /candlepin/owners/kumarcorp/imports) /opt/rh/ruby193/root/usr/share/gems/gems/rest-client-1.6.1/lib/restclient/abstract_response.rb:48:in `return!' /opt/rh/ruby193/root/usr/share/gems/gems/rest-client-1.6.1/lib/restclient/request.rb:220:in `process_result' /opt/rh/ruby193/root/usr/share/gems/gems/rest-client-1.6.1/lib/restclient/request.rb:169:in `block in transmit' /opt/rh/ruby193/root/usr/share/ruby/net/http.rb:746:in `start' /opt/rh/ruby193/root/usr/share/gems/gems/rest-client-1.6.1/lib/restclient/request.rb:166:in `transmit' /opt/rh/ruby193/root/usr/share/gems/gems/rest-client-1.6.1/lib/restclient/request.rb:60:in `execute' /opt/rh/ruby193/root/usr/share/gems/gems/rest-client-1.6.1/lib/restclient/request.rb:31:in `execute' /opt/rh/ruby193/root/usr/share/gems/gems/rest-client-1.6.1/lib/restclient/resource.rb:59:in `post' /usr/share/katello/app/lib/http_resource.rb:110:in `post' /usr/share/katello/app/lib/resources/candlepin.rb:352:in `import' /usr/share/katello/app/models/glue/provider.rb:188:in `owner_import' /usr/share/katello/app/models/glue.rb:174:in `execute' /usr/share/katello/app/models/glue.rb:124:in `block in process' /usr/share/katello/app/models/glue.rb:111:in `each' /usr/share/katello/app/models/glue.rb:111:in `process' /usr/share/katello/app/models/glue.rb:35:in `on_save' /opt/rh/ruby193/root/usr/share/gems/gems/activesupport-3.2.8/lib/active_support/callbacks.rb:214:in `_conditional_callback_around_3457' /opt/rh/ruby193/root/usr/share/gems/gems/activesupport-3.2.8/lib/active_support/callbacks.rb:403:in `_run__1635507987786152592__save__1814581965109045033__callbacks' /opt/rh/ruby193/root/usr/share/gems/gems/activesupport-3.2.8/lib/active_support/callbacks.rb:405:in `__run_callback' /opt/rh/ruby193/root/usr/share/gems/gems/activesupport-3.2.8/lib/active_support/callbacks.rb:385:in `_run_save_callbacks' /opt/rh/ruby193/root/usr/share/gems/gems/activesupport-3.2.8/lib/active_support/callbacks.rb:81:in `run_callbacks' /opt/rh/ruby193/root/usr/share/gems/gems/activerecord-3.2.8/lib/active_record/callbacks.rb:264:in `create_or_update' /opt/rh/ruby193/root/usr/share/gems/gems/activerecord-3.2.8/lib/active_record/persistence.rb:104:in `save!' /opt/rh/ruby193/root/usr/share/gems/gems/activerecord-3.2.8/lib/active_record/validations.rb:56:in `save!' /opt/rh/ruby193/root/usr/share/gems/gems/activerecord-3.2.8/lib/active_record/attribute_methods/dirty.rb:33:in `save!' /opt/rh/ruby193/root/usr/share/gems/gems/activerecord-3.2.8/lib/active_record/transactions.rb:246:in `block in save!' /opt/rh/ruby193/root/usr/share/gems/gems/activerecord-3.2.8/lib/active_record/transactions.rb:295:in `block in with_transaction_returning_status' /opt/rh/ruby193/root/usr/share/gems/gems/activerecord-3.2.8/lib/active_record/connection_adapters/abstract/database_statements.rb:192:in `transaction' /opt/rh/ruby193/root/usr/share/gems/gems/activerecord-3.2.8/lib/active_record/transactions.rb:208:in `transaction' /opt/rh/ruby193/root/usr/share/gems/gems/activerecord-3.2.8/lib/active_record/transactions.rb:293:in `with_transaction_returning_status' /opt/rh/ruby193/root/usr/share/gems/gems/activerecord-3.2.8/lib/active_record/transactions.rb:246:in `save!' /usr/share/katello/app/models/glue/provider.rb:308:in `queue_import_manifest' /usr/share/katello/app/models/async_operation.rb:58:in `perform' /opt/rh/ruby193/root/usr/share/gems/gems/delayed_job-3.0.2/lib/delayed/backend/base.rb:94:in `block in invoke_job' /opt/rh/ruby193/root/usr/share/gems/gems/delayed_job-3.0.2/lib/delayed/lifecycle.rb:60:in `call' /opt/rh/ruby193/root/usr/share/gems/gems/delayed_job-3.0.2/lib/delayed/lifecycle.rb:60:in `block in initialize' /opt/rh/ruby193/root/usr/share/gems/gems/delayed_job-3.0.2/lib/delayed/lifecycle.rb:65:in `call' /opt/rh/ruby193/root/usr/share/gems/gems/delayed_job-3.0.2/lib/delayed/lifecycle.rb:65:in `execute' /opt/rh/ruby193/root/usr/share/gems/gems/delayed_job-3.0.2/lib/delayed/lifecycle.rb:38:in `run_callbacks' /opt/rh/ruby193/root/usr/share/gems/gems/delayed_job-3.0.2/lib/delayed/backend/base.rb:91:in `invoke_job' /opt/rh/ruby193/root/usr/share/gems/gems/delayed_job-3.0.2/lib/delayed/worker.rb:178:in `block (2 levels) in run' /opt/rh/ruby193/root/usr/share/ruby/timeout.rb:69:in `timeout' /opt/rh/ruby193/root/usr/share/gems/gems/delayed_job-3.0.2/lib/delayed/worker.rb:178:in `block in run' /opt/rh/ruby193/root/usr/share/ruby/benchmark.rb:295:in `realtime' /opt/rh/ruby193/root/usr/share/gems/gems/delayed_job-3.0.2/lib/delayed/worker.rb:177:in `run' /opt/rh/ruby193/root/usr/share/gems/gems/delayed_job-3.0.2/lib/delayed/worker.rb:234:in `block in reserve_and_run_one_job' /opt/rh/ruby193/root/usr/share/gems/gems/delayed_job-3.0.2/lib/delayed/lifecycle.rb:60:in `call' /opt/rh/ruby193/root/usr/share/gems/gems/delayed_job-3.0.2/lib/delayed/lifecycle.rb:60:in `block in initialize' /opt/rh/ruby193/root/usr/share/gems/gems/delayed_job-3.0.2/lib/delayed/lifecycle.rb:65:in `call' /opt/rh/ruby193/root/usr/share/gems/gems/delayed_job-3.0.2/lib/delayed/lifecycle.rb:65:in `execute' /opt/rh/ruby193/root/usr/share/gems/gems/delayed_job-3.0.2/lib/delayed/lifecycle.rb:38:in `run_callbacks' /opt/rh/ruby193/root/usr/share/gems/gems/delayed_job-3.0.2/lib/delayed/worker.rb:234:in `reserve_and_run_one_job' /opt/rh/ruby193/root/usr/share/gems/gems/delayed_job-3.0.2/lib/delayed/worker.rb:162:in `block in work_off' /opt/rh/ruby193/root/usr/share/gems/gems/delayed_job-3.0.2/lib/delayed/worker.rb:161:in `times' /opt/rh/ruby193/root/usr/share/gems/gems/delayed_job-3.0.2/lib/delayed/worker.rb:161:in `work_off' /opt/rh/ruby193/root/usr/share/gems/gems/delayed_job-3.0.2/lib/delayed/worker.rb:133:in `block (4 levels) in start' /opt/rh/ruby193/root/usr/share/ruby/benchmark.rb:295:in `realtime' /opt/rh/ruby193/root/usr/share/gems/gems/delayed_job-3.0.2/lib/delayed/worker.rb:132:in `block (3 levels) in start' /opt/rh/ruby193/root/usr/share/gems/gems/delayed_job-3.0.2/lib/delayed/lifecycle.rb:60:in `call' /opt/rh/ruby193/root/usr/share/gems/gems/delayed_job-3.0.2/lib/delayed/lifecycle.rb:60:in `block in initialize' /opt/rh/ruby193/root/usr/share/gems/gems/delayed_job-3.0.2/lib/delayed/lifecycle.rb:65:in `call' /opt/rh/ruby193/root/usr/share/gems/gems/delayed_job-3.0.2/lib/delayed/lifecycle.rb:65:in `execute' /opt/rh/ruby193/root/usr/share/gems/gems/delayed_job-3.0.2/lib/delayed/lifecycle.rb:38:in `run_callbacks' /opt/rh/ruby193/root/usr/share/gems/gems/delayed_job-3.0.2/lib/delayed/worker.rb:129:in `block (2 levels) in start' /opt/rh/ruby193/root/usr/share/gems/gems/delayed_job-3.0.2/lib/delayed/worker.rb:128:in `loop' /opt/rh/ruby193/root/usr/share/gems/gems/delayed_job-3.0.2/lib/delayed/worker.rb:128:in `block in start' /opt/rh/ruby193/root/usr/share/gems/gems/delayed_job-3.0.2/lib/delayed/plugins/clear_locks.rb:7:in `call' /opt/rh/ruby193/root/usr/share/gems/gems/delayed_job-3.0.2/lib/delayed/plugins/clear_locks.rb:7:in `block (2 levels) in <class:ClearLocks>' /opt/rh/ruby193/root/usr/share/gems/gems/delayed_job-3.0.2/lib/delayed/lifecycle.rb:78:in `call' /opt/rh/ruby193/root/usr/share/gems/gems/delayed_job-3.0.2/lib/delayed/lifecycle.rb:78:in `block (2 levels) in add' /opt/rh/ruby193/root/usr/share/gems/gems/delayed_job-3.0.2/lib/delayed/lifecycle.rb:60:in `call' /opt/rh/ruby193/root/usr/share/gems/gems/delayed_job-3.0.2/lib/delayed/lifecycle.rb:60:in `block in initialize' /opt/rh/ruby193/root/usr/share/gems/gems/delayed_job-3.0.2/lib/delayed/lifecycle.rb:78:in `call' /opt/rh/ruby193/root/usr/share/gems/gems/delayed_job-3.0.2/lib/delayed/lifecycle.rb:78:in `block in add' /opt/rh/ruby193/root/usr/share/gems/gems/delayed_job-3.0.2/lib/delayed/lifecycle.rb:65:in `call' /opt/rh/ruby193/root/usr/share/gems/gems/delayed_job-3.0.2/lib/delayed/lifecycle.rb:65:in `execute' /opt/rh/ruby193/root/usr/share/gems/gems/delayed_job-3.0.2/lib/delayed/lifecycle.rb:38:in `run_callbacks' /opt/rh/ruby193/root/usr/share/gems/gems/delayed_job-3.0.2/lib/delayed/worker.rb:127:in `start' /opt/rh/ruby193/root/usr/share/gems/gems/delayed_job-3.0.2/lib/delayed/command.rb:101:in `run' /opt/rh/ruby193/root/usr/share/gems/gems/delayed_job-3.0.2/lib/delayed/command.rb:89:in `block in run_process' /opt/rh/ruby193/root/usr/share/gems/gems/daemons-1.1.4/lib/daemons/application.rb:249:in `call' /opt/rh/ruby193/root/usr/share/gems/gems/daemons-1.1.4/lib/daemons/application.rb:249:in `block in start_proc' /opt/rh/ruby193/root/usr/share/gems/gems/daemons-1.1.4/lib/daemons/daemonize.rb:197:in `call' /opt/rh/ruby193/root/usr/share/gems/gems/daemons-1.1.4/lib/daemons/daemonize.rb:197:in `call_as_daemon' /opt/rh/ruby193/root/usr/share/gems/gems/daemons-1.1.4/lib/daemons/application.rb:253:in `start_proc' /opt/rh/ruby193/root/usr/share/gems/gems/daemons-1.1.4/lib/daemons/application.rb:293:in `start' /opt/rh/ruby193/root/usr/share/gems/gems/daemons-1.1.4/lib/daemons/controller.rb:70:in `run' /opt/rh/ruby193/root/usr/share/gems/gems/daemons-1.1.4/lib/daemons.rb:195:in `block in run_proc' /opt/rh/ruby193/root/usr/share/gems/gems/daemons-1.1.4/lib/daemons/cmdline.rb:109:in `call' /opt/rh/ruby193/root/usr/share/gems/gems/daemons-1.1.4/lib/daemons/cmdline.rb:109:in `catch_exceptions' /opt/rh/ruby193/root/usr/share/gems/gems/daemons-1.1.4/lib/daemons.rb:194:in `run_proc' /opt/rh/ruby193/root/usr/share/gems/gems/delayed_job-3.0.2/lib/delayed/command.rb:87:in `run_process' /opt/rh/ruby193/root/usr/share/gems/gems/delayed_job-3.0.2/lib/delayed/command.rb:80:in `block in daemonize' /opt/rh/ruby193/root/usr/share/gems/gems/delayed_job-3.0.2/lib/delayed/command.rb:78:in `times' /opt/rh/ruby193/root/usr/share/gems/gems/delayed_job-3.0.2/lib/delayed/command.rb:78:in `daemonize' script/delayed_job:5:in `<main>' 2013-09-19T22:02:59-0400: [Worker(delayed_job host:cloud-qe-6.idm.lab.bos.redhat.com pid:21469)] 1 jobs processed at 2.7289 j/s, 1 failed ... Expected results: Manifest upload successful Additional info:
FWIW this manifest is created in stage
Not sure what the root cause is, but proposing to be a blocker for now.
Analysis so far: Manifest is generated with one entitlement for SKU RH0103708, but in products/ directory we end up with JSON for MCT1790, and nothing for RH0103708. Nothing in the manifest references MCT1790. Checked the accounts pools (stage_test_12) and NONE of them reference that SKU. When making a manifest Candlepin iterates the consumer's entitlements, and fetches data for all marketing and eng products, then exports them one by one to a file. Filename comes from the product object returned by the adapter (not necessarily the product ID we asked for), so in theory, if somehow we asked product adapter for product A, but got a product back for product B, that *could* cause this. Naturally that should be impossible. The Exporter is one of very few Candlepin objects registered as a singleton. This should have been and still appears to be safe, but it does not look required, so I may remove this just in case.
We spent some time looking into this, but are completely unable to come up with any plausible way this could happen. According to aedwards there aren't even any accounts in stage that have subscriptions for MCT1790, so how this value could be appearing is a complete mystery. My instinct is that somehow, we're asking the product adapter for info on RH0103708, and it's returning product data for MCT1790. However I cannot claim how or why that would happen but at this point it seems like this must be something on the adapter/Oracle side. Andrew have you had any luck on your end?
The release of Satellite 5.8 we are deprecating the support of Subscription Asset Manager. The release notes for 5.8 can be found at https://access.redhat.com/documentation/en-us/red_hat_satellite/5.8/pdf/release_notes/Red_Hat_Satellite-5.8-Release_Notes-en-US.pdf. I am therefore closing out this bug as WONTFIX. If you believe this to be an error, please feel free tor each out to either Rich Jerrido or Bryan Kearney. Thank you!
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 1000 days