Red Hat Bugzilla – Bug 1225541
fleecing vsphere6 vms shows unable to determine port error with vddk6
Last modified: 2017-01-31 16:18:40 EST
Description of problem: ============================ Finally set this up and fleecing a vsphere6 vm with vddk6.0.0 shows the 'unable to determine port' error. I reached out to Jerry who took a quick peek and believes the ffi code needs to be updated to support version 6 Version-Release number of selected component (if applicable): ============================== 5.4.0.2 How reproducible: ==================== 100% Steps to Reproduce: ======================== 1. deploy appliance on vsphere6 2. configure host credentials 3. install vddk 6.0.0 4. reboot 5. enable smartproxy role 6. manage vsphere6 provider 7. scan vm Actual results: ========================= unable to determine port Expected results: ========================= success, vm metadata retrieved
We are working to implement the changes necessary for vddk 6.0 in both ManageIQ and the ffi-vix_disk_lib gem, where necessary.
Same error with 5.5.0.8-beta1.4.20151027164951_4ab7fea and VDDK 6.0.1 evm.log excerpt: [----] E, [2015-10-30T14:52:27.537705 #26549:f75990] ERROR -- : Unable to mount filesystem. Reason:[ERROR: VixDiskLib.connect() Unable to determine port used by VixDiskLib Server.] for VM [[datastore_name] vm_name/vm_name.vmx] [----] E, [2015-10-30T14:52:27.537989 #26549:f75990] ERROR -- : MIQExtract.new /var/www/miq/vmdb/gems/pending/VixDiskLib/VixDiskLib.rb:141:in `get_uri' [----] E, [2015-10-30T14:52:27.538036 #26549:f75990] ERROR -- : MIQExtract.new /var/www/miq/vmdb/gems/pending/VixDiskLib/VixDiskLib.rb:125:in `start_service' [----] E, [2015-10-30T14:52:27.538074 #26549:f75990] ERROR -- : MIQExtract.new /var/www/miq/vmdb/gems/pending/VixDiskLib/VixDiskLib.rb:42:in `block in connect' [----] E, [2015-10-30T14:52:27.538151 #26549:f75990] ERROR -- : MIQExtract.new /opt/rh/rh-ruby22/root/usr/share/ruby/sync.rb:233:in `block in sync_synchronize' [----] E, [2015-10-30T14:52:27.538189 #26549:f75990] ERROR -- : MIQExtract.new /opt/rh/rh-ruby22/root/usr/share/ruby/sync.rb:230:in `handle_interrupt' [----] E, [2015-10-30T14:52:27.538228 #26549:f75990] ERROR -- : MIQExtract.new /opt/rh/rh-ruby22/root/usr/share/ruby/sync.rb:230:in `sync_synchronize' [----] E, [2015-10-30T14:52:27.538265 #26549:f75990] ERROR -- : MIQExtract.new /var/www/miq/vmdb/gems/pending/VixDiskLib/VixDiskLib.rb:39:in `connect' [----] E, [2015-10-30T14:52:27.538301 #26549:f75990] ERROR -- : MIQExtract.new /var/www/miq/vmdb/gems/pending/VMwareWebService/MiqVimVdlMod.rb:40:in `vdlVcConnection' [----] E, [2015-10-30T14:52:27.538337 #26549:f75990] ERROR -- : MIQExtract.new /var/www/miq/vmdb/gems/pending/MiqVm/MiqVm.rb:90:in `block in openDisks' [----] E, [2015-10-30T14:52:27.538405 #26549:f75990] ERROR -- : MIQExtract.new /var/www/miq/vmdb/gems/pending/MiqVm/MiqVm.rb:74:in `each' [----] E, [2015-10-30T14:52:27.538443 #26549:f75990] ERROR -- : MIQExtract.new /var/www/miq/vmdb/gems/pending/MiqVm/MiqVm.rb:74:in `openDisks' [----] E, [2015-10-30T14:52:27.538480 #26549:f75990] ERROR -- : MIQExtract.new /var/www/miq/vmdb/gems/pending/MiqVm/MiqVm.rb:62:in `vmDisks' [----] E, [2015-10-30T14:52:27.538517 #26549:f75990] ERROR -- : MIQExtract.new /var/www/miq/vmdb/gems/pending/MiqVm/MiqVm.rb:167:in `volumeManager' [----] E, [2015-10-30T14:52:27.538553 #26549:f75990] ERROR -- : MIQExtract.new /var/www/miq/vmdb/gems/pending/MiqVm/MiqVm.rb:161:in `rootTrees' [----] E, [2015-10-30T14:52:27.538677 #26549:f75990] ERROR -- : MIQExtract.new /var/www/miq/vmdb/gems/pending/metadata/MIQExtract/MIQExtract.rb:58:in `initialize' [----] E, [2015-10-30T14:52:27.538716 #26549:f75990] ERROR -- : MIQExtract.new /var/www/miq/vmdb/app/models/mixins/scanning_mixin.rb:237:in `new' [----] E, [2015-10-30T14:52:27.538753 #26549:f75990] ERROR -- : MIQExtract.new /var/www/miq/vmdb/app/models/mixins/scanning_mixin.rb:237:in `scan_via_miq_vm' [----] E, [2015-10-30T14:52:27.538789 #26549:f75990] ERROR -- : MIQExtract.new /var/www/miq/vmdb/app/models/manageiq/providers/vmware/infra_manager/vm_or_template_shared/scanning.rb:17:in `perform_metadata_scan' [----] E, [2015-10-30T14:52:27.538835 #26549:f75990] ERROR -- : MIQExtract.new /var/www/miq/vmdb/app/models/miq_server/server_smart_proxy.rb:123:in `scan_metadata' [----] E, [2015-10-30T14:52:27.538874 #26549:f75990] ERROR -- : MIQExtract.new /var/www/miq/vmdb/app/models/miq_queue.rb:345:in `block in deliver' [----] E, [2015-10-30T14:52:27.538909 #26549:f75990] ERROR -- : MIQExtract.new /opt/rh/rh-ruby22/root/usr/share/ruby/timeout.rb:89:in `block in timeout' [----] E, [2015-10-30T14:52:27.538945 #26549:f75990] ERROR -- : MIQExtract.new /opt/rh/rh-ruby22/root/usr/share/ruby/timeout.rb:34:in `block in catch' [----] E, [2015-10-30T14:52:27.539014 #26549:f75990] ERROR -- : MIQExtract.new /opt/rh/rh-ruby22/root/usr/share/ruby/timeout.rb:34:in `catch' [----] E, [2015-10-30T14:52:27.539052 #26549:f75990] ERROR -- : MIQExtract.new /opt/rh/rh-ruby22/root/usr/share/ruby/timeout.rb:34:in `catch' [----] E, [2015-10-30T14:52:27.539088 #26549:f75990] ERROR -- : MIQExtract.new /opt/rh/rh-ruby22/root/usr/share/ruby/timeout.rb:104:in `timeout' [----] E, [2015-10-30T14:52:27.539128 #26549:f75990] ERROR -- : MIQExtract.new /var/www/miq/vmdb/app/models/miq_queue.rb:341:in `deliver' [----] E, [2015-10-30T14:52:27.539164 #26549:f75990] ERROR -- : MIQExtract.new /var/www/miq/vmdb/app/models/miq_queue_worker_base/runner.rb:106:in `deliver_queue_message' [----] E, [2015-10-30T14:52:27.539247 #26549:f75990] ERROR -- : MIQExtract.new /var/www/miq/vmdb/app/models/miq_queue_worker_base/runner.rb:134:in `deliver_message' [----] E, [2015-10-30T14:52:27.539284 #26549:f75990] ERROR -- : MIQExtract.new /var/www/miq/vmdb/app/models/miq_queue_worker_base/runner.rb:151:in `block in do_work' [----] E, [2015-10-30T14:52:27.539320 #26549:f75990] ERROR -- : MIQExtract.new /var/www/miq/vmdb/app/models/miq_queue_worker_base/runner.rb:145:in `loop' [----] E, [2015-10-30T14:52:27.539355 #26549:f75990] ERROR -- : MIQExtract.new /var/www/miq/vmdb/app/models/miq_queue_worker_base/runner.rb:145:in `do_work' [----] E, [2015-10-30T14:52:27.539391 #26549:f75990] ERROR -- : MIQExtract.new /var/www/miq/vmdb/app/models/miq_worker/runner.rb:323:in `block in do_work_loop' [----] E, [2015-10-30T14:52:27.539427 #26549:f75990] ERROR -- : MIQExtract.new /var/www/miq/vmdb/app/models/miq_worker/runner.rb:320:in `loop' [----] E, [2015-10-30T14:52:27.539462 #26549:f75990] ERROR -- : MIQExtract.new /var/www/miq/vmdb/app/models/miq_worker/runner.rb:320:in `do_work_loop' [----] E, [2015-10-30T14:52:27.539498 #26549:f75990] ERROR -- : MIQExtract.new /var/www/miq/vmdb/app/models/miq_worker/runner.rb:142:in `run' [----] E, [2015-10-30T14:52:27.539600 #26549:f75990] ERROR -- : MIQExtract.new /var/www/miq/vmdb/app/models/miq_worker/runner.rb:123:in `start' [----] E, [2015-10-30T14:52:27.539654 #26549:f75990] ERROR -- : MIQExtract.new /var/www/miq/vmdb/app/models/miq_worker/runner.rb:24:in `start_worker' [----] E, [2015-10-30T14:52:27.539692 #26549:f75990] ERROR -- : MIQExtract.new /var/www/miq/vmdb/lib/workers/bin/worker.rb:2:in `<top (required)>' [----] E, [2015-10-30T14:52:27.539728 #26549:f75990] ERROR -- : MIQExtract.new /opt/rh/cfme-gemset/bundler/gems/rails-e2fcb2b4aec6/railties/lib/rails/commands/runner.rb:60:in `load' [----] E, [2015-10-30T14:52:27.539763 #26549:f75990] ERROR -- : MIQExtract.new /opt/rh/cfme-gemset/bundler/gems/rails-e2fcb2b4aec6/railties/lib/rails/commands/runner.rb:60:in `<top (required)>' [----] E, [2015-10-30T14:52:27.539799 #26549:f75990] ERROR -- : MIQExtract.new /opt/rh/cfme-gemset/bundler/gems/rails-e2fcb2b4aec6/railties/lib/rails/commands/commands_tasks.rb:123:in `require' [----] E, [2015-10-30T14:52:27.539881 #26549:f75990] ERROR -- : MIQExtract.new /opt/rh/cfme-gemset/bundler/gems/rails-e2fcb2b4aec6/railties/lib/rails/commands/commands_tasks.rb:123:in `require_command!' [----] E, [2015-10-30T14:52:27.539919 #26549:f75990] ERROR -- : MIQExtract.new /opt/rh/cfme-gemset/bundler/gems/rails-e2fcb2b4aec6/railties/lib/rails/commands/commands_tasks.rb:90:in `runner' [----] E, [2015-10-30T14:52:27.539955 #26549:f75990] ERROR -- : MIQExtract.new /opt/rh/cfme-gemset/bundler/gems/rails-e2fcb2b4aec6/railties/lib/rails/commands/commands_tasks.rb:39:in `run_command!' [----] E, [2015-10-30T14:52:27.539997 #26549:f75990] ERROR -- : MIQExtract.new /opt/rh/cfme-gemset/bundler/gems/rails-e2fcb2b4aec6/railties/lib/rails/commands.rb:17:in `<top (required)>' [----] E, [2015-10-30T14:52:27.540035 #26549:f75990] ERROR -- : MIQExtract.new /var/www/miq/vmdb/bin/rails:4:in `require' [----] E, [2015-10-30T14:52:27.540071 #26549:f75990] ERROR -- : MIQExtract.new /var/www/miq/vmdb/bin/rails:4:in `<main>' vim.log excerpt: ffi-vix_disk_lib: failed to load 6.0.0 version with error: Could not open library 'vixDiskLib.so.6.0.0': vixDiskLib.so.6.0.0: cannot open shared object file: No such file or directory. Could not open library 'libvixDiskLib.so.6.0.0': libvixDiskLib.so.6.0.0: cannot open shared object file: No such file or directory. ffi-vix_disk_lib: failed to load 5.5.4 version with error: Could not open library 'vixDiskLib.so.5.5.4': vixDiskLib.so.5.5.4: cannot open shared object file: No such file or directory. Could not open library 'libvixDiskLib.so.5.5.4': libvixDiskLib.so.5.5.4: cannot open shared object file: No such file or directory. ffi-vix_disk_lib: failed to load 5.5.2 version with error: Could not open library 'vixDiskLib.so.5.5.2': vixDiskLib.so.5.5.2: cannot open shared object file: No such file or directory. Could not open library 'libvixDiskLib.so.5.5.2': libvixDiskLib.so.5.5.2: cannot open shared object file: No such file or directory. ffi-vix_disk_lib: failed to load 5.5.1 version with error: Could not open library 'vixDiskLib.so.5.5.1': vixDiskLib.so.5.5.1: cannot open shared object file: No such file or directory. Could not open library 'libvixDiskLib.so.5.5.1': libvixDiskLib.so.5.5.1: cannot open shared object file: No such file or directory. ffi-vix_disk_lib: failed to load 5.5.0 version with error: Could not open library 'vixDiskLib.so.5.5.0': vixDiskLib.so.5.5.0: cannot open shared object file: No such file or directory. Could not open library 'libvixDiskLib.so.5.5.0': libvixDiskLib.so.5.5.0: cannot open shared object file: No such file or directory. ffi-vix_disk_lib: failed to load 5.1.3 version with error: Could not open library 'vixDiskLib.so.5.1.3': vixDiskLib.so.5.1.3: cannot open shared object file: No such file or directory. Could not open library 'libvixDiskLib.so.5.1.3': libvixDiskLib.so.5.1.3: cannot open shared object file: No such file or directory. ffi-vix_disk_lib: failed to load 5.1.2 version with error: Could not open library 'vixDiskLib.so.5.1.2': vixDiskLib.so.5.1.2: cannot open shared object file: No such file or directory. Could not open library 'libvixDiskLib.so.5.1.2': libvixDiskLib.so.5.1.2: cannot open shared object file: No such file or directory. ffi-vix_disk_lib: failed to load 5.1.1 version with error: Could not open library 'vixDiskLib.so.5.1.1': vixDiskLib.so.5.1.1: cannot open shared object file: No such file or directory. Could not open library 'libvixDiskLib.so.5.1.1': libvixDiskLib.so.5.1.1: cannot open shared object file: No such file or directory. ffi-vix_disk_lib: failed to load 5.1.0 version with error: Could not open library 'vixDiskLib.so.5.1.0': vixDiskLib.so.5.1.0: cannot open shared object file: No such file or directory. Could not open library 'libvixDiskLib.so.5.1.0': libvixDiskLib.so.5.1.0: cannot open shared object file: No such file or directory. ffi-vix_disk_lib: failed to load 5.0.4 version with error: Could not open library 'vixDiskLib.so.5.0.4': vixDiskLib.so.5.0.4: cannot open shared object file: No such file or directory. Could not open library 'libvixDiskLib.so.5.0.4': libvixDiskLib.so.5.0.4: cannot open shared object file: No such file or directory. ffi-vix_disk_lib: failed to load 5.0.0 version with error: Could not open library 'vixDiskLib.so.5.0.0': vixDiskLib.so.5.0.0: cannot open shared object file: No such file or directory. Could not open library 'libvixDiskLib.so.5.0.0': libvixDiskLib.so.5.0.0: cannot open shared object file: No such file or directory. ffi-vix_disk_lib: failed to load 1.2.0 version with error: Could not open library 'vixDiskLib.so.1.2.0': vixDiskLib.so.1.2.0: cannot open shared object file: No such file or directory. Could not open library 'libvixDiskLib.so.1.2.0': libvixDiskLib.so.1.2.0: cannot open shared object file: No such file or directory. ffi-vix_disk_lib: failed to load 1.1.2 version with error: Could not open library 'vixDiskLib.so.1.1.2': vixDiskLib.so.1.1.2: cannot open shared object file: No such file or directory. Could not open library 'libvixDiskLib.so.1.1.2': libvixDiskLib.so.1.1.2: cannot open shared object file: No such file or directory. /opt/rh/rh-ruby22/root/usr/share/gems/gems/ffi-1.9.8/lib/ffi/library.rb:162:in `ffi_libraries': no library specified (LoadError) from /opt/rh/rh-ruby22/root/usr/share/gems/gems/ffi-1.9.8/lib/ffi/library.rb:240:in `attach_function' from /opt/rh/cfme-gemset/gems/ffi-vix_disk_lib-1.0.2/lib/ffi-vix_disk_lib/api.rb:98:in `<module:API>' from /opt/rh/cfme-gemset/gems/ffi-vix_disk_lib-1.0.2/lib/ffi-vix_disk_lib/api.rb:5:in `<module:VixDiskLib>' from /opt/rh/cfme-gemset/gems/ffi-vix_disk_lib-1.0.2/lib/ffi-vix_disk_lib/api.rb:4:in `<module:FFI>' from /opt/rh/cfme-gemset/gems/ffi-vix_disk_lib-1.0.2/lib/ffi-vix_disk_lib/api.rb:3:in `<top (required)>' from /opt/rh/rh-ruby22/root/usr/share/rubygems/rubygems/core_ext/kernel_require.rb:54:in `require' from /opt/rh/rh-ruby22/root/usr/share/rubygems/rubygems/core_ext/kernel_require.rb:54:in `require' from /opt/rh/cfme-gemset/gems/ffi-vix_disk_lib-1.0.2/lib/ffi-vix_disk_lib.rb:7:in `<top (required)>' from /opt/rh/rh-ruby22/root/usr/share/rubygems/rubygems/core_ext/kernel_require.rb:54:in `require' from /opt/rh/rh-ruby22/root/usr/share/rubygems/rubygems/core_ext/kernel_require.rb:54:in `require' from /opt/rh/cfme-gemset/gems/ffi-vix_disk_lib-1.0.2/lib/ffi-vix_disk_lib/api_wrapper.rb:1:in `<top (required)>' from /opt/rh/rh-ruby22/root/usr/share/rubygems/rubygems/core_ext/kernel_require.rb:128:in `require' from /opt/rh/rh-ruby22/root/usr/share/rubygems/rubygems/core_ext/kernel_require.rb:128:in `rescue in require' from /opt/rh/rh-ruby22/root/usr/share/rubygems/rubygems/core_ext/kernel_require.rb:39:in `require' from /var/www/miq/vmdb/gems/pending/VixDiskLib/vdl_wrapper.rb:3:in `<top (required)>' from /opt/rh/rh-ruby22/root/usr/share/rubygems/rubygems/core_ext/kernel_require.rb:54:in `require' from /opt/rh/rh-ruby22/root/usr/share/rubygems/rubygems/core_ext/kernel_require.rb:54:in `require' from /var/www/miq/vmdb/gems/pending/VixDiskLib/VixDiskLibServer.rb:11:in `<main>' Could this be due to removal of the vddk install?
Templates have the wait_for_policy error (never starts the task) which masks this error.
We did not add VDDK 6.0.1 support. We added 6.0.0.
Thom - can you determine if this error happens with VDK 6.0.0? We only tested against and coded for 6.0.0 at this point. Thank you.
The problem is you have the VDDK libraries installed in the directory /usr/lib/vmware-vix-disklib-distrib/lib64 instead of /usr/lib/vmware-vix-disklib/lib64. That is why the libraries cannot be found. You can see on some of your other appliances, like 10.8.58.83, for example, that the library is is the correct directory and you are not getting the error of the library not being found - the VDDK is being invoked (and getting other errors that are being handled by other BZs. I would like to close this BZ with no trouble found if you agree.
After discussion with Thom Carlin, it was determined that some end-user doc is required for this issue. Prior to VDDK 6.0, VMware shipped an installer with the libraries that installed everything in /usr/lib/vmware-vix-disklib. As of 6.0, the installer is no longer shipped. We need to let the users know to install the libs in this directory or our Smart State Analysis code for VMware will not work.
Can you tell me what steps they need to take to get the vddk installed/copied into the correct place?
A KB needs to be written describing installation of VDDK 6.0 and higher (assuming that VMware does not decide to recreate their installation script). The info should be along the lines of: 1. Prior to release 6.0 of the VDDK VMware provided an installation script that installed the necessary files into the library directory at /usr/lib/vmware-vix-disklib. 2. In order to install a 6.0+ release of the VDDK onto a ManageIQ appliance, do the following: 3. Copy the downloaded gzip file to /tmp on the appliance. 4. unzip the archive file. 5. If /usr/lib/vmware-vix-disklib already exists, backup and remove the contents of the directory. 6. if the directory does not exist, create it by running "mkdir /usr/lib/vmware-vix-disklib" 7. Under the unzipped archive file in /tmp, there are several directories. Copy the following directories and their contents into /usr/lib/vmware-vix-disklib: bin64 include lib64
Andrew, can you please ensure that this is in the RNs for 5.5? This is a big issue. I can create a KB, but would be more appropriate in a Release Note.
If this is dropped from the advisory, does that mean that the doc text doesn't show up in release notes?
This known issue has been added to the Release Notes, the doc text of which is now live on the Customer Portal. Closing.
Thanks Andrew! ( I was bleary eyed and missed it) Additional steps required to add symlinks: ln -s /usr/lib/vmware-vix-disklib/lib64/libvixDiskLib.so /usr/lib/libvixDiskLib.so ln -s /usr/lib/vmware-vix-disklib/lib64/libvixDiskLib.so.6 /usr/lib/libvixDiskLib.so.6 reboot
https://access.redhat.com/articles/2078103