Bug 752464
Summary: | Cannot register ovirt-node image to ovirt-engine | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|
Product: | [Retired] oVirt | Reporter: | Mike Burns <mburns> | ||||||||
Component: | vdsm | Assignee: | Douglas Schilling Landgraf <dougsland> | ||||||||
Status: | CLOSED CURRENTRELEASE | QA Contact: | yeylon <yeylon> | ||||||||
Severity: | high | Docs Contact: | |||||||||
Priority: | high | ||||||||||
Version: | unspecified | CC: | abaron, apevec, bazulay, danken, dfediuck, iheim, mburns, srevivo, ykaul | ||||||||
Target Milestone: | --- | ||||||||||
Target Release: | --- | ||||||||||
Hardware: | Unspecified | ||||||||||
OS: | Unspecified | ||||||||||
Whiteboard: | |||||||||||
Fixed In Version: | v4.9.3.1 | Doc Type: | Bug Fix | ||||||||
Doc Text: | Story Points: | --- | |||||||||
Clone Of: | Environment: | ||||||||||
Last Closed: | 2012-06-19 16:54:50 UTC | Type: | --- | ||||||||
Regression: | --- | Mount Type: | --- | ||||||||
Documentation: | --- | CRM: | |||||||||
Verified Versions: | Category: | --- | |||||||||
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||||||
Cloudforms Team: | --- | Target Upstream Version: | |||||||||
Embargoed: | |||||||||||
Bug Depends On: | 753571, 753879, 755749, 755753, 756136, 760756, 769052, 769693 | ||||||||||
Bug Blocks: | 753297 | ||||||||||
Attachments: |
|
Description
Mike Burns
2011-11-09 15:33:07 UTC
Created attachment 532836 [details]
Proposed patch
Created attachment 532869 [details]
Proposed patch
Patch that factors out all bash checks into a function. Ideally, the python should do something similar, but I'm not familiar enough with python and how vdsm works to do that change
It seems there are other problems, however, that I don't know how to fix. vdsm-reg is still defaulting to /RHEVManagerWeb/VdsAutoRegistration.aspx I don't know where to point this to for registration. Due to the use of sed, /RHEVManagerWeb/VdsAutoRegistration.aspx was replaced by /ENGINEanagerWeb/VdsAutoRegistration.aspx We need to fix the typo on both sides (engine-core and vdsm), and we need to think about supporting previous nodes already installed. iirc, the ManagerWeb suffix (not to mention the .aspx) is legacy from C# days. let's take the opportunity and clean this up to the correct url I fully agree. Anything under /RegisterServlet/* will work. the VdsAutoRegistration.aspx is just for backwards compatibility. Created attachment 534239 [details]
vdsm-reg.log
Update:
- Manually hacked around release file by creating rhev-hypervisor-release file (issue fixed in vdsm but no build available)
- manually changed rwtab in ovirt-node to include /var/lib/vdsm (fix pending for vdsm)
- Updated vdsm-reg.conf to point to /RegisterServlet/register instead of the .aspx file mentioned previously
Registration failed with the attache logfile
Just internal update: First, Thanks Mike for your help! Here a new version of patch that already update: - manually changed rwtab in ovirt-node to include /var/lib/vdsm (fix pending for vdsm) - Updated vdsm-reg.conf to point to /RegisterServlet/register instead of the .aspx file mentioned previously (still pointing to /ENGINEanagerWeb/VdsAutoRegistration.aspx) Tried to open manyally via browser http://192.168.1.211:8080/RegisterServlet/register and got 404. Doron, Am I missing something? http://gerrit.ovirt.org/#change,268 Using (/ENGINEanagerWeb/VdsAutoRegistration.aspx) I got: MainThread::DEBUG::2011-11-18 02:33:27,585::deployUtil::1323::root::/rhevm.ssh.key.txt failed in HTTPS. Retrying using HTTP. Traceback (most recent call last): File "/usr/share/vdsm-reg/deployUtil.py", line 1318, in getRemoteFile File "/usr/share/vdsm-reg/deployUtil.py", line 1162, in getSSLSocket File "/usr/lib64/python2.7/ssl.py", line 372, in wrap_socket File "/usr/lib64/python2.7/ssl.py", line 132, in __init__ SSLError: [Errno 185090050] _ssl.c:340: error:0B084002:x509 certificate routines:X509_load_cert_crl_file:system lib MainThread::ERROR::2011-11-18 02:33:27,588::deployUtil::1348::root::Failed to fetch /rhevm.ssh.key.txt status 404 I don't have http://192.168.1.211:8080/rhevm.ssh.key.txt too, error 404. (In reply to comment #7) self.vdcURL 10.16.74.232 self.vdcPORT 8443 ... error: [Errno 111] Connection refused Seems that you don't have 8443 port open or jboss is listening on 8080 only? # telnet 10.16.74.232 8443 Trying 10.16.74.232... telnet: connect to address 10.16.74.232: Connection refused (In reply to comment #9) > I don't have http://192.168.1.211:8080/rhevm.ssh.key.txt too, error 404. backend/manager/conf/ca/installCA.sh should install /engine.ssh.key.txt but looks like ovirt-engine installation doesn't execute that script. What is equivalent of "rhevm-setup" for ovirt-engine? (In reply to comment #10) > (In reply to comment #7) > self.vdcURL 10.16.74.232 > self.vdcPORT 8443 > ... > error: [Errno 111] Connection refused > > Seems that you don't have 8443 port open or jboss is listening on 8080 only? > > # telnet 10.16.74.232 8443 > Trying 10.16.74.232... > telnet: connect to address 10.16.74.232: Connection refused 8443 is open, but jboss appears to not be listening there for some reason. I switched config to 8080 instead: MainThread::DEBUG::2011-11-17 19:39:58,669::deployUtil::105::root:: MainThread::DEBUG::2011-11-17 19:39:58,670::deployUtil::274::root::getHostID: 9A2FCF21-04E9-3737-6C3C-85011CE1A90C_52:54:00:c7:c4:5d MainThread::DEBUG::2011-11-17 19:39:58,670::vdsm-reg-setup::73::root::Setup::__init__ vars: self.vdcURL 10.16.74.232 self.vdcPORT 8080 self.vdcURI /RegisterServlet/register self.vdcRegPort 54321 self.ovirtURL 192.168.122.155 self.ovirtName localhost.localdomain self.ovirtUID 9A2FCF21-04E9-3737-6C3C-85011CE1A90C_52:54:00:c7:c4:5d self.vdcName 10.16.74.232 self.ticket MainThread::DEBUG::2011-11-17 19:39:58,671::vdsm-reg-setup::77::root::validate start MainThread::DEBUG::2011-11-17 19:39:58,671::vdsm-reg-setup::84::root::validate end. return: True MainThread::DEBUG::2011-11-17 19:39:58,672::vdsm-reg-setup::188::root::execute start. MainThread::DEBUG::2011-11-17 19:39:58,672::deployUtil::344::root::Bridge rhevm already exists. MainThread::DEBUG::2011-11-17 19:39:58,673::vdsm-reg-setup::192::root::execute: found existing management bridge. Skipping rename. MainThread::DEBUG::2011-11-17 19:39:58,673::deployUtil::1298::root::getRemoteFile start. IP = 10.16.74.232 port = 8080 fileName = "/rhevm.ssh.key.txt" MainThread::DEBUG::2011-11-17 19:39:58,841::deployUtil::1318::root::/rhevm.ssh.key.txt failed in HTTPS. Retrying using HTTP. Traceback (most recent call last): File "/usr/share/vdsm-reg/deployUtil.py", line 1313, in getRemoteFile File "/usr/share/vdsm-reg/deployUtil.py", line 1157, in getSSLSocket File "/usr/lib64/python2.7/ssl.py", line 372, in wrap_socket File "/usr/lib64/python2.7/ssl.py", line 132, in __init__ SSLError: [Errno 185090050] _ssl.c:340: error:0B084002:x509 certificate routines:X509_load_cert_crl_file:system lib MainThread::ERROR::2011-11-17 19:39:59,196::deployUtil::1343::root::Failed to fetch /rhevm.ssh.key.txt status 404 MainThread::DEBUG::2011-11-17 19:39:59,197::deployUtil::1354::root::getRemoteFile end. MainThread::DEBUG::2011-11-17 19:39:59,199::vdsm-reg-setup::204::root::execute: after getAuthKeysFile: False MainThread::DEBUG::2011-11-17 19:39:59,200::vdsm-reg-setup::215::root::Registration status:False ovirt-engine is not configured with ssl by default. need to enable the secure mode (by scripts right now). maybe we should add to node a checkbox if to use encryption or not to engine? (In reply to comment #8) > Just internal update: > > Tried to open manyally via browser > http://192.168.1.211:8080/RegisterServlet/register and got 404. Doron, Am I > missing something? > Just to answer myself: http://<IP>:8080/ENGINEanagerWeb/RegisterServlet/register (In reply to comment #14) > Just to answer myself: > http://<IP>:8080/ENGINEanagerWeb/RegisterServlet/register That's sed (rhevm->engine replacement) damage in engine.ear: engineanager.war engineanagerweb.war Failed to fetch /rhevm.ssh.key.txt status 404 The registration needs the engine's public ssh key. This key shoule be under jboss xxx/default/deploy/ROOT.war/ and will be used during the approval process. Since by default ovirt node uses key auth only, there's no simple way around it. ie- we need to use a key. Hi Doron, It seems that the key was not generated during my ovirt-engine deploy/install. Is it expected? Shouldn't be created during the ovirt-engine deploy? Is there any command line like rhevm-setup or should we generated manually? # pwd /usr/local/jboss/server/default/deploy/ROOT.war # ls -R .: css favicon.ico images index.html manager WEB-INF ./css: jboss.css ./images: logo.gif ./manager: xform.xsl ./WEB-INF: web.xml also: ============= # pwd /usr/local/jboss/server/default/deploy/engine.ear # ls components.war engineanager.war engineanagerweb.war engine-bll.jar engine-genericapi.jar engine-scheduler.jar engine-vdsbroker.jar lib META-INF restapi.war userportal.war webadmin.war # ls -R | grep -i ssh sshd-core-0.6.0.jar crosshair.png Thanks! Douglas Hi Douglas, true. We're working on a wiki which will give you a simple howto instructions to setup a proper pki environment. This is still under construction, but you can take a look in: http://ovirt.org/wiki/Engine_Node_Integration Hi Doron, Thanks for that. I am going to work with it. Cheers Douglas Hi Doron, I have got this report from a registration, can we chat about it? 2011-11-29 19:53:18,156 INFO [org.ovirt.engine.core.bll.RegisterVdsQuery] (http-0.0.0.0-8080-7) Running Command: RegisterVds 2011-11-29 19:53:18,284 INFO [org.ovirt.engine.core.bll.UpdateVdsCommand] (pool-22-thread-6) Running command: UpdateVdsCommand internal: true. Entities affected : ID: 016dc072-1aed-11e1-bbd0-00265a767e20 Type: VDS 2011-11-29 19:53:18,374 INFO [org.ovirt.engine.core.bll.RegisterVdsQuery] (http-0.0.0.0-8080-7) RegisterVdsQuery::Register -Updated a Pending registered VDS - Name: localhost.localdomain, Hostname: 192.168.1.20, UniqueID: 27452530-6AA3-11DE-9C27-00112550A074_00:01:02:8d:b0:88 2011-11-29 19:53:18,391 INFO [org.ovirt.engine.core.register.RegisterServlet] (http-0.0.0.0-8080-7) Succeeded to run RegisterVds. 2011-11-29 20:07:16,979 INFO [org.ovirt.engine.core.bll.LoginAdminUserCommand] (http-0.0.0.0-8080-8) Running command: LoginAdminUserCommand internal: false. 2011-11-29 20:07:17,003 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (http-0.0.0.0-8080-8) Running command: LogoutUserCommand internal: false. 2011-11-29 20:07:17,045 INFO [org.ovirt.engine.core.bll.LoginAdminUserCommand] (http-0.0.0.0-8080-8) Running command: LoginAdminUserCommand internal: false. 2011-11-29 20:07:17,089 INFO [org.ovirt.engine.core.bll.ApproveVdsCommand] (http-0.0.0.0-8080-8) Running command: ApproveVdsCommand internal: false. Entities affected : ID: 016dc072-1aed-11e1-bbd0-00265a767e20 Type: VDS 2011-11-29 20:07:17,090 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (http-0.0.0.0-8080-8) Before Installation http-0.0.0.0-8080-8, Powerclient/oVirtNode case: setting status to installing 2011-11-29 20:07:17,093 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (http-0.0.0.0-8080-8) START, SetVdsStatusVDSCommand(vdsId = 016dc072-1aed-11e1-bbd0-00265a767e20, status=Installing, nonOperationalReason=NONE), log id: 45d8db73 2011-11-29 20:07:17,113 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (http-0.0.0.0-8080-8) FINISH, SetVdsStatusVDSCommand, log id: 45d8db73 2011-11-29 20:07:17,141 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (http-0.0.0.0-8080-8) Before Installation http-0.0.0.0-8080-8 2011-11-29 20:07:17,141 INFO [org.ovirt.engine.core.bll.CBCInstaller] (http-0.0.0.0-8080-8) Installation of 192.168.1.20. Executing oVirt installation stage. (Stage: Starting Host installation) 2011-11-29 20:07:17,142 INFO [org.ovirt.engine.core.bll.CBCInstaller] (http-0.0.0.0-8080-8) Installation of 192.168.1.20. Executing oVirt installation stage. (Stage: Connecting to Host) 2011-11-29 20:07:17,228 INFO [org.ovirt.engine.core.bll.VdsInstaller] (http-0.0.0.0-8080-8) Installation of 192.168.1.20. Recieved message: <BSTRAP component='RHEV_INSTALL' status='OK' message='Connected to Host 192.168.1.20 with SSH key fingerprint: f3:52:b6:9c:c7:68:ac:a8:5a:02:f3:0a:d2:18:db:e4'/>. FYI. (Stage: Connecting to Host) 2011-11-29 20:07:17,244 INFO [org.ovirt.engine.core.bll.VdsInstaller] (http-0.0.0.0-8080-8) Installation of 192.168.1.20. Successfully connected to server ssh. (Stage: Connecting to Host) 2011-11-29 20:07:17,244 INFO [org.ovirt.engine.core.bll.CBCInstaller] (http-0.0.0.0-8080-8) Installation of 192.168.1.20. Executing oVirt installation stage. (Stage: Running first installation script on Host) 2011-11-29 20:07:17,244 INFO [org.ovirt.engine.core.utils.hostinstall.MinaInstallWrapper] (http-0.0.0.0-8080-8) Invoking /usr/share/vdsm-reg/vdsm-gen-cert -O "Redhat" 192.168.1.20 a6b79433-ce48-44ff-b329-54c984c88bfb on 192.168.1.20 2011-11-29 20:07:18,762 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (QuartzScheduler_Worker-98) Initializing Host: localhost.localdomain 2011-11-29 20:07:32,576 INFO [org.ovirt.engine.core.bll.VdsInstaller] (http-0.0.0.0-8080-8) Installation of 192.168.1.20. Recieved message: <BSTRAP component='Encryption setup' status='OK'/> <BSTRAP component='RHEV_INSTALL' status='OK'/> . Stage completed. (Stage: Running first installation script on Host) 2011-11-29 20:07:32,596 INFO [org.ovirt.engine.core.utils.hostinstall.MinaInstallWrapper] (http-0.0.0.0-8080-8) RunSSHCommand returns true 2011-11-29 20:07:32,597 INFO [org.ovirt.engine.core.bll.CBCInstaller] (http-0.0.0.0-8080-8) Installation of 192.168.1.20. Executing oVirt installation stage. (Stage: Downloading certificate request from Host) 2011-11-29 20:07:32,597 INFO [org.ovirt.engine.core.utils.hostinstall.MinaInstallWrapper] (http-0.0.0.0-8080-8) Downloading file /tmp/cert_a6b79433-ce48-44ff-b329-54c984c88bfb.req from 192.168.1.20 to /etc/pki/engine/ca//requests/cert_a6b79433-ce48-44ff-b329-54c984c88bfb.req 2011-11-29 20:07:32,803 INFO [org.ovirt.engine.core.bll.VdsInstaller] (http-0.0.0.0-8080-8) Installation of 192.168.1.20. successfully done sftp operation ( Stage: Downloading certificate request from Host) 2011-11-29 20:07:32,803 INFO [org.ovirt.engine.core.utils.hostinstall.MinaInstallWrapper] (http-0.0.0.0-8080-8) return true 2011-11-29 20:07:32,804 INFO [org.ovirt.engine.core.bll.VdsInstaller] (http-0.0.0.0-8080-8) DownloadCertificateRequest ended:true 2011-11-29 20:07:32,804 INFO [org.ovirt.engine.core.bll.CBCInstaller] (http-0.0.0.0-8080-8) Installation of 192.168.1.20. Executing oVirt installation stage. (Stage: Sign certificate request and generate certificate) 2011-11-29 20:07:33,855 ERROR [org.ovirt.engine.core.utils.hostinstall.OpenSslCAWrapper] (http-0.0.0.0-8080-8) Sign Certificate request failed with exit code 9 2011-11-29 20:07:33,857 ERROR [org.ovirt.engine.core.utils.hostinstall.OpenSslCAWrapper] (http-0.0.0.0-8080-8) Sign Certificate request script errors: null/etc/pki/engine/ca//SignReq.sh: line 79: /var/lock/engine/.openssl.exclusivelock: No such file or directory 2011-11-29 20:07:33,857 INFO [org.ovirt.engine.core.bll.VdsInstaller] (http-0.0.0.0-8080-8) SignCertificateRequest ended:false 2011-11-29 20:07:33,857 ERROR [org.ovirt.engine.core.bll.VdsInstaller] (http-0.0.0.0-8080-8) Error signing certificate request 2011-11-29 20:07:33,857 ERROR [org.ovirt.engine.core.bll.VdsInstaller] (http-0.0.0.0-8080-8) Installation of 192.168.1.20. Operation failure. (Stage: Sign certificate request and generate certificate) 2011-11-29 20:07:33,857 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (http-0.0.0.0-8080-8) After Installation http-0.0.0.0-8080-8 2011-11-29 20:07:33,859 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (http-0.0.0.0-8080-8) START, SetVdsStatusVDSCommand(vdsId = 016dc072-1aed-11e1-bbd0-00265a767e20, status=InstallFailed, nonOperationalReason=NONE), log id: 4f88134f Thanks Douglas Hi Douglas, You need to generate a folder called "/var/lock/engine" and allow write permissions to jboss user (the account jboss is running as). Hello Doron, Thanks for your previous comment, it resolved the problem. However, right now I am facing a different situation, I can register an oVirt Node, Create Data Storage and Iso storage, everything is UP here but when I try to upload a ISO to my ISO Storage the path seems not correct, for example: It created: /ovirtISO/70d2dd41-3155-4850-a61c-63f8885c47a6/images shouldn't be? /ovirtISO/70d2dd41-3155-4850-a61c-63f8885c47a6/images/11111111-1111-1111-1111-111111111111/ Even creating it manually, setting right permissions and uploading the iso , it doesn't work... I cannot create a virtual machine to execute a test. Can you please have a look? Here the logs: 2011-12-01 02:17:49,920 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (http-0.0.0.0-8080-3) Error code GetIsoListError and error message VDSGenericException: VDSErrorException: Failed to HsmGetIsoListVDS, error = Cannot get ISO list: ('c5ce633e-0f00-11e1-982d-00265a767e20',) 2011-12-01 02:17:49,921 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (http-0.0.0.0-8080-3) Command org.ovirt.engine.core.vdsbroker.vdsbroker.HsmGetIsoListVDSCommand return value Class Name: org.ovirt.engine.core.vdsbroker.irsbroker.IsoListReturnForXmlRpc mVMList Null mStatus Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc mCode 314 mMessage Cannot get ISO list: ('c5ce633e-0f00-11e1-982d-00265a767e20',) Thanks Douglas Hi, To setup the environment that I can register oVirt Node I have used the below steps, if anyone is interested to test as well and provide a feedback: - Executed the following procedures on oVirt engine: http://ovirt.org/wiki/Engine_Node_Integration#Engine_core_machine - Added this change to ovirt Node image: https://bugzilla.redhat.com/show_bug.cgi?id=756136 - Used the following patches into vdsm (still working on it): =================================================================== * replace /rhev repository to /ovirt http://gerrit.ovirt.org/#change,449 * Make /var/lib/vdsm read and write, otherwise it will break vdsm_reg http://gerrit.ovirt.org/#change,301 (already merged) * deployUtil: isOvirt() check oVirt release http://gerrit.ovirt.org/452 * storage/misc.py: Add validation to oVirt Node http://gerrit.ovirt.org/#change,453 * vdsm/caps.py: getos() include oVirt Node http://gerrit.ovirt.org/#change,454 * vdsm-reg: change registration servlet name http://gerrit.ovirt.org/#change,310 * vds_boostrap: rename rhevm to engine http://gerrit.ovirt.org/#change,311 * [WIP] scripts: Add oVirt validation http://gerrit.ovirt.org/455 I have recreated the ISO Domain and it's working normally. I do believe that last night I have created a Data Domain as ISO Domain, that's why I got the previous error to uploade the ISO. Right now I will continue my tests with virtual machines, as soon as I have more news I will provide here. Good news, I got a virtual machine running. Next week I will work to get the patches finsihed and merged into vdsm tree. Also, I will check any remaining problem to get vdsm working automatically. To test if the Virtual machine is working I have used spice only (yum install spice-xpi -y) + firefox. Created the wiki page: http://ovirt.org/wiki/Opening_guests Here output from vdsClient: # vdsClient 192.168.1.30 list e338bd30-e8f0-4041-8c58-0635b507c2a3 Status = Up cdrom = /ovirt/data-center/0c55ad0e-1d0b-11e1-a004-00265a767e20/8c2bb911-55b3-45eb-a1e2-274b9a22db8f/images/11111111-1111-1111-1111-111111111111/rhel-server-6.0-x86_64-dvd.iso bridge = engine acpiEnable = true emulatedMachine = pc-0.14 transparentHugePages = true displaySecurePort = -1 timeOffset = 0 cpuType = Penryn smp = 1 pauseCode = NOERR macAddr = 00:1a:4a:16:01:51 boot = d custom = {} vmType = kvm memSize = 512 smpCoresPerSocket = 1 vmName = supervm spiceMonitors = 1 nice = 0 pid = 29506 displayIp = 0 drives = [{'index': '0', 'domainID': '87bfc99e-8870-4046-9f99-da20550152d0', 'apparentsize': '5368709120', 'format': 'raw', 'bus': '0', 'boot': 'true', 'volumeID': '9905b22e-4a74-4855-bf7b-98270ed091f1', 'imageID': 'd879c5dd-f2be-459f-9661-c63b123d97fe', 'blockDev': False, 'propagateErrors': 'off', 'poolID': '0c55ad0e-1d0b-11e1-a004-00265a767e20', 'path': '/ovirt/data-center/0c55ad0e-1d0b-11e1-a004-00265a767e20/87bfc99e-8870-4046-9f99-da20550152d0/images/d879c5dd-f2be-459f-9661-c63b123d97fe/9905b22e-4a74-4855-bf7b-98270ed091f1', 'serial': '9f-9661-c63b123d97fe', 'truesize': '1250177024', 'if': 'virtio'}] displayPort = 5900 nicModel = pv keyboardLayout = en-us kvmEnable = true clientIp = display = qxl Adding one more patch to track: http://gerrit.ovirt.org/#change,492 All code that is required from vdsm side to register ovirt-node into ovirt-engine is available at upstream project tree (http://gerrit.ovirt.org/p/vdsm.git) From ovirt-engine side: - engine-core: fix context and registration servlet name http://gerrit.ovirt.org/#change,318 - rename bridge from 'engine'to 'ovirtmgmt' Thanks |