RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1278637 - Support for encrypted Hyper-V connection in virt-who
Summary: Support for encrypted Hyper-V connection in virt-who
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: virt-who
Version: 7.1
Hardware: All
OS: Linux
urgent
urgent
Target Milestone: rc
: 7.1
Assignee: Radek Novacek
QA Contact: Eko
Yehuda Zimmerman
URL:
Whiteboard:
: 1167283 (view as bug list)
Depends On: 1291737
Blocks: 1172231 1203710 1272873 1298337 1313485
TreeView+ depends on / blocked
 
Reported: 2015-11-06 04:02 UTC by Anand Vaddarapu
Modified: 2019-10-10 10:27 UTC (History)
14 users (show)

Fixed In Version:
Doc Type: Enhancement
Doc Text:
Encrypted Hyper-V connections supported in _virt-who_ Previously, _virt-who_ used unencrypted Hyper-V connections. All data was sent in plain text. This had security implications and needed special configuration on Hyper-V servers to be allowed. With this update, _virt-who_ now uses Windows NT LAN Manager (NTLM) sealing and signing to protect communication with Hyper-V servers.
Clone Of:
Environment:
Last Closed: 2016-11-04 05:06:50 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
Disallowed auth with GPO.png (74.70 KB, image/png)
2016-02-02 03:57 UTC, Liushihui
no flags Details
Disallowed auth without GPO.png (71.76 KB, image/png)
2016-02-02 03:59 UTC, Liushihui
no flags Details
Allow auth with GPO.png (70.36 KB, image/png)
2016-02-02 03:59 UTC, Liushihui
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Red Hat Knowledge Base (Solution) 1261263 0 None None None 2018-12-09 19:14:26 UTC
Red Hat Product Errata RHBA-2016:2387 0 normal SHIPPED_LIVE virt-who bug fix and enhancement update 2016-11-03 13:53:39 UTC

Description Anand Vaddarapu 2015-11-06 04:02:13 UTC
Description of problem:

virt-who configs:

#cat /etc/virt-who.d/hyperv_config
[hypervname]
type=hyperv
#server=hyperV-FQDN
server=http://10.0.0.0
username=username
encrypted_password=<encrypted using virt-who-password>
owner=OWNER
env=GA-Static-Patching-RHEL-7
hypervisor_id=hostname

>winrm get winrm/config/service
Service
    RootSDDL = XXXXX
    MaxConcurrentOperations = 4294967295
    MaxConcurrentOperationsPerUser = 1500
    EnumerationTimeoutms = 240000
    MaxConnections = 300
    MaxPacketRetrievalTimeSeconds = 120
    AllowUnencrypted = true [Source="GPO"]
    Auth
        Basic = false [Source="GPO"]
        Kerberos = true
        Negotiate = true [Source="GPO"]
        Certificate = false
        CredSSP = true
        CbtHardeningLevel = Relaxed
    DefaultPorts
        HTTP = 5985
        HTTPS = 5986
    IPv4Filter = * [Source="GPO"]
    IPv6Filter = * [Source="GPO"]
    EnableCompatibilityHttpListener = true
    EnableCompatibilityHttpsListener = true
    CertificateThumbprint
    AllowRemoteAccess = true [Source="GPO"]

virt-who -o -d
ERROR:root:Configuration file katello-ca-consumer-latest.noarch.rpm contains no section headers
2015-11-05 15:33:57,762 DEBUG: Using config named 'hyperv-hostname1'
2015-11-05 15:33:57,762 DEBUG: Using config named 'hyperv-hostname'
2015-11-05 15:33:57,763 INFO: Using configuration "hyperv-hostname1" ("hyperv" mode)
2015-11-05 15:33:57,763 INFO: Using configuration "hyperv-hostname2" ("hyperv" mode)
2015-11-05 15:33:57,829 DEBUG: Hyper-V url: http://10.0.0.0:5985/wsman
2015-11-05 15:33:57,833 DEBUG: Hyper-V url: http://10.0.0.0:5985/wsman
2015-11-05 15:33:57,841 DEBUG: Using NTLM authentication
2015-11-05 15:33:57,843 DEBUG: Using NTLM authentication
2015-11-05 15:33:57,887 ERROR: Virt backend 'hyperv-hostname1' fails with error: NTLM negotiation failed
2015-11-05 15:33:57,890 ERROR: Virt backend 'hyperv-hostname2' fails with error: NTLM negotiation failed

Version-Release number of selected component (if applicable):


How reproducible:
1. Install laste virt-who virt-who-0.14-4.el7.noarch.rpm
2. Configure virt-who with above configs
3. run virt-who -o -d to check autentication

Actual results:
ERROR: Virt backend 'hyperv-hostname2' fails with error: NTLM negotiation failed

Expected results:
Authentication should succeed.

Additional info:
Hyper-V are part of GPO (Microsoft Group Policy objects). When customer removes servers from GPO, virt-who can authenticate successfully.

Comment 1 Liushihui 2015-11-06 04:38:16 UTC
Please notice that: when Hyper-V is not part of GPO policy, virt-who can authenticate successfully only after set "AllowUnencrypted = false", However, if set "AllowUnencrypted = true",it still show " NTLM negotiation failed"

Comment 3 Radek Novacek 2015-11-24 07:34:30 UTC
Please, can you tell the customer to check if following setting in the Group Policy is set to "Disabled"

Computer > Policies > Administrative Templates > Windows Components > Windows Remote Management > WinRM Service: Disallow Negotiate Authentication.

Thanks.

Comment 7 Radek Novacek 2015-12-02 08:07:02 UTC
Ok, thanks. Did the customer try suggestion from comment #3?

Comment 10 Radek Novacek 2015-12-08 07:09:58 UTC
Jay,

looking at that screenshot, there is "Allow unencrypted traffic" option right above "Disallow Negotiate Authentication" and it is set to Disabled. virt-who is using unencrypted communication to talk to Hyper-V. Can the customer try to allow it?

Comment 13 Radek Novacek 2015-12-10 07:40:11 UTC
Anand,

can you please clarify what config options need to be changed in order to make virt-who work? I can work on finding some workaround but I need to know precisely what options in their GPO policy are not compatible with current virt-who requirements.

Comment 14 Radek Novacek 2015-12-10 07:47:17 UTC
Could the "AllowUnencrypted" option be the problem? virt-who needs this to be set to "true". In the bug description, this options is "true". In comment #4, there is "AllowUnencrypted = true [Source="GPO"]", but in comment #11, there is "AllowUnencrypted = false [Source="GPO"]"

Which one is customers original GPO setting?

Comment 16 Eko 2015-12-10 09:51:28 UTC
Hi Radek, I have set up an environment to reproduce this issue, my test result as following:
if set AllowUnencrypted = false [Source="GPO"], virt-who Failed;
if set AllowUnencrypted = true [Source="GPO"], virt-who pass;
if set AllowUnencrypted = true, virt-who pass;

so maybe it's not a GPO issue, the problem is how to make virt-who can work normally with AllowUnencrypted = false?

Comment 17 Radek Novacek 2015-12-10 11:54:43 UTC
Thanks, I'll take a look what would it need to support encrypted Hyper-V connection in virt-who.

Comment 21 Radek Novacek 2015-12-22 07:06:28 UTC
Anand, 

the workaround it to enable unencrypted connections. I'll give it a try to implement it in January. If I succeed, we can do a z-stream fix for this issue.

Comment 30 Liushihui 2016-02-02 03:56:11 UTC
In the latest In the latest virt-who-0.16-2.el6.noarch. test with the following three GPS settings, virt-who can get through NTLM authentication and send host/guest mapping to server successfully.

1. set AllowUnencrypted = false [Source="GPO"], virt-who pass;see the screenshot "Disallowed auth with GPO.png"
 "AllowUnencrypted = false [Source="GPO"]
    Auth
        Basic = false [Source="GPO"]
        Negotiate = true [Source="GPO"]
        IPv4Filter = * [Source="GPO"]
        IPv6Filter = * [Source="GPO"]
        AllowRemoteAccess = true [Source="GPO"] "

2. set AllowUnencrypted = false, without GPO, virt-who pass;see the screenshot "Disallowed auth without GPO.png"
 "AllowUnencrypted = false
    Auth
        Basic = false [Source="GPO"]
        Negotiate = true
        IPv4Filter = * [Source="GPO"]
        IPv6Filter = * [Source="GPO"]
        AllowRemoteAccess = true [Source="GPO"] "

3. set AllowUnencrypted = true [Source="GPO"], virt-who pass; see the screenshot "allow auth with GPO.png"
AllowUnencrypted = true [Source="GPO"]
    Auth
        Basic = false [Source="GPO"]
        Kerberos = true
        Negotiate = true [Source="GPO"]
    IPv4Filter = * [Source="GPO"]
    IPv6Filter = * [Source="GPO"]
    AllowRemoteAccess = true [Source="GPO"]

Comment 31 Liushihui 2016-02-02 03:57:33 UTC
Created attachment 1120307 [details]
Disallowed auth with GPO.png

Comment 32 Liushihui 2016-02-02 03:59:10 UTC
Created attachment 1120308 [details]
Disallowed auth without GPO.png

Comment 33 Liushihui 2016-02-02 03:59:49 UTC
Created attachment 1120309 [details]
Allow auth with GPO.png

Comment 37 Radek Novacek 2016-02-23 08:19:46 UTC
Anand, we can't just release it. We have to follow the process. That means to follow a z-stream process. But I don't think this bug qualifies for z-stream. This is clearly a new feature request. z-stream is meant to be for critical bug fixes only.

There is a risk that this change will break deployments for existing customers because it substantially changes how virt-who connects to hyper-v. I think we should wait for RHEL 7.3 GA.

If the customer really needs to have this ASAP, feel free to investigate options we have (z-stream, hotfix, async, etc.).

Comment 42 Radek Novacek 2016-03-08 06:50:15 UTC
The encrypted connection is already implemented upstream and will be resolved by rebase in 7.3. I will do the hotfix process ASAP.

Comment 44 Radek Novacek 2016-03-08 14:03:06 UTC
Hotfix build done: https://brewweb.devel.redhat.com/taskinfo?taskID=10620147

Comment 46 Radek Novacek 2016-03-09 07:44:57 UTC
Yes, although I would prefer QE team to test it first.

Comment 47 Liushihui 2016-03-10 04:38:07 UTC
Verified it on virt-who-0.14-9.el7.0.0.hotfix.1.bz1278637.noarch since virt-who can get through NTLM authentication and send host/guest mapping to server successfully when set "AllowUnencrypted = false" or "AllowUnencrypted = true".Meanwhile, guest can subscribe bonus pool successfully after hypervisor subscribe physical pool.

Checked version:
virt-who-0.14-9.el7.0.0.hotfix.1.bz1278637.noarch
subscription-manager-1.15.9-15.el7.x86_64
python-rhsm-1.15.4-5.el7.x86_64

Checked process:
1. Update virt-who version to the hostfix version
[root@hp-xl220agen8v2-01 ~]# rpm -q virt-who
virt-who-0.14-9.el7.noarch
[root@hp-xl220agen8v2-01 ~]# yum install -y python-requests
[root@hp-xl220agen8v2-01 ~]# rpm -Uvh virt-who-0.14-9.el7.0.0.hotfix.1.bz1278637.noarch.rpm 
Preparing...                          ################################# [100%]
Updating / installing...
   1:virt-who-0.14-9.el7.0.0.hotfix.1.################################# [ 50%]
Cleaning up / removing...
   2:virt-who-0.14-9.el7              ################################# [100%]
[root@hp-xl220agen8v2-01 ~]# rpm -q virt-who
virt-who-0.14-9.el7.0.0.hotfix.1.bz1278637.noarch
2. Register system to satellite6.1
3. In hyperv, configure local group policy as the following three conditions.
Config1: set AllowUnencrypted = false [Source="GPO"], virt-who pass;see the screenshot "Disallowed auth with GPO.png"
 "AllowUnencrypted = false [Source="GPO"]
    Auth
        Basic = false [Source="GPO"]
        Negotiate = true [Source="GPO"]
        IPv4Filter = * [Source="GPO"]
        IPv6Filter = * [Source="GPO"]
        AllowRemoteAccess = true [Source="GPO"] "
Config2. set AllowUnencrypted = false, without GPO, virt-who pass;see the screenshot "Disallowed auth without GPO.png"
 "AllowUnencrypted = false
    Auth
        Basic = false [Source="GPO"]
        Negotiate = true
        IPv4Filter = * [Source="GPO"]
        IPv6Filter = * [Source="GPO"]
        AllowRemoteAccess = true [Source="GPO"] "
Config3. set AllowUnencrypted = true [Source="GPO"], virt-who pass; see the screenshot "allow auth with GPO.png"
AllowUnencrypted = true [Source="GPO"]
    Auth
        Basic = false [Source="GPO"]
        Kerberos = true
        Negotiate = true [Source="GPO"]
    IPv4Filter = * [Source="GPO"]
    IPv6Filter = * [Source="GPO"]
    AllowRemoteAccess = true [Source="GPO"]
4. Configure virt-who run at hyperv mode ,restart virt-who and check virt-who's log
[root@hp-xl220agen8v2-01 ~]# cat /etc/virt-who.d/virt 
[test-hyperv1]
type=hyperv
server=10.73.5.227
username=administrator
password=Welcome1
owner=ACME_Corporation
env=Library 
[root@hp-xl220agen8v2-01 ~]# service virt-who restart && tail -f /var/log/rhsm/rhsm.log
2016-03-09 23:32:37,320 [INFO]  @virtwho.py:697 - Using configuration "test-hyperv1" ("hyperv" mode)
2016-03-09 23:32:37,320 [DEBUG]  @virtwho.py:216 - Starting infinite loop with 5 seconds interval
2016-03-09 23:32:37,358 [DEBUG]  @hyperv.py:477 - Hyper-V url: http://10.73.5.227:5985/wsman
2016-03-09 23:32:38,660 [DEBUG]  @hyperv.py:71 - Using NTLM authentication
2016-03-09 23:32:39,979 [DEBUG]  @hyperv.py:84 - Sending NTLM authentication data
2016-03-09 23:32:40,585 [DEBUG]  @hyperv.py:107 - NTLM authentication successful
2016-03-09 23:32:40,590 [DEBUG]  @hyperv.py:511 - Unable to enumerate using root/virtualization namespace, trying root/virtualization/v2 namespace
2016-03-09 23:32:44,461 [DEBUG]  @virt.py:343 - Getting the host/guests association took too long, interval waiting is skipped
2016-03-09 23:32:44,463 [DEBUG]  @subscriptionmanager.py:112 - Authenticating with certificate: /etc/pki/consumer/cert.pem
2016-03-09 23:32:44,589 [DEBUG]  @subscriptionmanager.py:146 - Checking if server has capability 'hypervisor_async'
2016-03-09 23:32:44,709 [DEBUG]  @subscriptionmanager.py:158 - Server does not have 'hypervisors_async' capability
2016-03-09 23:32:44,710 [INFO]  @subscriptionmanager.py:165 - Sending update in hosts-to-guests mapping: {
    "hyperv_01": [
        {
            "guestId": "32710A7E-94A9-A445-944E-16C01BFA63B3", 
            "state": 1, 
            "attributes": {
                "active": 1, 
                "virtWhoType": "hyperv", 
                "hypervisorType": "hyperv"
            }
        }, 
        {
            "guestId": "0E32F0E5-05CA-014A-BD59-F63D75843D5D", 
            "state": 1, 
            "attributes": {
                "active": 1, 
                "virtWhoType": "hyperv", 
                "hypervisorType": "hyperv"
            }
        }
    ]
}

5. In satellite webUI, Go to "content host" --> choose the [hyperv_hostnae]-->"Subscriptions" --> "Add", choose physical pool which can generate bonus pool on hypervisor ,then subscribe it.
6. In the Guest, list the bonus pool and subscribe the bonus pool

Result:
Virt-who send correct host/guest mapping info to satellite, guest can subscribe bonus pool successfully.

Comment 49 Radek Novacek 2016-03-10 06:41:22 UTC
Anand, yes, you can provide the package to the customer.

Comment 50 Eko 2016-04-27 05:40:35 UTC
*** Bug 1167283 has been marked as a duplicate of this bug. ***

Comment 51 Eko 2016-04-27 05:42:49 UTC
Verified it on virt-who-0.14-9.el7.0.0.hotfix.1.bz1278637.noarch,
and can't reproduce it in in virt-who-0.16-8.el6.noarch

Comment 53 Radek Novacek 2016-09-29 14:36:22 UTC
The updated Doc Text is fine. Thanks.

Comment 55 errata-xmlrpc 2016-11-04 05:06:50 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2016-2387.html


Note You need to log in before you can comment on or make changes to this bug.