Bug 807423

We have had some speculations on this, and I am afraid there is not much we can do from TCMS side at this stage.

Assumed that we are supporting authentication using the said principle approach, which in actuality is easy to implement btw, the first problem we'll be confronting is the relationship between this principle identify and TCMS data. That known, TCMS data is user-centric, accessing the data will have to provide a specific user identity. This user identify is not merely a start point of filtering the relevant data, also it is vital in terms of permission control.

In order to do that, either you agree with this principle name being a TCMS-wise user identify, in which case TCMS will clone/create data for this 'user' as a shared resource for your team to access, or this principle-based authentication mechanism is somehow able to inform TCMS a user identify besides the successful authentication for your automation environment host in a hacked fashion.

The former seemed workable, but there are some obvious downsides about it. First, since this principle is a shared identify being used by a team of users to access shared testing data, it is unpredictable that the data should be reliable. Second, the permission control on the data is becoming unuseful. Anyone can create/update/delete the shared data. This is like hell to go.

The latter is what can be hacked into TCMS to support the authentication on a host using principle. With the successful authentication on a host, say your automation environment, instead of using the principle name as the username, we use a dedicated argument passed in when creating a TCMS XMLRPC connection, say the argument name is 'username', as the username to ID the accessor. While this is actually possible and we have talked internally about its practicality, this is not an orthodox way to go, not to mention that it also has security blackhole. Red Hat has been using kerberos for most of our internal application authentications, it explicitly differentiates two scenarios, one for kerberos user that is to authenticate a regular Red Hat employee, the other for the machines/hosts using the said principle. By doing what's explained before is against this tradition, this is equivalent to bypassing the kerberos authentication, because in this case anyone has the access to your shared host will have access to TCMS(or Beaker, Portal ...), this is not allowed and should not be encouraged, not to mention that this is not our decision to make, but a guideline for us to follow.

However, we are not at a standing point that your request is unreasonable. Actually, your request has some generic aspects that all our hosted internal applications should think about, that is how to provide open web service in a kerberos defended environment. Realizing this, I have talked with other developers working on different applications, not surprisingly, they have had the identical issue. Unfortunately there are not good practices in solving this problem we have discovered yet.

Then we think back about your automations, isn't that the ticket cache is valid at least for 10 hours in default, and this is further configurable, which means that as long as your job is periodic and can be completed in a time frame of 10 hours, your tester can manually invoke kinit to renew the cache, instead of dangerously literally storing their kerberos credentials. Also, it is possible to scp the ticket cache across machines. Worstly, if all that turned impossible for you somehow, using proper encryption for the stored credentials also will also help greatly.

(In reply to comment #1)
> Assumed that we are supporting authentication using the said principle
> approach, which in actuality is easy to implement btw, the first problem
> we'll be confronting is the relationship between this principle identify and
> TCMS data. That known, TCMS data is user-centric, accessing the data will
> have to provide a specific user identity. This user identify is not merely a
> start point of filtering the relevant data, also it is vital in terms of
> permission control.
> 
> In order to do that, either you agree with this principle name being a
> TCMS-wise user identify, in which case TCMS will clone/create data for this
> 'user' as a shared resource for your team to access, 

This means it will own test-cases,plans and runs and will be on par with other users and subject to TCMS' access control? That is acceptable for BaseOS QE automation. 

And it is even a preferred solution: it's cleaner, more transparent and more secure. 

Other users should be granted/denied access to objects owned by automation via ACL. Will that be possible with the reworked permission in TCMSv4?. In the meantime (TCMSv3), read-only access for everyone and full access via admin rights for automation admins will suffice.

> The latter is what can be hacked into TCMS to support the authentication on
> a host using principle. With the successful authentication on a host, say
> your automation environment, instead of using the principle name as the
> username, we use a dedicated argument passed in when creating a TCMS XMLRPC
> connection, say the argument name is 'username', as the username to ID the
> accessor. 

While I understand the need for such type of access, it is currently not a requirement for BaseOS QE automation(*).

> Then we think back about your automations, isn't that the ticket cache is
> valid at least for 10 hours in default, and this is further configurable,
> which means that as long as your job is periodic and can be completed in a
> time frame of 10 hours, your tester can manually invoke kinit to renew the
> cache, instead of dangerously literally storing their kerberos credentials.
> Also, it is possible to scp the ticket cache across machines. Worstly, if
> all that turned impossible for you somehow, using proper encryption for the
> stored credentials also will also help greatly.

This won't work for us because:

 a) automation have to be have more than one maintainer to ensure continuous operation
 b) using ones personal kerberos principal on a shared machine is unacceptable (no matter the method of distribution - direct kinit, scp,...)
c)


(*) CCing the other BaseOS QE auotomation admins, in case I've missed something.

Hello,
 
> your job is periodic and can be completed in a
> time frame of 10 hours, your tester can manually invoke kinit to renew the
> cache

our automated workflow is following:
1, download thousands test cases from tcms, ~ 5hour
2, start it on beaker ~ 5-24 hours
3, report results into tcms ~ 1 minute
---
* that's why we need to be authenticated more than 10 hours
* manually intervention is impossible 

We need a way how to do it without using our personal kerberos principal.

I agree with what Ales said in Comment 2, having the service principal as a user would work for us. There's nothing wrong with that user creating test runs, etc.

"First, since this principle is a shared identify being used by a team of users to access shared testing data, it is unpredictable that the data should be reliable."

The data would represent the test flows from a service. We could see which service added the test run (ATOM, Jenkins, etc.) from the principal name and it would be the responsibility of the service operators to include information about the tests, like who started it, or what triggered the test run.

"Second, the permission control on the data is becoming unuseful. Anyone can create/update/delete the shared data. This is like hell to go."

If you're worried about the user being a shared account, you could restrict its permissions. For example it could only add new test runs, but not delete or modify them, and not touching anything else either.

My vote is on the "service principal as a user" solution as well.

A.

We have made certain changes in the TCMS authentication backend to accommodate situation where the authentication is based on keytab.

But be warned that the assumptions are made, and it is well specified in the comment #2.

This is going alive in the coming release of 3.8.

Verify 3.8.0-1 on stage -->PASS

Verify steps:
1. Login with Kerberos service principal account.

  $ kinit -k -t /home/tcmsfile/filename HTTP/REALM

2. enter into python dev env:
    $ python
    a. import natrite:
        >>from nitrate import NitrateKerbXmlrpc
    b. Specify server:
         >>n=NitrateKerbXmlrpc('XXX').server
note: 'XXX' is TCMS env, such as: 'https://tcms.englab.nay.redhat.com/'.
3. Call one or more meths.
    such as, 
    $ n.TestRun.get(21981)
4. Call following command,
   $ User.get_me()

Actual result:
In this way, user don't need any user's password. And username should be Kerberos service principal account.

Hi, I'm getting an internal server error when trying the above-mentioned test:

$ klist
Ticket cache: FILE:/tmp/krb5cc_502
Default principal: auto/liver.englab.brq.redhat.com

Valid starting     Expires            Service principal
08/21/12 18:37:34  08/22/12 04:37:34  krbtgt/REDHAT.COM
	renew until 08/21/12 18:37:34
08/21/12 18:38:20  08/22/12 04:37:34  HTTP/tcms.app.eng.bos.redhat.com
	renew until 08/21/12 18:37:34

$ python
Python 2.6.6 (r266:84292, May  1 2012, 13:52:17) 
[GCC 4.4.6 20110731 (Red Hat 4.4.6-3)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from nitrate import NitrateKerbXmlrpc
>>> n=NitrateKerbXmlrpc('https://tcms.engineering.redhat.com').server
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.6/site-packages/nitrate/xmlrpc.py", line 508, in __init__
    login_dict = self.do_command("Auth.login_krbv", [])
  File "/usr/lib/python2.6/site-packages/nitrate/xmlrpc.py", line 461, in do_command
    raise NitrateXmlrpcError(verb, params, e)
nitrate.xmlrpc.NitrateXmlrpcError: Error while executing cmd 'Auth.login_krbv()' --> <ProtocolError for tcms.engineering.redhat.com/RPC2: 500 INTERNAL SERVER ERROR>
>>>

(In reply to comment #7)
> Hi, I'm getting an internal server error when trying the above-mentioned
> test:
> 
Hi Ales,

Is this a happened-once error or your principal never works well?

(In reply to comment #7)
> Hi, I'm getting an internal server error when trying the above-mentioned
> test:
> 
> $ klist
> Ticket cache: FILE:/tmp/krb5cc_502
> Default principal: auto/liver.englab.brq.redhat.com
> 
> Valid starting     Expires            Service principal
> 08/21/12 18:37:34  08/22/12 04:37:34  krbtgt/REDHAT.COM
> 	renew until 08/21/12 18:37:34
> 08/21/12 18:38:20  08/22/12 04:37:34 
> HTTP/tcms.app.eng.bos.redhat.com
> 	renew until 08/21/12 18:37:34
> 
> $ python
> Python 2.6.6 (r266:84292, May  1 2012, 13:52:17) 
> [GCC 4.4.6 20110731 (Red Hat 4.4.6-3)] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
> >>> from nitrate import NitrateKerbXmlrpc
> >>> n=NitrateKerbXmlrpc('https://tcms.engineering.redhat.com').server

I believe the right url you should use is 'https://tcms.engineering.redhat.com/xmlrpc/'.
So that's why it cause a error.

> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
>   File "/usr/lib/python2.6/site-packages/nitrate/xmlrpc.py", line 508, in
> __init__
>     login_dict = self.do_command("Auth.login_krbv", [])
>   File "/usr/lib/python2.6/site-packages/nitrate/xmlrpc.py", line 461, in
> do_command
>     raise NitrateXmlrpcError(verb, params, e)
> nitrate.xmlrpc.NitrateXmlrpcError: Error while executing cmd
> 'Auth.login_krbv()' --> <ProtocolError for tcms.engineering.redhat.com/RPC2:
> 500 INTERNAL SERVER ERROR>
> >>>

I get an ISE with the xmlrpc in URL as well:

>>> n=NitrateKerbXmlrpc('https://tcms.engineering.redhat.com/xmlrpc/').server
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.6/site-packages/nitrate/xmlrpc.py", line 508, in __init__
    login_dict = self.do_command("Auth.login_krbv", [])
  File "/usr/lib/python2.6/site-packages/nitrate/xmlrpc.py", line 461, in do_command
    raise NitrateXmlrpcError(verb, params, e)
nitrate.xmlrpc.NitrateXmlrpcError: Error while executing cmd 'Auth.login_krbv()' --> <ProtocolError for tcms.engineering.redhat.com/xmlrpc/: 500 INTERNAL SERVER ERROR>

(In reply to comment #8)
> Is this a happened-once error or your principal never works well?

Unfortunately, it's never worked, I always get the ISE.

Thanks for looking into this.

FYI: I've tested it against TCMS stage, as instructed by Yawei Li, and it works well:

>>> n=NitrateKerbXmlrpc('https://tcms-stage.englab.bne.redhat.com/xmlrpc/').server
>>> n.User.get_me()
{'username': 'liver.englab.brq.redhat.com', 'first_name': '', 'last_name': '', 'is_active': True, 'id': 3662, 'is_superuser': False, 'is_staff': False, 'last_login': '2012-08-29 16:43:03', 'groups': [3], 'user_permissions': [], 'email': 'liver.englab.brq.redhat.com', 'date_joined': '2012-08-29 16:41:49'}
>>> n.TestRun.get(21981)
{'environment_id': 0, 'run_id': 21981, 'estimated_time': '00:00:00', 'product_version': '3.4', 'manager': 'jzhao', 'tag': [], 'build_id': 1358, 'plan_id': 3677, 'cc': [], 'build': '3.4.1', 'start_date': '2011-06-07 10:08:03', 'manager_id': 2778, 'errata_id': None, 'stop_date': None, 'auto_update_run_status': False, 'plan': 'Test Plan for TCMS 3.4(Full function)', 'plan_text_version': 2, 'default_tester': 'xgao', 'env_value': [], 'notes': '', 'summary': 'Test run for Acceptance(hotfix) - TCMS 3.4.1 - stage', 'case_run_status': '2:100,3:9', 'default_tester_id': 2923}

Eng-ops and TCMS devs are now investigating why are prod and stage behaving differently.

Kerberos Principle authentication works well on TCMS production now, after eng-ops engineer helped to update the configuration.

Please help to verify it. Thanks.

@Ales,
Hi Ales, please help to verify this problem when y're online and free.
Thanks very much.

(In reply to comment #12)
> Kerberos Principle authentication works well on TCMS production now, after
> eng-ops engineer helped to update the configuration.
> 
> Please help to verify it. Thanks.

It works:

>>>> from nitrate import NitrateKerbXmlrpc
>>>> n=NitrateKerbXmlrpc('https://tcms.engineering.redhat.com/xmlrpc/').server
>>>> n.User.get_me()
> {'username': 'liver.englab.brq.redhat.com', 'first_name': '', 'last_name': '', 'is_active': True, 'id': 3737, 'is_superuser': False, 'is_staff': False, 'last_login': '2012-09-17 19:32:07', 'groups': [3], 'user_permissions': [], 'email': 'liver.englab.brq.redhat.com', 'date_joined': '2012-09-17 19:31:53'}
>>>> n.TestRun.get(21981)
> {'environment_id': 0, 'run_id': 21981, 'estimated_time': '00:00:00', 'product_version': '3.4', 'manager': 'jzhao', 'tag': [], 'build_id': 1358, 'plan_id': 3677, 'cc': [], 'build': '3.4.1', 'start_date': '2011-06-07 10:08:03', 'manager_id': 2778, 'errata_id': None, 'stop_date': None, 'auto_update_run_status': False, 'plan': 'Test Plan for TCMS 3.4(Full function)', 'plan_text_version': 2, 'default_tester': 'xgao', 'env_value': [], 'notes': '', 'summary': 'Test run for Acceptance(hotfix) - TCMS 3.4.1 - stage', 'case_run_status': '2:100,3:9', 'default_tester_id': 2923}
>>>> 
> -bash-4.1$ python
> Python 2.6.6 (r266:84292, Sep 12 2011, 14:03:14) 
> [GCC 4.4.5 20110214 (Red Hat 4.4.5-6)] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
>>>> from nitrate import *
>>>> tp=TestPlan(652)
>>>> for case in tp:
> ...  print case
> ... 
> TC#6144   - Errata Workflow: Summary
> TC#36548  - Errata Workflow: Test urgent bugs
> TC#36549  - Errata Workflow: Test high bugs
> TC#36550  - Errata Workflow: Test medium bugs
> TC#6125   - Errata Workflow: RPMdiff
> TC#6126   - Errata Workflow: TPS
> TC#109967 - Errata Workflow: cppcheck
> TC#6128   - Errata Workflow: Automated tests
> TC#77009  - Errata Workflow: File Lists
> TC#6130   - Errata Workflow: Spec file
> TC#6131   - Errata Workflow: Patches
> TC#6132   - Errata Workflow: Test all bugs by priority
> TC#6134   - Errata Workflow: TCP
> TC#6135   - Errata Workflow: Performance
> TC#6136   - Errata Workflow: Desktop file sanity
> TC#6137   - Errata Workflow: Run as an ordinary user
> TC#6255   - Errata Workflow: Integration
> TC#6140   - Errata Workflow: Localization
> TC#6141   - Errata Workflow: RHN
> TC#6142   - Errata Workflow: Sanity
> TC#6143   - Errata Workflow: Regression


Thank you!

Hello,
I am able to read but not write:
I guess it has some low permission.

>>> a=nitrate.TestCase(65795)
>>> a.tags.add('anything')
>>> a.update
<bound method TestCase.update of TestCase(65795)>
>>> a.update()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.6/site-packages/nitrate/api.py", line 2953, in update
    self.tags.update()
  File "/usr/lib/python2.6/site-packages/nitrate/api.py", line 1301, in update
    self._update()
  File "/usr/lib/python2.6/site-packages/nitrate/api.py", line 1423, in _update
    if added: self._add(added)
  File "/usr/lib/python2.6/site-packages/nitrate/api.py", line 2062, in _add
    self._server.TestCase.add_tag(self.id, list(tags))
  File "/usr/lib64/python2.6/xmlrpclib.py", line 1199, in __call__
    return self.__send(self.__name, args)
  File "/usr/lib64/python2.6/xmlrpclib.py", line 1489, in __request
    verbose=self.__verbose
  File "/usr/lib/python2.6/site-packages/nitrate/xmlrpc.py", line 127, in request_with_cookies
    return self._parse_response(h.getfile(), sock)
  File "/usr/lib64/python2.6/xmlrpclib.py", line 1392, in _parse_response
    return u.close()
  File "/usr/lib64/python2.6/xmlrpclib.py", line 838, in close
    raise Fault(**self._stack[0])
xmlrpclib.Fault: <Fault 1: 'PermissionDenied: Permission denied.'>

(In reply to comment #15)
forget to mention that I used the same ticket as ales in Comment 14

(In reply to comment #16)
> (In reply to comment #15)
> forget to mention that I used the same ticket as ales in Comment 14
ticket to tcms system admin via RT, the RT Queue is eng-ops-appsupport. Request to add permissions for user.

This is the eng ops app support associate, I am attempting to add permissions to the user within the TCMS UI. But unfortunately I am receiving a message where it does not let me proceed with granting the group permissions. Steps are as follows:

1. User Name:  host/atom.qa.lab.tlv.redhat.com

2. User name is found

3. Scroll down to group permissions access to grant Tester and Default

4. Throws a message where I consider it an error: This value may contain only letters, numbers and @/./+/-/_ characters.

5. According to what is written next to the user name: 

Required. 30 characters or fewer. Letters, digits and @/./+/-/_ only, so I believe the "/" should not be an issue that causes not to proceed with granting the permission.

Please note that I have this ticket since 08/08/2012 and the requestor is still waiting for an update.


Thank you,

Zeena Al-juboori
Application Support Associate

(In reply to comment #18)
> This is the eng ops app support associate, I am attempting to add
> permissions to the user within the TCMS UI. But unfortunately I am receiving
> a message where it does not let me proceed with granting the group
> permissions. Steps are as follows:
> 
> 1. User Name:  host/atom.qa.lab.tlv.redhat.com
> 
> 2. User name is found
> 
> 3. Scroll down to group permissions access to grant Tester and Default
> 
> 4. Throws a message where I consider it an error: This value may contain
> only letters, numbers and @/./+/-/_ characters.
> 
> 5. According to what is written next to the user name: 
> 
> Required. 30 characters or fewer. Letters, digits and @/./+/-/_ only, so I
> believe the "/" should not be an issue that causes not to proceed with
> granting the permission.
> 
> Please note that I have this ticket since 08/08/2012 and the requestor is
> still waiting for an update.
> 
> 
> Thank you,
> 
> Zeena Al-juboori
> Application Support Associate
TCMS's User Name length is 30 character or fewer. Because this principle username is beyond 30 characters, you can try search this username:host/atom.qa.lab.tlv.redhat.co.

(In reply to comment #19)
> (In reply to comment #18)
> > This is the eng ops app support associate, I am attempting to add
> > permissions to the user within the TCMS UI. But unfortunately I am receiving
> > a message where it does not let me proceed with granting the group
> > permissions. Steps are as follows:
> > 
> > 1. User Name:  host/atom.qa.lab.tlv.redhat.com
> > 
> > 2. User name is found
> > 
> > 3. Scroll down to group permissions access to grant Tester and Default
> > 
> > 4. Throws a message where I consider it an error: This value may contain
> > only letters, numbers and @/./+/-/_ characters.
> > 
> > 5. According to what is written next to the user name: 
> > 
> > Required. 30 characters or fewer. Letters, digits and @/./+/-/_ only, so I
> > believe the "/" should not be an issue that causes not to proceed with
> > granting the permission.
> > 
> > Please note that I have this ticket since 08/08/2012 and the requestor is
> > still waiting for an update.
> > 
> > 
> > Thank you,
> > 
> > Zeena Al-juboori
> > Application Support Associate
> TCMS's User Name length is 30 character or fewer. Because this principle
> username is beyond 30 characters, you can try search this
> username:host/atom.qa.lab.tlv.redhat.co.

Hi, 

I have tried your suggestion, it gave me the same message "This value may contain only letters, numbers and @/./+/-/_ characters". 

Regards,
Zeena

Actually. the username contains "/" characters is illegal. Those illegality username generate by user use principal authentication before our patch merge in tcms package. Currently TCMS will filter and verify principle username, the next user use principal will generate a legal username, add permissions to new username working now.

It works now, confirmed by customer. the ticket is https://engineering.redhat.com/rt/Ticket/Display.html?id=163989

Verify 3.8.1-1 on tcms-test -->PASS

Verify steps:
1. >>> kinit -k -t tcms-test.app.eng.nay.redhat.com.http.keytab HTTP/tcms-test.app.eng.nay.redhat.com
2. change the permission of tcms-test.app.eng.nay.redhat.com.
2. >>> User.get_me()
3. >>> TestPlan.update(3954, {'name': 'Test Plan for Vault 2.2 test so so'})


Actual result:
1. ====================
{'date_joined': '2012-10-15 16:34:51',
 'email': 'tcms-test.app.eng.nay.redhat.c',
 'first_name': '',
 'groups': [2, 4],
 'id': 3763,
 'is_active': True,
 'is_staff': True,
 'is_superuser': False,
 'last_login': '2012-10-16 11:20:51',
 'last_name': '',
 'user_permissions': [],
 'username': 'tcms-test.app.eng.nay.redhat.c'}

=======================
2. ====================
[{'attachment': [],
  'author': 'jzhao',
  'author_id': 2778,
  'case': [93176,
   93162,
   93163,
   93173,
   93172,
   93171,
   93178,
   93177,
   93159,
   93170,
   93152,
   93154,
   93153,
   93168,
   93164,
   93165,
   93166,
   93157,
   93158,
   93160,
   93155,
   93149,
   93150,
   93183,
   93217,
   93179,
   93161,
   93175,
   93156,
   93167,
   93180,
   93181,
   93151,
   93182,
   93857,
   93859,
   93858,
   93174,
   93169,
   93867,
   93868,
   93864,
   94255,
   93862,
   93863,
   93861,
   93866,
   93865,
   93860,
   93144,
   93147,
   93148,
   93146,
   93021,
   93025,
   93024,
   93023,
   93184,
   93186,
   93185,
   93214,
   93215,
   93211,
   93210,
   93212,
   93216,
   93213,
   93199,
   93200,
   93204,
   93198,
   93194,
   93195,
   93193,
   93197,
   93196,
   93206,
   93205,
   93209,
   93208,
   93207,
   93201,
   93203,
   93202,
   93187,
   93188,
   93192,
   93191,
   93190,
   93189],
  'component': [],
  'create_date': '2011-05-24 21:54:05',
  'default_product_version': '2.2',
  'env_group': [],
  'extra_link': '',
  'is_active': True,
  'name': 'Test Plan for Vault 2.2 test so so',    < ======================
  'owner': 'jzhao',
  'owner_id': 2778,
  'parent': None,
  'parent_id': None,
  'plan_id': 3954,
  'product': 'Vault',
  'product_id': 257,
  'product_version': '2.2',
  'product_version_id': 1094,
  'tag': [],
  'type': 'Function',
  'type_id': 3}]

=======================

Above all, Kerberos Service principal account can do any operation and this account permission can be modified now.