Bug 1466874

Summary: [Intel OSP13] Discover and compose Intel RSD nodes
Product: Red Hat OpenStack Reporter: Krish Raghuram <krishnan.raghuram>
Component: python-rsdclientAssignee: Ilya Etingof <ietingof>
Status: CLOSED ERRATA QA Contact: Dan Yasny <dyasny>
Severity: high Docs Contact:
Priority: medium    
Version: 13.0 (Queens)CC: apevec, bfournie, emacchi, jjung, jschluet, krishnan.raghuram, lhh, lin.a.yang, markmc, mburns, mlammon, nathaniel.potter, paul.von.behren, pchavva, priyank.durugkar, racedoro, rhel-osp-director-maint, robert.h.armstrong, sclewis, srevivo
Target Milestone: Upstream M3Keywords: FutureFeature, OtherQA, TechPreview, Triaged
Target Release: 13.0 (Queens)   
Hardware: x86_64   
OS: Linux   
URL: https://github.com/openstack/python-rsdclient
Whiteboard:
Fixed In Version: python-rsd-lib-0.1.1-0.20171222201255.d911e6e.el7ost python-rsdclient-0.1.1-0.20180110154722.f02d3fc.el7ost Doc Type: Technology Preview
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-06-27 13:31:39 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1514115, 1514119    
Bug Blocks: 1412053, 1414581, 1419948, 1422243    

Description Krish Raghuram 2017-06-30 15:38:59 UTC
Description of feature:
RSD platforms currently support the Redfish API, and are expected to soon support the Swordfish and Yang-to-Redfish APIs. Red Hat has already written a Redfish library (Sushy) to provide access from within OpenStack to the RSD platforms. Functionality is needed within OpenStack (eg. through the CLI), to view and manage the compute resources through the open APIs. Specifically:

-	discover and list compute resources
-	Allow composition of nodes 
-	Allow decomposition of nodes with erase and no erase option for data on drives

Version-Release number of selected component (if applicable):
OpenStack Client version (potentially) in OpenStack Queens release

2. Business Justification:
  a) Why is this feature needed?
     RSD is a new architecture that realizes an agile infrastructure where the hardware resources can be pooled according to application needs. It also enables a more easily scaled infrastructure, so CPU, memory, network and storage resources can be added as needed, without the need to do complete replacements of nodes
  b) What hardware does this enable?
   New platforms based on RSD architecture
  c) Is this hardware on-board in a system (eg, LOM) or an add-on card? 
  RSD nodes are disaggregated (CPU, memory, storage, accelerators etc.)
  d) Business impact? N/A
  
  e) Other business drivers: N/A

3. Primary contact at Partner, email, phone (chat)
   Priyank Durugkar - priyank.durugkar

4. Expected results:
- DC admin installs the undercloud
- Admin then uses the CLI to discover compute resources
- Admin then composes a compute node on a RSD rack with the required resources
- TripleO then deploys the node into the overcloud
- Once a node is no longer needed, admin decomposes the node 

Additional info:

Comment 1 Red Hat Bugzilla Rules Engine 2017-06-30 15:39:05 UTC
This bugzilla has been removed from the release and needs to be reviewed and Triaged for another Target Release.

Comment 2 Nate Potter 2017-07-24 20:16:31 UTC
I'm working on enabling this starting with adding RSD functionality to the python-valenceclient using a custom extension of Sushy that allows for node composition logic. From there we can expose the work to OSC through a plugin.

Comment 3 Nate Potter 2017-08-03 18:02:57 UTC
Here is the code I wrote extending Sushy to be able to discover composed nodes via the RSD redfish API https://review.openstack.org/#/c/490253/. It was merged to the new rsd-lib project, which will be used by the python-rsdclient plugin for OSC.

Comment 4 Krish Raghuram 2017-09-21 14:45:38 UTC
This is high priority for Intel for Queens release. We've got all the code tested and merged into https://github.com/openstack/rsd-lib and https://github.com/openstack/python-rsdclient

Comment 6 Ramon Acedo 2017-10-19 10:36:33 UTC
Hi Krish, our understanding is this functionality will be included in the python-rsdclient you are working on. 

Adding python-rsdclient to RDO as a package first and then to OSP will make composing and listing the RSD nodes possible. 

We intend to do this for OSP 13 as requested. This BZ can be a placeholder to track the progress and to later test it in OSP when available by the Intel team with OSP.

Including it for OSP 13, pending the usual reviewing of dependencies, security, legal, etc.

BZ will need a new component for python-rsdclient, for now it's under python-openstackclient.

Comment 7 Ramon Acedo 2017-10-20 10:13:58 UTC
*** Bug 1466873 has been marked as a duplicate of this bug. ***

Comment 8 Ramon Acedo 2017-10-20 10:15:42 UTC
*** Bug 1466880 has been marked as a duplicate of this bug. ***

Comment 12 Jon Schlueter 2017-10-23 15:00:32 UTC
Also is there work for getting python-rsdclient packaged and available in RDO by upstream Milestone 2 for Queens cycle?  (if not already stated packaging and package review in RDO/fedora needs to be underway soon.)

Comment 15 Krish Raghuram 2017-10-23 17:29:28 UTC
(In reply to Jon Schlueter from comment #12)
> Also is there work for getting python-rsdclient packaged and available in
> RDO by upstream Milestone 2 for Queens cycle?  (if not already stated
> packaging and package review in RDO/fedora needs to be underway soon.)

Nate Potter on our team is investigating this, as we don't have experience packaging stuff into RDO. Would Milestone 2 be the Dec.4 deadline that I've heard being mentioned?

Comment 16 Jon Schlueter 2017-10-24 03:13:37 UTC
(In reply to Krish Raghuram from comment #15)
> (In reply to Jon Schlueter from comment #12)
> > Also is there work for getting python-rsdclient packaged and available in
> > RDO by upstream Milestone 2 for Queens cycle?  (if not already stated
> > packaging and package review in RDO/fedora needs to be underway soon.)
> 
> Nate Potter on our team is investigating this, as we don't have experience
> packaging stuff into RDO. Would Milestone 2 be the Dec.4 deadline that I've
> heard being mentioned?

Yes Dec 4-8 this Queens Milestone 2 [1]  and we should have all packages available in RDO before then.

[1] https://releases.openstack.org/queens/schedule.html


First step is work with RDO guys to get .spec created and reviewed for these packages, get Review BZ's filed for RDO, and reviews well underway.

Alan Pevec and others can help coordinate getting this process rolling.

Comment 17 Mike Burns 2017-10-24 11:57:50 UTC
(In reply to Krish Raghuram from comment #15)
> (In reply to Jon Schlueter from comment #12)
> > Also is there work for getting python-rsdclient packaged and available in
> > RDO by upstream Milestone 2 for Queens cycle?  (if not already stated
> > packaging and package review in RDO/fedora needs to be underway soon.)
> 
> Nate Potter on our team is investigating this, as we don't have experience
> packaging stuff into RDO. Would Milestone 2 be the Dec.4 deadline that I've
> heard being mentioned?

http://rdoproject.org/documentation/add-packages/ talks about how to add packages.

Dec 4 is when we need the package in RDO for us to get it into OSP 13.  

If you need assistance, people on #rdo on Freenode are generally pretty helpful.

Comment 18 Alan Pevec 2017-10-24 12:00:02 UTC
Also check packaging guidelines at https://www.rdoproject.org/documentation/rdo-packaging/ then ask any questions after that on Freenode #rdo

Comment 21 Ramon Acedo 2017-12-18 16:31:14 UTC
Based on: 

https://bugzilla.redhat.com/show_bug.cgi?id=1514115 (rsd-lib inclusion in RDO)
https://bugzilla.redhat.com/show_bug.cgi?id=1514119 (python-rsdclient inclusion in RDO)

The inclusion of both packages has been merged:

https://review.rdoproject.org/r/#/c/10491/
https://review.rdoproject.org/r/#/c/10490/

So we assume the packages are ready in RDO. 

Moving to POST accordingly.

Comment 22 Jon Schlueter 2018-01-03 19:03:58 UTC
I found and imported python-rsd-lib-0.1.1-0.20171222201255.d911e6e.el7ost  but python-rsdclient I am not finding in RDO Trunk Master (DLRN)

https://trunk.rdoproject.org/centos7-queens/current/

Comment 23 Jon Schlueter 2018-01-03 19:05:44 UTC
rdo git repo for rsdclient-distgit is empty and missing .spec

Comment 24 Bob Fournier 2018-01-03 20:35:36 UTC
The spec file was added for rsd-lib-distgit here -https://review.rdoproject.org/r/#/c/10619/

A spec file is also needed for rsdclient-distgit, its not apparent if a patch has been created, if not can you add it?

Comment 25 Bob Fournier 2018-01-03 20:44:29 UTC
NM, patch for spec file is here - https://review.rdoproject.org/r/#/c/10620/ and has an external tracker, it just needs to merge.

Comment 29 Lin Yang 2018-01-06 02:00:08 UTC
Now two RDO package specs for rsd-lib-distgit and rsdclient-distgit have been merged.

https://review.rdoproject.org/r/#/c/10620/
https://review.rdoproject.org/r/#/c/10619/

Comment 30 Bob Fournier 2018-01-06 12:53:11 UTC
Thanks, moving this back to POST.

Comment 35 Jon Schlueter 2018-06-22 16:20:59 UTC
The content for this fix is included in OSP 13 RC which is available on the Red Hat OpenStack Platform beta channels currently.  Any efforts to verify that these bugs have been fixed would be appreciated.

Comment 36 Krish Raghuram 2018-06-25 20:05:34 UTC
(In reply to Jon Schlueter from comment #35)
> The content for this fix is included in OSP 13 RC which is available on the
> Red Hat OpenStack Platform beta channels currently.  Any efforts to verify
> that these bugs have been fixed would be appreciated.
By when do you need this to be tested?

Comment 37 Jon Schlueter 2018-06-25 20:20:57 UTC
GA is scheduled for this week, if it's not verified by then by default bug will closed shipped advisory.  If you want/need something different please respond as soon as possible.

Comment 38 Krish Raghuram 2018-06-25 21:50:44 UTC
(In reply to Jon Schlueter from comment #37)
> GA is scheduled for this week, if it's not verified by then by default bug
> will closed shipped advisory.  If you want/need something different please
> respond as soon as possible.

What is meant by "closed shipped advisory"? Will it not be "Tech Preview" for the feature?

Comment 40 errata-xmlrpc 2018-06-27 13:31:39 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2018:2086