Bug 108827 - RHEL4: Infiniband support
RHEL4: Infiniband support
Status: CLOSED ERRATA
Product: Red Hat Enterprise Linux 4
Classification: Red Hat
Component: kernel (Show other bugs)
4.0
All Linux
high Severity medium
: ---
: ---
Assigned To: Doug Ledford
IT_10188 (Dell); IT_14002, IT_14003, ...
Kernel
: FutureFeature
: 108541 109127 113652 (view as bug list)
Depends On: 158747
Blocks: 130530 168430 172741 173386 175120
  Show dependency treegraph
 
Reported: 2003-11-02 12:22 EST by Susan Denham
Modified: 2010-10-21 22:27 EDT (History)
11 users (show)

See Also:
Fixed In Version: RHSA-2006-0132
Doc Type: Enhancement
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2006-03-07 13:29:07 EST
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Susan Denham 2003-11-02 12:22:12 EST
Arjan noted (and Steve Carbonari of Intel didn't object) that Infiniband is not
on the Intel chipset disclosure documents for CY04.  Arjan feels that 10GB
ethernet and similar technologies will supersede Infiniband.
Comment 2 Susan Denham 2003-11-02 16:48:28 EST
*** Bug 108541 has been marked as a duplicate of this bug. ***
Comment 3 Susan Denham 2003-11-02 16:50:18 EST
IBM specific info:  Priority: 1
Status: low-latency connection.  RAC interconnect and middle-tier to DB
connections are sensitive to latency and speed.
Description: Include Infiniband drivers
Business Justification: Provides a faster interconnect.
Comment 5 Bob Johnson 2003-11-04 19:19:51 EST
*** Bug 109127 has been marked as a duplicate of this bug. ***
Comment 6 Tim Burke 2004-01-18 20:53:02 EST
*** Bug 113652 has been marked as a duplicate of this bug. ***
Comment 8 Susan Denham 2004-03-03 12:49:29 EST
Sun wants Infiniband as a backplane interconnect rather than as a replacement
for 10GB ethernet.  

The request for Infiniband doesn't seem to be going away from any of the OEMs....
Comment 11 Larry Troan 2004-05-12 13:59:36 EDT
Adding HP
Comment 12 Arjan van de Ven 2004-05-19 07:31:12 EDT
significantly doubtful for RHEL4. There are like 3 codebases, all really really bad.
Comment 13 Larry Troan 2004-05-20 14:29:45 EDT
Adding SLAC
Comment 14 Larry Troan 2004-05-20 14:41:41 EDT
From PM:

Based on Arjan's comments, this one looks like it's too ill-defined and the code
is in too poor a shape for RHEL4 consideration. If it turns out that Topspin and
its 10G patch are accepted upstream and it passes Community code inspection
quality, then we should reconsifder.

For now , setting status to CLOSE=DEFERRED. Suggest we consider again for RHEL5.
Comment 15 Ben Woodard 2004-05-20 15:52:16 EDT
I believe Larry's comment is unacceptable in this case. In this case the
business justification is so compelling in the time frame of RHEL4 that we need
to take a proactive stance on the issue and work with the people at TopSpin and
the other OpenIB partners to bring their code into shape.

For this issue Red Hat must act not as an aggregator of already exant technology
but a contributor to the technology to bring it in line with acceptable upstream
coding standards. Virtually every big partner that we have is asking for this
feature and we have lost significant market advantage if we try to put this
feature out to RHEL5. 
Comment 16 Ben Woodard 2004-05-20 16:14:26 EDT
I think that all parties will agree that Infiniband is not ready yet. The big
thing that we need to decide as a company is are we going to be a market leader
deploying innovative technology in response to customer needs or are we going to
be a follower and just gather together the already extant technology and resell it.

There is a very notable advantage in being the first to market and this should
not be discounted. At the moment SuSE wins every single time in those races.
Comment 25 Robert Perkins 2004-06-22 11:53:04 EDT
Right now this is stuck for RHEL4 and will not be delivered.
We need hardware.
We need engineers to implement the IB upstream, and port it back to RHEL3.
This is a good candidate for a separate project for Red Hat to take on, but it
needs a Marketing Reqts Doc or some other justification as well as a project
plan and schedule.  Will mark this as a key feature for RHEL4 timeframe, perhaps
as a layered add-on.  

Could this be released as a vendor-supported driver?
Comment 27 Ben Woodard 2004-07-07 20:45:31 EDT
First of all we need the low level drivers that support the hardware.

We would like IP networking over IB. We run NFS and other protocols over this
network interface.

We also need text based IB configuration tools sort of like ifconfig, ethtool
and ethdiag.

MPI over IB is really important.

low level kernel data transfer capabilities are also necessary. On top of this
the CFS people will build the interface that lustre uses.

API that the batch job scheduler will use to initialize the MPI interfaces for
the job.
Comment 28 Ben Woodard 2004-07-07 21:03:35 EDT
Along with the development of the software stack, they want an integrated
testing suite and regression tests.

http://www.llnl.gov/asci/pathforward_trilab/iba_rfp/OSSODA_IBA_RFPSOW_DRAFT.pdf
http://www.llnl.gov/asci/pathforward_trilab/iba_rfp/OSSODA_IBA_RFP_EVAL_PREP_DRAFT.pdf

The way that they put it in the RFP on page 5 is:

1) complete InfiniBand access layer including host channel adapter (HCA) device
drivers, user and kernel space Verbs interface, communication management,
resource management; 
2) subnet management and subnet administration; and 
3) HPC upper layer protocols for MPI, IPoIB, SDP, and Sandia Portals.
Comment 31 Ben Woodard 2004-07-12 12:18:52 EDT
For reference regarding SDP:

To: Ben Woodard <woodard@redhat.com>
Subject: Re: [Fwd: (UPDATED) Issue #36125 ([RHEL 4] Implement Infiniband
        Support)[FEG-RHEL4/Feature Escalations Group/LLNL (HPC)]]
X-Message-Flag: Warning: May contain useful information
References: <1089328133.16745.1512.camel@quince.llnl.gov>
From: Roland Dreier <roland@topspin.com>
Date: Thu, 08 Jul 2004 21:42:08 -0700
In-Reply-To: <1089328133.16745.1512.camel@quince.llnl.gov> (Ben Woodard's
message of "Thu, 08 Jul 2004 16:08:53 -0700")
Message-ID: <52vfgxbxlr.fsf@topspin.com>
User-Agent: Gnus/5.1006 (Gnus v5.10.6) XEmacs/21.4 (Security Through
        Obscurity, linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
X-OriginalArrivalTime: 09 Jul 2004 04:42:10.0152 (UTC)
        FILETIME=[1896AE80:01C4656F]
X-RedHat-Spam-Score: 0
X-Evolution-Source: imap://woodard@pobox.corp.redhat.com/
Content-Transfer-Encoding: 8bit
 
This is something I wrote up a while ago about SDP:
 
SDP intellectual property
 
  Microsoft believes that they own certain intellectual property
  relating to the Sockets Direct Protocol (SDP)[2].  Therefore, we are
  including the following disclaimer required by Microsoft's license
  in SDP source that relates to the implementation of the protocol:
 
    "This source code may incorporate intellectual property owned by
    Microsoft Corporation. Our provision of this source code does not
    include any licenses or any other rights to you under any
    Microsoft intellectual property. If you would like a license from
    Microsoft (e.g., to rebrand, redistribute), you need to contact
    Microsoft directly."
 
  We realize that this is incompatible with open source licensing, and
  we are working to find a more satisfactory solution, but for the
  time being we are forced to comply with Microsoft's license.
 
  Please make sure you have fully understood the implications of
  Microsoft's claims before you redistribute any of the SDP source
  that contains the above disclaimer.
 
  [2]  <http://www.microsoft.com/mscorp/ip/standards/>
Comment 35 Robert Perkins 2004-08-04 08:08:47 EDT
PM is managing this from an overall program standpoint, in/out/when.  Greg is
project prime and will report on the Infiniband plans.
Comment 38 Robert Perkins 2004-08-17 11:49:55 EDT
Greg, please update with:
1.  What we are doing for RHEL4 or RHEL4 U1 (specify release)
    Please provide a synopsis of the functionality we are delivering as part of
the RFP.

2.  If we are delivering on RHEL3 U4, please provide a new FZ and provide the
same sort of info.

Thanks, this is a hot item for LLNL as you know and we have a SLA to meet to
respond to feature queries.  Also, many other vendors are interested in this.
Comment 49 Larry Troan 2004-10-06 15:53:05 EDT
Per Rob Perkins, plan is to support Infiniband via Vendor Provided "top spin"
driver.
Comment 57 Tim Burke 2005-03-02 10:53:54 EST
Not in U1.

Being reviewed for a later update.  Still not mature enough upstream.  Unlikely
to be in U2.
Comment 63 Tim Burke 2005-07-07 17:55:55 EDT
This is not in U2. We are pursuing initial IB support in the context of U3.
Comment 97 Red Hat Bugzilla 2006-03-07 11:43:58 EST
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on the solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHBA-2006-0013.html
Comment 98 Red Hat Bugzilla 2006-03-07 13:18:04 EST
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on the solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHEA-2006-0136.html
Comment 99 Red Hat Bugzilla 2006-03-07 13:18:26 EST
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on the solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHSA-2006-0016.html
Comment 100 Red Hat Bugzilla 2006-03-07 13:29:08 EST
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on the solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHSA-2006-0132.html

Note You need to log in before you can comment on or make changes to this bug.