Showing posts with label Infiniband. Show all posts
Showing posts with label Infiniband. Show all posts

Monday, January 13, 2020

Mellanox supplies 200G HDR InfiniBand to ECMWF

Mellanox Technologies will supply its 200 Gigabit HDR InfiniBand to the European Centre for Medium-Range Weather Forecasts (ECMWF) to accelerate their new world-leading supercomputer, which is based on Atos’ latest BullSequana XH2000 technology.

ECMWF's new supercomputer will be one of the world’s most powerful meteorological supercomputers, supporting weather forecasting and prediction researchers from over 30 countries across Europe. The new platform, utilizing HDR InfiniBand, will enable running nearly 2 times higher-resolution probabilistic weather forecasts in under an hour, improving the ability to monitor and predict increasingly severe weather phenomena and enable European countries to take proactive precautions to protect lives and property.

“We are proud to have our 200 Gigabit HDR InfiniBand solutions accelerate one of the most powerful meteorological services supercomputers in the world, at the European Centre for Medium-Range Weather Forecasts,” said Gilad Shainer, senior vice president of marketing at Mellanox Technologies. “Climate and weather simulations are compute and data intensive, and require the most advanced interconnect technology to ensure fast and accurate results. HDR InfiniBand includes multiple data acceleration and analysis engines, making it the leading technology for such applications. We look forward to continuing work with ECMWF and Atos, to develop the supercomputing capabilities needed for even more accurate and complex simulations in the future.”

Thursday, November 14, 2019

Mellanox extends Infiniband to 40km

Mellanox Technologies introduced its Quantum LongReach series of long-distance InfiniBand switches for connecting remote InfiniBand data centers together, or to provide high-speed and full RDMA (remote direct memory access) connectivity between remote compute and storage infrastructures.

Based on the 200 gigabit HDR Mellanox Quantum InfiniBand switch, the LongReach solution provides up to two long-reach InfiniBand ports and eight local InfiniBand ports. The long reach ports can deliver up to 100 Gbps data throughput for distances of 10 and 40 kilometers.

Key capabilities

  • Connect remote InfiniBand based data centers together to create a single virtual data center, effectively combining the compute power of multiple distributed data centers for higher overall performance and scalability. With LongReach, users can leverage the In-Network Computing capabilities such as the Scalable Hierarchical Aggregation and Reduction Protocol (SHARP)™, congestion control, enhanced telemetry and more between the multiple remote data centers.
  • Seamlessly migrate user applications from one data center to another to support different user demands, to provide load balancing between InfiniBand data centers, or to provide continuous compute services in cases of data-center failures.
  • Enable fast and efficient connectivity between remote compute and storage infrastructures, enabling fast disaster recovery and more.

“The Mellanox Quantum LongReach appliance enables native InfiniBand connectivity between remote InfiniBand-based data centers, or between data center and remote storage infrastructure, allowing users to enjoy native RDMA, In-Network Computing acceleration engines, congestion control and other InfiniBand technology advantages globally,” said Gilad Shainer, senior vice president of marketing at Mellanox Technologies. “Our existing and new customers, who wish to expand their clusters seamlessly over local and distributed InfiniBand networks that are kilometers apart, will find Mellanox Quantum LongReach to be the best cost-effective and easily managed solution.”

The products will be available in the first half of 2020.

https://www.mellanox.com/page/longreach/?ls=pr&lsd=191114-LongReach-1

Tuesday, June 18, 2019

Mellanox supplies 200G InfiniBand for Lenovo’s liquid cooled servers

Mellanox Technologies has begun shipping liquid cooled HDR 200G Multi-Host InfiniBand adapters for the Lenovo ThinkSystem SD650 server platform, which features Lenovo's "Neptune" liquid cooling technologies.

“Our collaboration with Lenovo delivers a scalable and highly energy efficient platform that delivers nearly 90% heat removal efficiency and can reduce data center energy costs by nearly 40%, and takes full advantage of the best-of-breed capabilities from Mellanox InfiniBand, including the Mellanox smart acceleration engines, RDMA, GPUDirect, Multi-Host and more,” said Gilad Shainer, Senior Vice President of Marketing at Mellanox Technologies.

Monday, June 17, 2019

Mellanox cites supercomputing momentum for HDR 200G Infiniband

Mellanox Technologies reports that HDR 200G InfiniBand continues to gain traction with next generation of supercomputers worldwide due to its highest data throughput, extremely low latency, and smart In-Network Computing acceleration engines.

Mellanox's HDR 200G InfiniBand solutions include its ConnectX-6 adapters, Mellanox Quantum switches, LinkX cables and transceivers and software packages.

“We are proud to have our HDR InfiniBand solutions accelerate supercomputers around the world, enhance research and discoveries, and advancing Exascale programs,” said Gilad Shainer, senior vice president of marketing at Mellanox Technologies. “InfiniBand continues to gain market share, and be selected by many research, educational and government institutes, weather and climate facilities, and commercial organizations. The technology advantages of InfiniBand make it the interconnect of choice for compute and storage infrastructures.”

Examples

  • The Texas Advanced Computing Center’s (TACC) Frontera supercomputer -- ranked #5 on the June 2019 TOP500 Supercomputers list, Frontera utilizes HDR InfiniBand, and in particular multiple 800-port HDR InfiniBand switches.
  • The new HDR InfiniBand-based Orion supercomputer located at the Mississippi State University High Performance Computing Collaboratory -- ranked #62 on the June 2019 TOP500 list, the 1800-node supercomputer leverages the performance advantages of HDR InfiniBand and its application acceleration engines to provide new levels of application performance and scalability.
  • CSC, the Finnish IT Center for Science, and the Finnish Meteorological Institute -- ranked #166 on the TOP500 list.
  • Cygnus -- the first HDR InfiniBand supercomputer in Japan and ranked #264 on the TOP500 list.
  • India's Center for Development of Advanced Computing (C-DAC) 

Tuesday, January 22, 2019

Mellanox supplies 200 Gigabit HDR InfiniBand to Finnish IT Center for Science

Mellanox Technologies will supply its 200 Gigabit HDR InfiniBand solutions to accelerate a multi-phase supercomputer system by CSC – the Finnish IT Center for Science. The new supercomputers, set to be deployed in 2019 and 2020, will serve the Finnish researchers in universities and research institutes, enhancing climate, renewable energy, astrophysics, nanomaterials and bioscience, among a wide range of exploration activities. The Finnish Meteorological Institute (FMI) will have their own separate partition for diverse simulation tasks ranging from ocean fluxes to atmospheric modeling and space physics.

Mellanox said its HDR InfiniBand interconnect solution was selected for its fast data throughout, extremely low latency, smart In-Network Computing acceleration engines, and enhanced Dragonfly network topology.

Sunday, July 22, 2018

Majority of TOP500 supercomputers connect with Infiniband

InfiniBand now powers four of the top five supercomputer systems, according to The InfinBand Trade Association and based on the latest TOP500 List. This includes Oak Ridge National Laboratory’s Summit system, which currently holds the title of world's fastest supercomputer

IBTA said InfiniBand technology continues to dominate the HPC category of the TOP500 List, representing nearly 60% of all HPC systems listed, due to its network bandwidth, latency, scalability and efficiency metrics.

The trade association notes that almost half of the platforms in the latest results can be categorized as non-HPC, most of which are Ethernet-based. There is a growing number of RDMA over Converged Ethernet (RoCE) capable systems, which constitute over 30% of total Ethernet listings.

“As the makeup of the world’s fastest supercomputers evolve to include more non-HPC systems such as cloud and hyperscale, the IBTA remains confident in the InfiniBand Architecture’s flexibility to support the increasing variety of demanding deployments,” said Bill Lee, IBTA Marketing Working Group Co-Chair.

Tuesday, June 19, 2018

Mellanox supplies Infiniband for Sandia's ARM supercomputer

Mellanox Technologies will supply an InfiniBand solution to accelerate the world’s top Arm-based supercomputer to be deployed in Sandia National Laboratory in the second half of 2018.

The Astra supercomputer will include nearly 2600 nodes, and will leverage InfiniBand In-Network Computing acceleration engines. Astra is the first system in a series of the Vanguard program of advanced architecture platforms, supporting the US Department of Energy’s National Nuclear Security Administration (NNSA) missions.

“InfiniBand smart In-Network Computing acceleration engines will enable the highest performance and productivity for Astra, the first large scale Arm-based supercomputer,” said Gilad Shainer, vice president of marketing at Mellanox Technologies. “InfiniBand is the world-leading interconnect technology that supports the full range of compute and storage infrastructures, including X86, Power, Arm, GPUs, FPGAs and more. We are happy to support the Department of Energy’s efforts to explore new supercomputing platforms for their future needs.”

http://www.mellanox.com

Thursday, February 2, 2017

Mellanox Says Infiniband Continues to Grow

Mellanox Technologies reported Q4 sales of $221.7 million and $857.5 million in fiscal year 2016. GAAP gross margins were 66.8 percent in the fourth quarter, and 64.8 percent in fiscal year 2016.

“During the fourth quarter we saw continued sequential growth in our InfiniBand business, driven by robust customer adoption of our 100 Gigabit EDR solutions into artificial intelligence, machine learning, high-performance computing, storage, database and more. Our quarterly, and full-year 2016 results, highlight InfiniBand’s continued leadership in high-performance interconnects,” said Eyal Waldman, president and CEO of Mellanox Technologies. “Customer adoption of our 25, 50, and 100 gigabit Ethernet solutions continued to grow in the fourth quarter. Adoption of Spectrum Ethernet switches by customers worldwide generated positive momentum exiting 2016. Our fourth quarter and full-year 2016 results demonstrate Mellanox’s diversification, and leadership in both Ethernet and InfiniBand. We anticipate growth in 2017 from all Mellanox product lines.”

http://www.mellanox.com


Monday, April 4, 2016

Mellanox Introduces InfiniBand Router

Mellanox Technologies introduced a new line of InfiniBand router systems based on its Switch-IB switch ASIC and aimed at next generation mega data centers.

The new EDR 100Gb/s InfiniBand Routers offers 36 EDR 100Gb/s ports, which can be split among six different subnets.

Mellanox said its new InfiniBand Router brings increases resiliency by segregating the data center’s network into several subnets; with each subnet running its own subnet-manager (SM) thereby effectively isolating each subnet, and thus providing better availability and stability. For example, the storage subnets may use a Fat-Tree topology while the compute subnets may use 3D-torus, DragonFly+, Fat-Tree or other topologies that best fit the local application. The SB7780 can also help split the cluster in order to segregate between applications that run best on localized resources and between applications that require a full fabric. The design also enables scaling the fabric up to a virtually unlimited number of nodes.

"The SB7780 InfiniBand Router adds another layer to Mellanox's solutions that pave the road to Exascale solutions," said Gilad Shainer, vice president of marketing at Mellanox. "This new InfiniBand Router gives us the ability to scale up to a virtually unlimited number of nodes and yet sustain the data processing demands of machine learning, IoT, HPC and cloud applications. Mellanox's EDR 100Gb/s InfiniBand solutions, together with the SB7780 router, represent the only scalable solution currently available on the market that support these needs."

http://www.mellanox.com/

Mellanox Intros 100Gbps InfiniBand Switch

Mellanox Technologies introduced its next generation InfiniBand switch for High-Performance Computing, Web 2.0, database and cloud data centers.

The Mellanox Switch-IB 2 is capable of 100 Gbps per port speeds and is the world's first smart network switch that offloads MPI operations from the CPU to the network to deliver 10X performance improvements. It integrates 144 SerDes which can operate at 1Gbps to 25Gbps speeds per lane and delivers 7.02 billion messages-per-second, 90ns switch latency and low power consumption, making Switch-IB 2 the best solution for high-performance computing, cloud, Web 2.0, database and storage centers.

Mellanox said a major innovation is that the switch can manage collective communications (MPI and SHMEM) using embedded hardware, decreading the amount of data traversing the network, reducing application latency with additional benefit of freeing up CPU resources for computation rather than using them to process communication.

Wednesday, February 17, 2016

Mellanox Supplies100G InfiniBand for European Supercomputer Center

The Flemish Supercomputer Center (VSC) in Belgium has selected Mellanox’s end-to-end 100Gb/s EDR interconnect solutions to be integrated into a new LX-series supercomputer from NEC. The system will be the fastest supercomputer (peak performance of 623 Teraflops) and the first complete end-to-end EDR 100Gb/s InfiniBand system in Belgium.

“Mellanox is thrilled to contribute to a project that will accelerate scientific discovery and has the capability to solve some of the top problems plaguing scientists and researchers today,” said Gilad Shainer, vice president of marketing, Mellanox Technologies. “This new supercomputer will position the university to make discoveries critical for the continuous advancement of science. VSC selected Mellanox for the performance levels only EDR 100Gb/s InfiniBand can deliver and is a prime example of the growing global demand for intelligent, fast, highly reliable, and cost efficient interconnect technology.”

https://www.vscentrum.be
http://www.mellanox.com

Thursday, November 12, 2015

Mellanox Intros 100Gbps InfiniBand Switch

Mellanox Technologies introduced its next generation InfiniBand switch for High-Performance Computing, Web 2.0, database and cloud data centers.

The Mellanox Switch-IB 2 is capable of 100 Gbps per port speeds and is the world's first smart network switch that offloads MPI operations from the CPU to the network to deliver 10X performance improvements. It integrates 144 SerDes which can operate at 1Gbps to 25Gbps speeds per lane and delivers 7.02 billion messages-per-second, 90ns switch latency and low power consumption, making Switch-IB 2 the best solution for high-performance computing, cloud, Web 2.0, database and storage centers.

Mellanox said a major innovation is that the switch can manage collective communications (MPI and SHMEM) using embedded hardware, decreading the amount of data traversing the network, reducing application latency with additional benefit of freeing up CPU resources for computation rather than using them to process communication.

“Mellanox smart interconnect solutions will continue to bring not just the widest and fastest interconnect but also more hardware offloads to free the CPU to focus on the application,” said Gilad Shainer, vice president of marketing, Mellanox Technologies. “Mellanox’s smart interconnect technology is a full generation ahead of the competition in terms of performance and return-on-investment advantages it brings to IT and applications managers.”

http://www.mellanox.com/page/press_release_item?id=1627

Thursday, October 8, 2015

Qualcomm Samples 24-core Server Chip based on ARMv8-A

Qualcomm has begun sampling a 24-core SoC based on the ARMv8-A instruction set and built using advanced FinFet technology.  The company's Server Development Platform (SDP) is aimed at high-density data centers.

Qualcomm also announced two key partnerships.  The company is partnering with Xilinx to deliver heterogeneous computing solutions for data centers with Qualcomm’s server processor and Xilinx FPGAs. Qualcomm is partnering with Mellanox to enable advanced, cost effective platforms for servers and storage that deliver the fastest, most efficient interconnect solutions for data transfer and analysis with Qualcomm’s server CPU and Mellanox’s Ethernet and InfiniBand interconnect solutions.

"The release of our evaluation system is a major milestone for Qualcomm Technologies. As data centers evolve to support the exponential growth and innovation in data, connectivity and cloud services, Qualcomm Technologies is creating an ecosystem to meet the needs of these next-generation data centers with Qualcomm-based server technologies. Our customers are eager to test and evaluate our Server Development Platform and begin porting their software. We are incorporating their feedback into our product offering with the goal of ensuring system and software readiness by the time we are in full production,” stated Anand Chandrasekher, senior vice president, Qualcomm Technologies.

http://www.qualcomm.com

Wednesday, September 2, 2015

Crehan: Strong Rebound for Data Center Ethernet Switch Market

The data center Ethernet switch market posted strong growth in the second quarter of 2015, according to a recent report from Crehan Research Inc. The firm reported that revenues and shipments both grew by more than 10 percent, helped by strong increases from 40 gigabit Ethernet (40GbE) and 10GBASE-T (10 gigabit-over-copper).

“We continue to see more 40GbE data center switch deployments, especially from the hyper-scale cloud market segment," said Seamus Crehan, president of Crehan Research. "At the same time, we are also seeing strong adoption of 10GBASE-T from both the enterprise and tier-2 cloud service provider segments," he added.

Besides the robust performance of 40GbE and 10GBASE-T, Crehan’s 2Q15 data center switch report includes these notable results:

  • HP Networking saw the strongest quarter-over-quarter data center Ethernet switch revenue growth, while Arista Networks saw the strongest year-over-year growth.
  • Cisco maintained its market share leadership position, accounting for almost 60% of total data center Ethernet Switch revenue. 
  • While all data center Ethernet switch types saw strong shipment growth in 2Q15, the Modular segment had the largest increase.
  • Infiniband switch shipments nearly doubled year-over-year, resulting in a record quarter.
  • In contrast with Ethernet and Infiniband, Fibre Channel switch revenue and shipments declined, both sequentially and year-over-year.

Although 40GbE data center switching is currently growing at a rapid rate, with shipments more than doubling year-over-year, Crehan’s last long-range forecast report noted that the impending arrival of low-priced 100GbE-capable data center switches could significantly slow down this growth, especially in the hyper-scale cloud segment.

http://www.crehanresearch.com/wp-content/uploads/2015/09/CREHAN-Data-Center-Switch-2Q15-PR.pdf

Thursday, August 27, 2015

InfiniBand Trade Association Tests RDMA over Converged Ethernet

The InfiniBand Trade Association (IBTA) has conducted the first Plugfest for RDMA over Converged Ethernet (RoCE) solutions. ifteen member companies participated, bringing their RoCE adapters, cables and switches for testing to the event.

A new RoCE Interoperability List has been posted on the IBTA website.

“The inclusion of RoCE interoperability testing as part of IBTA’s rigorous Plugfest events is a boon to the industry as we see RoCE gaining traction in enterprise-level data centers,” said Rupert Dance, chair of IBTA’s Compliance and Interoperability Workgroup. “System integrators want to be able to select products that meet their needs from a robust ecosystem, and interoperability testing helps provide assurance that those products will work together. The publication of the RoCE Interoperability List provides a critical resource in the advancement of this ecosystem.”

http://www.infinibandta.org
http://www.RoCEInitiative.org

Tuesday, August 25, 2015

Mellanox Supplies End-to-End EDR 100Gb/s InfiniBand to KTH Royal Institute

Sweden's KTH Royal Institute of Technology has selected Mellanox Technologies to supply its end-to-end, EDR 100Gb/s InfiniBand solutions for use in their PDC Center for High Performance Computing (PDC Center), whose scientific research work includes advanced modeling for climate changes, brain functions and protein-drug interactions.

The PDC Center is the latest deployment of Mellanox EDR InfiniBand technology to validate the expanding global adoption of this high-performance interconnect solution. Mellanox’s EDR InfiniBand solution offers higher interconnect speed, lower latency and smart accelerations.

The PDC Center for High Performance Computing at the KTH Royal Institute of Technology provides HPC services to Swedish academia as part of the Swedish National Infrastructure for Computing (SNIC), as well as internationally via the Partnership for Advanced Computing in Europe (PRACE) infrastructure. Those services include supercomputing and storage resources plus assistance from a range of application experts.

http://www.mellanox.com

Monday, July 13, 2015

Intel Shows its Omni-Path Architecture for HPC

Intel conducted the first public "powered-on" demonstration of its Omni-Path Architecture, a next-generation fabric technology for high performance computing (HPC) clusters.

The demonstration, conducted at the ISC2015 show in Frankfurt, featured Intel Omni-Path Architecture (Intel OPA), an end-to-end solution, including PCIe* adapters, silicon, switches, cables, and management software, that builds on the existing Intel True Scale Fabric and Infiniband. Intel OPA was designed to address the challenge that processor capacity and memory bandwidth have been scaling faster than system I/O.  It accelerates the message passing interface (MPI) rates in next gen systems. Intel OPA also promises the ability to scale to tens — and eventually hundreds — of thousands of nodes.

Intel Omni-Path Architecture uses technologies acquired from both QLogic and Cray, as well as Intel-developed technologies. In the near future, Intel says it will integrate the Intel Omni-Path Host Fabric Interface onto future generations of Intel Xeon processors and Intel Xeon Phi processors.

Intel also announced new a collaboration with HP to develop purpose-built HP Apollo systems designed to expand the use of HPC solutions to enterprises of all sizes.  The purpose built HP Apollo compute platforms will utilize the Intel HPC scalable system framework, including next generation Intel Xeon processors, the Intel Xeon Phi product family, Intel Omni-Path Architecture and the Intel Enterprise Edition of Lustre software.

http://www.intel.com/content/www/us/en/high-performance-computing-fabrics/omni-path-architecture-fabric-overview.html


In April 2015, Intel and Cray were selected to build two next generation, high-performance computing (HPC) systems that will be five to seven times more powerful than the fastest supercomputers today.

Intel will serve as prime contractor to deliver the supercomputers for the U.S. Department of Energy’s (DOE) Argonne Leadership Computing Facility (ALCF). The Aurora system will be based on Intel’s HPC scalable system framework and will be a next-generation Cray “Shasta” supercomputer. Intel said the Aurora system will be delivered in 2018 and have a peak performance of 180 petaflops, making it the world’s most powerful system currently announced to date. Aurora will use future generations of Intel Xeon Phi processors and the Intel Omni-Path Fabric high-speed interconnect technology, a new non-volatile memory architecture and advanced file system storage using Intel Lustre software.



In November 2014, Intel confirmed that its third-generation Intel Xeon Phi product family, code-named Knights Hill, will be built using 10nm process technology and that it will integrate Intel Omni-Path Fabric technology. Knights Hill will follow the upcoming Knights Landing product, with first commercial systems based on Knights Landing expected to begin shipping next year.

Intel also disclosed that its Intel Omni-Path Architecture will achieve 100 Gbps line speed and up to 56 percent lower switch fabric latency in medium-to-large clusters than InfiniBand alternatives. The architecture targets a 48 port switch chip compared to the current 36 port InfiniBand alternatives. This will reduce the number of switches required in HPC clusters.

Thursday, June 25, 2015

New RDMA over Converged Ethernet (RoCE) Initiative Gets Underway

A new RDMA over Converged Ethernet (RoCE) Initiative has been launched by the InfiniBand Trade Association (IBTA) to raise awareness about the benefits that RoCE delivers for cloud, storage, virtualization and hyper-converged infrastructures.

Remote Direct Memory Access (RDMA) enables faster movement of data between servers and between servers and storage with much less work being done by the CPU. RoCE utilizes RDMA to enhance infrastructure solutions for hyper-converged data centers, cloud, storage, and virtualized environments (see RoCE video). The technology brings greater network utilization with lower latency and improved CPU efficiency, in addition to reducing overall financial investment by increasing server productivity while leveraging Ethernet technology. RoCE technology transports data across Layer 2 and Layer 3 networks, providing better traffic isolation and enabling hyperscale data center deployments.

“The RoCE Initiative will be the leading source for information on RDMA over Ethernet solutions,” said Barry Barnet, co-chair, IBTA Steering Committee. “The IBTA remains committed to furthering the InfiniBand specification, of which RoCE is a part. The RoCE Initiative expands our potential audience and will enable us to deliver solution information and resources to those requiring the highest performing Ethernet networks.”

http://www.RoCEInitiative.org

Thursday, December 11, 2014

Blueprint: InfiniBand Moves from SuperComputing to Cloud

By: Dr. David Southwell, CVO, Obsidian Strategics

Some technology found inside modern supercomputers may prove to be surprisingly applicable to new data center architectures. To understand how, we must first look at the anatomy of contemporary supercomputers. Classics like the Cray vector supercomputers have long since given way to commodity silicon based designs - the vast majority of supercomputers today are huge clusters of servers lashed together with high-performance networks.  Built for massively parallel large-scale simulations, the application work load is distributed across the server nodes which coordinate via messages passed across their shared communications fabric.  The server nodes usually feature floating point heavy CPUs and GPU-based math accelerators and enjoy large main memories, but they are essentially just Linux servers.

InfiniBand is the Fast Interconnect Fabric for SuperComputing

 Most supercomputers attach their storage to the same communications fabric, as is used for inter-processor communication.  Storage must also be fast and parallel to facilitate large data set loading and also periodic checkpointing to save simulation state in case of a failure.  The interconnect is thus a unified fabric carrying management, compute and storage traffic over a single fiber connection to each node.

Reducing cost per node is a key consideration for most, and budget determines a supercomputer’s performance.  For this reason commodity, standards-based hardware components are preferred.  An open standard called InfiniBand (IB) has been the dominant cluster interconnect since its introduction, with specifications first published by an industry consortium that included Intel, IBM, HP and Microsoft in 1999.

IB is attractive due to features such as extreme scalability, low latency (sub microsecond end-to-end), high bandwidth (100GBits/s per port) and hardware offload, which includes a very powerful feature called RDMA (Remote Direct Memory Access).  RDMA allows data to flow “zero copy” from one application’s memory space to that residing on another server at wire speed, without the intervention of the OS, or even the CPU, allowing data movement to scale with memory speeds, not just CPU core speeds (which have stalled). More information on IB can be found here.

InfiniBand Takes on Data Center Scalability and East-West Traffic Challenges

What does InfiniBand have to do with data center design?  The components of good server farm design create a balance of compute, storage and network performance.  Many factors today reveal the shortcomings of the legacy 37-year old TCP/IP Ethernet:

  • Multiple virtual machines are consolidated onto single physical machines via virtualization, which has the effect of further multiplying the network performance requirements per socket and pushing towards supercomputer-class loading levels.  For instance, a TCP/IP stack running over 1Gb Ethernet could require up to 1GHz worth of CPU – overlay 20 such machines on a single node and even many-core CPUs are saturated by the OS before the application sees a single cycle.
  • Many-core processors use billions of transistors to tile tens to hundreds of CPU cores per chip, and server chips are trending strongly in this direction.  It is easy to see that the networking capability must be proportionately and radically scaled up to maintain architectural balance, or the cores will be forever waiting on network I/O.
  • Current data center work flow requirements, which tend to strongly emphasize East-West traffic, require new fabric topologies. Ethernet spanning tree limitations preclude efficient implementations such as “fat tree” featuring aggregated trunks between switches.
  • Rotating storage is being displaced by Solid State Disks (SSDs) – and not just in their early critical applications such as database indexing and metadata storage.  Legacy NAS interconnects that were able to hide behind tens of milliseconds of rotating disk latency are suddenly found to be hampering SSDs and their microsecond-range response times.  SSDs also deliver order of magnitude throughput increases, again stressing older interconnects.
  • Because they minimize network adapters, cables and switches, unified fabrics are highly desirable. They improve a host of system-level metrics such as capital costs, airflow, heat generation, management complexity and the number of channel interfaces per host.  Micro- and Blade-form-factor servers can ill-afford three separate interfaces per node.  Due to its lossy flow control and high latency, TCP/IP Ethernet is not a good match for high performance storage networks.

InfiniBand is in a unique position; it is able to take on all these challenges as well as smooth migration paths – for example, via IPoIB, InfiniBand can carry legacy IP traffic at great speed and while this does not immediately expose all of the protocol’s benefits, it provides a bridge to more efficient implementations that can be rolled out over time.  Furthermore—and contrary to popular misconception—InfiniBand is actually the most cost-effective protocol in terms of $/Gbits/s of any comparable standards-based interconnect technology, and dramatically so if deployed as a unified fabric.

Extending InfiniBand from Local Subnets to Global Distances

It’s true that InfiniBand has plenty of power and scale. It’s also true that an open standard supercomputer interconnect may hold the key to efficient future data center implementations. However, does InfiniBand have what it takes for production deployments?

In the past, InfiniBand implementations were limited to single subnet topologies and lacked security mechanisms such as link encryption. They could only manage very short links between racks by the standard’s precise lossless flow control scheme. However, today’s InfiniBand solutions enable the spaning of global distances over standard optical infrastructure, with strong link encryption and multi-subnet segmentation. Those who make use of the new IB stand to catch the bleeding edge of innovation that the supercomputer world continues to offer.

About the author

Dr. David Southwell co-founded Obsidian Research Corporation. Dr. Southwell was also a founding member of YottaYotta, Inc. in 2000 and served as its director of Hardware Development until 2004. Dr. Southwell worked at British Telecom's Research Laboratory at Martlesham Heath in the UK, participated in several other high technology start-ups, operated a design consultancy business, and taught Computer Science and Engineering at the University of Alberta. Dr. Southwell graduated with honors from the University of York, United Kingdom, in 1990 with a M.Eng. in Electronic Systems Engineering and a Ph.D in Electronics in 1993 and holds a Professional Engineer (P.Eng.) designation. 

About Obsidian Strategics
Obsidian Strategics Inc. is a private Canadian corporation offering enterprise-class, commercial off the shelf (COTS) devices supporting the InfiniBand protocol used in Supercomputer and HPC environments. The Obsidian Longbow™ technology was first developed for use in mission-critical military and intelligence environments that imposed operational requirements new to InfiniBand. http://www.obsidianresearch.com/

Got an idea for a Blueprint column?  We welcome your ideas on next gen network architecture.
See our guidelines.

Wednesday, November 19, 2014

Mellanox Intros Programmable Network Adapter with FPGA

Mellanox Technologies introduced its Programmable ConnectX-3 Pro adapter card with Virtual Protocol Interconnect (VPI) technology and aimed at modern data centers, public and private clouds, Web 2.0 infrastructures, telecommunication, and high-performance computing systems.

The new adapter uses an on-board integrated FPGA and memory to enable users to bring their own customized applications such as IPSEC encryption, enhanced flow steering and Network Address Translation (NAT), overlay networks bridge or router, data inspection, data compression or deduplication offloads, and others.

The programmable adapter card supports both InfiniBand and Ethernet protocols at bandwidths up to 56Gb/s. In addition, the FPGA can be engaged on the PCIe bus, the network interface, or in both locations simultaneously, giving the card complete flexibility for HPC, cloud, Web 2.0 and enterprise data center environments.

“Data center administrators and application users have looked for simple, powerful and more flexible ways to run applications at their highest potential, and to enable their own innovations as a competitive advantage,” said Gilad Shainer, vice president of marketing at Mellanox Technologies. “The Programmable ConnectX-3 Pro adapter card with FPGA completely revitalizes a data center’s ability to boost application performance by enabling a flexible and efficient programmable capability as data enters the network interface or is sent out.”

http://www.mellanox.com

Sunday, November 16, 2014

100G InfiniBand Connects Supercomputers across Continents

Obsidian Strategics and Singapore's A*STAR Computational Resource Centre are pioneering long-distance, 100G InfiniBand as a means of interconnecting geographically distributed supercomputers. A demonstration planned for this week's SC14 conference in New Orleans aims to demonstrate the feasibility of using the technology for global supercomputer collaboration. The key is to transmit RDMA over transcontinental distances.

The demo will show real time analysis and control of a plasma physics experiment at the Tokyo Institute of Technology involving distributed data processing concurrently occurring in Singapore, Georgia Tech and equipment on the SC14 exhibition floor.  Tata Communications is providing the long-haul cable.

Obsidian Strategics is a privately-held company based in Canada that offers InfiniBand products featuring range extension, routing and encryption.  Obsidian's Longbow technology allows an InfiniBand fabric -- normally a short-range network (up to 17m) used in supercomputers and data centers -- to be securely extended via optical fiber over global distances. Obsidian also provides its Crossbow technology that enables very high performance routing of InfiniBand traffic between many separate subnets.

 “Supercomputers have played crucial roles in military, higher-education, R&D labs and industry for decades. Today, at the rate data usage is increasing, the uses for high performance computing have also increased. Most supercomputers today are built around a very capable open-standard LAN technology called InfiniBand, but this is limited to very short reach connections. Our work on mission critical global communication requirements from the U.S. government has allowed us to pioneer the technology capable of transparently extending InfiniBand over arbitrary distances. The result is a highly deployable communications fabric suitable for demanding applications in not only scientific computing, but also storage, data center, cloud and enterprise environments. We couldn’t be more pleased to be a part of this groundbreaking collaboration,” stated. Dr. David Southwell, co-founder and chief visionary officer, Obsidian Strategics.

The InfiniCortex demostration is at SC14 booth #548 booth #2520.

http://www.obsidianstrategics.com
http://www.a-star.edu.sg/

See also