Sign in

Hi, good question.

I actually measured testpmd performance in a previous part where it was running on the Bluefield (hence, it was in Embedded CPU mode).

Overall the performance was similar to what I measured here: with 64-byte packets, RSS enabled, most of the cores assigned (i.e., the max of 3), the throughput was around 16 Gbps.

Here, I used Separated mode just to avoid one more abstraction and also for investigating the scaling without hardware offload. Using the host CPU cores should be better than using the ARM cores on the Bluefield.

Regarding your first question:

OvS when offloaded…


We continue our quest on the offloading topic (Part VII/A) and investigate the packet processing performance with pktgen.

Architecture

For the measurements, I prepare the machines to materialize the architecture depicted below.

Architecture used in this measurement study

The setup is more or less the same we had previously. The Host on the left-hand side (let’s call it H1) runs the Bluefield-2 DPU in EMBEDDED_CPU Mode (i.e., in SmartNIC mode). It implements OvS and the whole packet processing logic on the ARM cores. On the other hand, the other Host on the right-hand side (let’s call it H2) does only utilize the SmartNIC as a NIC, i.e…


Is it beneficial to offload OvS datapath to the hardware? Does it matter if the kernel or DPDK datapath is offloaded? In this episode, I dig a bit deeper into the OvS offloading matters, and I also explain how the packet processing is done with OvS running on the SmartNIC.

In the previous episodes, I have already been dealing with OvS and DPDK on the Bluefield-2 DPU SmartNIC; however, the OFFLOADing part was not working completely. Then, I had to manually reinstall different parts of the whole ecosystem, starting from the BlueField OS through DPDK and OvS.

In this episode…


How I get DPDK up and running on a Bluefield-2 SmartNIC, and what is its performance when measured with pktgen and testpmd.

In the previous part, we have installed the latest Ubuntu operating system on the Bluefield, supplied by the vendor itself. This release includes DOCA, DPDK, OvS, and all necessary tools by default. Hence, if you follow Part V in installing this system, you can probably skip the DPDK-related part of my blog post in Part II. :)

Let’s see whether “probably” can be transformed to “surely”.

Enable SmartNIC Mode

First of all, we have to switch from the default Separated Host…


In this episode, we will install the latest Bluefield OS on the Bluefield-2 DPU from scratch. As a result, we will be given a fresh system with DPDK and DOCA pre-installed.

We will install BlueOS from scratch on the Bluefield-2 SmartNIC

Preamble

In the last episode, Part IV., I had some trouble accessing the Bluefield after firing up my ultimate Cloudlab setup in Part III. As a result of getting permission denied every time I tried to log in to the Bluefield, I decided to reinstall the whole operating system on it…I had the feeling that maybe someone has changed the password for fun. …


In this episode, we scrutinize the performance of the Bluefield-2 SmartNIC in our two-node cluster deployed at Cloudlab. We had quite a bumpy road in reaching this point, but we finally did it. If you missed the former parts, read Part I to know how to install the necessary drivers; read Part II to see how DPDK can be installed, and read Part III to see how the environment we further work in can be achieved in Cloudlab.

As a quick recap, the topology we use is shown below. Two servers are connected via three links. Two of them practically…


In this tutorial, we go through all necessary steps to configure and fire up an experiment with two Bluefield-2 SmartNICs connected back to back atthe Cloudlab facility at the University of Clemson.

Ultimate Cloudlab Setup for NVIDIA / Mellanox Bluefield-2 Experiments

Bluefield-2 Lan Setup at Cloudlab.

Disclaimer

While this part is a sequel of my series with the same title as “NVIDIA Mellanox Bluefield-2 SmartNIC + DPDK: “Rig for Dive” (c.f. Part I. and Part II.), here I only talk about the Cloudlab setup. Hence, if you lucky and have your own servers with your own Bluefields, you can skip this part and continue with Part IV.

TL;DR


After firing up an experiment in Cloudlab and getting our hands dirty with Bluefield-2, we continue our journey by elaborating on the different modes of operation and install DPDK on the Bluefield-2.

We will install and configure the latest DPDK on Bluefield-2 DPU

In Part I., we have installed all necessary drivers on r7525 machines at the Clemson facility of Cloudlabs. Then, we became able to access Bluefield-2 and provide Internet access to it.

Modes of Operation

So, we are given an Arm64-based server inside a server with internet access. We can download and install packages, and once we figured out what applications we want to materialize on top, we can delve deeper.

First…


I have gotten my hands dirty with NVIDIA Bluefield-2 SmartNIC deployed at Cloudlab’s facility @ Clemson. If you ever wondered to buy a Bluefield SmartNIC, now I can show you how to test them and get your first impressions for free.

NVIDIA Bluefield-2 DPU

What is Mellanox Bluefield-2

For some time ago, the networking industry is going through a big revolution. There have been many headlines lately about the end of Moore’s law and how the continuous close-to-exponential improvement of general-purpose processors is degrading. However, the data and the corresponding network traffic are doing the opposite. To efficiently keep up with this increasing data processing need, common…

cs.lev

He is an academic researcher with a PhD degree in Computer Science. He writes papers not blogs, but converting notes to blogs turned out to be more fun :)

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store