• DPDK环境搭建

    （1）虚拟环境：VMware® Workstation 16 Pro

    网上随便下载一个也行

    （2）操作系统：ubuntu-22.04-beta-desktop-amd64.iso

    下载地址：oldubuntu-releases-releases-22.04安装包下载_开源镜像站-阿里云

    （3）DPDK版本：22.07

    下载地址：http://fast.dpdk.org/rel/dpdk-22.07.tar.xz

    （4）使用VMware安装ubuntu22.04，怎么安装去网上找

    安装后的配置，添加两个网络适配器，用于DPDK通信

    （5）DPDK的编译

    访问链接：Ubuntu22.04 虚拟机中搭建 DPDK 开发环境

    缺啥就安装啥

    先安装一些依赖的软件包：

    1. apt-get install meson
    2. apt install python3-pyelftools
    3. apt-get install pkg-config

    ninja安装：Linux(15)Ubuntu安装ninja构建工具_一歲抬頭的博客-CSDN博客

    再编译安装 DPDK：

    1. wget https://fast.dpdk.org/rel/dpdk-22.07.tar.xz
    2. tar xf dpdk-22.07.tar.xz
    3. cd dpdk-22.07
    4. meson  build
    5. cd build
    6. ninja
    7. ninja install

    执行完之后所有的库都安装在 /usr/local/lib/x86_64-linux-gnu/ 目录。

    可执行程序和脚本都安装在 /usr/local/bin/ 目录。

    要卸载只需执行 ninja uninstall 即可。

    编译 igb uio 驱动

    1. git clone http://dpdk.org/git/dpdk-kmods
    2. cd dpdk-kmods/linux/igb_uio
    3. make

    编译出来 igb_uio.ko

    为了验证我们的环境是没问题的，先编译出 l2fwd 程序：

    1. cd examples/l2fwd
    2. make

    接着加载 igb_uio 驱动：

    1. cd dpdk-kmods/linux/igb_uio
    2. modprobe uio
    3. insmod igb_uio.ko intr_mode=legacy

    注意： 加载驱动时要带着参数intr_mode=legacy，如果不加参数，将会有问题！

    分配一些大页内存【这里是1G】：

    echo 512 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages

    绑定两个网卡【虚拟机有 3 个网卡，最后两个网卡供 DPDK 使用】：

    1. ifconfig ens34 down
    2. ifconfig ens35 down
    3. dpdk-devbind.py -b igb_uio ens34 ens35

    运行 l2fwd 程序：

    1. cd examples/l2fwd/build/
    2. ./l2fwd -l 0-1 -- -p 0x3 -T 1

    如果看到以下信息，说明 DPDK 环境没问题！

    1. EAL: Detected CPU lcores: 8
    2. EAL: Detected NUMA nodes: 1
    3. EAL: Detected shared linkage of DPDK
    4. EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
    5. EAL: Selected IOVA mode 'PA'
    6. EAL: VFIO support initialized
    7. EAL: Probe PCI driver: net_e1000_em (8086:100f) device: 0000:02:02.0 (socket 0)
    8. EAL: Error reading from file descriptor 20: Input/output error
    9. EAL: Probe PCI driver: net_e1000_em (8086:100f) device: 0000:02:03.0 (socket 0)
    10. EAL: Error reading from file descriptor 6: Input/output error
    11. TELEMETRY: No legacy callbacks, legacy socket not created
    12. MAC updating enabled
    13. Lcore 0: RX port 0 TX port 1
    14. Lcore 1: RX port 1 TX port 0
    15. Initializing port 0... EAL: Error enabling interrupts for fd 20 (Input/output error)
    16. done: 
    17. Port 0, MAC address: 00:0C:29:0C:53:91Initializing port 1... EAL: Error enabling interrupts for fd 6 (Input/output error)
    18. done: 
    19. Port 1, MAC address: 00:0C:29:0C:53:9BChecking link statusdone
    20. Port 0 Link up at 1 Gbps FDX Autoneg
    21. Port 1 Link up at 1 Gbps FDX Autoneg
    22. L2FWD: entering main loop on lcore 1
    23. L2FWD:  -- lcoreid=1 portid=1
    24. L2FWD: entering main loop on lcore 0
    25. L2FWD:  -- lcoreid=0 portid=0Port statistics ====================================
    26. Statistics for port 0 ------------------------------
    27. Packets sent:                        0
    28. Packets received:                    0
    29. Packets dropped:                     0
    30. Statistics for port 1 ------------------------------
    31. Packets sent:                        0
    32. Packets received:                    0
    33. Packets dropped:                     0
    34. Aggregate statistics ===============================
    35. Total packets sent:                  0
    36. Total packets received:              0
    37. Total packets dropped:               0
    38. ====================================================
    39. ^CSignal 2 received, preparing to exit...
    40. EAL: Error disabling interrupts for fd 20 (Input/output error)
    41. Closing port 0... Done
    42. EAL: Error disabling interrupts for fd 6 (Input/output error)
    43. Closing port 1... Done
    44. Bye...

    EAL: Error enabling interrupts for fd 20 (Input/output error) 等错误可以忽略。

    解绑网卡

    两个网卡 ens34 和 ens35 被 DPDK 占用后，ifconfig 里是没有的，要恢复请进行如下操作。

    首先查看两个网卡 pci 设备号：

    1. root@ubuntu2204:~# lspci | grep Eth
    2. 02:00.0 Ethernet controller: Intel Corporation 82545EM Gigabit Ethernet Controller (Copper) (rev 01)
    3. 02:01.0 Ethernet controller: Intel Corporation 82545EM Gigabit Ethernet Controller (Copper) (rev 01)
    4. 02:02.0 Ethernet controller: Intel Corporation 82545EM Gigabit Ethernet Controller (Copper) (rev 01)
    5. 02:03.0 Ethernet controller: Intel Corporation 82545EM Gigabit Ethernet Controller (Copper) (rev 01)

    可看到最后两个网卡设备号是 02:02.0 和 02.03.0。

    然后解绑两个网卡的 igb_uio 驱动，绑定 e1000 驱动：

    1. dpdk-devbind.py -u 02:02.0 02:03.0
    2. dpdk-devbind.py -b e1000  02:02.0 02:03.0

    最后将网卡up起来：

    1. ifconfig ens34 up
    2. ifconfig ens35 up

    （6）DPDK间的发送与接收

    上面环境配置完后，克隆出另一个环境，这样就有两个DPDK环境

    DPDK接收数据代码：

    1. #include 
    2. #include 
    3. #include 
    4. #include 
    5. #include 
    6. #include 
    7. #include 
    8. #include 
    9. #include 
    10. #include 
    11. #include 
    12. #include 
    13. #include 
    14.  
    15. #define NB_MBUF 512
    16. #define MAX_PKT_BURST 32
    17. #define SELF_PROTO_TYPE 0x0888
    18. static struct rte_eth_conf port_conf = {
    19.       .rxmode = {
    20.             .split_hdr_size = 0
    21.       }
    22. };
    23.  
    24.  
    25. // cat /proc/interrupts
    26. void init_port(struct rte_mempool *mbuf_pool){
    27.       uint16_t nb_ports = 0;
    28.       int ret = 0;
    29.       int portid = 0;
    30.       struct rte_eth_dev_info dev_info;
    31.       struct rte_ether_addr addr;
    32.       const int num_rx_queues = 1;
    33.       const int num_tx_queues = 0;
    34.       nb_ports = rte_eth_dev_count_avail();
    35.       if(nb_ports == 0){
    36.             rte_exit(EXIT_FAILURE, "No support eth found\n");
    37.       }
    38.       for(portid = 0; portid < nb_ports; portid++){
    39.             ret = rte_eth_macaddr_get(portid, &addr);
    40.             if (ret != 0){
    41.                   rte_exit(EXIT_FAILURE, "macaddr get failed\n");
    42.             } 
    43.             printf("Port %u MAC: %02"PRIx8" %02"PRIx8" %02"PRIx8
    44.                         " %02"PRIx8" %02"PRIx8" %02"PRIx8"\n",
    45.                         portid, RTE_ETHER_ADDR_BYTES(&addr));
    46.             ret = rte_eth_dev_info_get(portid, &dev_info);
    47.             ret = rte_eth_dev_configure(portid, num_rx_queues, num_tx_queues, &port_conf);
    48.             ret = rte_eth_rx_queue_setup(portid, 0, 128, rte_eth_dev_socket_id(portid), NULL, mbuf_pool);
    49.             ret = rte_eth_dev_start(portid);
    50.             if (ret < 0) {
    51.                   rte_exit(EXIT_FAILURE, "rte_eth_dev_start:err=%d, port=%u\n", ret, portid);
    52.             }
    53.       }   
    54. }
    55. /*
    56.       发送方注意设置LP信息
    57. */
    58. int
    59. main(int argc, char **argv)
    60. {
    61.       int ret;
    62. 	unsigned lcore_id;
    63.       int i = 0;
    64.       int portid = 0;
    65.       int nb_rx = 0;
    66.       /* 初始化环境 */
    67. 	ret = rte_eal_init(argc, argv);
    68. 	if (ret < 0)
    69. 		rte_panic("Cannot init EAL\n");
    70.       /* 创建内存池 */
    71.       struct rte_mempool *mbuf_pool = rte_pktmbuf_pool_create("my pool", NB_MBUF, 32, 0,
    72. 			RTE_MBUF_DEFAULT_BUF_SIZE, rte_socket_id());
    73. 	if (mbuf_pool == NULL){
    74.             return -1;
    75.       }
    76.       init_port(mbuf_pool);
    77.       while(1){
    78.             struct rte_mbuf* pkts_burst[MAX_PKT_BURST];
    79.             nb_rx = rte_eth_rx_burst(portid, 0, pkts_burst, MAX_PKT_BURST);
    80.             if(nb_rx == 0){
    81.                   sleep(1);
    82.                   continue;
    83.             }
    84.             printf("recv data start : %d \n", nb_rx);
    85.             for(i = 0; i < nb_rx; i++){
    86.                   // ether
    87.                   struct rte_ether_hdr *hdr = rte_pktmbuf_mtod(pkts_burst[i], struct rte_ether_hdr *);
    88.                   // ip
    89.                   printf("ether_type = %x \n", hdr->ether_type);
    90.                   if(hdr->ether_type == rte_cpu_to_be_16(RTE_ETHER_TYPE_IPV4)){
    91.                         struct rte_ipv4_hdr *iphdr =  rte_pktmbuf_mtod_offset(pkts_burst[i], struct rte_ipv4_hdr *, sizeof(struct rte_ether_hdr));
    92.                         // upd
    93.                         if(iphdr->next_proto_id == IPPROTO_UDP){
    94.                               // (struct rte_udp_hdr *)RTE_PTR_ADD(iphdr, sizeof(struct rte_ipv4_hdr));
    95.                               struct rte_udp_hdr* udphdr = (struct rte_udp_hdr*)(iphdr + 1);
    96.                               uint16_t length = ntohs(udphdr->dgram_len);
    97.                               *(char*)(udphdr + length) = '\0';
    98.                               struct in_addr addr;
    99.                               addr.s_addr = iphdr->src_addr;
    100.                               printf("src:%s:%d \n", inet_ntoa(addr), ntohs(udphdr->src_port));
    101.                               addr.s_addr = iphdr->dst_addr;
    102.                               printf("dst:%s:%d, %s \n", inet_ntoa(addr), ntohs(udphdr->dst_port), (char*)(udphdr+1));
    103.                         }
    104.                   }else if(hdr->ether_type == rte_cpu_to_be_16(SELF_PROTO_TYPE)){
    105.                         char *data =  rte_pktmbuf_mtod_offset(pkts_burst[i], char *, sizeof(struct rte_ether_hdr));
    106.                         printf("recv data: %s \n", data);
    107.                   }
    108.                   rte_pktmbuf_free(pkts_burst[i]);
    109.             }
    110.       }
    111.       
    112.       return 0;
    113. }

    makefile文件

    1. # SPDX-License-Identifier: BSD-3-Clause
    2. # Copyright(c) 2010-2014 Intel Corporation
    3.  
    4. # binary name
    5. APP = demo1
    6.  
    7. # all source are stored in SRCS-y
    8. SRCS-y := main.c
    9.  
    10. PKGCONF ?= pkg-config
    11.  
    12. # Build using pkg-config variables if possible
    13. ifneq ($(shell $(PKGCONF) --exists libdpdk && echo 0),0)
    14. $(error "no installation of DPDK found")
    15. endif
    16.  
    17. all: shared
    18. .PHONY: shared static
    19. shared: build/$(APP)-shared
    20. 	ln -sf $(APP)-shared build/$(APP)
    21. static: build/$(APP)-static
    22. 	ln -sf $(APP)-static build/$(APP)
    23.  
    24. PC_FILE := $(shell $(PKGCONF) --path libdpdk 2>/dev/null)
    25. CFLAGS += -O3 $(shell $(PKGCONF) --cflags libdpdk)
    26. # Add flag to allow experimental API as l2fwd uses rte_ethdev_set_ptype API
    27. CFLAGS += -DALLOW_EXPERIMENTAL_API
    28. LDFLAGS_SHARED = $(shell $(PKGCONF) --libs libdpdk)
    29. LDFLAGS_STATIC = $(shell $(PKGCONF) --static --libs libdpdk)
    30.  
    31. ifeq ($(MAKECMDGOALS),static)
    32. # check for broken pkg-config
    33. ifeq ($(shell echo $(LDFLAGS_STATIC) | grep 'whole-archive.*l:lib.*no-whole-archive'),)
    34. $(warning "pkg-config output list does not contain drivers between 'whole-archive'/'no-whole-archive' flags.")
    35. $(error "Cannot generate statically-linked binaries with this version of pkg-config")
    36. endif
    37. endif
    38.  
    39. build/$(APP)-shared: $(SRCS-y) Makefile $(PC_FILE) | build
    40. 	$(CC) $(CFLAGS) $(SRCS-y) -o $@ $(LDFLAGS) $(LDFLAGS_SHARED)
    41.  
    42. build/$(APP)-static: $(SRCS-y) Makefile $(PC_FILE) | build
    43. 	$(CC) $(CFLAGS) $(SRCS-y) -o $@ $(LDFLAGS) $(LDFLAGS_STATIC)
    44.  
    45. build:
    46. 	@mkdir -p $@
    47.  
    48. .PHONY: clean
    49. clean:
    50. 	rm -f build/$(APP) build/$(APP)-static build/$(APP)-shared
    51. 	test -d build && rmdir -p build || true

    DPDK发送数据代码

    1. #include 
    2. #include 
    3. #include 
    4. #include 
    5. #include 
    6. #include 
    7. #include 
    8. #include 
    9. #include 
    10. #include 
    11. #include 
    12. #include 
    13. #include 
    14. #include 
    15. #include 
    16.  
    17. #define NB_MBUF 512
    18. #define MAX_PKT_BURST 32
    19. #define PKG_LEN 128
    20. #define DST_MAC "005056224E38"
    21. #define SELF_PROTO_TYPE 0x0888
    22. static struct rte_eth_conf port_conf;
    23.  
    24. // cat /proc/interrupts
    25. void init_port(struct rte_mempool *mbuf_pool){
    26.       uint16_t nb_ports = 0;
    27.       int ret = 0;
    28.       int portid = 0;
    29.       struct rte_eth_dev_info dev_info;
    30.       struct rte_ether_addr addr;
    31.       const int num_rx_queues = 0;
    32.       const int num_tx_queues = 1;
    33.       nb_ports = rte_eth_dev_count_avail();
    34.       if(nb_ports == 0){
    35.             rte_exit(EXIT_FAILURE, "No support eth found\n");
    36.       }
    37.       for(portid = 0; portid < nb_ports; portid++){
    38.             ret = rte_eth_macaddr_get(portid, &addr);
    39.             if (ret != 0){
    40.                   rte_exit(EXIT_FAILURE, "macaddr get failed\n");
    41.             } 
    42.             printf("Port %u MAC: %02"PRIx8" %02"PRIx8" %02"PRIx8
    43.                         " %02"PRIx8" %02"PRIx8" %02"PRIx8"\n",
    44.                         portid, RTE_ETHER_ADDR_BYTES(&addr));
    45.             ret = rte_eth_dev_info_get(portid, &dev_info);
    46.             ret = rte_eth_dev_configure(portid, num_rx_queues, num_tx_queues, &port_conf);
    47.             ret = rte_eth_tx_queue_setup(portid, 0, 128, rte_eth_dev_socket_id(portid), NULL);
    48.             ret = rte_eth_dev_start(portid);
    49.             if (ret < 0) {
    50.                   rte_exit(EXIT_FAILURE, "rte_eth_dev_start:err=%d, port=%u\n", ret, portid);
    51.             }
    52.             ret = rte_eth_promiscuous_enable(portid);
    53.             if (ret != 0) {
    54.                   rte_exit(EXIT_FAILURE, "promiscuous mode enable failed: %s\n", rte_strerror(-ret));
    55.                   return;
    56.             }
    57.       }   
    58. }
    59.  
    60. void sendData(uint16_t portId, struct rte_mempool *mbuf_pool){
    61.       struct rte_ether_addr addr_src;
    62.       struct rte_ether_addr addr_dst;
    63.       int ret = 0;
    64.       int queue_id = 0;
    65.       int nb_prep = 0;
    66.       int nb_send = 0;
    67.       struct rte_mbuf *pkt = NULL;
    68.       ret = rte_eth_macaddr_get(portId, &addr_src);
    69.       addr_dst.addr_bytes[0] = 0x00;
    70.       addr_dst.addr_bytes[1] = 0x50;
    71.       addr_dst.addr_bytes[2] = 0x56;
    72.       addr_dst.addr_bytes[3] = 0x22;
    73.       addr_dst.addr_bytes[4] = 0x4E;
    74.       addr_dst.addr_bytes[5] = 0x38;
    75.       
    76.       pkt = rte_pktmbuf_alloc(mbuf_pool);
    77.       if(!pkt){
    78.             printf("rte_pktmbuf_alloc faild \n");
    79.             return;
    80.       }
    81.       pkt->data_len = PKG_LEN;
    82.       struct rte_ether_hdr *hdr = rte_pktmbuf_mtod(pkt, struct rte_ether_hdr *);
    83.       // 源地址
    84.       rte_ether_addr_copy(&addr_src, &(hdr->src_addr));
    85.       // 目的地址
    86.       rte_ether_addr_copy(&addr_dst, &(hdr->dst_addr));
    87.       // 帧协议
    88.       hdr->ether_type = rte_cpu_to_be_16(SELF_PROTO_TYPE);
    89.       char* data =  rte_pktmbuf_mtod_offset(pkt, char *, sizeof(struct rte_ether_hdr));
    90.       rte_memcpy(data, "hello", strlen("hello") + 1);
    91.       nb_prep = rte_eth_tx_prepare(portId, queue_id, &pkt, 1); //准备数据
    92. 	nb_send = rte_eth_tx_burst(portId, queue_id, &pkt, 1); //发送数据
    93.       // 释放
    94.       rte_pktmbuf_free(pkt);
    95. }
    96.  
    97. int
    98. main(int argc, char **argv)
    99. {
    100.       int ret;
    101. 	unsigned lcore_id;
    102.       int i = 0;
    103.       int portid = 0;
    104.       int nb_rx = 0;
    105.       /* 初始化环境 */
    106. 	ret = rte_eal_init(argc, argv);
    107. 	if (ret < 0)
    108. 		rte_panic("Cannot init EAL\n");
    109.       /* 创建内存池 */
    110.       struct rte_mempool *mbuf_pool = rte_pktmbuf_pool_create("my pool", NB_MBUF, 32, 0,
    111. 			RTE_MBUF_DEFAULT_BUF_SIZE, rte_socket_id());
    112. 	if (mbuf_pool == NULL){
    113.             return -1;
    114.       }
    115.       init_port(mbuf_pool);
    116.       while(1){
    117.             sendData(portid, mbuf_pool);
    118.             // 1000000微秒 = 1秒
    119. 		rte_delay_us(1000000);
    120.       }
    121.       return 0;
    122. }

    发送数据代码中，目标MAC地址需要改为接收虚拟机中的MAC

    这个MAC在接收数据代码里也会打印，复制即可

    makefile文件

    1. # SPDX-License-Identifier: BSD-3-Clause
    2. # Copyright(c) 2010-2014 Intel Corporation
    3.  
    4. # binary name
    5. APP = send
    6.  
    7. # all source are stored in SRCS-y
    8. SRCS-y := main.c
    9.  
    10. PKGCONF ?= pkg-config
    11.  
    12. # Build using pkg-config variables if possible
    13. ifneq ($(shell $(PKGCONF) --exists libdpdk && echo 0),0)
    14. $(error "no installation of DPDK found")
    15. endif
    16.  
    17. all: shared
    18. .PHONY: shared static
    19. shared: build/$(APP)-shared
    20. 	ln -sf $(APP)-shared build/$(APP)
    21. static: build/$(APP)-static
    22. 	ln -sf $(APP)-static build/$(APP)
    23.  
    24. PC_FILE := $(shell $(PKGCONF) --path libdpdk 2>/dev/null)
    25. CFLAGS += -O3 $(shell $(PKGCONF) --cflags libdpdk)
    26. # Add flag to allow experimental API as l2fwd uses rte_ethdev_set_ptype API
    27. CFLAGS += -DALLOW_EXPERIMENTAL_API
    28. LDFLAGS_SHARED = $(shell $(PKGCONF) --libs libdpdk)
    29. LDFLAGS_STATIC = $(shell $(PKGCONF) --static --libs libdpdk)
    30.  
    31. ifeq ($(MAKECMDGOALS),static)
    32. # check for broken pkg-config
    33. ifeq ($(shell echo $(LDFLAGS_STATIC) | grep 'whole-archive.*l:lib.*no-whole-archive'),)
    34. $(warning "pkg-config output list does not contain drivers between 'whole-archive'/'no-whole-archive' flags.")
    35. $(error "Cannot generate statically-linked binaries with this version of pkg-config")
    36. endif
    37. endif
    38.  
    39. build/$(APP)-shared: $(SRCS-y) Makefile $(PC_FILE) | build
    40. 	$(CC) $(CFLAGS) $(SRCS-y) -o $@ $(LDFLAGS) $(LDFLAGS_SHARED)
    41.  
    42. build/$(APP)-static: $(SRCS-y) Makefile $(PC_FILE) | build
    43. 	$(CC) $(CFLAGS) $(SRCS-y) -o $@ $(LDFLAGS) $(LDFLAGS_STATIC)
    44.  
    45. build:
    46. 	@mkdir -p $@
    47.  
    48. .PHONY: clean
    49. clean:
    50. 	rm -f build/$(APP) build/$(APP)-static build/$(APP)-shared
    51. 	test -d build && rmdir -p build || true

    测试

    自己编译就行，make一下

    发送数据：

    接收数据：

  • 相关阅读:
    java字符
    mongodb——概念介绍（文档，集合，固定集合，元数据，常用数据类型）
    亚马逊加湿器UL998测试报告，测试项目
    基于dhtmlxGantt的Blazor甘特图组件
    Nacos注册中心1-nacos源码环境搭建
    生活消费分销系统搭建开发制作
    裸辞后吊打大厂面试官，四面拿到阿里offer后我还是选择了美团
    OLED透明拼接屏在乌海湖旅游区的应用探究
    基于Python开发的玛丽大冒险小游戏(源码+可执行程序exe文件+程序配置说明书+程序使用说明书)
    阿里面试官：聊聊如何格式化Instant
  • 原文地址：https://blog.csdn.net/hdsHDS6/article/details/132946610