Network Receive Path

• 内核网络包接收流程
  • 《网络包的内核漂流记》系列介绍
    • 风格、样式、本文的交互阅读方式 📖
  • ISO 网络模型
  • 网络包接收流程概述
  • 网络包接收步骤
    • 初始化与配置
    • IRQ 处理
    • SoftIRQ 中加载网络包到内存
    • SoftIRQ 中推送网络包到上层
  • 结尾
  • 相关
• Illustrated Guide to Monitoring and Tuning the Linux Networking Stack: Receiving Data
  • TL;DR
  • Getting started
  • Initial setup
  • Data arrives
  • Network data processing begins
  • Network data processing continues
  • Protocol stacks and userland sockets
  • Conclusion
• Monitoring and Tuning the Linux Networking Stack: Receiving Data
  • Overview
  • Detailed Look
    • Network Device Driver
      • Initialization
        • struct net_device_ops
        • ethtool registration
        • IRQs
        • NAPI
        • Preparing to receive data from the network
        • Enable NAPI
        • Register an interrupt handler
        • Enable Interrupts
        • The network device is now up
      • Monitor
        • Using ethtool -S
        • Using sysfs
        • Using /proc/net/dev
      • Tuning network devices
        • Check the number of RX queues being used
        • Adjusting the number of RX queues
        • Adjusting the size of the RX queues
        • ntuple filtering for steering network flows
    • SoftIRQs
      • What is a softirq?
      • ksoftirqd
      • __do_softirq
      • Monitoring
      • /proc/softirqs
    • Linux network device subsystem
      • Initialization of network device subsystem
        • Initialization of struct softnet_data structures
        • Initialization of softirq handlers
      • Data arrives
        • Interrupt handler
        • NAPI and napi_schedule
        • A note about CPU and network data processing
        • Monitoring network data arrival
          • Hardware interrupt requests
        • Tuning network data arrival
          • Interrupt coalescing
          • Adjusting IRQ affinities
      • Network data processing begins
        • net_rx_action processing loop
        • NAPI poll function and weight
        • The NAPI / network device driver contract
        • Finishing the net_rx_action loop
        • Exiting the loop when limits are reached
        • NAPI poll
          • igb_poll
          • igb_clean_rx_irq
        • Monitoring network data processing
          • /proc/net/softnet_stat
        • Tuning network data processing
          • Adjusting the net_rx_action budget
      • Generic Receive Offloading (GRO)
        • Tuning: Adjusting GRO settings with ethtool
      • napi_gro_receive
        • dev_gro_receive
      • napi_skb_finish
    • Receive Packet Steering (RPS)
      • Tuning: Enabling RPS
    • Receive Flow Steering (RFS)
      • Tuning: Enabling RFS
    • Hardware accelerated Receive Flow Steering (aRFS)
      • Tuning: Enabling accelerated RFS (aRFS)
    • Moving up the network stack with netif_receive_skb
      • Tuning: RX packet timestamping
    • netif_receive_skb
      • Without RPS (default setting)
      • With RPS enabled
        • enqueue_to_backlog
        • Flow limits
        • Monitoring: Monitor drops due to full input_pkt_queue or flow limit
        • Tuning
          • Tuning: Adjusting netdev_max_backlog to prevent drops
          • Tuning: Adjust the NAPI weight of the backlog poll loop
          • Tuning: Enabling flow limits and tuning flow limit hash table size
      • backlog queue NAPI poller
      • process_backlog
      • __netif_receive_skb_core delivers data to packet taps and protocol layers
      • Packet tap delivery
      • Protocol layer delivery
    • Protocol layer registration
      • IP protocol layer
        • ip_rcv
        • netfilter and iptables
        • ip_rcv_finish
          • Tuning: adjusting IP protocol early demux
        • ip_local_deliver
        • ip_local_deliver_finish
        • Monitoring: IP protocol layer statistics
      • Higher level protocol registration
      • UDP protocol layer
        • udp_rcv
        • __udp4_lib_rcv
        • The udp_queue_rcv_skb
        • sk_rcvqueues_full
          • Tuning: Socket receive queue memory
      • udp_queue_rcv_skb
        • __udp_queue_rcv_skb
        • Monitoring: UDP protocol layer statistics
          • /proc/net/udp
      • Queuing data to a socket
    • Extras
      • Timestamping
      • Busy polling for low latency sockets
      • SO_INCOMING_CPU
  • Conclusion
• The Performance Analysis of Linux Networking – Packet Receiving
  • Linux Networking subsystem: Packet Receiving Process
  • NIC & Device Driver Processing
  • Kernel Protocol Stack – IP
  • Kernel Protocol Stack – TCP
    • Kernel Protocol Stack – TCP 1
    • Kernel Protocol Stack – TCP 2
  • Mathematical Model
  • Experiment
    • Experiment 3: Data receive process
  • Conclusion
• The Linux networking stack - Testing Cassandra Throughput Performance Part 3
  • The Linux networking stack
  • Measuring is better than sampling
  • Tracing packet receipt
  • Monitoring message flow
  • Wrapping up
• Debugging network stalls on Kubernetes
  • Deep dive into Linux kernel packet processing
  • What is cadvisor doing to stall things?
• Linux 网络栈接收数据（RX）：原理及内核实现（2022）
• 0 收包过程（RX）俯瞰
• 1 网卡驱动初始化
  • 1.1 A little bit things about Mellanox NIC drivers
  • 1.2 驱动模块注册：module_init() -> init() -> pci/mlx5e init
  • 1.3 PCI 相关初始化
    • 1.3.1 PCI 驱动列表注册：pci_register_driver()
    • 1.3.2 内核为网卡搜索和加载驱动：pci_driver->probe()
      • 调用栈和流程图
      • 初始化 devlink：mlx5_devlink_alloc()
      • 初始化 debugfs 和一些 WQ：mlx5_mdev_init()
      • 初始化 PCI 相关部分：mlx5_pci_init()
      • 初始化硬中断（IRQ）、设置网卡状态为 UP：mlx5_load_one()
  • 1.4 以太网相关初始化：mlx5e_init()
    • 1.4.1 调用栈和流程图
    • 1.4.2 mlx5e_init() -> mlx5_register_interface() -> mlx5_add_device()
    • 1.4.3 mlx5_add_device() -> intf.add() -> mlx5e_add()
      • mlx5e_create_netdev()：创建 netdev、注册 ethtool 方法
      • mlx5e_attach()
        • RX 队列及 RX handlers 初始化
        • 启用网卡，注册 NAPI poll 方法
        • 启用硬中断，等待数据包进来
• 2 网卡收包
• 3 DMA 将包复制到 RX 队列 (ring buffer)
  • 3.1 工作流程
    • 第一次数据复制
  • 3.2 下一步：更上层来收包
    • 3.2.1 收包方式：100% 轮询 vs 硬件中断
    • 3.2.2 中断方式改进：NAPI 机制（轮询+中断）
• 4 触发硬件中断（IRQ）
  • 4.1 中断处理函数（ISR）注册
  • 4.2 触发硬件中断：mlx5_irq_int_handler()
  • 4.3 中断完成时的回调：irq_nb.notifier_call() -> mlx5_eq_comp_int()
  • 4.4 cq->comp() -> mlx5e_completion_event() -> napi_schedule()
  • 4.5 napi_schedule() -> ____napi_schedule()
  • 4.6 触发软件中断：____napi_schedule() -> __raise_softirq_irqoff()
• 5 内核调度到 ksoftirqd 线程
  • 5.1 内核网络设备子系统初始化
    • 5.1.1 注册 /proc/net/{dev,softnet_data,ptype}
    • 5.1.2 struct softnet_data
    • 5.1.3 SoftIRQ handler 初始化
  • 5.2 内核调度器与调用栈概览
    • 5.2.1 调用栈
    • 5.2.2 软中断线程初始化：注册 run_ksoftirqd()
  • 5.3 处理器调度循环：smpboot_thread_fn() -> run_ksoftirqd()
  • 5.4 run_ksoftirqd()
  • 5.5 __do_softirq() -> net_rx_action()
    • 流程图
    • 代码
• 6 软中断处理：从 ringbuffer 取数据送到协议栈
  • 6.1 net_rx_action() -> napi_poll()：从 ring buffer 读数据
    • 6.1.1 Softirq 收包预算：netdev_budget 和 netdev_budget_usecs
    • 6.1.2 多队列网卡（多 NAPI 实例）与 IRQ affinity
    • 6.1.3 time_squeeze 计数：ringbuffer 还有包，但 softirq 预算用完了
  • 6.2 napi_poll() -> mlx5e_napi_poll() -> mlx5e_poll_rx_cq()
    • 6.2.1 NAPI 和设备驱动之间的契约（contract）
    • 6.2.2 mlx5e_napi_poll() -> mlx5e_poll_rx_cq()
  • 6.3 mlx5e_poll_rx_cq() -> mlx5e_handle_rx_cqe()
  • 6.4 mlx5e_handle_rx_cqe() -> napi_gro_receive()
    • skb = mlx5e_skb_from_cqe_linear()：创建内核数据包
      • 第二次数据复制
    • mlx5e_complete_rx_cqe() -> mlx5e_build_rx_skb()：初始化 L2 header、IPSec、时间戳等
    • mlx5e_build_rx_skb() -> mlx5e_post_rx_wqes()：重新填充 RX ring
  • 6.6 napi_gro_receive()
    • 6.6.1 GRO（Generic Receive Offloading）
    • 6.6.2 流程图和调用栈
    • 6.6.3 dev_gro_receive()
    • 6.6.4 napi_gro_complete()
    • 6.6.5 napi_skb_finish() -> gro_normal_one() -> gro_normal_list()
    • 6.6.6 gro_normal_list() -> netif_receive_skb_list_internal()：批量将包送到协议栈
  • 6.8 其他高级特性
    • 6.8.1 RSS (Receive Side Scaling)：多队列分发
    • 6.8.2 RPS (Receive Packet Steering)：RSS 的软件实现
    • 6.8.3 RFS (Receive Flow Steering)：相同 flow 的包送到同一 CPU
    • 6.8.4 aRFS (Hardware accelerated RFS)
  • 6.9 小结
• 7 进入内核协议栈：L2 处理
  • 7.1 流程图
  • 7.2 netif_receive_skb_list_internal() -> __netif_receive_skb_list()
  • 7.3 __netif_receive_skb_core()：送到协议层
    • 7.3.1 ptype_base[]：L2/L3 协议层注册
    • 7.3.2 处理 skb 时间戳
    • 7.3.3 Generic XDP 处理（软件 XDP）
    • 7.3.4 VLAN 处理
    • 7.3.5 Tap 设备处理（例如 libpcap/tcpdump 抓包点）
    • 7.3.6 TC 处理
      • TC BPF redirection：将包从宿主机网卡直接送到容器内网卡
    • 7.3.7 Netfilter 处理
    • 7.3.8 进入 L3 协议层
    • 7.3.9 更新 softnet_data 统计
  • 7.4 启用了 RPS：先 enqueue_to_backlog() 再转 7.3
    • 7.4.1 enqueue_to_backlog()
    • Flow limits
    • 处理 backlog 队列：NAPI poller
    • process_backlog()
• 8 L3 协议层处理（IPv4）
  • 8.1 IP 协议层
  • 8.2 ip_rcv() -> NF_HOOK(PRE_ROUTING, ip_rcv_finish())
  • 8.3 Netfilter NF_INET_PRE_ROUTING hook
  • 8.4 ip_rcv_finish() -> routing -> dst_entry.input()
  • 8.5 ip_local_deliver()
  • 8.6 ip_local_deliver_finish() -> ip_protocol_deliver_rcu() -> l4proto.callback
• 9 L4 协议层处理（UDP）
  • 9.0 UDP 协议初始化
  • 9.1 L4 handler 注册：udp_v4_early_demux()/udp_v4_rcv()
  • 9.2 udp_rcv() -> __udp4_lib_rcv()
  • 9.3 __udp4_lib_rcv() -> udp_unicast_rcv_skb()
  • 9.4 udp_unicast_rcv_skb() -> udp_queue_rcv_skb()
  • 9.5 udp_queue_rcv_skb() -> udp_queue_rcv_one_skb() -> __udp_queue_rcv_skb()
  • 9.6 __udp_queue_rcv_skb() -> __skb_queue_tail() -> socket's receive queue
  • 9.7 小结
• 10 其他问题讨论
  • 10.1 打时间戳 (timestamping)
  • 10.2 socket 低延迟选项：busy polling
  • 10.3 Netpoll：内核挂掉后，设备仍然能收发数据
  • 10.4 SO_INCOMING_CPU：判断哪个 CPU 在处理给定 socket
  • 10.5 DMA 引擎
• 10 总结
• 参考资料