Kube Proxy

• Cracking Kubernetes Node Proxy (aka kube-proxy)
• 1. Background knowledge
  • 1.1 Netfilter
  • 1.2 eBPF
  • 1.3 VIP and load balancing (LB)
  • 1.4 Cross-host networking model
• 2. Kubernetes node proxy model
  • 2.1 Service types
  • 2.2 Service/proxy model
  • 2.3 Role of the node proxy: reverse proxy
  • 2.4 Performance issues
• 3. Test environment
  • 3.1 Cluster topology and test environment
  • 3.2 Create a Service
  • 3.3 Reachability test
• 4. Implementation 1: proxy via userspace socket
  • 4.1 The middleman model
  • 4.2 Implementation
    • 4.2.1 Code
    • 4.2.2 Some explanations
      • 1. Traffic interception
      • 2. Create proxy
    • 4.3 Reachability test
  • 4.3 Clean up
  • 4.4 Improvements
  • 4.5 Pros and Cons
• 5. Implementation 2: proxy via iptables
  • 5.1 Host -> ClusterIP (single backend)
    • 5.1.1 Where to perform the DNAT
    • 5.1.2 Check current NAT rules
    • 5.1.3 Add DNAT rules
    • 5.1.4 Test reachability
    • 5.1.5 Cleanup
  • 5.2 Host -> ClusterIP (multiple backends)
    • 5.2.1 Scale up webapp
    • 5.2.2 Add DNAT rules with load balancing
    • 5.2.3 Verify load balancing
    • 5.2.4 Cleanup
  • 5.3 Pod (app A) -> ClusterIP (app B)
  • 5.4 Re-structure the iptables rules
  • 5.5 Further Re-structure the iptables rules
• 6. Implementation 3: proxy via IPVS
  • 6.1 Install IPVS
  • 6.2 Add virtual/real servers
  • 6.3 Verify
  • 6.4 Clean up
• 7. Implementation 4: proxy via tc-level ebpf
  • 7.1 Prerequisites
  • 7.2 Implementation
  • 7.3 Compile and load into kernel
  • 7.4 Verify
  • 7.5 Cleanup
  • 7.6 Explanations
  • 7.7 Improvements
• 8. Implementation 5: proxy via socket-level ebpf
  • 8.1 Hook earlier
  • 8.2 Implementation
  • 8.3 Compile, load and attach BPF code
  • 8.3 Verify
  • 8.4 Clean up
  • 8.5 Performance comparison
  • 8.6 Other explanations
• 9. Summary
• References
• Appendix
  • A1. webapp.yaml
  • A2. Service struct definition in Kubernetes