🌐 Cilium eBPF Networking on AlmaLinux 9: Complete Guide

Welcome to the future of Kubernetes networking! 🚀 Today we’ll set up Cilium on AlmaLinux 9, unleashing the power of eBPF for blazing-fast, secure networking! Let’s revolutionize your cluster! ✨🔥

🤔 Why is Cilium Important?

Imagine networking that’s 10x faster and smarter! 🎯 That’s Cilium’s magic! Here’s why it’s revolutionary:

• ⚡ Lightning Performance - eBPF bypasses iptables for incredible speed!
• 🔍 Deep Observability - See every packet with Hubble’s X-ray vision
• 🛡️ Identity-Based Security - Protect based on service identity, not IPs
• 🌍 Multi-Cluster - Connect clusters across clouds seamlessly
• 📊 Service Mesh - Built-in L7 load balancing without sidecars
• 🔐 Transparent Encryption - Automatic WireGuard encryption
• 🎨 Network Policies - Advanced L3-L7 policies with DNS awareness
• 🚀 No Overhead - Kernel-native performance with eBPF

🎯 What You Need

Before we supercharge your networking, gather these:

• ✅ AlmaLinux 9 server (8GB RAM minimum, 16GB recommended)
• ✅ Kubernetes cluster 1.16+ (K3s, K8s, or any flavor)
• ✅ Kernel 4.19+ (5.4+ recommended for all features)
• ✅ Helm 3.0+ installed
• ✅ kubectl configured
• ✅ Multi-node cluster (3+ nodes for production)
• ✅ Root or sudo access
• ✅ Ready for networking magic! 🎉

📝 Step 1: Prepare AlmaLinux for Cilium

Let’s prepare your system for eBPF awesomeness! 🛠️

Verify System Requirements

# Check kernel version (needs 4.19+, ideally 5.4+)
uname -r  # AlmaLinux 9 has 5.14+, perfect!

# Check for eBPF support
ls /sys/fs/bpf  # Should exist
mount | grep bpf  # Should show bpf filesystem

# Install required packages
sudo dnf install -y \
  kernel-devel-$(uname -r) \
  kernel-headers-$(uname -r) \
  bpftool \
  iproute-tc

# Enable IP forwarding (required)
echo 'net.ipv4.ip_forward = 1' | sudo tee -a /etc/sysctl.conf
echo 'net.ipv6.conf.all.forwarding = 1' | sudo tee -a /etc/sysctl.conf
sudo sysctl -p

# Disable swap (Kubernetes requirement)
sudo swapoff -a
sudo sed -i '/ swap / s/^/#/' /etc/fstab

Prepare Kubernetes Cluster

# Remove existing CNI if present (like Flannel)
kubectl delete -f https://raw.githubusercontent.com/flannel-io/flannel/master/Documentation/kube-flannel.yml 2>/dev/null || true

# Clean up CNI configuration
sudo rm -rf /etc/cni/net.d/*

# Label nodes for Cilium
kubectl label nodes --all kubernetes.io/hostname-

# Verify cluster is ready
kubectl get nodes  # Should show NotReady (no CNI yet)
kubectl get pods -A  # CoreDNS should be pending

🔧 Step 2: Install Cilium CLI

Let’s get the Cilium command-line tool! 🎊

Install Cilium CLI

# Download latest Cilium CLI
CILIUM_CLI_VERSION=$(curl -s https://raw.githubusercontent.com/cilium/cilium-cli/master/stable.txt)
CLI_ARCH=amd64
if [ "$(uname -m)" = "aarch64" ]; then CLI_ARCH=arm64; fi

curl -L --fail --remote-name-all https://github.com/cilium/cilium-cli/releases/download/${CILIUM_CLI_VERSION}/cilium-linux-${CLI_ARCH}.tar.gz{,.sha256sum}

# Verify checksum
sha256sum --check cilium-linux-${CLI_ARCH}.tar.gz.sha256sum

# Extract and install
sudo tar xzvfC cilium-linux-${CLI_ARCH}.tar.gz /usr/local/bin
rm cilium-linux-${CLI_ARCH}.tar.gz{,.sha256sum}

# Verify installation
cilium version  # Shows CLI version

Install Hubble CLI

# Download Hubble CLI for observability
HUBBLE_VERSION=$(curl -s https://raw.githubusercontent.com/cilium/hubble/master/stable.txt)
HUBBLE_ARCH=amd64
if [ "$(uname -m)" = "aarch64" ]; then HUBBLE_ARCH=arm64; fi

curl -L --fail --remote-name-all https://github.com/cilium/hubble/releases/download/$HUBBLE_VERSION/hubble-linux-${HUBBLE_ARCH}.tar.gz{,.sha256sum}

# Verify and install
sha256sum --check hubble-linux-${HUBBLE_ARCH}.tar.gz.sha256sum
sudo tar xzvfC hubble-linux-${HUBBLE_ARCH}.tar.gz /usr/local/bin
rm hubble-linux-${HUBBLE_ARCH}.tar.gz{,.sha256sum}

# Verify Hubble
hubble version  # Shows Hubble version

🌟 Step 3: Deploy Cilium

Time to unleash the eBPF power! 🚀

Install Cilium with CLI (Recommended)

# Install Cilium with default configuration
cilium install

# Wait for Cilium to be ready
cilium status --wait

# You should see:
#  /¯¯\
# /¯¯\__/¯¯\    Cilium:         OK
# \__/¯¯\__/    Operator:       OK
# /¯¯\__/¯¯\    Hubble:         disabled
# \__/¯¯\__/    ClusterMesh:    disabled
#    \__/

# Enable Hubble for observability
cilium hubble enable --ui

# Wait for Hubble to be ready
cilium status --wait

Alternative: Install with Helm

# Add Cilium Helm repository
helm repo add cilium https://helm.cilium.io/
helm repo update

# Create values file for customization
cat <<EOF > cilium-values.yaml
# eBPF configuration
bpf:
  masquerade: true
  clockProbe: true
  preallocateMaps: true

# Hubble configuration
hubble:
  enabled: true
  relay:
    enabled: true
  ui:
    enabled: true
    ingress:
      enabled: true
      hosts:
        - hubble.yourdomain.com

# Network configuration
tunnel: disabled  # Use native routing
ipam:
  mode: kubernetes
  
# Enable advanced features
l7Proxy: true
encryption:
  enabled: true
  type: wireguard

# Performance tuning
operator:
  replicas: 2
  
resources:
  limits:
    cpu: 1000m
    memory: 1Gi
  requests:
    cpu: 100m
    memory: 128Mi
EOF

# Install Cilium
helm install cilium cilium/cilium \
  --version 1.14.5 \
  --namespace kube-system \
  --values cilium-values.yaml

# Verify installation
kubectl -n kube-system get pods -l k8s-app=cilium

✅ Step 4: Configure Network Policies

Let’s secure your network with powerful policies! 🔒

Create L3/L4 Network Policy

# Create a namespace for testing
kubectl create namespace production

# Deploy sample application
kubectl apply -n production -f - <<EOF
apiVersion: apps/v1
kind: Deployment
metadata:
  name: webapp
spec:
  replicas: 3
  selector:
    matchLabels:
      app: webapp
  template:
    metadata:
      labels:
        app: webapp
    spec:
      containers:
      - name: nginx
        image: nginx
        ports:
        - containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
  name: webapp
spec:
  selector:
    app: webapp
  ports:
  - port: 80
EOF

# Create Cilium Network Policy
kubectl apply -n production -f - <<EOF
apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
  name: webapp-policy
spec:
  endpointSelector:
    matchLabels:
      app: webapp
  ingress:
  - fromEndpoints:
    - matchLabels:
        app: frontend
    toPorts:
    - ports:
      - port: "80"
        protocol: TCP
  egress:
  - toEndpoints:
    - matchLabels:
        app: database
    toPorts:
    - ports:
      - port: "5432"
        protocol: TCP
  - toFQDNs:
    - matchPattern: "*.example.com"
    toPorts:
    - ports:
      - port: "443"
        protocol: TCP
EOF

# Verify policy is applied
kubectl -n production get cnp
cilium endpoint list

Create L7 Application Policy

# Create L7 policy with HTTP rules
kubectl apply -n production -f - <<EOF
apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
  name: api-l7-policy
spec:
  endpointSelector:
    matchLabels:
      app: api
  ingress:
  - fromEndpoints:
    - matchLabels:
        role: frontend
    toPorts:
    - ports:
      - port: "8080"
        protocol: TCP
      rules:
        http:
        - method: "GET"
          path: "/api/v1/users"
        - method: "POST"
          path: "/api/v1/users"
          headers:
          - 'Authorization: Bearer .*'
EOF

🎮 Quick Examples

Let’s explore Cilium’s amazing features! 🎬

Example 1: Access Hubble UI

# Port-forward Hubble UI
cilium hubble ui

# Access at http://localhost:12000
# You'll see beautiful service map!

# Or expose via NodePort
kubectl patch svc hubble-ui -n kube-system \
  -p '{"spec": {"type": "NodePort", "ports": [{"port": 80, "nodePort": 30080}]}}'

# Access at http://NODE_IP:30080

Example 2: Observe Network Flows

# Watch all flows in real-time
hubble observe

# Filter by namespace
hubble observe --namespace production

# See HTTP traffic
hubble observe --protocol http

# Watch specific pod
hubble observe --pod production/webapp-xxx

# See dropped packets (policy violations)
hubble observe --verdict DROPPED

# Export flows as JSON
hubble observe -o json | jq

Example 3: Enable Transparent Encryption

# Enable WireGuard encryption
cilium encrypt enable

# Verify encryption status
cilium encrypt status

# You should see:
# Encryption: Wireguard [NodeEncryption: Enabled, IPsec: Disabled]

# Check node-to-node encryption
kubectl exec -n kube-system ds/cilium -- cilium encrypt status

🚨 Fix Common Problems

Hit a snag? Here are solutions! 💪

Problem 1: Nodes Not Ready

# Check Cilium status
cilium status

# If Cilium not running, check logs
kubectl -n kube-system logs ds/cilium

# Common fix: restart Cilium
kubectl -n kube-system rollout restart ds/cilium

# Verify eBPF programs loaded
sudo bpftool prog list | grep cilium

# Check for CNI configuration
ls -la /etc/cni/net.d/

Problem 2: Pods Cannot Communicate

# Run connectivity test
cilium connectivity test

# Check endpoint status
cilium endpoint list

# Verify network policies
kubectl get cnp -A

# Test pod-to-pod connectivity
kubectl run test --image=busybox --rm -it -- sh
# Inside pod:
nslookup kubernetes
ping another-pod-ip

# Check for policy drops
hubble observe --verdict DROPPED

Problem 3: Hubble Not Working

# Check Hubble relay status
kubectl -n kube-system get pods -l k8s-app=hubble-relay

# Restart Hubble
cilium hubble disable
cilium hubble enable

# Check Hubble API
hubble status

# Port-forward manually if needed
kubectl port-forward -n kube-system svc/hubble-relay 4245:80 &
hubble --server localhost:4245 status

📋 Simple Commands Summary

Your Cilium command toolkit! 📚

💡 Tips for Success

Master Cilium with these pro tips! 🏆

Performance Optimization

• ⚡ Use native routing mode for best performance
• 🎯 Enable BPF host routing for node traffic
• 📊 Preallocate BPF maps for consistency
• 🚀 Tune MTU size for your network
• 💾 Monitor BPF map usage

Security Best Practices

• 🔐 Always enable encryption in production
• 🛡️ Start with deny-all network policies
• 📝 Use L7 policies for API protection
• 🔍 Monitor dropped packets regularly
• 🎯 Use identity-based policies over IP-based

Observability Tips

• 📈 Set up Grafana dashboards for metrics
• 🔍 Use Hubble CLI for troubleshooting
• 📊 Export flows to external systems
• ⚠️ Set alerts for policy violations
• 📝 Enable flow logs for audit

🏆 What You Learned

Incredible work! You’re now a Cilium expert! 🎉 You can:

• ✅ Install Cilium CNI on AlmaLinux 9
• ✅ Configure eBPF-based networking
• ✅ Enable Hubble observability platform
• ✅ Create advanced network policies
• ✅ Monitor network flows in real-time
• ✅ Enable transparent encryption
• ✅ Troubleshoot connectivity issues
• ✅ Optimize network performance

🎯 Why This Matters

You’ve revolutionized your Kubernetes networking! 🚀 With Cilium:

• 10x Performance - eBPF bypasses iptables overhead
• Complete Visibility - See every packet in your cluster
• Identity Security - Protect services, not IP addresses
• Zero Trust - Automatic encryption everywhere
• Service Mesh - L7 load balancing without sidecars
• Multi-Cloud - Connect clusters across providers
• Future Proof - eBPF is the future of networking

Your cluster now has the most advanced networking stack available! No more iptables complexity, no more networking black boxes. Everything is fast, secure, and observable.

Keep exploring features like ClusterMesh for multi-cluster, BGP for advanced routing, and bandwidth management. You’re running the same technology as the world’s largest clouds! 🌟

Remember: With great power comes great networking - Cilium delivers both! Happy networking! 🎊🌐

P.S. - Join the Cilium community, attend eBPF Summit, and share your networking wins! The future is eBPF-powered! ⭐🙌