EKS Best Practices: A Practical Guide from the Trenches

Source: https://www.linkedin.com/pulse/getting-started-kubernetes-amazon-eks-part-1-omar-alaa/

Last week, while debugging a production issue, I realized how crucial it is to follow best practices when running Amazon EKS clusters. After years of managing Kubernetes deployments, I’ve learned (sometimes the hard way) what works and what doesn’t. Let me share some key practices that have saved my team countless headaches.

Cluster Management: Start with the Basics

The foundation of a well-running EKS cluster starts with proper setup. During my first major deployment, I made the mistake of not planning our node groups properly. Now, I always ensure we follow these core practices:

Use managed node groups whenever possible. They’ve simplified our life tremendously by handling the EC2 instance provisioning and lifecycle management. Here’s a quick example of how we define our node groups:

apiVersion: eksctl.io/v1alpha5
kind: ClusterConfig
metadata:
  name: prod-cluster
  region: us-west-2
managedNodeGroups:
  - name: managed-ng-1
    instanceType: m5.large
    minSize: 2
    maxSize: 5
    desiredCapacity: 3
    labels: {role: worker}

Resource Management: Learn from Our Mistakes

I remember the day our production services started failing because we hadn’t set resource requests and limits properly. It was a rough afternoon of firefighting that taught us valuable lessons. Here’s what we do now:

Always set both requests and limits for your pods. It’s like setting ground rules for a shared apartment — everyone needs to know their boundaries. A practical example from our web service deployment:

resources:
  requests:
    memory: "256Mi"
    cpu: "250m"
  limits:
    memory: "512Mi"
    cpu: "500m"

Security: Don’t Wait Until It’s Too Late

Security isn’t just a checkbox — it’s an ongoing process. After a close call with an exposed dashboard (thankfully caught during an internal audit), we’ve implemented several crucial security measures:

1. Use AWS IAM roles for service accounts (IRSA). This provides fine-grained access control for your pods. Here’s how we configure it:

apiVersion: v1
kind: ServiceAccount
metadata:
  name: app-service-account
  annotations:
    eks.amazonaws.com/role-arn: arn:aws:iam::ACCOUNT_ID:role/app-role

Monitoring and Logging: Your Future Self Will Thank You

The importance of good observability hit home when we spent hours trying to debug a performance issue that could have been solved in minutes with proper monitoring. Now we maintain:

- Prometheus for metrics collection
- Grafana for visualization
- AWS CloudWatch for log aggregation

Here’s a snippet of our Prometheus configuration:

apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: app-monitor
spec:
  selector:
    matchLabels:
      app: my-app
  endpoints:
  - port: metrics

Cost Optimization: The Often Forgotten Aspect

During our last quarterly review, we found we were overspending on our EKS clusters. Here’s what we did to optimize costs:

Use Spot Instances for non-critical workloads. We created a separate node group for our dev environment:

managedNodeGroups:
  - name: spot-ng
    instanceType: m5.large
    spot: true
    minSize: 1
    maxSize: 3

Networking: Keep It Simple but Secure

After experiencing network issues during our multi-region expansion, we’ve adopted these networking practices:

- Use security groups effectively
- Implement proper network policies
- Plan your VPC CIDR ranges carefully

Here’s an example of a network policy we use to restrict pod communication:

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: api-allow
spec:
  podSelector:
    matchLabels:
      app: api
  ingress:
  - from:
    - podSelector:
        matchLabels:
          role: frontend

Conclusion

These practices have evolved through real-world experience and countless problem-solving sessions. They’re not just theoretical — they’re battle-tested solutions that have helped us maintain reliable, secure, and efficient EKS clusters.

Remember, what works for one team might not work for another. Start with these practices as a foundation, but don’t be afraid to adapt them to your specific needs. The key is to stay curious, keep learning, and always be ready to improve your infrastructure.

What best practices have you found useful in your EKS journey? I’d love to hear about your experiences in the comments below.

Found this helpful? Let’s connect!

🔗 Follow me on LinkedIn for more tech insights and best practices.

💡 Have thoughts or questions? Drop them in the comments below — I’d love to hear your perspective.

If this article added value to your day, consider giving it a 👏 to help others discover it too.

Until next time!