Karpenter

AWS EKS ์Šคํ„ฐ๋”” 5์ฃผ์ฐจ - Autoscaling

  • kkumtree

2023-05-22T19:23:37+09:00

์ด๋ฒˆ ์ฃผ์ฐจ๋Š” ์˜คํ† ์Šค์ผ€์ผ๋ง์„ ๋ฉ”์ธ์œผ๋กœ ํ•˜์—ฌ, ์ˆ˜ํ‰/์ˆ˜์ง ํ”„๋กœ๋น„์ €๋‹์„ ํ•™์Šตํ•ด๋ณด์•˜์Šต๋‹ˆ๋‹ค.
๋งˆ์ง€๋ง‰์—๋Š” ๊ณ ์„ฑ๋Šฅ ์˜คํ† ์Šค์ผ€์ผ๋Ÿฌ์ธ Karpenter๋ฅผ ๋ณ„๋„๋กœ ์‹ค์Šตํ•ด๋ณด์•˜์Šต๋‹ˆ๋‹ค. ํŠนํžˆ..

  • HPA custom metrics(์‚ฌ์šฉ์ž ์ •์˜ ๋ฉ”ํŠธ๋ฆญ) ์ ์šฉ

  • YAML ์„ค์ •๊ฐ’์„ CPU๋กœ ๋งž์ถ˜ ๊ฒƒ์„ ์žŠ๊ณ , ํ”„๋กœ๋น„์ €๋‹์„ ์ž˜๋ชป ์˜ˆ์ธกํ•œ ๊ฒƒ๋„ ํ•จ๊ป˜ ๊ณต์œ ํ•ฉ๋‹ˆ๋‹ค.

  • AutoScaling

    • HPA: Horizontal Pod Autoscaler
    • VPA: Vertical Pod Autoscaler
    • CA: Cluster Autoscaler
      • ๊ฐ CSP ์˜์กด์ , ์›Œ์ปค ๋…ธ๋“œ ๋ ˆ๋ฒจ์—์„œ์˜ ์˜คํ† ์Šค์ผ€์ผ๋ง

1. ์‹ค์Šต ํ™˜๊ฒฝ ๋ฐฐํฌ

  • 4์ฃผ์ฐจ์˜ ์ดˆ๊ธฐ ๋ฐฐํฌ ๋‚ด์šฉ์— p8s ๋ฐ Grafana๋ฅผ ์ถ”๊ฐ€ํ•˜์—ฌ ๋ฐฐํฌ
    • verticalPodAutoscaler ํ™œ์„ฑํ™”
    • ์ถ”์ฒœ ๋Œ€์‹œ๋ณด๋“œ: 15757, 17900, 15172
curl -O https://s3.ap-northeast-2.amazonaws.com/cloudformation.cloudneta.net/K8S/eks-oneclick4.yaml

# ์ดํ•˜ ์ค‘๋žต

## Prometheus & Grafana ์„ค์น˜

# ์ธ์ฆ์„œ ARN
CERT_ARN=`aws acm list-certificates --query 'CertificateSummaryList[].CertificateArn[]' --output text`
echo $CERT_ARN

# ํŒŒ๋ผ๋ฏธํ„ฐ ํŒŒ์ผ ์ƒ์„ฑ ๋ฐ ๋ฐฐํฌ
cat <<EOT > monitor-values.yaml
prometheus:
  prometheusSpec:
    podMonitorSelectorNilUsesHelmValues: false
    serviceMonitorSelectorNilUsesHelmValues: false
    retention: 5d
    retentionSize: "10GiB"

  verticalPodAutoscaler:
    enabled: true

  ingress:
    enabled: true
    ingressClassName: alb
    hosts: 
      - prometheus.$MyDomain
    paths: 
      - /*
    annotations:
      alb.ingress.kubernetes.io/scheme: internet-facing
      alb.ingress.kubernetes.io/target-type: ip
      alb.ingress.kubernetes.io/listen-ports: '[{"HTTPS":443}, {"HTTP":80}]'
      alb.ingress.kubernetes.io/certificate-arn: $CERT_ARN
      alb.ingress.kubernetes.io/success-codes: 200-399
      alb.ingress.kubernetes.io/load-balancer-name: myeks-ingress-alb
      alb.ingress.kubernetes.io/group.name: study
      alb.ingress.kubernetes.io/ssl-redirect: '443'

grafana:
  defaultDashboardsTimezone: Asia/Seoul
  adminPassword: prom-operator

  ingress:
    enabled: true
    ingressClassName: alb
    hosts: 
      - grafana.$MyDomain
    paths: 
      - /*
    annotations:
      alb.ingress.kubernetes.io/scheme: internet-facing
      alb.ingress.kubernetes.io/target-type: ip
      alb.ingress.kubernetes.io/listen-ports: '[{"HTTPS":443}, {"HTTP":80}]'
      alb.ingress.kubernetes.io/certificate-arn: $CERT_ARN
      alb.ingress.kubernetes.io/success-codes: 200-399
      alb.ingress.kubernetes.io/load-balancer-name: myeks-ingress-alb
      alb.ingress.kubernetes.io/group.name: study
      alb.ingress.kubernetes.io/ssl-redirect: '443'

defaultRules:
  create: false
kubeControllerManager:
  enabled: false
kubeEtcd:
  enabled: false
kubeScheduler:
  enabled: false
alertmanager:
  enabled: false
EOT

kubectl create ns monitoring
helm install kube-prometheus-stack prometheus-community/kube-prometheus-stack --version 45.27.2 \
--set prometheus.prometheusSpec.scrapeInterval='15s' --set prometheus.prometheusSpec.evaluationInterval='15s' \
-f monitor-values.yaml --namespace monitoring

# metrics-server ๋ฐฐํฌ
kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml

1-1. EKS Node Viewer ์„ค์น˜

  • ํŒŒ๋“œ ๋ฆฌ์†Œ์Šค์— ๋Œ€ํ•œ ์š”์ฒญ ์ •๋ณด๋ฅผ ํ™•์ธํ•  ์ˆ˜ ์žˆ๋Š” ๋Œ€์‹œ๋ณด๋“œ
    • ํ•ด๋‹น ๋…ธ๋“œ์— ํ• ๋‹น ๊ฐ€๋Šฅํ•œ ์šฉ๋Ÿ‰์„ ์‹œ๊ฐ์ ์œผ๋กœ ํ‘œ์‹œ
  • ์‹ค์ œ ์‚ฌ์šฉ๋Ÿ‰์ด ์•„๋‹ˆ๋ผ, ์š”์ฒญ๋œ ๋ฆฌ์†Œ์Šค(CPU, Memory)์— ๋Œ€ํ•œ ํ‘œ์‹œ
  • ์‹ค์Šต ์Šค์ฑ… ์ƒ์—์„œ go ์„ค์น˜ ๋ฐ ๋ทฐ์–ด ์„ค์น˜์‹œ ๋‹ค์†Œ ์‹œ๊ฐ„์ด ์†Œ์š” (์•ฝ 5๋ถ„)
  • Karpenter ์‹ค์Šต ์‹œ์—๋„ ์–ธ๊ธ‰๋˜๊ฒ ์ง€๋งŒ, EKS๊ฐ€ ๊ตฌ์ถ•๋œ ๋’ค์— ์‚ฌ์šฉ์ด ๊ฐ€๋Šฅํ•˜๋‹ค.
# go ๋ฐ EKS Node Viewer ์„ค์น˜
yum install -y go
go install github.com/awslabs/eks-node-viewer/cmd/eks-node-viewer@latest

# EKS Node Viewer ์‹คํ–‰
tree ~/go/bin
cd ~/go/bin && ./eks-node-viewer

## EKS Node Viewer ๋ช…๋ น ์ƒ˜ํ”Œ
# Display both CPU and Memory Usage
./eks-node-viewer --resources cpu,memory

# Karenter nodes only
./eks-node-viewer --node-selector "karpenter.sh/provisioner-name"

# Display extra labels, i.e. AZ
./eks-node-viewer --extra-labels topology.kubernetes.io/zone

# Specify a particular AWS profile and region
AWS_PROFILE=myprofile AWS_REGION=ap-northeast-2

## ๊ธฐ๋ณธ ์˜ต์…˜ ํ™˜๊ฒฝ ๋ณ€์ˆ˜
# select only Karpenter managed nodes
node-selector=karpenter.sh/provisioner-name

# display both CPU and memory
resources=cpu,memory

EKS node viewer

kkumtree

Source code on GitHub

ยฉ 2025 kkumtree and contributors All rights reserved.
Licensed under
CC BY-NC-ND 4.0