Storage

AWS EKS ์Šคํ„ฐ๋”” 3์ฃผ์ฐจ - Storage

  • kkumtree

2023-05-12T05:36:38+09:00

์ด๋ฒˆ ์ฃผ์ฐจ์—๋Š” ์Šคํ† ๋ฆฌ์ง€์— ๋Œ€ํ•ด ์‹ค์Šต์„ ์ง„ํ–‰ํ•ด๋ณด์•˜์Šต๋‹ˆ๋‹ค. ์ง€๋‚œ๋ฒˆ kOps ์Šคํ„ฐ๋””์—์„œ ๋‹ค๋ฃจ์—ˆ๋˜ ๋‚ด์šฉ์ด์ง€๋งŒ, ๋ถ€์กฑํ–ˆ๋˜ ๋‚ด์šฉ์„ ๋ณด์ถฉํ•˜๋ฉด์„œ ์ž‘์„ฑ์„ ํ•ด๋ณด์•˜์Šต๋‹ˆ๋‹ค.

์ฃผ์š”ํ•œ ๋‚ด์šฉ์€…

  • NodeAffinity๋ฅผ ์ด์šฉํ•œ ๋ผ๋ฒจ๋ง
  • AWS EBS controller์˜ ๊ฒฝ์šฐ, AWS managed policy๋ฅผ ํ™œ์šฉ
  • AWS Volume SnapShots Controller๋ฅผ ํ†ตํ•œ ๋ณผ๋ฅจ ๋ฐฑ์—…
  • AWS EFS controller์—์„œ์˜ ๋™์  ํ”„๋กœ๋น„์ €๋‹
  • AWS EKS ์‹ ๊ทœ ๋…ธ๋“œ๊ทธ๋ฃน ์ƒ์„ฑ

๋ณ„๋„๋กœ kube-ops-view์˜ ๊ฒฝ์šฐ, ์›น์œผ๋กœ ํ™•์ธํ•  ์ˆ˜ ์žˆ์„ ๋•Œ๊นŒ์ง€ ์‹œ๊ฐ„์ด ์†Œ์š”๋œ๋‹ค๋Š” ์ ์ด ์žˆ์Šต๋‹ˆ๋‹ค.

1. ์‹ค์Šต ํ™˜๊ฒฝ ๋ฐฐํฌ

  • 2์ฃผ์ฐจ์— ์‹ค์Šตํ–ˆ๋˜ ๋‚ด์šฉ๋“ค์„ ๋ฏธ๋ฆฌ ๋ฐฐํฌ
    1. AWS LB
    2. ExternalDNS
    3. kube-ops-view
  • context ์ด๋ฆ„ ๋ณ€๊ฒฝ
    • ์ง€๋‚œ ๋ฒˆ๊นŒ์ง€ pkos๊ฐ€ ๋œจ๋Š” ํ˜„์ƒ์ด ์žˆ์—ˆ๋Š”๋ฐ, ๋‹‰๋„ค์ž„์„ ๋ณ„๋„ ์ง€์ •ํ•  ์ˆ˜ ์žˆ์Œ
  • EFS ์ƒ์„ฑ ๊ด€๋ จ cloudformation์ด ์ถ”๊ฐ€๋˜์—ˆ์Œ
# ์‹ค์Šต YAML ํŒŒ์ผ
curl -O https://s3.ap-northeast-2.amazonaws.com/cloudformation.cloudneta.net/K8S/eks-oneclick2.yaml

# cloudformation ์Šคํƒ ์ƒ์„ฑ
aws cloudformation deploy --template-file eks-oneclick2.yaml --stack-name myeks --parameter-overrides KeyName=aews SgIngressSshCidr=$(curl -s ipinfo.io/ip)/32  MyIamUserAccessKeyID=AKIA5... MyIamUserSecretAccessKey=CVNa2... ClusterBaseName=myeks --region ap-northeast-2

ssh -i ~/.ssh/aews.pem ec2-user@$(aws cloudformation describe-stacks --stack-name myeks --query 'Stacks[*].Outputs[0].OutputValue' --output text)

# default ๋„ค์ž„์ŠคํŽ˜์ด์Šค ์ ์šฉ
kubectl ns default

# (์˜ต์…˜) context ์ด๋ฆ„ ๋ณ€๊ฒฝ
NICK=kkumtree
kubectl ctx
kubectl config rename-context [email protected] $NICK@myeks

# EFS ํ™•์ธ : AWS ๊ด€๋ฆฌ์ฝ˜์†” EFS ํ™•์ธ
EfsFsId=$(aws efs describe-file-systems --query 'FileSystems[*].FileSystemId' --output text)
echo $EfsFsId
mount -t nfs4 -o nfsvers=4.1,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2,noresvport $EfsFsId.efs.ap-northeast-2.amazonaws.com:/ /mnt/myefs
df -hT --type nfs4
mount | grep nfs4
echo "Test efs exist with file " > /mnt/myefs/memo.txt
cat /mnt/myefs/memo.txt
rm -f /mnt/myefs/memo.txt

# ์Šคํ† ๋ฆฌ์ง€ํด๋ž˜์Šค ๋ฐ CSI ๋…ธ๋“œ ํ™•์ธ
kubectl get sc
kubectl get sc gp2 -o yaml | yh
kubectl get csinodes

# ๋…ธ๋“œ ์ •๋ณด ํ™•์ธ
kubectl get node --label-columns=node.kubernetes.io/instance-type,eks.amazonaws.com/capacityType,topology.kubernetes.io/zone
eksctl get iamidentitymapping --cluster myeks

# ๋…ธ๋“œ IP ํ™•์ธ ๋ฐ PrivateIP ๋ณ€์ˆ˜ ์ง€์ •
N1=$(kubectl get node --label-columns=topology.kubernetes.io/zone --selector=topology.kubernetes.io/zone=ap-northeast-2a -o jsonpath={.items[0].status.addresses[0].address})
N2=$(kubectl get node --label-columns=topology.kubernetes.io/zone --selector=topology.kubernetes.io/zone=ap-northeast-2b -o jsonpath={.items[0].status.addresses[0].address})
N3=$(kubectl get node --label-columns=topology.kubernetes.io/zone --selector=topology.kubernetes.io/zone=ap-northeast-2c -o jsonpath={.items[0].status.addresses[0].address})
echo "export N1=$N1" >> /etc/profile
echo "export N2=$N2" >> /etc/profile
echo "export N3=$N3" >> /etc/profile
echo $N1, $N2, $N3

# ๋…ธ๋“œ ๋ณด์•ˆ๊ทธ๋ฃน ID ํ™•์ธ
NGSGID=$(aws ec2 describe-security-groups --filters Name=group-name,Values=*ng1* --query "SecurityGroups[*].[GroupId]" --output text)
aws ec2 authorize-security-group-ingress --group-id $NGSGID --protocol '-1' --cidr 192.168.1.100/32

# ์›Œ์ปค ๋…ธ๋“œ SSH ์ ‘์†
ssh ec2-user@$N1 hostname
ssh ec2-user@$N2 hostname
ssh ec2-user@$N3 hostname

# ๋…ธ๋“œ์— ํˆด ์„ค์น˜
ssh ec2-user@$N1 sudo yum install links tree jq tcpdump sysstat -y
ssh ec2-user@$N2 sudo yum install links tree jq tcpdump sysstat -y
ssh ec2-user@$N3 sudo yum install links tree jq tcpdump sysstat -y

# AWS LB, ExternalDNS ์„ค์น˜
helm repo add eks https://aws.github.io/eks-charts
helm repo update
helm install aws-load-balancer-controller eks/aws-load-balancer-controller -n kube-system --set clusterName=$CLUSTER_NAME \
  --set serviceAccount.create=false --set serviceAccount.name=aws-load-balancer-controller

# ExternalDNS
MyDomain=awskops.click
MyDnzHostedZoneId=$(aws route53 list-hosted-zones-by-name --dns-name "${MyDomain}." --query "HostedZones[0].Id" --output text)
echo $MyDomain, $MyDnzHostedZoneId
curl -s -O https://raw.githubusercontent.com/gasida/PKOS/main/aews/externaldns.yaml
MyDomain=$MyDomain MyDnzHostedZoneId=$MyDnzHostedZoneId envsubst < externaldns.yaml | kubectl apply -f -

1-1. kube-ops-view

  • ์‹œ๊ฐ์ ์œผ๋กœ ํ˜„์žฌ k8s์˜ ์ƒํƒœ๋ฅผ ๋ณผ ์ˆ˜ ์žˆ๋Š” ํˆด
  • ์•ˆ๋˜๋Š” ์ค„ ์•Œ์•˜๋Š”๋ฐ, ๋ทฐ์–ด๊ฐ€ ๋œฐ ๋•Œ๊นŒ์ง€ ์‹œ๊ฐ„์ด ๊ฑธ๋ฆฌ๋Š” ๊ฒƒ์ด์—ˆ์Œ.

1-kube-ops-view

kkumtree

Source code on GitHub

ยฉ 2025 kkumtree and contributors All rights reserved.
Licensed under
CC BY-NC-ND 4.0