Skip to content

Instantly share code, notes, and snippets.

@chirauki
Last active November 4, 2024 16:48
Show Gist options
  • Select an option

  • Save chirauki/dc6e07bbf3e431aa2f90aa7c1e13d203 to your computer and use it in GitHub Desktop.

Select an option

Save chirauki/dc6e07bbf3e431aa2f90aa7c1e13d203 to your computer and use it in GitHub Desktop.
#
# kubectl create -f https://raw.githubusercontent.com/projectcalico/calico/v3.28.2/manifests/tigera-operator.yaml
#
# For MKE
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
labels:
subjectName: tigera-operator-tigera-operator
name: tigera-operator-tigera-operator:cluster-admin
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: cluster-admin
subjects:
- kind: ServiceAccount
name: tigera-operator
namespace: tigera-operator
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
labels:
subjectName: calico-system-calico-typha
name: calico-system-calico-typha:cluster-admin
namespace: calico-system
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: cluster-admin
subjects:
- kind: ServiceAccount
name: calico-typha
namespace: calico-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: istio-system-tsb-operator-control-plane
namespace: istio-system
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: cluster-admin
subjects:
- kind: ServiceAccount
name: tsb-operator-control-plane
namespace: istio-system
---
# For more information, see: https://docs.tigera.io/calico/latest/reference/installation/api#operator.tigera.io/v1.Installation
apiVersion: operator.tigera.io/v1
kind: Installation
metadata:
name: default
spec:
# Configures Calico networking.
calicoNetwork:
linuxDataplane: BPF
# ipPools:
# - name: default-ipv4-ippool
# blockSize: 26
# # This is MKE default
# cidr: 192.168.0.0/16
# encapsulation: VXLANCrossSubnet
# natOutgoing: Enabled
# nodeSelector: all()
# serviceCIDRs:
# # Again, MKE default
# - 10.96.0.0/16
variant: Calico
controlPlaneTolerations:
- effect: NoSchedule
operator: Exists
key: node.kubernetes.io/not-ready
- effect: NoSchedule
operator: Exists
key: com.docker.ucp.manager
- effect: NoSchedule
operator: Exists
key: node.cloudprovider.kubernetes.io/uninitialized
calicoNodeDaemonSet:
spec:
template:
spec:
tolerations:
- effect: NoSchedule
operator: Exists
key: node.kubernetes.io/not-ready
- effect: NoSchedule
operator: Exists
key: com.docker.ucp.manager
- effect: NoSchedule
operator: Exists
key: node.cloudprovider.kubernetes.io/uninitialized
csiNodeDriverDaemonSet:
spec:
template:
spec:
tolerations:
- effect: NoSchedule
operator: Exists
key: node.kubernetes.io/not-ready
- effect: NoSchedule
operator: Exists
key: com.docker.ucp.manager
- effect: NoSchedule
operator: Exists
key: node.cloudprovider.kubernetes.io/uninitialized
typhaDeployment:
spec:
template:
spec:
tolerations:
- effect: NoSchedule
operator: Exists
key: node.kubernetes.io/not-ready
- effect: NoSchedule
operator: Exists
key: com.docker.ucp.manager
- effect: NoSchedule
operator: Exists
key: node.cloudprovider.kubernetes.io/uninitialized
---
# This section configures the Calico API server.
# For more information, see: https://docs.tigera.io/calico/latest/reference/installation/api#operator.tigera.io/v1.APIServer
apiVersion: operator.tigera.io/v1
kind: APIServer
metadata:
name: default
spec:
apiServerDeployment:
spec:
template:
spec:
tolerations:
- effect: NoSchedule
operator: Exists
key: node.kubernetes.io/not-ready
- effect: NoSchedule
operator: Exists
key: com.docker.ucp.manager
- effect: NoSchedule
operator: Exists
key: node.cloudprovider.kubernetes.io/uninitialized
---
# For eBPF, we need to provide an API address. The address will be the NLB address created for the cluster
kind: ConfigMap
apiVersion: v1
metadata:
name: kubernetes-services-endpoint
namespace: tigera-operator
data:
KUBERNETES_SERVICE_HOST: 'ucp-nlb-mkeebpf0-fa2d1f98d6638b9a.elb.us-east-2.amazonaws.com'
KUBERNETES_SERVICE_PORT: '6443'
#
# Afer all of the above, some calico pods will still fail as core dns is not started due to masters not being ready.
# Operators need an special grant in MKE so they can set tolerations.
# TOKEN=$(curl -k -s -H "Content-Type: application/json" -X POST -d '{"username": "admin", "password": "tccTetrate"}' 'https://<MKE_API>/auth/login' | jq -r .auth_token)
# curl -k -H "Authorization: Bearer ${TOKEN}" -X PUT https://<MKE_API>/collectionGrants/system:serviceaccount:tigera-operator:tigera-operator/swarm/scheduler
# curl -k -H "Authorization: Bearer ${TOKEN}" -X PUT https://<MKE_API>/collectionGrants/system:serviceaccount:calico-system:calico-typha/swarm/scheduler
#
# Install AWS EBS CSI driver
# helm upgrade --install aws-ebs-csi-driver \
# --namespace kube-system \
# aws-ebs-csi-driver/aws-ebs-csi-driver
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment