Skip to content

Instantly share code, notes, and snippets.

@sudermanjr

sudermanjr/KIAM_Notes.md

Last active February 6, 2019 20:32
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save sudermanjr/c71532a11b5049d21382373ad91fe26e to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save sudermanjr/c71532a11b5049d21382373ad91fe26e to your computer and use it in GitHub Desktop.
Download ZIP
Raw
KIAM_Notes.md

KIAM Implementation Notes

Roles and IAM

Masters already have assume-role (presumably), so we just need to create the roles. Module found here works quite well: uswitch/kiam#25 (comment)

Certs

For some reason kiam-server expects the certs to be valid for IP 127.0.0.1 as well as for the name of the service. I generated the certs using the instructions in the kiam repo, and it worked. Cert-manager doesn’t do IPs as well as DNS names yet, but it’s on the way in v0.7: https://github.com/jetstack/cert-manager/pull/1128/files

The manually generated certs have this SAN block: X509v3 Subject Alternative Name: DNS:kiam-server, DNS:127.0.0.1:443, DNS:127.0.0.1:9610, IP Address:127.0.0.1, URI:kiam-server:443

and the cert-manager generated ones the best I can get is this: X509v3 Subject Alternative Name: DNS:kiam-server, DNS:kiam-server.kiam, DNS:kiam-server:443, DNS:localhost, DNS:localhost:443, DNS:localhost:9610, DNS:127.0.0.1, DNS:127.0.0.1:443, DNS:127.0.0.1:9610

I tried using localhost instead and setting the healthchecks to use localhost, but it doesn't seem to help.

This all seems to be caused by the findings in this issue where they reached the same conclusion I did. uswitch/kiam#162

Host CA Certs

KIAM doesn't add ca-certificates package to the container, so you have to mount the host /etc/ssl/certs. The chart has a flag to deal with this:

server:
  extraHostPathMounts:
    - name: cacerts
      mountPath: /etc/ssl/certs
      hostPath: /etc/ssl/certs
      readOnly: true

Flannel

According to the readme, the agent needs to set an iptables rule to intercept metadata traffic. I believe this means that we need to configure the helm chart like so:

agent:
  host:
    iptables: true
    interface: flannel+

Other Notes

  • The server must run on masters, the agent only on the nodes.
  • Versions seem to be moving quickly. Keep an eye on them (v3.0 just dropped, but chart doesn't necessarily keep up)
Raw
test-kiam-deploy.yaml
---
apiVersion: v1
kind: Namespace
metadata:
name: iam-example
annotations:
iam.amazonaws.com/permitted: "arn:aws:iam::ACCOUNTID:role/CLUSTERNAME_external-dns"
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: utilities
namespace: iam-example
labels:
app: utilities
spec:
replicas: 1
selector:
matchLabels:
app: utilities
template:
metadata:
annotations:
iam.amazonaws.com/role: "arn:aws:iam::ACCOUNTID:role/CLUSTERNAME_external-dns"
labels:
app: utilities
spec:
containers:
- name: utilities
image: quay.io/sudermanjr/utilities:latest
# Just spin & wait forever
command: [ "/bin/bash", "-c", "--" ]
args: [ "while true; do sleep 30; done;" ]
securityContext:
readOnlyRootFilesystem: true
allowPrivilegeEscalation: false
runAsNonRoot: true
runAsUser: 10324
capabilities:
drop:
- ALL
resources:
requests:
cpu: 30m
memory: 64Mi
limits:
cpu: 100m
memory: 128Mi
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment