Quick start failing to register instances with API server

[faermanj]

/kind bug

What steps did you take and what happened:
I followed the quick start guide, but failed to create a working cluster.

Here are the commands executed:

kind create cluster
kubectl cluster-info
clusterawsadm bootstrap iam create-cloudformation-stack
export AWS_B64ENCODED_CREDENTIALS=$(clusterawsadm bootstrap credentials encode-as-profile)
echo $AWS_B64ENCODED_CREDENTIALS
clusterctl init --infrastructure aws

export AWS_SSH_KEY_NAME=capa-quickstart

if aws ec2 describe-key-pairs --key-names $AWS_SSH_KEY_NAME 2>/dev/null; then
    echo "Key pair $AWS_SSH_KEY_NAME already exists."
else
  aws ec2 create-key-pair --key-name $AWS_SSH_KEY_NAME --query 'KeyMaterial' --output text > ${AWS_SSH_KEY_NAME}.pem
  chmod 400 ${AWS_SSH_KEY_NAME}.pem
  ls -l ${AWS_SSH_KEY_NAME}.pem
fi

export AWS_CONTROL_PLANE_MACHINE_TYPE=t3.large
export AWS_NODE_MACHINE_TYPE=t3.large

export KUBERNETES_VERSION=$(clusterawsadm ami list -o json | jq -r '.items[0].spec.kubernetesVersion')
echo $KUBERNETES_VERSION

clusterctl generate cluster capa-quickstart \
  --kubernetes-version $KUBERNETES_VERSION \
  --control-plane-machine-count=3 \
  --worker-machine-count=3 \
  > capa-quickstart.yaml

kubectl apply -f capa-quickstart.yaml

Here is a demonstration step by step:
https://youtu.be/i6d5J7XC3AM

What did you expect to happen:

Have a cluster control plane provisioned to proceed with next steps, kubeconfig retrieval and CNI deployment

Anything else you would like to add:

Should I expect v1.30.5 to work? Would other version be more recommendable?

Environment:

• Cluster-api-provider-aws version: built from main branch

clusterawsadm version: &version.Info{Major:"", Minor:"", GitVersion:"", GitCommit:"", GitTreeState:"", BuildDate:"", GoVersion:"go1.23.4", AwsSdkVersion:"v1.55.5", Compiler:"gc", Platform:"linux/amd64"}

• Kubernetes version: (use kubectl version):

Client Version: v1.31.0
Kustomize Version: v5.4.2
Server Version: v1.31.2

• OS (e.g. from /etc/os-release):

PRETTY_NAME="Debian GNU/Linux 12 (bookworm)"
NAME="Debian GNU/Linux"
VERSION_ID="12"
VERSION="12 (bookworm)"
VERSION_CODENAME=bookworm
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"

[dlipovetsky]

I wonder if this is caused by the default control plane endpoint being an ELB? That's a known issue #5139. I'll check. If that's indeed the issue, we'll need to update the quick start to use NLB instead.

[dlipovetsky]

/triage accepted

[dlipovetsky]

/priority important-soon

[yoctozepto]

I confirm this looks like a dup of #5139

@faermanj please confirm whether merging this with your AWSCluster definition helps (though you need to recreate it, i.e., delete the previous broken cluster before applying):

spec:
  controlPlaneLoadBalancer:
    loadBalancerType: nlb