Configure SASL_SSL authentication for Kafka scalers

[vimallearnz]

Discussed in #6484

^{Originally posted by vimallearnz January 13, 2025}
Hi,

I would like to know how to configure SASL_SSL authentication mechanism for KAFKA scaler trigger authentication. Based on the KEDA help articles, I found the below however I am getting the error mentioned under error section.

sasl - Kafka SASL auth mode. (Values: plaintext, scram_sha256, scram_sha512, gssapi, oauthbearer, or none, Default: none, Optional). This parameter could also be specified in sasl in TriggerAuthentication

tls - To enable SSL auth for Kafka, set this to enable. If not set, TLS for Kafka is not used. (Values: enable, disable, Default: disable, Optional). This parameter could also be specified in tls in TriggerAuthentication

unsafeSsl - Skip certificate validation when connecting over HTTPS. (Values: true, false, Default: false, Optional)

Attached the secret, triggerauthentication and scaled object script.
kafka-aeh-consumer-app-poc-new.yaml.zip

Please share your thoughts to configure SASL_SSL for kafka connectivity.

Errors:

2025-01-13T23:43:13Z ERROR scale_handler error resolving auth params {"type": "ScaledObject", "namespace": "", "name": "kafka-aeh-consumer-app-poc", "triggerIndex": 0, "error": "error creating kafka client: kafka: client has run out of available brokers to talk to: EOF"}
github.com/kedacore/keda/v2/pkg/scaling.(scaleHandler).buildScalers
/workspace/pkg/scaling/scalers_builder.go:99
github.com/kedacore/keda/v2/pkg/scaling.(scaleHandler).performGetScalersCache
/workspace/pkg/scaling/scale_handler.go:357
github.com/kedacore/keda/v2/pkg/scaling.(scaleHandler).GetScalersCache
/workspace/pkg/scaling/scale_handler.go:282
github.com/kedacore/keda/v2/controllers/keda.(ScaledObjectReconciler).getScaledObjectMetricSpecs
/workspace/controllers/keda/hpa.go:217
github.com/kedacore/keda/v2/controllers/keda.(ScaledObjectReconciler).newHPAForScaledObject
/workspace/controllers/keda/hpa.go:72
github.com/kedacore/keda/v2/controllers/keda.(ScaledObjectReconciler).createAndDeployNewHPA
/workspace/controllers/keda/hpa.go:45
github.com/kedacore/keda/v2/controllers/keda.(ScaledObjectReconciler).ensureHPAForScaledObjectExists
/workspace/controllers/keda/scaledobject_controller.go:444
github.com/kedacore/keda/v2/controllers/keda.(ScaledObjectReconciler).reconcileScaledObject
/workspace/controllers/keda/scaledobject_controller.go:283
github.com/kedacore/keda/v2/controllers/keda.(ScaledObjectReconciler).Reconcile
/workspace/controllers/keda/scaledobject_controller.go:194
sigs.k8s.io/controller-runtime/pkg/internal/controller.(Controller).Reconcile
/workspace/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:119
sigs.k8s.io/controller-runtime/pkg/internal/controller.(Controller).reconcileHandler
/workspace/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:316
sigs.k8s.io/controller-runtime/pkg/internal/controller.(Controller).processNextWorkItem
/workspace/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:266
sigs.k8s.io/controller-runtime/pkg/internal/controller.(Controller).Start.func2.2
/workspace/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:227
2025-01-13T23:43:13Z ERROR Error getting scalers {"controller": "scaledobject", "controllerGroup": "keda.sh", "controllerKind": "ScaledObject", "ScaledObject": {"name":"kafka-aeh-consumer-app-poc","namespace":""}, "namespace": "*****", "name": "kafka-aeh-consumer-app-poc", "reconcileID": "c3104498-4e6e-4122-a811-b9b568e4b18e", "error": "error creating kafka client: kafka: client has run out of available brokers to talk to: EOF"}
github.com/kedacore/keda/v2/controllers/keda.(*ScaledObjectReconciler).getScaledObjectMetricSpecs
/workspace/controllers/keda/hpa.go:219
github.com/kedacore/keda/v2/controllers/keda.(*ScaledObjectReconciler).newHPAForScaledObject
/workspace/controllers/keda/hpa.go:72
github.com/kedacore/keda/v2/controllers/keda.(*ScaledObjectReconciler).createAndDeployNewHPA
/workspace/controllers/keda/hpa.go:45
github.com/kedacore/keda/v2/controllers/keda.(*ScaledObjectReconciler).ensureHPAForScaledObjectExists
/workspace/controllers/keda/scaledobject_controller.go:444
github.com/kedacore/keda/v2/controllers/keda.(*ScaledObjectReconciler).reconcileScaledObject
/workspace/controllers/keda/scaledobject_controller.go:283
github.com/kedacore/keda/v2/controllers/keda.(*ScaledObjectReconciler).Reconcile
/workspace/controllers/keda/scaledobject_controller.go:194
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile
/workspace/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:119
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
/workspace/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:316
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
/workspace/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:266
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
/workspace/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:227

[vimallearnz]

After configuring cert and key parameters, the issue got fixed

apiVersion: v1
kind: Secret
metadata:
name: keda-aeh-kafka-secrets-with-cert-with-cert
namespace: ns1
data:
cert: LS0tZjhQbHdVSeUtLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tDQoNCj0=
key: LS0tLGbRSBLRVktLS0tLQ==
sasl: "cGxhaW50ZXh0"
username: "JENvbm5lY3Rpb25TdHJpbmc="
password: "RW5kcG9pbnQ9c2I6Ly9laHViLnNlcnZpY2VidXMud2luZG93cy5uZXQvO1NoYXJlZEFjY2Vzc0tleU5hbWU9cmVhZDtTaGFyZWRBY2Nlc3NLZXk9bkU5SllLM1IvKytBRWhPVUtiZjg9O0VudGl0eVBhdGg9cG9jLXBsYXRmb3JtLXRvcGljMg=="
tls: "ZW5hYmxl"

apiVersion: keda.sh/v1alpha1
kind: TriggerAuthentication
metadata:
name: keda-trigger-auth-aeh-kafka-credential-with-cert
namespace: 90b8b1e4-f80a-4703-a5ed-bf5c25df6a89
spec:
secretTargetRef:

• parameter: sasl
  name: keda-aeh-kafka-secrets-with-cert
  key: sasl
• parameter: username
  name: keda-aeh-kafka-secrets-with-cert
  key: username
• parameter: password
  name: keda-aeh-kafka-secrets-with-cert
  key: password
• parameter: tls
  name: keda-aeh-kafka-secrets-with-cert
  key: tls
• parameter: cert
  name: keda-aeh-kafka-secrets-with-cert
  key: cert
• parameter: key
  name: keda-aeh-kafka-secrets-with-cert
  key: key

---

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
name: kafka-aeh-consumer-app-poc
namespace: ns1
spec:
scaleTargetRef:
name: kafka-aeh-consumer-app-poc
pollingInterval: 1
#lagThreshold: "1"
cooldownPeriod: 2
minReplicaCount: 2
maxReplicaCount: 5
triggers:

• type: kafka
  metadata:
  bootstrapServers: "ehub.servicebus.windows.net:9093"
  consumerGroup: "platform-consumer-east" # Make sure that this consumer group name is the same one as the one that is consuming topics
  topic: "poc-platform-topic2"

  Optional

  lagThreshold: "1"
  #cooldownPeriod: "2"
  #minReplicaCount: "0"
  #maxReplicaCount: "50"
  offsetResetPolicy: latest
  authenticationRef:
  name: keda-trigger-auth-aeh-kafka-credential-with-cert