BackendTLSPolicy's Service attachment is problematic

[howardjohn]

These comments were made during the initial design, but I wanted to persist them into an issue.

BackendTLSPolicy currently attaches to a Service.

In the simple case, this works fine. For example, I may have:

Gateway:
  port: 443
  protocol: HTTPS
  tls: TERMINATE
HTTPRoute:
  backendRef: foo, port 443
BackendTLSPolicy:
  targetRef: Service foo

and I will terminate TLS and encrypt it on the upstream. This works great.

---

Where things fall apart is when we have multiple paths.

For example, imagine I have the above config, but then also have a service mesh. An application in the service mesh calls curl https://foo.

The foo service has the TLS config saying "Add TLS to this request". The service mesh adds TLS and now it has 2 layers of TLS which is clearly broken.

If the API semantic is supposed to be "make sure this request is TLS" not "Add TLS to the this request" (which I don't think it is, nor do I think it should be, just covering the bases here) that is infeasible to implement since it would require any implementation to know whether a request is already TLS encrypted or not.

This same issue can occur without mesh as well. For example, if in addition to my configs above I have:

Gateway:
  port: 443
  protocol: HTTPS
  tls: PASSTHROUGH
TLSRoute:
  backendRef: foo, port 443

then I have the same problem of TLS being added again. Again, we could say the impl should detect this is already TLS and it should not be added. However, this falls apart in other cases:

Gateway:
  port: 443
  protocol: TCP
TCPRoute:
  backendRef: foo, port 443

As it this is opaque TCP, we are not aware if it is TLS or not.

---

Overall, these issues prevent BackendTLSPolicy from being a viable replacement for DestinationRule for Istio

[youngnick]

Thanks for writing this down, @howardjohn, it does make sense.

Do you have any suggestions here? How is DestinationRule different that it doesn't have these problems? Is it just that it is effectively at the equivalent of the HTTPRoute level?

[howardjohn]

DestinationRule actually sort of has the same problem but has an escape hatch.

You can provide the TLS config in the top level to make it apply to the service generally, which has the possibility of the same issue, but you can also scope it to a subset and then route directly to that:

apiVersion: networking.istio.io/v1
kind: DestinationRule
metadata:
  name: originate-tls
spec:
  host: foo
  subsets:
  - name: originate-tls
    tls:
      mode: SIMPLE
      sni: foo.com
---
apiVersion: networking.istio.io/v1
kind: VirtualService
metadata:
  name: originate-tls
spec:
  hosts:
  - foo.com
  http:
  - route:
    - destination:
        host: foo
        subset: originate-tls

Note: "subset" is usually used to filter a service into a smaller set; here its used in a less common way where the set of endpoints is all of them, and we just customizing the configuration.

One way you could actually do the same today is to make two services (foo, and foo-originate-tls). Then you can attached the BackendTLSPolicy only to foo-originate-tls and route to foo-originate-tls from the Gateway's we want to originate TLS; others can call foo. A bit tedious for the user, though.

Another way would be to allow the BackendTLSPolicy to be scoped to a specific route in some way, rather than the Service directly. Like maybe:

backendRefs:
  filters: # "filter" name is awkward here...
  - extensionRef:
      kind: BackendTlsPolicy
      name: foo-policy
  name: foo

but then we have the same policy be used as an extensionRef and targetRef which is probably not good.

You could targetRef the HTTPRoute (and further scope with sectionName?), but its still not quite the same since you cannot say "attach it to this backend in this rule in this route"

[mikemorris]

You could targetRef the HTTPRoute (and further scope with sectionName?), but its still not quite the same since you cannot say "attach it to this backend in this rule in this route"

This is pretty close to what I was trying to achieve with a from clause in #3573 (comment) - adding a secondary targeting mechanism to a policy, so it could still target a Service, but allow scoping an effective policy down to only apply to requests from a particular GatewayClass (or Mesh?), Gateway, Listener, HTTPRoute or even a named HTTPRoute rule.