Extend RateLimitFilter to support multiple ordered RateLimit rules

[nezdolik]

Description:
In certain cases users may need to specify multiple ratelimit rules that should be applied in certain order. E.g. a global RL rule per ip and subsequent rule per ip per upstream/backend service. The Api could look like:

#first RL rule
apiVersion: gateway.envoyproxy.io/v1alpha1
kind: RateLimitFilter
metadata:
  name: global-ip-rule
spec:
  type: Global
  priority: 0
  global:
    rules:
    - clientSelectors:
      - headers:
        - name: x-real-ip
      limit:
        requests: 3
        unit: Hour

#second RL rule
apiVersion: gateway.envoyproxy.io/v1alpha1
kind: RateLimitFilter
metadata:
  name: per-service-per-ip-rule
spec:
  type: Global
  priority: 1
  global:
    rules:
    - clientSelectors:
      - headers:
        - name: x-real-ip
        - name: some-header-that-denotes-backend-name # destination cluster is calculated at router?
      limit:
        requests: 2
        unit: Hour

The config would translate into multiple Envoy filters ordered according to priority. Note, in my personal opinion is easier to operate with multiple envoy filters/multiple RL domains than try to order descriptors within single RL domain. This is due how envoy ratelimit service descriptor matcing algo works, e.g. match the longest descriptor first (which will not work for the example above). So Envoy filter chain config would look like:

- name: envoy.filters.http.ratelimit
  typed_config:
    "@type": type.googleapis.com/envoy.extensions.filters.http.ratelimit.v3.RateLimit
    domain: global-ip-rule
    ...
- name: envoy.filters.http.ratelimit
  typed_config:
    "@type": type.googleapis.com/envoy.extensions.filters.http.ratelimit.v3.RateLimit
    domain: per-service-per-ip-rule
    ...

And RL service descriptors will be populated accordingly.
Looking forward to feedback, if this proposal makes sense.

[optional Relevant Links:]

Any extra documentation required to understand the issue.

[nezdolik]

cc @envoyproxy/gateway-maintainers

[arkodg]

@nezdolik this API RateLimitFilter has been removed (although chatgpt still remembers it :) ), and RateLimit exists in BackendTrafficPolicy https://gateway.envoyproxy.io/docs/tasks/traffic/global-rate-limit/

[rsaelim]

@arkodg Does the premise of this issue still stand with BackendTrafficPolicy? What if I want all my routes exposed via the Gateway to have a high general rate limit rule, but for a specific HTTPRoute, I apply a more specific BackendTrafficPolicy? Does one take precedent over the other or is it non-deterministic?

I'm using v1.2.5 of EG, and I remember in older version of the doc (v0.6), there's a specific BackendTrafficPolicy design page that spelled out this collision: "xRoute resource will win in a conflict", just want to make sure this is still the behavior.

[arkodg]

What if I want all my routes exposed via the Gateway to have a high general rate limit rule, but for a specific HTTPRoute, I apply a more specific BackendTrafficPolicy? Does one take precedent over the other or is it non-deterministic?

For this case, you'd need to define 2 BackendTrafficPolicies - one that targets the top level Gateway or all the other routes and a second that has a more specific rule for a specific route

[ryanhristovski]

@arkodg would it make sense to define the top level gateway ratelimiting within the ClientTrafficPolicy rather than a BackendTrafficPolicy (and keep the route level limiting in the btp)?

[arkodg]

@ryanhristovski I think this intent

In certain cases users may need to specify multiple ratelimit rules that should be applied in certain order. E.g. a global RL rule per ip and subsequent rule per ip per upstream/backend service

can be achieved by

1. Defining the RL Rule in a BTP and applying it at the Gateway level.
2. Above RL Rule is special, and needs a new field like shared: true or scope: gateway so its treated as a common bucket when applying across all routes
3. Using Merge Semantics to merge 2 BTPs - applied to the Gateway and xRoute Add the ability to override only a portion of the spec using Policies #1934

Im not sure if priority is needed here, when merging we can ensure gateway rules are set before route rules

[ryanhristovski]

@arkodg wdyt the functionality should be if an HTTPRoute level btp had shared: true or scope: gateway (I am leaning shared as the field).

For example:

apiVersion: gateway.envoyproxy.io/v1alpha1
kind: BackendTrafficPolicy 
metadata:
  name: per-service-per-ip-rule
spec:
  targetRefs:
  - group: gateway.networking.k8s.io
    kind: HTTPRoute
    name: http-ratelimit
  rateLimit:
    type: Global
    shared: true
    global:
      rules:
      - clientSelectors:
        - headers:
          - name: x-real-ip
        limit:
          requests: 3
          unit: Hour

Should I add some logic that only allows shared on Gateway & shared=false to be default?

[arkodg]

hey @ryanhristovski @nezdolik can we close this issue and create a new one or update title and description to better define shared ratelimit bucket feature

[ryanhristovski]

@arkodg yup, @nezdolik is ooo now so I'll create a new one if you can close this one once i'm done

[ryanhristovski]

@arkodg #5265