Migrate to Policy v1 Specification for Filter and Policy Engine

[abhisek]

This is a larger project. Multiple PRs should be raised against this ticket to get it done in small, incremental chunks. Do not raise one large monolithic PR for this.

As part of our larger revamping effort to standardise APIs, we have already integrated vet with Insights v2 in #274 However, to be able to really leverage the benefit of Insights v2, we need to revamp our policy evaluation infrastructure to support Policy v1 specification as defined in https://buf.build/safedep/api/docs/main:safedep.messages.policy.v1

To avoid confusion in this ticket,

• Filter v1 is vet's current filter specification
• Filter v2 is our new Policy v1 specification defined at https://buf.build/safedep/api/docs/main:safedep.messages.policy.v1

The migration involves:

1. Adopt new policy input based on the specification, feat: Support Package Meta Enricher using Insights V2 API #274 supports this data using Insights v2
2. Adopt spec based Policy and Rule entities
3. Support new policy evaluation without breaking existing filter based policy infrastructure

Currently, following features have dependency on old filter format:

• CEL Filter Analyser
• CEL Filter Suite Analyzer

Both of them internally use the filter package for CEL evaluation. While it is easy to have a new Filter v2 spec introduced and have new analyzers work on it, we made a wrong decision of exposing the Filter outside the package through AnalyzerEvent. Multiple reporters depend on the filter v1 for example

• JSON
• SARIF
• Graph
• Sync

The old filter spec is also used in exceptions package to generate exceptions.

For non-breaking migration, we need to support both filter v1 and v2 analyzer. Then we need to figure out a way to support v1 and v2 analyzer event in reporters because they are the consumer of filter evaluation result. There are a few options:

• Make the reporters complex i.e. check if v1 or v2 and act accordingly
• Refactor into a common Filter interface as per current reporter's need
• Migrate all reporters to use v2, this is unacceptable because it will be a breaking change

We first need to define a filter/v2 package that defines the contract for new spec based policy evaluation.

• Evaluate Package
• Evaluate Package Manifest
• [...]

We will keep this extensible and not be restricting to only evaluating a package model.

This requires:

1. Introduce a filter/v2 package that provides the necessary service for spec based policy evaluation
2. Refactor analysers to use Filter as a wrapper for both v1 and v2 objects, expose this instead of filter v1 directly
3. Refactor all reporters to use the Filter object instead of directly using the filter v1 object
4. Enhance Filter Suite to support both v1 and v2 input files, this will require doing a minimal YAML parsing to check if the version is explicitly set in which case use the new policy model. Internally use Filter v1 or v2 evaluator
5. Update test cases to check for compatibility between v1 and v2 usage scenarios i.e. vet should pass test cases using both v1 and v2 filters
6. Add deprecation notice in vet for filter v1 usage with docs URL for filter v2 migration