Skip to content

Commit

Permalink
refactor: kpt-kcl code
Browse files Browse the repository at this point in the history
Signed-off-by: peefy <[email protected]>
  • Loading branch information
Peefy committed May 8, 2024
1 parent 6f22037 commit da41563
Show file tree
Hide file tree
Showing 9 changed files with 168 additions and 82 deletions.
85 changes: 21 additions & 64 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,57 +83,27 @@ items:
selector:
app: MyApp
functionConfig:
apiVersion: v1
kind: ConfigMap
apiVersion: krm.kcl.dev/v1alpha1
kind: KCLRun
metadata:
name: set-replicas
annotations:
config.kubernetes.io/index: '0'
config.kubernetes.io/path: 'fn-config.yaml'
internal.config.kubernetes.io/index: '0'
internal.config.kubernetes.io/path: 'fn-config.yaml'
internal.config.kubernetes.io/seqindent: 'compact'
data:
replicas: "5"
source: |
resources = option("resource_list")
setReplicas = lambda items, replicas {
[item | {if item.kind == "Deployment": spec.replicas = replicas} for item in items]
}
setReplicas(resources.items or [], resources.functionConfig.data.replicas)
name: conditionally-add-annotations
spec:
params:
replicas: "5"
source: |
params = option("params")
replicas = params.replicas
setReplicas = lambda items, replicas {
[item | {if item.kind == "Deployment": spec.replicas = replicas} for item in items]
}
items = setReplicas(option("items"), replicas)
```
Thus, the `spec.replicas` of `Deployment` in the `resource_list.yaml` is changed to `5` from `2`.

## FunctionConfig

There are 2 kinds of `functionConfig` supported by this function:

+ ConfigMap
+ A custom resource of kind `KCLRun`

To use a ConfigMap as the functionConfig, the KCL script source must be specified in the data.source field. Additional parameters can be specified in the data field.

Here's an example:

```yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: set-replicas
data:
replicas: "5"
source: |
resources = option("resource_list")
setReplicas = lambda items, replicas {
[item | {if item.kind == "Deployment": spec.replicas = replicas} for item in items]
}
setReplicas(resources.items or [], resources.functionConfig.data.replicas)
```

In the example above, the script accesses the replicas parameters using `option("resource_list").functionConfig.data.replicas`.

To use a KCLRun as the functionConfig, the KCL source must be specified in the source field. Additional parameters can be specified in the params field. The params field supports any complex data structure as long as it can be represented in YAML.
The KCL source must be specified in the source field. Additional parameters can be specified in the params field. The params field supports any complex data structure as long as it can be represented in YAML.

```yaml
apiVersion: krm.kcl.dev/v1alpha1
Expand All @@ -148,20 +118,19 @@ spec:
configmanagement.gke.io/managed: disabled
source: |
resource = option("resource_list")
items = resource.items
params = resource.functionConfig.spec.params
toMatch = params.toMatch
toAdd = params.toAdd
[item | {
items = [item | {
# If all annotations are matched, patch more annotations
if all key, value in toMatch {
item.metadata.annotations[key] == value
}:
metadata.annotations: toAdd
} for item in items]
} for item in resource.items]
```

In the example above, the script accesses the `toMatch` parameters using `option("resource_list").functionConfig.spec.params.toMatch`.
In the example above, the script accesses the `toMatch` parameters using `option("params").toMatch`.

## Integrate the Function into kpt

Expand All @@ -175,26 +144,15 @@ export TAG=<Your KRM function tag>
docker build . -t ${FN_CONTAINER_REGISTRY}/${FUNCTION_NAME}:${TAG}
```

There are 2 ways to run the function declaratively.

+ Have your Kptfile with the inline ConfigMap as the functionConfig.
+ Have your Kptfile pointing to a functionConfig file that contains either a ConfigMap or a KCLRun.
Have your Kptfile pointing to a functionConfig file that contains either a KCLRun.

After that, you can render it in the folder that contains KRM with:

```bash
kpt fn render
```

There are 2 ways to run the function imperatively.

+ Run it using a ConfigMap that is generated from the command line arguments. The KCL script lives in `main.k` file.

```bash
kpt fn eval --image ${FN_CONTAINER_REGISTRY}/${FUNCTION_NAME}:${TAG} -- source="$(cat main.k)" param1=value1 param2=value2
```

+ Or use the function config file.
Or use the function config file.

```bash
kpt fn eval --image ${FN_CONTAINER_REGISTRY}/${FUNCTION_NAME}:${TAG} --fn-config fn-config.yaml
Expand All @@ -214,11 +172,10 @@ Then the Kubernetes resource file `resources.yaml` will be modified in place.

Here's what you can do in the KCL script:

+ Read resources from `option("resource_list")`. The `option("resource_list")` complies with the [KRM Functions Specification](https://github.com/kubernetes-sigs/kustomize/blob/master/cmd/config/docs/api-conventions/functions-spec.md#krm-functions-specification). You can read the input resources from `option("resource_list")["items"]` and the `functionConfig` from `option("resource_list")["functionConfig"]`.
+ Read resources from `option("resource_list")`. The `option("resource_list")` complies with the [KRM Functions Specification](https://github.com/kubernetes-sigs/kustomize/blob/master/cmd/config/docs/api-conventions/functions-spec.md#krm-functions-specification). You can read the input resources from `option("items")`.
+ Return a KPM list for output resources.
+ Return an error using `assert {condition}, {error_message}`.
+ Read the environment variables. e.g. `option("PATH")` (Not yet implemented).
+ Read the OpenAPI schema. e.g. `option("open_api")["definitions"]["io.k8s.api.apps.v1.Deployment"]` (Not yet implemented).
+ Read the environment variables. e.g. `option("PATH")`.

## Examples

Expand Down
25 changes: 14 additions & 11 deletions examples/resource_list/set_replicas.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -7,15 +7,18 @@ items:
replicas: '2'
- kind: Service
functionConfig:
apiVersion: v1
kind: ConfigMap
apiVersion: krm.kcl.dev/v1alpha1
kind: KCLRun
metadata:
name: set-replicas
data:
replicas: '5'
source: |
resources = option("resource_list")
setReplicas = lambda items, replicas {
[item | {if item.kind == "Deployment": spec.replicas = replicas} for item in items]
}
setReplicas(resources.items or [], resources.functionConfig.data.replicas)
name: conditionally-add-annotations
spec:
params:
replicas: "5"
source: |
params = option("params")
items = option("items")
replicas = params.replicas
setReplicas = lambda items, replicas {
[item | {if item.kind == "Deployment": spec.replicas = replicas} for item in items]
}
items = setReplicas(items, replicas)
4 changes: 2 additions & 2 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,8 @@ go 1.21

require (
github.com/GoogleContainerTools/kpt-functions-sdk/go/fn v0.0.0-20230427202446-3255accc518d
kcl-lang.io/krm-kcl v0.8.6
kcl-lang.io/krm-kcl v0.8.7-0.20240508062136-5e7a4f68cdcf
sigs.k8s.io/kustomize/kyaml v0.16.0
)

require (
Expand Down Expand Up @@ -175,7 +176,6 @@ require (
oras.land/oras-go v1.2.3 // indirect
oras.land/oras-go/v2 v2.3.0 // indirect
sigs.k8s.io/json v0.0.0-20221116044647-bc3834ca7abd // indirect
sigs.k8s.io/kustomize/kyaml v0.16.0 // indirect
sigs.k8s.io/structured-merge-diff/v4 v4.2.3 // indirect
sigs.k8s.io/yaml v1.4.0 // indirect
)
4 changes: 2 additions & 2 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -1718,8 +1718,8 @@ kcl-lang.io/kcl-openapi v0.6.1 h1:iPH0EvPgDGZS5Lk00/Su5Av6AQP5IBG8f7gAUyevkHE=
kcl-lang.io/kcl-openapi v0.6.1/go.mod h1:Ai9mFztCVKkRSFabczO/r5hCNdqaNtAc2ZIRxTeV0Mk=
kcl-lang.io/kpm v0.8.6 h1:uGaZjfkyG2ot9xVqdbE7ZsHcUfi6qYDa3UZUM77hhR8=
kcl-lang.io/kpm v0.8.6/go.mod h1:buvccvOf1JdN9WiPG1bOYHUnzapWNBLfURJf8W/VswU=
kcl-lang.io/krm-kcl v0.8.6 h1:XDthlR22e21ttFFxG8YhPh5ymB+aCOdledEpz7wgvoU=
kcl-lang.io/krm-kcl v0.8.6/go.mod h1:CAb+LBz4Zn670W+tKurWDpfErBWxPHCNSCeI2L9b++4=
kcl-lang.io/krm-kcl v0.8.7-0.20240508062136-5e7a4f68cdcf h1:ZvFQmQ1BI1WLJOMtsUbfgkZuuwFInV8/yMasVOpIfYA=
kcl-lang.io/krm-kcl v0.8.7-0.20240508062136-5e7a4f68cdcf/go.mod h1:8AGFPu6lYrvvusD2wgow0b41/gG4yms7wwUhX5jHPW4=
kcl-lang.io/lib v0.8.6 h1:y7+t19DZX/hJzgEsfAAN22gUHXVVJ1VOGJoWx8LGORo=
kcl-lang.io/lib v0.8.6/go.mod h1:ubsalGXxJaa5II/EsHmsI/tL2EluYHIcW+BwzQPt+uY=
oras.land/oras-go v1.2.3 h1:v8PJl+gEAntI1pJ/LCrDgsuk+1PKVavVEPsYIHFE5uY=
Expand Down
50 changes: 50 additions & 0 deletions pkg/process/process.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
package process

import (
"os"

"github.com/GoogleContainerTools/kpt-functions-sdk/go/fn"
"kcl-lang.io/krm-kcl/pkg/config"
"kcl-lang.io/krm-kcl/pkg/kube"
)

func init() {
// Set the fast eval mode for KCL
os.Setenv("KCL_FAST_EVAL", "1")
}

// Process is a function that takes a pointer to a ResourceList and processes
// it using the KCL function. It returns a boolean indicating whether the
// processing was successful, and an error (if any).
func Process(resourceList *fn.ResourceList) (bool, error) {
resourceListYAML, err := resourceList.ToYAML()
if err != nil {
return false, err
}
res, err := kube.ParseResourceList(resourceListYAML)
if err != nil {
return false, err
}
err = func() error {
r := &config.KCLRun{}
if err := r.Config(res.FunctionConfig); err != nil {
return err
}
return r.TransformResourceList(res)
}()
if err != nil {
resourceList.Results = []*fn.Result{
{
Message: err.Error(),
Severity: fn.Error,
},
}
return false, nil
}
processedItems, err := fn.ParseKubeObjects([]byte(res.Items.MustString()))
if err != nil {
return false, err
}
resourceList.Items = processedItems
return true, nil
}
76 changes: 76 additions & 0 deletions pkg/process/process_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
package process

import (
"testing"

"github.com/GoogleContainerTools/kpt-functions-sdk/go/fn"
"sigs.k8s.io/kustomize/kyaml/yaml"
)

func TestProcess(t *testing.T) {
p := fn.ResourceListProcessorFunc(Process)
input := `kind: ResourceList
items:
- apiVersion: apps/v1
kind: Deployment
spec:
replicas: '2'
- kind: Service
functionConfig:
apiVersion: krm.kcl.dev/v1alpha1
kind: KCLRun
metadata:
name: conditionally-add-annotations
spec:
params:
replicas: "5"
source: |
params = option("params")
replicas = params.replicas
setReplicas = lambda items, replicas {
[item | {if item.kind == "Deployment": spec.replicas = replicas} for item in items]
}
items = setReplicas(option("items"), replicas)
`
expected := `apiVersion: config.kubernetes.io/v1
kind: ResourceList
items:
- kind: Service
- apiVersion: apps/v1
kind: Deployment
spec:
replicas: '5'
functionConfig:
apiVersion: krm.kcl.dev/v1alpha1
kind: KCLRun
metadata:
name: conditionally-add-annotations
spec:
params:
replicas: "5"
source: |
params = option("params")
replicas = params.replicas
setReplicas = lambda items, replicas {
[item | {if item.kind == "Deployment": spec.replicas = replicas} for item in items]
}
items = setReplicas(option("items"), replicas)
`
output, err := fn.Run(p, []byte(input))
if err != nil {
t.Fatal(err)
}
expectedYaml, err := yaml.Parse(expected)
if err != nil {
t.Fatal(err)
}
outputYaml, err := yaml.Parse(string(output))
if err != nil {
t.Fatal(err)
}
expectedYamlString := expectedYaml.MustString()
outputYamlString := outputYaml.MustString()
if expectedYamlString != outputYamlString {
t.Errorf("test failed, expected %s got %s", expectedYamlString, outputYamlString)
}
}
2 changes: 1 addition & 1 deletion pkg/runner/runner.go
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ package runner

import (
"github.com/GoogleContainerTools/kpt-functions-sdk/go/fn"
"kcl-lang.io/krm-kcl/pkg/process"
"kcl-lang.io/kpt-kcl/pkg/process"
)

// Run evaluates the ResourceList from STDIN to STDOUT
Expand Down
2 changes: 1 addition & 1 deletion testdata/fn-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,6 @@ metadata:
name: set-replicas
spec:
params:
replicas: 6
replicas: 5
source: |
[item | {if item.kind == "Deployment": spec.replicas = option("params").replicas} for item in option("items") or []]
2 changes: 1 addition & 1 deletion testdata/resources.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ metadata:
labels:
app: nginx
spec:
replicas: 6
replicas: 2
selector:
matchLabels:
app: nginx
Expand Down

0 comments on commit da41563

Please sign in to comment.