From 624cae0ced47026e2b4a529d11fc6a6570854fc0 Mon Sep 17 00:00:00 2001 From: Ramon Perez Date: Wed, 12 Feb 2025 16:17:51 +0100 Subject: [PATCH 1/2] Document all individual projects --- README.md | 124 +-------- cnf-app-mac-operator/README.md | 75 ++++-- documentation/testing.md | 6 +- nfv-example-cnf-index/README.md | 2 +- testpmd-container-app/README.md | 5 + testpmd-container-app/cnfapp/README.md | 70 ++++- testpmd-operator/DEV.md | 60 ----- testpmd-operator/README.md | 48 ++-- testpmd-operator/localtest.yaml | 24 -- testpmd-operator/play.yaml | 18 -- testpmd-operator/testpmd-allinone.yaml | 231 ---------------- trex-container-app/README.md | 6 + trex-container-app/app/README.md | 19 ++ trex-container-app/app/scripts/README.md | 11 - trex-container-app/server/README.md | 63 +++++ ...ci-fix-VF-bus-error-for-memory-access.diff | 55 ---- trex-container-app/tests/annotations | 3 - trex-operator/README.md | 46 ++-- trex-operator/localtest.yaml | 35 --- trex-operator/trex-allinone.yaml | 252 ------------------ utils/README.md | 20 ++ 21 files changed, 304 insertions(+), 869 deletions(-) create mode 100644 testpmd-container-app/README.md delete mode 100644 testpmd-operator/DEV.md delete mode 100644 testpmd-operator/localtest.yaml delete mode 100644 testpmd-operator/play.yaml delete mode 100644 testpmd-operator/testpmd-allinone.yaml create mode 100644 trex-container-app/README.md create mode 100644 trex-container-app/app/README.md delete mode 100644 trex-container-app/app/scripts/README.md create mode 100644 trex-container-app/server/README.md delete mode 100644 trex-container-app/server/v3-bus-pci-fix-VF-bus-error-for-memory-access.diff delete mode 100644 trex-container-app/tests/annotations delete mode 100644 trex-operator/localtest.yaml delete mode 100644 trex-operator/trex-allinone.yaml create mode 100644 utils/README.md diff --git a/README.md b/README.md index d85bb03e..d0a1ba6f 100644 --- a/README.md +++ b/README.md @@ -6,36 +6,13 @@ Example CNF is an OpenShift workload to exercice an SRIOV setup, based on [TestP ![Flow](documentation/network_setup.png) -It is providing the following operators: +It provides the following operators: -* trex-operator - * It provides TRex Traffic Generator, decomposed in three components: - * TRexConfig CR, pinning the config of the TRex server instance. Related pod is `trexconfig-` pod. - * TRexApp CR, setting up the job that launches TRex execution to generate traffic towards the system under test. Related pod is `trex-app` pod. - * TRex Operator, ensuring CR reconciliation via controller-manager pod. Related pod is `trex-operator-controller-manager-` pod. - * Following information can be extracted from pod logs: - * In `trexconfig-` pod logs, you can see the trex statistics printed periodically. - * The summary of the test execution can be seen at the end of the `trex-app` job logs. - * In `trex-operator-controller-manager-` pod, you can see the execution of the Ansible playbooks that ensures the reconciliation loop of the operator. +- [trex-operator](trex-operator) +- [testpmd-operator](testpmd-operator) +- [cnf-app-mac-operator](cnf-app-mac-operator) -![Operator behavior](documentation/trex-operator.png) - -* testpmd-operator - * Final application, also known as CNF Application, which is a standard TestPMD instance using the default MAC forwarding module. It uses two components: - * TestPMD CR, which creates a pod to implement the MAC forwarding module as final application. Related pod is `testpmd-app-` pod (only one replica is used). - * TestPMD Operator, ensuring CR reconciliation via controller-manager pod. Related pod is `testpmd-operator-controller-manager-` pod. - * Following information can be extracted from pod logs: - * To see the TestPMD statistics printed periodically for this module, you can rely on `testpmd-app-` pod logs. - * In `testpmd-operator-controller-manager-` pod, you can see the execution of the Ansible playbooks that ensures the reconciliation loop of the operator. - -![Operator behavior](documentation/testpmd-operator.png) - -* cnf-app-mac-operator - * Auxiliary operator just composed by one component, which is CNFAppMac Operator, a Golang-based operator in charge of ensuring reconciliation for CNFAppMac CR, which is a wrapper created for the `testpmd-app-` pod and linked to it, and that is used to extract the network information of the pods (network, MAC and PCI addresses), to be offered to other components of the solution. - -![Operator behavior](documentation/cnf-app-mac-operator.png) - -You can use them from the [Example CNF Catalog](https://quay.io/repository/rh-nfv-int/nfv-example-cnf-catalog?tab=tags). Image generation is automated with [Github workflows](.github/workflows) in this repo. +You can use them from the [Example CNF Catalog](https://quay.io/repository/rh-nfv-int/nfv-example-cnf-catalog?tab=tags). Image generation is automated with [Github workflows](.github/workflows) in this repo, using this [Makefile](Makefile). ## Pre-requirements @@ -53,78 +30,8 @@ The three operators defined in this repository are built with [Operator SDK tool We can differentiate between these two cases: -### Ansible-based operators - -This is the case of testpmd-operator and trex-operator. - -Base structure for each case is achieved with the following commands, then it's just a matter of accommodating the required code for each operator in the corresponding files and folders: - -- testpmd-operator - -``` -$ mkdir testpmd-operator; cd testpmd-operator -$ operator-sdk init --domain openshift.io --plugins ansible -$ operator-sdk create api --version v1 --generate-role --group examplecnf --kind TestPMD -``` - -- trex-operator - -``` -$ mkdir trex-operator; cd trex-operator -$ operator-sdk init --domain openshift.io --plugins ansible -$ operator-sdk create api --version v1 --generate-role --group examplecnf --kind TRexApp -$ operator-sdk create api --version v1 --generate-role --group examplecnf --kind TRexConfig -``` - -### Go-based operators - -This is the case of cnf-app-mac-operator. - -Base structure for this case is achieved with the following commands, then it's just a matter of accommodating the required code for the operator in the corresponding files and folders: - -- cnf-app-mac-operator - -For operator-sdk v1.38.0, you need to have installed the same Go version used in operator-sdk, which is at least Go 1.22.5+. - -``` -$ operator-sdk version -operator-sdk version: "v1.38.0", commit: "0735b20c84e5c33cda4ed87daa3c21dcdc01bb79", kubernetes version: "1.30.0", go version: "go1.22.5", GOOS: "linux", GOARCH: "amd64" -``` - -Create the project structure and the CNFAppMac API: - -``` -$ mkdir cnf-app-mac-operator; cd cnf-app-mac-operator -$ operator-sdk init --domain openshift.io --repo github.com/openshift-kni/example-cnf/tree/main/cnf-app-mac-operator -$ operator-sdk create api --version v1 --group examplecnf --kind CNFAppMac --controller --resource -``` - -At this point, remove RBAC resource creation in Makefile > manifests task. Then, review cmd/main.go and api/v1/cnfappmac_types.go, then run: - -``` -$ make generate -$ make manifests -``` - -Create webhook and certmanager: - -``` -$ operator-sdk create webhook --version v1 --group examplecnf --kind CNFAppMac --defaulting --programmatic-validation -``` - -Review the generated files properly, then: - -``` -$ make manifests -``` - -Comment webhook references in PROJECT and cmd/main.go files (older versions were not using this), review internal/controller/cnfappmac_controller.go and review the rest of files. - -To conclude, build the main.go file to check it's working fine: - -``` -$ go build cmd/main.go -``` +- Ansible-based operators: this is the case of [testpmd-operator](testpmd-operator/README.md#how-to-build-the-operator) and [trex-operator](trex-operator/README.md#how-to-build-the-operator). +- Go-based operators: this is the case of [cnf-app-mac-operator](cnf-app-mac-operator/README.md#how-to-build-the-operator). ## Pod affinity rules @@ -273,23 +180,6 @@ $ ip -s -d link show dev ens2f0 Under [utils](utils) folder, you can find some utilities included in Example CNF to extend the functionalities offered by the tool. -- [webserver.go](utils/webserver.go): a Golang-based webserver to implement liveness, readiness and startup probes in the container images offered in [testpmd-container-app](testpmd-container-app) and [trex-container-app](trex-container-app) folders. The Makefiles offered in these directories take care of copying the webserver code from the utils directory to each image's directory. -- [required-annotations.yaml](utils/required-annotations.yaml): annotations to be appended to the CSVs to pass Preflight's RequiredAnnotations tests. They are appended automatically thanks to the Makefile tasks from each operator. -- [support-images](support_images): projects where you can find the Dockerfile required to build some of the images used as build images by the Example CNF images. These images can be found on quay.io/rh-nfv-int and they are publicly available, you only need credentials to access quay.io. The images can be built with the following commands (you need to run it in a RHEL host with a valid RHEL subscription to be able to download the packages installed in the images, and you need a valid quay.io credentials to push it to quay.io): - -``` -# build images -$ cd utils/support-images -$ podman build dpdk-23.11 -f dpdk-23.11/Dockerfile -t "quay.io/rh-nfv-int/dpdk-23.11:v0.0.1" -$ podman build ubi8-base-testpmd -f ubi8-base-testpmd/Dockerfile -t "quay.io/rh-nfv-int/ubi8-base-testpmd:v0.0.1" -$ podman build ubi8-base-trex -f ubi8-base-trex/Dockerfile -t "quay.io/rh-nfv-int/ubi8-base-trex:v0.0.1" - -# push images (to quay.io/rh-nfv-int) -$ podman push quay.io/rh-nfv-int/dpdk-23.11:v0.0.1 -$ podman push quay.io/rh-nfv-int/ubi8-base-testpmd:v0.0.1 -$ podman push quay.io/rh-nfv-int/ubi8-base-trex:v0.0.1 -``` - ## Acknowledgements Please write to telcoci@redhat.com in case you need more information for using and testing Example CNF in the scenarios that have been proposed in this repository. diff --git a/cnf-app-mac-operator/README.md b/cnf-app-mac-operator/README.md index d0fd22e0..a759540f 100644 --- a/cnf-app-mac-operator/README.md +++ b/cnf-app-mac-operator/README.md @@ -1,26 +1,63 @@ -CNF App MAC FETCH -================= +# cnf-app-mac-operator -An operator helps in creating `CNFAppMac` CR for all the pods in a namespace -which has VFs attached to it. +![Operator behavior](../documentation/cnf-app-mac-operator.png) -TODO: ------------------------- -* Create CR only for a custom set of pods (based on label or name) +Auxiliary operator just composed by one component, which is CNFAppMac Operator, a Golang-based operator in charge of ensuring reconciliation for CNFAppMac CR, which is a wrapper created for the `testpmd-app-` pod and linked to it, and that is used to extract the network information of the pods (network, MAC and PCI addresses), to be offered to other components of the solution. -License ------------------------- +## How to build the operator -Copyright 2024. +Base structure for this case is achieved with the following commands, then it's just a matter of accommodating the required code for the operator in the corresponding files and folders: -Licensed under the Apache License, Version 2.0 (the "License"); -you may not use this file except in compliance with the License. -You may obtain a copy of the License at +For operator-sdk v1.38.0, you need to have installed the same Go version used in operator-sdk, which is at least Go 1.22.5+. - http://www.apache.org/licenses/LICENSE-2.0 +``` +$ operator-sdk version +operator-sdk version: "v1.38.0", commit: "0735b20c84e5c33cda4ed87daa3c21dcdc01bb79", kubernetes version: "1.30.0", go version: "go1.22.5", GOOS: "linux", GOARCH: "amd64" +``` -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. +Create the project structure and the CNFAppMac API: + +``` +$ mkdir cnf-app-mac-operator; cd cnf-app-mac-operator +$ operator-sdk init --domain openshift.io --repo github.com/openshift-kni/example-cnf/tree/main/cnf-app-mac-operator +$ operator-sdk create api --version v1 --group examplecnf --kind CNFAppMac --controller --resource +``` + +At this point, remove RBAC resource creation in Makefile > manifests task. Then, review cmd/main.go and api/v1/cnfappmac_types.go, then run: + +``` +$ make generate +$ make manifests +``` + +Create webhook and certmanager: + +``` +$ operator-sdk create webhook --version v1 --group examplecnf --kind CNFAppMac --defaulting --programmatic-validation +``` + +Review the generated files properly, then: + +``` +$ make manifests +``` + +Comment webhook references in PROJECT and cmd/main.go files (older versions were not using this), review internal/controller/cnfappmac_controller.go and review the rest of files. + +To conclude, build the main.go file to check it's working fine: + +``` +$ go build cmd/main.go +``` + +## What to update if bumping operator version + +Apart from the modifications you have to do, you also need to update the operator version in these files: + +- [CHANGELOG.md](CHANGELOG.md). +- [Makefile](Makefile). +- [Dockerfile](Dockerfile). + +Also, make sure that the operator version is within the interval defined in [required-annotations.yaml](../utils/required-annotations.yaml) file for `olm.skipRange` annotation, else update that file to modify the current range. + +A classical change is the update of Operator SDK version used in the operator. Here's an [example](https://github.com/openshift-kni/example-cnf/pull/108) where this is done. diff --git a/documentation/testing.md b/documentation/testing.md index eccb146c..fc6de2b7 100644 --- a/documentation/testing.md +++ b/documentation/testing.md @@ -174,9 +174,9 @@ We can see these pods: The three data-plane pods (`testpmd-app`, `trexconfig` and `job-trex-app`) execute different scripts to configure all the software pieces that involves the traffic exchange between TestPMD and TRex: -- `testpmd-app`: it runs `/usr/local/bin/example-cnf/testpmd-wrapper`. [This script](../testpmd-container-app/cnfapp/scripts/testpmd-wrapper) retrieves context information to build the command to run `testpmd` command-line tool. In this baseline scenario, once finished, `testpmd` is invoked in auto-start mode, printing the statistics every minute. -- `trexconfig`: it runs `/usr/local/bin/trex-wrapper`. [This script](../trex-container-app/server/scripts/trex-wrapper) takes also some context information to build a config file for TRex, then it launches `_t-rex-64` binary using the generated config file, and printing statistics regularly. This does not launch the traffic generation, since this is done with the TRexApp job. -- `job-trex-app`: it runs `/usr/local/bin/trex-wrapper` (but a different one compared to `trexconfig`). [This script](../trex-container-app/app/scripts/trex-wrapper) triggers a custom Python script that builds the traffic profile, according to the TRex variables provided in the pipeline (duration of the job, packet size, data rate, etc.), and start sending the traffic. If defining a duration, a timeout will be enabled to stop the execution after the given duration, and then statistics will be printed and packet loss will be calculated. If packet loss is equal or less than 0, this means there's no packet loss, else, packet loss is present and the pod status will be different than Completed. +- `testpmd-app`: check [cnfapp container image docs](../testpmd-container-app/cnfapp/README.md) for more details. In this baseline scenario, once finished, `testpmd` is invoked in auto-start mode, printing the statistics every minute. +- `trexconfig`: check [TRex server container image docs](../trex-container-app/server/README.md) for more details. +- `job-trex-app`: check [TRex app container image docs](../trex-container-app/app/README.md) for more details. ### What logs should I check? diff --git a/nfv-example-cnf-index/README.md b/nfv-example-cnf-index/README.md index f7d685ab..db1db2fb 100644 --- a/nfv-example-cnf-index/README.md +++ b/nfv-example-cnf-index/README.md @@ -1,5 +1,5 @@ # nfv-example-cnf-index -This repository builds an index image that includes the [Example cnf operators](https://github.com/openshift-kni/example-cnf) used by the [example_cnf_deploy role](https://github.com/redhatci/ansible-collection-redhatci-ocp/blob/main/roles/example_cnf_deploy/README.md). +This repository builds an index image that includes the [Example CNF operators](../README.md) used by the [example_cnf_deploy role](https://github.com/redhatci/ansible-collection-redhatci-ocp/blob/main/roles/example_cnf_deploy/README.md). The [CHANGELOG](./CHANGELOG.md) includes the information about the version of the image as well as the operators and their versions included in the image. diff --git a/testpmd-container-app/README.md b/testpmd-container-app/README.md new file mode 100644 index 00000000..4e1240a2 --- /dev/null +++ b/testpmd-container-app/README.md @@ -0,0 +1,5 @@ +# testpmd-container-app + +Container images used by [testpmd-operator](../testpmd-operator/). We have the following projects: + +- [cnfapp](cnfapp). diff --git a/testpmd-container-app/cnfapp/README.md b/testpmd-container-app/cnfapp/README.md index f531c44e..dc1dad0d 100644 --- a/testpmd-container-app/cnfapp/README.md +++ b/testpmd-container-app/cnfapp/README.md @@ -1,6 +1,68 @@ -Pushing images to Red Hat registry ----------------------------------- +# cnfapp -Follow [partner guide](https://redhat-connect.gitbook.io/partner-guide-for-red-hat-openshift-and-container/certify-your-application/image-upload) to upload the image for certification. The image will be available in the respective partner (fredco) project at https://connect.redhat.com/project. +Container image that allows you to configure TestPMD in a container and prepare it to launch it. -Follow the push images manually steps. +## What is it executed? + +This container runs `/usr/local/bin/example-cnf/testpmd-wrapper`. [This script](scripts/testpmd-wrapper) retrieves context information to build the command to run `testpmd` command-line tool. Then, depending on the selected mode to launch Example CNF, `testpmd` will be executed in auto-start mode (baseline scenario), or, if launching the troubleshooting scenario, you can select between that mode and the interactive mode. + +The scripts (generated by `testpmd-wrapper`) to deploy each mode are placed in the same directory: + +``` +sh-4.4$ pwd +/usr/local/bin/example-cnf/run +sh-4.4$ ls +testpmd-interactive testpmd-run +``` + +They present some differences: + +- `testpmd-run`: runs `testpmd` in auto-start mode. + +``` +sh-4.4$ cat testpmd-run +/usr/local/bin/example-cnf/testpmd -l 3,5,53 --in-memory -a 0000:86:02.2 -a 0000:86:03.0 --socket-mem 0,1024 -n 6 --proc-type auto --file-prefix pg -- --nb-cores=2 --rxq=1 --txq=1 --rxd=2048 --txd=2048 --auto-start --eth-peer 0,20:04:0f:f1:89:01 --eth-peer 1,20:04:0f:f1:89:02 --forward-mode=mac --stats-period 1 2>&1 | tee /var/log/testpmd/app.log +``` + +The call to `testpmd` receives, as arguments (remember we have to differentiate between [EAL parameters](https://doc.dpdk.org/guides/linux_gsg/linux_eal_parameters.html) and [TestPMD parameters](https://doc.dpdk.org/guides/testpmd_app_ug/run_app.html), they're separated by `--`), the following parameters: + +- CPU cores to bind to `testpmd`. This matches with the number of cores assigned for the pod that is launching this container. If using reduced mode, only three cores are used: first CPU for the console, and second core CPU (two siblings) for `testpmd` execution. This modification is managed by `testpmd-wrapper`. +- Option to not create any shared data structures and run entirely in memory. +- PCI addresses. +- Preallocation of specific amounts of memory per socket. Depending on the NUMA node that is used, `testpmd-wrapper` will tune this value. +- Number of memory channels to use. +- Type of current process as `auto`. +- Shared data file prefix for DPDK process named as `pg` +- Number of forwarding cores. +- Number of TX/RX queues. +- Number of descriptors in the TX/RX rings. These values are doubled if using reduced mode, everything managed by `testpmd-wrapper`. +- Enable auto-start mode. +- Provide the pair of port-MAC address for each port. +- Enable MAC forwarding mode. +- Print the statistics every minute. + +Finally, redirect the logs to `/var/log/testpmd/app.log` + +- `testpmd-interactive`: runs `testpmd` in interactive mode. + +``` +sh-4.4$ cat testpmd-interactive +/usr/local/bin/example-cnf/testpmd -l 3,5,53 --in-memory -a 0000:86:02.2 -a 0000:86:03.0 --socket-mem 0,1024 -n 6 --proc-type auto --file-prefix pg -- --nb-cores=2 --rxq=1 --txq=1 --rxd=2048 --txd=2048 --i --eth-peer 0,20:04:0f:f1:89:01 --eth-peer 1,20:04:0f:f1:89:02 --forward-mode=mac +``` + +Parameters are mostly similar than in the previous case, but with the following differences: + +- Use interactive mode (`--i`) instead of auto-start mode (`--auto-start`). +- Do not print statistics, since they can be retrieved manually. + +You have more details about testing this feature in the [testing docs](../../documentation/testing.md). + +## What to update if bumping container version + +Apart from the modifications you have to do, you also need to update the container version in these files: + +- [Dockerfile](Dockerfile). +- [build.sh](../build.sh) (from parent folder). +- [Makefile](../Makefile) (from parent folder). + +Here's an [example](https://github.com/openshift-kni/example-cnf/pull/111) where this is done. diff --git a/testpmd-operator/DEV.md b/testpmd-operator/DEV.md deleted file mode 100644 index c16901e7..00000000 --- a/testpmd-operator/DEV.md +++ /dev/null @@ -1,60 +0,0 @@ -testpmd-operator -================ - -Ansible based operator to deploy TestPMD application in OpenShift. - -Container images for the TestPMD application is built using [testpmd-container-app](https://github.com/openshift-kni/example-cnf/testpmd-container-app). - -Preparation ----------- -* Deploy a OpenShift cluster with Baremetal worker nodes - -* Baremetal Node Configuration (Kernel args and IOMMU) -``` -# TODO - -``` - -* Configure static CPU manager for Kubelet configuration -``` -NODENAME=worker-0 -oc label node $NODENAME cpumanager=true -oc label machineconfigpool worker custom-kubelet=cpumanager-enabled -git clone https://github.com/krsacme/ocp-templates-nfv.git -cd ocp-templates-nfv -oc -n openshift-machine-api apply -f cpumanager-kubeletconfig.yaml -``` - -* Deploy ``sriov-network-operator`` (simplest way is to deploy via source) -``` -go get github.com/openshift/sriov-network-operator -export KUBECONFIG=/home/kni/dev-scripts/ocp/vnf/auth/kubeconfig -make deploy-setup - -# Disable webhook (to disable PF whitelisting) -oc -n openshift-sriov-network-operator patch sriovoperatorconfig default --type=merge -p '{"spec":{"enableOperatorWebhook":false}}' -``` -* Create SR-IOV networks & policy, modify the ``policy1.yaml`` file as per the cluster interface names -``` -git clone https://github.com/krsacme/ocp-templates-nfv.git -cd ocp-templates-nfv - -oc apply -f sriov/policy1.yaml -oc apply -f sriov/policy2.yaml -oc apply -f sriov/network1.yaml -oc apply -f sriov/network2.yaml -``` - -TestPMD operator deployment ---------------------------- - -``` -oc -n example-cnf apply -f deploy/crds/examplecnf.openshift.io_testpmds_crd.yaml -oc -n example-cnf apply -f deploy/service_account.yaml -oc -n example-cnf apply -f deploy/role.yaml -oc -n example-cnf apply -f deploy/role_binding.yaml -oc -n example-cnf apply -f deploy/operator.yaml -oc -n example-cnf apply -f deploy/scc.yaml - -oc -n example-cnf apply -f deploy/crds/examplecnf.openshift.io_v1_testpmd_cr.yaml -``` diff --git a/testpmd-operator/README.md b/testpmd-operator/README.md index 4bcfd2f6..a56aa970 100644 --- a/testpmd-operator/README.md +++ b/testpmd-operator/README.md @@ -1,27 +1,35 @@ -testpmd-operator -================ +# testpmd-operator -Ansible based operator to deploy TestPMD application in OpenShift. +![Operator behavior](../documentation/testpmd-operator.png) -Container images for the TestPMD application is built using [testpmd-container-app](https://github.com/openshift-kni/example-cnf/testpmd-container-app). +Final application, also known as CNF Application, which is a standard TestPMD instance using the default MAC forwarding module. It uses two components: -Preparation ----------- -* Deploy a OpenShift cluster with Baremetal worker nodes -* Deploy Performance Addon Operator and configure nodes with ``worker-cnf`` role -* Deploy SR-IOV Network Operator -* Create Peformance Profile -* Create SR-IOV Network Policy and Network +- TestPMD CR, which creates a pod to implement the MAC forwarding module as final application. Related pod is `testpmd-app-` pod (only one replica is used). +- TestPMD Operator, ensuring CR reconciliation via controller-manager pod. Related pod is `testpmd-operator-controller-manager-` pod. -TestPMD operator deployment ---------------------------- -Deploy the operator -``` -cd testpmd-operator -oc kustomize | oc -n example-cnf apply -f - -``` +The following information can be extracted from pod logs: + +- To see the TestPMD statistics printed periodically for this module, you can rely on `testpmd-app-` pod logs. +- In `testpmd-operator-controller-manager-` pod, you can see the execution of the Ansible playbooks that ensures the reconciliation loop of the operator. + +## How to build the operator + +Base structure is achieved with the following commands, then it's just a matter of accommodating the required code for the operator in the corresponding files and folders: -Modify the CR as per the cluster and create it: ``` -oc -n example-cnf apply -f deploy/crds/examplecnf.openshift.io_v1_testpmd_cr.yaml +$ mkdir testpmd-operator; cd testpmd-operator +$ operator-sdk init --domain openshift.io --plugins ansible +$ operator-sdk create api --version v1 --generate-role --group examplecnf --kind TestPMD ``` + +## What to update if bumping operator version + +Apart from the modifications you have to do, you also need to update the operator version in these files: + +- [CHANGELOG.md](CHANGELOG.md). +- [Makefile](Makefile). +- [Dockerfile](Dockerfile). + +Also, make sure that the operator version is within the interval defined in [required-annotations.yaml](../utils/required-annotations.yaml) file for `olm.skipRange` annotation, else update that file to modify the current range. + +A classical change is the update of Operator SDK version used in the operator. Here's an [example](https://github.com/openshift-kni/example-cnf/pull/108) where this is done. diff --git a/testpmd-operator/localtest.yaml b/testpmd-operator/localtest.yaml deleted file mode 100644 index 83f8317f..00000000 --- a/testpmd-operator/localtest.yaml +++ /dev/null @@ -1,24 +0,0 @@ -# Helper file, this playbook is not used inside the operator -# This playbook is useful to runnin locally for easy development -# Command: ansible-playbook -i playbook.yaml -vv ---- -- hosts: localhost - gather_facts: no - vars: - ansible_operator_meta: - name: testpmd-manual1 - namespace: example-cnf - tasks: - - include_role: - name: testpmd - vars: - privileged: true - image: jumphost.cluster5.dfwt5g.lab:5000/nfv-example-cnf/testpmd-container-app:v0.1.0 - image_pull_policy: IfNotPresent - ethpeer_maclist: - - 3c:fd:fe:79:2a:a0 - - 3c:fd:fe:79:2a:a1 - size: 1 - networks: - - name: vfdpdk1 - count: 2 diff --git a/testpmd-operator/play.yaml b/testpmd-operator/play.yaml deleted file mode 100644 index cee23ef4..00000000 --- a/testpmd-operator/play.yaml +++ /dev/null @@ -1,18 +0,0 @@ ---- -- hosts: localhost - gather_facts: no - vars: - networks: - - name: example-cnf-net1 - count: 2 - cluster_name: '' - operator_version: v0.2.4 - app_version: v0.2.2 - registry_url: quay.io - repo_name: rh-nfv-int - testpmd_node_count: 1 - ansible_operator_meta: - name: testpmd-1 - namespace: example-cnf - roles: - - testpmd diff --git a/testpmd-operator/testpmd-allinone.yaml b/testpmd-operator/testpmd-allinone.yaml deleted file mode 100644 index 4fc8a500..00000000 --- a/testpmd-operator/testpmd-allinone.yaml +++ /dev/null @@ -1,231 +0,0 @@ -apiVersion: v1 -kind: Namespace -metadata: - labels: - control-plane: controller-manager - name: example-cnf ---- -apiVersion: apiextensions.k8s.io/v1 -kind: CustomResourceDefinition -metadata: - name: testpmds.examplecnf.openshift.io -spec: - group: examplecnf.openshift.io - names: - kind: TestPMD - listKind: TestPMDList - plural: testpmds - singular: testpmd - scope: Namespaced - versions: - - name: v1 - schema: - openAPIV3Schema: - description: TestPMD is the Schema for the testpmds API - properties: - apiVersion: - description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' - type: string - kind: - description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' - type: string - metadata: - type: object - spec: - description: Spec defines the desired state of TestPMD - type: object - x-kubernetes-preserve-unknown-fields: true - status: - description: Status defines the observed state of TestPMD - type: object - x-kubernetes-preserve-unknown-fields: true - type: object - served: true - storage: true - subresources: - status: {} ---- -apiVersion: v1 -kind: ServiceAccount -metadata: - name: testpmd-operator-controller-manager - namespace: example-cnf ---- -apiVersion: rbac.authorization.k8s.io/v1 -kind: Role -metadata: - name: testpmd-operator-leader-election-role - namespace: example-cnf -rules: -- apiGroups: - - "" - resources: - - configmaps - verbs: - - get - - list - - watch - - create - - update - - patch - - delete -- apiGroups: - - "" - resources: - - events - verbs: - - create - - patch ---- -apiVersion: rbac.authorization.k8s.io/v1 -kind: ClusterRole -metadata: - name: testpmd-operator-manager-role -rules: -- apiGroups: - - "" - resources: - - secrets - - pods - - pods/exec - - pods/log - - configmaps - verbs: - - create - - delete - - get - - list - - patch - - update - - watch -- apiGroups: - - apps - resources: - - deployments - - daemonsets - - replicasets - - statefulsets - verbs: - - create - - delete - - get - - list - - patch - - update - - watch -- apiGroups: - - examplecnf.openshift.io - resources: - - testpmds - - testpmds/status - verbs: - - create - - delete - - get - - list - - patch - - update - - watch -- apiGroups: - - k8s.cni.cncf.io - resources: - - network-attachment-definitions - verbs: - - get - - list - - watch -- apiGroups: - - security.openshift.io - resourceNames: - - privileged - resources: - - securitycontextconstraints - verbs: - - use ---- -apiVersion: rbac.authorization.k8s.io/v1 -kind: RoleBinding -metadata: - name: testpmd-operator-leader-election-rolebinding - namespace: example-cnf -roleRef: - apiGroup: rbac.authorization.k8s.io - kind: Role - name: testpmd-operator-leader-election-role -subjects: -- kind: ServiceAccount - name: testpmd-operator-controller-manager - namespace: example-cnf ---- -apiVersion: rbac.authorization.k8s.io/v1 -kind: ClusterRoleBinding -metadata: - name: testpmd-operator-manager-rolebinding -roleRef: - apiGroup: rbac.authorization.k8s.io - kind: ClusterRole - name: testpmd-operator-manager-role -subjects: -- kind: ServiceAccount - name: testpmd-operator-controller-manager - namespace: example-cnf ---- -apiVersion: apps/v1 -kind: Deployment -metadata: - labels: - control-plane: controller-manager - name: testpmd-operator-controller-manager - namespace: example-cnf -spec: - replicas: 1 - selector: - matchLabels: - control-plane: controller-manager - template: - metadata: - labels: - control-plane: controller-manager - example-cnf-type: testpmd-operator - spec: - containers: - - args: - - --metrics-addr=127.0.0.1:8080 - - --enable-leader-election - - --leader-election-id=testpmd-operator - env: - - name: ANSIBLE_GATHERING - value: explicit - image: quay.io/rh-nfv-int/testpmd-operator:v0.2.4 - imagePullPolicy: IfNotPresent - lifecycle: - postStart: - exec: - command: ["/bin/sh", "-c", "echo Hello from the postStart handler"] - preStop: - exec: - command: ["/bin/sh", "-c", "echo Hello from the preStop handler"] - livenessProbe: - httpGet: - path: /healthz - port: 6789 - initialDelaySeconds: 15 - periodSeconds: 20 - name: manager - readinessProbe: - httpGet: - path: /readyz - port: 6789 - initialDelaySeconds: 5 - periodSeconds: 10 - # operator-sdk is not implementing startup probe up to now, using liveness probe endpoint - # while this feature is developed - startupProbe: - httpGet: - path: /healthz - port: 6789 - initialDelaySeconds: 5 - periodSeconds: 10 - terminationMessagePolicy: FallbackToLogsOnError - serviceAccountName: testpmd-operator-controller-manager - terminationGracePeriodSeconds: 10 diff --git a/trex-container-app/README.md b/trex-container-app/README.md new file mode 100644 index 00000000..43064829 --- /dev/null +++ b/trex-container-app/README.md @@ -0,0 +1,6 @@ +# trex-container-app + +Container images used by [trex-operator](../trex-operator/). We have the following projects: + +- [app](app) +- [server](server) diff --git a/trex-container-app/app/README.md b/trex-container-app/app/README.md new file mode 100644 index 00000000..a94665eb --- /dev/null +++ b/trex-container-app/app/README.md @@ -0,0 +1,19 @@ +# app + +Container image that allows you to configure a traffic profile for TRex and deploy it in a running TRex instance. + +## What is it executed? + +This container runs `/usr/local/bin/trex-wrapper` (but a different one compared to `server`). [This script](scripts/trex-wrapper) triggers a [custom Python script](scripts/run-trex) that builds the traffic profile, according to the TRex variables provided in the pipeline (duration of the job, packet size, data rate, etc.), and start sending the traffic. If defining a duration, a timeout will be enabled to stop the execution after the given duration, and then statistics will be printed and packet loss will be calculated. If packet loss is equal or less than 0, this means there's no packet loss, else, packet loss is present and the pod status will be different than Completed. + +You have more details about testing this feature in the [testing docs](../../documentation/testing.md). + +## What to update if bumping container version + +Apart from the modifications you have to do, you also need to update the container version in these files: + +- [Dockerfile](Dockerfile). +- [build.sh](../build.sh) (from parent folder). +- [Makefile](../Makefile) (from parent folder). + +Here's an [example](https://github.com/openshift-kni/example-cnf/pull/111) where this is done. diff --git a/trex-container-app/app/scripts/README.md b/trex-container-app/app/scripts/README.md deleted file mode 100644 index cd67763e..00000000 --- a/trex-container-app/app/scripts/README.md +++ /dev/null @@ -1,11 +0,0 @@ -Entrypoints -=========== -`trex-wrapper` is the default entry point for trex app container image. - -Choosing Run Apps -================= -Within trex app, there are multiple entrypoints for running each use case. -In order to choose a usecase, provide `run_app` with appropriate value in -the container environment, while creating the POD. - -`run_app=1` - Run the app with default profile. Monitory for mac update CR. diff --git a/trex-container-app/server/README.md b/trex-container-app/server/README.md new file mode 100644 index 00000000..603fe825 --- /dev/null +++ b/trex-container-app/server/README.md @@ -0,0 +1,63 @@ +# server + +Container image that allows you to configure TRex in a container and prepare it to launch it. + +## What is it executed? + +This container runs `/usr/local/bin/trex-wrapper`. [This script](scripts/trex-wrapper) takes also some context information to build a config file for TRex, such as this one: + +``` +sh-4.4$ pwd +/usr/local/bin/example-cnf +sh-4.4$ ls +trex-server-run trex_cfg.yaml +sh-4.4$ cat trex_cfg.yaml +- c: 4 + interfaces: + - '86:02.3' + - '86:03.2' + platform: + dual_if: + - socket: 1 + threads: + - 7 + - 51 + - 53 + - 55 + latency_thread_id: 5 + master_thread_id: 3 + port_info: + - dest_mac: 80:04:0f:f1:89:01 + src_mac: 20:04:0f:f1:89:01 + - dest_mac: 80:04:0f:f1:89:02 + src_mac: 20:04:0f:f1:89:02 + version: 2 +``` + +You can see it saves in the config file the following information, among others: + +- PCI addresses to use. +- Number of sockets. +- CPUs to be used. +- Pair of destination-source MAC addresses for each port. + +Then, it builds a script file, placed in `/usr/local/bin/example-cnf/trex-server-run`, with the following content: + +``` +sh-4.4$ cat /usr/local/bin/example-cnf/trex-server-run +/usr/local/bin/trex-server 4 +``` + +This calls to [trex-server](scripts/trex-server) script, passing 4 as argument which represents the number of coures to use, which launches `_t-rex-64` binary using the generated config file (`--cfg` argument), using interactive mode (`-i` argument), which allows to print statistics regularly. This does not launch the traffic generation, since this is done with the TRexApp job. + +You have more details about testing this feature in the [testing docs](../../documentation/testing.md). + +## What to update if bumping container version + +Apart from the modifications you have to do, you also need to update the container version in these files: + +- [Dockerfile](Dockerfile). +- [build.sh](../build.sh) (from parent folder). +- [Makefile](../Makefile) (from parent folder). + +Here's an [example](https://github.com/openshift-kni/example-cnf/pull/111) where this is done. diff --git a/trex-container-app/server/v3-bus-pci-fix-VF-bus-error-for-memory-access.diff b/trex-container-app/server/v3-bus-pci-fix-VF-bus-error-for-memory-access.diff deleted file mode 100644 index af9d8384..00000000 --- a/trex-container-app/server/v3-bus-pci-fix-VF-bus-error-for-memory-access.diff +++ /dev/null @@ -1,55 +0,0 @@ -diff --git a/src/dpdk/drivers/bus/pci/linux/pci_vfio.c b/src/dpdk/drivers/bus/pci/linux/pci_vfio.c -index 64cd84a6..03cc98ee 100644 ---- a/src/dpdk/drivers/bus/pci/linux/pci_vfio.c -+++ b/src/dpdk/drivers/bus/pci/linux/pci_vfio.c -@@ -149,6 +149,38 @@ pci_vfio_get_msix_bar(int fd, struct pci_msix_table *msix_table) - return 0; - } - -+/* enable PCI bus memory space */ -+static int -+pci_vfio_enable_bus_memory(int dev_fd) -+{ -+ uint16_t cmd; -+ int ret; -+ -+ ret = pread64(dev_fd, &cmd, sizeof(cmd), -+ VFIO_GET_REGION_ADDR(VFIO_PCI_CONFIG_REGION_INDEX) + -+ PCI_COMMAND); -+ -+ if (ret != sizeof(cmd)) { -+ RTE_LOG(ERR, EAL, "Cannot read command from PCI config space!\n"); -+ return -1; -+ } -+ -+ if (cmd & PCI_COMMAND_MEMORY) -+ return 0; -+ -+ cmd |= PCI_COMMAND_MEMORY; -+ ret = pwrite64(dev_fd, &cmd, sizeof(cmd), -+ VFIO_GET_REGION_ADDR(VFIO_PCI_CONFIG_REGION_INDEX) + -+ PCI_COMMAND); -+ -+ if (ret != sizeof(cmd)) { -+ RTE_LOG(ERR, EAL, "Cannot write command to PCI config space!\n"); -+ return -1; -+ } -+ -+ return 0; -+} -+ - /* set PCI bus mastering */ - static int - pci_vfio_set_bus_master(int dev_fd, bool op) -@@ -427,6 +459,11 @@ pci_rte_vfio_setup_device(struct rte_pci_device *dev, int vfio_dev_fd) - return -1; - } - -+ if (pci_vfio_enable_bus_memory(vfio_dev_fd)) { -+ RTE_LOG(ERR, EAL, "Cannot enable bus memory!\n"); -+ return -1; -+ } -+ - /* set bus mastering for the device */ - if (pci_vfio_set_bus_master(vfio_dev_fd, true)) { - RTE_LOG(ERR, EAL, "Cannot set up bus mastering!\n"); diff --git a/trex-container-app/tests/annotations b/trex-container-app/tests/annotations deleted file mode 100644 index 4932960e..00000000 --- a/trex-container-app/tests/annotations +++ /dev/null @@ -1,3 +0,0 @@ -k8s.v1.cni.cncf.io/networks="[ { \"name\": \"vfdpdk1\", \"mac\": \"20:04:0f:f1:89:01\", \"namespace\": \"example-cnf\" }, { \"name\": \"vfdpdk2\", \"mac\": \"20:04:0f:f1:89:02\", \"namespace\": \"example-cnf\" } ]" -kubernetes.io/config.seen="2020-07-22T13:10:35.720154784Z" -kubernetes.io/config.source="api" diff --git a/trex-operator/README.md b/trex-operator/README.md index 9fb8270a..affb023b 100644 --- a/trex-operator/README.md +++ b/trex-operator/README.md @@ -1,24 +1,38 @@ -## How to rebuild trex-operator +# trex-operator -1. Build new version of the application container (vx.y.z is currently v0.2.6) +![Operator behavior](../documentation/trex-operator.png) -``` -trex-container-app]$ bash build.sh app -Successfully tagged quay.io/rh-nfv-int/trex-container-app:vx.y.z -trex-container-app]$ podman push quay.io/rh-nfv-int/trex-container-app:vx.y.z -``` +It provides TRex Traffic Generator, decomposed in three components: -2. Build new version of the server. +- TRexConfig CR, pinning the config of the TRex server instance. Related pod is `trexconfig-` pod. +- TRexApp CR, setting up the job that launches TRex execution to generate traffic towards the system under test. Related pod is `trex-app` pod. +- TRex Operator, ensuring CR reconciliation via controller-manager pod. Related pod is `trex-operator-controller-manager-` pod. -``` -trex-container-app]$ bash build.sh server -Successfully tagged quay.io/rh-nfv-int/trex-container-server:vx.y.z -trex-container-app]$ podman push quay.io/rh-nfv-int/trex-container-server:vx.y.z -``` +Following information can be extracted from pod logs: + +- In `trexconfig-` pod logs, you can see the trex statistics printed periodically. +- The summary of the test execution can be seen at the end of the `trex-app` job logs. +- In `trex-operator-controller-manager-` pod, you can see the execution of the Ansible playbooks that ensures the reconciliation loop of the operator. + +## How to build the operator -3. Rebuild trex-operator va.b.c (va.b.c is currently v0.2.12) +Base structure is achieved with the following commands, then it's just a matter of accommodating the required code for the operator in the corresponding files and folders: ``` -trex-operator]$ make operator-all -trex-operator]$ make bundle-all +$ mkdir trex-operator; cd trex-operator +$ operator-sdk init --domain openshift.io --plugins ansible +$ operator-sdk create api --version v1 --generate-role --group examplecnf --kind TRexApp +$ operator-sdk create api --version v1 --generate-role --group examplecnf --kind TRexConfig ``` + +## What to update if bumping operator version + +Apart from the modifications you have to do, you also need to update the operator version in these files: + +- [CHANGELOG.md](CHANGELOG.md). +- [Makefile](Makefile). +- [Dockerfile](Dockerfile). + +Also, make sure that the operator version is within the interval defined in [required-annotations.yaml](../utils/required-annotations.yaml) file for `olm.skipRange` annotation, else update that file to modify the current range. + +A classical change is the update of Operator SDK version used in the operator. Here's an [example](https://github.com/openshift-kni/example-cnf/pull/108) where this is done. diff --git a/trex-operator/localtest.yaml b/trex-operator/localtest.yaml deleted file mode 100644 index f7d38fd4..00000000 --- a/trex-operator/localtest.yaml +++ /dev/null @@ -1,35 +0,0 @@ -# Helper file, this playbook is not used inside the operator -# This playbook is useful to runnin locally for easy development -# Command: ansible-playbook -i playbook.yaml -vv ---- -- hosts: localhost - gather_facts: no - vars: - ansible_operator_meta: - name: trex-manual1 - namespace: example-cnf - privileged: false - image: quay.io/krsacme/trex-container-app:v0.0.1 - imagePullPolicy: IfNotPresent - size: 1 - networks: - - name: vfdpdk1 - count: 1 - - name: vfdpdk2 - count: 1 - command: ["sleep infinity"] - - tasks: - - template: - src: roles/trexconfig/templates/deployment.yml - dest: /tmp/deployment.yaml - vars: - network_name_list: ["vfdpdk1","vfdpdk2"] - hugepage_1gb_count: 3 - memory: 1000 - cpu: 4 - network_resources: {} - - fail: - - include_role: - name: trex - diff --git a/trex-operator/trex-allinone.yaml b/trex-operator/trex-allinone.yaml deleted file mode 100644 index 4e4a550c..00000000 --- a/trex-operator/trex-allinone.yaml +++ /dev/null @@ -1,252 +0,0 @@ -apiVersion: v1 -kind: Namespace -metadata: - labels: - control-plane: controller-manager - name: example-cnf ---- -apiVersion: apiextensions.k8s.io/v1 -kind: CustomResourceDefinition -metadata: - name: trexconfigs.examplecnf.openshift.io -spec: - group: examplecnf.openshift.io - names: - kind: TRexConfig - listKind: TRexConfigList - plural: trexconfigs - singular: trexconfig - scope: Namespaced - versions: - - name: v1 - schema: - openAPIV3Schema: - description: TRexConfig is the Schema for the trexconfigs API - properties: - apiVersion: - description: 'APIVersion defines the versioned schema of this representation - of an object. Servers should convert recognized schemas to the latest - internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' - type: string - kind: - description: 'Kind is a string value representing the REST resource this - object represents. Servers may infer this from the endpoint the client - submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' - type: string - metadata: - type: object - spec: - description: Spec defines the desired state of TRexConfig - type: object - x-kubernetes-preserve-unknown-fields: true - status: - description: Status defines the observed state of TRexConfig - type: object - x-kubernetes-preserve-unknown-fields: true - type: object - served: true - storage: true - subresources: - status: {} ---- -apiVersion: rbac.authorization.k8s.io/v1 -kind: Role -metadata: - name: trex-operator-leader-election-role - namespace: example-cnf -rules: -- apiGroups: - - "" - resources: - - configmaps - verbs: - - get - - list - - watch - - create - - update - - patch - - delete -- apiGroups: - - "" - resources: - - events - verbs: - - create - - patch ---- -apiVersion: rbac.authorization.k8s.io/v1 -kind: ClusterRole -metadata: - name: trex-operator-manager-role -rules: -- apiGroups: - - "" - resources: - - secrets - - pods - - pods/exec - - pods/log - verbs: - - create - - delete - - get - - list - - patch - - update - - watch -- apiGroups: - - apps - resources: - - deployments - - daemonsets - - replicasets - - statefulsets - verbs: - - create - - delete - - get - - list - - patch - - update - - watch -- apiGroups: - - examplecnf.openshift.io - resources: - - trexconfigs - - trexconfigs/status - verbs: - - create - - delete - - get - - list - - patch - - update - - watch -- apiGroups: - - security.openshift.io - resourceNames: - - privileged - resources: - - securitycontextconstraints - verbs: - - use ---- -apiVersion: rbac.authorization.k8s.io/v1 -kind: ClusterRole -metadata: - name: trex-operator-proxy-role -rules: -- apiGroups: - - authentication.k8s.io - resources: - - tokenreviews - verbs: - - create -- apiGroups: - - authorization.k8s.io - resources: - - subjectaccessreviews - verbs: - - create ---- -apiVersion: rbac.authorization.k8s.io/v1beta1 -kind: ClusterRole -metadata: - name: trex-operator-metrics-reader -rules: -- nonResourceURLs: - - /metrics - verbs: - - get ---- -apiVersion: rbac.authorization.k8s.io/v1 -kind: RoleBinding -metadata: - name: trex-operator-leader-election-rolebinding - namespace: example-cnf -roleRef: - apiGroup: rbac.authorization.k8s.io - kind: Role - name: trex-operator-leader-election-role -subjects: -- kind: ServiceAccount - name: default - namespace: example-cnf ---- -apiVersion: rbac.authorization.k8s.io/v1 -kind: ClusterRoleBinding -metadata: - name: trex-operator-manager-rolebinding -roleRef: - apiGroup: rbac.authorization.k8s.io - kind: ClusterRole - name: trex-operator-manager-role -subjects: -- kind: ServiceAccount - name: default - namespace: example-cnf ---- -apiVersion: rbac.authorization.k8s.io/v1 -kind: ClusterRoleBinding -metadata: - name: trex-operator-proxy-rolebinding -roleRef: - apiGroup: rbac.authorization.k8s.io - kind: ClusterRole - name: trex-operator-proxy-role -subjects: -- kind: ServiceAccount - name: default - namespace: example-cnf ---- -apiVersion: v1 -kind: Service -metadata: - labels: - control-plane: controller-manager - name: trex-operator-controller-manager-metrics-service - namespace: example-cnf -spec: - ipFamilyPolicy: "PreferDualStack" - ports: - - name: https - port: 8443 - targetPort: https - selector: - control-plane: controller-manager ---- -apiVersion: apps/v1 -kind: Deployment -metadata: - labels: - control-plane: controller-manager - name: trex-operator-controller-manager - namespace: example-cnf -spec: - replicas: 1 - selector: - matchLabels: - control-plane: controller-manager - template: - metadata: - labels: - control-plane: controller-manager - spec: - containers: - - args: - - --metrics-addr=127.0.0.1:8080 - - --enable-leader-election - - --leader-election-id=trex-operator - image: jumphost.cluster5.dfwt5g.lab:5000/nfv-example-cnf/trex-operator:v0.1.0 - imagePullPolicy: IfNotPresent - name: manager - lifecycle: - postStart: - exec: - command: ["/bin/sh", "-c", "echo Hello from the postStart handler"] - preStop: - exec: - command: ["/bin/sh", "-c", "echo Hello from the preStop handler"] - terminationMessagePolicy: FallbackToLogsOnError - terminationGracePeriodSeconds: 10 diff --git a/utils/README.md b/utils/README.md new file mode 100644 index 00000000..f772765d --- /dev/null +++ b/utils/README.md @@ -0,0 +1,20 @@ +# Utils + +Here, you can find some utilities included in Example CNF to extend the functionalities offered by the tool. + +- [webserver.go](webserver.go): a Golang-based webserver to implement liveness, readiness and startup probes in the container images offered in [testpmd-container-app](../testpmd-container-app) and [trex-container-app](../trex-container-app) folders. The Makefiles offered in these directories take care of copying the webserver code from the utils directory to each image's directory. +- [required-annotations.yaml](required-annotations.yaml): annotations to be appended to the CSVs to pass Preflight's RequiredAnnotations tests. They are appended automatically thanks to the Makefile tasks from each operator. +- [support-images](support_images): projects where you can find the Dockerfile required to build some of the images used as build images by the Example CNF images. These images can be found on quay.io/rh-nfv-int and they are publicly available, you only need credentials to access quay.io. The images can be built with the following commands (you need to run it in a RHEL host with a valid RHEL subscription to be able to download the packages installed in the images, and you need a valid quay.io credentials to push it to quay.io): + +``` +# build images +$ cd support-images +$ podman build dpdk-23.11 -f dpdk-23.11/Dockerfile -t "quay.io/rh-nfv-int/dpdk-23.11:v0.0.1" +$ podman build ubi8-base-testpmd -f ubi8-base-testpmd/Dockerfile -t "quay.io/rh-nfv-int/ubi8-base-testpmd:v0.0.1" +$ podman build ubi8-base-trex -f ubi8-base-trex/Dockerfile -t "quay.io/rh-nfv-int/ubi8-base-trex:v0.0.1" + +# push images (to quay.io/rh-nfv-int) +$ podman push quay.io/rh-nfv-int/dpdk-23.11:v0.0.1 +$ podman push quay.io/rh-nfv-int/ubi8-base-testpmd:v0.0.1 +$ podman push quay.io/rh-nfv-int/ubi8-base-trex:v0.0.1 +``` From 822ce31b5b76f27351b953d75b6913fc3f254ad4 Mon Sep 17 00:00:00 2001 From: Ramon Perez Date: Thu, 13 Feb 2025 08:50:27 +0100 Subject: [PATCH 2/2] Replace classical by common --- cnf-app-mac-operator/README.md | 2 +- testpmd-operator/README.md | 2 +- trex-operator/README.md | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/cnf-app-mac-operator/README.md b/cnf-app-mac-operator/README.md index a759540f..4bd490b8 100644 --- a/cnf-app-mac-operator/README.md +++ b/cnf-app-mac-operator/README.md @@ -60,4 +60,4 @@ Apart from the modifications you have to do, you also need to update the operato Also, make sure that the operator version is within the interval defined in [required-annotations.yaml](../utils/required-annotations.yaml) file for `olm.skipRange` annotation, else update that file to modify the current range. -A classical change is the update of Operator SDK version used in the operator. Here's an [example](https://github.com/openshift-kni/example-cnf/pull/108) where this is done. +A common change is the update of Operator SDK version used in the operator. Here's an [example](https://github.com/openshift-kni/example-cnf/pull/108) where this is done. diff --git a/testpmd-operator/README.md b/testpmd-operator/README.md index a56aa970..4f07b4a1 100644 --- a/testpmd-operator/README.md +++ b/testpmd-operator/README.md @@ -32,4 +32,4 @@ Apart from the modifications you have to do, you also need to update the operato Also, make sure that the operator version is within the interval defined in [required-annotations.yaml](../utils/required-annotations.yaml) file for `olm.skipRange` annotation, else update that file to modify the current range. -A classical change is the update of Operator SDK version used in the operator. Here's an [example](https://github.com/openshift-kni/example-cnf/pull/108) where this is done. +A common change is the update of Operator SDK version used in the operator. Here's an [example](https://github.com/openshift-kni/example-cnf/pull/108) where this is done. diff --git a/trex-operator/README.md b/trex-operator/README.md index affb023b..339de6b4 100644 --- a/trex-operator/README.md +++ b/trex-operator/README.md @@ -35,4 +35,4 @@ Apart from the modifications you have to do, you also need to update the operato Also, make sure that the operator version is within the interval defined in [required-annotations.yaml](../utils/required-annotations.yaml) file for `olm.skipRange` annotation, else update that file to modify the current range. -A classical change is the update of Operator SDK version used in the operator. Here's an [example](https://github.com/openshift-kni/example-cnf/pull/108) where this is done. +A common change is the update of Operator SDK version used in the operator. Here's an [example](https://github.com/openshift-kni/example-cnf/pull/108) where this is done.