Updating the readme

README.md

@@ -4,122 +4,145 @@ Kubernetes manifest files for [notification.canada.ca](https://notification.cana
 
 ## How does this repository work?
 
-This repository uses version 2.0.3 of [Kustomize](https://github.com/kubernetes-sigs/kustomize/tree/v2.0.3), which is baked into `kubectl` to apply different environment overlays (staging, production), on to an existing base configuration. As a result the `base` directory describes all the commonalities between all environments, while the [`env/staging`](env/staging) and [`env/production`](env/production) directories contain the environment specific configurations. These include:
+Notify components are deployed to Kubernetes using [helm](https://helm.sh/) and [helmfile](https://github.com/helmfile/helmfile). This allows us to do "diffs" on our environments, and apply applications surgically using labeles.
 
-- Environment variables
-- Target group bindings between the AWS network infrastructure and the Kubernetes cluster
-- Replica count patches (ex. How many pods of each type run in each environment)
+```bash
+helmfile -e dev -l app=notify-admin diff
+```
+
+```bash
+helmfile -e dev -l app=notify-admin sync
+```
 
 ## How are environment variables set?
 
-To set the variables for the API Lambda see [this](https://github.com/cds-snc/notification-terraform#awslambda-api)
+### Environment Variables
 
-`Kustomize` can dynamically inject environment variables when it compiles the configuration. To do this it reads out the environment variables and creates a `ConfigMap` object using an `.env` file that is in the same directory as the overlay that is being called from (ex. `/env/staging/.env`). As it is bad practice to save environment variables to a Git repository, the `.env` is ignored, and instead saved in an encrypted envelope using AWS KMS as `.env.enc.aws` files.
+Environment variables are set using a key/value pair in the values.yaml of the GC Notify helm charts, along with their overrides for each environment. [See the below section for adding a new environment variable.](##HowdoIaddanewenvironmentvariable?)
 
-This means that before the overlay is applied, the file needs to be decrypted.
+### Secrets
 
-### Decrypting environment variables
+Secrets are set using Kubernetes secrets, which ensures that we do not ever spit sensitive information out on helmfile diffs or logs. [To add a secret please follow this guide.](https://github.com/cds-snc/notification-terraform/blob/main/docs/creatingSecrets.md)
 
-You should leverage the appropriate commands in the Makefile:
 
-- the `make decrypt-staging` command that will decrypt environment variables in staging ;
-- the `make decrypt-production` command that will decrypt environment variables in production.
+## How do I add a new environment variable?
 
-```sh
-# You need to have the AWS credentials set up on your machine
-# under the profile name `notify-staging`.
-AWS_PROFILE=notify-staging make decrypt-staging
-# This creates a new decrypted file at env/staging/.env
-```
+### READ THIS
+
+If the value you are adding is a secret, DO NOT USE THIS METHOD. [Instead follow the instructions in the adding a secret document.](https://github.com/cds-snc/notification-terraform/blob/main/docs/creatingSecrets.md)
+
+### Steps for everything but API
 
-### Encrypting variables in staging/production
+To add an environment variable for admin or celery, you will need to update up to 3 files. 
 
-You should leverage the appropriate commands in the Makefile:
-- the `make encrypt-staging` command that will encrypt environment variables in staging ;
+1. In the values.yaml file in helmfile/charts/notify-\<component\> add a new key value pair with the expected default value under the main component tag (ex. admin: or celeryCommon: etc)
 
-```sh
-AWS_PROFILE=notify-staging make decrypt-staging
-# Change values in the decrypted file at env/staging/.env
-# Encrypt the decrypted file that you just edited
-AWS_PROFILE=notify-staging make encrypt-staging
-# Creates a new file at env/staging/.env.enc.aws which is safe to commit
+```yaml
+admin:
+  NOTIFY_ENVIRONMENT: "default"
+  AWS_REGION: "ca-central-1"
+  ALLOW_HTML_SERVICE_IDS: "1,2,3,4"
+  ........
+  GC_ORGANISATIONS_BUCKET_NAME: "notification-canada-ca-default-gc-organisations"
+  SENDING_DOMAIN: "notification.canada.ca"
+  SOME_NEW_ENV_VAR: "someDefaultValue"
 ```
 
-- the `make encrypt-production` command that will encrypt environment variables in production.
+2. Add a corresponding value in the overrides file in helmfile/overrides/notify/\<component\>.yaml.gotmpl for the component. You have choices on how to configure this. 
 
+If this is a straight up true/false feature flag, use 
 
-```sh
-AWS_PROFILE=notify-production make decrypt-production
-# Change values in the decrypted file at env/production/.env
-# Encrypt the decrypted file that you just edited
-AWS_PROFILE=notify-production make encrypt-production
-# Creates a new file at env/production/.env.enc.aws which is safe to commit
+```yaml
+SOME_NEW_ENV_VAR: "{{ .StateValues.SOME_NEW_ENV_VAR }}"
 ```
 
-## How do I add a new environment variable?
+If this is a string that changes based on environment name, you can leverage built in helmfile keywords:
 
-As mentioned above, you will need to make changes to the `.env` file to include them in the `ConfigMap` object. In addition, you need to set up the `kustomization.yaml` file to include the new environment variable in the `ConfigMap`. This would look something like this if you wanted to add the variable `FOO` with the value `BAR`:
+```yaml
+SOME_NEW_ENV_VAR: "notify-new-env-var-{{ .Environment.Name }}"
+```
 
-In a `.env` file add:
+If this is a URL, you can leverage the BASE_DOMAIN environment variable that is set in getContext.sh
 
+```yaml
+SOME_NEW_ENV_VAR: "https://some-new-url.{{ requiredEnv "BASE_DOMAIN" }}"
 ```
-FOO=BAR
+
+3. If the value of the new environment variable can't be set dynamically with the environment name or required environment variables (i.e. a true/false feature flag), you must now modify each environment.env file in helmfile/overrides/\<environmnent\>.env (State Value Files) to include your new variable and its configuration for each environment.
+
+staging.env
+
+```yaml
+
+ADMIN_DOCKER_TAG: "1d912b8"
+API_DOCKER_TAG: "1270b81"
+DOCUMENT_DOWNLOAD_DOCKER_TAG: "7f5dc9d"
+......
+GC_ORGANISATIONS_BUCKET_NAME: "notification-canada-ca-staging-gc-organisations"
+SOME_NEW_ENV_VAR: "true"
+
 ```
 
-and then in `kustomization.yaml` add:
+production.env 
 
 ```yaml
-- name: FOO
-  objref:
-    kind: ConfigMap
-    name: application-config
-    apiVersion: v1
-  fieldref:
-    fieldpath: data.FOO
+
+ADMIN_DOCKER_TAG: "1d912b8"
+API_DOCKER_TAG: "1270b81"
+DOCUMENT_DOWNLOAD_DOCKER_TAG: "7f5dc9d"
+......
+GC_ORGANISATIONS_BUCKET_NAME: "notification-canada-ca-production-gc-organisations"
+SOME_NEW_ENV_VAR: "false"
 
 ```
 
-under the `vars:` key.
+### Steps for API
 
-You can now reference the variable in you other manifest files using `$(FOO)`.
+Since the API is run on both kubernetes and AWS Lambda, you will need to modify the Terraform parameter store configuration as well. Follow the steps in the preceeding section and then [update the terraform file as necessary.](https://github.com/cds-snc/notification-terraform/blob/main/aws/lambda-api/secrets_manager.tf)
 
-You also need to add the new variable in `env.example` so that we can run CI without using any actual live variables.
+```terraform
 
-The last thing you need to do is re-encrypt the `.env` file to make sure the variable gets saved. You can use the `make encrypt-staging` command to do this.
+resource "aws_ssm_parameter" "environment_variables" {
+  count       = var.bootstrap ? 1 : 0
+  name        = "ENVIRONMENT_VARIABLES"
+  description = "Environment variables for the API Lambda function"
+  type        = "SecureString"
+  tier        = "Advanced"
+  value       = <<EOF
+ADMIN_CLIENT_SECRET=${var.manifest_admin_client_secret}
+ALLOW_DEBUG_ROUTE=false
+ALLOW_HTML_SERVICE_IDS=4de8b784-03a8-4ba8-a440-3bfea1b04fe6,ea608120-148a-4eba-a64c-4d9a8010e7b0
+API_HOST_NAME=https://api.${var.base_domain}
 
-## How are image tag sets?
+........
+
+CYPRESS_USER_PW_SECRET=${var.manifest_cypress_user_pw_secret}
+CYPRESS_AUTH_CLIENT_SECRET=${var.manifest_cypress_auth_client_secret}
+SOME_NEW_ENV_VAR: "some-value"
+EOF
+}
 
-To adjust what images are used in the environments, you need to set them in the environment `kustomization.yaml` file:
 
-```yaml
-images:
-  - name: admin
-    newName: public.ecr.aws/cds-snc/notify-admin:latest
-  - name: api
-    newName: public.ecr.aws/cds-snc/notify-api:latest
-  - name: document-download-api
-    newName: public.ecr.aws/cds-snc/notify-document-download-api:latest
 ```
 
-Will set the images in the base deployment to use `latest`.
+## How are image tags set?
 
-In production, we use set image hashes directly, take a look at [`env/production/kustomization.yaml`](env/production/kustomization.yaml).
+Image tags are automatically set via Notify PR Bot in the helmfile/overrides/\<environment\>.env file
 
-## Connecting to the database
+```yaml
 
-1. First shell into the `jump-box` container inside the Kubernetes cluster (note the `-848d9c6787-p4r2v` suffix will be different):
-```sh
-kubectl exec -n notification-canada-ca -it jump-box-848d9c6787-p4r2v -- /bin/sh 
-```
+ADMIN_DOCKER_TAG: "3efe89b"
+API_DOCKER_TAG: "1270b81"
+DOCUMENT_DOWNLOAD_DOCKER_TAG: "7f5dc9d"
+DOCUMENTATION_DOCKER_TAG: "025892e"
 
-2. Use `socat` to forward all traffic from the `jump-box`'s port 5430 to the `DB_HOST_NAME` port 5432. `DB_HOST_NAME` should be something like `notification-canada-ca-staging-cluster.cluster-....ca-central-1.rds.amazonaws.com `
-```sh
-socat TCP-LISTEN:5430,fork TCP:DB_HOST_NAME:5432
 ```
 
-3. Last, on your local machine, map the `jump-box` remote port 5430 to your local port 5430 (note the `-848d9c6787-p4r2v` suffix will be different):
-```sh
-kubectl port-forward -n notification-canada-ca jump-box-848d9c6787-p4r2v 5430:5430 
-```
+## Connecting to the database
 
-4. You can now connect to the database on your local port 5430 using the username and password. Please do not forget to terminate the `socat` connection with `Ctrl-C` once you are done.
+1. Obtain the postgres credentials from the 1Password TERRAFORM_SECRETS_<ENVIRONMENT> secret: 
+- app_db_user
+- app_db_user_password
+2. [Obtain the postgres URL from AWS](https://ca-central-1.console.aws.amazon.com/rds/home?region=ca-central-1#databases:)
+3. Connect to the appropriate environment VPN
+4. Connect to the database