v0.17.0

Thanks to everyone who created feature requests, bug reports and tested fixes. Your help is instrumental to continue to improve Cupdate!

Features

Tracking workflow runs

Cupdate uses a workflow to process images, the workflow runs jobs and jobs consist of steps. If you're familiar with Github Actions, Cupdate's workflows work much the same.

Being a core part of Cupdate, observability of the workflows progress is key to debugging issues. In v0.17.0, Cupdate now tracks each and every workflow run, making their durations, statuses and errors available via the API and UI as soon as they've completed.

The graph shows each job of the latest workflow run. If you click one of the jobs, a summary of the steps run as part of the job is shown. If you're using tracing, a tracing id is shown, enabling you to continue correlating the run using your observability platform of choice. See docs/observability for more information.

If a step would fail, the error causing it to fail is shown.

Tracking the workflows also allows Cupdate to show a summary of failed images on the dashboard and via the API, for use with services like Grafana or Homepage. If you're already using Homepage, see the updated documentation in docs/cookbook.

This also allows us to tag failing images, making it easy to find them.

For now, only the latest workflow run is exposed through APIs. Workflow runs are by default kept for 48h. See docs/config for details on the new environment variables CUPDATE_WORKFLOW_CLEANUP_MAX_AGE and CUPDATE_WORKFLOW_CLEANUP_INTERVAL.

Ignore images using labels

Cupdate can now be configured to ignore images, containers, deployments, any Docker or Kubernetes resource that takes a label.

# deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: cupdate
    config.cupdate/ignore: "true"
# ...

# compose.yaml
services:
  cupdate:
    labels:
      - cupdate.config.ignore: "true"

docker run --rm -it --label "cupdate.config.ignore=true" alpine:3

See docs/config for more information.

Tracking data changes

Whenever Cupdate processes an image and founds new data, that data is stored to an sqlite database. Starting in v0.17.0, Cupdate now internally tracks changes made to the data stored. Although not a front-facing feature, this enables improvements in existing features and lays the foundation for future improvements. If you're using the API, you can now get events whenever an image is processed or when there's a new version available. See the API docs for more details.

One improvement made to the existing feature set is that the web app can now tell what was updated and only prompt the user once data affecting the current page changes. This ensures that you won't get the toast every time Cupdate processes the image and updates basic fields such as when the image was last processed.

Worker queue rewrite

In previous Cupdate versions, the internal queue used for processing references had a fixed max size and provided bad observability. At times, it made it impossible to manually schedule an image for processing via the UI or APIs.

In v0.17.0, the worker queue implementation is rewritten to be unsized, with unique items, ensuring that you're always free to schedule images for processing, even if Cupdate's busy. It also makes sure that Cupdate won't schedule an image if it's already scheduled.

The rewrite also greatly improves the observability of the queue by making sure the cupdate_worker_available_burst gauge is always up-to-date. The change also adds a new gauge, cupdate_worker_queue_length which contains the current queue length.

See the example Grafana dashboard in docs/observability.

The change also means that the CUPDATE_PROCESSING_QUEUE_SIZE no longer has any effect.

Improvements and fixes

• Set correct cursor on image update button, graph navigation buttons
• Only show update toast on change detection
• Bubble errors returned by steps and jobs to the workflow, improving context in error logs
• Color negative counters on the dashboard green when they're zeroed
• Remove the word 'images' from the dashboard, reducing space and repetitiveness
• Add skeleton animations to the UI when its loading for a long time (should only be shown during times of bad network connectivity)
• Add missing cascade delete to image tags, fixing tags staying after their images have been removed
• Add backdrop blur to top bar
• Fix z-index of surface navigation
• Fix filter not being included on pagination
• Fix settings card not showing on Cupdate's image page
• Fix Cupdate processing Kubernetes references before a digest is resolved by the runtime
• Fix typo causing nil check to pass

Deprecations

• CUPDATE_PROCESSING_QUEUE_SIZE has been removed and no longer has any meaning.

Breaking changes

• Cupdate now stores links and vulnerabilities as blobs instead of rows. The tables images_links and images_vulnerabilities are removed when Cupdate starts. In their place, images_linksv2 and images_vulnerabilitiesv2 are created. if you're just using the UI or API, you won't notice the change after Cupdate has processed the images, again updating the links and vulnerabilities to the new tables.

Full Changelog: v0.16.0...v0.17.0

v0.17.0-beta.2

Thanks to everyone who created feature requests, bug reports and tested fixes. Your help is instrumental to continue to improve Cupdate!

The lists below refer to changes made since v0.17.0-beta.1.

Features

Ignore images using labels

Cupdate can now be configured to ignore images, containers, deployments, any Docker or Kubernetes resource that takes a label.

# deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: cupdate
    config.cupdate/ignore: "true"
# ...

# compose.yaml
services:
  cupdate:
    labels:
      - cupdate.config.ignore: "true"

docker run --rm -it --label "cupdate.config.ignore=true" alpine:3

See docs/config for more information.

Improvements and fixes

• Add backdrop blur to top bar
• Fix z-index of surface navigation

Full Changelog: v0.17.0-beta.1...v0.17.0-beta.2

v0.17.0-beta.1

Thanks to everyone who created feature requests, bug reports and tested fixes. Your help is instrumental to continue to improve Cupdate!

Features

Tracking workflow runs

Cupdate uses a workflow to process images, the workflow runs jobs and jobs consist of steps. If you're familiar with Github Actions, Cupdate's workflows work much the same.

Being a core part of Cupdate, observability of the workflows progress is key to debugging issues. In v0.17.0, Cupdate now tracks each and every workflow run, making their durations, statuses and errors available via the API and UI as soon as they've completed.

The graph shows each job of the latest workflow run. If you click one of the jobs, a summary of the steps run as part of the job is shown. If you're using tracing, a tracing id is shown, enabling you to continue correlating the run using your observability platform of choice. See docs/observability for more information.

If a step would fail, the error causing it to fail is shown.

Tracking the workflows also allows Cupdate to show a summary of failed images on the dashboard and via the API, for use with services like Grafana or Homepage. If you're already using Homepage, see the updated documentation in docs/cookbook.

This also allows us to tag failing images, making it easy to find them.

For now, only the latest workflow run is exposed through APIs. Workflow runs are by default kept for 48h. See docs/config for details on the new environment variables CUPDATE_WORKFLOW_CLEANUP_MAX_AGE and CUPDATE_WORKFLOW_CLEANUP_INTERVAL.

Tracking data changes

Whenever Cupdate processes an image and founds new data, that data is stored to an sqlite database. Starting in v0.17.0, Cupdate now internally tracks changes made to the data stored. Although not a front-facing feature, this enables improvements in existing features and lays the foundation for future improvements. If you're using the API, you can now get events whenever an image is processed or when there's a new version available. See the API docs for more details.

One improvement made to the existing feature set is that the web app can now tell what was updated and only prompt the user once data affecting the current page changes. This ensures that you won't get the toast every time Cupdate processes the image and updates basic fields such as when the image was last processed.

Worker queue rewrite

In previous Cupdate versions, the internal queue used for processing references had a fixed max size and provided bad observability. At times, it made it impossible to manually schedule an image for processing via the UI or APIs.

In v0.17.0, the worker queue implementation is rewritten to be unsized, with unique items, ensuring that you're always free to schedule images for processing, even if Cupdate's busy. It also makes sure that Cupdate won't schedule an image if it's already scheduled.

The rewrite also greatly improves the observability of the queue by making sure the cupdate_worker_available_burst gauge is always up-to-date. The change also adds a new gauge, cupdate_worker_queue_length which contains the current queue length.

See the example Grafana dashboard in docs/observability.

The change also means that the CUPDATE_PROCESSING_QUEUE_SIZE no longer has any effect.

Improvements and fixes

• Set correct cursor on image update button, graph navigation buttons
• Only show update toast on change detection
• Bubble errors returned by steps and jobs to the workflow, improving context in error logs
• Color negative counters on the dashboard green when they're zeroed
• Remove the word 'images' from the dashboard, reducing space and repetitiveness
• Add skeleton animations to the UI when its loading for a long time (should only be shown during times of bad network connectivity)
• Add missing cascade delete to image tags, fixing tags staying after their images have been removed
• Fix filter not being included on pagination
• Fix settings card not showing on Cupdate's image page
• Fix Cupdate processing Kubernetes references before a digest is resolved by the runtime

Deprecations

• CUPDATE_PROCESSING_QUEUE_SIZE has been removed and no longer has any meaning.

Breaking changes

• Cupdate now stores links and vulnerabilities as blobs instead of rows. The tables images_links and images_vulnerabilities are removed when Cupdate starts. In their place, images_linksv2 and images_vulnerabilitiesv2 are created. if you're just using the UI or API, you won't notice the change after Cupdate has processed the images, again updating the links and vulnerabilities to the new tables.

Full Changelog: v0.16.0...v0.17.0-beta.1

v0.16.0

Thanks to everyone who created feature requests, bug reports and tested fixes. Your help is instrumental to continue to improve Cupdate!

A special thanks to @thespad for helping out with discussing and testing some of the new features.

Features

• Add grid view layout option to UI
• Track digests used by the runtime
• Track changes made to tags like latest and v8
• Add support for custom registries like Harbor and Zot, as well as authentication towards registries like Docker Hub and GitHub Container Registry

Grid view

A compact grid layout has been added. The layout will fit as many images as possible on one row, adapting to your viewport's width. The chosen layout is stored in the browser's local storage so that it's available the next time you use the UI.

Custom registries

Cupdate now supports custom registries like Harbor and Zot as well as tracking private images from registries like Docker Hub and GitHub Container Registry using a token.

The implementation closely follows the format used by Docker and Kubernetes. To use the feature, you'll need to specify a config file via CUPDATE_REGISTRY_SECRETS:

{
  "auths": {
    "registry.example.com": {
      "username": "username",
      "password": "password"
    }
  }
}

{
  "auths": {
    "registry.example.com": {
      "auth": "c29tZSB0b2tlbg=="
    }
  }
}

{
  "HttpHeaders": {
    "x-some-special-auth": "letmein"
  }
}

For now, there's no support for insecure registries served without TLS.

Tracking digests

Tags like latest, v8 and v3.0.0 have one thing in common - they all refer to a manfiest containing information about a container image. The manifest is always referenced by its hash sum - its digest. Tags can be changed at any time to point to any digest, but digests will forever uniquely refer to the same manifest. In most cases, the manifests nor the digests themselves contain any information about what "version" of a software it actually contains.

This causes a couple of problems when trying to identify what "version" of a service is actually in use, so that Cupdate can find a newer version. In part, it means that your "latest" might not be the same as mine and that the "latest" today is not the same as yesterday.

Up until now, Cupdate has only supported "well-behaved" image tags that follow anything resembling a semantic version, assuming that they are never overwritten. This works great for tags like v1.2.3 and it works alright for tags like v1.2 and v1 as Cupdate can identify the version in use will be able to promote updates to v1.2.4 and beyond. It doesn't work at all for tags like latest. Tags like v1 and latest are typically overwritten, meaning Cupdate's information would become outdated over time.

With this new feature, Cupdate will try its best to identify the digest used by the underlying container runtime - again, uniquely refering to the specific manifest in use by its digest. When checking for updates, Cupdate will notice if the digest that a tag currently points to differs from the one used locally. This now enables Cupdate to promote you to update alpine:latest and mongo:v8 to a new version because the tags have been overwritten.

Please be aware that due to the limitations of the information made available by container registries and the image manifests themselves, it will be impossible in most cases to present a typical version like v3.0.0 if you use tags like latest, v3 and v3.3. There's simply no technical way of knowing what latest means in your context. As always, the best practice and what works best with Cupdate is to use tags like v3.3.0.

The full version is always shown in the UI when hovering over a version:

Improvements and fixes

• Add a favicon and logo override for Cupdate
• Improve intuitiveness of full-text search
• Improve support for references with IPs (v4 and v6) and pinned digests
• Update data shown on the page without a refresh (when shown the update toast)
• Make page indexes start at 1
• Minor UI improvements
  • Fix colors of layout button, use appropriate cursor
  • Remove divider in summary
• Keep query when going back from an image page
• Fix Bézier curves not showing in graph view in Firefox
• Set referrer policy on video elements
• Fix error with images missing quay vulnerability scans
• Fix tags being shown in the image name
• Fix GitHub packages data for packages owned by organizations

Deprecations

• CUPDATE_KUBERNETES_INCLUDE_OLD_REPLICAS environment variable now does nothing.

Breaking changes

• References will now contain the digest as well in almost all cases. This means that the reference and latestReference field returned by APIs can look like alpine:v3@sha256:56fa17d2a7e7f168a043a2712e63aed1f8543aeafdcee47c58dcffe38ed51099 as opposed to just alpine:v3. As the reference is the primary key of the database, it also means that all entries of the database will be removed updated over time. No user intervention is required.
• The Kubernetes dependency graph no longer includes templated resources like a job's pod template, which in turn can have a pod template which in turn depends on an image.
• Getting old replicas from Kubernetes is no longer supported.
• Page indexes now start at 1. Specifying page=0 to /api/v1/images will return 400 bad request.

Full Changelog: v0.15.0...v0.16.0

v0.16.0-rc.1

Thanks to everyone who created feature requests, bug reports and tested fixes. Your help is instrumental to continue to improve Cupdate!

The lists below refer to changes made since v0.16.0-beta.2.

Improvements and fixes

• Revert 400 returned when page index is out of bounds

Full Changelog: v0.16.0-beta.2...v0.16.0-rc.1

v0.16.0-beta.2

Thanks to everyone who created feature requests, bug reports and tested fixes. Your help is instrumental to continue to improve Cupdate!

The lists below refer to changes made since v0.16.0-beta.1.

Improvements and fixes

• Update data shown on the page without a refresh (when shown the update toast)
• Make page indexes start at 1
• Minor UI improvements
  • Fix colors of layout button, use appropriate cursor
  • Remove divider in summary
• Keep query when going back from an image page
• Fix Bézier curves not showing in graph view in Firefox
• Set referrer policy on video elements

Breaking changes

• Page indexes now start at 1. Specifying page=0 or a non-existing page to /api/v1/images will return 400 bad request.

Full Changelog: v0.16.0-beta.1...v0.16.0-beta.2

v0.16.0-beta.1

Thanks to everyone who created feature requests, bug reports and tested fixes. Your help is instrumental to continue to improve Cupdate!

A special thanks to @thespad for helping out with discussing and testing some of the new features.

Features

• Add grid view layout option to UI
• Track digests used by the runtime
• Track changes made to tags like latest and v8
• Add support for custom registries like Harbor and Zot, as well as authentication towards registries like Docker Hub and GitHub Container Registry

Grid view

A compact grid layout has been added. The layout will fit as many images as possible on one row, adapting to your viewport's width. The chosen layout is stored in the browser's local storage so that it's available the next time you use the UI.

Custom registries

Cupdate now supports custom registries like Harbor and Zot as well as tracking private images from registries like Docker Hub and GitHub Container Registry using a token.

The implementation closely follows the format used by Docker and Kubernetes. To use the feature, you'll need to specify a config file via CUPDATE_REGISTRY_SECRETS:

{
  "auths": {
    "registry.example.com": {
      "username": "username",
      "password": "password"
    }
  }
}

{
  "auths": {
    "registry.example.com": {
      "auth": "c29tZSB0b2tlbg=="
    }
  }
}

{
  "HttpHeaders": {
    "x-some-special-auth": "letmein"
  }
}

For now, there's no support for insecure registries served without TLS.

Tracking digests

Tags like latest, v8 and v3.0.0 have one thing in common - they all refer to a manfiest containing information about a container image. The manifest is always referenced by its hash sum - its digest. Tags can be changed at any time to point to any digest, but digests will forever uniquely refer to the same manifest. In most cases, the manifests nor the digests themselves contain any information about what "version" of a software it actually contains.

This causes a couple of problems when trying to identify what "version" of a service is actually in use, so that Cupdate can find a newer version. In part, it means that your "latest" might not be the same as mine and that the "latest" today is not the same as yesterday.

Up until now, Cupdate has only supported "well-behaved" image tags that follow anything resembling a semantic version, assuming that they are never overwritten. This works great for tags like v1.2.3 and it works alright for tags like v1.2 and v1 as Cupdate can identify the version in use will be able to promote updates to v1.2.4 and beyond. It doesn't work at all for tags like latest. Tags like v1 and latest are typically overwritten, meaning Cupdate's information would become outdated over time.

With this new feature, Cupdate will try its best to identify the digest used by the underlying container runtime - again, uniquely refering to the specific manifest in use by its digest. When checking for updates, Cupdate will notice if the digest that a tag currently points to differs from the one used locally. This now enables Cupdate to promote you to update alpine:latest and mongo:v8 to a new version because the tags have been overwritten.

Please be aware that due to the limitations of the information made available by container registries and the image manifests themselves, it will be impossible in most cases to present a typical version like v3.0.0 if you use tags like latest, v3 and v3.3. There's simply no technical way of knowing what latest means in your context. As always, the best practice and what works best with Cupdate is to use tags like v3.3.0.

The full version is always shown in the UI when hovering over a version:

Improvements and fixes

• Add a favicon and logo override for Cupdate
• Improve intuitiveness of full-text search
• Improve support for references with IPs (v4 and v6) and pinned digests
• Fix error with images missing quay vulnerability scans
• Fix tags being shown in the image name
• Fix GitHub packages data for packages owned by organizations

Deprecations

• CUPDATE_KUBERNETES_INCLUDE_OLD_REPLICAS environment variable now does nothing.

Breaking changes

• References will now contain the digest as well in almost all cases. This means that the reference and latestReference field returned by APIs can look like alpine:v3@sha256:56fa17d2a7e7f168a043a2712e63aed1f8543aeafdcee47c58dcffe38ed51099 as opposed to just alpine:v3. As the reference is the primary key of the database, it also means that all entries of the database will be removed updated over time. No user intervention is required.
• The Kubernetes dependency graph no longer includes templated resources like a job's pod template, which in turn can have a pod template which in turn depends on an image.
• Getting old replicas from Kubernetes is no longer supported.

Full Changelog: v0.15.0...v0.16.0-beta.1

v0.15.0

Thanks to everyone who created feature requests, bug reports and tested fixes. Your help is instrumental to continue to improve Cupdate!

Features

• Add support for multiple Docker hosts (comma-separated in CUPDATE_DOCKER_HOST)
• Add support for remote Docker hosts (tcp://<hostname>:<port> rather than just unix://)
• Add API endpoint for getting only the summary, useful for integration with homepage (/api/v1/summary)
• Improve Docker Swarm and Compose support for graphs, now showing tasks, services and namespaces / projects
• Expose Kubernetes namespaces, Docker Swarm namespaces and Docker Compose Projects as tags
• Add support for gcr.io, gke.gcr.io
• Make tags clickable in all places

Improvements and fixes

• Improve UX of manually scheduling updates
• Improve UI of images with digests rather than tags
• Sort tags lexically, non-prefixed tags first
• Make tag selection scrollable
• Improve handling of standalone docker v2 manifests
• Fix crash when running in Kubernetes and pods were created by "v1/Node" resources or CRDs like GitHub actions
• Fix tag selection not working on Windows
• Fix tag selection being too narrow on Windows, when scrollbar is visible
• Fix crash when a server-side HTTP error was instrumented
• Fix server-sent events not flushing when using telemetry handler

Breaking changes

None.

Full Changelog: v0.14.1...v0.15.0

v0.14.1

This is the first release of Cupdate meant for general use.

Please refer to the README for more information about Cupdate.