-
Notifications
You must be signed in to change notification settings - Fork 29
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
disallow fallback to Make, add devcontainers CI
- Loading branch information
Showing
9 changed files
with
268 additions
and
2 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,36 @@ | ||
# syntax=docker/dockerfile:1.5 | ||
|
||
ARG BASE | ||
ARG PYTHON_PACKAGE_MANAGER=conda | ||
|
||
FROM ${BASE} as pip-base | ||
|
||
ENV DEFAULT_VIRTUAL_ENV=rapids | ||
ENV RAPIDS_LIBUCX_PREFER_SYSTEM_LIBRARY=true | ||
|
||
RUN apt update -y \ | ||
&& DEBIAN_FRONTEND=noninteractive apt install -y \ | ||
libblas-dev liblapack-dev \ | ||
&& rm -rf /tmp/* /var/tmp/* /var/cache/apt/* /var/lib/apt/lists/*; | ||
|
||
FROM ${BASE} as conda-base | ||
|
||
ENV DEFAULT_CONDA_ENV=rapids | ||
|
||
FROM ${PYTHON_PACKAGE_MANAGER}-base | ||
|
||
ARG CUDA | ||
ENV CUDAARCHS="RAPIDS" | ||
ENV CUDA_VERSION="${CUDA_VERSION:-${CUDA}}" | ||
|
||
ARG PYTHON_PACKAGE_MANAGER | ||
ENV PYTHON_PACKAGE_MANAGER="${PYTHON_PACKAGE_MANAGER}" | ||
|
||
ENV PYTHONSAFEPATH="1" | ||
ENV PYTHONUNBUFFERED="1" | ||
ENV PYTHONDONTWRITEBYTECODE="1" | ||
|
||
ENV SCCACHE_REGION="us-east-2" | ||
ENV SCCACHE_BUCKET="rapids-sccache-devs" | ||
ENV AWS_ROLE_ARN="arn:aws:iam::279114543810:role/nv-gha-token-sccache-devs" | ||
ENV HISTFILE="/home/coder/.cache/._bash_history" |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,34 @@ | ||
# UCXX Development Containers | ||
|
||
This directory contains [devcontainer configurations](https://containers.dev/implementors/json_reference/) for using VSCode to [develop in a container](https://code.visualstudio.com/docs/devcontainers/containers) via the `Remote Containers` [extension](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-containers) or [GitHub Codespaces](https://github.com/codespaces). | ||
|
||
This container is a turnkey development environment for building and testing the UCXX C++ and Python libraries. | ||
|
||
## Table of Contents | ||
|
||
* [Prerequisites](#prerequisites) | ||
* [Host bind mounts](#host-bind-mounts) | ||
* [Launch a Dev Container](#launch-a-dev-container) | ||
## Prerequisites | ||
|
||
* [VSCode](https://code.visualstudio.com/download) | ||
* [VSCode Remote Containers extension](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-containers) | ||
|
||
## Host bind mounts | ||
|
||
By default, the following directories are bind-mounted into the devcontainer: | ||
|
||
* `${repo}:/home/coder/ucxx` | ||
* `${repo}/../.aws:/home/coder/.aws` | ||
* `${repo}/../.local:/home/coder/.local` | ||
* `${repo}/../.cache:/home/coder/.cache` | ||
* `${repo}/../.conda:/home/coder/.conda` | ||
* `${repo}/../.config:/home/coder/.config` | ||
|
||
This ensures caches, configurations, dependencies, and your commits are persisted on the host across container runs. | ||
|
||
## Launch a Dev Container | ||
|
||
To launch a devcontainer from VSCode, open the UCXX repo and select the "Reopen in Container" button in the bottom right:<br/><img src="https://user-images.githubusercontent.com/178183/221771999-97ab29d5-e718-4e5f-b32f-2cdd51bba25c.png"/> | ||
|
||
Alternatively, open the VSCode command palette (typically `cmd/ctrl + shift + P`) and run the "Rebuild and Reopen in Container" command. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,42 @@ | ||
{ | ||
"build": { | ||
"context": "${localWorkspaceFolder}/.devcontainer", | ||
"dockerfile": "${localWorkspaceFolder}/.devcontainer/Dockerfile", | ||
"args": { | ||
"CUDA": "11.8", | ||
"PYTHON_PACKAGE_MANAGER": "conda", | ||
"BASE": "rapidsai/devcontainers:25.04-cpp-cuda11.8-mambaforge-ubuntu22.04" | ||
} | ||
}, | ||
"runArgs": [ | ||
"--rm", | ||
"--name", | ||
"${localEnv:USER:anon}-rapids-${localWorkspaceFolderBasename}-25.04-cuda11.8-conda" | ||
], | ||
"hostRequirements": {"gpu": "optional"}, | ||
"features": { | ||
"ghcr.io/rapidsai/devcontainers/features/rapids-build-utils:25.4": {} | ||
}, | ||
"overrideFeatureInstallOrder": [ | ||
"ghcr.io/rapidsai/devcontainers/features/rapids-build-utils" | ||
], | ||
"initializeCommand": ["/bin/bash", "-c", "mkdir -m 0755 -p ${localWorkspaceFolder}/../.{aws,cache,config,conda/pkgs,conda/${localWorkspaceFolderBasename}-cuda11.8-envs}"], | ||
"postAttachCommand": ["/bin/bash", "-c", "if [ ${CODESPACES:-false} = 'true' ]; then . devcontainer-utils-post-attach-command; . rapids-post-attach-command; fi"], | ||
"workspaceFolder": "/home/coder", | ||
"workspaceMount": "source=${localWorkspaceFolder},target=/home/coder/ucxx,type=bind,consistency=consistent", | ||
"mounts": [ | ||
"source=${localWorkspaceFolder}/../.aws,target=/home/coder/.aws,type=bind,consistency=consistent", | ||
"source=${localWorkspaceFolder}/../.cache,target=/home/coder/.cache,type=bind,consistency=consistent", | ||
"source=${localWorkspaceFolder}/../.config,target=/home/coder/.config,type=bind,consistency=consistent", | ||
"source=${localWorkspaceFolder}/../.conda/pkgs,target=/home/coder/.conda/pkgs,type=bind,consistency=consistent", | ||
"source=${localWorkspaceFolder}/../.conda/${localWorkspaceFolderBasename}-cuda11.8-envs,target=/home/coder/.conda/envs,type=bind,consistency=consistent" | ||
], | ||
"customizations": { | ||
"vscode": { | ||
"extensions": [ | ||
"ms-python.flake8", | ||
"nvidia.nsight-vscode-edition" | ||
] | ||
} | ||
} | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,49 @@ | ||
{ | ||
"build": { | ||
"context": "${localWorkspaceFolder}/.devcontainer", | ||
"dockerfile": "${localWorkspaceFolder}/.devcontainer/Dockerfile", | ||
"args": { | ||
"CUDA": "11.8", | ||
"PYTHON_PACKAGE_MANAGER": "pip", | ||
"BASE": "rapidsai/devcontainers:25.04-cpp-cuda11.8-ucx1.18.0-openmpi-ubuntu22.04" | ||
} | ||
}, | ||
"runArgs": [ | ||
"--rm", | ||
"--name", | ||
"${localEnv:USER:anon}-rapids-${localWorkspaceFolderBasename}-25.04-cuda11.8-pip" | ||
], | ||
"hostRequirements": {"gpu": "optional"}, | ||
"features": { | ||
"ghcr.io/rapidsai/devcontainers/features/cuda:25.4": { | ||
"version": "11.8", | ||
"installcuBLAS": true, | ||
"installcuSOLVER": true, | ||
"installcuRAND": true, | ||
"installcuSPARSE": true | ||
}, | ||
"ghcr.io/rapidsai/devcontainers/features/rapids-build-utils:25.4": {} | ||
}, | ||
"overrideFeatureInstallOrder": [ | ||
"ghcr.io/rapidsai/devcontainers/features/cuda", | ||
"ghcr.io/rapidsai/devcontainers/features/rapids-build-utils" | ||
], | ||
"initializeCommand": ["/bin/bash", "-c", "mkdir -m 0755 -p ${localWorkspaceFolder}/../.{aws,cache,config/pip,local/share/${localWorkspaceFolderBasename}-cuda11.8-venvs}"], | ||
"postAttachCommand": ["/bin/bash", "-c", "if [ ${CODESPACES:-false} = 'true' ]; then . devcontainer-utils-post-attach-command; . rapids-post-attach-command; fi"], | ||
"workspaceFolder": "/home/coder", | ||
"workspaceMount": "source=${localWorkspaceFolder},target=/home/coder/ucxx,type=bind,consistency=consistent", | ||
"mounts": [ | ||
"source=${localWorkspaceFolder}/../.aws,target=/home/coder/.aws,type=bind,consistency=consistent", | ||
"source=${localWorkspaceFolder}/../.cache,target=/home/coder/.cache,type=bind,consistency=consistent", | ||
"source=${localWorkspaceFolder}/../.config,target=/home/coder/.config,type=bind,consistency=consistent", | ||
"source=${localWorkspaceFolder}/../.local/share/${localWorkspaceFolderBasename}-cuda11.8-venvs,target=/home/coder/.local/share/venvs,type=bind,consistency=consistent" | ||
], | ||
"customizations": { | ||
"vscode": { | ||
"extensions": [ | ||
"ms-python.flake8", | ||
"nvidia.nsight-vscode-edition" | ||
] | ||
} | ||
} | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,42 @@ | ||
{ | ||
"build": { | ||
"context": "${localWorkspaceFolder}/.devcontainer", | ||
"dockerfile": "${localWorkspaceFolder}/.devcontainer/Dockerfile", | ||
"args": { | ||
"CUDA": "12.8", | ||
"PYTHON_PACKAGE_MANAGER": "conda", | ||
"BASE": "rapidsai/devcontainers:25.04-cpp-mambaforge-ubuntu22.04" | ||
} | ||
}, | ||
"runArgs": [ | ||
"--rm", | ||
"--name", | ||
"${localEnv:USER:anon}-rapids-${localWorkspaceFolderBasename}-25.04-cuda12.8-conda" | ||
], | ||
"hostRequirements": {"gpu": "optional"}, | ||
"features": { | ||
"ghcr.io/rapidsai/devcontainers/features/rapids-build-utils:25.4": {} | ||
}, | ||
"overrideFeatureInstallOrder": [ | ||
"ghcr.io/rapidsai/devcontainers/features/rapids-build-utils" | ||
], | ||
"initializeCommand": ["/bin/bash", "-c", "mkdir -m 0755 -p ${localWorkspaceFolder}/../.{aws,cache,config,conda/pkgs,conda/${localWorkspaceFolderBasename}-cuda12.8-envs}"], | ||
"postAttachCommand": ["/bin/bash", "-c", "if [ ${CODESPACES:-false} = 'true' ]; then . devcontainer-utils-post-attach-command; . rapids-post-attach-command; fi"], | ||
"workspaceFolder": "/home/coder", | ||
"workspaceMount": "source=${localWorkspaceFolder},target=/home/coder/ucxx,type=bind,consistency=consistent", | ||
"mounts": [ | ||
"source=${localWorkspaceFolder}/../.aws,target=/home/coder/.aws,type=bind,consistency=consistent", | ||
"source=${localWorkspaceFolder}/../.cache,target=/home/coder/.cache,type=bind,consistency=consistent", | ||
"source=${localWorkspaceFolder}/../.config,target=/home/coder/.config,type=bind,consistency=consistent", | ||
"source=${localWorkspaceFolder}/../.conda/pkgs,target=/home/coder/.conda/pkgs,type=bind,consistency=consistent", | ||
"source=${localWorkspaceFolder}/../.conda/${localWorkspaceFolderBasename}-cuda12.8-envs,target=/home/coder/.conda/envs,type=bind,consistency=consistent" | ||
], | ||
"customizations": { | ||
"vscode": { | ||
"extensions": [ | ||
"ms-python.flake8", | ||
"nvidia.nsight-vscode-edition" | ||
] | ||
} | ||
} | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,49 @@ | ||
{ | ||
"build": { | ||
"context": "${localWorkspaceFolder}/.devcontainer", | ||
"dockerfile": "${localWorkspaceFolder}/.devcontainer/Dockerfile", | ||
"args": { | ||
"CUDA": "12.8", | ||
"PYTHON_PACKAGE_MANAGER": "pip", | ||
"BASE": "rapidsai/devcontainers:25.04-cpp-cuda12.8-ucx1.18.0-openmpi-ubuntu22.04" | ||
} | ||
}, | ||
"runArgs": [ | ||
"--rm", | ||
"--name", | ||
"${localEnv:USER:anon}-rapids-${localWorkspaceFolderBasename}-25.04-cuda12.8-pip" | ||
], | ||
"hostRequirements": {"gpu": "optional"}, | ||
"features": { | ||
"ghcr.io/rapidsai/devcontainers/features/cuda:25.4": { | ||
"version": "12.8", | ||
"installcuBLAS": true, | ||
"installcuSOLVER": true, | ||
"installcuRAND": true, | ||
"installcuSPARSE": true | ||
}, | ||
"ghcr.io/rapidsai/devcontainers/features/rapids-build-utils:25.4": {} | ||
}, | ||
"overrideFeatureInstallOrder": [ | ||
"ghcr.io/rapidsai/devcontainers/features/cuda", | ||
"ghcr.io/rapidsai/devcontainers/features/rapids-build-utils" | ||
], | ||
"initializeCommand": ["/bin/bash", "-c", "mkdir -m 0755 -p ${localWorkspaceFolder}/../.{aws,cache,config/pip,local/share/${localWorkspaceFolderBasename}-cuda12.8-venvs}"], | ||
"postAttachCommand": ["/bin/bash", "-c", "if [ ${CODESPACES:-false} = 'true' ]; then . devcontainer-utils-post-attach-command; . rapids-post-attach-command; fi"], | ||
"workspaceFolder": "/home/coder", | ||
"workspaceMount": "source=${localWorkspaceFolder},target=/home/coder/ucxx,type=bind,consistency=consistent", | ||
"mounts": [ | ||
"source=${localWorkspaceFolder}/../.aws,target=/home/coder/.aws,type=bind,consistency=consistent", | ||
"source=${localWorkspaceFolder}/../.cache,target=/home/coder/.cache,type=bind,consistency=consistent", | ||
"source=${localWorkspaceFolder}/../.config,target=/home/coder/.config,type=bind,consistency=consistent", | ||
"source=${localWorkspaceFolder}/../.local/share/${localWorkspaceFolderBasename}-cuda12.8-venvs,target=/home/coder/.local/share/venvs,type=bind,consistency=consistent" | ||
], | ||
"customizations": { | ||
"vscode": { | ||
"extensions": [ | ||
"ms-python.flake8", | ||
"nvidia.nsight-vscode-edition" | ||
] | ||
} | ||
} | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -16,6 +16,7 @@ jobs: | |
- changed-files | ||
- checks | ||
- conda-cpp-build | ||
- devcontainer | ||
- docs-build | ||
- conda-cpp-tests | ||
- conda-python-tests | ||
|
@@ -49,12 +50,14 @@ jobs: | |
files_yaml: | | ||
test_cpp: | ||
- '**' | ||
- '!.devcontainer/**' | ||
- '!.pre-commit-config.yaml' | ||
- '!README.md' | ||
- '!docs/**' | ||
- '!python/**' | ||
test_python: | ||
- '**' | ||
- '!.devcontainer/**' | ||
- '!.pre-commit-config.yaml' | ||
- '!README.md' | ||
- '!docs/**' | ||
|
@@ -145,3 +148,14 @@ jobs: | |
build_type: pull-request | ||
container-options: "--cap-add CAP_SYS_PTRACE --shm-size=8g --ulimit=nofile=1000000:1000000" | ||
script: ci/test_wheel_distributed_ucxx.sh | ||
devcontainer: | ||
secrets: inherit | ||
needs: checks | ||
uses: rapidsai/shared-workflows/.github/workflows/[email protected] | ||
with: | ||
arch: '["amd64"]' | ||
cuda: '["12.8"]' | ||
build_command: | | ||
sccache -z; | ||
build-all --verbose; | ||
sccache -s; |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters