Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow all list based config to be assigned dictionaries #845

Merged
merged 4 commits into from
Jul 29, 2024
Merged

Allow all list based config to be assigned dictionaries #845

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
530a00a
Allow more pod properties to be defined as dictionaries
sunu Jul 26, 2024
1e25f43
Minor improvements to help text
sunu Jul 27, 2024
322133f
Use proper hyperlink syntax for links to spec
sunu Jul 27, 2024
7c080a6
Replace type comparison with isinstance
sunu Jul 27, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
227 changes: 182 additions & 45 deletions kubespawner/spawner.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -781,7 +781,7 @@ def _deprecated_changed(self, change):

@validate('image_pull_secrets')
def _validate_image_pull_secrets(self, proposal):
if type(proposal['value']) == str:
if isinstance(proposal['value'], str):
warnings.warn(
"""Passing KubeSpawner.image_pull_secrets string values is
deprecated since KubeSpawner 0.14.0. The recommended
Expand Down Expand Up @@ -1262,14 +1262,21 @@ def _validate_image_pull_secrets(self, proposal):
""",
)

init_containers = List(
init_containers = Union(
trait_types=[
List(),
Dict(),
],
config=True,
help="""
List of initialization containers belonging to the pod.
List or dictionary of initialization containers belonging to the pod.

This list will be directly added under `initContainers` in the kubernetes pod spec,
so you should use the same structure. Each item in the dict must a field
of the `V1Container specification <https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.28/#container-v1-core>`__
If provided as a list, this list will be directly added under `initContainers` in the kubernetes pod spec.
If provided as a dictionary, the items will be sorted lexicographically by the dictionary keys and
then the sorted values will be added to the `initContainers` key.

Each item (whether in the list or dictionary values) must be a dictionary which follows the spec at
`V1Container specification <https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.28/#container-v1-core>`__

One usage is disabling access to metadata service from single-user
notebook server with configuration below::
Expand All @@ -1285,11 +1292,25 @@ def _validate_image_pull_secrets(self, proposal):
}
}]

Or as a dictionary::

c.KubeSpawner.init_containers = {
"01-iptables": {
"name": "init-iptables",
"image": "<image with iptables installed>",
"command": ["iptables", "-A", "OUTPUT", "-p", "tcp", "--dport", "80", "-d", "169.254.169.254", "-j", "DROP"],
"securityContext": {
"capabilities": {
"add": ["NET_ADMIN"]
}
}
}
}

See `the Kubernetes documentation <https://kubernetes.io/docs/concepts/workloads/pods/init-containers/>`__
for more info on what init containers are and why you might want to use them!

To user this feature, Kubernetes version must greater than 1.6.
To use this feature, Kubernetes version must be greater than 1.6.
""",
)

Expand Down Expand Up @@ -1342,15 +1363,22 @@ def _validate_image_pull_secrets(self, proposal):
""",
)

extra_containers = List(
extra_containers = Union(
trait_types=[
List(),
Dict(),
],
config=True,
help="""
List of containers belonging to the pod which besides to the container generated for notebook server.
List or dictionary of containers belonging to the pod in addition to
the container generated for the notebook server.

This list will be directly appended under `containers` in the kubernetes pod spec,
so you should use the same structure. Each item in the list is container configuration
which follows spec at https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.28/#container-v1-core
If provided as a list, this list will be directly appended under `containers` in the kubernetes pod spec.
If provided as a dictionary, the items will be sorted lexicographically by the dictionary keys and
then the sorted values will be appended to the `containers` key.

Each item (whether in the list or dictionary values) is container configuration
which follows the spec at https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.28/#container-v1-core

One usage is setting crontab in a container to clean sensitive data with configuration below::

Expand All @@ -1360,6 +1388,16 @@ def _validate_image_pull_secrets(self, proposal):
"command": ["/usr/local/bin/supercronic", "/etc/crontab"]
}]

Or as a dictionary::

c.KubeSpawner.extra_containers = {
"01-crontab": {
"name": "crontab",
"image": "supercronic",
"command": ["/usr/local/bin/supercronic", "/etc/crontab"]
}
}

`{username}`, `{userid}`, `{servername}`, `{hubnamespace}`,
`{unescaped_username}`, and `{unescaped_servername}` will be expanded if
found within strings of this configuration. The username and servername
Expand All @@ -1379,19 +1417,27 @@ def _validate_image_pull_secrets(self, proposal):
""",
)

tolerations = List(
tolerations = Union(
trait_types=[
List(),
Dict(),
],
config=True,
help="""
List of tolerations that are to be assigned to the pod in order to be able to schedule the pod
List or dictionary of tolerations that are to be assigned to the pod in order to be able to schedule the pod
on a node with the corresponding taints. See the official Kubernetes documentation for additional details
https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/

Pass this field an array of "Toleration" objects
* https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.28/#toleration-v1-core
If provided as a list, each item should be a "Toleration" object.
If provided as a dictionary, the keys can be any descriptive name and the values should be "Toleration" objects.
The items will be sorted lexicographically by the dictionary keys and the sorted values will be added to the pod spec.

Example::
Each "Toleration" object should follow the specification at:
https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.28/#toleration-v1-core

[
Example as a list::

c.KubeSpawner.tolerations = [
{
'key': 'key',
'operator': 'Equal',
Expand All @@ -1405,83 +1451,160 @@ def _validate_image_pull_secrets(self, proposal):
}
]

Example as a dictionary::

c.KubeSpawner.tolerations = {
"01-gpu-toleration": {
'key': 'gpu',
'operator': 'Equal',
'value': 'true',
'effect': 'NoSchedule'
},
"02-general-toleration": {
'key': 'key',
'operator': 'Exists',
'effect': 'NoSchedule'
}
}

""",
)

node_affinity_preferred = List(
node_affinity_preferred = Union(
trait_types=[
List(),
Dict(),
],
config=True,
help="""
List or dictionary of preferred node affinities.

Affinities describe where pods prefer or require to be scheduled, they
may prefer or require a node to have a certain label or be in proximity
/ remoteness to another pod. To learn more visit
https://kubernetes.io/docs/concepts/configuration/assign-pod-node/

Pass this field an array of "PreferredSchedulingTerm" objects.*
* https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.28/#preferredschedulingterm-v1-core
If provided as a list, each item should be a "PreferredSchedulingTerm" object.
If provided as a dictionary, the keys can be any descriptive name and the values should be "PreferredSchedulingTerm" objects.
The items will be sorted lexicographically by the dictionary keys and the sorted values will be added to the pod spec.

Each item should follow the `"PreferredSchedulingTerm" specification
<https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.28/#preferredschedulingterm-v1-core>`__.
""",
)
node_affinity_required = List(

node_affinity_required = Union(
trait_types=[
List(),
Dict(),
],
config=True,
help="""
List or dictionary of required node affinities.

Affinities describe where pods prefer or require to be scheduled, they
may prefer or require a node to have a certain label or be in proximity
/ remoteness to another pod. To learn more visit
https://kubernetes.io/docs/concepts/configuration/assign-pod-node/

Pass this field an array of "NodeSelectorTerm" objects.*
* https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.28/#nodeselectorterm-v1-core
If provided as a list, each item should be a "NodeSelectorTerm" object.
If provided as a dictionary, the keys can be any descriptive name and the values should be "NodeSelectorTerm" objects.
The items will be sorted lexicographically by the dictionary keys and the sorted values will be added to the pod spec.

Each item should follow the `"NodeSelectorTerm" specification
<https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.28/#nodeselectorterm-v1-core>`__.
""",
)
pod_affinity_preferred = List(

pod_affinity_preferred = Union(
trait_types=[
List(),
Dict(),
],
config=True,
help="""
List or dictionary of preferred pod affinities.

Affinities describe where pods prefer or require to be scheduled, they
may prefer or require a node to have a certain label or be in proximity
/ remoteness to another pod. To learn more visit
https://kubernetes.io/docs/concepts/configuration/assign-pod-node/

Pass this field an array of "WeightedPodAffinityTerm" objects.*
* https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.28/#weightedpodaffinityterm-v1-core
If provided as a list, each item should be a "WeightedPodAffinityTerm" object.
If provided as a dictionary, the keys can be any descriptive name and the values should be "WeightedPodAffinityTerm" objects.
The items will be sorted lexicographically by the dictionary keys and the sorted values will be added to the pod spec.

Each item should follow the `"WeightedPodAffinityTerm" specification
<https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.28/#weightedpodaffinityterm-v1-core>`__.
""",
)
pod_affinity_required = List(

pod_affinity_required = Union(
trait_types=[
List(),
Dict(),
],
config=True,
help="""
List or dictionary of required pod affinities.

Affinities describe where pods prefer or require to be scheduled, they
may prefer or require a node to have a certain label or be in proximity
/ remoteness to another pod. To learn more visit
https://kubernetes.io/docs/concepts/configuration/assign-pod-node/

Pass this field an array of "PodAffinityTerm" objects.*
* https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.28/#podaffinityterm-v1-core
If provided as a list, each item should be a "PodAffinityTerm" object.
If provided as a dictionary, the keys can be any descriptive name and the values should be "PodAffinityTerm" objects.
The items will be sorted lexicographically by the dictionary keys and the sorted values will be added to the pod spec.

Each item should follow the `"PodAffinityTerm" specification
<https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.28/#podaffinityterm-v1-core>`__.
""",
)
pod_anti_affinity_preferred = List(

pod_anti_affinity_preferred = Union(
trait_types=[
List(),
Dict(),
],
config=True,
help="""
List or dictionary of preferred pod anti-affinities.

Affinities describe where pods prefer or require to be scheduled, they
may prefer or require a node to have a certain label or be in proximity
/ remoteness to another pod. To learn more visit
https://kubernetes.io/docs/concepts/configuration/assign-pod-node/

Pass this field an array of "WeightedPodAffinityTerm" objects.*
* https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.28/#weightedpodaffinityterm-v1-core
If provided as a list, each item should be a "WeightedPodAffinityTerm" object.
If provided as a dictionary, the keys can be any descriptive name and the values should be "WeightedPodAffinityTerm" objects.
The items will be sorted lexicographically by the dictionary keys and the sorted values will be added to the pod spec.

Each item should follow the `"WeightedPodAffinityTerm" specification
<https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.28/#weightedpodaffinityterm-v1-core>`__.
""",
)
pod_anti_affinity_required = List(

pod_anti_affinity_required = Union(
trait_types=[
List(),
Dict(),
],
config=True,
help="""
List or dictionary of required pod anti-affinities.

Affinities describe where pods prefer or require to be scheduled, they
may prefer or require a node to have a certain label or be in proximity
/ remoteness to another pod. To learn more visit
https://kubernetes.io/docs/concepts/configuration/assign-pod-node/

Pass this field an array of "PodAffinityTerm" objects.*
* https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.28/#podaffinityterm-v1-core
If provided as a list, each item should be a "PodAffinityTerm" object.
If provided as a dictionary, the keys can be any descriptive name and the values should be "PodAffinityTerm" objects.
The items will be sorted lexicographically by the dictionary keys and the sorted values will be added to the pod spec.

Each item should follow the `"PodAffinityTerm" specification
<https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.28/#podaffinityterm-v1-core>`__.
""",
)

Expand Down Expand Up @@ -2077,20 +2200,34 @@ async def get_pod_manifest(self):
extra_resource_limits=self.extra_resource_limits,
extra_resource_guarantees=self.extra_resource_guarantees,
lifecycle_hooks=self.lifecycle_hooks,
init_containers=self._expand_all(self.init_containers),
init_containers=self._expand_all(
self._sorted_dict_values(self.init_containers)
),
service_account=self._expand_all(self.service_account),
automount_service_account_token=self.automount_service_account_token,
extra_container_config=self.extra_container_config,
extra_pod_config=self._expand_all(self.extra_pod_config),
extra_containers=self._expand_all(self.extra_containers),
extra_containers=self._expand_all(
self._sorted_dict_values(self.extra_containers)
),
scheduler_name=self.scheduler_name,
tolerations=self.tolerations,
node_affinity_preferred=self.node_affinity_preferred,
node_affinity_required=self.node_affinity_required,
pod_affinity_preferred=self.pod_affinity_preferred,
pod_affinity_required=self.pod_affinity_required,
pod_anti_affinity_preferred=self.pod_anti_affinity_preferred,
pod_anti_affinity_required=self.pod_anti_affinity_required,
tolerations=self._sorted_dict_values(self.tolerations),
node_affinity_preferred=self._sorted_dict_values(
self.node_affinity_preferred
),
node_affinity_required=self._sorted_dict_values(
self.node_affinity_required
),
pod_affinity_preferred=self._sorted_dict_values(
self.pod_affinity_preferred
),
pod_affinity_required=self._sorted_dict_values(self.pod_affinity_required),
pod_anti_affinity_preferred=self._sorted_dict_values(
self.pod_anti_affinity_preferred
),
pod_anti_affinity_required=self._sorted_dict_values(
self.pod_anti_affinity_required
),
priority_class_name=self.priority_class_name,
ssl_secret_name=self.secret_name if self.internal_ssl else None,
ssl_secret_mount_path=self.secret_mount_path,
Expand Down
Loading