Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

typing.TypeVar doesn't distinguish between (bound=None) and not providing a bound argument #129543

Closed
bzoracler opened this issue Feb 1, 2025 · 11 comments
Labels
stdlib Python modules in the Lib dir topic-typing type-bug An unexpected behavior, bug, or error

Comments

@bzoracler
Copy link

bzoracler commented Feb 1, 2025

Bug report

Bug description:

I was trying to reconstruct typing interfaces in a .pyi stub file by using runtime symbols. When trying to extract the value of the bound= argument to typing.TypeVar, I came across this result (which to be fair is documented in the signature):

>>> from typing import TypeVar
>>>
>>> T = TypeVar("T")
>>> print(T.__bound__ is None)
True

In static type checkers, an omitted bound= is equivalent to bound=builtins.object. The inconsistency with the runtime default value is problematic, because explicitly specifying bound=None also has a well-defined meaning to static type checkers (even if such a specification is rare):

(mypy Playground, pyright Playground)

from typing import TypeVar

T = TypeVar("T", bound=None)

def f(val: T, /) -> T:
    return val

f(None)  # OK
f(1)  # type-checker error

Not sure what the best course of action is here as any change to this will have backwards-compatibility concerns, but I think it's clear that static typing will not change an omitted bound= argument's interpretation to bound=None, as that would break most existing typed code.

CPython versions tested on:

3.9, 3.14

Operating systems tested on:

No response

@bzoracler bzoracler added the type-bug An unexpected behavior, bug, or error label Feb 1, 2025
@Viicos
Copy link
Contributor

Viicos commented Feb 1, 2025

typing.NoDefault was created for this purpose for the default argument. I'm wondering if the same could be applied to the bound?

@picnixz
Copy link
Member

picnixz commented Feb 1, 2025

AFAIK, None can be regarded as a valid sentinel but not as a valid argument to Typevar. The bound, when specified, should be a type, which None is not (even though it's allowed in unions for the sake of readability). So I'm not sure if it's not an error on the typechecker side instead (what do you mean by "also has a well-defined meaning to static type checkers (even if such a specification is rare)"?)

cc @JelleZijlstra

@picnixz picnixz added topic-typing stdlib Python modules in the Lib dir labels Feb 1, 2025
@bzoracler
Copy link
Author

bzoracler commented Feb 1, 2025

@picnixz I meant, it's rare for users to specify T = TypeVar("T", bound=None), because there is rarely any practical difference between this specification of T and just using None, so most of the time users would opt for None rather than T. As a possible way to resolve this issue, I expect that there would be minimal disruption if the runtime default for bound= was changed to builtins.object instead.

Addressing your comment about "The bound, when specified, should be a type", type-checkers are expected to follow this section of the typing specification, which allows None as the argument to bound=. Quoting a few relevant parts:

A type expression is any expression that validly expresses a type. Type expressions are always acceptable in annotations and also in various other places. Specifically, type expressions are used in the following locations:

  • ...
  • The bounds and constraints of a TypeVar ...
  • ...

...

The following grammar describes the allowed elements of type and annotation expressions:

...
type_expression       ::=  <Any>
                           | ...
                           | <None>
                           | ...
...

Many type expressions would not be instances of builtins.types at runtime (e.g. those resolved from string annotations, which are also allowed in TypeVar("T", bound="None")), and some type checkers (e.g. mypy) complain if you use types.NoneType instead of None as a type expression.

@sobolevn
Copy link
Member

sobolevn commented Feb 1, 2025

I don't think that it ever makes sense to create a TypeVar with bound=None. I initially thought that maybe int | None can fit bound=None, but no: https://mypy-play.net/?mypy=latest&python=3.12&gist=aa9303391066fae2e35eb2b73f14adf5

@bzoracler
Copy link
Author

@sobolevn there are probably some use-cases, such as in conditional definitions:

if sys.version_info >= (3, 10):
    TypeArg: TypeAlias = int
else:
    TypeArg: TypeAlias = None

T = TypeVar("T", bound=TypeArg)

class Data(Generic[T]):
    val: T

@picnixz
Copy link
Member

picnixz commented Feb 1, 2025

In this case, I think the None value is not meant to be considered as a standalone type expression. Also, if the bound is NoneType, then the only possible type is NoneType right? so there is no need for a type variable I think.

When I said "types" I didn't meant runtime types but rather valid type expressions, my bad.While None can be a valid standalone type expression I don't think we should support it as is for a bound of a type variable. I however wasn't aware of the exact specs so thank you

I would be -0.5 on changing the default unless there is a real use case that would benefit from it (as you observed, using a None bound can be simply replaced by directly using None)

@picnixz
Copy link
Member

picnixz commented Feb 1, 2025

there are probably some use-cases, such as in conditional definitions

I am afraid that those use cases are mostly artificial. Such code should rather change the genericity of the class instead or provide different class definitions if it's version-specific.

@bzoracler
Copy link
Author

I'm fine with leaving this with the runtime default as None, but at the current state, the runtime value is still ambiguous. I guess the resolution would be to delegate it to a typing specification update, to ban users from specifying None to bound=?

@AlexWaygood
Copy link
Member

I agree with @sobolevn's comment in #129543 (comment). If we could go back in time and make it so that the TypeVar constructor had never had bound=None as its default, I think we probably would. But given that we've had bound=None for so many years without any complaint (until now), and given that there's no realistic reason why you'd ever want to have a TypeVar bound to None (or any other singleton type) in real code, I don't see any reason to change this now.

@sobolevn there are probably some use-cases, such as in conditional definitions:

if sys.version_info >= (3, 10):
    TypeArg: TypeAlias = int
else:
    TypeArg: TypeAlias = None

T = TypeVar("T", bound=TypeArg)

@bzoracler, is this a problem you've encountered in real code, or is this just a hypothetical?


@picnixz, your argument in #129543 (comment) also doesn't make much sense to me in this case, however, I'm afraid :-) None is special-cased in the type system, but the special case is very consistently applied (except here). In type expressions, None always means "instance of NoneType", and the bound parameter of TypeVar always takes a type expression.

So I agree with @bzoracler that if we could go back in time and change it so that the default for the bound parameter had never been None, we would probably do so; it would make it consistent with other parts of the typing system. I just don't think consistency is worth it here, practically speaking, given how long it's been this way and how it doesn't seem to have actually caused any real-world problems. As PEP 8 tells us:

A Foolish Consistency is the Hobgoblin of Little Minds


I'm fine with leaving this with the runtime default as None, but at the current state, the runtime value is still ambiguous. I guess the resolution would be to delegate it to a typing specification update, to ban users from specifying None to bound=?

Sure, I suppose. But again, I'm not really sure how much the inconsistency really matters in this case :-)

@bzoracler
Copy link
Author

@AlexWaygood, that particular snippet was artificial, but my issue was not that someone would use such a definition, but rather how to interpret a TypeVar.__bound__ of None if I were to generate typing interfaces or schema for a generic class by inspecting the runtime values (I'm trying to solve this at the very moment). I only posted this issue here because the runtime doesn't match the behaviour of static type checkers and TypeVar, but this issue could well be shifted to the typing repository instead if changing the runtime value is unlikely to happen.

Changing the runtime default to builtins.object, or banning users from specifying bound=None in the typing specifications, are both very clear on what the interpretation is (although the interpretations would be wildly different) and provides a way forward on how to generate the interfaces and schema.

Thanks all for the input - it seems like the consensus is "don't expect an explicit None to be passed to bound=", so I'll close this.

@picnixz picnixz reopened this Feb 1, 2025
@picnixz picnixz closed this as not planned Won't fix, can't repro, duplicate, stale Feb 1, 2025
@JelleZijlstra
Copy link
Member

This is fixed on 3.14 for new-style TypeVars: the evaluate_bound attribute is non-None only if a bound, including None, is given.

>>> def f[T, U: None](): pass
... 
>>> T, U = f.__type_params__
>>> T.evaluate_bound
>>> U.evaluate_bound
<function U at 0x101c9fdd0>

It's not fixed for legacy (pre-PEP 695) TypeVars, but they're legacy for a reason.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
stdlib Python modules in the Lib dir topic-typing type-bug An unexpected behavior, bug, or error
Projects
None yet
Development

No branches or pull requests

6 participants