WARNING: THIS SITE IS A MIRROR OF GITHUB.COM / IT CANNOT LOGIN OR REGISTER ACCOUNTS / THE CONTENTS ARE PROVIDED AS-IS / THIS SITE ASSUMES NO RESPONSIBILITY FOR ANY DISPLAYED CONTENT OR LINKS / IF YOU FOUND SOMETHING MAY NOT GOOD FOR EVERYONE, CONTACT ADMIN AT ilovescratch@foxmail.com
Skip to content

Conversation

@shifqu
Copy link
Contributor

@shifqu shifqu commented Dec 4, 2025

The type NormalizedName is just a NewType of str so it's compatible with return-type str. This means casting it to a str is never required.

no changelog needed

Contributor checklist
  • Included tests for the changes. No-op refactor, tests not required
  • A change note is created in changelog.d/ (see changelog.d/README.md for instructions) or the PR text says "no changelog needed".
Maintainer checklist
  • If no changelog is needed, apply the bot:chronographer:skip label.
  • Assign the PR to an existing or new milestone for the target version (following Semantic Versioning).

The type NormalizedName is just a NewType of str so it's compatible
with return-type str. This means casting it to a str is never required.
@shifqu
Copy link
Contributor Author

shifqu commented Dec 4, 2025

Heya

Initially I did not understand why the mypy check was failing because canonicalize_name is effectively typed. I then started digging in the pre-commit config and noticed that we use pip==20.3.4. This version's vendored packaging does not have a typed definition indeed.

However, when upgrading the pre-commit config to use the first pip version where canonicalize_name is typed v21.2, I opened a can of worms and Found 102 errors in 11 files (checked 46 source files)

Then I decided to give it a go on the latest pip-supported (25.3) -> Found 118 errors in 11 files (checked 46 source files)

So my question now is: Why was mypy's pip version pinned at 20.3.4?

An easy fix for this PR is to just add a typing.cast(NormalizedName, ...) or even typing.cast(str, ...). But that seems to defeat the purpose of this PR.

Any suggestions/ideas?

To be complete, here is the log when using pre-commit with the newest pip
mypy.....................................................................Failed
- hook id: mypy
- duration: 17.37s
- exit code: 1

piptools/utils.py:32: error: Module "pip._internal.vcs" does not explicitly export attribute "is_url"  [attr-defined]
piptools/utils.py:73: error: Argument 1 to "key_from_req" has incompatible type "pip._vendor.packaging.requirements.Requirement | None"; expected "InstallRequirement | pip._vendor.packaging.requirements.Requirement | pip._internal.resolution.resolvelib.base.Requirement"  [arg-type]
piptools/utils.py:90: error: Argument 1 to "canonicalize_name" has incompatible type "str | None"; expected "str"  [arg-type]
piptools/utils.py:147: error: Item "None" of "Link | None" has no attribute "url"  [union-attr]
piptools/utils.py:154: error: Item "None" of "Requirement | None" has no attribute "name"  [union-attr]
piptools/utils.py:175: error: Item "None" of "Link | None" has no attribute "url"  [union-attr]
piptools/utils.py:178: error: Item "None" of "Link | None" has no attribute "is_file"  [union-attr]
piptools/utils.py:178: error: Item "None" of "Link | None" has no attribute "path"  [union-attr]
piptools/utils.py:179: error: Item "None" of "Link | None" has no attribute "url"  [union-attr]
piptools/utils.py:185: error: Item "None" of "Link | None" has no attribute "url_without_fragment"  [union-attr]
piptools/utils.py:189: error: Item "None" of "Link | None" has no attribute "subdirectory_fragment"  [union-attr]
piptools/utils.py:190: error: Item "None" of "Link | None" has no attribute "subdirectory_fragment"  [union-attr]
piptools/utils.py:192: error: Item "None" of "Link | None" has no attribute "has_hash"  [union-attr]
piptools/utils.py:193: error: Item "None" of "Link | None" has no attribute "hash_name"  [union-attr]
piptools/utils.py:193: error: Item "None" of "Link | None" has no attribute "hash"  [union-attr]
piptools/utils.py:211: error: Incompatible types in assignment (expression has type "list[Specifier]", variable has type "SpecifierSet")  [assignment]
piptools/utils.py:460: error: Returning Any from function declared to return "SpecifierSet"  [no-any-return]
piptools/utils.py:516: error: "InstallRequirement" has no attribute "use_pep517"  [attr-defined]
piptools/utils.py:517: error: "InstallRequirement" has no attribute "global_options"  [attr-defined]
piptools/utils.py:526: error: "InstallRequirement" has no attribute "install_options"  [attr-defined]
piptools/utils.py:534: error: Incompatible types in assignment (expression has type "Requirement | None", target has type "Collection[str] | bool | InstallRequirement | Link | Marker | None")  [assignment]
piptools/utils.py:536: error: Argument 2 to "map" has incompatible type "Collection[str] | bool | InstallRequirement | Link | Marker | None"; expected "Iterable[str]"  [arg-type]
piptools/utils.py:538: error: Item "Collection[str]" of "Collection[str] | Literal[True] | InstallRequirement | Link | Marker" has no attribute "extras"  [union-attr]
piptools/utils.py:538: error: Item "bool" of "Collection[str] | Literal[True] | InstallRequirement | Link | Marker" has no attribute "extras"  [union-attr]
piptools/utils.py:538: error: Item "Link" of "Collection[str] | Literal[True] | InstallRequirement | Link | Marker" has no attribute "extras"  [union-attr]
piptools/utils.py:538: error: Item "Marker" of "Collection[str] | Literal[True] | InstallRequirement | Link | Marker" has no attribute "extras"  [union-attr]
piptools/utils.py:538: error: Argument 1 to "set" has incompatible type "Collection[str] | bool | InstallRequirement | Link | Marker | None"; expected "Iterable[str | Any]"  [arg-type]
piptools/utils.py:540: error: Argument 1 to "InstallRequirement" has incompatible type "**dict[str, Collection[str] | bool | InstallRequirement | Link | Marker | None]"; expected "Requirement | None"  [arg-type]
piptools/utils.py:540: error: Argument 1 to "InstallRequirement" has incompatible type "**dict[str, Collection[str] | bool | InstallRequirement | Link | Marker | None]"; expected "str | InstallRequirement | None"  [arg-type]
piptools/utils.py:540: error: Argument 1 to "InstallRequirement" has incompatible type "**dict[str, Collection[str] | bool | InstallRequirement | Link | Marker | None]"; expected "bool"  [arg-type]
piptools/utils.py:540: error: Argument 1 to "InstallRequirement" has incompatible type "**dict[str, Collection[str] | bool | InstallRequirement | Link | Marker | None]"; expected "Link | None"  [arg-type]
piptools/utils.py:540: error: Argument 1 to "InstallRequirement" has incompatible type "**dict[str, Collection[str] | bool | InstallRequirement | Link | Marker | None]"; expected "Marker | None"  [arg-type]
piptools/utils.py:540: error: Argument 1 to "InstallRequirement" has incompatible type "**dict[str, Collection[str] | bool | InstallRequirement | Link | Marker | None]"; expected "dict[str, list[str]] | None"  [arg-type]
piptools/utils.py:540: error: Argument 1 to "InstallRequirement" has incompatible type "**dict[str, Collection[str] | bool | InstallRequirement | Link | Marker | None]"; expected "dict[str, str | list[str]] | None"  [arg-type]
piptools/utils.py:540: error: Argument 1 to "InstallRequirement" has incompatible type "**dict[str, Collection[str] | bool | InstallRequirement | Link | Marker | None]"; expected "Collection[str]"  [arg-type]
piptools/utils.py:545: error: Incompatible types in assignment (expression has type "Link | Collection[str] | bool | InstallRequirement | Marker | None", variable has type "Link | None")  [assignment]
piptools/build.py:311: error: Item "None" of "Requirement | None" has no attribute "name"  [union-attr]
piptools/_compat/pip_compat.py:48: error: Argument 1 to "_from_importlib" of "Distribution" has incompatible type "BaseDistribution"; expected "Distribution"  [arg-type]
piptools/_compat/pip_compat.py:73: error: Unused "type: ignore" comment  [unused-ignore]
piptools/_compat/pip_compat.py:115: error: Item "None" of "Link | None" has no attribute "url"  [union-attr]
piptools/_compat/pip_compat.py:120: error: Argument 1 to "_rewrite_comes_from_to_hardcoded_stdin_value" has incompatible type "str | InstallRequirement | None"; expected "str"  [arg-type]
piptools/_compat/pip_compat.py:204: error: Argument 1 to "WheelCache" has incompatible type "**dict[str, str | None]"; expected "str"  [arg-type]
piptools/_compat/pip_compat.py:209: error: Module "pip._internal.commands.freeze" has no attribute "DEV_PKGS"  [attr-defined]
piptools/_compat/pip_compat.py:215: error: Redundant cast to "set[str]"  [redundant-cast]
piptools/sync.py:79: error: Incompatible return value type (got "set[NormalizedName]", expected "set[str]")  [return-value]
piptools/sync.py:79: note: Perhaps you need a type annotation for "dependencies"? Suggestion: "set[str]"
piptools/sync.py:93: error: Argument 1 to "dependency_tree" has incompatible type "dict[NormalizedName, Distribution]"; expected "Mapping[str, Distribution]"  [arg-type]
piptools/sync.py:136: error: Item "None" of "Link | None" has no attribute "has_hash"  [union-attr]
piptools/sync.py:139: error: Argument 1 to "direct_url_from_link" has incompatible type "Link | None"; expected "Link"  [arg-type]
piptools/sync.py:139: error: Item "None" of "Requirement | None" has no attribute "name"  [union-attr]
piptools/sync.py:155: error: Incompatible types in assignment (expression has type "str", variable has type "NormalizedName")  [assignment]
piptools/sync.py:220: error: Value of type variable "SupportsRichComparisonT" of "sorted" cannot be "InstallRequirement"  [type-var]
piptools/sync.py:244: error: Value of type variable "SupportsRichComparisonT" of "sorted" cannot be "InstallRequirement"  [type-var]
piptools/sync.py:244: error: List item 6 has incompatible type "list[InstallRequirement]"; expected "str | bytes | PathLike[str] | PathLike[bytes]"  [list-item]
piptools/resolver.py:113: error: Argument 1 to "__ior__" of "set" has incompatible type "Collection[str]"; expected "AbstractSet[str]"  [arg-type]
piptools/resolver.py:146: error: "InstallRequirement" has no attribute "_source_ireqs"  [attr-defined]
piptools/resolver.py:388: error: Argument "key" to "sorted" has incompatible type "Callable[[InstallRequirement], str]"; expected "Callable[[RequirementSummary], SupportsDunderLT[Any] | SupportsDunderGT[Any]]"  [arg-type]
piptools/resolver.py:392: error: Argument "key" to "sorted" has incompatible type "Callable[[InstallRequirement], str]"; expected "Callable[[RequirementSummary], SupportsDunderLT[Any] | SupportsDunderGT[Any]]"  [arg-type]
piptools/resolver.py:440: error: "InstallRequirement" has no attribute "_source_ireqs"  [attr-defined]
piptools/resolver.py:611: error: Argument 1 to "make_requirement_preparer" of "RequirementCommand" has incompatible type "**dict[str, object]"; expected "TempDirectory"  [arg-type]
piptools/resolver.py:611: error: Argument 1 to "make_requirement_preparer" of "RequirementCommand" has incompatible type "**dict[str, object]"; expected "Values"  [arg-type]
piptools/resolver.py:611: error: Argument 1 to "make_requirement_preparer" of "RequirementCommand" has incompatible type "**dict[str, object]"; expected "BuildTracker"  [arg-type]
piptools/resolver.py:611: error: Argument 1 to "make_requirement_preparer" of "RequirementCommand" has incompatible type "**dict[str, object]"; expected "PipSession"  [arg-type]
piptools/resolver.py:611: error: Argument 1 to "make_requirement_preparer" of "RequirementCommand" has incompatible type "**dict[str, object]"; expected "PackageFinder"  [arg-type]
piptools/resolver.py:611: error: Argument 1 to "make_requirement_preparer" of "RequirementCommand" has incompatible type "**dict[str, object]"; expected "bool"  [arg-type]
piptools/resolver.py:611: error: Argument 1 to "make_requirement_preparer" of "RequirementCommand" has incompatible type "**dict[str, object]"; expected "str | None"  [arg-type]
piptools/resolver.py:611: error: Argument 1 to "make_requirement_preparer" of "RequirementCommand" has incompatible type "**dict[str, object]"; expected "int"  [arg-type]
piptools/resolver.py:646: error: Argument "resolver" to "_do_resolve" of "BacktrackingResolver" has incompatible type "BaseResolver"; expected "Resolver"  [arg-type]
piptools/resolver.py:652: error: "BaseResolver" has no attribute "_result"  [attr-defined]
piptools/resolver.py:720: error: Missing type parameters for generic type "Result"  [type-arg]
piptools/resolver.py:750: error: Unsupported left operand type for | ("Collection[str]")  [operator]
piptools/resolver.py:751: error: Item "None" of "Requirement | None" has no attribute "extras"  [union-attr]
piptools/resolver.py:757: error: Missing type parameters for generic type "Result"  [type-arg]
piptools/resolver.py:809: error: Item "None" of "Requirement | None" has no attribute "name"  [union-attr]
piptools/resolver.py:812: error: Item "None" of "Requirement | None" has no attribute "specifier"  [union-attr]
piptools/resolver.py:818: error: "InstallRequirement" has no attribute "_required_by"  [attr-defined]
piptools/resolver.py:825: error: "InstallRequirement" has no attribute "_source_ireqs"  [attr-defined]
piptools/resolver.py:828: error: "InstallRequirement" has no attribute "_source_ireqs"  [attr-defined]
piptools/repositories/pypi.py:32: error: Module "pip._vendor.requests" does not explicitly export attribute "RequestException"  [attr-defined]
piptools/repositories/pypi.py:32: error: Module "pip._vendor.requests" does not explicitly export attribute "Session"  [attr-defined]
piptools/repositories/pypi.py:68: error: Incompatible types in assignment (expression has type "Command", variable has type "InstallCommand")  [assignment]
piptools/repositories/pypi.py:140: error: Argument 1 to "find_all_candidates" of "PyPIRepository" has incompatible type "str | None"; expected "str"  [arg-type]
piptools/repositories/pypi.py:154: error: Argument 1 to "make_candidate_evaluator" of "PackageFinder" has incompatible type "str | None"; expected "str"  [arg-type]
piptools/repositories/pypi.py:160: error: Item "None" of "InstallationCandidate | None" has no attribute "name"  [union-attr]
piptools/repositories/pypi.py:161: error: Item "None" of "InstallationCandidate | None" has no attribute "version"  [union-attr]
piptools/repositories/pypi.py:185: error: Argument 1 to "make_requirement_preparer" of "RequirementCommand" has incompatible type "**dict[str, object]"; expected "TempDirectory"  [arg-type]
piptools/repositories/pypi.py:185: error: Argument 1 to "make_requirement_preparer" of "RequirementCommand" has incompatible type "**dict[str, object]"; expected "Values"  [arg-type]
piptools/repositories/pypi.py:185: error: Argument 1 to "make_requirement_preparer" of "RequirementCommand" has incompatible type "**dict[str, object]"; expected "BuildTracker"  [arg-type]
piptools/repositories/pypi.py:185: error: Argument 1 to "make_requirement_preparer" of "RequirementCommand" has incompatible type "**dict[str, object]"; expected "PipSession"  [arg-type]
piptools/repositories/pypi.py:185: error: Argument 1 to "make_requirement_preparer" of "RequirementCommand" has incompatible type "**dict[str, object]"; expected "PackageFinder"  [arg-type]
piptools/repositories/pypi.py:185: error: Argument 1 to "make_requirement_preparer" of "RequirementCommand" has incompatible type "**dict[str, object]"; expected "bool"  [arg-type]
piptools/repositories/pypi.py:185: error: Argument 1 to "make_requirement_preparer" of "RequirementCommand" has incompatible type "**dict[str, object]"; expected "str | None"  [arg-type]
piptools/repositories/pypi.py:185: error: Argument 1 to "make_requirement_preparer" of "RequirementCommand" has incompatible type "**dict[str, object]"; expected "int"  [arg-type]
piptools/repositories/pypi.py:205: error: "BaseResolver" has no attribute "_resolve_one"  [attr-defined]
piptools/repositories/pypi.py:209: error: "BaseResolver" has no attribute "_get_dist_for"  [attr-defined]
piptools/repositories/pypi.py:245: error: Incompatible types in assignment (expression has type "set[InstallationCandidate]", target has type "set[InstallRequirement]")  [assignment]
piptools/repositories/pypi.py:329: error: Argument 1 to "debug" of "LogContext" has incompatible type "str | None"; expected "str"  [arg-type]
piptools/repositories/pypi.py:391: error: Argument 1 to "find_all_candidates" of "PyPIRepository" has incompatible type "str | None"; expected "str"  [arg-type]
piptools/repositories/pypi.py:449: error: Cannot assign to a method  [method-assign]
piptools/repositories/pypi.py:449: error: Incompatible types in assignment (expression has type "Callable[[Wheel, list[Tag]], bool]", variable has type "Callable[[Wheel, Iterable[Tag]], bool]")  [assignment]
piptools/repositories/pypi.py:450: error: Cannot assign to a method  [method-assign]
piptools/repositories/pypi.py:464: error: Cannot assign to a method  [method-assign]
piptools/repositories/pypi.py:465: error: Cannot assign to a method  [method-assign]
piptools/repositories/pypi.py:493: error: Call to untyped function "get" in typed context  [no-untyped-call]
piptools/repositories/local.py:28: error: "InstallationCandidate" has no attribute "req"  [attr-defined]
piptools/repositories/local.py:29: error: Item "None" of "Requirement | None" has no attribute "specifier"  [union-attr]
piptools/repositories/local.py:30: error: "InstallationCandidate" has no attribute "req"  [attr-defined]
piptools/repositories/local.py:76: error: Signature of "find_best_match" incompatible with supertype "piptools.repositories.base.BaseRepository"  [override]
piptools/repositories/local.py:76: note:      Superclass:
piptools/repositories/local.py:76: note:          def find_best_match(self, ireq: InstallRequirement, prereleases: bool | None) -> InstallRequirement
piptools/repositories/local.py:76: note:      Subclass:
piptools/repositories/local.py:76: note:          def find_best_match(self, ireq: InstallRequirement, prereleases: bool | None = ...) -> InstallationCandidate
piptools/repositories/local.py:82: error: Argument 1 to "as_tuple" has incompatible type "InstallationCandidate"; expected "InstallRequirement"  [arg-type]
piptools/repositories/local.py:83: error: Incompatible return value type (got "InstallRequirement", expected "InstallationCandidate")  [return-value]
piptools/repositories/local.py:85: error: Incompatible return value type (got "InstallRequirement", expected "InstallationCandidate")  [return-value]
piptools/repositories/local.py:95: error: "InstallationCandidate" has no attribute "hash_options"  [attr-defined]
piptools/scripts/sync.py:150: error: Argument 2 to "sync" has incompatible type "set[str]"; expected "Iterable[InstallRequirement]"  [arg-type]
piptools/scripts/compile.py:335: error: Argument 1 to "LocalRequirementsRepository" has incompatible type "dict[str, InstallRequirement]"; expected "Mapping[str, InstallationCandidate]"  [arg-type]
tests/test_repository_pypi.py:10: error: Module "pip._vendor.requests" does not explicitly export attribute "HTTPError"  [attr-defined]
tests/test_repository_pypi.py:10: error: Module "pip._vendor.requests" does not explicitly export attribute "Session"  [attr-defined]
tests/conftest.py:120: error: Missing return statement  [empty-body]
tests/conftest.py:120: note: If the method is meant to be abstract, use @abc.abstractmethod
tests/conftest.py:124: error: Missing return statement  [empty-body]
tests/conftest.py:124: note: If the method is meant to be abstract, use @abc.abstractmethod
tests/conftest.py:128: error: Missing return statement  [empty-body]
tests/conftest.py:128: note: If the method is meant to be abstract, use @abc.abstractmethod
Found 118 errors in 11 files (checked 46 source files)

@shifqu
Copy link
Contributor Author

shifqu commented Dec 4, 2025

So my question now is: Why was mypy's pip version pinned at 20.3.4?

Ok, checking the commit history on the file actually revealed that this was introduced initially to silence mypy going to v21. 1efce5f

I am not sure what is the preferred way forward.

@sirosen
Copy link
Member

sirosen commented Dec 6, 2025

Thanks for getting this moving!

Our mypy situation right now is poor -- we're currently stuck on the version in pre-commit and, as you note, there's some work to be done to un-wedge it (we have some other issues about that, like #2200). However, I think we should be able to land this even without fixing the mypy setup yet.

I think the right approach is to wrap canonicalize_name in a shim module which presents it as the appropriate type. Something like...

# _packaging_compat.py
import typing as _t

if _t.TYPE_CHECKING:
    def canonicalize_name(name: str) -> str: ...
else:
    from pip._vendor.packaging.utils import canonicalize_name

That way, the rest of the codebase can grab canonicalize_name from the compat/shim module and not need to cast or use other typing-related tricks. I think that will also make it easier to "convince" the mypy version setup in pre-commit config to accept your work. 😄

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants