Update types using fresh dependencies

[atugushev]

Since 21.1 pip has py.typed file, hence many updates on type annotations.

Contributor checklist

• Provided the tests for the changes.
• Assure PR title is short, clear, and good to be included in the user-oriented changelog

Maintainer checklist

• Assure one of these labels is present: backwards incompatible, feature, enhancement, deprecation, bug, dependency, docs or skip-changelog as they determine changelog listing.
• Assign the PR to an existing or new milestone for the target version (following Semantic Versioning).

[atugushev]

@AndydeCleyre

FYI, this is pre-step towards addressing the following comments:

• Fix a bug where pip-sync would unexpectedly uninstall some packages #1919 (comment)
• Fix a bug where pip-sync would unexpectedly uninstall some packages #1919 (comment)

[webknjaz]

FWIW it might be useful to run mypy against several pip and Python versions (like min + max). Just the last one isn't going to cover a lot of stuff..

[atugushev]

FWIW it might be useful to run mypy against several pip and Python versions (like min + max). Just the last one isn't going to cover a lot of stuff..

@webknjaz I wonder if there are any library/apps in wild that do this?

[webknjaz]

Pip itself used to have py3 and py2 runs both in pre-commit, before 2020. I also do so in some places (like Cheroot). One possible strategy is to test every other version or min/max to cover most stuff while still not using too much resources.
I think that ansible-test runs mypy under every supported interpreter version (both for the ansible-core itself and the external collections).

[webknjaz]

pre-commit tip: if you add several check invocations, it's important to add aliases because several checks have the same id and there's no way to select a specific check without this.

[atugushev]

I also do so in some places (like Cheroot)

@webknjaz how do you handle type incompatibilities between different versions in the same code?

[atugushev]

FWIW it might be useful to run mypy against several pip and Python versions (like min + max). Just the last one isn't going to cover a lot of stuff..

@webknjaz opened a separate issue #1928 to address the suggestion.

[chrysle]

We haven't got these requirements pinned in pyproject.toml - Should we do this, or instead remove the boundary here?

[webknjaz]

@chrysle this is kinda a special case. https://pre-commit.ci is heavily cached and the cache virtually never expires. Locally we can use a pre-commit subcommand to clean it but no in the CI service. This kinda forces us to have something that would update in the cache key. But also, not having pins will cause difference in behavior on people's machines depending on when they first ran pre-commit and got the env cached.

[webknjaz]

how do you handle type incompatibilities between different versions in the same code?

MyPy's assert-based type narrowing can handle this. Though, there might be the need to have multiple MyPy invocations running against different versions of pip pinned.