1

I've recently started using mypy, and have run into some weird problems that i cannot for the life of me seem to figure out.

I'm using mypy 0.950, django-stubs 1.11.0, django 4.0.5 and python 3.10.2.

Running mypy through the command line returns this:

project/suppliers/models.py:6: error: Name "Optional" is not defined
project/suppliers/models.py:6: note: Did you forget to import it from "typing"? (Suggestion: "from typing import Optional")
project/users/models.py:6: error: Name "Optional" is not defined
project/users/models.py:6: note: Did you forget to import it from "typing"? (Suggestion: "from typing import Optional")
project/products/models.py:6: error: Name "Optional" is not defined
project/products/models.py:6: note: Did you forget to import it from "typing"? (Suggestion: "from typing import Optional")(Suggestion: "from typing import Optional")

However, line 6 in project/suppliers/models.py is completely empty:

from django.contrib.sites.managers import CurrentSiteManager
from django.contrib.sites.models import Site
from django.db import models
from django.utils.text import slugify
from django.utils.translation import gettext_lazy as _

from django_countries.fields import CountryField

from project.core.models import BaseImageModel, BaseModel
from project.suppliers.managers import SupplierQuerySet

_SupplierManager = models.Manager.from_queryset(SupplierQuerySet)


class Supplier(BaseModel, BaseImageModel):
    ...

Line 6 in project/users/models.py is a django import from django.contrib.contenttypes.models import ContentType:

import random
from typing import Any

from django.conf import settings
from django.contrib.auth.models import AbstractBaseUser, PermissionsMixin
from django.contrib.contenttypes.models import ContentType
from django.contrib.sites.managers import CurrentSiteManager
from django.contrib.sites.models import Site
from django.core import signing
from django.core.mail import send_mail
from django.db import models
from django.forms import ValidationError
from django.http import HttpRequest
from django.template.loader import render_to_string
from django.utils import timezone
from django.utils.encoding import force_bytes, force_str
from django.utils.http import (
    urlsafe_base64_decode as uid_decoder,
    urlsafe_base64_encode as uid_encoder,
)
from django.utils.translation import gettext_lazy as _

import phonenumbers

from project.users.enums import AvatarColors
from project.users.managers import UserQuerySet
from project.users.schemas.records import (
    UserAuditLogsRecord,
    UserNotesRecord,
    UserProfileRecord,
)

_UserManager = models.Manager.from_queryset(UserQuerySet)


class User(AbstractBaseUser, PermissionsMixin):

And line 6 in project/products/models.py is yet another django import from django.utils.text import slugify:

from decimal import Decimal

from django.contrib.sites.managers import CurrentSiteManager
from django.contrib.sites.models import Site
from django.db import models
from django.utils.text import slugify
from django.utils.translation import gettext_lazy as _

from imagekit.models import ImageSpecField
from imagekit.processors import ResizeToFill
from mptt.models import TreeManyToManyField
...

My mypy config is as follows:

[tool.mypy]
plugins = ["mypy_django_plugin.main", "pydantic.mypy"]
follow_imports = "normal"

ignore_missing_imports = true

disallow_untyped_calls = true
disallow_untyped_defs = true
disallow_incomplete_defs = true

warn_unused_configs = true
warn_redundant_casts = true
warn_unused_ignores = true
warn_return_any = true
warn_unreachable = true

no_implicit_optional = true
no_implicit_reexport = true
check_untyped_defs = true
strict_equality = true

[tool.django-stubs]
django_settings_module = "project.settings"

[tool.pydantic-mypy]
init_forbid_extra = true
init_typed = true
warn_required_dynamic_aliases = true
warn_untyped_fields = true

# Admin files uses some patterns that are not easily typed
[[tool.mypy.overrides]]
module = "project.*.admin"
ignore_errors = true

[[tool.mypy.overrides]]
module = "project.*.tests.*"
ignore_errors = true

[[tool.mypy.overrides]]
module = "project.*.migrations.*"
ignore_errors = "true"

[[tool.mypy.overrides]]
module = "project.*.management.*"
disallow_untyped_defs = false

I've tried googling around, but can not seem to find anyone else that has experienced this. Anything obvious that I've missed, or does it look like a bug? Seems to me at least that this would affect quite a lot if it was something wrong with mypy/django stubs.

DanielK
  • 257
  • 2
  • 15
  • Could you try to invoke `mypy` with option `--config-file=` (without value)? It should then ignore config file, so we can determine whether it is config problem or your code. – STerliakov Jun 11 '22 at 16:46
  • 1
    It is suspicious that line 6 is always referred to. – STerliakov Jun 11 '22 at 16:47
  • The errors did not surface when ran without a config file, i think, however, i've hunted it down to be the CurrentSiteManager. It's present in all files and the error goes away when it's commented out. Must be something weird with the django-stubs package. This would also be supported by the fact that the errors goes away as the stubs package is not loaded when it's run without a config – DanielK Jun 11 '22 at 17:02
  • Much more funny... Please provide MRE (remove all unrelated code) with `CurrentSiteManager` - I'll try to investigate this and update `django-stubs` if needed. – STerliakov Jun 11 '22 at 17:03
  • `Optional` is really used in `django.contrib.sites.managers`, but is [imported](https://github.com/typeddjango/django-stubs/blob/master/django-stubs/contrib/sites/managers.pyi) – STerliakov Jun 11 '22 at 17:05
  • Could you try using `master` branch of django-stubs (`pip install git+https://github.com/typeddjango/django-stubs`) and latest `mypy`? (yes, it is not compatible with `django-stubs`, but should've been unpinned long ago in requirements, because `mypy` releases are more frequent than `django-stubs`, it should work ok) – STerliakov Jun 11 '22 at 17:11
  • Here is MRE: https://github.com/danielkjellid/django-stubs-current-site-manager-mre - I'll try with with using the master branch of django-stubs. I'll update you! – DanielK Jun 11 '22 at 17:28
  • An update: I experience the same issue with django-stubs 1.11.1 and mypy 0.961. – DanielK Jun 11 '22 at 20:01
  • Yeah, I see the bug origin, looking for workaround. – STerliakov Jun 11 '22 at 20:06

1 Answers1

7

Fast 1st-party fix

You can add from typing import Optional to files that use CurrentSiteManager. It will resolve this problem (yes, # noqa: F401 is your friend).

One-time fix for package

As a quick workaround, you can modify django-stubs/contrib/sites/managers.pyi to have the following content:

from typing import Optional, TypeVar
from django.db import models

_T = TypeVar('_T', bound=models.Model)

class CurrentSiteManager(models.Manager[_T]):
    def __init__(self, field_name: Optional[str] = ...) -> None: ...

Problem root

It is not a real solution and is a bug in mypy_django_plugin. I'm looking for a solution now. The problem is that helpers.copy_method_to_another_class (and consequently transformers.models.AddManagers.create_new_model_parametrized_manager) is using context of model class definition. I don't understand now how to get and pass another context properly without symmetrical issue, and merging is definitely not an option. I'll update this on success and raise a PR to maintainer. It would be great if you file an issue to django-stubs (and attach link to this question) so that I have less to explain (or somebody else could help if I fail).

Better fix #1

here we can add original_module_name=base_manager_info.module_name call argument. It will allow resolving everything inherited from first MRO parent. However, it seems like this should fail for longer inheritance chains. My bad, we're actually iterating here over methods of last ancestor only, so this seems to be the final solution.

(I'm sorry for using this as a changelog, but once started...)

This bug was fixed in this my PR.

STerliakov
  • 4,983
  • 3
  • 15
  • 37
  • I saw that the PR was merged now. Thank you for taking the time out of your day to fix it. I'll set your reply as the solution, since it both fixed the immediate issue, although a bit hacky, and it won't be a problem when the next version of django-stubs is released. Again, thank you! – DanielK Jun 15 '22 at 07:51