We are using sqlmodel 0.0.8 with a pre-existing sqlite database that has a column with
Field(sa_column=sa.Column(sam.types.CompressedJSONType))
We are implementing a very simple rest API fetching data from the said database.
This is the SQLModel model (stripped down to demo the issue):
class ZIPCode(SQLModel, table=True):
__tablename__ = 'simple_zipcode'
zipcode: str = Field(primary_key=True)
common_city_list: list = Field(sa_column=sa.Column(sam.types.CompressedJSONType))
Using this for the fastapi part:
@app.get("/zip/{zipcode}", response_model=ZIPCode) # notice the response_model
async def by_zipcode(*, db: Session = Depends(get_db),
zipcode: str = Path(regex=ZIP_RE)):
zipc = db.get(ZIPCode, zipcode)
if not zipc:
raise HTTPException(status_code=404, detail=f'{zipcode} not found')
return zipc
Just starting results in the following, lengthy exception:
$ uvicorn main:app --host 127.0.0.1 --port 8000 --reload
INFO: Will watch for changes in these directories: ['/mnt/e/src/sgt-geodata/src']
INFO: Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)
INFO: Started reloader process [9162] using WatchFiles
Loading settings for: Local
2023-04-05 11:39:16,150 INFO sqlalchemy.engine.Engine BEGIN (implicit)
2023-04-05 11:39:16,150 INFO sqlalchemy.engine.Engine PRAGMA main.table_info("simple_zipcode")
2023-04-05 11:39:16,150 INFO sqlalchemy.engine.Engine [raw sql] ()
2023-04-05 11:39:16,152 INFO sqlalchemy.engine.Engine COMMIT
db created
Process SpawnProcess-1:
Traceback (most recent call last):
File "/usr/lib/python3.11/multiprocessing/process.py", line 314, in _bootstrap
self.run()
File "/usr/lib/python3.11/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/mnt/e/src/sgt-geodata/venv_linux/lib/python3.11/site-packages/uvicorn/_subprocess.py", line 76, in subprocess_started
target(sockets=sockets)
File "/mnt/e/src/sgt-geodata/venv_linux/lib/python3.11/site-packages/uvicorn/server.py", line 59, in run
return asyncio.run(self.serve(sockets=sockets))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/asyncio/runners.py", line 190, in run
return runner.run(main)
^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/asyncio/runners.py", line 118, in run
return self._loop.run_until_complete(task)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "uvloop/loop.pyx", line 1517, in uvloop.loop.Loop.run_until_complete
File "/mnt/e/src/sgt-geodata/venv_linux/lib/python3.11/site-packages/uvicorn/server.py", line 66, in serve
config.load()
File "/mnt/e/src/sgt-geodata/venv_linux/lib/python3.11/site-packages/uvicorn/config.py", line 471, in load
self.loaded_app = import_from_string(self.app)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/e/src/sgt-geodata/venv_linux/lib/python3.11/site-packages/uvicorn/importer.py", line 21, in import_from_string
module = importlib.import_module(module_str)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "<frozen importlib._bootstrap>", line 1206, in _gcd_import
File "<frozen importlib._bootstrap>", line 1178, in _find_and_load
File "<frozen importlib._bootstrap>", line 1149, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 940, in exec_module
File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
File "/mnt/e/src/sgt-geodata/src/main.py", line 55, in <module>
@app.get("/zip/{zipcode}", response_model=ZIPCode)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/e/src/sgt-geodata/venv_linux/lib/python3.11/site-packages/fastapi/routing.py", line 661, in decorator
self.add_api_route(
File "/mnt/e/src/sgt-geodata/venv_linux/lib/python3.11/site-packages/fastapi/routing.py", line 600, in add_api_route
route = route_class(
^^^^^^^^^^^^
File "/mnt/e/src/sgt-geodata/venv_linux/lib/python3.11/site-packages/fastapi/routing.py", line 417, in __init__
] = create_cloned_field(self.response_field)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/e/src/sgt-geodata/venv_linux/lib/python3.11/site-packages/fastapi/utils.py", line 114, in create_cloned_field
use_type = create_model(original_type.__name__, __base__=original_type)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "pydantic/main.py", line 1026, in pydantic.main.create_model
File "/mnt/e/src/sgt-geodata/venv_linux/lib/python3.11/site-packages/sqlmodel/main.py", line 272, in __new__
new_cls = super().__new__(cls, name, bases, dict_used, **config_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "pydantic/main.py", line 138, in pydantic.main.ModelMetaclass.__new__
File "pydantic/utils.py", line 693, in pydantic.utils.smart_deepcopy
File "/usr/lib/python3.11/copy.py", line 146, in deepcopy
y = copier(x, memo)
^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/copy.py", line 231, in _deepcopy_dict
y[deepcopy(key, memo)] = deepcopy(value, memo)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/copy.py", line 172, in deepcopy
y = _reconstruct(x, memo, *rv)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/copy.py", line 271, in _reconstruct
state = deepcopy(state, memo)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/copy.py", line 146, in deepcopy
y = copier(x, memo)
^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/copy.py", line 211, in _deepcopy_tuple
y = [deepcopy(a, memo) for a in x]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/copy.py", line 211, in <listcomp>
y = [deepcopy(a, memo) for a in x]
^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/copy.py", line 146, in deepcopy
y = copier(x, memo)
^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/copy.py", line 231, in _deepcopy_dict
y[deepcopy(key, memo)] = deepcopy(value, memo)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/copy.py", line 172, in deepcopy
y = _reconstruct(x, memo, *rv)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/copy.py", line 271, in _reconstruct
state = deepcopy(state, memo)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/copy.py", line 146, in deepcopy
y = copier(x, memo)
^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/copy.py", line 211, in _deepcopy_tuple
y = [deepcopy(a, memo) for a in x]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/copy.py", line 211, in <listcomp>
y = [deepcopy(a, memo) for a in x]
^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/copy.py", line 146, in deepcopy
y = copier(x, memo)
^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/copy.py", line 231, in _deepcopy_dict
y[deepcopy(key, memo)] = deepcopy(value, memo)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/copy.py", line 172, in deepcopy
y = _reconstruct(x, memo, *rv)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/copy.py", line 271, in _reconstruct
state = deepcopy(state, memo)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/copy.py", line 146, in deepcopy
y = copier(x, memo)
^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/copy.py", line 231, in _deepcopy_dict
y[deepcopy(key, memo)] = deepcopy(value, memo)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/copy.py", line 172, in deepcopy
y = _reconstruct(x, memo, *rv)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/copy.py", line 271, in _reconstruct
state = deepcopy(state, memo)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/copy.py", line 146, in deepcopy
y = copier(x, memo)
^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/copy.py", line 231, in _deepcopy_dict
y[deepcopy(key, memo)] = deepcopy(value, memo)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/copy.py", line 161, in deepcopy
rv = reductor(4)
^^^^^^^^^^^
TypeError: cannot pickle 'module' object
If we remove response_model=ZIPCode from the endpoint definition
@app.get("/zip/{zipcode}" # no response_model
async def by_zipcode(*, db: Session = Depends(get_db),
zipcode: str = Path(regex=ZIP_RE)):
......
OR
@app.get("/zip/{zipcode}", response_model=dict) # plain dict as response_model
async def by_zipcode(*, db: Session = Depends(get_db),
zipcode: str = Path(regex=ZIP_RE)):
......
It's all good, no explosions were observed.
Also, since I prefer to have the response model for documentation purposes, for example, we tried defining a "pure" pydantic model:
class ZIPCodeResponse(BaseModel): # inheriting from BaseModel
zipcode: str
common_city_list: list
and setting this as response_model
@app.get("/zip/{zipcode}", response_model=ZIPCodeResponse)
async def by_zipcode(*, db: Session = Depends(get_db),
zipcode: str = Path(regex=ZIP_RE)):
......
Works as expected - no exceptions thrown, data is returned, and all. But I don't reslly think it should be done like this,
Looked at https://github.com/tiangolo/sqlmodel/issues/540, https://github.com/tiangolo/fastapi/issues/5514, https://github.com/pydantic/pydantic/issues/380, How to use JSON columns with SQLModel
Using
....
class Config:
arbitrary_types_allowed = True
Makes no difference, still dies with TypeError: cannot pickle 'module' object
Environment:
$ pip freeze
anyio==3.6.2
bcrypt==4.0.1
click==8.1.3
fastapi==0.95.0
greenlet==2.0.2
gunicorn==20.1.0
h11==0.14.0
haversine==2.8.0
httptools==0.5.0
idna==3.4
passlib==1.7.4
prettytable==3.6.0
pydantic==1.10.7
python-dotenv==1.0.0
PyYAML==6.0
sniffio==1.3.0
SQLAlchemy==1.4.41
sqlalchemy-mate==1.4.28.4
sqlalchemy2-stubs==0.0.2a32
sqlmodel==0.0.8
starlette==0.26.1
typing_extensions==4.5.0
uvicorn==0.21.1
uvloop==0.17.0
watchfiles==0.19.0
wcwidth==0.2.6
websockets==11.0
$ python -V
Python 3.11.2
$ uname -a
Linux pc 5.15.90.1-microsoft-standard-WSL2 #1 SMP Fri Jan 27 02:56:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Operating System
Linux, Windows, Other
SQLModel Version
0.0.8
Python Version
Python 3.11.2
Please advice on what we are doing wrong here
