Testing Unmanaged Models - Using the SchemaEditor to create db tables

[Emmanuel Katchy]

Hi everyone!

I'd like to get your thoughts on something.

Unmanaged models mean that Django no longer handles creating and managing 
schema at the database level (hence the name).
When running tests, this means these tables aren't created, and we can't 
run queries against that model. The general solution I found is to monkey-patch 
the TestSuiteRunner to temporarily treat models as managed 

.

Doing a bit of research I however came up with a solution using SchemaEditor 
, to create the 
model tables directly, viz:

```
"""
A cleaner approach to temporarily creating unmanaged model db tables for 
tests
"""

from unittest import TestCase

from django.db import connections, models

class create_unmanaged_model_tables:
"""
Create db tables for unmanaged models for tests
Adapted from: https://stackoverflow.com/a/49800437
Examples:
with create_unmanaged_model_tables(UnmanagedModel):
...
@create_unmanaged_model_tables(UnmanagedModel, FooModel)
def test_generate_data():
...

@create_unmanaged_model_tables(UnmanagedModel, FooModel)
def MyTestCase(unittest.TestCase):
...
"""

def __init__(self, unmanaged_models: list[ModelBase], db_alias: str = 
"default"):
"""
:param str db_alias: Name of the database to connect to, defaults 
to "default"
"""
self.unmanaged_models = unmanaged_models
self.db_alias = db_alias
self.connection = connections[db_alias]

def __call__(self, obj):
if issubclass(obj, TestCase):
return self.decorate_class(obj)
return self.decorate_callable(obj)

def __enter__(self):
self.start()

def __exit__(self, exc_type, exc_value, traceback):
self.stop()

def start(self):
with self.connection.schema_editor() as schema_editor:
for model in self.unmanaged_models:
schema_editor.create_model(model)

if (
model._meta.db_table
not in self.connection.introspection.table_names()
):
raise ValueError(
"Table `{table_name}` is missing in test 
database.".format(
table_name=model._meta.db_table
)
)

def stop(self):
with self.connection.schema_editor() as schema_editor:
for model in self.unmanaged_models:
schema_editor.delete_model(model)

def copy(self):
return self.__class__(
unmanaged_models=self.unmanaged_models, db_alias=self.db_alias
)

def decorate_class(self, klass):
# Modify setUpClass and tearDownClass
orig_setUpClass = klass.setUpClass
orig_tearDownClass = klass.tearDownClass

@classmethod
def setUpClass(cls):
self.start()
if orig_setUpClass is not None:
orig_setUpClass()


@classmethod
def tearDownClass(cls):
if orig_tearDownClass is not None:
orig_tearDownClass()
self.stop()

klass.setUpClass = setUpClass
klass.tearDownClass = tearDownClass

return klass

def decorate_callable(self, callable_obj):
@functools.wraps(callable_obj)
def wrapper(*args, **kwargs):
with self.copy():
return callable_obj(*args, **kwargs)

return wrapper
```

Would this make a good addition to *django.test.utils*?

P.S: First time posting here :P


-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/6f5c668b-8f72-42be-9e41-01c786c12027n%40googlegroups.com.

Re: Testing Unmanaged Models - Using the SchemaEditor to create db tables

[Emmanuel Katchy]

Hi Adam,

Thanks for your response!

I understand your point about unmanaged models being a niche use case of 
Django. I've decided to proceed with creating a package and see how it goes.

The new enterContext() and other methods in unittest seem interesting. I'll 
definitely be using them more from now on.

Best,
Emmanuel

On Friday, February 9, 2024 at 11:23:36 PM UTC+1 Adam Johnson wrote:

> Hi Emmanuel
>
> Most activity from this mailing list has moved to Django Internals 
> category on the forum: https://forum.djangoproject.com/c/internals/5 . 
> Better to post there in future, or you could even duplicate this post.
>
> I think your approach is worth sharing in a blog post, or even a package, 
> rather than adding to Django itself.  Your code is worth sharing but may be 
> too specific for the framework.
>
> Unmanaged models aren’t particularly popular. When they are used, it can 
> be for many reasons. As a result, projects may create the tables in various 
> ways during tests, such as loading an existing database dump or calling an 
> external tool. So using Django’s migrations to create them (through 
> managed=True or SchemaEditor) is just one option among many.
>
> By the way, you may be able to simplify your implementation with the new 
> context methods in unittest from Python 3.11: 
> https://adamj.eu/tech/2022/11/14/unittest-context-methods-python-3-11-backports/
>  
> .
>
> Thank you for sharing, and welcome to the Django community!
>
> On Sun, Jan 28, 2024, at 11:00 PM, Emmanuel Katchy wrote:
>
> Hi everyone!
>
> I'd like to get your thoughts on something.
>
> Unmanaged models mean that Django no longer handles creating and managing 
> schema at the database level (hence the name).
> When running tests, this means these tables aren't created, and we can't 
> run queries against that model. The general solution I found is to 
> monkey-patch 
> the TestSuiteRunner to temporarily treat models as managed 
> <https://www.caktusgroup.com/blog/2010/09/24/simplifying-the-testing-of-unmanaged-database-models-in-django/>
> .
>
> Doing a bit of research I however came up with a solution using 
> SchemaEditor <https://docs.djangoproject.com/en/5.0/ref/schema-editor/>, 
> to create the model tables directly, viz:
>
> ```
> """
> A cleaner approach to temporarily creating unmanaged model db tables for 
> tests
> """
>
> from unittest import TestCase
>
> from django.db import connections, models
>
> class create_unmanaged_model_tables:
> """
> Create db tables for unmanaged models for tests
> Adapted from: https://stackoverflow.com/a/49800437
> Examples:
> with create_unmanaged_model_tables(UnmanagedModel):
> ...
> @create_unmanaged_model_tables(UnmanagedModel, FooModel)
> def test_generate_data():
> ...
> 
> @create_unmanaged_model_tables(UnmanagedModel, FooModel)
> def MyTestCase(unittest.TestCase):
> ...
> """
>
> def __init__(self, unmanaged_models: list[ModelBase], db_alias: str = 
> "default"):
> """
> :param str db_alias: Name of the database to connect to, defaults 
> to "default"
> """
> self.unmanaged_models = unmanaged_models
> self.db_alias = db_alias
> self.connection = connections[db_alias]
>
> def __call__(self, obj):
> if issubclass(obj, TestCase):
> return self.decorate_class(obj)
> return self.decorate_callable(obj)
>
> def __enter__(self):
> self.start()
>
> def __exit__(self, exc_type, exc_value, traceback):
> self.stop()
>
> def start(self):
> with self.connection.schema_editor() as schema_editor:
> for model in self.unmanaged_models:
> schema_editor.create_model(model)
>
> if (
> model._meta.db_table
> not in self.connection.introspection.table_names()
> ):
> raise ValueError(
> "Table `{table_name}` is missing in test 
> database.".format(
> table_name=model._meta.db_table
> )
> )
>
> def stop(self):
> with self.connection.schema_editor() as schema_editor:
> for model in self.unmanaged_models:
> schema_editor.delete_model(model)
>
> def copy(self):
> return self.__class__(
> unmanaged_models=self.