Re: [GSOC] Shifting to Py.Test and Improving the Test Suite
Some Pytest advocacy: 1) Pytest has convenient tests collection options - you can just specify folder to run all tests in it. It is also possible to filter tests by regex, and select specific ones. 2) PyUnit: class MyTestCase(TestCase): def test_something(self): expected_content = ... response = self.client(...) self.assertEqual(expected_content, response.content) Pytest: def test_something(client): expected_content = ... response = self.client(...) assert expected_content == response.content Pytest style is cleaner and complies PEP8. Assertions with assert are more readable. It is also possible to group tests, by placing them into class, without subclassing something. 4) In Pytest it is possible to define setup functions for modules, classes and specific tests. 5) Pytest also has fixture mechanism, which allows to use additional resources in tests by specifying fixture names as an arguments (like client) - it is good alternative to using setUp's and subclassing something. 6) pytest-xdist plugin provides parallel execution of tests. 7) Pytest has features for writing parametrized tests. So Pytest is kind of cool and it is easy to migrate to it. On Friday, February 28, 2014 3:42:17 AM UTC+4, Russell Keith-Magee wrote: > > > On Fri, Feb 28, 2014 at 4:17 AM, Gwildor Sok > > wrote: > >> Personally I'm a big fan of Py.test, simply because it's so simple and >> Pythonic to use. Simple functions with simple assert statements. That's >> all. For me this significantly lowers the threshold to write tests and >> requires less effort, which in the end results in way more tests written in >> Py.test than the testsuite Django is currently using. >> > > This is an argument that I've never understood. Why is typing: > > class MyTestCase(TestCase): > > such a burden? Because once you've typed that, everything else in the > class is "just a method". However, you also get the added benefit of test > grouping, and if you've got common setup/teardown requirements, the class > provides a place to put them. > > I know py.test has a lot of fans, but I'm seriously unconvinced by the > "it's so easy to write tests" argument. > > For this proposal to be acceptable to core, it's going to need to start > with a *comprehensive* analysis of why py.test is a better approach to > take. Don't just assume everyone is on board. > > Yours, > Russ Magee %-) > -- You received this message because you are subscribed to the Google Groups "Django developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To post to this group, send email to django-developers@googlegroups.com. Visit this group at http://groups.google.com/group/django-developers. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/f1133fe6-bd1c-4641-857a-e721e26e4e23%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [GSOC] Shifting to Py.Test and Improving the Test Suite
Hi Andrew, I'm not a Django core contributor but just a user and occasional patcher. I submit some comments on this proposal in the hope that they will be useful. On Sun, 6 Apr 2014, Andrew Pashkin wrote: Some Pytest advocacy: 1) Pytest has convenient tests collection options - you can just specify folder to run all tests in it. It is also possible to filter tests by regex, and select specific ones. I do use pytest and it has a lot of good points. Running all tests in a directory or a specific file is one of them, as is the better test selection logic. class MyTestCase(TestCase): def test_something(self): expected_content = ... response = self.client(...) self.assertEqual(expected_content, response.content) Pytest: def test_something(client): expected_content = ... response = self.client(...) assert expected_content == response.content But this is not one of them, at least not for me. I strongly dislike this style of assertions: Pytest style is cleaner and complies PEP8. Assertions with assert are more readable. * The difference between "self.assertEquals(a, b)" and "assert a == b" is minimal for me. * It hides a LOT of magic in how pytest goes back and evaluates the assertion's arguments again if the assertion fails. Sometimes it gets different results (if one of the tested methods has side effects) and then the assertion message makes no sense, hides the problem, or pytest breaks. * It makes it much harder to write custom assertions and get meaningful display on error. It is also possible to group tests, by placing them into class without subclassing something. Grouping tests with decorators allows tests to be members of multiple suites, it's true. However this is not something that I've ever needed. And losing the power of inheritance to add methods to my test classes, by not putting tests into classes, is enough to kill this style completely for me. I write a LOT of my own assertions: https://github.com/aptivate/django-harness/tree/master/django_harness 4) In Pytest it is possible to define setup functions for modules, classes and specific tests. I can do this by adding mixins to test classes. Works fine for me, and less magical. 5) Pytest also has fixture mechanism, which allows to use additional resources in tests by specifying fixture names as an arguments (like client) - it is good alternative to using setUp's and subclassing something. We have named fixtures in Django test classes already, so I don't see that as much of an advantage. We can't easily add fixtures in mixins at the moment. I've not needed that so far. 6) pytest-xdist plugin provides parallel execution of tests. 7) Pytest has features for writing parametrized tests. So Pytest is kind of cool and it is easy to migrate to it. I don't think it's necessary or wise to rewrite the test suite to use pytest-style assertions. But better integration of Django and Pytest, and the ability to write tests with Pytest-style assertions if you like, would certainly have my support (fwiw). Cheers, Chris. -- Aptivate | http://www.aptivate.org | Phone: +44 1223 967 838 Citylife House, Sturton Street, Cambridge, CB1 2QF, UK Aptivate is a not-for-profit company registered in England and Wales with company number 04980791. -- You received this message because you are subscribed to the Google Groups "Django developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To post to this group, send email to django-developers@googlegroups.com. Visit this group at http://groups.google.com/group/django-developers. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/alpine.DEB.2.02.1404061751310.7500%40lap-x201. For more options, visit https://groups.google.com/d/optout.
Re: [GSOC] Shifting to Py.Test and Improving the Test Suite
On 6 April 2014 17:24, Andrew Pashkin wrote: > Some Pytest advocacy: > 1) Pytest has convenient tests collection options - you can just specify > folder to run all tests in it. It is also possible to filter tests by regex, > and select specific ones. Sounds good, but unittest's test discovery is not that bad and can be easily improved. > > 2) PyUnit: > > class MyTestCase(TestCase): > def test_something(self): >expected_content = ... >response = self.client(...) >self.assertEqual(expected_content, response.content) > > Pytest: > > def test_something(client): > expected_content = ... > response = self.client(...) > assert expected_content == response.content > > Pytest style is cleaner and complies PEP8. That is very subjective. Also, Python's standards library uses unittest, so it's hard to argue that Pytest is more in line with Python standard library recommendation (which what PEP8 is). > Assertions with assert are more readable. Again subjective and the magic involved in Pytest's "assertion introspection" is rather scary. And you still need custom methods/functions for non-trivial assertions like assertHTMLEquals or assertQuerysetEquals. > It is also possible to group tests, by placing them into class, without > subclassing something. True, but Python is not Java, so I never found that to be a problem. > 4) In Pytest it is possible to define setup functions for modules, classes > and specific tests. As in unittest: https://docs.python.org/2/library/unittest.html#class-and-module-fixtures > 6) pytest-xdist plugin provides parallel execution of tests. The hard part when running tests in parallel is usually how to manage shared resources (like the database - and no creating a new Oracle instance in every subprocess is a non-starter!). So while having some support for that is good, it's not something Django's test suite would get for free. > 5) Pytest also has fixture mechanism, which allows to use additional > resources in tests by specifying fixture names as an arguments (like client) > - it is good alternative to using setUp's and subclassing something. > 7) Pytest has features for writing parametrized tests. > These two features are actually something worth looking at (parametrized tests would allow cleaning up a lot of template tests), but I'm guessing that for the core developers to get convinced you'd have to show that this two will make a real difference (instead of focusing on the less important stuff mentioned earlier). Best regards, Lucas Rekucki -- You received this message because you are subscribed to the Google Groups "Django developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To post to this group, send email to django-developers@googlegroups.com. Visit this group at http://groups.google.com/group/django-developers. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAEZs-EJM6tGxYsihgebYWXgX5uX3OLaJnrFZR3J1LDqxSa2xQg%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
Re: [GSOC] Shifting to Py.Test and Improving the Test Suite
...Sometimes it gets different results (if one of the tested methods has side effects) and then the assertion message makes no sense, hides the problem, or pytest breaks. * It makes it much harder to write custom assertions and get meaningful display on error. Can you give an examples for cases with messages/breakings and for custom assertions? Regarding magic in Pytest - yeah - there is some =)) But if think deeper, inheritance and self is also magic, and Pytest just uses its own magic (dependency injection) that is differs from OO magic =) Actually it would be interesting to discuss cases where Pytest cant handle (or handle with convennience) what PyUnit can and vice versa. On 06.04.2014 21:02, Chris Wilson wrote: Hi Andrew, I'm not a Django core contributor but just a user and occasional patcher. I submit some comments on this proposal in the hope that they will be useful. On Sun, 6 Apr 2014, Andrew Pashkin wrote: Some Pytest advocacy: 1) Pytest has convenient tests collection options - you can just specify folder to run all tests in it. It is also possible to filter tests by regex, and select specific ones. I do use pytest and it has a lot of good points. Running all tests in a directory or a specific file is one of them, as is the better test selection logic. class MyTestCase(TestCase): def test_something(self): expected_content = ... response = self.client(...) self.assertEqual(expected_content, response.content) Pytest: def test_something(client): expected_content = ... response = self.client(...) assert expected_content == response.content But this is not one of them, at least not for me. I strongly dislike this style of assertions: Pytest style is cleaner and complies PEP8. Assertions with assert are more readable. * The difference between "self.assertEquals(a, b)" and "assert a == b" is minimal for me. * It hides a LOT of magic in how pytest goes back and evaluates the assertion's arguments again if the assertion fails. Sometimes it gets different results (if one of the tested methods has side effects) and then the assertion message makes no sense, hides the problem, or pytest breaks. * It makes it much harder to write custom assertions and get meaningful display on error. It is also possible to group tests, by placing them into class without subclassing something. Grouping tests with decorators allows tests to be members of multiple suites, it's true. However this is not something that I've ever needed. And losing the power of inheritance to add methods to my test classes, by not putting tests into classes, is enough to kill this style completely for me. I write a LOT of my own assertions: https://github.com/aptivate/django-harness/tree/master/django_harness 4) In Pytest it is possible to define setup functions for modules, classes and specific tests. I can do this by adding mixins to test classes. Works fine for me, and less magical. 5) Pytest also has fixture mechanism, which allows to use additional resources in tests by specifying fixture names as an arguments (like client) - it is good alternative to using setUp's and subclassing something. We have named fixtures in Django test classes already, so I don't see that as much of an advantage. We can't easily add fixtures in mixins at the moment. I've not needed that so far. 6) pytest-xdist plugin provides parallel execution of tests. 7) Pytest has features for writing parametrized tests. So Pytest is kind of cool and it is easy to migrate to it. I don't think it's necessary or wise to rewrite the test suite to use pytest-style assertions. But better integration of Django and Pytest, and the ability to write tests with Pytest-style assertions if you like, would certainly have my support (fwiw). Cheers, Chris. -- С наилучшими пожеланиями, Андрей Пашкин. м.т - +7 (985) 898 57 59 Skype - waves_in_fluids e-mail - andrew.pash...@gmx.co.uk -- You received this message because you are subscribed to the Google Groups "Django developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To post to this group, send email to django-developers@googlegroups.com. Visit this group at http://groups.google.com/group/django-developers. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/5341952A.8000201%40gmx.co.uk. For more options, visit https://groups.google.com/d/optout.
Re: [GSOC] Shifting to Py.Test and Improving the Test Suite
Hi Andrew,
On Sun, 6 Apr 2014, Andrew Pashkin wrote:
* It makes it much harder to write custom assertions and get meaningful
display on error.
Can you give an examples for cases with messages/breakings and for custom
assertions?
I don't have an example of breakage to hand, I ripped out the offending
code in disgust and replaced it with standard assertions that worked. If I
find another example I'll be sure to let you know before I do the same
again.
Regarding custom assertions, how would you write assert_not_redirected
from
https://github.com/aptivate/django-harness/blob/master/django_harness/fast_dispatch.py
as a PyTest assertion? Consider the following tests:
def not_redirected(response):
return (response.status_code != 302)
class RedirectViewTests(FastDispatchMixin, TestCase):
def test_redirect_view_with_custom_assertion(self):
response = self.fast_dispatch('my-redirecting-view')
self.assert_not_redirected(response)
def test_redirect_view_with_pytest_assertion(self):
response = self.fast_dispatch('my-redirecting-view')
assert not_redirected(response)
assert_not_redirected is from FastDispatchMixin in
https://github.com/aptivate/django-harness/blob/master/django_harness/fast_dispatch.py,
not_redirected is a simple assertion I wrote as a demo, since I don't use
pytest much myself for this reason. Now compare the output (reformatted
for readability):
RedirectViewTests.test_redirect_view_with_custom_assertion
main/tests/tests.py:71: in test_redirect_view_with_custom_assertion
self.assert_not_redirected(response)
.ve/src/django-harness/django_harness/fast_dispatch.py:114: in
assert_not_redirected
msg_prefix)
E AssertionError: 302 != 200 : unexpectedly redirected to
http://www.google.com/
RedirectViewTests.test_redirect_view_with_pytest_assertion
main/tests/tests.py:75: in test_redirect_view_with_pytest_assertion
assert not_redirected(response)
E AssertionError: assert
not_redirected(0x493da10>)
I.e. my assertion outputs:
AssertionError: 302 != 200 : unexpectedly redirected to
http://www.google.com/
and the simple Python assertion outputs this:
AssertionError: assert
not_redirected()
How would you generate a useful error message, bearing in mind that you
can't really access response._headers['location'][1] without first having
checked that the response is a redirect, which isn't the normal case?
Regarding magic in Pytest - yeah - there is some =)) But if think
deeper, inheritance and self is also magic, and Pytest just uses its own
magic (dependency injection) that is differs from OO magic =)
OO "magic" is well known and understood by most Python developers.
Pytest's magic is (a) largely undocumented (afaik) and (b) relies on
understanding of Python bytecode which is definitely not common knowledge.
Cheers, Chris.
--
Aptivate | http://www.aptivate.org | Phone: +44 1223 967 838
Citylife House, Sturton Street, Cambridge, CB1 2QF, UK
Aptivate is a not-for-profit company registered in England and Wales
with company number 04980791.
--
You received this message because you are subscribed to the Google Groups "Django
developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-developers/alpine.DEB.2.02.1404061926030.7500%40lap-x201.
For more options, visit https://groups.google.com/d/optout.
