Hypothesis 1.0: A production quality property-based testing library for Python
Hypothesis is a Python library for turning unit tests into generative tests, covering a far wider range of cases than you can manually. Rather than just testing for the things you already know about, Hypothesis goes out and actively hunts for bugs in your code. It usually finds them, and when it does it gives you simple and easy to read examples to demonstrate. Hypothesis is based on Quickcheck ( https://wiki.haskell.org/Introduction_to_QuickCheck2) but is designed to have a naturally Pythonic API and integrate well with Python testing libraries. It's easy to use, extremely solid, and probably more devious than you are at finding edge cases. The 1.0 release of Hypothesis has a stable and well documented public API. It's more than ready for you to use and it's easy to get started. Full documentation is available at http://hypothesis.readthedocs.org/en/latest/, or if you prefer you can skip straight to the quick start guide: http://hypothesis.readthedocs.org/en/latest/quickstart.html -- https://mail.python.org/mailman/listinfo/python-list
Re: Hypothesis 1.0: A production quality property-based testing library for Python
On 30 March 2015 at 22:37, Terry Reedy wrote: > On 3/30/2015 2:44 PM, David MacIver wrote: > >> Hypothesis is a Python library for turningunit tests into generative >> tests, covering a far wider range of cases thanyou can manually. Rather >> than just testing for thethings you already know about, Hypothesis goes >> out and actively hunts forbugs in your code.It usually finds them, and >> when it does it gives you simple and easy to read examples to demonstrate. >> > > Iteresting. Some years ago, Vickor Stinner wrote a fuzzing module > (fusil?). I believe he found some bugs in the stdlib with it. > Cool. I'd somehow missed fusil, thanks. It looks interesting and I'll have a dig through it for ideas :-) Hypothesis and fusil are aimed at somewhat different levels. Hypothesis is closer to being for unit testing (I mean you can use it for integration testing and other things just as easily, but that's not strictly its strength) whileas fusil looks better for testing whole programs. > > https://www.python.org/dev/peps/pep-0484/ > proposes the addition of a 'typing' module for defining types beyond the > builtins and ABCs, such as List(int), Union(tuple, list). If and when it > is accepted and added (maybe 3.5, maybe later), you should consider having > Hypothesis accept the notations that it can work with. Yes, definitely. Having something like that as standard would be great for Hypothesis and I intend to support it once it becomes available. (It will force me to finally figure out how to do staged APIs, with some versions of the API only supported on some versions of python, but I need to do that anyway. This might involve just shipping a compatibility layer for previous versions of Python in with Hypothesis) > > > Full documentation is available at >> http://hypothesis.readthedocs.org/en/latest/, or if you prefer you can >> skip straight to the quick start guide: >> http://hypothesis.readthedocs.org/en/latest/quickstart.html >> > > > -- > Terry Jan Reedy > > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: Hypothesis 1.0: A production quality property-based testing library for Python
On 31 March 2015 at 02:46, Terry Reedy wrote: > On 3/30/2015 4:46 PM, David MacIver wrote: > >> On 30 March 2015 at 22:37, Terry Reedy > <mailto:[email protected]>> wrote: >> > > https://www.python.org/dev/__peps/pep-0484/ >> <https://www.python.org/dev/peps/pep-0484/> >> proposes the addition of a 'typing' module for defining types beyond >> the builtins and ABCs, such as List(int), Union(tuple, list). If >> and when it is accepted and added (maybe 3.5, maybe later), you >> should consider having Hypothesis accept the notations that it can >> work with. >> >> Yes, definitely. Having something like that as standard would be great >> for Hypothesis and I intend to support it once it becomes available. >> > > I just posted "PEP 484: Generating test inputs from type hints" to > python-ideas list. Thanks. I'll pop over there and join in the discussion. > > > (It will force me to finally figure out how to do staged APIs, with some >> versions of the API only supported on some versions of python, but I >> need to do that anyway. This might involve just shipping a compatibility >> layer for previous versions of Python in with Hypothesis) >> > > I believe Guido intends that typing.py should be available on PyPI for use > with current versions. > > Ah, that would be useful. Does that include Python 2.7? The examples suggest yes. (I really hate having to ask that. I don't want to support Python 2.7 at all, but...) > Full documentation is available at >> http://hypothesis.readthedocs.__org/en/latest/ >> you can >> skip straight to the quick start guide: >> http://hypothesis.readthedocs.__org/en/latest/quickstart.html >> > > -- > Terry Jan Reedy > > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: Hypothesis 1.0: A production quality property-based testing library for Python
On 31 March 2015 at 02:51, Paul Rubin wrote: > David MacIver writes: > > Hypothesis is based on Quickcheck > > (https://wiki.haskell.org/Introduction_to_QuickCheck2) > > This is great. Have you looked at the Erlang version of Quickcheck? It > may have aspects more directly applicable to Python, since Erlang is > dynamically typed like Python is. > I actually haven't looked at it much. I've read some of the associated papers, but due to a mix of my not really knowing erlang well and it being proprietary it's not as useful for idea mining as I'd like it to be. I have some friends who are more extensive users of it who I've talked to a bit about it for feature comparisons though. -- https://mail.python.org/mailman/listinfo/python-list
Looking for people who are using Hypothesis and are willing to say so
Hi there, Author of Hypothesis here. (If you don't know what Hypothesis is, you're probably not the target audience for this email but you should totally check it out: https://hypothesis.readthedocs.org/ Unless you like spending ages writing tests and still shipping buggy code). I keep finding out about new people using Hypothesis who I've never heard of. e.g. turns out that depending on how you count there are between two and four talks about Hypothesis happening at Europython this year, and many of them are from people I don't know. On the one hand, it's great that people are using and excited about it! No complaints from me there. I was bowled over when I realised about the EuroPython talks. On the other hand, it's really quite useful to have more visibility of usage - both for me to have it and also for other people to see - it's a much easier sell that people should start using it if they can see that lots of other people are too. SO, the point. If you are one of those people using Hypothesis, I'd really like it if you would say so publicly. The #1 best way for you to do this for me is to add your name and usage to https://github.com/DRMacIver/hypothesis/blob/master/docs/endorsements.rst so it will show up on the endorsements page at http://hypothesis.readthedocs.org/en/latest/endorsements.html I'm also thrilled that people are speaking at it, and would love more talks and blog posts about it. Even tweeting enthusiastically about it is good too. Finally, if you are using Hypothesis but can't/don't want to speak about doing so publicly because your company is doing super top secret stuff (or any other reason), I'd really appreciate just a short email saying roughly what sort of things you're using it for, maybe give me an idea of your workflow. The other reason that I want to know who is using it is so I can learn where it needs to improve and also help other people use it better (I'm pretty sure some of the users of Hypothesis at this point have a better idea how to deploy it than I do). Regards and thanks, David R. MacIver -- https://mail.python.org/mailman/listinfo/python-list
Re: Looking for people who are using Hypothesis and are willing to say so
Actually one of the things that's helped the most in the course of designing Hypothesis is the realisation that types are something of a red herring for this sort of testing. Thinking purely in terms of custom generators helps a lot, because it means you can do things like specify size bounds on lists, integers, etc. as well as map and filter over the resulting data (e.g. lists(integers().map(lambda x: x * 2), min_size=1, max_size=10). Lists of length between 1 and 10 only containing even integers. So in this regard the design of Hypothesis should be considered more closely related to that of the Erlang than the Haskell quickcheck (although I'd only ever used statically typed quickchecks before writing Hypothesis). In particular Hypothesis's strategies are best thought of in terms of how to provide data as the argument rather than a type - you can't check whether a given value is producable from a given strategy. I was originally thinking it would be worth writing some auto derivation functionality for using the new python 3 type annotations, but I actually don't think it would be useful. You lose far too much flexibility. On 24 June 2015 at 23:13, Paul Rubin wrote: > David MacIver writes: > > Author of Hypothesis here. (If you don't know what Hypothesis is, you're > > probably not the target audience for this email but you should totally > > check it out: https://hypothesis.readthedocs.org/ > > Oh very cool: a QuickCheck-like unit test library. I heard of something > like that for Python recently, that might or might not have been > Hypothesis. I certainly plan to try it out. The original QuickCheck > (for Haskell) used the static type signatures on the functions under > test to know what test cases to generate, but Erlang QuickCheck has had > some good successes, including finding some subtle bugs during > development in the HAMT (Clojure-like hash array mapped trie) > implementation just released with Erlang/OTP 18.0 this week. > > I see Hypothesis use decorators that look sort of like Erlang Dialyzer > so that can help with test cases. Maybe later, it use Python 3 type > annotations, though I think those are still much less precise than > Dialyzer or Haskell types. > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
