[ANNOUNCE] Campaign to support the notmm project!
Hi All, I'm raising a campaign to support the notmm project, a freely accessible open source project i created to develop an advanced web framework for Django. Furthermore the project is using ConfigObj internally for allowing flexible configuration and Cython for extending Django apps in C. If you would like thus supporting the time and work I invested into the project I would really appreciate a small donation, as I'm now really poor and cannot even finance the server hosting. Your donation would therefore be used to put back the website online and continue active development of an alternative web framework for Python which doesn't necessarily force its users to log in Facebook or Twitter for commenting, or syndication purposes.. Lastly if you have any comments on this letter or would like further info before making a donation it will be my pleasure to help.. Kind regards, Etienne http://pledgie.com/campaigns/16268 -- Etienne Robillard Green Tea Hackers Club Fine Software Carpentry For The Rest Of Us! http://gthc.org/ [email protected] “It is easy to fly into a passion... anybody can do that, but to be angry with the right person to the right extent and at the right time and in the right way that is not easy.” -Aristotle “You have to accept whatever comes and the only important thing is that you meet it with courage and with the best that you have to give.” - Eleanor Roosevelt -- http://mail.python.org/mailman/listinfo/python-list
notmm is dead!
Dear list, Due to lack of energy and resources i'm really sad to announce the removal of notmm from pypi and bitbucket. I deleted also my account from bitbucket as it was not really useful for me. notmm will continue to be accessible from my master site at http://gthc.org/dist/notmm until the server get down, as I cannot find money to pay for the hosting of notmm.org, neither anyone to encourage the project so it can grow further. I have tried to develop a coherent extension for Django using the open source model but I'm afraid to have been bitten by its failure to encourage a free market over one dictated by profit and the use of cheap tricks to compete unfairly with perhaps too much openness. I always will also continue to love and use free softwares but sadly it seems asking for a little fairness is too much asked to competitors dedicated in stealing and subverting my work for their own advantages... I therefore refuse to continue any longer being mocked by competitors asking excessive prices for having a broken Internet dictated by a few companies and decide the content I should be visiting. Shall you have anything you wish saying I'll be open to discuss further on this list. I wish also to thanks the supporters of the project who have invested time and energy into my business and dedication to the notmm project. Best wishes, Etienne -- Etienne Robillard Green Tea Hackers Club Fine Software Carpentry For The Rest Of Us! http://gthc.org/ [email protected] "If a free society cannot help the many who are poor, it cannot save the few who are rich." -John F. Kennedy -- http://mail.python.org/mailman/listinfo/python-list
Re: notmm is dead!
Thanks, but I tried all that and don't have much energy for continuing. If you're serious about open source then maybe you can forward the thread to django-developers and get some fundings to pay for a minimalistic fee to get the project maintained by someone else, otherwise I'd prefer sticking with more profitable activities. E On Fri, 5 Oct 2012 08:23:05 +1000 Chris Angelico wrote: > On Fri, Oct 5, 2012 at 8:11 AM, Etienne Robillard > wrote: > > For $399 i give you the current source code and you can do whatever you > > like with it, including > > putting in github or sf. i think to have lost too many time already with > > further > > maintaining the open source version without getting a penny out of it. > > Like I said, I'm not personally interested; but perhaps say that to > the django list and someone'll bite. But if you're really tired of it, > then just post the source code and someone may end up taking your > project to places you never had the energy to. > > > There was - and, incidentally, still kinda is - a project called Gmud, > a 32-bit (but Win32s compatible) Windows MUD client. Its author asked > people to send him money if they liked and used the program - $US20 I > think - but almost nobody ever did. For years, Gmud was the > recommended Windows client for Threshold RPG, and yet still something > like *four* people ever sent the author money. So the author threw the > source out to the world and said "I'm done, have fun". > > Enter the Threshold RPG community. Gmud has been extremely popular (in > fact, some people still use it today), in spite of some limitations > that may have been reasonable a few years ago, but are ridiculous now, > like a fixed 500-line scrollback buffer. So some of the people there > ask me to grab the source, tweak a few things, and recompile. I'm a > geek, I do these sorts of things. > > Well, it turned out to be not that simple, for a few reasons. But > eventually, after a near-complete rewrite, I produced a new MUD client > that uses the same look and feel as Gmud, as an acknowledged > derivative. RosMud++ is now the officially recommended Windows client > for Threshold, and it would never have happened if Gmud's source > hadn't been given away. > > > It's really hard to make money off software, these days. Which is a > pity, because there's lots of good software that'd be worth money. I > do see your pain. :( This is part of why my newest MUD client, Gypsum, > is open-sourced from the very beginning. > > ChrisA -- Etienne Robillard Green Tea Hackers Club Fine Software Carpentry For The Rest Of Us! http://gthc.org/ [email protected] -- http://mail.python.org/mailman/listinfo/python-list
Re: notmm is dead!
Err not exactly.. :) Firstly notmm is still ISC licensed and available from here http://gthc.org/notmm/dist/. Secondly i don't want to leave it to the hands of people without I can get a single dime for the work did, however some peoples don't seem to get this point yet.. My apologies if you feel this removal was rude anyways. Feel free to contact me again if you need further clarification or would like to take over maintainership of a branch for a minimal fee. Kind regards, Etienne On 05 Oct 2012 00:22:06 GMT Steven D'Aprano wrote: > On Thu, 04 Oct 2012 14:10:46 -0400, Etienne Robillard wrote: > > > Dear list, > > > > Due to lack of energy and resources i'm really sad to announce the > > removal of notmm from pypi and bitbucket. > > Well that's just rude. Even if you don't intend to maintain the software > any more, why are you removing it from pypi? Since you say you are a fan > of Open Source software, just flag it as unmaintained and leave it for > somebody else to pick up. > > If you are going to abandon the project, release it on PyPI with a dual > MIT and GPL licence, and let it be taken over by somebody else. > > If you were looking for sympathy here, starting off by removing your > project from free hosting, then complaining that you can't pay for the > non-free hosting, was NOT the right way to do so. > > By the way, the latest version of notmm (0.4.4) has an empty licence > file. No licence means that everyone using it is unlicenced and therefore > infringing your copyright. > > > > -- > Steven > -- > http://mail.python.org/mailman/listinfo/python-list -- Etienne Robillard Green Tea Hackers Club Fine Software Carpentry For The Rest Of Us! http://gthc.org/ [email protected] -- http://mail.python.org/mailman/listinfo/python-list
Re: notmm is dead!
You probably have a old tarball or something... $ wget http://gthc.org/dist/notmm/notmm-0.4.4.tar.gz $ md5sum notmm-0.4.4.tar.gz dff1b2ec5373b5157cf79d57169a336e notmm-0.4.4.tar.gz Cheers, Etienne On Thu, 4 Oct 2012 18:46:18 -0600 Ian Kelly wrote: > On Thu, Oct 4, 2012 at 6:22 PM, Steven D'Aprano > wrote: > > By the way, the latest version of notmm (0.4.4) has an empty licence > > file. No licence means that everyone using it is unlicenced and therefore > > infringing your copyright. > > It's an ISC license. The notmm-0.4.4/LICENSE file is a link to the > notmm-0.4.4/notmm-0.4.4-rc7/LICENSE file, unless your archiver fails > to reconstruct the link when untarring, in which case the former ends > up as an empty file (but the latter is still present). > -- > http://mail.python.org/mailman/listinfo/python-list -- Etienne Robillard Green Tea Hackers Club Fine Software Carpentry For The Rest Of Us! http://gthc.org/ [email protected] -- http://mail.python.org/mailman/listinfo/python-list
Re: notmm is dead!
No. All past notmm licenses were and still ARE ISC licensed. The license fee is simply because I'm shifting into commercial license for new releases, including the newer 0.4.5 version... Pypi was not the authority source for notmm and neither anyone can claim the license was left blank, thats once again a form of plagiarism. Shall anyone is really serious about supporting the work i did then he'll want to pay a minimal fee to get the proper license rights and can contact me off-list if he want to fork a branch to github of google code. Thanks, Etienne On Thu, 04 Oct 2012 20:39:55 -0600 Michael Torrie wrote: > On 10/04/2012 05:13 PM, Etienne Robillard wrote: > > Thanks, but I tried all that and don't have much energy for continuing. If > > you're > > serious about open source then maybe you can forward the thread to > > django-developers > > and get some fundings to pay for a minimalistic fee to get the project > > maintained > > by someone else, otherwise I'd prefer sticking with more profitable > > activities. > > Nothing in the existing license prevents someone from taking the latest > source and posting it back on Pypi as an unmaintained package. Isn't > that correct? > > What are you referring to when you say "minimalistic fee." Would this > be a fee you require for transferring copyright assignment? I know of > no fee necessary for a new maintainer to take over should one wish to. > Copyright assignment is not strictly necessary. > > -- > http://mail.python.org/mailman/listinfo/python-list -- Etienne Robillard Green Tea Hackers Club Fine Software Carpentry For The Rest Of Us! http://gthc.org/ [email protected] -- http://mail.python.org/mailman/listinfo/python-list
Re: When was the last time you did something for the first time?
Micro$oft :) On Fri, 5 Oct 2012 04:03:52 -0700 (PDT) The Matchmaker wrote: > What do you want to talk about today? > -- > http://mail.python.org/mailman/listinfo/python-list -- Etienne Robillard Green Tea Hackers Club Fine Software Carpentry For The Rest Of Us! http://gthc.org/ [email protected] -- http://mail.python.org/mailman/listinfo/python-list
Re: notmm is dead!
On Fri, 5 Oct 2012 09:29:39 -0600 Ian Kelly wrote: > On Oct 4, 2012 6:56 PM, "Etienne Robillard" wrote: > > > > You probably have a old tarball or something... > > Not unless you've replaced it since I made my post, as I had just > downloaded it to check the license. The 0.4.4 release is old, however, so you might really got a wrong tarball. :-) Cheers, Etienne -- Etienne Robillard Green Tea Hackers Club Fine Software Carpentry For The Rest Of Us! http://gthc.org/ [email protected] -- http://mail.python.org/mailman/listinfo/python-list
Re: Generating C++ code
On Tue, 09 Oct 2012 20:55:48 +0100 Andrea Crotti wrote: > On 10/09/2012 05:00 PM, Jean-Michel Pichavant wrote: > > Greetings, > > > > I'm trying to generate C++ code from an XML file. I'd like to use a > > template engine, which imo produce something readable and maintainable. > > My google search about this subject has been quite unsuccessful, I've been > > redirected to template engine specific to html mostly. > > > > Does anybody knows a python template engine for generating C++ code ? > > > > Here's my flow: > > > > XML file -> nice python app -> C++ code > > > > From what I know I could use Cheetah, a generic template engine. I never > > used it though, I'm not sure this is what I need. > > I'm familiar with jinja2 but I'm not sure I could use it to generate C++ > > code, did anybody try ? (maybe that's a silly question) > > > > Any advice would be appreciated. > > > > JM > > I think you can use anything to generate C++ code, but is it a good idea? > Are you going to produce this code only one time and then maintain it > manually? > > And are you sure that the design that you would get from the XML file > actually makes sense when > translated in C++? > -- > http://mail.python.org/mailman/listinfo/python-list You can build nice python app with Cython/Pyrex too if you got sufficient knowledge in C/C++ programming to extend the app in C. :-) Otherwise that seem like a little counter-productive to produce a python template only for generating C stubs when python is good for many things without requiring advanced C++ skills like memory management, etc. HTH, E -- Etienne Robillard Green Tea Hackers Club Fine Software Carpentry For The Rest Of Us! http://gthc.org/ [email protected] -- http://mail.python.org/mailman/listinfo/python-list
Porting notmm to pep 3333 and mod_wsgi
Hi, Has PEP- introduced backward incompatibilities or changes in unicode encoding and decoding or should this updated pep require python 3 to make mod_wsgi working with wsgi 1.0 (pep-333) scripts ? Moreover is it possible to use python 2.7 with mod_wsgi at all without the utf-8 encoding errors? Besides theses minor Unicode encoding problems I'm glad to announce experimental mod_wsgi support for notmm on python 2.7.3. I will also include a sample app into the next notmm release as a proof-of-concept for Django-based projects but so far I haven't found more serious problems following the standard documentation in mod_wsgi 3.4... Furthermore this newer support for mod_wsgi should allow experimented Django developers to use mod_wsgi and Apache as an alternative to FastCGI (and nginx) for serving high-performance Django apps compiled into optimized C bindings using Cython and notmm! :-) Cheers, Etienne -- Etienne Robillard Green Tea Hackers Club Fine Software Carpentry For The Rest Of Us! http://gthc.org/ [email protected] Donations for the notmm project are accepted through Pledgie.com. For more info please visit: http://pledgie.com/campaigns/16268 -- http://mail.python.org/mailman/listinfo/python-list
Re: Generating C++ code
On Wed, 10 Oct 2012 11:59:50 +0200 (CEST)
Jean-Michel Pichavant wrote:
> Well, the C++ code will end up running on a MIPS on a SOC, unfortunately,
> python is not an option here.
> The xml to C++ makes a lot of sense, because only a small part of the code is
> generated that way (everything related to log & fatal events). Everything
> else is written directly in C++.
sorry but i don't get what you mean with a "MIPS on a SOC". Is not Python well
supported on MIPS ?
> Currently we already have a python script that translate this xml file to
> C++, but it's done in a way that is difficult to maintain. Basically, when
> parsing the xml file, it writes the generated C++ code. Something like:
> if 'blabla' in xml:
> h_file.write("#define blabla 55", append="top")
> c_file.write("someglobal = blabla", append="bottom")
Don't do that! This is a good example of ambigous coding (to say the least..)
and you'll make C++ programmers eyes to bleed
at this.
> This is working, but the python code is quite difficult to maintain, there's
> a lot of escaping going on, it's almost impossible to see the structure of
> the c files unless generating one and hopping it's successful. It's also
> quite difficult to insert code exactly where you want, because you do not
> know the order in which the xml trees are defined then parsed.
Its maybe working but why then are you stuck asking for help ? I suggest you
either write plain C++ code or learn to
use Python more efficiently...
> JM
> --
> http://mail.python.org/mailman/listinfo/python-list
Kind regards,
Etienne
--
http://mail.python.org/mailman/listinfo/python-list
Re: Generating C++ code
On Wed, 10 Oct 2012 20:12:36 -0700 Tim Roberts wrote: > Jean-Michel Pichavant wrote: > > > >I'm trying to generate C++ code from an XML file. I'd like to use a template > >engine, which imo produce something readable and maintainable. > >My google search about this subject has been quite unsuccessful, I've been > >redirected to template engine specific to html mostly. > > > >Does anybody knows a python template engine for generating C++ code ? > > I'm a big fan of Cheetah. It's simple but flexible enough to be useful. > Besides the many web projects I've done with it, I also I use it in one > project to generate PHP code (it generates data access objects from a live > database schema). > -- > Tim Roberts, [email protected] > Providenza & Boekelheide, Inc. > -- > http://mail.python.org/mailman/listinfo/python-list Also take a look at IDL, for a proper way to handle interface generation in C++. No python or cheetah required as by definition your interfaces should be portable. What your describing is more or less look like a hack or something which would be a pain to maintain without SWIG or something more suited for this purposes than XML. http://en.wikipedia.org/wiki/Interface_description_language http://en.wikipedia.org/wiki/SWIG -- http://mail.python.org/mailman/listinfo/python-list
Re: serialization and versioning
On Fri, 12 Oct 2012 06:42:03 -0400 Neal Becker wrote: > I wonder if there is a recommended approach to handle this issue. > > Suppose objects of a class C are serialized using python standard pickling. > Later, suppose class C is changed, perhaps by adding a data member and a new > constructor argument. > > It would see the pickling protocol does not directly provide for this - but > is > there a recommended method? > > I could imagine that a class could include a class __version__ property that > might be useful - although I would further expect that it would not have been > defined in the original version of class C (but only as an afterthought when > it > became necessary). > > -- > http://mail.python.org/mailman/listinfo/python-list i guess a easy answer is to say to try python 3.3 but how would this translate in python (2) code ? -- http://mail.python.org/mailman/listinfo/python-list
Re: How to use "while" within the command in -c option of python?
On Fri, 12 Oct 2012 15:51:19 -0700
Herman wrote:
> python -c "import os; while True: print('hello')"
> File "", line 1
> import os; while True: print('hello')
> ^
> SyntaxError: invalid syntax
> --
> http://mail.python.org/mailman/listinfo/python-list
You get a syntax error since you didn't used tabs indents in your string
which is normal for python AFAIK.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Feedback on my python framework I'm building.
On Fri, 12 Oct 2012 21:49:55 -0700 (PDT) [email protected] wrote: > http://giotto.readthedocs.org/en/latest/tutorial.html > > Can someone give me some feedback on what they think of this framework? I > came up with the idea of this framework a few months ago. I gave a talk at a > local python user group regarding these ideas, but no one seemed to think I > was onto anything special or useful. > > Basically its a framework that forces the developer(s) to strictly separate > the model from the view and controller. You can 'hook up' multiple > controllers to a project. The model layer can be completely mocked out so > front end designers don't have to bother setting up > Postgres/rabbitmq/whatever. > > Does anyone have any throughts or feedback? Docs are useless for developers with an quarter of imagination. :-) -- Etienne Robillard Green Tea Hackers Club Fine Software Carpentry For The Rest Of Us! http://gthc.org/ [email protected] -- http://mail.python.org/mailman/listinfo/python-list
Re: readline trick needed
On 13 Oct 2012 13:30:14 GMT Steven D'Aprano wrote: > I'm working with the readline module, and I'm trying to set a key > combination to process the current command line by calling a known > function, *and* enter the command line. > > Something along the lines of: > > * execute function spam() in some context where it can access > the current command line as a string > * enter the command line > > Function spam() may or may not modify the command line. > > Here is what I have got so far: I can discard the current line and call a > function: > > readline.parse_and_bind(r'"\C-p": "%cspam()\n"' % 0x15) # ^U > > binds ctrl-P to the key combinations `ctrl-U spam() Enter`, which clears > the command line before entering spam(). > > If I leave out the ctrl-U, I'll get a SyntaxError or other exception, > e.g. command line `x = 123` gets transformed into `x = 123spam()`. > > > This is not suitable: > > readline.parse_and_bind(r'"\C-p": "; spam()\n"') > > because it changes the command line. It's okay for spam() itself to > modify the command line, but the key binding should not. > > I tried to do this: > > readline.parse_and_bind(r'"\C-p": "\nspam()\n"') > > but it gives me a segmentation fault, which is a little less helpful than > I had expected. > > This Stackoverflow question suggests that what I want is not possible in > vanilla Python: > > http://stackoverflow.com/questions/11680356 > > > but I'm a stubborn guy and I have not given up yet. Any suggestions? > > > (P.S. I'm aware of IPython, I want to get this working in the standard > CPython interpreter.) > > > -- > Steven > -- > http://mail.python.org/mailman/listinfo/python-list Why dont you grow yourself some usable neurons instead ? Don't you realize now stackoverflow.com is starting to hurt your capacity to cogitate on your own or have you not realized this yet? Cheers, Etienne -- Etienne Robillard Green Tea Hackers Club Fine Software Carpentry For The Rest Of Us! http://gthc.org/ [email protected] -- http://mail.python.org/mailman/listinfo/python-list
Re: readline trick needed
On Sun, 14 Oct 2012 00:47:52 +1100 Chris Angelico wrote: > Excuse me? > > I'm not overly familiar with readline, so perhaps there is a really > obvious way to do what Steven's trying to do, but this post does not > appear to be the result of a lack of thinking. > > If it really IS that obvious to you, post a link to appropriate > documentation without the rudeness... that way it'll be useful to > everyone, not just cathartic to you. > > ChrisA > -- > http://mail.python.org/mailman/listinfo/python-list whatever. i don't feel much like replying to idiots today so your apologies and useless if not irrelevant. -- Etienne Robillard Green Tea Hackers Club Fine Software Carpentry For The Rest Of Us! http://gthc.org/ [email protected] -- http://mail.python.org/mailman/listinfo/python-list
Re: Feedback on my python framework I'm building.
On Sun, 14 Oct 2012 01:12:30 +1100 Chris Angelico wrote: > On Sat, Oct 13, 2012 at 3:49 PM, wrote: > > Basically its a framework that forces the developer(s) to strictly separate > > the model from the view and controller. You can 'hook up' multiple > > controllers to a project. The model layer can be completely mocked out so > > front end designers don't have to bother setting up > > Postgres/rabbitmq/whatever. > > I don't like frameworks that force too much. Actually, I guess that > means I don't like frameworks at all, I like toolsets. Let the > programmer decide what he wants to do. > > That said, though, there are times when a good framework can do 90% of > your work for you. The trouble comes when you want to do something the > author didn't think of - you might well end up either fighting against > the system, or modifying the framework to suit your task (and that > works only if you created it yourself). Thin frameworks are usually > immune to this, but on the flip side, they're less likely to be doing > most of your work for you. > > It's really easy to demo something and show how awesome it is. How > easily can it be turned to a task it was never built for? Perhaps we deserve such lame/mediocre web frameworks after all, as Imagination is for those who deserve the freedom to use it properly. > ChrisA > -- > http://mail.python.org/mailman/listinfo/python-list -- Etienne Robillard Green Tea Hackers Club Fine Software Carpentry For The Rest Of Us! http://gthc.org/ [email protected] -- http://mail.python.org/mailman/listinfo/python-list
Re: Aggressive language on python-list
On Sat, 13 Oct 2012 11:21:28 -0400 Zero Piraeus wrote: > : > > Not sure exactly how to put this ... > > I'm a mostly passive subscriber to this list - my posts here over the > years could probably be counted without having to take my socks off - > so perhaps I have no right to comment, but I've noticed a marked > increase in aggressive language here lately, so I'm putting my head > above the parapet to say that I don't appreciate it. > > Robust disagreement is one thing [and can be quite enjoyable for those > of us merely spectating], but there's really no need to go around > calling people idiots at the drop of a hat. Quite apart from anything > else, when the contributor you're calling names is as helpful and > knowledgeable a member of the community as some of those targeted have > been, it makes you look a bit daft. > > And, yes, I know bringing it up could be construed as stoking the > flames ... but, well, "silence = acquiescence" and all that. > > -[]z. > -- > http://mail.python.org/mailman/listinfo/python-list OT. you obviously has no clue what agressive behavior mean. :-) So please continue with the passive tone saying nothing relevant and login to facebook. -- Etienne Robillard Green Tea Hackers Club Fine Software Carpentry For The Rest Of Us! http://gthc.org/ [email protected] -- http://mail.python.org/mailman/listinfo/python-list
Re: Feedback on my python framework I'm building.
On Sat, 13 Oct 2012 08:57:47 -0700 (PDT) [email protected] wrote: > Do you have an example of a task that giotto can't handle that other > frameworks can? One of my goals is to have this framework "turing complete" > in the sense that everything that other frameworks can do, giotto should be > able to do. I think my controller -> input_middleware -> model -> cache -> > view -> output_middleware pattern pretty much allows for anything. Throughout > the development process, I've radically changed the API many times when it > was discovered that a certain task would be very hard (for instance > authentication) > -- > http://mail.python.org/mailman/listinfo/python-list how about eliminating poverty and employment discrimination? Or can your script prevent childs to kill themselves on facebook? let me know! -- http://mail.python.org/mailman/listinfo/python-list
Re: Uliweb release 0.1 version
Flexible is more English than "flexiable" i guess... :-) Besides the typing errors I look forward in checking this out for new concepts... Cheers and congratulations, Etienne -- Etienne Robillard Occupation: Software Developer Company:Green Tea Hackers Club Email: [email protected] Website:gthcfoundation.org Skype ID: incidah -- http://mail.python.org/mailman/listinfo/python-list
How to create a python extension module from a shared library?
Hi all, I would like to extend uWSGI by creating a CPython extension module for the libuwsgi.so shared library included in the distribution. My goal is to use this automatically generated CPython module for extending uWSGI. I have summarized my current approach here: https://mail.python.org/pipermail/tutor/2017-December/112475.html So far, I have tried to use CFFI and pycparser to generate the Python bindings, however CFFI and pycparser doesn't handle C directives like #include and #define. I also attempted using clang to preprocess the "uwsgi.h" header found in the uWSGI distro with pycparser.parse_file() to generate a AST, however the generated AST object seems incorrect. Is there any way to reflect a shared library into a CPython module? I know that using "nm -D libuwsgi.so" I can get a list of available functions for this module. However, I have very little experience with ctypes and CFFI. Do I need to modify CFFI to allow it to parse a C header with libclang? I'm guessing CFFI/clang or ctypes should allow me to reflect the shared library, but I'm not sure about which method is the most appropriate. What do you think? Sincerely, Etienne -- Etienne Robillard [email protected] https://www.isotopesoftware.ca/ -- https://mail.python.org/mailman/listinfo/python-list
Re: How to create a python extension module from a shared library?
So far I found the SWIG method quite good. But I really think that I will try patching CFFI to use libclang natively. Etienne Le 2017-12-29 à 20:00, Etienne Robillard a écrit : Hi all, I would like to extend uWSGI by creating a CPython extension module for the libuwsgi.so shared library included in the distribution. My goal is to use this automatically generated CPython module for extending uWSGI. I have summarized my current approach here: https://mail.python.org/pipermail/tutor/2017-December/112475.html So far, I have tried to use CFFI and pycparser to generate the Python bindings, however CFFI and pycparser doesn't handle C directives like #include and #define. I also attempted using clang to preprocess the "uwsgi.h" header found in the uWSGI distro with pycparser.parse_file() to generate a AST, however the generated AST object seems incorrect. Is there any way to reflect a shared library into a CPython module? I know that using "nm -D libuwsgi.so" I can get a list of available functions for this module. However, I have very little experience with ctypes and CFFI. Do I need to modify CFFI to allow it to parse a C header with libclang? I'm guessing CFFI/clang or ctypes should allow me to reflect the shared library, but I'm not sure about which method is the most appropriate. What do you think? Sincerely, Etienne -- Etienne Robillard [email protected] https://www.isotopesoftware.ca/ -- https://mail.python.org/mailman/listinfo/python-list
Re: How to create a python extension module from a shared library?
Hi James, Thanks for your input. I want to make a native Python application (uwsgictl) to dispatch FIFO commands to the master uWSGI process. uwsgictl would depend on libuwsgi.so, a uWSGI plugin included in the distribution. My goal is to use CFFI to generate python bindings for libuwsgi.so using clang interpreter. :) Best regards, Etienne Le 2018-01-02 à 05:06, James Chapman a écrit : Hi Etienne, I'm not familiar with uSWGI, so I started here: https://uwsgi-docs.readthedocs.io/en/latest/Hooks.html AFAIK there are a number of hooks already exposed and available to Python apps. However, if I've understood your question correctly, you want to extend uWSGI itself by writing some python code which is then translated/generated into C code? You refer to AST generation but why are you doing this? Maybe my unfamiliarity with uWSGI is preventing me from understanding exactly what it is you are trying to do, but further clarification would go a long way. If your goal is actually to extend uWSGI via a plugin, then there's more to just writing some python, you'll need to write some C code (I wouldn't bother trying to generate this as it'll probably not work): https://github.com/unbit/uwsgi-docs/blob/master/tutorials/WritingPlugins.rst If you look here, there's a link to each 3rd party plugin (mostly GitHub) where you can see the source code for each plugin. This is a very good starting point as there are many examples here: https://uwsgi-docs.readthedocs.io/en/latest/ThirdPartyPlugins.html Perhaps you can clarify. James On 30 December 2017 at 01:00, Etienne Robillard <mailto:[email protected]>> wrote: Hi all, I would like to extend uWSGI by creating a CPython extension module for the libuwsgi.so shared library included in the distribution. My goal is to use this automatically generated CPython module for extending uWSGI. -- Etienne Robillard [email protected] https://www.isotopesoftware.ca/ -- https://mail.python.org/mailman/listinfo/python-list
Re: How to create a python extension module from a shared library?
Here's my ffi_build.py script: https://bitbucket.org/tkadm30/libuwsgictl/src/956f6ca24f9111c1d8b3ce90cf17173a6e5ae3e2/ffi_build.py?at=default&fileviewer=file-view-default Not really sure how to preprocess the uwsgi.h header with clang.cindex in cffi. I can load the shared lib with : lib = clang.cindex.cdll.LoadLibrary(c_lib) Any ideas how to compile lib into a python extension module with cffi? Etienne Le 2018-01-02 à 05:17, Etienne Robillard a écrit : Hi James, Thanks for your input. I want to make a native Python application (uwsgictl) to dispatch FIFO commands to the master uWSGI process. uwsgictl would depend on libuwsgi.so, a uWSGI plugin included in the distribution. My goal is to use CFFI to generate python bindings for libuwsgi.so using clang interpreter. :) Best regards, Etienne Le 2018-01-02 à 05:06, James Chapman a écrit : Hi Etienne, I'm not familiar with uSWGI, so I started here: https://uwsgi-docs.readthedocs.io/en/latest/Hooks.html AFAIK there are a number of hooks already exposed and available to Python apps. However, if I've understood your question correctly, you want to extend uWSGI itself by writing some python code which is then translated/generated into C code? You refer to AST generation but why are you doing this? Maybe my unfamiliarity with uWSGI is preventing me from understanding exactly what it is you are trying to do, but further clarification would go a long way. If your goal is actually to extend uWSGI via a plugin, then there's more to just writing some python, you'll need to write some C code (I wouldn't bother trying to generate this as it'll probably not work): https://github.com/unbit/uwsgi-docs/blob/master/tutorials/WritingPlugins.rst If you look here, there's a link to each 3rd party plugin (mostly GitHub) where you can see the source code for each plugin. This is a very good starting point as there are many examples here: https://uwsgi-docs.readthedocs.io/en/latest/ThirdPartyPlugins.html Perhaps you can clarify. James On 30 December 2017 at 01:00, Etienne Robillard <mailto:[email protected]>> wrote: Hi all, I would like to extend uWSGI by creating a CPython extension module for the libuwsgi.so shared library included in the distribution. My goal is to use this automatically generated CPython module for extending uWSGI. -- Etienne Robillard [email protected] https://www.isotopesoftware.ca/ -- https://mail.python.org/mailman/listinfo/python-list
Re: How to create a python extension module from a shared library?
Hi James, Le 2018-01-02 à 09:30, James Chapman a écrit : What starts uWSGI? Is it started from a Python application or a webserver? Apologies for my lack of knowledge RE uWSGI. uWSGI is typically managed by the web server. > Any ideas how to compile lib into a python extension module with cffi? No. Do you mean python binary extension? (https://packaging.python.org/guides/packaging-binary-extensions/) Or Python package? (https://python-packaging.readthedocs.io/en/latest/) I just want CFFI to work with clang to produce automagically the python bindings for my app. James Best regards, Etienne -- Etienne Robillard [email protected] https://www.isotopesoftware.ca/ -- https://mail.python.org/mailman/listinfo/python-list
Re: How to create a python extension module from a shared library?
Hi James, Part of the problem is because the CFFI and uWSGI developers aren't interested to support this. I need to modify CFFI to support preprocessing C headers with clang.cindex myself. I also need to make sure its possible to attach my Python script to the master uWSGI process to dispatch FIFO commands. Clang is needed because CFFI doesn't support preprocessing C headers with #define or #include directives. Best regards, Etienne Le 2018-01-02 à 10:45, James Chapman a écrit : Again, apologies if I've dumbed this down, but if I understand this all correctly... The webserver starts uWSGI, the python application running within then does stuff via uWSGI. You want one of those things to be handled by a uWSGI extension that has been written in python, with CFFI, rather than in pure C. In other words, CFFI is going to generate the C part of the extension. Unfortunately I'm no CFFI expert, but the above sounds like something that might be better broken down into 2 parts. 1) Writing uWSGI extensions in Python and posted to a uWSGI development forum. 2) Generating python bindings for C calls with CFFI and clang and posted to a CFFI forum. (Why clang? GCC/G++ will work just as well and may be the default). Hope that helps. James -- Etienne Robillard [email protected] https://www.isotopesoftware.ca/ -- https://mail.python.org/mailman/listinfo/python-list
Re: How to create a python extension module from a shared library?
Hi James, I would love to write in C but I much prefer coding in Python :) I was thinking that I could use Cython to dlopen the shared library dynamically with ctypes. No need to compile anything. Best regards, Etienne Le 2018-01-03 à 05:30, James Chapman a écrit : In my opinion, just write your extension in C following the traditional extension development guidelines unless you plan to actively maintain and contribute your changes in the relevant projects (CFFI and uWSGI). Assuming you get this to work, you're going to have to support it and bug fix it. Something that initially will not be too difficult because it will all be fresh in your head, but 6 months down the line when you want to change something or discover a bug it's going to be very difficult. Will you be able to attach a debugger and step through the code? I've been down the route of trying to do something clever and even succeeded. I then later regretted it, because what seemed like a good idea at the time turned out to be a PITA to support. Go with what's supported, go with what's documented, don't modify core components if you don't absolutely have to, because you'll have to modify those components with each update and ultimately you just end up generating work for yourself. James On 2 January 2018 at 21:21, Etienne Robillard <mailto:[email protected]>> wrote: Hi James, Part of the problem is because the CFFI and uWSGI developers aren't interested to support this. I need to modify CFFI to support preprocessing C headers with clang.cindex myself. I also need to make sure its possible to attach my Python script to the master uWSGI process to dispatch FIFO commands. Clang is needed because CFFI doesn't support preprocessing C headers with #define or #include directives. Best regards, Etienne -- Etienne Robillard [email protected] https://www.isotopesoftware.ca/ -- https://mail.python.org/mailman/listinfo/python-list
Progress migrating cffi and pycparser to libclang
Hi,
I will be creating a repository for this:
https://bitbucket.org/tkadm30/cffi-libclang
The goal is to generate a AST object from a C header by preprocessing
with clang -E then compile the python bindings with CFFI...
ffi.cdef(open('uwsgi.h').read()) # <-- XXX need to modify internal
parsing code to use clang.cindex!
ffi.dlopen('/usr/local/lib/libuwsgi.so')
Right now the parsing of the C source file is done by pycparser, but i
would like to use the clang.cindex module.
What do you think?
Etienne
--
Etienne Robillard
[email protected]
https://www.isotopesoftware.ca/
--
https://mail.python.org/mailman/listinfo/python-list
Re: Progress migrating cffi and pycparser to libclang
Hi Paul, Le 2018-01-04 à 06:41, Paul Moore a écrit : Presumably that will introduce a dependency on some clang module? You mention clang.cindex - but the only clang package I can find on PyPI says "OBSOLETE. LLVM-CLANG NOW PUBLISHES PYTHON PACKAGE. JUST ADD THE OFFICIAL llvm-3.7 repo in your apt." but doesn't provide binaries or explain how to install clang on, say, Windows (to pick an example relevant to me :-)). I'm targeting the Linux/x86 machine class. On debian systems, it's pretty easy to install clang and the python bindings with apt-get. :-) As a fork/extension for cffi, I have no particular opinion (I'm unlikely to ever use it). But the advantage of pycparser is that it's cross-platform and pure Python, so I doubt this will be acceptable for inclusion into CFFI itself. CFFI/pycparser definitely need to be patched to support parsing standard C directives like #define and #include in the ffi.cdef() function. The easiest solution is to migrate the internal parsing code to libclang, a state-of-the art C/C++ compiler based on LLVM. Best regards, Etienne -- Etienne Robillard [email protected] https://www.isotopesoftware.ca/ -- https://mail.python.org/mailman/listinfo/python-list
Fwd: Re: Progress migrating cffi and pycparser to libclang
Forwarding this thread to the CFFI developers... Re Paul: Thanks for your feedback. My intended audience are developers who can use hg to fetch/build source code without pip. Best regards, Etienne Message transféré Sujet : Re: Progress migrating cffi and pycparser to libclang Date : Thu, 4 Jan 2018 21:25:27 + De :Paul Moore Pour : Etienne Robillard Copie à : Python On 4 January 2018 at 21:02, Etienne Robillard wrote: As a fork/extension for cffi, I have no particular opinion (I'm unlikely to ever use it). But the advantage of pycparser is that it's cross-platform and pure Python, so I doubt this will be acceptable for inclusion into CFFI itself. CFFI/pycparser definitely need to be patched to support parsing standard C directives like #define and #include in the ffi.cdef() function. The easiest solution is to migrate the internal parsing code to libclang, a state-of-the art C/C++ compiler based on LLVM. I would strongly object to adding a dependency to cffi that couldn't be automatically installed by pip as part of standard dependency resolution (i.e., a PyPI hosted Python project with wheels available for all common platforms - Linux, Mac OS and Windows). But ultimately if you're proposing this as a change to cffi, you should be getting the opinions of the cffi devs, not just asking on this list. (I notice you have posted to the cffi mailing list, but haven't had any response yet). Paul -- https://mail.python.org/mailman/listinfo/python-list
Spectre/Meltdown bug affecting Python ?
Hi all, What do you think about the latest Spectre/Meltdown security flaw found in Intel processors and Apple smartphones? Are Python 2.7 and 3.6 affected by speculative execution side-channel attacks when using the Linux kernel and Intel CPUs? Best regards, Etienne -- Etienne Robillard [email protected] https://www.isotopesoftware.ca/ -- https://mail.python.org/mailman/listinfo/python-list
Re: Spectre/Meltdown bug affecting Python ?
My understanding of this vulnerability is that speculative indirect calls in Linux kernel can be used to extract/filter memory content via side-channels. So, is it time to implement --enable-retpoline to CPython ? [1] Etienne 1. https://www.bleepingcomputer.com/news/google/google-unveils-new-retpoline-coding-technique-for-mitigating-spectre-attacks/ Le 2018-01-06 à 05:42, Etienne Robillard a écrit : Hi all, What do you think about the latest Spectre/Meltdown security flaw found in Intel processors and Apple smartphones? Are Python 2.7 and 3.6 affected by speculative execution side-channel attacks when using the Linux kernel and Intel CPUs? Best regards, Etienne -- Etienne Robillard [email protected] https://www.isotopesoftware.ca/ -- https://mail.python.org/mailman/listinfo/python-list
Re: Spectre/Meltdown bug affecting Python ?
Le 2018-01-06 à 15:49, J.O. Aho a écrit : On 01/06/18 13:43, Etienne Robillard wrote: My understanding of this vulnerability is that speculative indirect calls in Linux kernel can be used to extract/filter memory content via side-channels. Not just Linux, but all other OS:es, Microsoft and Apple been patching in secret as they have a closed source approach, but ms-windows needs at least one more patch before it can breath out, which will be released on Tuesday. It's unclear to me whether AMD CPUs are affected by theses design flaws. Furthermore, I'd like to know if Python can mitigate hardware-specific timing attacks. Best regards, Etienne -- Etienne Robillard [email protected] https://www.isotopesoftware.ca/ -- https://mail.python.org/mailman/listinfo/python-list
fun with cwrap: Unknown type kind 114
Dear Eli,
Can you please take a look at:
https://bitbucket.org/tkadm30/libuwsgictl/raw/048978bf2b51b1185302da98c5063978061293df/tests/cwrap/error
I'm playing around with cwrap: https://github.com/geggo/cwrap
Looks like this *gem* can generate Cython pxd files from C headers using
libclang. :)
Traceback (most recent call last):
File "/usr/local/bin/cwrap", line 46, in
ast_items = clang_parser.parse([('input.h', inputfile)], include_dirs, '')
File
"/usr/local/lib/python2.7/dist-packages/cwrap/frontends/clang/clang_parser.py",
line 889, in parse
parser.parse(cfile[0][0], include_dirs, language, unsaved_files=cfile)
File
"/usr/local/lib/python2.7/dist-packages/cwrap/frontends/clang/clang_parser.py",
line 123, in parse
self.parse_element(tu.cursor)
File
"/usr/local/lib/python2.7/dist-packages/cwrap/frontends/clang/clang_parser.py",
line 306, in parse_element
child = self.parse_element(c, level+1)
File
"/usr/local/lib/python2.7/dist-packages/cwrap/frontends/clang/clang_parser.py",
line 259, in parse_element
result = mth(cursor, level)
File
"/usr/local/lib/python2.7/dist-packages/cwrap/frontends/clang/clang_parser.py",
line 496, in visit_VAR_DECL
typ, id_ = self.type_to_c_ast_type(cursor.type, level)
File
"/usr/local/lib/python2.7/dist-packages/cwrap/frontends/clang/clang_parser.py",
line 167, in type_to_c_ast_type
level.show( 'in type to c_ast:', 'kind:', t.kind,
repr(t.get_declaration().spelling))
File
"/usr/local/lib/python2.7/dist-packages/cwrap/frontends/clang/clang/cindex.py",
line 1467, in kind
return TypeKind.from_id(self._kind_id)
File
"/usr/local/lib/python2.7/dist-packages/cwrap/frontends/clang/clang/cindex.py",
line 1407, in from_id
raise ValueError,'Unknown type kind %d' % id
ValueError: Unknown type kind 114
What do you think about this error?
I ran: $ cwrap -i . -i /usr/lib/llvm-3.8/lib/clang/3.8.1/include uwsgi.h
libuwsgi.pxd
Cheers,
Etienne
--
Etienne Robillard
[email protected]
https://www.isotopesoftware.ca/
--
https://mail.python.org/mailman/listinfo/python-list
How to embed a native JIT compiler to a django app?
Hi, I want to compile a Django application into a C source file and embed a JIT compiler into the binary. Is there any way of doing this with llvm/clang? Regards, Etienne -- Etienne Robillard [email protected] https://www.isotopesoftware.ca/ -- https://mail.python.org/mailman/listinfo/python-list
Re: How to embed a native JIT compiler to a django app?
Hi Stephane, Thank you for your reply. As you suggested, I believe my approach was too complex. So I decided to replace Cython with PyPy in order to enable experimental support of JIT compilation for my Python/Django web applications. Best regards, Etienne Le 2018-01-28 à 08:32, Stephane Wirtel via Python-list a écrit : On 01/27, Etienne Robillard wrote: Hi, I want to compile a Django application into a C source file and embed a JIT compiler into the binary. Is there any way of doing this with llvm/clang? Hi Etienne, I think no, Django will use Python and this one is interpreted. Answer, no... Now, you could try to create a wheel file with your django project and try that, but it is really complex. Why do you want to package a Django app into a C file with a compiler? -- Etienne Robillard [email protected] https://www.isotopesoftware.ca/ -- https://mail.python.org/mailman/listinfo/python-list
PyPy support breaking CPython compatibility?
Hi, I managed to patch Schevo and Durus to run under PyPy 5.9. However, I'm afraid the changes is breaking Python 2.7 compatibility. I'm not sure how I should distribute my changes to the respective projects. Since I decided to use more PyPy in my Django projects, should I drop Python 2.7 support and release the experimental code on PyPi ? What do you think? Etienne -- Etienne Robillard [email protected] https://www.isotopesoftware.ca/ -- https://mail.python.org/mailman/listinfo/python-list
Re: PyPy support breaking CPython compatibility?
Hi Chris, Le 2018-01-30 à 14:53, Chris Angelico a écrit : If you're supporting Python 3, I don't think there's any problem with saying "Python 2.7 support ceases as of Schevo v4.0, so if you need Py 2.7 use Schevo 3.x". (It's not as if the old versions will suddenly cease working or anything.) Fair enough. I'll gladly release libschevo and libdurus 4.0 very soon. :) Thank you, Etienne -- Etienne Robillard [email protected] https://www.isotopesoftware.ca/ -- https://mail.python.org/mailman/listinfo/python-list
Re: PyPy support breaking CPython compatibility?
Hi Ned, Le 2018-01-30 à 15:14, Ned Batchelder a écrit : I'm curious what you had to change for PyPy? (Unless it's a Py2/Py3 thing as Chris mentions.) Please take a look at the changesets: https://bitbucket.org/tkadm30/libschevo/commits/745d1aeab5c6ee0d336790cf13d16f327e10c2f8 https://bitbucket.org/tkadm30/libdurus/commits/875636e9b6caa840fd50ca87d69217d87fc06f43 In short, it seems PyPy automagically adds a __weakref__ attribute to __slots__, causing the CPython interpreter to raise a TypeError... Cheers, Etienne -- Etienne Robillard [email protected] https://www.isotopesoftware.ca/ -- https://mail.python.org/mailman/listinfo/python-list
Re: PyPy support breaking CPython compatibility?
Le 2018-01-30 à 16:38, Ned Batchelder a écrit :
I'm confused by this:
-if os.environ.get('SCHEVO_OPTIMIZE', '1') == '1':
+if os.environ.get('SCHEVO_OPTIMIZE', '1') == True:
I was also curious about this: when does os.environ.get return
anything but a string?
I was probably high when I coded this! ;)
I think a better solution is to write :
if os.environ.get('SCHEVO_OPTIMIZE', '0') == '1':
...
Anyways, what do you think about the weakref case?
Etienne
--
Etienne Robillard
[email protected]
https://www.isotopesoftware.ca/
--
https://mail.python.org/mailman/listinfo/python-list
Re: PyPy support breaking CPython compatibility?
Le 2018-01-31 à 05:21, Ned Batchelder a écrit : On 1/30/18 3:58 PM, Etienne Robillard wrote: Hi Ned, Le 2018-01-30 à 15:14, Ned Batchelder a écrit : I'm curious what you had to change for PyPy? (Unless it's a Py2/Py3 thing as Chris mentions.) Please take a look at the changesets: https://bitbucket.org/tkadm30/libschevo/commits/745d1aeab5c6ee0d336790cf13d16f327e10c2f8 https://bitbucket.org/tkadm30/libdurus/commits/875636e9b6caa840fd50ca87d69217d87fc06f43 In short, it seems PyPy automagically adds a __weakref__ attribute to __slots__, causing the CPython interpreter to raise a TypeError... PyPy and CPython are separate implementations, that never mix: you run your code in one, or in the other. How can PyPy do something at runtime (add a __weakref__ attribute to __slots__) that can cause CPython to do something at runtime (raise a TypeError)? Sorry Ned for the confusion. This is strictly a PyPy issue. CPython does not raise a TypeError. A small demonstration case would be very helpful for figuring out what is going on. Yes The problem is with the following code on PyPy 5.9: class MyClass(object): __slots__ = ['__weakref__',] --Ned. -- Etienne Robillard [email protected] https://www.isotopesoftware.ca/ -- https://mail.python.org/mailman/listinfo/python-list
What is django-hotsauce?
About Django-hotsauce: Scalable and high-performance WSGI microframework on top of Django and others: Django-hotsauce is the ultimate web development toolkit for rogue Python hackers and chronic weed users looking to build porn web sites in your mama basement! :) Typically used for advanced training of slackers, hackers, and unemployed people bored with life and looking to learn Schevo DBMS on PyPy. :) I think django-hotsauce is a great Python library for research/educational and experimental purpose. In specific, it is interesting to break Django ORM and use ZODB to develop your own models api... Anyways, have fun hacking django-hotsauce with PyPy. Hacking life is essential to happiness... :) Pragmatic hackers love to learn rogue ways to exploit Django api with JIT and PyPy. ;) cheers, Etienne -- Etienne Robillard [email protected] https://www.isotopesoftware.ca/ -- https://mail.python.org/mailman/listinfo/python-list
Re: What is django-hotsauce?
More like a platform to impress your cat with your JIT-powered web app :) Etienne Le 2018-02-02 à 10:02, Alexandre Brault a écrit : So it's a buzzword generator? Alex On 2018-02-02 04:17 AM, Etienne Robillard wrote: About Django-hotsauce: Scalable and high-performance WSGI microframework on top of Django and others: Django-hotsauce is the ultimate web development toolkit for rogue Python hackers and chronic weed users looking to build porn web sites in your mama basement! :) Typically used for advanced training of slackers, hackers, and unemployed people bored with life and looking to learn Schevo DBMS on PyPy. :) I think django-hotsauce is a great Python library for research/educational and experimental purpose. In specific, it is interesting to break Django ORM and use ZODB to develop your own models api... Anyways, have fun hacking django-hotsauce with PyPy. Hacking life is essential to happiness... :) Pragmatic hackers love to learn rogue ways to exploit Django api with JIT and PyPy. ;) cheers, Etienne -- Etienne Robillard [email protected] https://www.isotopesoftware.ca/ -- https://mail.python.org/mailman/listinfo/python-list
Re: What is django-hotsauce?
Django-hotsauce is a Python-powered library for pragmatic hackers looking to exploit django with JIT. A commercial edition is also available for pre-order: https://www.livestore.ca/product/django-hotsauce/ Cheers, Etienne Le 2018-02-02 à 10:02, Alexandre Brault a écrit : So it's a buzzword generator? Alex On 2018-02-02 04:17 AM, Etienne Robillard wrote: About Django-hotsauce: Scalable and high-performance WSGI microframework on top of Django and others: Django-hotsauce is the ultimate web development toolkit for rogue Python hackers and chronic weed users looking to build porn web sites in your mama basement! :) Typically used for advanced training of slackers, hackers, and unemployed people bored with life and looking to learn Schevo DBMS on PyPy. :) I think django-hotsauce is a great Python library for research/educational and experimental purpose. In specific, it is interesting to break Django ORM and use ZODB to develop your own models api... Anyways, have fun hacking django-hotsauce with PyPy. Hacking life is essential to happiness... :) Pragmatic hackers love to learn rogue ways to exploit Django api with JIT and PyPy. ;) cheers, Etienne -- Etienne Robillard [email protected] https://www.isotopesoftware.ca/ -- https://mail.python.org/mailman/listinfo/python-list
Benchmarking Django on PyPy with unittest?
Hi, is it possible to benchmark a django application with unittest module in order to compare and measure the speed/latency of the django orm with sqlite3 against ZODB databases? i'm interested in comparing raw sqlite3 performance versus ZODB (schevo). i would like to make specific testsuite(s) for benchmarking django 1.11.7, django 2.0, pypy, etc. What do you think? Etienne -- Etienne Robillard [email protected] https://www.isotopesoftware.ca/ -- https://mail.python.org/mailman/listinfo/python-list
Re: Benchmarking Django on PyPy with unittest?
Also, i need to isolate and measure the speed of gevent loop engine (gevent.monkey), epoll, and python-specific asyncio coroutines. :-) Etienne Le 2018-02-07 à 04:39, Etienne Robillard a écrit : Hi, is it possible to benchmark a django application with unittest module in order to compare and measure the speed/latency of the django orm with sqlite3 against ZODB databases? i'm interested in comparing raw sqlite3 performance versus ZODB (schevo). i would like to make specific testsuite(s) for benchmarking django 1.11.7, django 2.0, pypy, etc. What do you think? Etienne -- Etienne Robillard [email protected] https://www.isotopesoftware.ca/ -- https://mail.python.org/mailman/listinfo/python-list
Re: Benchmarking Django on PyPy with unittest?
Hi Dan, Thank you for your reply. I never used the requests module, but it looks like a very good pick to test the Django wsgi handler. :-) Currently my tests/benchmarks folder looks like this: tests/benchmarks/lib tests/benchmarks/lib/django_sqlite # Django 1.11 specific project directory tests/benchmarks/lib/django_sqlite/myapp # Test app for benchmarking templates, views, and core django api tests/benchmarks/lib/django_sqlite/polls # Tutorial app for benchmarking django orm/sqlite on pypy and cpython tests/benchmarks/uwsgi # Testsuite for uWSGI tests/benchmarks/django1 # Testsuite for Django 1.11 tests/benchmarks/django2 # Testsuite for Django 2 To run the benchmarks testsuite on pypy: $ cd tests $ source ./djangorc $ pypy ./run.py -C benchmarks/uwsgi # Run testsuite for the uwsgi handler $ pypy ./run.py -C benchmarks/django1 # Run testsuite for the django 1.11 api $ pypy ./run.py -C benchmarks/django2 # Run testsuite for the django 2.0 api Ideally, i would like to compile benchmarks data, but I have not yet understood how to do this. :-) Cheers, Etienne 2018-02-07 à 19:10, Dan Stromberg a écrit : You could probably use the "requests" module to time how long various operations take in your Django website. On Wed, Feb 7, 2018 at 2:26 AM, Etienne Robillard wrote: Also, i need to isolate and measure the speed of gevent loop engine (gevent.monkey), epoll, and python-specific asyncio coroutines. :-) Etienne Le 2018-02-07 à 04:39, Etienne Robillard a écrit : Hi, is it possible to benchmark a django application with unittest module in order to compare and measure the speed/latency of the django orm with sqlite3 against ZODB databases? i'm interested in comparing raw sqlite3 performance versus ZODB (schevo). i would like to make specific testsuite(s) for benchmarking django 1.11.7, django 2.0, pypy, etc. What do you think? Etienne -- Etienne Robillard [email protected] https://www.isotopesoftware.ca/ -- https://mail.python.org/mailman/listinfo/python-list -- Etienne Robillard [email protected] https://www.isotopesoftware.ca/ -- https://mail.python.org/mailman/listinfo/python-list
Re: Regex on a Dictionary
Hi Stanley, Le 2018-02-13 à 08:11, Stanley Denman a écrit : x=MyRegex.findall(MyDict) How about: x = [MyRegex.findall(item) for item in MyDict] Etienne -- Etienne Robillard [email protected] https://www.isotopesoftware.ca/ -- https://mail.python.org/mailman/listinfo/python-list
Django-hotsauce 1.0 commercial edition looking for beta testers!
Hello everyone, Django-hotsauce 1.0 commercial edition (LTS) is now available for preorder :) Checkout: https://www.livestore.ca/product/django-hotsauce/ I'm also looking for expert Django and Python programmers to test and review the design and API of Django-hotsauce 1.0 commercial edition in order to develop a low-cost, high-performance, Django alternative microframework to develop scalable web apps in Python and WSGI. Furthermore, i will always be happy to support a free version of Django-hotsauce in order to drive the development and research of Python/Django web programming. :) Final note, the Paypal payment gateway is experimental. I might one day decide to switch to monero or flashcoin... I'm not really sure it's a good idea yet. What do you think? Etienne -- Etienne Robillard [email protected] https://www.isotopesoftware.ca/ -- https://mail.python.org/mailman/listinfo/python-list
Re: Django-hotsauce 1.0 commercial edition looking for beta testers!
Hi Ned Le 2018-02-13 à 19:54, Ned Batchelder a écrit : There are a number of things that are confusing about this. You should state clearly what django-hotsauce is. You say "Django alternative microframework." Usually people use "Django" and "microframework" as opposites, not something you can combine together. Sorry for being unclear. I meant "Django-based alternative microframework". :-) I plan to commercialize a Django extension for allowing pragmatic and high-performance WSGI programming *on top* of Django. You say there will be a free version. Does it already exist? Link to it. https://www.isotopesoftware.ca/software/django-hotsauce/ https://pypi.python.org/pypi/Django-hotsauce/0.9.0 Finally, you seem to be asking experts to volunteer their time to help you design and test software that you will then sell. This seems unrealistic at best, and exploitive or unfair at worst. Yes. I'm really looking for serious people and/or companies to try/test/review django-hotsauce 1.0 commercial edition for *commercial purposes* in exchange of a small fee. --Ned. Best regards, Etienne -- Etienne Robillard [email protected] https://www.isotopesoftware.ca/ -- https://mail.python.org/mailman/listinfo/python-list
How to benchmark a HTTP connection with requests module?
Hi, Is it possible to emulate a range of concurrent connections to a http server with the requests module? import os import requests from test_support import unittest class HTTPBenchmarkTestCase(unittest.TestCase): url = 'http://localhost/benchmark/' def setUp(self): pass def test_benchmark_concurrency(self): # XXX it would be really nice to execute 10 concurrent requests # with 10 separated threads/connections. r = requests.get(self.url) self.assertEqual(r.status_code, 200) def tearDown(self): pass def runTest(self): pass What do you think? Etienne -- https://mail.python.org/mailman/listinfo/python-list
Re: How to benchmark a HTTP connection with requests module?
Hi Dennis, Nice pseudo code! :-) Is it possible benchmark/measure the latency of a HTTP connection for each threads by timing the duration of the requests.get method to complete? I also plan to test with gevent.monkey extension for enabling cooperative multithreading support: from gevent import monkey monkey.patch_all() Etienne Le 2018-02-15 à 11:56, Dennis Lee Bieber a écrit : Keyword: threads Create a number of threads, each handling one request, and use a global flag to start them. And maybe a queue to retrieve Pseudo-code (I haven't checked the manual for correct API): hold = True resultQ = Queue.Queue() def worker(ID): while hold: pass r = requests.get(...) resultQ.put( (ID, r.status_code) ) def benchmark(): requestTasks = [ threading.thread( worker, args=(ID) ) for ID in range(NUMBEROFREQUESTS) ] for rT in requestTasks.start() #or is it .run() #workers are now busy waiting for hold to False #better would be to use threading.Condition and .notifyAll() #having each thread wait on a global condition variable, #rather than spinning on a busy wait hold = False for _ in range(NUMBEROFREQUESTS): (ID, code) = resultQ.get() requestTasks[ID].join() #put any asserts here -- Etienne Robillard [email protected] https://www.isotopesoftware.ca/ -- https://mail.python.org/mailman/listinfo/python-list
Re: How to benchmark a HTTP connection with requests module?
Hi Dennis, Here's my code so far: import os import requests import threading import time from Queue import Queue from test_support import unittest test_queue = Queue() def worker(ident, url, queue): while True: start = time.clock() r = requests.get(url) end = time.clock() queue.put((ident, r.status_code, end - start)) queue.task_done() return queue class HTTPBenchmarkTestCase(unittest.TestCase): url = 'http://localhost/benchmark/' threads = 5 def setUp(self): self.hold = True def test_benchmark_concurrency(self): for i in range(self.threads): t = threading.Thread(target=worker, args=(i, self.url, test_queue)) t.daemon = True t.start() obj = test_queue.get() print obj test_queue.join() erob@marina:~/src/django-hotsauce-0.9/tests$ pypy ./run.py -C benchmarks DEBUG:Starting new HTTP connection (1): localhost DEBUG:http://localhost:80 "GET /benchmark/ HTTP/1.1" 200 76 DEBUG:Starting new HTTP connection (1): localhost (0, 200, 0.1299950349943) DEBUG:Starting new HTTP connection (1): localhost DEBUG:http://localhost:80 "GET /benchmark/ HTTP/1.1" 200 76 DEBUG:Starting new HTTP connection (1): localhost (1, 200, 0.05324292899606) DEBUG:Starting new HTTP connection (1): localhost DEBUG:http://localhost:80 "GET /benchmark/ HTTP/1.1" 200 76 DEBUG:Starting new HTTP connection (1): localhost (0, 200, 0.1122124810025) DEBUG:Starting new HTTP connection (1): localhost DEBUG:http://localhost:80 "GET /benchmark/ HTTP/1.1" 200 76 DEBUG:Starting new HTTP connection (1): localhost (1, 200, 0.1207582250033) DEBUG:Starting new HTTP connection (1): localhost DEBUG:http://localhost:80 "GET /benchmark/ HTTP/1.1" 200 76 DEBUG:Starting new HTTP connection (1): localhost DEBUG:http://localhost:80 "GET /benchmark/ HTTP/1.1" 200 76 DEBUG:Starting new HTTP connection (1): localhost (0, 200, 0.1259027660028) . -- Ran 1 test in 0.468s OK What do you think? :-) Etienne Le 2018-02-16 à 10:58, Dennis Lee Bieber a écrit : On Fri, 16 Feb 2018 06:22:04 -0500, Etienne Robillard declaimed the following: Hi Dennis, Nice pseudo code! :-) Is it possible benchmark/measure the latency of a HTTP connection for each threads by timing the duration of the requests.get method to complete? Wrap each .get with calls for start time and end time? (Need to check if time.time or time.clock is more precise on your hardware, docs recommend time.clock for benchmarking) def worker(ID): while hold: pass st = time.clock() r = requests.get(...) en = time.clock() resultQ.put( (ID, r.status_code, en - st) ) #return duration as third parameter for _ in range(NUMBEROFREQUESTS): (ID, code, duration) = resultQ.get() requestTasks[ID].join() -- Etienne Robillard [email protected] https://www.isotopesoftware.ca/ -- https://mail.python.org/mailman/listinfo/python-list
Python to Julia code generator?
Hi, Would it be possible to build a Python to Julia code generator?? i'm interested to learn Julia and would love to have the capacity to embed or run native Python code in Julia.. Thx Etienne -- Etienne Robillard [email protected] https://www.isotopesoftware.ca/ -- https://mail.python.org/mailman/listinfo/python-list
Re: Python to Julia code generator?
I found this: https://github.com/JuliaPy/PyCall.jl Looks pretty awesome already! :-) Thx E Le 2018-02-21 à 05:04, Steven D'Aprano a écrit : On Wed, 21 Feb 2018 04:13:56 -0500, Etienne Robillard wrote: Hi, Would it be possible to build a Python to Julia code generator?? i'm interested to learn Julia and would love to have the capacity to embed or run native Python code in Julia.. http://blog.leahhanson.us/post/julia/julia-calling-python.html -- Etienne Robillard [email protected] https://www.isotopesoftware.ca/ -- https://mail.python.org/mailman/listinfo/python-list
Re: Python to Julia code generator?
Le 2018-02-21 à 05:27, Chris Angelico a écrit : """If you find that your production code is too slow because you’re using mutual recursion between nine different languages, blame Dan Luu for this terrible idea.""" I have... NEVER gone as far as nine. That takes the cake. In fact, I don't recall ever going beyond three. At least, not in production... ChrisA Dude that is advanced stuff... :-) Etienne -- Etienne Robillard [email protected] https://www.isotopesoftware.ca/ -- https://mail.python.org/mailman/listinfo/python-list
Django-hotsauce 0.9.1 release
Hi everyone, I'm glad to announce the release of Django-hotsauce 0.9.1: https://www.isotopesoftware.ca/pub/django-hotsauce/django-hotsauce-0.9.1.tar.gz This is primarly a bugfix release for PyPy. Users of django-hotsauce are encouraged to test this release (with PyPy or Python 3) and submit bug reports here: https://bitbucket.org/tkadm30/django-hotsauce/issues For any questions or comments related to this project, please join the mailing list: https://groups.google.com/forum/#!forum/django-hotsauce Best regards, Etienne -- Etienne Robillard [email protected] https://www.isotopesoftware.ca/ -- https://mail.python.org/mailman/listinfo/python-list
Re: How to make Python run as fast (or faster) than Julia
So, how exactly can PyPy and JIT runs multithreaded Python applications
any faster than Julia on distributed systems?
Right now I think PyPy and JIT can run Python code on my old ia32
computer faster than with Python/Cython.
How do Julia scale on x86 machines ?
Etienne
Le 2018-02-22 à 11:00, Steven D'Aprano a écrit :
On Thu, 22 Feb 2018 12:03:09 +, bartc wrote:
The idea of the Fibonacci benchmark is to test how effectively an
implementation manages large numbers of recursive function calls. Then,
fib(36) would normally involve 48,315,633 calls.
This version does only 37, giving a misleading impression.
Who cares if it only does 37? There is *absolutely no benefit* to those
additional 48,315,596 calls, and thanks to the benchmark I discovered
that.[1]
I want to know what is the fastest implementation of Fibonacci I can
write. I'm not married to the idea that I have to make a ludicrous 48
million function calls just to get the 36th Fibonacci number.
(It then goes on to suggest using 'numba', and using its JIT compiler,
and using on that on an /iterative/ version of fib(). Way to miss the
point.
Indeed, you certainly have.
It might be a technique to bear in mind, but it is nonsensical to say
this gives a 17,000 times speed-up over the original code.
That's *exactly what it did*.
How many years, decades, have you been programming? Have you not realised
yet that the best way to optimize code is to pick the fastest algorithm
you can that will do the job? There's no law that says because you
started with a slow, inefficient algorithm you have to stay with it.
Here's another speed-up I found myself, although it was only 50 times
faster, not 17,000: just write the code in C, and call it via
os.system("fib.exe").
Did you include the time taken to capture the text output from stdout,
parse it, and convert to an int?
Did your C code support numbers greater than 2**64? My fibonacci function
can calculate the 10,000th Fibonacci number in a blink of the eye:
py> fibi(1)
336447648764317832666216120051075...976171121233066073310059947366875
Output truncated for brevity, it is a 2090-digit number.
Admittedly it did take about a minute and a half to generate all 208988
digits of the one millionth Fibonacci number, but the performance is fast
enough for my needs.
[1] Actually I already knew it, but then I didn't perform these
benchmarks, I'm just linking to them.
--
Etienne Robillard
[email protected]
https://www.isotopesoftware.ca/
--
https://mail.python.org/mailman/listinfo/python-list
do_handshake_on_connect should not be specified for non-blocking sockets
How can i fix this error?
I'm using gevent/uwsgi and python 2.7.13...
Looks like ssl.py really needs monkey-patching to work with
gevent/uwsgi, but i'm not sure exactly where/what triggers this error.
Any thoughts?
Etienne
File "/home/www/isotoperesearch.ca/trunk/www-bin/dispatch-django.uwsgi",
line 17, in application
return wsgi_app(environ, start_response)
File
"/home/erob/src/django-hotsauce-oauthclient/lib/wsgi_oauth2/middleware.py",
line 228, in __call__
code)
File
"/home/erob/src/django-hotsauce-oauthclient/lib/wsgi_oauth2/client.py",
line 114, in request_access_token
data=urlencode(form).encode('utf-8'))
File "/usr/lib/python2.7/urllib2.py", line 154, in urlopen
return opener.open(url, data, timeout)
File "/usr/lib/python2.7/urllib2.py", line 429, in open
response = self._open(req, data)
File "/usr/lib/python2.7/urllib2.py", line 447, in _open
'_open', req)
File "/usr/lib/python2.7/urllib2.py", line 407, in _call_chain
result = func(*args)
File "/usr/lib/python2.7/urllib2.py", line 1241, in https_open
context=self._context)
File "/usr/lib/python2.7/urllib2.py", line 1195, in do_open
h.request(req.get_method(), req.get_selector(), req.data, headers)
File "/usr/lib/python2.7/httplib.py", line 1042, in request
self._send_request(method, url, body, headers)
File "/usr/lib/python2.7/httplib.py", line 1082, in _send_request
self.endheaders(body)
File "/usr/lib/python2.7/httplib.py", line 1038, in endheaders
self._send_output(message_body)
File "/usr/lib/python2.7/httplib.py", line 882, in _send_output
self.send(msg)
File "/usr/lib/python2.7/httplib.py", line 844, in send
self.connect()
File "/usr/lib/python2.7/httplib.py", line 1263, in connect
server_hostname=server_hostname)
File "/usr/lib/python2.7/ssl.py", line 363, in wrap_socket
_context=self)
File "/usr/lib/python2.7/ssl.py", line 610, in __init__
raise ValueError("do_handshake_on_connect should not be specified
for non-blocking sockets")
ValueError: do_handshake_on_connect should not be specified for
non-blocking sockets
--
https://mail.python.org/mailman/listinfo/python-list
Detection of ultrasonic side channels in mobile devices with Python?
Hi, I would like to know if its possible to detect or decode ultrasonic signals in mobiles devices like Android with Python? For a introduction to ultrasonic tracking, see: https://www.wired.com/2016/11/block-ultrasonic-signals-didnt-know-tracking/ Thank you, Etienne -- Etienne Robillard [email protected] https://www.isotopesoftware.ca/ -- https://mail.python.org/mailman/listinfo/python-list
Re: Detection of ultrasonic side channels in mobile devices with Python?
Hey, I want to know why this question is being silently ignored by this group. I think it is essential to develop cryptoanalysis tools (like softwares) to detect and block ultrasonic signals in mobile devices. Using Python to detect or block ultrasonic side channels would be very interesting. I want to know what exactly do I need to develop open source softwares for analyzing and decoding ultrasonic side channels in smart tvs, mobile phones, and computers in Python. Please speak your mind out... Tell me how exactly ultrasonic side channels may activate remotely specific neural pathways implicated in aggressivity and how to block theses specific side channels from neuromodulating human behavior. Thank you. Etienne Le 2018-02-26 à 05:25, Etienne Robillard a écrit : Hi, I would like to know if its possible to detect or decode ultrasonic signals in mobiles devices like Android with Python? For a introduction to ultrasonic tracking, see: https://www.wired.com/2016/11/block-ultrasonic-signals-didnt-know-tracking/ Thank you, Etienne -- Etienne Robillard [email protected] https://www.isotopesoftware.ca/ -- https://mail.python.org/mailman/listinfo/python-list
Re: Detection of ultrasonic side channels in mobile devices with Python?
If ultrasonic side channels are a threat to privacy, it is because smart phones were designed to exploit this precise feature. A great number of studies have shown that ultrasonic neuromodulation of the central nervous system can be exploited via brain-computer interfaces... It is cutting edge science however, and my knowledge on techniques for activating human behavior by remote means is limited. I am only an amateur scientist/programmer with no experience whatsoever in Android or neuroscience. However, the possibility of ultrasonic side channels to be exploited by a third-party remotely without the user consent seem highly likely. I want to develop Python tools to provide experimental evidences that Android may exploit ultrasonic side channels to stimulate or impair the auditory cortex of someone. In other words, I plan to disclose the ability of the US government to use mobile devices as psychoenergetics (non-lethal) bioweapons. Etienne Le 2018-02-28 à 16:54, Chris Angelico a écrit : On Thu, Mar 1, 2018 at 8:49 AM, Ned Batchelder wrote: On 2/28/18 4:13 PM, Etienne Robillard wrote: I want to know why this question is being silently ignored by this group. If no one has any information about your topic, then no one will say anything. Python on Android is very specialized as it is, and I have no idea what ultrasonic side channels are, much less what it would take to detect or block them. There's no guarantee that any given question will get a response, much less a satisfactory answer. The silence is not malicious, you just haven't piqued anyone's interest enough for them to respond. I rather suspect that this is more an Android question than a Python one, so there's likely to be more interest in an Android community. But this part sounds like prime quality tinfoil hat material: Tell me how exactly ultrasonic side channels may activate remotely specific neural pathways implicated in aggressivity and how to block theses specific side channels from neuromodulating human behavior. Should be easy to find some whack-job newsgroups that would love to discuss that aspect of it. ChrisA -- Etienne Robillard [email protected] https://www.isotopesoftware.ca/ -- https://mail.python.org/mailman/listinfo/python-list
Re: Detection of ultrasonic side channels in mobile devices with Python?
What do rats find rewarding in play fighting? Le 2018-02-28 à 18:29, Chris Angelico a écrit : On Thu, Mar 1, 2018 at 10:23 AM, Steven D'Aprano wrote: I'd go further... what gave you the idea that ANYONE is an expert on ultrasonic activation of neural pathways? Definitely tinfoil hat material here; expertise is completely immaterial. ChrisA -- Etienne Robillard [email protected] https://www.isotopesoftware.ca/ -- https://mail.python.org/mailman/listinfo/python-list
psychocomputational analysis toolkit in Python?
Hi i want to develop a psychocomputational analysis toolkit in python for studying real-time neuroimaging data from focused ultrasound/brain-to-brain interfaces. Any suggestions where to look for cool projects in github? :) Best regards, Etienne -- Etienne Robillard [email protected] https://www.isotopesoftware.ca/ https://www.isotoperesearch.ca/ -- https://mail.python.org/mailman/listinfo/python-list
Re: psychocomputational analysis toolkit in Python?
Thanks for sharing this link. MNE looks pretty cool. :-) I'm mostly interested in mapping specific cortical pathways to brain-to-brain interfaces implicated in the ultrasonic neuromodulation of behavior. Would it be possible to emulate a minimally functional brain-to-brain coupling system entirely in Python? Best regards, Etienne Le 2018-03-08 à 09:44, Christopher Mullins a écrit : You might find MNE useful, and if what you're doing happens to fit somewhere in their package you could contribute to it -- they're a good group to work with. https://www.martinos.org/mne/stable/index.html https://github.com/mne-tools/mne-python -- Etienne Robillard [email protected] https://www.isotoperesearch.ca/ -- https://mail.python.org/mailman/listinfo/python-list
Re: psychocomputational analysis toolkit in Python?
OK I got it, thank you! :) Essentially, I want to develop a psychocomputational analysis toolkit in Python for studying real-time neuroimaging data (of cortical activity) from low-intensity, focused ultrasounds (LIFU) and possibly emulate a functional brain-to-brain coupling system on my 32-bit computer as a proof-of-concept to demonstrate the vulnerability of the human cochlear pathway to become a target of sophisticated ultrasonic side-channel attacks from mobile devices. Etienne Le 2018-03-08 à 10:31, Christopher Mullins a écrit : > Would it be possible to emulate a minimally functional brain-to-brain coupling system entirely in Python? I don't know what that would entail, but the links I shared have a mailing list and a very responsive gitter, both of which would be great places to ask about that! (You're welcome to ask here of course, but this being the general python mailing list, it's a shot in the dark.) Good luck! Chris -- Etienne Robillard [email protected] https://www.isotoperesearch.ca/ -- https://mail.python.org/mailman/listinfo/python-list
Thank you Python community!
You guys just made me realize something very obvious. :-) I'm in the process right now of watching the excellent documentary named "Drugs Inc." on Netflix and I'm basically stunned and deeply concerned about the major opioid epidemic in the US. I would like to thank you guys sincerely for helping a lot of people to stay clean, and focus on programming high-level stuff in Python instead of doing some really nasty drugs. I'm also wondering, could we exploit this strategy even further to help people willing to stop doing drugs by teaching them some stuff in Python? Murdering random drug dealers won't help out, i'm afraid. Neither do putting some of them in jail is sufficient to prevent people from trying out cocaine and heroin it seems. So, I would like to take this opportunity to advocate for this precise form of rehabilitation using open source software programming as a vector to make people use their brains in positive ways. The trick I think is to let people use programming as a way to socialize more in order to stimulate or distract their minds whiling keeping them away from drugs. It's time we regroup and combine our forces to help people who really needs our help. Sincerely, Etienne -- Etienne Robillard [email protected] https://www.isotopesoftware.ca/ -- https://mail.python.org/mailman/listinfo/python-list
Re: Thank you Python community!
Le 2018-03-19 à 15:21, Larry Martell a écrit : On Mon, Mar 19, 2018 at 12:08 PM, Etienne Robillard wrote: You guys just made me realize something very obvious. :-) I'm in the process right now of watching the excellent documentary named "Drugs Inc." on Netflix and I'm basically stunned and deeply concerned about the major opioid epidemic in the US. Have no clue what this has to do with python, but the opioid epidemic was created by collision between big pharma and the government. Quite simply, coding stuff in Python (or any other high-level programming language) is probably a very effective (and underestimated) way to help people in need finding a real goal in their life and the feeling of accomplishment. Etienne -- Etienne Robillard [email protected] https://www.isotopesoftware.ca/ -- https://mail.python.org/mailman/listinfo/python-list
Re: Thank you Python community!
Hi Ben, Thank you for your reply. I would like to make such an experimental research/investigation on the effects of Python software programming on opioid addiction. :-) Probably studying the learning of Python in people with cocaine and heroin addiction would be significant and interesting. Kind regards, Etienne Le 2018-03-19 à 17:58, Ben Finney a écrit : Etienne Robillard writes: I would like to thank you guys sincerely for helping a lot of people to stay clean, and focus on programming high-level stuff in Python instead of doing some really nasty drugs. Thank you for the kind words. I'd love to believe the Python community has the significant benefit you describe. That just raises the importance of being skeptical (because I *want* it to be true, means more rigour is needed to avoid the bias to believe it; read about “confirmation bias” to see the problem). So, in the interest of having our beliefs match the facts: What is the verifiable, statistically sound evidence that this effect actually occurs as a result of the Python community's help with programming? Where is it pbulished so we can see it? What is the size of the measured effect? It's time we regroup and combine our forces to help people who really needs our help. Combining forces to help the needy is a sentiment I can fully endorse, for sure. -- Etienne Robillard [email protected] https://www.isotoperesearch.ca/ -- https://mail.python.org/mailman/listinfo/python-list
Re: Thank you Python community!
Le 2018-03-19 à 19:36, Ben Finney a écrit : Etienne Robillard writes: I would like to make such an experimental research/investigation on the effects of Python software programming on opioid addiction. :-) Okay. The wording of your message implied that you know this already happens now, though. How did you come to know this? I'm not really sure what you mean here. Anyways, i found "Drugs Inc." a highly informative and educational documentary. I just thought that doing some Python coding should be a very effective way to stimulate your brain in positive ways. Etienne -- Etienne Robillard [email protected] https://www.isotopesoftware.ca/ -- https://mail.python.org/mailman/listinfo/python-list
Re: Thank you Python community!
Le 2018-03-19 à 22:21, Rick Johnson a écrit : On Monday, March 19, 2018 at 6:37:21 PM UTC-5, Ben Finney wrote: -- \ "Success is going from one failure to the next without a loss | `\ of enthusiasm." -- Winston Churchill | _o__) | Ben Finney -- \ “Experience is that marvelous thing that enables you to | `\ recognize a mistake when you make it again.” —Franklin P. Jones | _o__) | Ben Finney Okay Ben, i believe it's time to come clean and finally admit to this fine community, that these quotes of yours are not merely chosen at random, but are in fact, delectable morsels of laser-focused sarcasm. Here's another one I have in mind theses days: "The only thing necessary for the triumph of evil is for good men to do nothing." -Edmund Burke Etienne -- Etienne Robillard [email protected] https://www.isotoperesearch.ca/ -- https://mail.python.org/mailman/listinfo/python-list
Re: Python Developer Survey: Python 3 usage overtakes Python 2 usage
Hi, I was just wondering, could the fact that the Python community is willing to discontinue using and developing Python 2 softwares, does that mean we are stopping to support standard computers and laptops as well? Furthermore, does it bother you to develop code primarly oriented towards mobile devices in Python 3 while most of the world still cannot afford theses expensive products? Etienne -- Etienne Robillard [email protected] https://www.isotopesoftware.ca/ -- https://mail.python.org/mailman/listinfo/python-list
Re: Python Developer Survey: Python 3 usage overtakes Python 2 usage
Le 2018-03-31 à 11:40, Michael Torrie a écrit : On 03/31/2018 08:58 AM, Etienne Robillard wrote: I was just wondering, could the fact that the Python community is willing to discontinue using and developing Python 2 softwares, does that mean we are stopping to support standard computers and laptops as well? I've tried several times but I can't make sense of that paragraph. Please give me a break. That was just a simple question. Besides, I really don't understand why the Python development community is dropping support for Python 2 unless for stopping to support standard computers altogether... Furthermore, does it bother you to develop code primarly oriented towards mobile devices in Python 3 while most of the world still cannot afford theses expensive products? Or this one. What are you talking about? Are you trolling? Do you understand that a modern mobile device typically require a Internet subscription and an additional subscription for the smart phone? Do you really think people in Somalia can afford theses things like in the US? Etienne -- Etienne Robillard [email protected] https://www.isotopesoftware.ca/ -- https://mail.python.org/mailman/listinfo/python-list
Django-hotsauce 0.9.2 is out!
Hi everyone, I'm very happy to announce the Django-hotsauce 0.9.2 release: https://www.isotopesoftware.ca/pub/django-hotsauce-0.9.2.tar.gz https://pypi.python.org/pypi/Django-hotsauce/0.9.2 This is a minor bugfix/maintenance release. *What's new** * - It's now possible to use ZODB databases with Schevo backend without requiring libdurus library installed. - Updated support for PyPy 5.10 (CPython 2.7), uWSGI 2.0.17, and Django 1.11.10. For more info on Django-hotsauce, see: https://www.isotopesoftware.ca/software/django-hotsauce/ https://www.isotopesoftware.ca/wiki/DjangoHotSauce Have fun! Etienne -- Etienne Robillard [email protected] https://www.isotopesoftware.ca/ -- https://mail.python.org/mailman/listinfo/python-list
Re: [ZODB] Multithreaded connection pooling support for ZODB databases?
Heads up people!
I've finally managed to make a working `ThreadedConnectionPool` class
for supporting multiple ZODB databases connections on top of libschevo
API! :)
Technically, I decided to use gevent.Greenlet under the hood to spawn
multiple cooperative ClientStorage client/server connections.
I'm also making all my testing and development under PyPy 5.9.
You can check out the code here:
https://bitbucket.org/tkadm30/libschevo/commits/37feb029615d76f3d81233990e509b6eb6ffb5d7
Comments are welcome! :)
Etienne
Le 2018-04-16 à 18:27, Etienne Robillard a écrit :
Hi Jason,
I just made some more changes to my ThreadedConnectionPool class here:
https://bitbucket.org/tkadm30/django-hotsauce/commits/81d2e8f30019840d9c8bbdf7f82df6de2be024fc
In summary, the `ThreadedConnectionPool` class is now a subclass of
`threading.Thread` and is extending the `run` method to populate a
thread local dictionary with ClientStorage connections.
Kind regards,
Etienne
Le 2018-04-16 à 09:14, Jason Madden a écrit :
On Apr 15, 2018, at 8:17 PM, Etienne Robillard
wrote:
I would like to define a `ThreadedConnectionPoll` class to allow
multithreaded caching of ZODB databases into memory.
The ZODB.DB object is *already* thread safe for connection
management. You shouldn't need to do anything other than use
ZODB.DB().open() and conn.close() from all the threads you're using.
(That is, create exactly one ZODB.DB object for each database you are
using and share that object amongst your threads, using
db.open()/conn.close() as needed. For storages like RelStorage that
have caches at the ZODB.DB level---which are also thread safe---this
is important.)
If you are managing multiple databases and want to be able to make
references between them (a multiple-database) it is critical that
they share the same `databases` object. But that is an advanced usage.
Here's a updated version of my code so far using threading.local :
from threading import local
from notmm.dbapi.orm import ClientStorageProxy
class ThreadedConnectionPool(object):
def __init__(self, d, debug=True):
if debug:
assert isinstance(d, dict) == True
local.pool = d
for key,value in local.pool.iteritems():
Unless you left out part of the code, it appears that you're trying
to set (and read) a class attribute on the threading.local class.
This does not create thread-local state. It is also not portable and
doesn't work when local is an extension type:
py> from threading import local
py> import sys
py> sys.version_info # CPython 3.7; same result in 2.7
sys.version_info(major=3, minor=7, micro=0, releaselevel='beta',
serial=3)
py> local
py> local.foo = 1
Traceback (most recent call last):
File "", line 1, in
TypeError: can't set attributes of built-in/extension type
'_thread._local'
pypy> sys.version_info # pypy 2
(major=2, minor=7, micro=13, releaselevel='final', serial=42)
pypy> from threading import local
pypy> local.foo = 1
Traceback (most recent call last):
File "", line 1, in
TypeError: can't set attributes on type object '_local'
It generally also shouldn't be necessary as the parameters to
__init__ are preserved and used again in each thread; you can use
this to share information if those parameters are mutable:
py> from threading import get_ident
py> from threading import Thread
py> class MyLocal(local):
... def __init__(self, l):
... print("Creating in thread", get_ident(), "param", l)
... l.append(get_ident())
...
py> l = []
py> mylocal = MyLocal(l)
Creating in thread 140736147411840 param []
py> l
[140736147411840]
py> def target():
... mylocal.foo = 1
...
py> t = Thread(target=target)
py> t.start()
Creating in thread 123145538318336 param [140736147411840]
None
py> l
[140736147411840, 123145538318336]
Jason
--
Etienne Robillard
[email protected]
https://www.isotopesoftware.ca/
--
https://mail.python.org/mailman/listinfo/python-list
Re: [ZODB] Multithreaded connection pooling support for ZODB databases?
Hi,
I would like to know if you could please share some input about this
ThreadConnectionPool class for libschevo/ZODB. :)
Thanks!
Etienne
Le 2018-04-20 à 04:49, Etienne Robillard a écrit :
Heads up people!
I've finally managed to make a working `ThreadedConnectionPool` class
for supporting multiple ZODB databases connections on top of libschevo
API! :)
Technically, I decided to use gevent.Greenlet under the hood to spawn
multiple cooperative ClientStorage client/server connections.
I'm also making all my testing and development under PyPy 5.9.
You can check out the code here:
https://bitbucket.org/tkadm30/libschevo/commits/37feb029615d76f3d81233990e509b6eb6ffb5d7
Comments are welcome! :)
Etienne
Le 2018-04-16 à 18:27, Etienne Robillard a écrit :
Hi Jason,
I just made some more changes to my ThreadedConnectionPool class
here:
https://bitbucket.org/tkadm30/django-hotsauce/commits/81d2e8f30019840d9c8bbdf7f82df6de2be024fc
In summary, the `ThreadedConnectionPool` class is now a subclass of
`threading.Thread` and is extending the `run` method to populate a
thread local dictionary with ClientStorage connections.
Kind regards,
Etienne
Le 2018-04-16 à 09:14, Jason Madden a écrit :
On Apr 15, 2018, at 8:17 PM, Etienne Robillard
wrote:
I would like to define a `ThreadedConnectionPoll` class to allow
multithreaded caching of ZODB databases into memory.
The ZODB.DB object is *already* thread safe for connection
management. You shouldn't need to do anything other than use
ZODB.DB().open() and conn.close() from all the threads you're using.
(That is, create exactly one ZODB.DB object for each database you
are using and share that object amongst your threads, using
db.open()/conn.close() as needed. For storages like RelStorage that
have caches at the ZODB.DB level---which are also thread safe---this
is important.)
If you are managing multiple databases and want to be able to make
references between them (a multiple-database) it is critical that
they share the same `databases` object. But that is an advanced usage.
Here's a updated version of my code so far using threading.local :
from threading import local
from notmm.dbapi.orm import ClientStorageProxy
class ThreadedConnectionPool(object):
def __init__(self, d, debug=True):
if debug:
assert isinstance(d, dict) == True
local.pool = d
for key,value in local.pool.iteritems():
Unless you left out part of the code, it appears that you're trying
to set (and read) a class attribute on the threading.local class.
This does not create thread-local state. It is also not portable and
doesn't work when local is an extension type:
py> from threading import local
py> import sys
py> sys.version_info # CPython 3.7; same result in 2.7
sys.version_info(major=3, minor=7, micro=0, releaselevel='beta',
serial=3)
py> local
py> local.foo = 1
Traceback (most recent call last):
File "", line 1, in
TypeError: can't set attributes of built-in/extension type
'_thread._local'
pypy> sys.version_info # pypy 2
(major=2, minor=7, micro=13, releaselevel='final', serial=42)
pypy> from threading import local
pypy> local.foo = 1
Traceback (most recent call last):
File "", line 1, in
TypeError: can't set attributes on type object '_local'
It generally also shouldn't be necessary as the parameters to
__init__ are preserved and used again in each thread; you can use
this to share information if those parameters are mutable:
py> from threading import get_ident
py> from threading import Thread
py> class MyLocal(local):
... def __init__(self, l):
... print("Creating in thread", get_ident(), "param", l)
... l.append(get_ident())
...
py> l = []
py> mylocal = MyLocal(l)
Creating in thread 140736147411840 param []
py> l
[140736147411840]
py> def target():
... mylocal.foo = 1
...
py> t = Thread(target=target)
py> t.start()
Creating in thread 123145538318336 param [140736147411840]
None
py> l
[140736147411840, 123145538318336]
Jason
--
Etienne Robillard
[email protected]
https://www.isotopesoftware.ca/
--
https://mail.python.org/mailman/listinfo/python-list
Problem with pickle in zodb
Hi, Any ideas why this is happening ? erob@marina:/home/www/open-neurosecurity.org/trunk$ schevo editor zodb://127.0.0.1:4545 07:32:16 environ No en_CA translation found for domain kiwi 07:32:16 environ No en_CA translation found for domain gazpacho libschevo 4.0.1 :: Database Navigator () () > /home/erob/src/libschevo/lib/schevo/gtk2/window.py(295)database_open() -> self._db = schevo.database.open(db_alias) (Pdb) n zodb://127.0.0.1:4545 127.0.0.1:4545 Found epoll library, good! /home/erob/src/libschevo/lib/schevo/backends/zodb.py:7: RuntimeWarning: Patching more than once will result in the union of all True parameters being patched patch_all(thread=True) 07:32:23 ZEO.asyncio.marshal can't decode message: '\x80\x03(K\x01K\x00U\x06.replyU\x08\x03\xc7\xf6]\xe0\xd...' 07:32:23 ZEO.asyncio.base data_received 4 0 True Traceback (most recent call last): File "/home/erob/work/ZEO-5.1.0/src/ZEO/asyncio/base.py", line 128, in data_received self.message_received(collected) File "/home/erob/work/ZEO-5.1.0/src/ZEO/asyncio/client.py", line 206, in message_received msgid, async, name, args = self.decode(data) File "/home/erob/work/ZEO-5.1.0/src/ZEO/asyncio/marshal.py", line 91, in pickle_decode return unpickler.load() # msgid, flags, name, args ValueError: unsupported pickle protocol: 3 07:32:52 ZEO.asyncio.client Registration or cache validation failed, Closed Traceback (most recent call last): File "/home/erob/work/ZEO-5.1.0/src/ZEO/asyncio/client.py", line 188, in finish_connect *credentials) ClientDisconnected: Closed ClientDisconnected: ClientDi...ection',) Thank you in advance, Etienne -- https://mail.python.org/mailman/listinfo/python-list
Re: [ZODB] Problem with pickle in zodb
Yup, that is correct! Thanks. :-) Etienne Le 2018-06-03 à 07:39, Jason Madden a écrit : You appear to have mismatched versions of ZEO on the client and server, 5.2 on one side and 5.1.0 on the other. Both need to be 5.2. On Jun 3, 2018, at 06:35, Etienne Robillard wrote: Hi, Any ideas why this is happening ? erob@marina:/home/www/open-neurosecurity.org/trunk$ schevo editor zodb://127.0.0.1:4545 07:32:16 environ No en_CA translation found for domain kiwi 07:32:16 environ No en_CA translation found for domain gazpacho libschevo 4.0.1 :: Database Navigator () () /home/erob/src/libschevo/lib/schevo/gtk2/window.py(295)database_open() -> self._db = schevo.database.open(db_alias) (Pdb) n zodb://127.0.0.1:4545 127.0.0.1:4545 Found epoll library, good! /home/erob/src/libschevo/lib/schevo/backends/zodb.py:7: RuntimeWarning: Patching more than once will result in the union of all True parameters being patched patch_all(thread=True) 07:32:23 ZEO.asyncio.marshal can't decode message: '\x80\x03(K\x01K\x00U\x06.replyU\x08\x03\xc7\xf6]\xe0\xd...' 07:32:23 ZEO.asyncio.base data_received 4 0 True Traceback (most recent call last): File "/home/erob/work/ZEO-5.1.0/src/ZEO/asyncio/base.py", line 128, in data_received self.message_received(collected) File "/home/erob/work/ZEO-5.1.0/src/ZEO/asyncio/client.py", line 206, in message_received msgid, async, name, args = self.decode(data) File "/home/erob/work/ZEO-5.1.0/src/ZEO/asyncio/marshal.py", line 91, in pickle_decode return unpickler.load() # msgid, flags, name, args ValueError: unsupported pickle protocol: 3 07:32:52 ZEO.asyncio.client Registration or cache validation failed, Closed Traceback (most recent call last): File "/home/erob/work/ZEO-5.1.0/src/ZEO/asyncio/client.py", line 188, in finish_connect *credentials) ClientDisconnected: Closed ClientDisconnected: ClientDi...ection',) -- Etienne Robillard [email protected] https://www.isotopesoftware.ca/ -- https://mail.python.org/mailman/listinfo/python-list
Django-hotsauce/ZODB 5.4.0/PyPy nightly sprint!!
Yo people I'm doing a nightly hacking sprint for django-hotsauce on pypy
and got some cool bugs I would like to share:
Traceback (most recent call last):
File "/usr/local/bin/schevo", line 11, in
load_entry_point('libschevo', 'console_scripts', 'schevo')()
File "/home/erob/src/libschevo/lib/schevo/script/command.py", line
32, in __call__
return self.main(arg0, args)
File "/home/erob/src/libschevo/lib/schevo/script/command.py", line
74, in main
return command()(*args)
File "/home/erob/src/libschevo/lib/schevo/script/command.py", line
32, in __call__
return self.main(arg0, args)
File "/home/erob/src/libschevo/lib/schevo/script/command.py", line
74, in main
return command()(*args)
File "/home/erob/src/libschevo/lib/schevo/script/command.py", line
32, in __call__
return self.main(arg0, args)
File "/home/erob/src/libschevo/lib/schevo/script/db_evolve.py", line
86, in main
db = schevo.database.open(url)
File "/home/erob/src/libschevo/lib/schevo/database.py", line 371, in open
db = Database(backend)
File "/home/erob/src/libschevo/lib/schevo/database2.py", line 95, in
__init__
self._update_extent_maps_by_name()
File "/home/erob/src/libschevo/lib/schevo/database2.py", line 1633,
in _update_extent_maps_by_name
for extent in self._extent_maps_by_id.itervalues():
File "/usr/local/lib/python2.7/dist-packages/ZODB/Connection.py",
line 791, in setstate
p, serial = self._storage.load(oid)
File "/usr/local/lib/python2.7/dist-packages/ZODB/mvccadapter.py",
line 143, in load
r = self._storage.loadBefore(oid, self._start)
File "/home/erob/work/ZEO-5.1.0/src/ZEO/ClientStorage.py", line 520,
in loadBefore
return self._server.load_before(oid, tid)
File "/home/erob/work/ZEO-5.1.0/src/ZEO/asyncio/client.py", line 783,
in load_before
return self.__call(self.client.load_before_threadsafe, oid, tid)
File "/home/erob/work/ZEO-5.1.0/src/ZEO/asyncio/client.py", line 748,
in call
return self.wait_for_result(result, self.timeout)
File "/home/erob/work/ZEO-5.1.0/src/ZEO/asyncio/client.py", line 756,
in wait_for_result
return future.result(timeout)
File
"/usr/local/lib/python2.7/dist-packages/futures-3.0.5-py2.7.egg/concurrent/futures/_base.py",
line 405, in result
return self.__get_result()
File
"/usr/local/lib/python2.7/dist-packages/futures-3.0.5-py2.7.egg/concurrent/futures/_base.py",
line 357, in __get_result
raise type(self._exception), self._exception, self._traceback
ZEO.Exceptions.ClientDisconnected: connection lost
erob@marina:/home/www/isotopesoftware.ca/trunk$
Not sure about this first one! :)
The command I'm trying to run is:
% schevo db evolve --app blogengine2 zodb://127.0.0.1:4545 31
The ZODB 5.4.0 server then produce the following traceback:
2018-06-07T21:14:55 INFO ZEO.asyncio.base Connected server protocol
--
2018-06-07T21:14:55 INFO ZEO.asyncio.server received handshake 'Z5'
--
2018-06-07T21:14:55 ERROR ZEO.asyncio.marshal can't decode message:
'((ccopy_reg\n_reconstructor\n(czodbpickle\nbinary\nc__b...'
--
2018-06-07T21:14:55 ERROR ZEO.asyncio.server Can't deserialize message
Traceback (most recent call last):
File "/home/erob/work/ZEO-5.1.0/src/ZEO/asyncio/server.py", line 89,
in message_received
message_id, async, name, args = self.decode(message)
File "/home/erob/work/ZEO-5.1.0/src/ZEO/asyncio/marshal.py", line
114, in pickle_server_decode
return unpickler.load() # msgid, flags, name, args
File "/home/erob/work/ZEO-5.1.0/src/ZEO/asyncio/marshal.py", line
164, in server_find_global
raise ImportError("import error %s: %s" % (module, msg))
ImportError: import error copy_reg:
--
2018-06-07T21:14:55 ERROR ZEO.asyncio.base data_received 4 0 True
Traceback (most recent call last):
File "/home/erob/work/ZEO-5.1.0/src/ZEO/asyncio/base.py", line 128,
in data_received
self.message_received(collected)
File "/home/erob/work/ZEO-5.1.0/src/ZEO/asyncio/server.py", line 94,
in message_received
if message_id == -1:
UnboundLocalError: local variable 'message_id' referenced before assignment
--
2018-06-07T21:14:55 INFO ZEO.StorageServer (127.0.0.1:4545) disconnected
--
2018-06-07T21:14:55 INFO ZEO.asyncio.base Connected server protocol
--
2018-06-07T21:14:55 INFO ZEO.asyncio.server received handshake 'Z5'
--
2018-06-07T21:14:55 INFO ZEO.StorageServer (127.0.0.1:4545) disconnected
Please hit me up if you know how to fix theses errors! :)
I'm using PyPy 5.9 and 5.10 for dev and Python 2.7.13 for production
with Cython bindings!
Cheers,
Etienne
--
https://mail.python.org/mailman/listinfo/python-list
Re: [ZODB] Django-hotsauce/ZODB 5.4.0/PyPy nightly sprint!!
Le 2018-06-08 à 02:25, Mauro Amico a écrit : seems to me really similar to https://github.com/zopefoundation/ZEO/pull/96 try to upgrade to ZEO 5.1.2 mauro. Hey man! Thanks for the heads up! Looks like I messed up again... :) That error was caused because my zodb database file was out of sync with the schema definition. peace, Etienne Il Ven 8 Giu 2018, 03:29 Etienne Robillard <mailto:[email protected]>> ha scritto: Yo people I'm doing a nightly hacking sprint for django-hotsauce on pypy and got some cool bugs I would like to share: Traceback (most recent call last): File "/usr/local/bin/schevo", line 11, in load_entry_point('libschevo', 'console_scripts', 'schevo')() File "/home/erob/src/libschevo/lib/schevo/script/command.py", line 32, in __call__ return self.main(arg0, args) File "/home/erob/src/libschevo/lib/schevo/script/command.py", line 74, in main return command()(*args) File "/home/erob/src/libschevo/lib/schevo/script/command.py", line 32, in __call__ return self.main(arg0, args) File "/home/erob/src/libschevo/lib/schevo/script/command.py", line 74, in main return command()(*args) File "/home/erob/src/libschevo/lib/schevo/script/command.py", line 32, in __call__ return self.main(arg0, args) File "/home/erob/src/libschevo/lib/schevo/script/db_evolve.py", line 86, in main db = schevo.database.open(url) File "/home/erob/src/libschevo/lib/schevo/database.py", line 371, in open db = Database(backend) File "/home/erob/src/libschevo/lib/schevo/database2.py", line 95, in __init__ self._update_extent_maps_by_name() File "/home/erob/src/libschevo/lib/schevo/database2.py", line 1633, in _update_extent_maps_by_name for extent in self._extent_maps_by_id.itervalues(): File "/usr/local/lib/python2.7/dist-packages/ZODB/Connection.py", line 791, in setstate p, serial = self._storage.load(oid) File "/usr/local/lib/python2.7/dist-packages/ZODB/mvccadapter.py", line 143, in load r = self._storage.loadBefore(oid, self._start) File "/home/erob/work/ZEO-5.1.0/src/ZEO/ClientStorage.py", line 520, in loadBefore return self._server.load_before(oid, tid) File "/home/erob/work/ZEO-5.1.0/src/ZEO/asyncio/client.py", line 783, in load_before return self.__call(self.client.load_before_threadsafe, oid, tid) File "/home/erob/work/ZEO-5.1.0/src/ZEO/asyncio/client.py", line 748, in call return self.wait_for_result(result, self.timeout) File "/home/erob/work/ZEO-5.1.0/src/ZEO/asyncio/client.py", line 756, in wait_for_result return future.result(timeout) File "/usr/local/lib/python2.7/dist-packages/futures-3.0.5-py2.7.egg/concurrent/futures/_base.py", line 405, in result return self.__get_result() File "/usr/local/lib/python2.7/dist-packages/futures-3.0.5-py2.7.egg/concurrent/futures/_base.py", line 357, in __get_result raise type(self._exception), self._exception, self._traceback ZEO.Exceptions.ClientDisconnected: connection lost erob@marina:/home/www/isotopesoftware.ca/trunk$ <http://isotopesoftware.ca/trunk$> Not sure about this first one! :) The command I'm trying to run is: % schevo db evolve --app blogengine2 zodb://127.0.0.1:4545 <http://127.0.0.1:4545> 31 The ZODB 5.4.0 server then produce the following traceback: 2018-06-07T21:14:55 INFO ZEO.asyncio.base Connected server protocol -- 2018-06-07T21:14:55 INFO ZEO.asyncio.server received handshake 'Z5' -- 2018-06-07T21:14:55 ERROR ZEO.asyncio.marshal can't decode message: '((ccopy_reg\n_reconstructor\n(czodbpickle\nbinary\nc__b...' -- 2018-06-07T21:14:55 ERROR ZEO.asyncio.server Can't deserialize message Traceback (most recent call last): File "/home/erob/work/ZEO-5.1.0/src/ZEO/asyncio/server.py", line 89, in message_received message_id, async, name, args = self.decode(message) File "/home/erob/work/ZEO-5.1.0/src/ZEO/asyncio/marshal.py", line 114, in pickle_server_decode return unpickler.load() # msgid, flags, name, args File "/home/erob/work/ZEO-5.1.0/src/ZEO/asyncio/marshal.py", line 164, in server_find_global raise ImportError("import error %s: %s" % (module, msg)) ImportError: import error copy_reg: -- 2018-06-07T21:14:55 ERROR ZEO.asyncio.base data_received 4 0 True
Re: [ZODB] Django-hotsauce/ZODB 5.4.0/PyPy nightly sprint!!
Le 2018-06-08 à 02:25, Mauro Amico a écrit : seems to me really similar to https://github.com/zopefoundation/ZEO/pull/96 try to upgrade to ZEO 5.1.2 mauro. Hey man! Thanks for the heads up! Looks like I messed up again... :) That error was caused because my zodb database file was out of sync with the schema definition. peace, Etienne Il Ven 8 Giu 2018, 03:29 Etienne Robillard <mailto:[email protected]>> ha scritto: Yo people I'm doing a nightly hacking sprint for django-hotsauce on pypy and got some cool bugs I would like to share: Traceback (most recent call last): File "/usr/local/bin/schevo", line 11, in load_entry_point('libschevo', 'console_scripts', 'schevo')() File "/home/erob/src/libschevo/lib/schevo/script/command.py", line 32, in __call__ return self.main(arg0, args) File "/home/erob/src/libschevo/lib/schevo/script/command.py", line 74, in main return command()(*args) File "/home/erob/src/libschevo/lib/schevo/script/command.py", line 32, in __call__ return self.main(arg0, args) File "/home/erob/src/libschevo/lib/schevo/script/command.py", line 74, in main return command()(*args) File "/home/erob/src/libschevo/lib/schevo/script/command.py", line 32, in __call__ return self.main(arg0, args) File "/home/erob/src/libschevo/lib/schevo/script/db_evolve.py", line 86, in main db = schevo.database.open(url) File "/home/erob/src/libschevo/lib/schevo/database.py", line 371, in open db = Database(backend) File "/home/erob/src/libschevo/lib/schevo/database2.py", line 95, in __init__ self._update_extent_maps_by_name() File "/home/erob/src/libschevo/lib/schevo/database2.py", line 1633, in _update_extent_maps_by_name for extent in self._extent_maps_by_id.itervalues(): File "/usr/local/lib/python2.7/dist-packages/ZODB/Connection.py", line 791, in setstate p, serial = self._storage.load(oid) File "/usr/local/lib/python2.7/dist-packages/ZODB/mvccadapter.py", line 143, in load r = self._storage.loadBefore(oid, self._start) File "/home/erob/work/ZEO-5.1.0/src/ZEO/ClientStorage.py", line 520, in loadBefore return self._server.load_before(oid, tid) File "/home/erob/work/ZEO-5.1.0/src/ZEO/asyncio/client.py", line 783, in load_before return self.__call(self.client.load_before_threadsafe, oid, tid) File "/home/erob/work/ZEO-5.1.0/src/ZEO/asyncio/client.py", line 748, in call return self.wait_for_result(result, self.timeout) File "/home/erob/work/ZEO-5.1.0/src/ZEO/asyncio/client.py", line 756, in wait_for_result return future.result(timeout) File "/usr/local/lib/python2.7/dist-packages/futures-3.0.5-py2.7.egg/concurrent/futures/_base.py", line 405, in result return self.__get_result() File "/usr/local/lib/python2.7/dist-packages/futures-3.0.5-py2.7.egg/concurrent/futures/_base.py", line 357, in __get_result raise type(self._exception), self._exception, self._traceback ZEO.Exceptions.ClientDisconnected: connection lost erob@marina:/home/www/isotopesoftware.ca/trunk$ <http://isotopesoftware.ca/trunk$> Not sure about this first one! :) The command I'm trying to run is: % schevo db evolve --app blogengine2 zodb://127.0.0.1:4545 <http://127.0.0.1:4545> 31 The ZODB 5.4.0 server then produce the following traceback: 2018-06-07T21:14:55 INFO ZEO.asyncio.base Connected server protocol -- 2018-06-07T21:14:55 INFO ZEO.asyncio.server received handshake 'Z5' -- 2018-06-07T21:14:55 ERROR ZEO.asyncio.marshal can't decode message: '((ccopy_reg\n_reconstructor\n(czodbpickle\nbinary\nc__b...' -- 2018-06-07T21:14:55 ERROR ZEO.asyncio.server Can't deserialize message Traceback (most recent call last): File "/home/erob/work/ZEO-5.1.0/src/ZEO/asyncio/server.py", line 89, in message_received message_id, async, name, args = self.decode(message) File "/home/erob/work/ZEO-5.1.0/src/ZEO/asyncio/marshal.py", line 114, in pickle_server_decode return unpickler.load() # msgid, flags, name, args File "/home/erob/work/ZEO-5.1.0/src/ZEO/asyncio/marshal.py", line 164, in server_find_global raise ImportError("import error %s: %s" % (module, msg)) ImportError: import error copy_reg: -- 2018-06-07T21:14:55 ERROR ZEO.asyncio.base data_received 4 0 True
Django-hotsauce 0.9.4.1 is out
Hi everyone, Django-hotsauce 0.9.4.1 (open source edition) is out! What's new - Maintenance bugfixes for ZODBController class (cPython, PyPy) - Minor bugfixes and updates Download - https://pypi.org/pypi/Django-hotsauce/ - https://www.isotopesoftware.ca/pub/django-hotsauce/django-hotsauce-0.9.4.1.tar.gz Have fun! Etienne -- Etienne Robillard [email protected] https://www.isotopesoftware.ca/ -- https://mail.python.org/mailman/listinfo/python-list
Django-hotsauce 1.0 LTS (Commercial Edition) now available for preorder!!
Dear all, It is my pleasure to announce the availability of Django-hotsauce 1.0 LTS (Commercial Edition) for preorder: https://www.livestore.ca/product/django-hotsauce/ Purchase this gem using Paypal before October 1st 2018 with the discount code "djangohotsauce" and get 50% off the regular price! Everyone who will make this purchase will receive their registered copy of Django-hotsauce 1.0 on October 1st 2018. One more thing, this discount code also works for all commercial softwares listed on livestore.ca! :) Have fun!! Etienne -- Etienne Robillard [email protected] https://www.isotopesoftware.ca/ -- https://mail.python.org/mailman/listinfo/python-list
Re: Django-hotsauce 1.0 LTS (Commercial Edition) now available for preorder!!
Le 2018-06-14 à 02:38, Chris Angelico a écrit : On Thu, Jun 14, 2018 at 11:07 AM, Jim Lee wrote: I haven't purchased commercial software in decades, so I'm not up on the prevailing business model, but I have to ask: Why would anyone purchase software and then agree to wait 14 weeks for it to be delivered? I can see that model for hardware, where material resources are limited and a finite number of product is produced, but software? What's the point? For the 50% discount, I presume. If you wait 14 weeks, then buy, then own, you pay full price. >From the company's point of view: if the release date is in the future and ALL the revenue is also in the future, cash flow becomes tricky. By getting at least _some_ money in advance, they give themselves a way to pay the bills. Chris got it right I think! :) Thank you, Etienne -- Etienne Robillard [email protected] https://www.isotopesoftware.ca/ -- https://mail.python.org/mailman/listinfo/python-list
How to delete a branch permanently in mercurial/bitbucket?
Hi, I need to delete a branch in mercurial in order to move it into a private repository. I have heard of the "hg strip" method but Bitbucket is warning me that this operation will affect branches I would prefer keeping... So I just have no idea right now how to prune this specific branch out of my public repo! :( Any suggestions for doing this would be much appreciated! Kind regards, Etienne -- Etienne Robillard [email protected] https://www.isotopesoftware.ca/ -- https://mail.python.org/mailman/listinfo/python-list
Re: Django-hotsauce 1.0 LTS (Commercial Edition) now available for preorder!!
Le 2018-06-14 à 12:58, Jim Lee a écrit : On 06/13/2018 11:38 PM, Chris Angelico wrote: On Thu, Jun 14, 2018 at 11:07 AM, Jim Lee wrote: I haven't purchased commercial software in decades, so I'm not up on the prevailing business model, but I have to ask: Why would anyone purchase software and then agree to wait 14 weeks for it to be delivered? I can see that model for hardware, where material resources are limited and a finite number of product is produced, but software? What's the point? For the 50% discount, I presume. If you wait 14 weeks, then buy, then own, you pay full price. From the company's point of view: if the release date is in the future and ALL the revenue is also in the future, cash flow becomes tricky. By getting at least _some_ money in advance, they give themselves a way to pay the bills. ChrisA But the "50% discount" is supposedly good up until the release date. I could purchase the software the day before release and still enjoy the same benefit without the 14 week wait. I understand the advantages *to the company*, but to enjoy those advantages, they need to provide some kind of incentive to the buyer. I don't see one here. Anyway, I was just curious to see if there was any kind of thought process behind the "promotion". -Jim Thought process? I believe the current strategy I have in mind is to let any potential new customers to take the time needed to consider seriously Django-hotsauce for their commercial web applications. This includes allowing some time for peoples (developers) to at least try the free version and then possibly consider switching to the LTS version for extended support and troubleshooting. Anyways, thank you for your input! :) Etienne -- Etienne Robillard [email protected] https://www.isotopesoftware.ca/ -- https://mail.python.org/mailman/listinfo/python-list
A quick question for you!
Hi, Quick question: Does anyone of you know what is the effect of enabling gc.enable() in sitecustomize.py when using PyPy? Can it reduce latency for long-lived WSGI applications? Thanks, Etienne -- Etienne Robillard [email protected] https://www.isotopesoftware.ca/ -- https://mail.python.org/mailman/listinfo/python-list
Re: [pypy-dev] A quick question for you!
Le 2018-06-18 à 22:47, William ML Leslie a écrit : On 18 June 2018 at 22:18, Etienne Robillard wrote: Hi, Quick question: Does anyone of you know what is the effect of enabling gc.enable() in sitecustomize.py when using PyPy? Can it reduce latency for long-lived WSGI applications? gc is enabled by default. you only need to use gc.enable() if you have earlier run gc.disable(). Good to know man! I didn't knew that. Thanks for sharing. Etienne -- Etienne Robillard [email protected] https://www.isotopesoftware.ca/ -- https://mail.python.org/mailman/listinfo/python-list
Django-hotsauce 0.9.5 (Realistic Scenario) and libschevo 4.0.2 are out!
Hi everyone, I'm really happy to announce the public release of Django-hotsauce 0.9.5 (Realistic Scenario) and libschevo 4.0.2 for Python! :-) Downloads: PyPi https://pypi.org/project/Django-hotsauce/ https://pypi.org/project/libschevo/ Master site https://www.isotopesoftware.ca/pub/django-hotsauce/django-hotsauce-0.9.5.tar.gz https://www.isotopesoftware.ca/pub/libschevo/libschevo-4.0.2.tar.gz Release notes: - Maintenance bugfixes (django-hotsauce) - Fixed several critical issues with transaction management and schema migration for the ZODB backend (libschevo) Have fun!! Etienne -- Etienne Robillard [email protected] https://www.isotopesoftware.ca/ -- https://mail.python.org/mailman/listinfo/python-list
Re: Invalid error in python program
I'm guessing you may need some help in english writing too! Anyways, I don't think anyone here may want to make your homeworks for you... Regards, Etienne Le 2018-07-08 à 12:43, Prafull Ks a écrit : I miss studying in class 11 cbse.College as introduced us python. I installed it in my laptop but when I save and runed in idle mod. The 3.7.0 the 7 is highlighted and syntax error is showen. As showen in the screen shot below. Plz help me resolve this problem Ur faithful user Prafull. Thank you -- Etienne Robillard [email protected] https://www.isotopesoftware.ca/ -- https://mail.python.org/mailman/listinfo/python-list
Generic OAuth2 authorization inside view-based decorator/middleware?
Hi!
This is quick post about my current attempt to develop a specific
GoogleController extension for django-hotsauce 0.9.6 in order to allow
OAuth2 authentication and authorization on a regular Django view.
Code:
from notmm.controllers.wsgi import WSGIController, sessionmanager
from wsgi_oauth2 import client
__all__ = ['OAuthController']
class OAuthController(WSGIController):
debug = True
def __init__(self, **kwargs):
super(OAuthController, self).__init__(**kwargs)
class GoogleController(OAuthController):
scope = 'email'
def __init__(self, request, **kwargs):
super(GoogleController, self).__init__()
self._request = request
self._client = client.GoogleClient(
self.settings.OAUTH2_CLIENT_ID,
access_token=self.settings.OAUTH2_ACCESS_TOKEN,
scope=self.scope,
redirect_url=self.settings.OAUTH2_REDIRECT_URL,
)
def application(self, env, start_response):
with sessionmanager(self.request):
# wsgi_oauth2 response middleware
response =
self._client.wsgi_middleware(self.get_response(request=self.request),
secret=self.settings.SECRET_KEY,
login_path=self.settings.OAUTH2_LOGIN_URL)
return response(env, start_response)
Decorator:
from functools import wraps
from notmm.controllers.oauth import GoogleController
from notmm.utils.django_settings import LazySettings
_settings = LazySettings()
__all__ = ('require_oauth', 'with_schevo_database',)
def require_oauth(view_func):
controller_class = GoogleController
def decorator(view_func, **kwargs):
def _wrapper(*args, **kwargs):
req = args[0]
wsgi_app = controller_class(req, **kwargs)
# XXX this does not work :-)
response = wsgi_app.get_response(request=req)
return response
return wraps(view_func)(_wrapper, **kwargs)
return decorator(view_func)
I would like to know what's wrong with the code above... :-)
Thank you,
Etienne
--
Etienne Robillard
[email protected]
https://www.isotopesoftware.ca/
--
https://mail.python.org/mailman/listinfo/python-list
Re: Guido van Rossum resigns as Python leader
I'm really sad to hear that news! Python is the language which made me gain the most skills in modern software programming ever! I have not encountered a better programming language than Python for quickly becoming a mature developer! I just hope the Python software foundation will continue to provide to his community all the same services than before and that the project will continue to evolve! Thanks for everything Guido! :-) Etienne -- Etienne Robillard [email protected] https://www.isotopesoftware.ca/ -- https://mail.python.org/mailman/listinfo/python-list
Re: Cult-like behaviour [was Re: Kindness]
Le 2018-07-14 à 16:02, Chris Angelico a écrit : No, it isn't. Shunning would be killfiling you, which is apparently "more polite" than telling you how utterly and completely wrong you are. What I would like to do is ban you for endemic racism, honestly. By recommending and preferring eight-bit text strings, you're saying "Chinese text doesn't matter". And by stipulating Latin-1, you're also saying "Russian text doesn't matter" and "Thai text doesn't matter" and "Hebrew text doesn't matter" and more. You are declaring that YOUR culture is the only one that matters. When I see behaviour like that in a Twitch stream that I moderate, I smack it with a banhammer, because that is utterly unacceptable. Why should we tolerate it in programming? lol +1 ChrisA -- Etienne Robillard [email protected] https://www.isotopesoftware.ca/ -- https://mail.python.org/mailman/listinfo/python-list
