Re: documentation on read.encode
On Thu, Jan 18, 2018 at 12:47 AM, Steven D'Aprano wrote: > On Wed, 17 Jan 2018 16:54:37 -0500, Larry Martell wrote: > >> The code that was receiving the >> PNG was not reading and writing the file as binary. Strangely that >> worked on Linux but not on Windows. > > Nothing strange about it -- on Unix and Linux systems (with the possible > exception of Mac OS?) in Python 2 there's no difference between text and > binary mode for ASCII-only files. In Python 2, strings are byte strings, > not Unicode, and reading from files returns such sequences of bytes. > > On Windows, reading from files in text mode treats \r\n as the end of > line, and converts[1] such \r\n pairs to \n; it also treats ^Z byte as > the end of file[2], or at least it used to back in the 2.5 or so days. I > haven't tested it in more recent versions. > > > [1] Technically this is a build-time option, but as far as I know it is > not just the default but pretty much universal. > > [2] https://blogs.msdn.microsoft.com/oldnewthing/20040316-00/?p=40233/ Thanks for the clarification. I have been programming since 1975 and thankfully have had very little exposure to Windows. -- https://mail.python.org/mailman/listinfo/python-list
Very strange issues with collections.Mapping
Hello! I am running into a very perplexing issue that is very rare, but creeps up and is crashing my app. The root cause of the issue comes down to the following check returning true: isinstance([], collections.Mapping) Obviously you can get this behavior if you register `list` as a subclass of the Mapping ABC, but I'm not doing that. Because the issue is so rare (but still common enough that I need to address it), it's hard to reproduce in a bench test. What I am going to try is to essentially monkey-patch collections.Mapping.register with a method that dumps a stack trace whenever it's called at the time of initial import so I can get an idea of where this method could *possibly* be getting called with a list as its argument. The annoying thing here is that wherever the bug is happening, the crash happens *way* far away (in a third-party library). I've also determined it as the root cause of two crashes that seem completely unrelated (except that the crash is caused by a list not behaving like a dict shortly after isinstance(obj, collections.Mapping) returns True). These are the libraries I'm using: amqp billiard celery dj-database-url Django django-redis-cache enum34 gunicorn kombu newrelic psycopg2 pyasn1 pytz redis requests rsa six vine voluptuous It's a web application, as you can probably tell. The main reason I ask here is because I'm wondering if anybody has encountered this before and managed to hunt down which of these libraries is doing something naughty? Thanks! Jason -- Jason M. Swails -- https://mail.python.org/mailman/listinfo/python-list
Re: Very strange issues with collections.Mapping
On Fri, Jan 19, 2018 at 8:37 AM, Jason Swails wrote: > The root cause of the issue comes down to the following check returning > true: > > isinstance([], collections.Mapping) > > Obviously you can get this behavior if you register `list` as a subclass of > the Mapping ABC, but I'm not doing that. Because the issue is so rare (but > still common enough that I need to address it), it's hard to reproduce in a > bench test. Just a quickie: Have you confirmed for sure that it is a vanilla list - that "type(x) is type([])" - not some subclass thereof? Other than that, I don't have any specific tips; have fun instrumenting your code to try to figure this out! (And I mean that sincerely - this kind of thing CAN be fun.) ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Where are the moderators?
Hi, What happened to the moderators? I have always liked this forum, but there's so much spam now. Is there a way to become a moderator so this can be cleaned up? Thanks, Mike -- https://mail.python.org/mailman/listinfo/python-list
Re: Where are the moderators?
On 18 January 2018 at 22:38, Mike Driscoll wrote: > Hi, > > What happened to the moderators? I have always liked this forum, but there's > so much spam now. Is there a way to become a moderator so this can be cleaned > up? > > Thanks, > Mike You seem to be using the Google Groups interface. The moderators can't control what appears on that list, and it is as you say essentially unusable (IMO) because of spam. For a reasonably spam-free experience, I suggest subscribing to the mailing list itself (the web page to do that is attached to mails posted here - https://mail.python.org/mailman/listinfo/python-list) Paul -- https://mail.python.org/mailman/listinfo/python-list
Re: Where are the moderators?
> What happened to the moderators? I have always liked this forum, but there's > so much spam now. Is there a way to become a moderator so this can be cleaned > up? Just to emphasize what Paul said, the only moderators I'm aware of are those who moderate [email protected]. A bidirectional gateway runs on mail.python.org which ships messages back and forth. I take care of the SpamBayes setup on mail.python.org, whose primary task is to divert spam which arrives from comp.lang.python. Skip -- https://mail.python.org/mailman/listinfo/python-list
Re: Speeding up the implementation of Stochastic Gradient Ascent in Python
On 17/01/18 14:29, [email protected] wrote: > > Hello everyone, > > I am implementing a time-dependent Recommender System which applies BPR > (Bayesian Personalized Ranking), where Stochastic Gradient Ascent is used to > learn the parameters of the model. Such that, one iteration involves sampling > randomly the quadruple (i.e. userID, positive_item, negative_item, > epoch_index) for n times, where n is the total number of positive feedbacks > (i.e. the number of ratings given to all items). But, as my implementation > takes too much time for learning the parameters (since it requires 100 > iterations to learn the parameters), I was wondering if there is a way to > improve my code and speed up the learning process of the parameters. > > Please find the code of my implementation (the update function named as > updateFactors is the one that learns the parameters, and such that I guess is > the one that should be improved in order to speed up the process of learning > the parameter values) in the following link: > https://codereview.stackexchange.com/questions/183707/speeding-up-the-implementation-of-stochastic-gradient-ascent-in-python > dim0 = [] dim1 = [] ... dim19 = [] dim0.append((asin, self.theta_item_per_bin[bin][itemID][0])) dim1.append((asin,self.theta_item_per_bin[bin][itemID][1])) ... dim19.append((asin,self.theta_item_per_bin[bin][itemID][19])) for d in range(self.K2): if d == 0: max_value = max(dim0, key=operator.itemgetter(1)) asin_ = max_value[0] value_ = max_value[1] print 'dim:',d,', itemID: ', asin_, ', value = ', value_ if d == 1: max_value = max(dim1, key=operator.itemgetter(1)) asin_ = max_value[0] value_ = max_value[1] print 'dim:',d,', itemID: ', asin_, ', value = ', value_ if d == 19: max_value = max(dim19, key=operator.itemgetter(1)) asin_ = max_value[0] value_ = max_value[1] print 'dim:',d,', itemID: ', asin_, ', value = ', value_ How about something like, dims = [[] for _ in range(20)] for i in range(20): dims[i].append((asin, self.theta_item_per_bin[bin][itemID][i])) for j in range(self.K2): max_value = max(dims[j], key=operator.itemgetter(1)) asin_ = max_value[0] value_ = max_value[1] print 'dim:', j,', itemID: ', asin_, ', value = ', value_ I haven't looked at it in detail, and I'm not sure why you have exactly 20 lists or what happens if self.K2 is not equal to 20. But you can make the code simpler and shorter, and marginally more efficient by not testing all those if statements. Duncan -- https://mail.python.org/mailman/listinfo/python-list
How to use asyncore with SSL?
I've been trying to use the secure smtpd module from https://github.com/bcoe/secure-smtpd, but the SSL support seems to be fundamentally broken. That module simply wraps a socket and then expects to use it in the normal way via asyncore. Of course that fails the first time an ssl-wrapped-socket's send or recv method raises SSLWantReadError or SSLWantWriteError. Those exceptions aren't handled and it crashes. That makes the SSL support pretty much useless. I'm trying to fix that, but I can't find any information or documentation about using asyncore with SSL. Alternatively, a pointer to a simpler smtp server library that supports SSL would be great. The use of asyncore and multiprocessing process pools by this module is _way_ overkill for my needs and results in something that 1) doesn't work, and 2) can't be debugged. -- Grant Edwards grant.b.edwardsYow! Pardon me, but do you at know what it means to be gmail.comTRULY ONE with your BOOTH! -- https://mail.python.org/mailman/listinfo/python-list
Re: Where are the moderators?
On Thursday, January 18, 2018 at 10:38:18 PM UTC, Mike Driscoll wrote: > Hi, > > What happened to the moderators? I have always liked this forum, but there's > so much spam now. Is there a way to become a moderator so this can be cleaned > up? > > Thanks, > Mike Simply point your email client at news.gmane.org and take your pick from hundreds of Python lists and thousands of other technical lists that are all spam free. -- Kindest regards. Mark Lawrence. -- https://mail.python.org/mailman/listinfo/python-list
Re: Where are the moderators?
On 2018-01-18 22:38, Mike Driscoll wrote: Hi, What happened to the moderators? I have always liked this forum, but there's so much spam now. Is there a way to become a moderator so this can be cleaned up? How are you viewing the list? If you're viewing via Google Groups, then complain about the spam to Google. I'm subscribed to the list proper, and I'm not having a problem with spam; it's filtered out. See: https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: Very strange issues with collections.Mapping
Probably not what you want to hear, but this might be a good place for an SSCCE. On Thu, Jan 18, 2018 at 1:37 PM, Jason Swails wrote: > Hello! > > I am running into a very perplexing issue that is very rare, but creeps up > and is crashing my app. > > The root cause of the issue comes down to the following check returning > true: > > isinstance([], collections.Mapping) > > Obviously you can get this behavior if you register `list` as a subclass of > the Mapping ABC, but I'm not doing that. Because the issue is so rare (but > still common enough that I need to address it), it's hard to reproduce in a > bench test. > > What I am going to try is to essentially monkey-patch > collections.Mapping.register with a method that dumps a stack trace > whenever it's called at the time of initial import so I can get an idea of > where this method could *possibly* be getting called with a list as its > argument. > > The annoying thing here is that wherever the bug is happening, the crash > happens *way* far away (in a third-party library). I've also determined it > as the root cause of two crashes that seem completely unrelated (except > that the crash is caused by a list not behaving like a dict shortly after > isinstance(obj, collections.Mapping) returns True). These are the > libraries I'm using: > > amqp > billiard > celery > dj-database-url > Django > django-redis-cache > enum34 > gunicorn > kombu > newrelic > psycopg2 > pyasn1 > pytz > redis > requests > rsa > six > vine > voluptuous > > It's a web application, as you can probably tell. The main reason I ask > here is because I'm wondering if anybody has encountered this before and > managed to hunt down which of these libraries is doing something naughty? > > Thanks! > Jason > > -- > Jason M. Swails > -- > https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: Where are the moderators?
On Thu, Jan 18, 2018 at 4:28 PM, wrote: > On Thursday, January 18, 2018 at 10:38:18 PM UTC, Mike Driscoll wrote: > > Hi, > > > > What happened to the moderators? I have always liked this forum, but > there's so much spam now. Is there a way to become a moderator so this can > be cleaned up? > > > > Thanks, > > Mike > > Simply point your email client at news.gmane.org and take your pick from > hundreds of Python lists and thousands of other technical lists that are > all spam free. > > Actually, this link is broken. But, gmane.org (without the news prefix) does work. -- Listen to my FREE CD at http://www.mellowood.ca/music/cedars Bob van der Poel ** Wynndel, British Columbia, CANADA ** EMAIL: [email protected] WWW: http://www.mellowood.ca -- https://mail.python.org/mailman/listinfo/python-list
Re: Where are the moderators?
>> Simply point your email client at news.gmane.org and take your pick from >> hundreds of Python lists and thousands of other technical lists that are >> all spam free. >> >> > Actually, this link is broken. But, gmane.org (without the news prefix) > does work. Neither works for me. The news link clearly fails, but the gmane.org link has no search functionality either. Do you have a direct URL to comp.lang.python? Skip -- https://mail.python.org/mailman/listinfo/python-list
Re: Where are the moderators?
On Fri, 19 Jan 2018 00:43:36 +, MRAB wrote: > If you're viewing via Google Groups, then complain about the spam to > Google. Alternatively, and just as effectively, you could repeatedly hit yourself on the head with a ball-peen hammer. That will be just as effective at filtering the spam, but with the added benefit that it will feel so much better when you stop. *wink* -- Steve -- https://mail.python.org/mailman/listinfo/python-list
Re: Very strange issues with collections.Mapping
On Thu, 18 Jan 2018 16:37:18 -0500, Jason Swails wrote: > The root cause of the issue comes down to the following check returning > true: > > isinstance([], collections.Mapping) I re-iterate Chris' suggestion that you check that the instance is an actual list, not a subclass. Can you grep the offending files for something like class .*\(.*list.*\): to see if anything subclasses list? If so, does it look like the subclass is intended to offer a mapping interface? > Obviously you can get this behavior if you register `list` as a subclass > of the Mapping ABC, but I'm not doing that. Because the issue is so > rare (but still common enough that I need to address it), it's hard to > reproduce in a bench test. Ah lovely, a heisenbug. > What I am going to try is to essentially monkey-patch > collections.Mapping.register with a method that dumps a stack trace > whenever it's called at the time of initial import so I can get an idea > of where this method could *possibly* be getting called with a list as > its argument. Seems like a reasonable step to me. Also you could try identifying the offending library by interleaving assertions between the imports: import collections assert not isinstance([], collections.Mapping) import amqp assert not isinstance([], collections.Mapping) import billiard assert not isinstance([], collections.Mapping) import celery etc. When (if) the assertion fails, you know which library is to blame. Not exactly the nicest code, but it is only there until you locate (and fix) the problem. -- Steve -- https://mail.python.org/mailman/listinfo/python-list
Context manager able to write to the caller's namespace
I'm looking for a solution, or at least hints, to this problem. I want to define a context manager in one module: # a.py def CM: def __enter__(self): return self def __exit__(self, *args): pass Then call it from another module: # b.py import a with a.CM() as spam: x = 1 y = 2 in such a way that the spam context manager can, on exit, see the callers namespace and write to it. E.g. as a toy example (this isn't what I actually want to do!) we might have this: with a.CM() as spam: x = 1 print(x) # prints 2, not 1 I stress that's not the intended functionality, it just demonstrates the requirement. Don't assume that the content manager is going to be called in the global scope. In fact, the most common use I'm expecting is to call it from inside a class: class Aardvark: with a.CM() as spam: x = 1 It's okay if it doesn't work inside a function: def foo(): with a.CM() as spam: x = 1 assert x == 2 since function scopes in CPython are weird. Any suggestions? -- Steve -- https://mail.python.org/mailman/listinfo/python-list
Anaconda installation problem
Hi, When I was installing windows Anaconda 3 64 bit, there is a pop up window saying that the 'python program is closing'. I ignored it and continued the installation. When the Anaconda Prompt is launched, I have this error popped at the top (refer to attached doc.) Does that mean that the batch file is missing? I have tried re-installing and restarted my PC, however, the problem still persists. Also, I further checked that the python version installed is python 2.3.4 instead ( refer to Python 2.3.4 existance). Kindly advise. Regards, JS (Jing Shen) -- https://mail.python.org/mailman/listinfo/python-list
Re: How to use asyncore with SSL?
Grant Edwards : > I've been trying to use the secure smtpd module from > https://github.com/bcoe/secure-smtpd, but the SSL support seems to be > fundamentally broken. That module simply wraps a socket and then > expects to use it in the normal way via asyncore. > > Of course that fails the first time an ssl-wrapped-socket's send or > recv method raises SSLWantReadError or SSLWantWriteError. Those > exceptions aren't handled and it crashes. > > That makes the SSL support pretty much useless. > > I'm trying to fix that, but I can't find any information or > documentation about using asyncore with SSL. I'm all in for asynchronous programming, but asyncore is a bit too naive of an approach and shouldn't be used for anything serious. Python3, of course, has the asyncio framework. Additionally, I seem to recall Python's TLS support really supported synchronous processing only (based on some experimentation of my own). I hope I'm wrong on that. > Alternatively, a pointer to a simpler smtp server library that > supports SSL would be great. The use of asyncore and multiprocessing > process pools by this module is _way_ overkill for my needs and > results in something that 1) doesn't work, and 2) can't be debugged. Haven't tried it myself, but I supposed Twisted might be what you're looking for. Myself, I've written several "asyncore" replacements in Python as well as an SMTP server for my personal email needs. You could also consider writing your own implementation. For async, there's select.epoll and the like (assuming Linux), and SMTP is rather a simple protocol. Marko -- https://mail.python.org/mailman/listinfo/python-list
Re: Context manager able to write to the caller's namespace
On Fri, Jan 19, 2018 at 3:48 PM, Steven D'Aprano wrote: > I want to define a context manager in one module: > > # a.py > def CM: > def __enter__(self): > return self > def __exit__(self, *args): > pass > > > Then call it from another module: > > # b.py > import a > with a.CM() as spam: > x = 1 > y = 2 > > > in such a way that the spam context manager can, on exit, see the callers > namespace and write to it. E.g. as a toy example (this isn't what I > actually want to do!) we might have this: > > with a.CM() as spam: > x = 1 > print(x) > # prints 2, not 1 > > I stress that's not the intended functionality, it just demonstrates the > requirement. I'm assuming that you don't have to magically know that x was assigned to, as that's its own separate problem. AIUI, you have three separate cases: 1) Context manager was called from global scope, and needs access to globals() or locals() as returned in the caller 2) Ditto ditto class scope, and needs access to the phantom environment that's going to get put into the new class's dict 3) Ditto ditto function scope, which you specifically said is okay to fail, but which also wants to be locals(). So basically, you want to call locals() in the caller's scope. I don't think there's any standard way to do that, so you're going to be stuck with sys._getframe() and the cross-implementation incompatibilities that entails. But in CPython (tested in 3.7, should be fine in older versions), this ought to work: >>> def mutate(): ... locals = sys._getframe(1).f_locals ... locals["x"] = 4 ... >>> x 1 >>> mutate() >>> x 4 >>> class Foo: ... x = 2 ... print(x) ... mutate() ... print(x) ... 2 4 Written as a context manager: >>> import contextlib >>> @contextlib.contextmanager ... def mutate(): ... yield ... locals = sys._getframe(2).f_locals ... locals["x"] = 4 ... >>> class Foo: ... x = 2 ... print(x) ... with mutate(): ... x = 3 ... print(x) ... print(x) ... 2 3 4 >>> There's basically zero guarantees about this though. Have fun. :) ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Context manager able to write to the caller's namespace
On Fri, 19 Jan 2018 16:49:49 +1100, Chris Angelico wrote: [...] > 1) Context manager was called from global scope, and needs access to > globals() or locals() as returned in the caller A! /facepalm Of course the caller can just pass locals() to the context manager. Why didn't I think of that? I think I'm okay with that as a solution, but... def mutate(): > ... locals = sys._getframe(1).f_locals ... locals["x"] = 4 > ... I'll play around with that too. Just for kicks :-) -- Steve -- https://mail.python.org/mailman/listinfo/python-list
Re: Re: Context manager able to write to the caller's namespace
On Fri, Jan 19, 2018 at 5:00 PM, wrote: > Hello, > > Thank you for your mail. I will answer as fast as possible. If you're going to subscribe to a mailing list, PLEASE disable your autoresponder. Otherwise, messages like this will get marked as Spam, and your legit mail will end up getting tarred with the same metaphorical brush. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
