Re: Waht do you think about my repeated_timer class
On 2022-02-03 at 05:52:19 +0100, Cecil Westerhof via Python-list wrote: > [email protected] writes: > > > FWIW, I'd find some way to tell users the units (seconds, milliseconds, > > fortnights, etc.) instead of making them wade through your code to find > > the call to (and possibly the [broken] help text of) Timer. > > You mean with docstring? Docstring, comments, error/exception text, external documentation, URLs or other references in the source code (docstring, comments), the name of the object in question (e.g., instead of "interval," call it "interval_seconds," or "seconds_between_runs"). *Something*. Or more than one of the above. A reference to the Timer class's documentation. There are a lot of options. Pick a place (or more than one!) that will be fairly conspicuous in as many situations to as many users (including future you) or potential users (including your future code) as possible. In a perfect world, you will receive feedback and/or use your own code, and improve your choice(s) and method(s) over time. -- https://mail.python.org/mailman/listinfo/python-list
Re: Waht do you think about my repeated_timer class
On Feb 2, 2022 23:31, Barry wrote: > On 2 Feb 2022, at 21:12, Marco Sulla wrote: > > You could add a __del__ that calls stop :) Didn't python3 make this non deterministic when del is called? I thought the recommendation is to not rely on __del__ in python3 code. ==> Adding __del__ also poses chalenges is you would like to support pypy: "There are a few extra implications from the difference in the GC. Most notably, if an object has a __del__, the __del__ is never called more than once in PyPy; but CPython will call the same __del__ several times if the object is resurrected and dies again (at least it is reliably so in older CPythons; newer CPythons try to call destructors not more than once, but there are counter-examples). The __del__ methods are called in "the right" order if they are on objects pointing to each other, as in CPython, but unlike CPython, if there is a dead cycle of objects referencing each other, their __del__ methods are called anyway; CPython would instead put them into the list garbage of the gc module." https://doc.pypy.org/en/latest/cpython_differences.html -- https://mail.python.org/mailman/listinfo/python-list
A Short Survey To Understand Practitioner' Perspectives Towards The Requirements Engineering
Dear Sir or Madam, We prepared a short survey to understand practitioners’ perspectives towards the requirements engineering. Our survey basically aims to clarify on many aspects of the requirements engineering applied in industry, including (i) requirements gathering and specifications, (ii) requirements modifications, (iii) requirements analysis, and (iv) requirements transformation. The survey results will be submitted to a reputable journal on software engineering. The survey takes about 2-5 minutes to participate, we would be so grateful if you could separate your time. Also, please circulate the email to anyone who may be interested. The survey link: https://forms.gle/DhLqr15GXVhJhzzy6 All the best, Etem Çetin Toptani -- *Bu mesajı yazdırmadan önce çevreye verebileceğiniz zararları bir kez daha düşününüz. * *Think of the environment once more before printing out this message.* -- *Bu mesajı yazdırmadan önce çevreye verebileceğiniz zararları bir kez daha düşününüz. * *Think of the environment once more before printing out this message.* -- https://mail.python.org/mailman/listinfo/python-list
Pypy with Cython
Hi, I inherited a fairly large codebase that I need to port to Python 3. Since the program was running quite slow I am also running the unittests against pypy3.8. It's a long running program that does lots of pairwise comparisons of string values in two files. Some parts of the program (e.g a modulo 11 digit check) are implemented in Cython. Should I use pure Python instead when using Pypy? I compiled the Cython modules for pypy and they work, but I'm afraid they might just slow things down. Thanks! Albert-Jan -- https://mail.python.org/mailman/listinfo/python-list
Re: Pypy with Cython
The best answer to "is this slower on Pypy" is probably to measure. Sometimes it makes sense to rewrite C extension modules in pure python for pypy. On Thu, Feb 3, 2022 at 7:33 AM Albert-Jan Roskam wrote: >Hi, >I inherited a fairly large codebase that I need to port to Python 3. > Since >the program was running quite slow I am also running the unittests > against >pypy3.8. It's a long running program that does lots of pairwise >comparisons of string values in two files. Some parts of the program > (e.g >a modulo 11 digit check) are implemented in Cython. Should I use pure >Python instead when using Pypy? I compiled the Cython modules for pypy > and >they work, but I'm afraid they might just slow things down. >Thanks! >Albert-Jan > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: Pypy with Cython
On Feb 3, 2022 17:01, Dan Stromberg wrote: > The best answer to "is this slower on > Pypy" is probably to measure. > Sometimes it makes sense to rewrite C > extension modules in pure python for pypy. Hi Dan, thanks. What profiler do you recommend I normally use cProfile, but I was thinking about this one: https://pyinstrument.readthedocs.io/en/latest/index.html -- https://mail.python.org/mailman/listinfo/python-list
ssl server: how to disable client cert verfication?
I've got a small ssl server app. I want to require a certificate from the client, so I'm using a context with context.verify_mode = ssl.CERT_REQUIRED But, I want all certificates accepted. How do I disable client certificate verification? -- Grant -- https://mail.python.org/mailman/listinfo/python-list
ssl: why wrap newly accept()ed connections?
According to the docs, when you accept() an ssl connection, you need to wrap the new connection: https://docs.python.org/3/library/ssl.html?highlight=ssl#ssl-sockets When a client connects, you’ll call accept() on the socket to get the new socket from the other end, and use the context’s SSLContext.wrap_socket() method to create a server-side SSL socket for the connection: while True: newsocket, fromaddr = bindsocket.accept() connstream = context.wrap_socket(newsocket, server_side=True) try: deal_with_client(connstream) finally: connstream.shutdown(socket.SHUT_RDWR) connstream.close() However, example server code I've found does not wrap the newly accepted connection. I've checked, and newsocket is already an object. The examples I've seen/tried simply call ..recv() and .send() methods of newsocket, and that seems to work fine. What is the purpose of wrapping newsocket? -- https://mail.python.org/mailman/listinfo/python-list
Re: ssl: why wrap newly accept()ed connections?
On Thu, Feb 03 2022 at 11:17:17 AM, Grant Edwards wrote: > According to the docs, when you accept() an ssl connection, > you need to wrap the new connection: > > https://docs.python.org/3/library/ssl.html?highlight=ssl#ssl-sockets > >When a client connects, you’ll call accept() on the socket to get >the new socket from the other end, and use the context’s >SSLContext.wrap_socket() method to create a server-side SSL socket >for the connection: > > while True: > newsocket, fromaddr = bindsocket.accept() > connstream = context.wrap_socket(newsocket, server_side=True) > try: > deal_with_client(connstream) > finally: > connstream.shutdown(socket.SHUT_RDWR) > connstream.close() > > However, example server code I've found does not wrap the newly > accepted connection. I've checked, and newsocket is already an > object. The examples I've seen/tried simply call > ..recv() and .send() methods of newsocket, and that seems to work fine. > > What is the purpose of wrapping newsocket? That section is talking about using an "ordinary" socket for the server. bindsocket is a socket.socket. If bindsocket was already a ssl.SSLSocket, the wrapping would be already done by accept. I suppose this kind of functionality is useful for protocols that start off as cleartext and then switch to TLS (such as the mail-related protocols that use STARTTLS). -- regards, kushal -- https://mail.python.org/mailman/listinfo/python-list
Re: ssl server: how to disable client cert verfication?
On Thu, Feb 03 2022 at 10:57:56 AM, Grant Edwards wrote: > I've got a small ssl server app. I want to require a certificate from > the client, so I'm using a context with > > context.verify_mode = ssl.CERT_REQUIRED > > But, I want all certificates accepted. How do I disable client > certificate verification? > Perhaps you can explain what your goal is. Which kinds of client certificates do you want to permit (to the best of my knowledge, none of these can be actually allowed): - expired certificates - self-signed certificates - certificates signed by untrusted CA - completely garbage certificates (bad signature, etc.) I don't see what benefit you expect from requiring client certificates if you don't care what the certificate says. Why not simply set verify_mode to SSL_NONE and use other authentication mechanisms? -- regards, kushal -- https://mail.python.org/mailman/listinfo/python-list
Re: ssl: why wrap newly accept()ed connections?
On 2022-02-03, Kushal Kumaran wrote: > >> [...] >> However, example server code I've found does not wrap the newly >> accepted connection. I've checked, and newsocket is already an >> object. [...] >> >> What is the purpose of wrapping newsocket? > > That section is talking about using an "ordinary" socket for the server. Yep, I missed that. > bindsocket is a socket.socket. If bindsocket was already a > ssl.SSLSocket, the wrapping would be already done by accept. Thanks! -- Grant -- https://mail.python.org/mailman/listinfo/python-list
Re: ssl server: how to disable client cert verfication?
On 2022-02-03, Kushal Kumaran wrote: > On Thu, Feb 03 2022 at 10:57:56 AM, Grant Edwards > wrote: >> I've got a small ssl server app. I want to require a certificate from >> the client, so I'm using a context with >> >> context.verify_mode = ssl.CERT_REQUIRED >> >> But, I want all certificates accepted. How do I disable client >> certificate verification? >> > > Perhaps you can explain what your goal is. It's a troubleshooting utility for displaying a client's certificate. > Which kinds of client certificates do you want to permit All of them. Anything that's parsable as an X509 certificate no matter how "invalid" it is. > (to the best of my knowledge, none of these can be actually allowed): > > - expired certificates > - self-signed certificates > - certificates signed by untrusted CA > - completely garbage certificates (bad signature, etc.) > > I don't see what benefit you expect from requiring client > certificates if you don't care what the certificate says. I do care what it says. The whole point is to find out what it says. I just don't want it validated by the SSL layer: I want to print it out. That seems to be trivial to do for server certificates using "openssl s_client", but I can't find any way to do it for client certficates. > Why not simply set verify_mode to SSL_NONE and use other > authentication mechanisms? I'm not interested in doing any authentication. I just want to require that the client provide a certificate and then print it out using print(connection.getpeercert()) -- Grant -- https://mail.python.org/mailman/listinfo/python-list
Re: Waht do you think about my repeated_timer class
> On 3 Feb 2022, at 04:45, Cecil Westerhof via Python-list > wrote: > > Have to be careful that timing keeps correct when target takes a 'lot' > of time. > Something to ponder about, but can wait. You have noticed that your class does call the function at the repeat interval but rather at the repeat interval plus processing time. The way to fix this is to subtract the last processing elapsed time for the next interval. Sort of a software phase locked loop. Just before you call the run function record the time.time() as start_time. Then you can calculate next_interval = max( .001, interval - time.time() - start_time) I use 1ms as the min interval. Barry -- https://mail.python.org/mailman/listinfo/python-list
Re: ssl server: how to disable client cert verfication?
> On 3 Feb 2022, at 21:34, Grant Edwards wrote: > > On 2022-02-03, Kushal Kumaran wrote: > >>> On Thu, Feb 03 2022 at 10:57:56 AM, Grant Edwards >>> wrote: >>> I've got a small ssl server app. I want to require a certificate from >>> the client, so I'm using a context with >>> >>> context.verify_mode = ssl.CERT_REQUIRED >>> >>> But, I want all certificates accepted. How do I disable client >>> certificate verification? >>> >> >> Perhaps you can explain what your goal is. > > It's a troubleshooting utility for displaying a client's certificate. > >> Which kinds of client certificates do you want to permit > > All of them. Anything that's parsable as an X509 certificate no matter > how "invalid" it is. > >> (to the best of my knowledge, none of these can be actually allowed): >> >> - expired certificates >> - self-signed certificates >> - certificates signed by untrusted CA >> - completely garbage certificates (bad signature, etc.) >> >> I don't see what benefit you expect from requiring client >> certificates if you don't care what the certificate says. > > I do care what it says. The whole point is to find out what it says. > > I just don't want it validated by the SSL layer: I want to print it > out. That seems to be trivial to do for server certificates using > "openssl s_client", but I can't find any way to do it for client > certficates. > >> Why not simply set verify_mode to SSL_NONE and use other >> authentication mechanisms? > > I'm not interested in doing any authentication. > > I just want to require that the client provide a certificate and then > print it out using print(connection.getpeercert()) I am not near the pc with the code on. But in outline you provide a ssl context that returns true for the validation of the cert always. You also get to have x509 cert in your hands. I use pyopenssl to play with x.509 certs. Let me know if this is not enough info and I will dig out the code I have that does this custom cert stuff. Barry > > -- > Grant > > > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: ssl server: how to disable client cert verfication?
On 2022-02-03, Barry wrote: > >> [...] I just want to require that the client provide a certificate >> and then print it out using print(connection.getpeercert()) > > I am not near the pc with the code on. But in outline you provide a > ssl context that returns true for the validation of the cert always. I thought that was what I was asking. How do you create an ssl context that requests a client certificate but then treats any received client certificate as valid? I've looked through the ssl.Context documentation multiple times, and haven't been able to spot any option or flag that disables client certificate validation or allows the user to override the actual client certificate validation process. > You also get to have x509 cert in your hands. I use pyopenssl to > play with x.509 certs. I don't have any problem getting and printing the certificate once the connection is established. The problem is preventing the handshake from failing when the client certificate isn't valid and signed by a CA provided to the context with .load_verify_locations(). > Let me know if this is not enough info and I will dig out the code I > have that does this custom cert stuff. -- Grant -- https://mail.python.org/mailman/listinfo/python-list
Re: Waht do you think about my repeated_timer class
Barry writes: >> On 3 Feb 2022, at 04:45, Cecil Westerhof via Python-list >> wrote: >> >> Have to be careful that timing keeps correct when target takes a 'lot' >> of time. >> Something to ponder about, but can wait. > > You have noticed that your class does call the function at the repeat > interval but > rather at the repeat interval plus processing time. Nope: def _next(self): self._timer = Timer(self._interval, self._run) self._timer.start() def _run(self): self._next() self._fn() In _run I first set the new timer and then I execute the function. So that will go mostly OK. > The way to fix this is to subtract the last processing elapsed time for the > next interval. > Sort of a software phase locked loop. > > Just before you call the run function record the time.time() as start_time. > Then you can calculate next_interval = max( .001, interval - time.time() - > start_time) > I use 1ms as the min interval. But I am working on a complete rewrite to create a more efficient class. (This means I have to change also the code that uses it.) There I have to do something like you suggest. (I am already working on it.) Personally I am also of the opinion that the function should finish in less as 10% from the interval. (That was one of my rewrites.) -- Cecil Westerhof Senior Software Engineer LinkedIn: http://www.linkedin.com/in/cecilwesterhof -- https://mail.python.org/mailman/listinfo/python-list
Re: Pypy with Cython
On 4/02/22 5:07 am, Albert-Jan Roskam wrote: On Feb 3, 2022 17:01, Dan Stromberg wrote: What profiler do you recommend If it runs for that long, just measuring execution time should be enough. Python comes with a "timeit" module to help with that, or you can use whatever your OS provides (e.g. the "time" shell command in unices). -- Greg -- https://mail.python.org/mailman/listinfo/python-list
Re: Pypy with Cython
On Thu, 3 Feb 2022 at 23:16, Greg Ewing wrote: > > On 4/02/22 5:07 am, Albert-Jan Roskam wrote: > > On Feb 3, 2022 17:01, Dan Stromberg wrote: > > > > What profiler do you recommend > > If it runs for that long, just measuring execution time should > be enough. Python comes with a "timeit" module to help with > that, or you can use whatever your OS provides (e.g. the > "time" shell command in unices). Yes, exactly. It's important to distinguish between timing or benchmarking as compared to profiling. When you use a profiler it does not usually give an accurate representation of the time taken by the same code when the profiler is not running because of the overhead added by the profiler itself. The purpose of the profiler is instead to give lots of information that can help you to *think* about how to make the code faster (e.g. what proportion of time is spent in different functions or how many times is a function called etc). This information is useful for considering at a *high level* what parts of the code could be improved to make it faster by e.g. calling a particular function less or making that function faster. The timeit module can be used to time micro-operations i.e. things that take nanoseconds or maybe milliseconds. It repeats an operation in a loop which is often needed to get reliable timings for things that are otherwise too fast to reliably time from a single run. It can give information that helps to identify good approaches to try at a *low level* e.g. when optimising a single line of code. If you want to *evaluate* whether or not a change actually makes your *whole* program faster you should just run your code normally but time how long it takes (which is what the unix "time" command does). You can also use time.time() from Python for this. Profilers and timeit help to identify ways to speed up your code but should not be used as the way to assess the final impact of the changes you make to a long running program. -- Oscar -- https://mail.python.org/mailman/listinfo/python-list
Re: ssl server: how to disable client cert verfication?
On Fri, 4 Feb 2022 at 09:37, Grant Edwards wrote: > I've looked through the ssl.Context documentation multiple times, and > haven't been able to spot any option or flag that disables client > certificate validation or allows the user to override the actual > client certificate validation process. What you're doing is a little unusual, so my first thought would be to subclass Context and override whatever method does the checks. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: ssl server: how to disable client cert verfication?
On Thu, Feb 03 2022 at 01:32:04 PM, Grant Edwards wrote: > On 2022-02-03, Kushal Kumaran wrote: > >> On Thu, Feb 03 2022 at 10:57:56 AM, Grant Edwards >> wrote: >>> I've got a small ssl server app. I want to require a certificate from >>> the client, so I'm using a context with >>> >>> context.verify_mode = ssl.CERT_REQUIRED >>> >>> But, I want all certificates accepted. How do I disable client >>> certificate verification? >>> >> >> Perhaps you can explain what your goal is. > > It's a troubleshooting utility for displaying a client's certificate. > >> Which kinds of client certificates do you want to permit > > All of them. Anything that's parsable as an X509 certificate no matter > how "invalid" it is. > Does `openssl x509 -in -text -noout` do what you want? >> (to the best of my knowledge, none of these can be actually allowed): >> >> - expired certificates >> - self-signed certificates >> - certificates signed by untrusted CA >> - completely garbage certificates (bad signature, etc.) >> >> I don't see what benefit you expect from requiring client >> certificates if you don't care what the certificate says. > > I do care what it says. The whole point is to find out what it says. > > I just don't want it validated by the SSL layer: I want to print it > out. That seems to be trivial to do for server certificates using > "openssl s_client", but I can't find any way to do it for client > certficates. > In your place, I would simply use the openssl x509 command. If I wanted more/different info, I would write a script to load the certificate and printed out the relevant info. If this functionality must be provided by a server, I would write it so that a certificate could be POSTed to the server (without using client certificates), and it would in turn do the parsing equivalent to what the standalone script would do and respond with the relevant info. (But I hear X.509 parsing is an esoteric mess, and it's unclear to me what demons lie in the area of parsing untrusted X.509 content). I don't know how to use the stdlib's ssl module to do this kind of parsing. The cryptography package makes this simple though: https://cryptography.io/en/latest/x509/reference/#loading-certificates >> Why not simply set verify_mode to SSL_NONE and use other >> authentication mechanisms? > > I'm not interested in doing any authentication. > > I just want to require that the client provide a certificate and then > print it out using print(connection.getpeercert()) > -- regards, kushal -- https://mail.python.org/mailman/listinfo/python-list
