Postgresql equivalent of Python's timeit?
Hi, This is more related to Postgresql than to Python, I hope this is ok. I want to measure Postgres queries N times, much like Python timeit (https://docs.python.org/3/library/timeit.html). I know about EXPLAIN ANALYZE and psql \timing, but there's quite a bit of variation in the times. Is there a timeit-like function in Postgresql? Thanks! Albert-Jan -- https://mail.python.org/mailman/listinfo/python-list
Re: Postgresql equivalent of Python's timeit?
On Sep 15, 2023 19:45, "Peter J. Holzer via Python-list" wrote: On 2023-09-15 17:42:06 +0200, Albert-Jan Roskam via Python-list wrote: > This is more related to Postgresql than to Python, I hope this is ok. > I want to measure Postgres queries N times, much like Python timeit > (https://docs.python.org/3/library/timeit.html). I know about EXPLAIN > ANALYZE and psql \timing, but there's quite a bit of variation in the > times. Is there a timeit-like function in Postgresql? Why not simply call it n times from Python? (But be aware that calling the same query n times in a row is likely to be unrealistically fast because most of the data will already be in memory.) = Thanks, I'll give this a shot. Hopefully the caching is not an issue if I don't re-use the same database connection. -- https://mail.python.org/mailman/listinfo/python-list
pathlib.Path.is_file vs os.path.isfile difference
Hi, I was replacing some os.path stuff with Pathlib and I discovered this: Path(256 * "x").is_file() # OSError os.path.isfile(256 * "x") # bool Is this intended? Does pathlib try to resemble os.path as closely as possible? Best wishes, Albert-Jan -- https://mail.python.org/mailman/listinfo/python-list
Re: pathlib.Path.is_file vs os.path.isfile difference
On Mar 8, 2024 19:35, Thomas Passin via Python-list wrote: On 3/8/2024 1:03 PM, Albert-Jan Roskam via Python-list wrote: > Hi, > I was replacing some os.path stuff with Pathlib and I discovered this: > Path(256 * "x").is_file() # OSError > os.path.isfile(256 * "x") # bool > Is this intended? Does pathlib try to resemble os.path as closely as > possible? You must have an very old version of Python. I'm running 3.12.2 and it returns False. Either that or that path name exists and throws some kind of unexpected exception. Hi, I tested this with Python 3.8. Good to know that this was fixed! -- https://mail.python.org/mailman/listinfo/python-list
Re: pathlib.Path.is_file vs os.path.isfile difference
On Mar 10, 2024 12:59, Thomas Passin via Python-list wrote: On 3/10/2024 6:17 AM, Barry wrote: > > >> On 8 Mar 2024, at 23:19, Thomas Passin via Python-list wrote: >> >> We just learned a few posts back that it might be specific to Linux; I ran it on Windows. > > Depending on the exact win32 api used there is a 257 limit on windows. > The 257 includes 2 for the device, C:, and 255 for the path part that will use 1 for the leading \. Getting an error for a name that is 255 is not surprising. > > Other api allow for 65535 limit, not sure on its additional limits. I seem to remember there is a setting to allow longer paths, but I forget any details. = You mean the "\\?\" prefix? https://learn.microsoft.com/en-us/windows/win32/fileio/maximum-file-path-limitation?tabs=registry -- https://mail.python.org/mailman/listinfo/python-list
Re: Suggested python feature: allowing except in context maneger
The example exception is not what bothers me. The syntax change is nowhere near as useful as `with` and context managers. They provide an excellent idiom for resource usage and release. Your suggestion complicates the `with` statement and brings only a tiny indentation reduction over the `with`-inside-`try` idiom. It brings no semantic changes or new features. I also don't see the added value. If you desperately want to get rid of an indentation level, you could use an except hook. https://docs.python.org/3/library/sys.html#sys.excepthook -- https://mail.python.org/mailman/listinfo/python-list
Re: Best use of "open" context manager
Or like below, although pylint complains about this: "consider using with". Less indentation this way. f = None try: f = open(FILENAME) records = f.readlines() except Exception: sys.exit(1) finally: if f is not None: f.close() -- https://mail.python.org/mailman/listinfo/python-list
Re: Error codes
On Aug 13, 2024 15:29, Barry Scott via Python-list wrote: > Could not find file 'C:\Users\Charl\OneDrive\Documents\The Sims 4 Mod Constructor\Projects\MetalMummysMods_Ehlers-DanlosMod\Python\__pycache__\MetalMummysMods_Ehlers-DanlosMod.cpython-37.pyc'. > Element ID: (No Element) > Element Name: (No Element) Wild guess: do you have sufficicient permissions to write the .pyc files? What happens if you run the program with the -B option? https://docs.python.org/3/using/cmdline.html#envvar-PYTHONDONTWRITEBYTECODE -- https://mail.python.org/mailman/listinfo/python-list
Re: ListAdmin: Is list/archive working correctly?
I also think that list/archive isn't working properly. Very little emails. Before, this was quite a busy list. -- https://mail.python.org/mailman/listinfo/python-list
Synchronise annotations -> docstring
Hi, Are there any tools that check whether type annotations and Numpydoc strings are consistent? I did find this Vim plugin: https://lxyuan0420.github.io/posts/til-vim-pydocstring-plugin. Looks incredibly useful, but I haven't tried it yet. Thanks! AJ -- https://mail.python.org/mailman/listinfo/python-list
Re: DRM is self-defeating (was: Encrypt python files)
- On Wed, May 6, 2015 9:41 AM CEST Ben Finney wrote: >Palpandi writes: > >> On Wednesday, May 6, 2015 at 12:07:13 PM UTC+5:30, Palpandi wrote: > >> > What are the ways to encrypt python files? >> >> No, I just want to hide the scripts from others. > >Which others? You can hide the scripts from them by never showing those >others the scripts. If you don't trust a recipient, don't let them >receive the file. >The only feasible solution is to distribute files only to those >recipients you want to have them, and can trust to do with them as you >ask. If you don't trust a recipient, don't hand the files to them. Can tools like py2exe or cx_freeze also be used to obfuscate python source code? -- https://mail.python.org/mailman/listinfo/python-list
Re: Encrypt python files
- On Wed, May 6, 2015 11:04 AM CEST Steven D'Aprano wrote: >On Wednesday 06 May 2015 17:23, Palpandi wrote: > >> On Wednesday, May 6, 2015 at 12:07:13 PM UTC+5:30, Palpandi wrote: >> Hi, >> >> What are the ways to encrypt python files? >> >> No, I just want to hide the scripts from others. > >Why, are you ashamed of your code? > >Python is free, open source software. Hiding the code from others is not a >priority for the developers of the language. Besides, you can't hide the >code unless you only operate the application via a web service, or similar. >As soon as you give people a copy of the code, whether it is binary code or >source code, they have a copy of it and can look at it and work out how what >it does. > >If "hiding the code" was good for security, why are there so many viruses >and spybots and worms and other malware for Windows? > >No, as far as I am concerned, trying to hide the code is a waste of time >with Python. But if you absolutely must, you can distribute the .pyc files >instead of the .py files, and that will discourage casual tinkerers from >poking around in the program. The .pyc file is compiled to byte-code rather >than source code, so it's not readable without running it through a >decompiler. I used the marshal module before as a faster alternative to shelve (I marshalled a huge dictionary). I always understood marshal files require the *exact* same interpreter version (incl. built number). Do the same limitations apply for .pyc files? Isn't it the same format? We have a VCS with an option to use private repos. I am always wary of users who want to use this feature. Why would you not want to share your code with your colleagues? Embarrassed about unreadable crappy code, perhaps? -- https://mail.python.org/mailman/listinfo/python-list
Re: Why does unicode-escape decode escape symbols that are already escaped?
-
On Sun, May 10, 2015 5:53 PM CEST Somelauw . wrote:
>In Python 3, decoding "€" with unicode-escape returns 'â\x82¬' which in my
>opinion doesn't make sense.
>The € already is decoded; if it were encoded it would look like this:
>'\u20ac'.
>So why is it doing this?
>
>In Python 2 the behaviour is similar, but slightly different.
>
>$ python3 -S
>Python 3.3.3 (default, Nov 27 2013, 17:12:35)
>[GCC 4.8.2] on linux
>>> import codecs
>>> codecs.decode('€', 'unicode-escape')
>'â\x82¬'
>>> codecs.encode('€', 'unicode-escape')
>b'\\u20ac'
>>>
>
>$ python2 -S
>Python 2.7.5+ (default, Sep 17 2013, 15:31:50)
>[GCC 4.8.1] on linux2
>>> import codecs
>>> codecs.decode('€', 'unicode-escape')
>u'\xe2\x82\xac'
>>> codecs.encode('€', 'unicode-escape')
>Traceback (most recent call last):
> File "", line 1, in
>UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 0:
>ordinal not in range(128)
>>>
Hi,
I only have Python 2 on my phone, but I am suprised that you (and are able to)
decode unicode strings. What result do you get when you do the following in
Python 3:
Python 2.7.2 (default, Oct 25 2014, 20:52:15)
[GCC 4.9 20140827 (prerelease)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import codecs
>>> codecs.decode(b'€', 'unicode-escape')
u'\xe2\x82\xac'
>>> codecs.encode(u'€', 'unicode-escape')
'\\xe2\\x82\\xac'
>>>
--
https://mail.python.org/mailman/listinfo/python-list
Re: Updating a package on PyPi, testing and etiquette
- On Tue, May 12, 2015 8:01 PM CEST Rob Gaddi wrote: >So I've got a package I put up on PyPi a while back (ctypes-bitfield, if >it matters). For version 0.2.6 I had access to some older versions of >Python and was able to run my test suite on Python 2.6 and 3.0. > >Well, I don't have them anymore. I've got no access right now to >anything older than 2.7 and 3.2, and my primary development environment >is 3.4. But I've also updated the package to support what I consider to >be some really nice new functionality. So, it comes down to two >questions: > >A) Is there any easy way to test against an older version of Python? >Preferably without trying to install entire old environments and keep >them nicely isolated from my actual doing work? I'm running Ubuntu for >what difference that makes. I use tox, it's available on pypi. Works with virtualenv. There's also detox, which runs your unittests in parellel, but I have not used that. -- https://mail.python.org/mailman/listinfo/python-list
Re: Looking for direction
- On Thu, May 14, 2015 3:35 PM CEST Dennis Lee Bieber wrote: >On Wed, 13 May 2015 16:24:30 -0700, 20/20 Lab declaimed >the following: > >>Now is were I have my problem: >> >>myList = [ [123, "XXX", "Item", "Qty", "Noise"], >>[72976, "YYY", "Item", "Qty", "Noise"], >>[123, "XXX" "ItemTypo", "Qty", "Noise"]] >> >>Basically, I need to check for rows with duplicate accounts row[0] and >>staff (row[1]), and if so, remove that row, and add it's Qty to the >>original row. I really dont have a clue how to go about this. The >>number of rows change based on which run it is, so I couldnt even get >>away with using hundreds of compare loops. >> >>If someone could point me to some documentation on the functions I would >>need, or a tutorial it would be a great help. >> > > This appears to be a matter of algorithm development -- there won't be >an pre-made "function" for it. The closest would be the summing functions >(control break http://en.wikipedia.org/wiki/Control_break ) of a report >writer application. > > The short gist would be: > > SORT the data by the account field > Initialize sum using first record > loop > read next record > if end of data > output sum record > exit > if record is same account as sum > add quantity to sum > else > output sum record > reset sum to the new record > > Granted -- loading the data into an SQL capable database would make >this simple... > > select account, sum(quantity) from table > order by account You could also use pandas. Read the data in a DataFrame, create a groupby object, use the sum() and the first() methods. http://pandas.pydata.org/pandas-docs/version/0.15.2/groupby.html -- https://mail.python.org/mailman/listinfo/python-list
Re: need help with an accessibility prototype
- On Fri, May 22, 2015 9:50 PM CEST Laura Creighton wrote: >In a message of Fri, 22 May 2015 12:29:20 -0400, "Eric S. Johansson" writes: >>2 needs. first is determining if NaturallySpeaking injects keycodes or >>ascii char into the windows input queue. second is building a test >>widget to capture and display text. >> >>I think I can solve both of these by building a simple text widget >>(tkinter? qt? ??) to capture keycodes. problem, is in >>yrs of programming, I've never written a GUI interface so I have no idea >>where to start. help, tutor, pointers to samples would be most welcome. >> >>--- eric > >The best online manual I know of for Tkinter is here: >http://effbot.org/tkinterbook/ despite being 10 years old. But then I >haven't looked for one for about that long, either. This book by John Shipman is also very good: http://infohost.nmt.edu/tcc/help/pubs/tkinter/ Albert-Jan -- https://mail.python.org/mailman/listinfo/python-list
programmatically change windows regional settings?
Hi, In Windows I can change the regional settings manually in the control panel. But how do I do this programmatically? I tried setting LANG but this does not work in Windows. Kernel32's SetLocaleInfo sounds promising, but "This setting only affects the user override portion of the locale settings; it does not set the system defaults." https://msdn.microsoft.com/en-us/library/ee491893(v=winembedded.60).aspx Another route might be _winreg (but that's probably more brittle and, more importantly, the registry sucks) My goal is to easily run my unittests in a number of locales (actually a platform x locale x python version matrix) Thanks! Albert-Jan -- https://mail.python.org/mailman/listinfo/python-list
Re: programmatically change windows regional settings?
- On Sun, May 24, 2015 3:07 PM CEST Mark Lawrence wrote: >On 24/05/2015 13:50, Albert-Jan Roskam via Python-list wrote: >> >> Hi, >> >> In Windows I can change the regional settings manually in the control panel. >> But how do I do this programmatically? I tried setting LANG but this does >> not work in Windows. >> >> Kernel32's SetLocaleInfo sounds promising, but "This setting only affects >> the user override portion of the locale settings; it does not set the system >> defaults." >> https://msdn.microsoft.com/en-us/library/ee491893(v=winembedded.60).aspx >> >> Another route might be _winreg (but that's probably more brittle and, more >> importantly, the registry sucks) >> >> My goal is to easily run my unittests in a number of locales (actually a >> platform x locale x python version matrix) >> >> Thanks! >> >> Albert-Jan >> > >You're probably better off asking this on >https://mail.python.org/mailman/listinfo/python-win32 which is also >available as gmane.comp.python.windows, although you might get lucky >here. Ok, I'll check that out, thanks. Hope that list is about more than win32com If you do a rather more specific statement than "this does not >work in Windows" would be helpful. One can set LANG in Windows (of course), but Windows does not appear to ever use it. So it's pointless. setx LANG France.French.1252 && python -c "from locale import *; setlocale(LC_ALL, ''); print(getlocale())" ... Does not print a French locale on my non-French system >Your Python and Windows version(s) >might possibly assist as well :) Windows Server 2012 R2 (x64), and preferably also Win 7 x64. Python 2.7, 3.3, 3.4, and ideally also pypy. >-- >My fellow Pythonistas, ask not what our language can do for you, ask >what you can do for our language. > >Mark Lawrence > >-- >https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: programmatically change windows regional settings?
- On Sun, May 24, 2015 4:45 PM CEST Laura Creighton wrote: >In a message of Sun, 24 May 2015 14:07:37 +0100, Mark Lawrence writes: >>On 24/05/2015 13:50, Albert-Jan Roskam via Python-list wrote: >> >> Hi, >> >> In Windows I can change the regional settings manually in the control panel. >> But how do I do this programmatically? I tried setting LANG but this does >> not work in Windows. >> >> Kernel32's SetLocaleInfo sounds promising, but "This setting only affects >> the user override portion of the locale settings; it does not set the system >> defaults." >> https://msdn.microsoft.com/en-us/library/ee491893(v=winembedded.60).aspx >> >> Another route might be _winreg (but that's probably more brittle and, more >> importantly, the registry sucks) >> >> My goal is to easily run my unittests in a number of locales (actually a >> platform x locale x python version matrix) >> >> Thanks! >> >> Albert-Jan > >>You're probably better off asking this on >>https://mail.python.org/mailman/listinfo/python-win32 which is also >>available as gmane.comp.python.windows, although you might get lucky >>here. If you do a rather more specific statement than "this does not >>work in Windows" would be helpful. Your Python and Windows version(s) >>might possibly assist as well :) >> >>-- >>My fellow Pythonistas, ask not what our language can do for you, ask >>what you can do for our language. >> >>Mark Lawrence > >I actually think he will get the best advice from the testing-in-python >mailing list. :-) That's indeed what I've done initially, but my message does not get posted. But maybe that list is moderated and I'm too impatient. But TIP is indeed the place to be for Tox-related questions. -- https://mail.python.org/mailman/listinfo/python-list
Re: What is considered an "advanced" topic in Python?
Metaclasses, abc, asyncio, ast, some of the dunder methods, eg __del__, weakref, perhaps gc-- https://mail.python.org/mailman/listinfo/python-list
Re: Find in ipython3
"To run any command at the system shell, simply prefix it with !" See: https://ipython.org/ipython-doc/dev/interactive/tutorial.html-- https://mail.python.org/mailman/listinfo/python-list
Re: How to check in script if Python or Jython is used
Sun, Jun 21, 2015 12:24 PM CEST Cecil Westerhof wrote: >On Sunday 21 Jun 2015 11:22 CEST, Laura Creighton wrote: > >> In a message of Sun, 21 Jun 2015 10:12:06 +0200, Cecil Westerhof >> writes: >> I installed Jython and will start playing with it. There probably >> will be differences between Python and Jython. Is there a way to >> determine if a script is run by Python or Jython? Then different >> execution paths could be taken. With sys.version(_info) you do not >> get this information. >> os.path.basename(sys.executable), if sys.implementation is not an option -- https://mail.python.org/mailman/listinfo/python-list
windows and file names > 256 bytes
Hi,
Consider the following calls, where very_long_path is more than 256 bytes:
[1] os.mkdir(very_long_path)
[2] os.getsize(very_long_path)
[3] shutil.rmtree(very_long_path)
I am using Python 2.7 and [1] and [2] fail under Windows XP [3] fails
under Win7 (not sure about XP). This is even when I use the "special"
notations \\?\c:\dir\file or \\?\UNC\server\share\file, e.g.
os.path.getsize("?\\" + "c:\\dir\\file")
(Oddly, os.path.getsize(os.path.join("?", "c:\\dir\\file")) will
truncate the prefix)
My questions:
1. How can I get the file size of very long paths under XP?
2. Is this a bug in Python? I would prefer if Python dealt with the gory
details of Windows' silly behavior.
Regards,
Albert-Jan
---
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus
--
https://mail.python.org/mailman/listinfo/python-list
Re: Chardet oddity
On Oct 24, 2024 17:51, Roland Mueller via Python-list wrote: ke 23. lokak. 2024 klo 20.11 Albert-Jan Roskam via Python-list ( [email protected]) kirjoitti: > Today I used chardet.detect in the repl and it returned windows-1252 > (incorrect, because it later resulted in a UnicodeDecodeError). When I > ran > chardet as a script (which uses UniversalLineDetector) this returned > MacRoman. Isn't charset.detect the correct way? I've used this method > many > times. > # Interpreter > >>> contents = open(FILENAME, "rb").read() > >>> chardet.detect(content) > {'encoding': 'Windows-1252', 'confidence': 0.7282676610947401, > 'language': > ''} > # Terminal > $ python -m chardet FILENAME > FILENAME: MacRoman with confidence 0.7167379080370483 > Thanks! > Albert-Jan > The entry point for the module chardet is chardet.cli.chardetect:main and main() calls function description_of(lines, name). 'lines' is an opened file in mode 'rb' and name will hold the filename. Following way I tried this in interactive mode: I think the crucial difference is that description_of(lines, name) reads the opened file line by line and stops after something has been detected in some line. When reading the whole file into the variable contents probably gives another result depending on the input. This behaviour I was not able to repeat. I am assuming that you used the same Python for both tests. >>> from chardet.cli import chardetect >>> chardetect.description_of(open('/tmp/DATE', 'rb'), 'some file') 'some file: ascii with confidence 1.0' >>> Your approach >>> from chardet import detect >>> detect(open('/tmp/DATE','rb').read()) {'encoding': 'ascii', 'confidence': 1.0, 'language': ''} from /usr/lib/python3/dist-packages/chardet/cli/chardetect.py def description_of(lines, name='stdin'): u = UniversalDetector() for line in lines: line = bytearray(line) u.feed(line) # shortcut out of the loop to save reading further - particularly useful if we read a BOM. if u.done: break u.close() result = u.result = Hi Mark, Roland, Thanks for your replies. I experimented a bit with both methods and the derived encoding still differed, even after I removed the "if u.done: break" (I removed that because I've seen cp1252 files with a utf8 BOM in the past. I kid you not!). BUT next day, at closer inspection I saw that the file was quite a mess. I contained mojibake. So I don't blame chardet for not being able to figure out the encoding. Albert-Jan -- https://mail.python.org/mailman/listinfo/python-list
Chardet oddity
Today I used chardet.detect in the repl and it returned windows-1252
(incorrect, because it later resulted in a UnicodeDecodeError). When I ran
chardet as a script (which uses UniversalLineDetector) this returned
MacRoman. Isn't charset.detect the correct way? I've used this method many
times.
# Interpreter
>>> contents = open(FILENAME, "rb").read()
>>> chardet.detect(content)
{'encoding': 'Windows-1252', 'confidence': 0.7282676610947401, 'language':
''}
# Terminal
$ python -m chardet FILENAME
FILENAME: MacRoman with confidence 0.7167379080370483
Thanks!
Albert-Jan
--
https://mail.python.org/mailman/listinfo/python-list
