Re: correct way to catch exception with Python 'with' statement
Steven D'Aprano :
> There is no need to catch the exception if you're not going to do
> anything with it.
Correct. However, the question of the subject line is still a good one.
See:
try:
with open("xyz") as f:
...[A]...
except FileNotFoundError:
...[B]...
The basic principle about exceptions is that if you are prepared to
catch one, catch it in the immediate vicinity of the operation that
might raise it. The code above might mistakenly catch an exception from
inside the "with" block.
You could then protect yourself like this:
try:
with open("xyz") as f:
try:
...[A]...
except FileNotFoundError as e:
raise SmuggleItOutException(e)
except FileNotFoundError:
...[B]...
except SmuggleItOutException as e:
raise e.exception_proper
However, I think the real answer is that you shouldn't mix the "with"
construct with exception handling. Instead you should write:
try:
f = open("xyz")
except FileNotFoundError:
...[B]...
try:
...[A]...
finally:
f.close()
Marko
--
https://mail.python.org/mailman/listinfo/python-list
Re: correct way to catch exception with Python 'with' statement
Marko Rauhamaa wrote:
> However, I think the real answer is that you shouldn't mix the "with"
> construct with exception handling. Instead you should write:
>
>try:
>f = open("xyz")
>except FileNotFoundError:
>...[B]...
>try:
>...[A]...
>finally:
>f.close()
What's the problem with spelling the above
try:
f = open(...)
except FileNotFoundError:
...
with f:
...
?
--
https://mail.python.org/mailman/listinfo/python-list
Re: correct way to catch exception with Python 'with' statement
Peter Otten <[email protected]>: > Marko Rauhamaa wrote: > >> However, I think the real answer is that you shouldn't mix the "with" >> construct with exception handling. Instead you should write: >> >>try: >>f = open("xyz") >>except FileNotFoundError: >>...[B]... >>try: >>...[A]... >>finally: >>f.close() > > What's the problem with spelling the above > > try: > f = open(...) > except FileNotFoundError: > ... > with f: > ... Nothing. Marko -- https://mail.python.org/mailman/listinfo/python-list
Last call for the Call For Proposals of PythonFOSDEM 2017
Because the deadline is imminent and because we have only received some proposals, we will extend the current deadline. The new submission deadline is 2016-12-18. Call For Proposals == This is the official call for sessions for the Python devroom at FOSDEM 2017. FOSDEM is the Free and Open source Software Developers' European Meeting, a free and non-commercial two-day week-end that offers open source contributors a place to meet, share ideas and collaborate. It's the biggest event in Europe with +5000 hackers, +400 speakers. For this edition, Python will be represented by its Community. If you want to discuss with a lot of Python Users, it's the place to be! Important dates === * Submission deadlines: 2016-12-18 * Acceptance notifications: 2016-12-23 Practical = * The duration for talks will be 30 minutes, including presentations and questions and answers. * Presentation can be recorded and streamed, sending your proposal implies giving permission to be recorded. * A mailing list for the Python devroom is available for discussions about devroom organisation. You can register at this address: https://lists.fosdem.org/listinfo/python-devroom How to submit = All submissions are made in the Pentabarf event planning tool at https://penta.fosdem.org/submission/FOSDEM17 When submitting your talk in Pentabarf, make sure to select the Python devroom as the Track. Of course, if you already have a user account, please reuse it. Questions = Any questions, please send an email to info AT python-fosdem DOT org Thank you for submitting your sessions and see you soon in Brussels to talk about Python. If you want to keep informed for this edition, you can follow our twitter account @PythonFOSDEM. * FOSDEM 2017: https://fosdem.org/2017 * Python Devroom: http://python-fosdem.org * Twitter: https://twitter.com/PythonFOSDEM Stephane -- Stéphane Wirtel - http://wirtel.be - @matrixise -- https://mail.python.org/mailman/listinfo/python-list
Re: best way to read a huge ascii file.
Hi all, Let me update my question, I have an ascii file(7G) which has around 100M lines. I read this file using : f=np.loadtxt(os.path.join(dir,myfile),delimiter=None,skiprows=0) x=f[:,1] y=f[:,2] z=f[:,3] id=f[:,0] I will need the x,y,z and id arrays later for interpolations. The problem is reading the file takes around 80 min while the interpolation only takes 15 mins. I tried to get the memory increment used by each line of the script using python memory_profiler module. The following line which reads the entire 7.4 GB file increments the memory usage by 3206.898 MiB (3.36 GB). First question is Why it does not increment the memory usage by 7.4 GB? f=np.loadtxt(os.path.join(dir,myfile),delimiter=None,skiprows=0) The following 4 lines do not increment the memory at all. x=f[:,1] y=f[:,2] z=f[:,3] id=f[:,0] Finally I still would appreciate if you could recommend me what is the most optimized way to read/write to files in python? are numpy np.loadtxt and np.savetxt the best? Thanks in Advance, -- https://mail.python.org/mailman/listinfo/python-list
async enumeration - possible?
Hi all
Python 3.6 has introduced Asynchronous Generators, which work very well.
Python 3.6.0b4 (default, Nov 22 2016, 05:30:12) [MSC v.1900 64 bit (AMD64)]
on win32
Type "help", "copyright", "credits" or "license" for more information.
import asyncio
loop = asyncio.get_event_loop()
async def counter(n):
... for i in range(n):
... yield i
...
async def main():
... c = counter(5)
... async for j in c:
... print(j)
... print('done')
...
loop.run_until_complete(main())
0
1
2
3
4
done
However, it does not allow you to enumerate over the generator output -
async def main():
... c = counter(5)
... async for j, k in enumerate(c):
... print(j, k)
... print('done')
...
loop.run_until_complete(main())
Traceback (most recent call last):
File "", line 1, in
File
"C:\Users\User\AppData\Local\Programs\Python\Python36\lib\asyncio\base_events.py",
line 466, in run_until_complete
return future.result()
TypeError: 'async_generator' object is not iterable
Is there any technical reason for this, or is it just that no-one has got
around to writing an asynchronous version yet?
Frank Millman
--
https://mail.python.org/mailman/listinfo/python-list
Re: best way to read a huge ascii file.
Heli writes:
> Hi all,
>
> Let me update my question, I have an ascii file(7G) which has around
> 100M lines. I read this file using :
>
> f=np.loadtxt(os.path.join(dir,myfile),delimiter=None,skiprows=0)
>
> x=f[:,1]
> y=f[:,2]
> z=f[:,3]
> id=f[:,0]
>
> I will need the x,y,z and id arrays later for interpolations. The
> problem is reading the file takes around 80 min while the
> interpolation only takes 15 mins.
(Are there only those four columns in the file? I guess yes.)
> The following line which reads the entire 7.4 GB file increments the
> memory usage by 3206.898 MiB (3.36 GB). First question is Why it does
> not increment the memory usage by 7.4 GB?
>
> f=np.loadtxt(os.path.join(dir,myfile),delimiter=None,skiprows=0)
In general, doubles take more space as text than as, well, doubles,
which (in those arrays) take eight bytes (64 bits) each:
>>> len("0.1411200080598672 -0.9899924966004454 -0.1425465430742778
>>> 20.085536923187668 ")
78
>>> 4*8
32
> Finally I still would appreciate if you could recommend me what is the
> most optimized way to read/write to files in python? are numpy
> np.loadtxt and np.savetxt the best?
A document I found says "This function aims to be a fast reader for
simply formatted files" so as long as you want to save the numbers as
text, this is probably meant to be the best way.
https://docs.scipy.org/doc/numpy/reference/generated/numpy.loadtxt.html
Perhaps there are binary load and save functions? They could be faster.
The binary data file would be opaque, but probably you are not editing
it by hand anyway.
--
https://mail.python.org/mailman/listinfo/python-list
Re: async enumeration - possible?
On Tue, Nov 29, 2016 at 7:25 AM, Frank Millman wrote:
> However, it does not allow you to enumerate over the generator output -
>
async def main():
>
> ... c = counter(5)
> ... async for j, k in enumerate(c):
> ... print(j, k)
> ... print('done')
> ...
loop.run_until_complete(main())
>
> Traceback (most recent call last):
> File "", line 1, in
> File
> "C:\Users\User\AppData\Local\Programs\Python\Python36\lib\asyncio\base_events.py",
> line 466, in run_until_complete
>return future.result()
> TypeError: 'async_generator' object is not iterable
>
> Is there any technical reason for this, or is it just that no-one has got
> around to writing an asynchronous version yet?
No one has written an async enumerate afaik. You may be interested in
the aitertools package in PyPI:
https://pypi.python.org/pypi/aitertools/0.1.0
It also doesn't have an async enumerate. It does have an aiter
function that can wrap an ordinary iterable into an async iterable, so
one way to write that would be aiter(enumerate(c)), with the caveat
that the wrapped iterator is still running synchronously.
Otherwise, you can write one yourself. This doesn't support all the
features of enumerate, but it serves as demonstration:
from itertools import count
async def aenumerate(aiterable):
counter = count()
async for x in aiterable:
yield next(counter), x
--
https://mail.python.org/mailman/listinfo/python-list
Re: best way to read a huge ascii file.
On Tuesday, November 29, 2016 at 3:18:29 PM UTC+1, Heli wrote:
> Hi all,
>
> Let me update my question, I have an ascii file(7G) which has around 100M
> lines. I read this file using :
>
> f=np.loadtxt(os.path.join(dir,myfile),delimiter=None,skiprows=0)
>
> x=f[:,1]
> y=f[:,2]
> z=f[:,3]
> id=f[:,0]
>
> I will need the x,y,z and id arrays later for interpolations. The problem is
> reading the file takes around 80 min while the interpolation only takes 15
> mins.
>
> I tried to get the memory increment used by each line of the script using
> python memory_profiler module.
>
> The following line which reads the entire 7.4 GB file increments the memory
> usage by 3206.898 MiB (3.36 GB). First question is Why it does not increment
> the memory usage by 7.4 GB?
>
> f=np.loadtxt(os.path.join(dir,myfile),delimiter=None,skiprows=0)
>
> The following 4 lines do not increment the memory at all.
> x=f[:,1]
> y=f[:,2]
> z=f[:,3]
> id=f[:,0]
>
> Finally I still would appreciate if you could recommend me what is the most
> optimized way to read/write to files in python? are numpy np.loadtxt and
> np.savetxt the best?
>
> Thanks in Advance,
Hi,
Have you considered storing the data in HDF5? There is an excellent Python
interface for this (see: http://www.h5py.org/). The advantage that you will
have is that no text to number conversion has to applied anymore. You can
directly operate on the datasets in the HDF5 database.
If you would go this direction, the following would get you started:
>>> import h5py
>>> import numpy
>>> from numpy import uint32, float32, arange
>>> fd = h5py.File('demo.h5', mode='w') # Note that this will truncate
# the file, use 'r' or 'a' if you
# want to open an existing file
>>> observations = fd.create_group('/observations')
>>> N = 100
>>> observations.create_dataset('id', data=arange(0, N, dtype=uint32))
>>> observations.create_dataset('x', data=numpy.random.random(N), dtype=float32)
>>> observations.create_dataset('y', data=numpy.random.random(N), dtype=float32)
>>> observations.create_dataset('z', data=numpy.random.random(N), dtype=float32)
>>>
>>> fd.close()
Note that you can also combine x,y and z in a single dataset if you want to.
See the documentation for datasets for more information
http://docs.h5py.org/en/latest/high/dataset.html
I would also advise you to carefully select the proper dtype for the arrays.
In particular if you know the value range for your datasets. This can save
you a lot of disk space and probably will increase the performance a little.
Marco
--
https://mail.python.org/mailman/listinfo/python-list
Re: best way to read a huge ascii file.
On 29/11/2016 14:17, Heli wrote: Hi all, Let me update my question, I have an ascii file(7G) which has around 100M lines. I read this file using : f=np.loadtxt(os.path.join(dir,myfile),delimiter=None,skiprows=0) x=f[:,1] y=f[:,2] z=f[:,3] id=f[:,0] I will need the x,y,z and id arrays later for interpolations. The problem is reading the file takes around 80 min while the interpolation only takes 15 mins. I tried to get the memory increment used by each line of the script using python memory_profiler module. The following line which reads the entire 7.4 GB file increments the memory usage by 3206.898 MiB (3.36 GB). First question is Why it does not increment the memory usage by 7.4 GB? Is there enough total RAM capacity for another 4.2GB? But if the file is text, and being read into binary data in memory, it will be different. Usually binary data takes less space. I assume the loader doesn't load the entire text file first, do the conversions to binary, then unloads file, as that would then require 10.6GB during that process! f=np.loadtxt(os.path.join(dir,myfile),delimiter=None,skiprows=0) The following 4 lines do not increment the memory at all. x=f[:,1] y=f[:,2] z=f[:,3] id=f[:,0] That's surprising because if those are slices, they would normally create a copy (I suppose you don't set f to 0 or something after those lines). But if numpy data is involved, I seem to remember that slices are actually views into the data. Finally I still would appreciate if you could recommend me what is the most optimized way to read/write to files in python? are numpy np.loadtxt and np.savetxt the best? Why not post a sample couple of lines from the file? (We don't need the other 99,999,998 assuming they are all have the same format.) Then we can see if there's anything obviously inefficient about it. -- Bartc -- https://mail.python.org/mailman/listinfo/python-list
Re: Asyncio -- delayed calculation
On Mon, Nov 28, 2016 at 10:42 PM, Chris Angelico wrote: > On Tue, Nov 29, 2016 at 4:13 PM, Paul Rubin wrote: >> >> I haven't gotten my head around Python asyncio and have been wanting >> to read this: >> >>http://lucumr.pocoo.org/2016/10/30/i-dont-understand-asyncio/ > > It's talking a lot about how we got here, which isn't all necessary if > you just want to give asyncio a whirl. The conclusion at the end says > that you should just use 'async def' and not bother with all the older > forms, which I agree with (subject to the usual caveat that this > implies no support for older Pythons). > > There's one thing that I really struggle with, though, and that's that > there's no easy and obvious way to demonstrate the lowest level of > operation. If "await x()" is like "yield from x()", how do you do the > innermost "yield" that actually does something? I have the same > confusion with Node.js, too. It's as if async primitives can't be > implemented in application code at all, they just have to be given to > you. Certainly it's not something made clear anywhere in the docs that > I've found. You mean how do you create something that can be awaited that doesn't await something else in turn? With a Future. import asyncio class Awaitable(asyncio.Future): def wake_up_later(self): asyncio.get_event_loop().call_later(3, self.set_result, 42) async def main(): awaitable = Awaitable() awaitable.wake_up_later() print(await awaitable) asyncio.get_event_loop().run_until_complete(main()) The trick is in arranging for the future's result to be set. For I/O events you would typically do that by associating a callback with a file descriptor on the event loop: https://docs.python.org/3/library/asyncio-eventloop.html#watch-file-descriptors If you need to do something particulary abstruse you can always write a custom Selector or event loop: https://docs.python.org/3/library/selectors.html#module-selectors https://docs.python.org/3/library/asyncio-eventloop.html#base-event-loop -- https://mail.python.org/mailman/listinfo/python-list
csv into multiple columns using split function using python
I am trying to split a specific column of csv into multiple column and then
appending the split values at the end of each row.
`enter code here`
import csv
fOpen1=open('Meta_D1.txt')
reader=csv.reader(fOpen1)
mylist=[elem[1].split(',') for elem in reader]
mylist1=[]
for elem in mylist1:
mylist1.append(elem)
#writing to a csv file
with open('out1.csv', 'wb') as fp:
myf = csv.writer(fp, delimiter=',')
myf.writerows(mylist1)
---
Here is the link to file I am working on 2 column.
https://spaces.hightail.com/space/4hFTj
Can someone guide me further?
--
https://mail.python.org/mailman/listinfo/python-list
Re: async enumeration - possible?
On 11/29/2016 9:25 AM, Frank Millman wrote: Is there any technical reason for this, or is it just that no-one has got around to writing an asynchronous version yet? Google's first hit for 'aenumerate' is https://pythonwise.blogspot.com/2015/11/aenumerate-enumerate-for-async-for.html Note that updated 3.5.2+ code is in response to my comment. -- Terry Jan Reedy -- https://mail.python.org/mailman/listinfo/python-list
Re: async enumeration - possible?
Terry Reedy : > On 11/29/2016 9:25 AM, Frank Millman wrote: > >> Is there any technical reason for this, or is it just that no-one has >> got around to writing an asynchronous version yet? > > Google's first hit for 'aenumerate' is > https://pythonwise.blogspot.com/2015/11/aenumerate-enumerate-for-async-for.html Ok, so how about: aall(aiterable) aany(aiterable) class abytearray(aiterable[, encoding[, errors]]) class adict(aiterable, **kwarg) class afilter(coro, aiterable) class afrozenset(aiterable) aiter(object[, sentinel]) class alist(aiterable) amap(coro, aiterable, ...) amax(aiterable, *[, key, default]) amin(aiterable, *[, key, default]) anext(aiterator[, default]) class aset(aiterable) asorted(aiterable[, key][, reverse]) asum(aiterable[, start]) atuple(aiterable) azip(*aiterables) to name a few... How about awaitable comprehensions? Marko -- https://mail.python.org/mailman/listinfo/python-list
Re: best way to read a huge ascii file.
On Wed, 30 Nov 2016 01:17 am, Heli wrote: > The following line which reads the entire 7.4 GB file increments the > memory usage by 3206.898 MiB (3.36 GB). First question is Why it does not > increment the memory usage by 7.4 GB? > > f=np.loadtxt(os.path.join(dir,myfile),delimiter=None,skiprows=0) Floating point numbers as strings typically take up far more space than do floats. On disk, a string like "3.141592653589793" requires 13 bytes. Plus there are additional bytes used as separators between fields, and at least one more byte (a newline) at the end of each record. Whereas, once converted to a float (a C 64-bit double) it only requires 8 bytes. In a numpy array, there's no separator needed and the values are tightly packed. So its quite reasonable to expect a saving of around 50%. > The following 4 lines do not increment the memory at all. > x=f[:,1] > y=f[:,2] > z=f[:,3] > id=f[:,0] Numpy slices are views, not copies. > Finally I still would appreciate if you could recommend me what is the > most optimized way to read/write to files in python? are numpy np.loadtxt > and np.savetxt the best? You're not just reading a file. You're reading a file and converting millions of strings to floats. You are processing 7GB of data in 80 minutes, or around 1.5MB per second. Do you have reason to think that's unreasonably slow? (Apart from wishing that it were faster.) Where are you reading the file from? How much RAM do you have? -- Steve “Cheer up,” they said, “things could be worse.” So I cheered up, and sure enough, things got worse. -- https://mail.python.org/mailman/listinfo/python-list
Re: async enumeration - possible?
On Wed, 30 Nov 2016 07:07 am, Marko Rauhamaa wrote: > Terry Reedy : > >> On 11/29/2016 9:25 AM, Frank Millman wrote: >> >>> Is there any technical reason for this, or is it just that no-one has >>> got around to writing an asynchronous version yet? >> >> Google's first hit for 'aenumerate' is >> https://pythonwise.blogspot.com/2015/11/aenumerate-enumerate-for-async-for.html > > Ok, so how about: > >aall(aiterable) >aany(aiterable) >class abytearray(aiterable[, encoding[, errors]]) [...] What about them? What's your question? -- Steve “Cheer up,” they said, “things could be worse.” So I cheered up, and sure enough, things got worse. -- https://mail.python.org/mailman/listinfo/python-list
Re: Asyncio -- delayed calculation
On Wed, 30 Nov 2016 05:41 am, Ian Kelly wrote: > You mean how do you create something that can be awaited that doesn't > await something else in turn? With a Future. > > import asyncio > > class Awaitable(asyncio.Future): > def wake_up_later(self): > asyncio.get_event_loop().call_later(3, self.set_result, 42) Not to be confused with concurrent.Futures. https://docs.python.org/3.5/library/concurrent.futures.html https://docs.python.org/3.5/library/asyncio-task.html#asyncio.Future -- Steve “Cheer up,” they said, “things could be worse.” So I cheered up, and sure enough, things got worse. -- https://mail.python.org/mailman/listinfo/python-list
Python while loop
Write a program which prints the sum of numbers from 1 to 101 ( 1 and 101 are included) that are divisible by 5 (Use while loop) This is the code: x=0 count=0 while x<=100: if x%5==0: count=count+x x=x+1 print(count) Question: How does python know what count means ? I see that its declared. What i wanna know is how does it know the iteration or each number it finds divisible by 5 is the "Count" ?? -- https://mail.python.org/mailman/listinfo/python-list
Re: csv into multiple columns using split function using python
Add some print statements to see what is happening, especially after the for elem in mylist1: statement -- https://mail.python.org/mailman/listinfo/python-list
Re: correct way to catch exception with Python 'with' statement
If you want to do something only if the file exists (or does not), use os.path.isfile(filename) -- https://mail.python.org/mailman/listinfo/python-list
Re: Python while loop
On 2016-11-29 23:58, [email protected] wrote: Write a program which prints the sum of numbers from 1 to 101 ( 1 and 101 are included) that are divisible by 5 (Use while loop) This is the code: x=0 count=0 while x<=100: if x%5==0: count=count+x x=x+1 print(count) Question: How does python know what count means ? I see that its declared. What i wanna know is how does it know the iteration or each number it finds divisible by 5 is the "Count" ?? It doesn't, it's just a name, and, anyway, in the code, 'x' is the count... If it _did_ know (which is doesn't), wouldn't it be complaining that 'count' isn't the count but the sum? :-) -- https://mail.python.org/mailman/listinfo/python-list
Re: Python while loop
On 29/11/2016 23:58, [email protected] wrote: Write a program which prints the sum of numbers from 1 to 101 ( 1 and 101 are included) that are divisible by 5 (Use while loop) This is the code: x=0 count=0 while x<=100: if x%5==0: count=count+x x=x+1 print(count) This looks at numbers from 0 to 100 inclusive, not 1 to 101. Although it doesn't affect the result. (It will add in 0 which is divisible by 5, but that doesn't change it either. If this is an exercise however, checking the correct range might be important!) -- Bartc -- https://mail.python.org/mailman/listinfo/python-list
Re: correct way to catch exception with Python 'with' statement
On Tue, 29 Nov 2016 at 23:59 wrote: > If you want to do something only if the file exists (or does not), use > os.path.isfile(filename) > This opens you up to a potential race condition (and has potential security implications, depending on the application), as you're using LBYL[0]. If you want to write robust code, you'll need to catch the FileNotFoundError anyway, so unless you're doing many* file lookups and expecting a large number of them* to not exist, you're probably better off with EAFP[1] unless you're really sure (i.e. you've followed the 3 rules of performance optimisation[2]) *I'm being quite vague here as you'd have to actually profile the application to work out which way is quicker and if the difference is worth worrying about. Catching the exception in theory might be more expensive, but that cost might be insignificant compared to the IO cost which you'll have either way. [0] https://docs.python.org/2/glossary.html#term-lbyl [1] https://docs.python.org/2/glossary.html#term-eafp [2] http://wiki.c2.com/?RulesOfOptimization -- https://mail.python.org/mailman/listinfo/python-list
Request Help With Byte/String Problem
For the purpose of learning I am writing a script that will
return different information about the Linux machine where
it is running. Sort of like the inxi utility.
Below is some code that I found that returns a list of the
network interface devices on the system. It runs as is
perfectly on Python2 but I get the error pasted below the
code when run on Python3, the desired version. I know it
has something to do with bytes vs. strings but I can' seem
to figure it out. Any help appreciated.
import array
import socket
import struct
def all_interfaces():
max_possible = 128 # arbitrary. raise if needed.
bytes = max_possible * 32
s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
names = array.array("B", '\0' * bytes)
outbytes = struct.unpack('iL', fcntl.ioctl(
s.fileno(),
0x8912, # SIOCGIFCONF
struct.pack('iL', bytes, names.buffer_info()[0])
))[0]
namestr = names.tostring()
lst = []
for i in range(0, outbytes, 40):
name = namestr[i:i+16].split('\0', 1)[0]
ip = namestr[i+20:i+24]
lst.append((name, ip))
return lst
def format_ip(addr):
return str(ord(addr[0])) + '.' + \
str(ord(addr[1])) + '.' + \
str(ord(addr[2])) + '.' + \
str(ord(addr[3]))
ifs = all_interfaces()
for i in ifs:
print("%12s %s" % (i[0], format_ip(i[1])))
Traceback (most recent call last):
File "./ifaces.py", line 32, in
ifs = all_interfaces()
File "./ifaces.py", line 11, in all_interfaces
names = array.array("B", '\0' * bytes)
TypeError: cannot use a str to initialize an array with typecode 'B'
--
GNU/Linux user #557453
The cow died so I don't need your bull!
--
https://mail.python.org/mailman/listinfo/python-list
Re: Request Help With Byte/String Problem
Wildman writes:
> names = array.array("B", '\0' * bytes)
> TypeError: cannot use a str to initialize an array with typecode 'B'
In Python 2, str is a byte string and you can do that. In Python 3,
str is a unicode string, and if you want a byte string you have to
specify that explicitly, like b'foo' instead of 'foo'. I.e.
names = array.array("B", b'\0' * bytes)
should work.
--
https://mail.python.org/mailman/listinfo/python-list
Re: async enumeration - possible?
On Wed, Nov 30, 2016 at 7:07 AM, Marko Rauhamaa wrote: > Ok, so how about: > >aall(aiterable) >aany(aiterable) >class abytearray(aiterable[, encoding[, errors]]) >class adict(aiterable, **kwarg) >class afilter(coro, aiterable) >class afrozenset(aiterable) >aiter(object[, sentinel]) >class alist(aiterable) >amap(coro, aiterable, ...) >amax(aiterable, *[, key, default]) >amin(aiterable, *[, key, default]) >anext(aiterator[, default]) >class aset(aiterable) >asorted(aiterable[, key][, reverse]) >asum(aiterable[, start]) >atuple(aiterable) >azip(*aiterables) > > to name a few... > > How about awaitable comprehensions? Any of these that depend on pumping the entire iterable can simply synchronify [1] the iterable: list(x async for x in aiterable) Interestingly, I can't do that in a list comp: >>> [x async for x in aiterable] File "", line 1 [x async for x in aiterable] ^ SyntaxError: invalid syntax Not sure why. Anyhow. That removes the need for alist, atuple, amax, etc. All you would need are the ones that process an iterable and yield values individually, like azip and amap. Those can be provided as recipes, third-party modules, or stdlib modules, until they prove their worth as potential builtins. ChrisA [1] is that even a word? -- https://mail.python.org/mailman/listinfo/python-list
Re: correct way to catch exception with Python 'with' statement
On Wednesday 30 November 2016 10:59, [email protected] wrote: > If you want to do something only if the file exists (or does not), use > os.path.isfile(filename) No, don't do that. Just because the file exists, doesn't mean that you have permission to read or write to it. Worse, the code is vulnerable to race conditions. Look at this: if os.path.isfile(filename): with open(filename) as f: process(f) Just because the file exists when you test it, doesn't mean it still exists a millisecond later when you go to open the file. On a modern multi-processing system, like Windows, OS X or Linux, a lot can happen in the microseconds between checking for the file's existence and actually accessing the file. This is called a "Time Of Check To Time Of Use" bug, and it can be a security vulnerability. -- Steven "Ever since I learned about confirmation bias, I've been seeing it everywhere." - Jon Ronson -- https://mail.python.org/mailman/listinfo/python-list
Re: Request Help With Byte/String Problem
On Tue, 29 Nov 2016 18:29:51 -0800, Paul Rubin wrote:
> Wildman writes:
>> names = array.array("B", '\0' * bytes)
>> TypeError: cannot use a str to initialize an array with typecode 'B'
>
> In Python 2, str is a byte string and you can do that. In Python 3,
> str is a unicode string, and if you want a byte string you have to
> specify that explicitly, like b'foo' instead of 'foo'. I.e.
>
> names = array.array("B", b'\0' * bytes)
>
> should work.
I really appreciate your reply. Your suggestion fixed that
problem, however, a new error appeared. I am doing some
research to try to figure it out but no luck so far.
Traceback (most recent call last):
File "./ifaces.py", line 33, in
ifs = all_interfaces()
File "./ifaces.py", line 21, in all_interfaces
name = namestr[i:i+16].split('\0', 1)[0]
TypeError: Type str doesn't support the buffer API
--
GNU/Linux user #557453
The cow died so I don't need your bull!
--
https://mail.python.org/mailman/listinfo/python-list
Re: async enumeration - possible?
"Frank Millman" wrote in message news:[email protected]... Hi all Python 3.6 has introduced Asynchronous Generators, which work very well. [...] However, it does not allow you to enumerate over the generator output - [...] Is there any technical reason for this, or is it just that no-one has got around to writing an asynchronous version yet? Thanks for the replies. @Ian Thanks for your explanation and example. The example is perfect for my simple requirement. @Terry I should have googled for aenumerate - didn't think of it. However, this recipe is based on Python 3.5 and earlier. Asynchronous Generators, introduced in 3.6, make it much easier. See PEP 525 for the details - https://www.python.org/dev/peps/pep-0525/ @Chris I agree, there is a big difference between functions which consume the entire iterable before returning the result, and those which behave asynchronously and return the value on-the-fly. I happened upon enumerate. You mentioned zip and map, which are also likely to be useful in the right circumstances. I did not quite follow your example, as I get the opposite result - Python 3.6.0b4 (default, Nov 22 2016, 05:30:12) [MSC v.1900 64 bit (AMD64)] on win32 Type "help", "copyright", "credits" or "license" for more information. import asyncio loop = asyncio.get_event_loop() async def gen(n): ... for i in range(n): ... yield i ... async def main(): ... print([x async for x in gen(5)]) ... loop.run_until_complete(main()) [0, 1, 2, 3, 4] async def main(): ... print(list(x async for x in gen(5))) ... loop.run_until_complete(main()) Traceback (most recent call last): File "", line 1, in File "C:\Users\User\AppData\Local\Programs\Python\Python36\lib\asyncio\base_events.py", line 466, in run_until_complete return future.result() TypeError: 'async_generator' object is not iterable Frank -- https://mail.python.org/mailman/listinfo/python-list
Re: async enumeration - possible?
Chris Angelico : > On Wed, Nov 30, 2016 at 7:07 AM, Marko Rauhamaa wrote: > Any of these that depend on pumping the entire iterable can simply > synchronify [1] the iterable: One of the more useful ones might be: o = await anext(ait) > list(x async for x in aiterable) > > Interestingly, I can't do that in a list comp: I have a couple of points to make with my question: * We are seeing the reduplication of a large subset of Python's facilities. I really wonder if the coroutine fad is worth the price. * I don't think bulk iteration in asynchronous programming is ever that great of an idea. You want to be prepared for more than one possible stimulus in any given state. IOW, a state machine matrix might be sparse but it is never diagonal. Marko -- https://mail.python.org/mailman/listinfo/python-list
