Re: Make a unique filesystem path, without creating the file
Ben Finney wrote: One valid filesystem path each time it's accessed. That is, behaviour equivalent to ‘tempfile.mktemp’. My question is because the standard library clearly has this useful functionality implemented, but simultaneously warns strongly against its use. But it *doesn't*, if your requirement is truly to not touch the filesystem at all, because tempfile.mktemp() *reads* the file system to make sure the name it's returning isn't in use. What's more, because you're *not* creating the file, mktemp() would be within its rights to return the same file name the second time you call it. If you want something that really doesn't go near the file system and/or is guaranteed to produce multiple different non-existing file names, you'll have to write it yourself. -- Greg -- https://mail.python.org/mailman/listinfo/python-list
Re: What is heating the memory here? hashlib?
Às 02:21 de 14-02-2016, Steven D'Aprano escreveu: > On Sun, 14 Feb 2016 06:29 am, Paulo da Silva wrote: ... Thanks Steven for your advices. This is a small script to solve a specific problem. It will be used in future to solve other similar problems probably with small changes. When I found it eating memory and, what I thought was the 1st reason for that was fixed and it still ate the memory, I thought of something less obvious. After all it seems there is nothing wrong with it (see my other post). > That's your first clue that, perhaps, you should be reading in relatively > small blocks, more like 4K than 4MB. Sure enough, a quick bit of googling > shows that typically you should read from files in small-ish chunks, and > that trying to read in large chunks is often counter-productive: > > https://duckduckgo.com/html/?q=file+read+buffer+size > > The first three links all talk about optimal sizes being measured in small > multiples of 4K, not 40MB. > I didn't know about this! Most of my files are about ~>30MB. So I chose 40MB to avoid python loops. After all, python should be able to optimize those things. > You can try to increase the system buffer, by changing the "open" line to: > > with open(pathname, 'rb', buffering=40*M) as f: > This is another thing. One thing is the requested amount of data I want another is to choose de "really" buffer size. (I didn't know about this argument - thanks). ... > By the way, do you need a cryptographic checksum? sha256 is expensive to > calculate. If all you are doing is trying to match files which could have > the same content, you could use a cheaper hash, like md5 or even crc32. I don't know the probability of collision of each of them. The script has sha256 and md5 as options. When the failed execution I had chosen sha256. I didn't check if it takes much more time. A collision might cause data loss. So ... Thank you. Paulo -- https://mail.python.org/mailman/listinfo/python-list
Re: What is heating the memory here? hashlib?
On 15.02.2016 03:21, Paulo da Silva wrote: > So far I tried the program twice and it ran perfectly. I think you measured your RAM consumption wrong. Linux uses all free RAM as HDD cache. That's what is used in "buffers". That is, it's not "free", but it would be free if any process would sbrk(). My guess is that you only looked at the "free" number going down and concluded your program is eating your RAM. Which it wasn't. Cheers Johannes -- >> Wo hattest Du das Beben nochmal GENAU vorhergesagt? > Zumindest nicht öffentlich! Ah, der neueste und bis heute genialste Streich unsere großen Kosmologen: Die Geheim-Vorhersage. - Karl Kaos über Rüdiger Thomas in dsa -- https://mail.python.org/mailman/listinfo/python-list
Re: Make a unique filesystem path, without creating the file
Ben Finney wrote: The existing behaviour of ‘tempfile.mktemp’ – actually of its internal class ‘tempfile._RandomNameSequence’ – is to generate unpredictable, unique, valid filesystem paths that are different each time. But that's not documented behaviour, so even if mktemp() weren't marked as deprecated, you'd still be relying on undocumented and potentially changeable behaviour. What I'm looking for is a way to use it (not re-implement it) that is public API and isn't scolded by the library documentation. Then you're looking for something that doesn't exist, I'm sorry to say, and it's unlikely you'll persuade anyone to make it exist. If you want to leverage stdlib functionality for this, I'd suggest something along the lines of: def fakefilename(dir, ext): return os.path.join(dir, str(uuid.uuid4())) + ext -- Greg -- https://mail.python.org/mailman/listinfo/python-list
Re: asyncio - run coroutine in the background
"Paul Rubin" wrote in message news:[email protected]... "Frank Millman" writes: > The benefit of my class is that it enables me to take the coroutine > and run it in another thread, without having to re-engineer the whole > thing. Threads in Python don't get you parallelism either, of course. Agreed. My class does not alter the total time taken, but it does free up the original task to carry on with other work. run_in_executor() uses threads by default, but it does allow you to specify processes as an alternative. I haven't used async/await yet and it's looking painful. I've been wanting to read this: http://www.snarky.ca/how-the-heck-does-async-await-work-in-python-3-5 but I start to think it isn't all that great an approach to concurrency. Thanks for that link. I had a quick scan, and it looks interesting, but some of it a bit above my head. I have bookmarked it, as I think that as my understanding increases, I will gain more from it on each re-read. Frank -- https://mail.python.org/mailman/listinfo/python-list
EuroPython 2016: Sending out the first gravitational waves
We are pleased to announce the launch of our all new EuroPython 2016 website. Over the last few weeks, we have been busy talking to sponsors and getting the website prepared for the launch. You may have heard about the recent direct observation of gravitational waves by the LIGO (Laser Interferometer Gravitational-wave Observatory). What you may not know is that Python helped in analyzing the data, so we now have two things to celebrate: 1. Python’s use in this phenomenal direct proof of Einstein’s prediction and 2. the launch of our 2016 edition of the EuroPython conference. So here it is: *** https://ep2016.europython.eu/ *** July 17 - 24 2016 Many thanks go to our launch sponsors who have signed up early to give us that extra boost in motivation to get the conference and it’s website set up. Meet our Launch Sponsors * Bilbao Ekintza * Intel * UPV/EHU * Udemy * Python Software Foundation * Blue Yonder * Jet Brains * Numberly * Flying Circus * Limejump * RedHat * Vzzual.com * Django-CMS * Riverbank PS: We’d like to thank the EuroPython Web WG for the web site improvements and our friends at Python Italia for making their code available. With gravitational regards, -- EuroPython 2016 Team http://ep2016.europython.eu/ http://www.europython-society.org/ -- https://mail.python.org/mailman/listinfo/python-list
Re: How to properly override the default factory of defaultdict?
Herman wrote: I want to pass in the key to the default_factory of defaultdict and I found that defaultdict somehow can intercept my call to dict.__getitem__(self, key), What's happening here is that defaultdict doesn't actually override __getitem__ at all. Instead, it overrides __missing__, which gets called by the standard dict's __getitem__ for a missing key. As Steven said, you don't need a defaultdict here at all, just a dict subclass that defines __missing__ the way you want. -- Greg -- https://mail.python.org/mailman/listinfo/python-list
Re: fetchall is taking much longer time while getting data from Sybase module in Python
reetesh nigam wrote: > Hi All, > > I am retrieving data from Sybase database using Sybase module of Python. > My query is not taking time however fecthall is taking longer time. > > Below is the test script : > > def run_query(db,query): ## Run query and resturn record result > t1 = datetime.now() > cursorObj = db.cursor() > t2 = datetime.now() > cursorObj.execute(query) > t3 = datetime.now() > import pdb > pdb.set_trace() > rowset = cursorObj.fetchall() > t4 = datetime.now() > cursorObj.close() > print "Time taken to make cursor --%s"%(t2-t1) > print "Time taken to execute query --%s"%(t3-t2) > print "Time taken for fetchall--%s"%(t4-t3) > return rowset > > Output: > Time taken to make cursor --0:00:00.37 > Time taken to execute query --0:00:00.379443 > Time taken for fetchall--0:00:14.739064 fetchall() probably has to transfer a lot of rows. If you want to process them one at a time you can turn run_query into a generator def run_query(db, query): ... while True: row = cursor.fetchone() if row is None: break yield row ... A complete implementation that guarantees that the cursor is closed might look like this (untested): @contextlib.contextmanager def run_query(db, query): cursor = db.cursor() try: cursor.execute(query) yield iter(cursor.fetchone, None) finally: cursor.close() # use it with run_query(db, query) as rows: for row in rows: print row This is likely to *increase* the overall time taken, but should drastically reduce the time you have to wait for the first record to be printed, i. e. the latency. -- https://mail.python.org/mailman/listinfo/python-list
Re: ftplib throws: IndexError: tuple index out of range
Antoon Pardon wrote: >> It looks like the actual error is socket.timeout which is probably raised >> from somewhere inside the stdlib without args. > > I think I know what is going on. I have my own timeout mechanism at work > here, that works with signals and alarm. When the SIGALRM fires I just > raise the socket.timeout without arguments, which then causes the problem. OK, then it's self-inflicted pain ;) -- https://mail.python.org/mailman/listinfo/python-list
Re: ftplib throws: IndexError: tuple index out of range
Antoon Pardon wrote: >> PS: How did you produce the overview over the local variables? That looks >> nice. > I started from this recipe: > http://code.activestate.com/recipes/52215-get-more-information-from-tracebacks/ > made it more to my liking and turned it into a module. So that instead of > a program being run like this: Ah, thanks! -- https://mail.python.org/mailman/listinfo/python-list
Re: Unable to insert data into MongoDB.
I changed the port number from 27017 to 5 in the code segment:
IP = "127.0.0.1"
PORT = 27017
BUFFER_SIZE = 1024
client = MongoClient('127.0.0.1', 27017)
And my output on Python shell is:
hello world!
Connection address: ('127.0.0.1', 16951)
Connection address: ('127.0.0.1', 16953)
Connection address: ('127.0.0.1', 16957)
Connection address: ('127.0.0.1', 16958)
Connection address: ('127.0.0.1', 16959)
Connection address: ('127.0.0.1', 16961)
Connection address: ('127.0.0.1', 16962)
Connection address: ('127.0.0.1', 16963)
Connection address: ('127.0.0.1', 16964)
Connection address: ('127.0.0.1', 16965)
and it goes on and on...
What does the above port numbers mean? And I can't still see the data being
inserted into Database yet.
And as you said, if I change server.close to server.close(), then I get a
traceback:
Traceback (most recent call last):
File "C:\Users\SRA2LO\Desktop\API.py", line 41, in
data = server.recv(BUFFER_SIZE)
File "C:\Python27\lib\socket.py", line 174, in _dummy
raise error(EBADF, 'Bad file descriptor')
error: [Errno 9] Bad file descriptor
On Thursday, February 11, 2016 at 5:09:53 PM UTC+1, MRAB wrote:
> On 2016-02-11 15:12, Arjun Srivatsa wrote:
> > Hi guys. I am basically transferring the data from PLC to PC (where the
> > Python API runs) but I'm unable to insert into MongoDB thereafter. When I
> > run the Python script on IDLE, the output is
> >
> > Hello World!
> > Traceback (most recent call last): File "C:\Users\SRA2LO\Desktop\API.py",
> > line 32, in s.bind((IP, PORT)) File "C:\Python27\lib\socket.py",
> > line 228, in meth return getattr(self._sock,name)(*args) error: [Errno
> > 10013] An attempt was made to access a socket in a way forbidden by its
> > access permissions
> > and when I change the IP of MongoDB server, it shows
> >
> >
> > Hello World!
> > Traceback (most recent call last): File "C:\Users\SRA2LO\Desktop\API.py",
> > line 32, in s.bind((IP, PORT)) File "C:\Python27\lib\socket.py",
> > line 228, in meth return getattr(self._sock,name)(*args) error: [Errno
> > 10049] The requested address is not valid in its context.
> >
> > Could you please help me out? I have disabled the firewall as well.
> >
> > Here's the API I have written.
> >
> > #!/usr/bin/python
> >
> > import socket
> > import socket
> > from pymongo import MongoClient
> > #from eve import Eve
> > import datetime
> >
> > # Connection to server (PLC) on port 27017
> > server = socket.socket()
> > host = "10.52.124.135" #IP of PLC
> > port = 27017
> > BUFFER_SIZE = 1024
> > ###
> >
> > server.connect((host, port))
> > print server.recv(1024)
> >
> > server.close
> >
> > #Connection to Client (Mongodb) on port 27017
> > IP = "127.0.0.1"
> > PORT = 27017
> > BUFFER_SIZE = 1024
> >
> > client = MongoClient('127.0.0.1', 27017)
> > db = client.test_database
> >
> > s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
> > s.bind((IP, PORT))
> > s.listen(1)
> >
> > #connections loop
> > while True:
> > conn, addr = s.accept()
> > print 'Connection address:',addr
> > try:
> > # read loop
> > while True:
> > data = server.recv(BUFFER_SIZE)
> >
> > if not data: break
> > conn.sendall(data)
> >
> >
> > # send to MongoDB
> >
> > mongodoc = { "data": data, "date" : datetime.datetime.utcnow() }
> >
> >
> > ABC = db.ABC
> > ABC_id = ABC.insert_one(mongodoc).inserted_id
> >
> > finally:
> > conn.close()
> >
> I don't know whether it's relevant, but you didn't close the server
> socket. You have "server.close" instead of "server.close()".
>
> Also, the code as posted won't compile because the block after the
> "while True:" isn't indented.
--
https://mail.python.org/mailman/listinfo/python-list
Re: Changing sound volume
i had had to play some wav files and the volume was very poor. i used "audacity" and used amplify effect to change the basic amplitude. -- https://mail.python.org/mailman/listinfo/python-list
Re: asyncio - run coroutine in the background
Paul Rubin : > Threads in Python don't get you parallelism either, of course. Ah, of course. Processes it is, then. Marko -- https://mail.python.org/mailman/listinfo/python-list
Re: Unable to insert data into MongoDB.
Arjun Srivatsa wrote: > I changed the port number from 27017 to 5 in the code segment: Instead of throwing arbitrary changes at your script in the hope that one works I recommend that you write two independent scripts, from scratch: (1) write_to_db.py: Write made-up data into the MongoDB on your local machine. (2) read_from_server.py: Read data from the server and print it to stdout. Reduce these scripts to the bare minimum. Debug them one ofter another. Once both scripts work to your satisfaction replace the sample data in script one with code from script two that reads actual data. -- https://mail.python.org/mailman/listinfo/python-list
repr( open('/etc/motd', 'rt').read() )
When I do at the interpreter prompt,
repr( open('/etc/motd', 'rt').read() )
i get # 1 #:
"'\\nThe programs included with the Debian GNU/Linux system are free
software;\\nthe exact distribution terms for each program are described
in the\\nindividual files in /usr/share/doc/*/copyright.\\n\\nDebian
GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent\\npermitted
by applicable law.\\n'"
whereas if i just do:
open('/etc/motd', 'rt').read()
i get # 2 #:
'\nThe programs included with the Debian GNU/Linux system are free
software;\nthe exact distribution terms for each program are described
in the\nindividual files in /usr/share/doc/*/copyright.\n\nDebian
GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent\npermitted by
applicable law.\n'
---
With # 2 # read returns a string that the interpreter displays by
calling __str__ via print so newlines are converted to \n.
What is happening with # 1 # (repr)?
repr calls __repr__ which gives you bytes.. why does this result in \\n
--
https://mail.python.org/mailman/listinfo/python-list
Re: There has to be a better way to split this string!
On 2/9/2016 10:50 PM, Cameron Simpson wrote: On 10Feb2016 07:34, srinivas devaki wrote: PS: trying to read mailing list when you are half woke, is a bad idea and trying reply to it is even bad idea. Regrettably, when one is half awake one is unable to realise what a bad idea it may be:-) Not to worry -- you find out quick enough. :) Emile -- https://mail.python.org/mailman/listinfo/python-list
Re: repr( open('/etc/motd', 'rt').read() )
On Mon, Feb 15, 2016, at 08:05, Veek. M wrote: > What is happening with # 1 # (repr)? > repr calls __repr__ which gives you bytes.. why does this result in \\n When you call a function that returns a string directly in the interpreter prompt (i.e. without print), it passes the result to repr, which means in this case repr has been called twice. -- https://mail.python.org/mailman/listinfo/python-list
Re: Syntax error (The Python Book) Linux User and Developer Bookazine
On Sunday, 14 February 2016 13:39:52 UTC, Geoff Munn wrote:
> Hi,
>
> Noob at the Python thing so here goes,
>
> I have copied a program to demonstrate control structures in Python but get a
> syntax error at line 31, isint = False. I'm using Python 2.7.6 and Linux Mint
> based around ubuntu14.04.1. I have pasted all the code below,
>
>
>
> #!/usr/bin/env python2
>
> '''
> We are going to write a program that will ask for the user to input an
> arbitary
> number of integers, store them in a collection, and then demonstrate how
> the collection would be used with various control structures
> '''
>
> import sys # Used for the sys.exit function
>
> target_int=raw_input("How many integers?")
>
> '''
> By now the variable target_int contains a string representation of
> whatever the user typed. We nee to try and convert that to an integer but
> be ready to deal with the error if it's not. Otherwise the program will crash
> '''
>
> try:
> target_int=int(target_int)
> except ValueError:
> sys.exit("You must enter an integer")
>
> ints=list() # list to store the integers
>
> count = 0 # Track how many integers have been inputted
>
> # Keep asking for a number until we have reached the required number
> while count < target_int:
> new_int=raw_input("Please enter integer {0}:".format(count +1)
> isint = False
> try:
> new_int=int(new_int) # If the above succeeds then isint will
> #be set to true: isint = True
>
> except:
> print("You must enter an integer")
>
> '''
> Only carry on if we have an integer. If not we will loop again.
> The == below is a comparision operator, a single = is an asignment
> operator
> '''
> if isnit==True:
> ints.append(new_int) # Adds the integer to the collection
> count += 1 # Count is incremented by 1
> # The for loop
> print ("Using a for loop")
> for values in ints:
> print (str(value))
> # The while loop
> print ("Using a while loop")
> total=len(ints) # We already have the total from above but using len we
> can determine from the ints list.
> count = 0
> while count < total:
> print (str(ints[count]))
> count += 1
Thanks Peter and Chris, yes missed the parentheses by taking the error as being
in line 31, DOH but a lesson learned. I have checked and checked the code I
entered against the provided code and had to make some more changes to at least
go through the first while loop but have given up on the rest of it. Given your
comments do you think its worth persevering with this book or is there a better
'entry' into Python programming?
--
https://mail.python.org/mailman/listinfo/python-list
Re: Unable to insert data into MongoDB.
Hi Peter.
Thank you for the reply.
This is the read_server code:
import socket
from pymongo import MongoClient
#import datetime
import sys
# Connection to server (PLC) on port 27017
host = "10.52.124.135"
port = 27017
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((host, port))
sys.stdout.write(s.recv(1024))
And the write_db code:
from pymongo import MongoClient
import datetime
import socket
import sys
client = MongoClient('mongodb://localhost:27017/')
db = client.test_database
mongodoc = { "data": 'data', "date" : datetime.datetime.utcnow() }
values = db.values
values_id = values.insert_one(mongodoc).inserted_id
So, both these scripts work independently. While, read_server shows the output
of the actual data from PLC, write_db inserts the sample data into the MongoDB.
I am not sure as to how to combine these both and get the desired output.
On Monday, February 15, 2016 at 12:37:02 PM UTC+1, Peter Otten wrote:
> Arjun Srivatsa wrote:
>
> > I changed the port number from 27017 to 5 in the code segment:
>
> Instead of throwing arbitrary changes at your script in the hope that one
> works I recommend that you write two independent scripts, from scratch:
>
> (1) write_to_db.py:
> Write made-up data into the MongoDB on your local machine.
>
> (2) read_from_server.py:
> Read data from the server and print it to stdout.
>
> Reduce these scripts to the bare minimum. Debug them one ofter another.
>
> Once both scripts work to your satisfaction replace the sample data in
> script one with code from script two that reads actual data.
--
https://mail.python.org/mailman/listinfo/python-list
Re: Syntax error (The Python Book) Linux User and Developer Bookazine
On Mon, Feb 15, 2016 at 9:56 AM, Geoff Munn wrote:
> On Sunday, 14 February 2016 13:39:52 UTC, Geoff Munn wrote:
> > Hi,
> >
> > Noob at the Python thing so here goes,
> >
> > I have copied a program to demonstrate control structures in Python but
> get a syntax error at line 31, isint = False. I'm using Python 2.7.6 and
> Linux Mint based around ubuntu14.04.1. I have pasted all the code below,
> >
> >
> >
> > #!/usr/bin/env python2
> >
> > '''
> > We are going to write a program that will ask for the user to input an
> arbitary
> > number of integers, store them in a collection, and then demonstrate how
> > the collection would be used with various control structures
> > '''
> >
> > import sys # Used for the sys.exit function
> >
> > target_int=raw_input("How many integers?")
> >
> > '''
> > By now the variable target_int contains a string representation of
> > whatever the user typed. We nee to try and convert that to an integer but
> > be ready to deal with the error if it's not. Otherwise the program will
> crash
> > '''
> >
> > try:
> > target_int=int(target_int)
> > except ValueError:
> > sys.exit("You must enter an integer")
> >
> > ints=list() # list to store the integers
> >
> > count = 0 # Track how many integers have been inputted
> >
> > # Keep asking for a number until we have reached the required number
> > while count < target_int:
> > new_int=raw_input("Please enter integer {0}:".format(count +1)
> > isint = False
> > try:
> > new_int=int(new_int) # If the above succeeds then isint will
> > #be set to true: isint = True
> >
> > except:
> > print("You must enter an integer")
> >
> > '''
> > Only carry on if we have an integer. If not we will loop again.
> > The == below is a comparision operator, a single = is an asignment
> operator
> > '''
> > if isnit==True:
> > ints.append(new_int) # Adds the integer to the collection
> > count += 1 # Count is incremented by 1
> > # The for loop
> > print ("Using a for loop")
> > for values in ints:
> > print (str(value))
> > # The while loop
> > print ("Using a while loop")
> > total=len(ints) # We already have the total from above but using
> len we can determine from the ints list.
> > count = 0
> > while count < total:
> > print (str(ints[count]))
> > count += 1
>
> Thanks Peter and Chris, yes missed the parentheses by taking the error as
> being in line 31, DOH but a lesson learned. I have checked and checked the
> code I entered against the provided code and had to make some more changes
> to at least go through the first while loop but have given up on the rest
> of it. Given your comments do you think its worth persevering with this
> book or is there a better 'entry' into Python programming?
> --
> https://mail.python.org/mailman/listinfo/python-list
>
Learn Python the Hard Way is pretty good some people say. Its online.
Also Diving into Python is online written by the now offline Mark Pilgrim.
--
Joel Goldstick
http://joelgoldstick.com/stats/birthdays
--
https://mail.python.org/mailman/listinfo/python-list
Re: Unable to insert data into MongoDB.
Okay,
I added
with open('yourfile','w') as f:
f.write(data)
to the read_server code in which the Actual data is stored in a file on my
desktop. Then, it must be possible to read this file in the Write_db script to
insert the data.
On Monday, February 15, 2016 at 4:00:37 PM UTC+1, Arjun Srivatsa wrote:
> Hi Peter.
>
> Thank you for the reply.
>
> This is the read_server code:
>
> import socket
> from pymongo import MongoClient
> #import datetime
> import sys
>
> # Connection to server (PLC) on port 27017
> host = "10.52.124.135"
> port = 27017
>
> s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
> s.connect((host, port))
> sys.stdout.write(s.recv(1024))
>
>
> And the write_db code:
>
> from pymongo import MongoClient
> import datetime
> import socket
> import sys
>
> client = MongoClient('mongodb://localhost:27017/')
> db = client.test_database
>
> mongodoc = { "data": 'data', "date" : datetime.datetime.utcnow() }
> values = db.values
> values_id = values.insert_one(mongodoc).inserted_id
>
>
> So, both these scripts work independently. While, read_server shows the
> output of the actual data from PLC, write_db inserts the sample data into the
> MongoDB.
>
> I am not sure as to how to combine these both and get the desired output.
>
>
>
> On Monday, February 15, 2016 at 12:37:02 PM UTC+1, Peter Otten wrote:
> > Arjun Srivatsa wrote:
> >
> > > I changed the port number from 27017 to 5 in the code segment:
> >
> > Instead of throwing arbitrary changes at your script in the hope that one
> > works I recommend that you write two independent scripts, from scratch:
> >
> > (1) write_to_db.py:
> > Write made-up data into the MongoDB on your local machine.
> >
> > (2) read_from_server.py:
> > Read data from the server and print it to stdout.
> >
> > Reduce these scripts to the bare minimum. Debug them one ofter another.
> >
> > Once both scripts work to your satisfaction replace the sample data in
> > script one with code from script two that reads actual data.
--
https://mail.python.org/mailman/listinfo/python-list
Re: Syntax error (The Python Book) Linux User and Developer Bookazine
On Monday, 15 February 2016 15:07:03 UTC, Joel Goldstick wrote:
> On Mon, Feb 15, 2016 at 9:56 AM, Geoff Munn wrote:
>
> > On Sunday, 14 February 2016 13:39:52 UTC, Geoff Munn wrote:
> > > Hi,
> > >
> > > Noob at the Python thing so here goes,
> > >
> > > I have copied a program to demonstrate control structures in Python but
> > get a syntax error at line 31, isint = False. I'm using Python 2.7.6 and
> > Linux Mint based around ubuntu14.04.1. I have pasted all the code below,
> > >
> > >
> > >
> > > #!/usr/bin/env python2
> > >
> > > '''
> > > We are going to write a program that will ask for the user to input an
> > arbitary
> > > number of integers, store them in a collection, and then demonstrate how
> > > the collection would be used with various control structures
> > > '''
> > >
> > > import sys # Used for the sys.exit function
> > >
> > > target_int=raw_input("How many integers?")
> > >
> > > '''
> > > By now the variable target_int contains a string representation of
> > > whatever the user typed. We nee to try and convert that to an integer but
> > > be ready to deal with the error if it's not. Otherwise the program will
> > crash
> > > '''
> > >
> > > try:
> > > target_int=int(target_int)
> > > except ValueError:
> > > sys.exit("You must enter an integer")
> > >
> > > ints=list() # list to store the integers
> > >
> > > count = 0 # Track how many integers have been inputted
> > >
> > > # Keep asking for a number until we have reached the required number
> > > while count < target_int:
> > > new_int=raw_input("Please enter integer {0}:".format(count +1)
> > > isint = False
> > > try:
> > > new_int=int(new_int) # If the above succeeds then isint will
> > > #be set to true: isint = True
> > >
> > > except:
> > > print("You must enter an integer")
> > >
> > > '''
> > > Only carry on if we have an integer. If not we will loop again.
> > > The == below is a comparision operator, a single = is an asignment
> > operator
> > > '''
> > > if isnit==True:
> > > ints.append(new_int) # Adds the integer to the collection
> > > count += 1 # Count is incremented by 1
> > > # The for loop
> > > print ("Using a for loop")
> > > for values in ints:
> > > print (str(value))
> > > # The while loop
> > > print ("Using a while loop")
> > > total=len(ints) # We already have the total from above but using
> > len we can determine from the ints list.
> > > count = 0
> > > while count < total:
> > > print (str(ints[count]))
> > > count += 1
> >
> > Thanks Peter and Chris, yes missed the parentheses by taking the error as
> > being in line 31, DOH but a lesson learned. I have checked and checked the
> > code I entered against the provided code and had to make some more changes
> > to at least go through the first while loop but have given up on the rest
> > of it. Given your comments do you think its worth persevering with this
> > book or is there a better 'entry' into Python programming?
> > --
> > https://mail.python.org/mailman/listinfo/python-list
> >
>
>
> Learn Python the Hard Way is pretty good some people say. Its online.
> Also Diving into Python is online written by the now offline Mark Pilgrim.
> --
> Joel Goldstick
> http://joelgoldstick.com/stats/birthdays
Thanks Joel
--
https://mail.python.org/mailman/listinfo/python-list
Re: Make a unique filesystem path, without creating the file
On 2016-02-14, Ben Finney wrote: > Howdy all, > > How should a program generate a unique filesystem path and *not* create > the filesystem entry? Short answer: you can't because it's the filesystem entry operation that is atomic and guarantees uniqueness. > [..] > What standard library function should I be using to generate > ‘tempfile.mktemp’-like unique paths, and *not* ever create a real file > by that path? What's the point of creating a unique path if you don't want to create the file? -- Grant Edwards grant.b.edwardsYow! I'm rated PG-34!! at gmail.com -- https://mail.python.org/mailman/listinfo/python-list
Re: Make a unique filesystem path, without creating the file
On 2016-02-15, Ben Finney wrote: > Dan Sommers writes: > >> On Mon, 15 Feb 2016 11:08:52 +1100, Ben Finney wrote: >> >> > I am unconcerned with whether there is a real filesystem entry of >> > that name; the goal entails having no filesystem activity for this. >> > I want a valid unique filesystem path, without touching the >> > filesystem. >> >> That's an odd use case. > > It's very common to want filesystem paths divorced from accessing a > filesystem entry. If the filesystem paths are not associated with a filesystem, what do you mean by "unique"? You want to make sure that path which doesn't exist in some filesystem is different from all other paths that don't exist in some filesystem? > For example: test paths in a unit test. Filesystem access is orders > of magnitude slower than accessing fake files in memory only, How is "fake files in memory" not a filesystem? -- Grant Edwards grant.b.edwardsYow! The Korean War must at have been fun. gmail.com -- https://mail.python.org/mailman/listinfo/python-list
Re: Unable to insert data into MongoDB.
Arjun Srivatsa wrote:
> Hi Peter.
>
> Thank you for the reply.
>
> This is the read_server code:
>
> import socket
> from pymongo import MongoClient
> #import datetime
> import sys
>
> # Connection to server (PLC) on port 27017
> host = "10.52.124.135"
> port = 27017
>
> s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
> s.connect((host, port))
> sys.stdout.write(s.recv(1024))
>
>
> And the write_db code:
>
> from pymongo import MongoClient
> import datetime
> import socket
> import sys
>
> client = MongoClient('mongodb://localhost:27017/')
> db = client.test_database
>
> mongodoc = { "data": 'data', "date" : datetime.datetime.utcnow() }
> values = db.values
> values_id = values.insert_one(mongodoc).inserted_id
>
>
> So, both these scripts work independently. While, read_server shows the
> output of the actual data from PLC, write_db inserts the sample data into
> the MongoDB.
>
> I am not sure as to how to combine these both and get the desired output.
What I mean is once you have working scripts
connect_to_mongodb()
while True:
record = make_fake_data()
insert_record_into_mongodb(record)
and
connect_to_server()
while True:
record = read_record_from_server()
print(record)
you can combine the code in a third script to
connect_to_server()
connect_to_mongodb()
while True:
record = read_record_from_server()
insert_record_into_mongodb(record)
and be fairly sure that the combination works, too.
--
https://mail.python.org/mailman/listinfo/python-list
Re: What is heating the memory here? hashlib?
Às 08:12 de 15-02-2016, Johannes Bauer escreveu: > On 15.02.2016 03:21, Paulo da Silva wrote: > >> So far I tried the program twice and it ran perfectly. > > I think you measured your RAM consumption wrong. > > Linux uses all free RAM as HDD cache. That's what is used in "buffers". > That is, it's not "free", but it would be free if any process would > sbrk(). My guess is that you only looked at the "free" number going down > and concluded your program is eating your RAM. Which it wasn't. > No, for sure. I monitored (using atop) free, cache and swap. In general, because I only have 2GB, freemem is almost always a few tens of MB. Remaining "free" memory is in Cache. When Cache goes low it begins to swap out. Paulo -- https://mail.python.org/mailman/listinfo/python-list
Re: repr( open('/etc/motd', 'rt').read() )
On 2/15/2016 8:05 AM, Veek. M wrote:
When I do at the interpreter prompt,
repr( open('/etc/motd', 'rt').read() )
i get # 1 #:
When posting questions here or at Stackoverflow or elsewhere, it is a
really good idea to develop and post a 'minimal, complete, verifiable
example' that demonstrates the behavior in question. In this case, the
open and read calls are just noise. A string with a newline illustrates
your question without distraction.
>>> s = '\n'
>>> len(s)
1
>>> len(str(s))
1
>>> len(repr(s))
4
>>> s
'\n'
>>> str(s)
'\n'
>>> repr(s)
"'\\n'"
>>> print(s)
>>> print(str(s))
>>> print(repr(s))
'\n'
>>>
For this question, 'at the interpreter prompt' is essential, so leaving
the >>> prompt is a good idea. I did the above with 3.5.1 also in IDLE
and got exactly the same result, which should be the case.
print('start')
s='\n'
print(s)
print(str(s))
print(repr(s))
print('add repr')
print(repr(s))
print(repr(str(s)))
print(repr(repr(s)))
print('end')
duplicates the collective >>> responses seen above and demonstrates, as
Random832 said, that '>>> expr' prints repr(expr).
start
'\n'
add repr
'\n'
'\n'
"'\\n'"
end
--
Terry Jan Reedy
--
https://mail.python.org/mailman/listinfo/python-list
Re: Syntax error (The Python Book) Linux User and Developer Bookazine
On 02/15/2016 07:06 AM, Joel Goldstick wrote: [snip a lot...] Learn Python the Hard Way is pretty good some people say. Its online. Also Diving into Python is online written by the now offline Mark Pilgrim. I have a couple of "Hard Way" books and personally, I don't like his style of teaching. Of course, take that as one person's opinion -- and as always, YMMV. :-) -=- Larry -=- -- https://mail.python.org/mailman/listinfo/python-list
Re: Make a unique filesystem path, without creating the file
Ben Finney schreef op 2016-02-14 22:46: How should a program generate a unique filesystem path and *not* create the filesystem entry? > ... What standard library function should I be using to generate ‘tempfile.mktemp’-like unique paths, and *not* ever create a real file by that path? Use uuid.uuid1()? -- The saddest aspect of life right now is that science gathers knowledge faster than society gathers wisdom. -- Isaac Asimov Roel Schroeven -- https://mail.python.org/mailman/listinfo/python-list
Re: Make a unique filesystem path, without creating the file
On 15Feb2016 12:19, Ben Finney wrote: Dan Sommers writes: On Mon, 15 Feb 2016 11:08:52 +1100, Ben Finney wrote: > I am unconcerned with whether there is a real filesystem entry of > that name; the goal entails having no filesystem activity for this. > I want a valid unique filesystem path, without touching the > filesystem. That's an odd use case. It's very common to want filesystem paths divorced from accessing a filesystem entry. For example: test paths in a unit test. Filesystem access is orders of magnitude slower than accessing fake files in memory only, it is more complex and prone to irrelevant failures. So in such a test case filesystem access should be avoided as unnecessary. But.. then why a filesystem path at all in that case? Why use a filesystem as a reference at all? I've been watching this for a few days, and am struggling to understand your use case. The only modes I can imagine for such a thing (a generated but unused filename) are: checking that the name is syntactly valid, for whatever constrains you may have (but if you're calling an opaque mktemp-like function, is this feasible or remediable?) checking that the name generated does in fact not correspond to an existing file (which presumes that the target directory has no other users, which also implies that you don't need mktemp - a simple prefix+unused-ordinal will do) generating test paths using a real filesystem as a reference but not making a test file - I'm having trouble imagining how this can be useful generating test paths without using a real filesystem as a reference, but then you can't even use mktemp I think I can contrive your test case scenario using #3: filepath = mktemp(existing_dir_path) fp = InMemoryFileLikeClassWithBogusName(filepath) do I/O on fp ... but I don't see how it is useful to have a notion of a filepath at all in this case, and therefore I don't see why you would want a mktemp-like function available. Can you elaborate with a concrete example and its purpose which would work with a mktemp-ish official function? You say: One valid filesystem path each time it's accessed. That is, behaviour equivalent to ‘tempfile.mktemp’. My question is because the standard library clearly has this useful functionality implemented, but simultaneously warns strongly against its use. I'm looking for how to get at that functionality in a non-deprecated way, without re-implementing it myself. I think "the standard library clearly has this useful functionality implemented, but simultaneously warns strongly against its use" pretty much precludes this. I think you probably need to reimplement. However if your intent is never to use the path you can use something very simple (my personal habit is prefix+ordinal where that doesn't already exist - keep the last ordinal to arrange a distinct name next time). Cheers, Cameron Simpson -- https://mail.python.org/mailman/listinfo/python-list
Re: Make a unique filesystem path, without creating the file
Gregory Ewing wrote: > Ben Finney wrote: >> One valid filesystem path each time it's accessed. That is, behaviour >> equivalent to ‘tempfile.mktemp’. >> >> My question is because the standard library clearly has this useful >> functionality implemented, but simultaneously warns strongly against its >> use. > > But it *doesn't*, Yes, it does. > if your requirement is truly to not touch the filesystem at all, because > tempfile.mktemp() *reads* the file system to make sure the name it's > returning isn't in use. But there is a race condition occurring between the moment that the filesystem has been read and is being written to by another user. Hence the deprecation in favor of tempfile.mkstemp() which also *creates* the file instead, and the warning about the security hole if tempfile.mktemp() is used anyway. You can use tempfile.mktemp() only as long as it is irrelevant if a file with that name already exists, or exists later but was not created by you. -- PointedEars Twitter: @PointedEars2 Please do not cc me. / Bitte keine Kopien per E-Mail. -- https://mail.python.org/mailman/listinfo/python-list
Re: Make a unique filesystem path, without creating the file
Roel Schroeven writes: > Use uuid.uuid1()? That has potential. A little counter-intuitive, for use in documentation about testing filesystem paths; but not frightening or dubious to the conscientious reader. I'll see whether that meets this use case, thank you. The bug report (to make a supported ‘tempfile’ API for generating filesystem paths only) remains, and fixing that would be the correct way to address this IMO. -- \ “I used to be a proofreader for a skywriting company.” —Steven | `\Wright | _o__) | Ben Finney -- https://mail.python.org/mailman/listinfo/python-list
Re: [STORY-TIME] THE BDFL AND HIS PYTHON PETTING ZOO
On Friday, February 12, 2016 at 1:51:35 AM UTC-6, John Ladasky wrote: Reguarding a migration from Python2 to Pyhton3, John said: > I had to wait until my favorite packages were ported > (numpy, scipy, matplotlib, pandas). WxPython is not ported either, much to my chagrin. > But once that happened, I moved from Py2 to Py3 years ago > with scarcely a bump, bruise, or scratch. So you have no Python2.x code remaining in your repos? Are you telling us that you moved *EVERYTHING* to Python3? If so, i can't imagine how something like that would even be possible, hmm, unless of course, you don't have much code to move...? I could imagine that *IF* someone's entire Python repo consisted of a couple hundred (or less) small scripts, with few, or no, dependencies, such a migration would be relatively easy. But even though i've only been writing Python code for a handful of years, my repo consists of thousands of scripts, millions of lines of code, and many, *MANY* dependencies -- because after all, DRY is very important, yes? (That reminds me, i need to write a script to compile some statistics on these files...) > I like lazy evaluation. Well, it is a "Pythonic feature" no doubt. And I'll admit, many of the changes are "good changes". But they are not good enough to risk blowing up millions of lines of code, John. And what will i gain if i did? Nothing. There are *NO* new features to offer my users, and even the features that *WOULD* improve my coding experiences, are not enough to warrant the misery.Heck, almost all of these festures i could implement myself. What's easier: migrating millions of lines into a hostile environment just so i can gain a few new functionalities, or writing a few lines to bring the needed functionalities to *ME*? I don't need, nor do i want, *EVERYTHING* that Python3 brings to the table. > I think that Unicode handling is vastly improved (and yes, > I'm fully aware that exactly two people in this newsgroup > don't agree, they make sure we all know). I have > encountered few surprises, and nothing that makes my job > harder. Well John, not everyone is going to experience the same "level of onerous" when they port their code over to Python3. Perhaps your case is unique. And there is no way for myself, or anyone one else, to make an objective judgment, without looking at the breath and depth of the code you had to migrate. Perhaps you don't have a lot of dependencies. Perhaps you have not engineered "feature-rich libraries". Perhaps you have have not written code that is on the bleeding edge, or pushed your mental power to it's extremes. Perhaps your code style is to "play it safe". Perhaps you're scripts consist mostly of simple maintenance and utilities. So even if you "believe" that the migration from Python2.x to Python3.x is smooth, you're only able to make that judgment relative to own *PERSONAL* experience, with your own *PERSONAL* library. So even though there is only two people in this group who don't care for Pyhton Unicode handling (hey, your stats, not mine!), there is only *ONE* of you who can judge the level of onerous required to migrate *YOUR* code! Until you make the depth and breath of you repo publicly view-able, your opinion is just that -- an *OPINION*! As for me, I've written vast libraries over the top of wxPython and Tkinter. I've extended existing functionality, added new functionality, and created a more consistent and intuitive API than either library could have dreamed. I've spent many long hours debugging, testing, and perfecting these libraries, and so far, they work very nicely. Heck, my refactored Tkinter API, and IDLE editor, is leaps and bounds ahead of anything Python3 could manage to do. And it's a shame too, because these two libraries are among some of the oldest and most neglected of all. But now i'm faced with a tough choice: (1) I could attempt to migrate all these libraries, and risk mountains of subtle bugs echoing down the inheritance/dependency chains, or (2) I could leave the code where it is, and milk-it until the majority of OS's will no longer run Python2. JUST FYI: I'M GOING WITH CHOICE NUMBER TWO! Why? Because only a fool would risk "exception hell" on code that has been stable for a reasonable amount of time. Ever heard the expression: "if it ain't broke, don't fix it"? Well my code ain't broke, and until it becomes broken, my time would be better spent forging new paths, instead of flogging the old ones like dead, quadrupedaled, "beasts of burden"! > To be sure, people who are entrenched in the Py2 way of > doing things, with a lot of legacy code, have some work to > do -- on their code, and possibly on their brains. Your "veiled ad hominem" is not justified John.With that statement, you've just insulted well over half of the Python community -- and the "better half", i might add! You're talking about people who have been coding python, and many other languages, probably long before you were even born! I'm not as "curm
Re: Make a unique filesystem path, without creating the file
On Mon, 15 Feb 2016 15:28:27 +1100, Ben Finney wrote: > The behaviour is already implemented in the standard library. What I'm > looking for is a way to use it (not re-implement it) that is public API > and isn't scolded by the library documentation. So, basically you want (essentially) the exact behaviour of tempfile.mktemp(), except without any mention of the (genuine) risks that such a function presents? I suspect that you'll have to settle for either a) using that function and simply documenting the reasons why it isn't an issue in this particular case, or b) re-implementing it (so that you can choose to avoid mentioning the issue in its documentation). At the outside, you *might* have a third option: c) persuade the maintainers to tweak the documentation to further clarify that the risk arises from creating a file with the returned name, not from simply calling the function. But actually it's already fairly clear if you actually read it. If it's the bold-face "Warning:" and the red background that you don't like, I wouldn't expect those to go away either for mktemp() or for any other function with similar behaviour (i.e. something which someone *might* try to use to actually create temporary files). The simple fact that it might get used that way is enough to warrant a prominent warning. -- https://mail.python.org/mailman/listinfo/python-list
Re: Make a unique filesystem path, without creating the file
On Sunday, February 14, 2016 at 10:55:11 PM UTC-6, Steven D'Aprano wrote: > If you want to guarantee that these faux pathnames can't > leak out of your test suite and touch the file system, > prepend an ASCII NUL to them. That will make it an illegal > path on all file systems that I'm aware of. Hmm, the unfounded fears in this thread are beginning to remind me of a famous Black Sabbath song. Finished with "py tempfile", 'cause it, couldn't help to, ease my mind. People think i'm insane, because, i want "faux paths", all the time. All day long i think of ways, but nothing seems to, satisfy. Think i'll loose my mind, if i don't, find a py-module to, pacify. CAN YOU HELP ME? MAKE "FAUX PATHS" TODY, OH YEAH... -- https://mail.python.org/mailman/listinfo/python-list
Re: [STORY-TIME] THE BDFL AND HIS PYTHON PETTING ZOO
On Tue, Feb 16, 2016 at 1:02 PM, Rick Johnson
wrote:
>> But once that happened, I moved from Py2 to Py3 years ago
>> with scarcely a bump, bruise, or scratch.
>
> So you have no Python2.x code remaining in your repos? Are
> you telling us that you moved *EVERYTHING* to Python3? If
> so, i can't imagine how something like that would even be
> possible, hmm, unless of course, you don't have much code to
> move...?
I have one project that's bound to Py2 by its one dependency, and I've
kept it Py3-compatible apart from the one block of code that sets that
up. (That includes renaming a function that was originally called
"await", even though that's not actually a keyword as of 3.6.) All the
rest of my code will run on Python 3. However, there is still a lot of
Python 2 code in my repos - it's the same code! Most of my stuff is
simple enough that I keep it 2/3 compatible. Sometimes there's a block
of code at the top to deal with {raw_,}input or a future directive or
a changed import name, but the bulk of the code is unchanged.
Of course, now that I go to actually *check*, it turns out that a lot
of my code has accidentally not been cross-version compatible. I use
the open(fn, encoding="...") form in quite a few places, so to make
that work, I would need "from io import open" (which seems to be a
safe no-op on Py3), and there are places where I use multi-arg
versions of print, which will end up displaying tuples in Py2 if I
don't slap in a __future__ directive. But that just means that I
really truly have *moved* to Python 3, to the extent that I don't
always even test my code on 2.7 any more.
ChrisA
--
https://mail.python.org/mailman/listinfo/python-list
Re: Changing sound volume
On Monday, February 15, 2016 at 4:58:04 AM UTC-6, izik43 wrote: > i had had to play some wav files and the volume was very > poor. i used "audacity" and used amplify effect to change > the basic amplitude. Yes, you can achieve a more powerful punch that way, but it's not the same as what the OP needs. Heck, with that method, he just might blow-out his cheap monitor speakers! ;-) Besides, editing an audio file is not something you should do without practice. If you're not careful, you can destroy the sound quality by "clipping-off" the extreme tips of the waveform. Sure, distortion can sometimes add a desirable effect, but too much of a good thing, well, you know... In my early days as an "amateur sound guy", i destroyed some valuable live recordings due to my ignorance of clipping. Yeah, i didn't realize i was editing the *ACTUAL FILE*, and not a copy of it! Had to learn the hard way. :-'( When you're an audiophile, those kinds of screw-ups are devastating. Magical musical moments are a rarity. -- https://mail.python.org/mailman/listinfo/python-list
Re: Make a unique filesystem path, without creating the file
I would create a RAM disk (http://www.cyberciti.biz/faq/howto-create-linux-ram-disk-filesystem/), generate all the path/files I want with any, or my own algorithm, run the tests, unmount it, destroy it, be happy ... Whats wrong with that?? AFAIK, RAM disks do not get logged, and even if they do, any "insecure" file created would also be gone. On Sunday, February 14, 2016 at 4:46:42 PM UTC-5, Ben Finney wrote: > Howdy all, > > How should a program generate a unique filesystem path and *not* create > the filesystem entry? > > The 'tempfile.mktemp' function is strongly deprecated, and rightly so > https://docs.python.org/3/library/tempfile.html#tempfile.mktemp> > because it leaves the program vulnerable to insecure file creation. > > In some code (e.g. unit tests) I am calling 'tempfile.mktemp' to > generate a unique path for a filesystem entry that I *do not want* to > exist on the real filesystem. In this case the filesystem security > concerns are irrelevant because there is no file. > > The deprecation of that function is a concern still, because I don't > want code that makes every conscientious reader need to decide whether > the code is a problem. Instead the code should avoid rightly-deprecated > APIs. > > It is also prone to that API function disappearing at some point in the > future, because it is explicitly and strongly deprecated. > > So I agree with the deprecation, but the library doesn't appear to > provide a replacement. > > What standard library function should I be using to generate > 'tempfile.mktemp'-like unique paths, and *not* ever create a real file > by that path? > > -- > \"If you have the facts on your side, pound the facts. If you | > `\ have the law on your side, pound the law. If you have neither | > _o__) on your side, pound the table." --anonymous | > Ben Finney -- https://mail.python.org/mailman/listinfo/python-list
Re: Make a unique filesystem path, without creating the file
"Mario R. Osorio" writes: > I would create a RAM disk > (http://www.cyberciti.biz/faq/howto-create-linux-ram-disk-filesystem/), > generate all the path/files I want with any, or my own algorithm, run > the tests, unmount it, destroy it, be happy ... Whats wrong with > that?? It is addressing the problem at a different level. I am not asking about writing a wrapper around the test suite, I am asking about an API to generate filesystem paths. Your solution is a fine response to a different question. -- \“Consider the daffodil. And while you're doing that, I'll be | `\ over here, looking through your stuff.” —Jack Handey | _o__) | Ben Finney -- https://mail.python.org/mailman/listinfo/python-list
Re: [STORY-TIME] THE BDFL AND HIS PYTHON PETTING ZOO
On Monday, February 15, 2016 at 6:02:24 PM UTC-8, Rick Johnson wrote: > On Friday, February 12, 2016 at 1:51:35 AM UTC-6, John Ladasky wrote: > > Reguarding a migration from Python2 to Pyhton3, John said: > > I had to wait until my favorite packages were ported > > (numpy, scipy, matplotlib, pandas). > > WxPython is not ported either, much to my chagrin. I was a big fan of WxPython myself. I waited for Phoenix for a while, then decided to learn PyQt5. Having been exposed to both GUI's ways of doing things, I'm not sure which I prefer. I never liked Tkinter. > > But once that happened, I moved from Py2 to Py3 years ago > > with scarcely a bump, bruise, or scratch. > > So you have no Python2.x code remaining in your repos? Are > you telling us that you moved *EVERYTHING* to Python3? If > so, i can't imagine how something like that would even be > possible, hmm, unless of course, you don't have much code to > move...? > > I could imagine that *IF* someone's entire Python repo > consisted of a couple hundred (or less) small scripts, with > few, or no, dependencies, such a migration would be > relatively easy. But even though i've only been writing > Python code for a handful of years, my repo consists of > thousands of scripts, millions of lines of code, and many, > *MANY* dependencies -- because after all, DRY is very > important, yes? It's true, I only have about 25,000 lines of code. My largest program suite is only about 10% of that. And indeed, it's now all in Py3. The 2to3 utility took care of most of my porting needs. Yes, I aspire to the DRY principle. I've written a few packages which extend Numpy's behavior in specific ways which I've re-used many times. I acknowledge that there's a vocal group of Py2 users who have decided that it's too much trouble to switch. The PSF has decided that its energies are better expended on growing Py3 than on maintaining Py2 for that diminishing user base. But no one owns Python. There is no law that states that Py2 adherents cannot maintain the language themselves, if they want it. Sheesh. Just do it. Make it yours. Or make something better if you want. You state: > I don't need Python3. And i reckon that by the time i do, > something more interesting will come along, or, i'll create > something more interesting myself. i've been drafting, and > dreaming, of a new language spec for over a year now.. And > the best thing about starting a new language, you can do > anything you want... no dependencies! If you have all the skills that you claim, you're a far better programmer than I. So -- exactly why are you complaining to people who are developing and/or using Py3? Go off and impress everyone. Become your own BDFL. Cheers. -- https://mail.python.org/mailman/listinfo/python-list
Project Xnod is created
I am happy to announce that project Xnod is created. https://sourceforge.net/projects/systemnode/ Xnod tries to develop a more uniform language or scaffolding, or kind of program deployment to reduce efforts for nearly every programming tasks. Xnod is currently based on Xarg, and has to be a collabrated project. So, consider join this project in this initial stage helping ourselves by sharing your know-how. -- https://mail.python.org/mailman/listinfo/python-list
Re: asyncio - run coroutine in the background
On Mon, Feb 15, 2016 at 6:39 PM, Paul Rubin wrote:
> "Frank Millman" writes:
>> The benefit of my class is that it enables me to take the coroutine
>> and run it in another thread, without having to re-engineer the whole
>> thing.
>
> Threads in Python don't get you parallelism either, of course.
>
They can. The only limitation is that, in CPython (and some others),
no two threads can concurrently be executing Python byte-code. The
instant you drop into a C-implemented function, it can release the GIL
and let another thread start running. Obviously this happens any time
there's going to be a blocking API call (eg if one thread waits on a
socket read, others can run), but it can also happen with
computational work:
import numpy
import threading
def thread1():
arr = numpy.zeros(1, dtype=numpy.int64)
while True:
print("1: %d" % arr[0])
arr += 1
arr = (arr * arr) % 142957
def thread2():
arr = numpy.zeros(1, dtype=numpy.int64)
while True:
print("2: %d" % arr[0])
arr += 2
arr = (arr * arr) % 142957
threading.Thread(target=thread1).start()
thread2()
This will happily keep two CPU cores occupied. Most of the work is
being done inside Numpy, which releases the GIL before doing any work.
So it's not strictly true that threading can't parallelise Python code
(and as mentioned, it depends on your interpreter - Jython can, I
believe, do true multithreading), but just that there are limitations
on what can execute concurrently.
ChrisA
--
https://mail.python.org/mailman/listinfo/python-list
Re: Make a unique filesystem path, without creating the file
Cameron Simpson writes:
> I've been watching this for a few days, and am struggling to
> understand your use case.
Yes, you're not alone. This surprises me, which is why I'm persisting.
> Can you elaborate with a concrete example and its purpose which would
> work with a mktemp-ish official function?
An example::
import io
import tempfile
names = tempfile._get_candidate_names()
def test_frobnicates_configured_spungfile():
""" ‘foo’ should frobnicate the configured spungfile. """
fake_file_path = os.path.join(tempfile.gettempdir(), names.next())
fake_file = io.BytesIO("Lorem ipsum, dolor sit amet".encode("utf-8"))
patch_builtins_open(
when_accessing_path=fake_file_path,
provide_file=fake_file)
system_under_test.config.spungfile_path = fake_file_path
system_under_test.foo()
assert_correctly_frobnicated(fake_file)
So the test case creates a fake file, makes a valid filesystem path to
associate with it, then patches the ‘open’ function so that it will
return the fake file when that specific path is requested.
Then the test case alters the system under test's configuration, giving
it the generated filesystem path for an important file. The test case
then calls the function about which the unit test is asserting
behaviour, ‘system_under_test.foo’. When that call returns, the test
case asserts some properties of the fake file to ensure the system under
test actually accessed that file.
With a supported standard library API for this – ‘tempfile.makepath’ for
example – the generation of the filesystem path would change from four
separate function calls, one of which is a private API::
names = tempfile._get_candidate_names()
fake_file_path = os.path.join(tempfile.gettempdir(), names.next())
to a simple public function call::
fake_file_path = tempfile.makepath()
This whole thread began because I expected such an API would exist.
> I don't see how it is useful to have a notion of a filepath at all
> in this case, and therefore I don't see why you would want a
> mktemp-like function available.
Because the system under test expects to be dealing with a filesystem,
including normal restrictions on filesystem paths.
The filesystem path needs to be valid because the test case isn't making
assertions about what the system does with invalid paths. A test case
should be very narrow in what it asserts so that the failure's cause is
as obvious as possible.
The filesystem path needs to be unpredictable to make sure we're not
using some hard-coded value; the test case asserts that the system under
test will access whatever file is named in the configuration.
The file object needs to be fake because the test case should not be
prone to irrelevant failures when the real filesystem isn't behaving as
expected; this test case makes assertions only about what
‘system_under_test.foo’ does internally, not what the filesystem does.
The system library functionality should be providing this because it's
*already implemented there* and well tested and maintained. It should be
in a public non-deprecated API because merely generating filesystem
paths is not a security risk.
> But.. then why a filesystem path at all in that case?
Because the system under test is expecting valid filesystem paths, and I
have no good reason to violate that constraint.
> Why use a filesystem as a reference at all?
An actual running filesystem is irrelevant to this inquiry.
I'm only wanting to use functionality, with the constraints I enumerated
earlier (already implemented in the standard library), to generate
filesystem paths.
> The only modes I can imagine for such a thing (a generated but unused
> filename) are:
>
> checking that the name is syntactly valid, for whatever constrains
> you may have (but if you're calling an opaque mktemp-like function, is
> this feasible or remediable?)
Almost. I want the filesystem paths to be valid because the system under
test expects them, it may perform its own validation, and I have no good
reason to complicate the unit test by possibly supplying an invalid path
when that's not relevant to the test case.
> generating test paths without using a real filesystem as a reference,
> but then you can't even use mktemp
I hadn't realised the filesystem was accessed by ‘tempfile.mktemp’, and
I apologise for the complication that entails.
I would prefer to access some standard public documented non-deprecated
function that internally uses ‘tempfile._get_candidate_names’ and
returns a new path each time.
> I think "the standard library clearly has this useful functionality
> implemented, but simultaneously warns strongly against its use" pretty
> much precludes this.
I hope to get that addressed with https://bugs.python.org/issue26362>.
--
\ “Timid men prefer the calm of despotism to the boisterous sea |
`\of liberty.” —Thomas Jefferson |
Re: repr( open('/etc/motd', 'rt').read() )
On Tuesday 16 February 2016 00:05, Veek. M wrote:
> When I do at the interpreter prompt,
> repr( open('/etc/motd', 'rt').read() )
Opening and reading MOTD is a needless distraction from your actual
question. This demonstrates the issue most simply:
# at the interactive interpreter
py> s = "text\n"
py> s # evaluate the string s
'text\n'
py> repr(s)
"'text\\n'"
Evaluating the string `s` in the REPL (Read-Eval-Print Loop) displays the
repr() of the string. Contrast that to using print directly:
py> print s # note the blank line
text
py> print repr(s)
'text\n'
So when you call print on a string, the string is printed in the most
accurate way possible. When you call repr() on a string, it returns a new
string containing the Python representation of a string.
So the string:
text
plus newline (but without the indent) has a representation in Python of:
'text\n'
so repr("text\n") == "'text\n'"
If you call repr() twice, you get this string:
py> print repr(repr("text\n"))
"'text\\n'"
That should look familiar:
py> repr(s)
"'text\\n'"
So when you evaluate a string on its own, the REPL prints the repr() of the
string. So if you evaluate repr(s), you see repr(repr(s)) printed.
> With # 2 # read returns a string that the interpreter displays by
> calling __str__ via print so newlines are converted to \n.
No, that's incorrect. The REPL uses the repr() of the string. We can test
this with a class that makes the difference between __str__ and __repr__
more obvious:
py> class C(object):
... def __repr__(self):
... return "the repr"
... def __str__(self):
... return "the str"
...
py> c = C()
py> print(c)
the str
py> c
the repr
--
Steve
--
https://mail.python.org/mailman/listinfo/python-list
