[Tutor] python at midnight
While processing weather data from the http://www.knmi.nl/ and building a small domotica system I ran into the following 'shortcomming' from the python doc: "class datetime.datetime(year, month, day[, hour[, minute[, second[, microsecond[, tzinfo]) The year, month and day arguments are required. tzinfo may be None, or an instance of a tzinfo subclass. The remaining arguments may be ints or longs, in the following ranges: [...] 0 <= hour < 24 [...] If an argument outside those ranges is given, ValueError is raised." from http://en.wikipedia.org/wiki/ISO_8601 : "ISO 8601 uses the 24-hour clock system. The basic format is [hh][mm][ss] and the extended format is [hh]:[mm]:[ss]. * [hh] refers to a zero-padded hour between 00 and 24 (where 24 is only used to notate midnight at the end of a calendar day). [...] Midnight is a special case and can be referred to as both "00:00" and "24:00". The notation "00:00" is used at the beginning of a calendar day and is the more frequently used. At the end of a day use "24:00". Note that "2007-04-05T24:00" is the same instant as "2007-04-06T00:00" (see Combined date and time representations below)." The use of 24:00 is very comfortable when using hourly datasets, the first set of a day is saved under 1:00, the fifth (4:00 to 5:00) under 5:00 and the last (23:00 - 24:00) under 24:00. No need to suddenly use 23:59:59 or 0:00 the next day. Actually in another part of Python SQLlite's date and time functions accept and output the 24:00. Adding some Python to an existing database made me aware of the problem. Questions, Is there a date time library that accepts the 24:00? mxDateTime doesn't. Is there a way to set the limit 'from the outside' (subclassing???) or a simple way around? How to get this functionality added to Python? i. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] python at midnight
On Sun, Nov 14, 2010 at 11:48 PM, Steven D'Aprano wrote: > > Write a wrapper function: > Thanks Steve, with this in hand I think I can solve the main problems for now i. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] python at midnight
On Sun, Nov 14, 2010 at 11:48 PM, Steven D'Aprano wrote: >> How to get this functionality added to Python? > > Make a feature request on the Python bug tracker: > > http://bugs.python.org/ done: http://bugs.python.org/issue10427 i. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
[Tutor] how to get the return value?
To make a time lapse video I've been playing with the sched module. There is one problem I run into, in the code below, how do I get the returned value t from printtime into main? import time from sched import scheduler class time_lapse(scheduler): def time_lapse(self, start_time, stop_time, interval, priority, action, argument): def lapse(): action(*argument) i=self.enter(interval, priority, lapse, ()) if stop_time: if stop_timehttp://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] how to get the return value?
On 2006/3/04, Anna Ravenscroft wrote: >>[...] > >main(printtime(strf=None)) > >should do it. Alternately: > >tt = printtime(strf=None) >main(tt) > Anna, that results in an syntax error / invalid syntax Ingo ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] how to get the return value?
in news:[EMAIL PROTECTED] Kent Johnson wrote: >>>>[...] >>> >>>main(printtime(strf=None)) >>> >>>[...] >> >> Anna, >> >> that results in an syntax error / invalid syntax > > It's very helpful to show the actual error and traceback. > Sorry, here it is: File "D:\Ingo\PyScript\VIDEOC~1.9\_ij\_time_lapse.py", line 23 def main(printtime(strf=None)): ^ SyntaxError: invalid syntax Ingo ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] how to get the return value?
On 2006/3/05, Kent Johnson wrote: >> >>>>[...] >>> >>>main(printtime(strf=None)) >>> >>>should do it. Alternately: >>> >>>tt = printtime(strf=None) >>>main(tt) >> >> that results in an syntax error / invalid syntax > >It's very helpful to show the actual error and traceback. Sorry, here it is File "D:\Ingo\PyScript\VIDEOC~1.9\_ij\_time_lapse.py", line 23 def main(printtime(strf=None)): ^ SyntaxError: invalid syntax Ingo ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] how to get the return value?
On 2006/3/06, Alan Gauld wrote: >> for every workingday of a week: >> [...] >> week >> ... > >To be honest I'd do this with a series of separate scripts triggered by the >OS - either cron on Unix or at in Windows... To be honest, I had my share of troubles with AT, so I try to stay away from it. No unix box here, so ... mmm, there seems to be a pycron maybe that's the tool I'm looking for. >Simply save the frames as individually named files, probably using the >time... Getting the frames and naming them is no big problem, for windows there's the great videocapture module http://videocapture.sourceforge.net/ >This follows my motto of keeping things as simple as possible! :-) I try, I'm trying to try God knows I try Thanks Alan, Ingo ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Python related mags
in news:[EMAIL PROTECTED] Evans Anyokwu wrote: > But then again, who is going to initiate the first move?? > Here's a name: Monthly Python ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
[Tutor] turn a path into nested list
Dear all, a probably simple problem I can't get my head around at the moment (old age?), some input would be appreciated. I have a path >>> path = >>> 'Audio/site-packages/pygame/examples/macosx/aliens_app_example/English.lproj' I'd like to turn taht into a list with a certain structure and order: [['Audio', 'Audio'], ['site-packages', ''Audio/site-packages'], ['pygame', 'Audio/site-packages/pygame'], ['examples', 'Audio/site-packages/pygame/examples'], ['macosx', ''Audio/site-packages/pygame/examples/macosx'], . . ] How to approach this? TIA, ingo ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] turn a path into nested list
On Thu, Jan 13, 2011 at 1:51 PM, Christian Witts wrote: > First split your path into segments, then iterate over those segments and > build your new list (item one being the current segment, and part two a > joined string of previous segments). > thanks Christian, path = 'Audio/site-packages/pygame/examples/macosx/aliens_app_example/English.lproj' path=urllib.url2pathname(path) print '\n',path pathlist=[] dir='' for item in path.split("\\"): dir=os.path.join(dir,item) a=[item,dir] pathlist.append(a) print pathlist ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] turn a path into nested list
On Thu, Jan 13, 2011 at 6:30 PM, Alan Gauld wrote: > Its not clear what exactly the sort criteria is, sorry Alan, I meant that I want to keep the order as is. > however have you looked > at the os.walk() function for traversing directory trees? It may be all > you need. Yes, but I'm not confident with it (the more reason to use it you will say :) ). Below the script I'm working on, rather unstable still. It will be put to work in Cherrypy so I browse through my media directories and send playlists to my media player(s). Very preliminary still. ingo --- %< --- %< --- import os, string, urllib topdirs={ 'Audio': 'c:', 'Video': 'd:', 'Photo': 'e:' } path = 'c:\\Python27' path = os.path.abspath(path) print path topdir = 'Audio' def mkPATH(path, topdirs=topdirs): path=urllib.url2pathname(path) path=string.split(path,'\\',1) return os.path.join(topdirs[path[0]],path[1]) def mkURL(dir): return urllib.pathname2url(dir) def mk_alias(dir, old=topdirs[topdir], new=topdir): return string.replace(dir,old,new) def mk_aliasURL(dir): return mkURL(mk_alias(dir)) def mk_pathlist(path): pathlist=[] dir='' for item in path.split("\\"): dir=os.path.join(dir,item) dir=mkURL(dir) a=[item,dir] pathlist.append(a) return pathlist aliaspath = mk_alias(path) directories = mk_pathlist(aliaspath) files = [] content = os.listdir(path) content.sort() for item in content: fullpath = os.path.join(path, item) if os.path.isdir(fullpath): directories.append([item, mk_aliasURL(fullpath)]) else: files.append([item, mk_aliasURL(fullpath)]) print '\n Path \n', path print '\n AliasPath \n', aliaspath print '\n Dirs \n', directories print '\n Files \n', files ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] turn a path into nested list
On Fri, Jan 14, 2011 at 1:21 AM, Alan Gauld wrote: >>>> for t in os.walk('Root'): > > ... print t > ... > And the result is:('Root', ['D1', 'D2', 'D3'], ['FA.txt', 'FB.txt']) > ('Root/D1', ['D1-1'], ['FC.txt']) As I only need the first result presented I didn't favor os.walk, yet tried it anyway. Below both versions: 1. aliaspath = mk_alias(path) directories = mk_pathlist(aliaspath) files = [] content = os.listdir(path) content.sort() for item in content: fullpath = os.path.join(path, item) if os.path.isdir(fullpath): directories.append((item, mk_aliasURL(fullpath))) else: files.append((item, mk_aliasURL(fullpath))) 2. content = list(os.walk(path).next()) content[0] = mk_alias(content[0]) directories = mk_pathlist(content[0]) directories.extend([(item, mkURL(os.path.join(content[0], item))) for item in content[1]]) files = [(item, mkURL(os.path.join(content[0], item))) for item in content[2]] And both with the same output: Path c:\Python25\Lib\site-packages\pygame\examples\macosx\aliens_app_example AliasPath Audio\site-packages\pygame\examples\macosx\aliens_app_example Dirs [('Audio', 'Audio'), ('site-packages', 'Audio/site-packages'), ('pygame', 'Audio/site-packages/pygame'), ('examples', 'Audio/site-packages/pygame/examples'), ('macosx', 'Audio/site-packages/pygame/examples/macosx'), ('aliens_app_example', 'Audio/site-packages/pygame/examples/macosx/aliens_app_example'), ('English.lproj', 'Audio/site-packages/pygame/examples/macosx/aliens_app_example/English.lproj')] Files [('aliens.py', 'Audio/site-packages/pygame/examples/macosx/aliens_app_example/aliens.py'), ('aliens.pyc', 'Audio/site-packages/pygame/examples/macosx/aliens_app_example/aliens.pyc'), ('aliens.pyo', 'Audio/site-packages/pygame/examples/macosx/aliens_app_example/aliens.pyo'), ('README.txt', 'Audio/site-packages/pygame/examples/macosx/aliens_app_example/README.txt'), ('setup.py', 'Audio/site-packages/pygame/examples/macosx/aliens_app_example/setup.py'), ('setup.pyc', 'Audio/site-packages/pygame/examples/macosx/aliens_app_example/setup.pyc'), ('setup.pyo', 'Audio/site-packages/pygame/examples/macosx/aliens_app_example/setup.pyo')] ingo ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
[Tutor] automatic setting of class property when another one changes
Rather stuck with this one, I'd like to automatically (re)set the propery "relaystate" when one of the others (Tm, Tset, Th) is set, regardless of wether their value has changed. Ingo My code so far: from collections import namedtuple import logging logger = logging.getLogger() logger.setLevel(logging.DEBUG) stream_handler = logging.StreamHandler() stream_handler.setLevel(logging.DEBUG) formatter = logging.Formatter( '[%(levelname)-8s] %(asctime)s (%(name)-8s) - %(message)s', '%Y-%m-%d %H:%M:%S', ) stream_handler.setFormatter(formatter) logger.addHandler(stream_handler) class Thermostat(): def __init__(self, Tm=20, Tset=20, Th=0.3): logger.debug("__INIT__") self.Tm = Tm self.Tset = Tset self.Th = Th self.relaystate = None @property def Tm(self): logger.debug("GETTER Tm") return self._Tm @Tm.setter def Tm(self, value): logger.debug("SETTER Tm") self._Tm = value @property def Tset(self): logger.debug("GETTER Tset") return self._Tset @Tset.setter def Tset(self, value): logger.debug("SETTER Tset") self._Tset = value @property def Th(self): logger.debug("GETTER Th") return self._Th @Th.setter def Th(self, value): logger.debug("SETTER Th") self._Th = value @property def relaystate(self): logger.debug("GETTER relaystate") return self._relaystate @relaystate.setter def relaystate(self, value): logger.debug("SETTER relaystate") #on init while setting Tm, Tset and Th are not known #so relaystate can not be calculated try: lower = self.Tset-self.Th upper = self.Tset+self.Th except AttributeError as e: logger.debug("SETTER relaystate : %s", e) self._relaystate = None rlstate = namedtuple('relaystate', ['heat', 'cool']) if self.Tm < lower: value = rlstate(1,0) elif self.Tm > upper: value = rlstate(0,1) elif self.Tm > lower and self.Tm < upper: value = rlstate(0,0) self._relaystate = value if __name__ == "__main__": TS1 = Thermostat() TS1.Tset = 44 print("heat : ",TS1.relaystate.heat,"cool : ",TS1.relaystate.cool) ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
[Tutor] type hint woes regex
In a first attempt to add types I ran into trouble. First a code snippet ( full code at https://gist.github.com/ingoogni/460676b11e5a04ed3a3ac93ae0e1fddd ) _matchline: Pattern[str] = re.compile(r"(?P^\s*$)|(?P[\t]*)" "((?P[A-Z]+):)?([ \t]*)(?P.*)") def proc_matchlns(momfile: TextIO) -> Iterator[Dict[str, Union[str, int]]]: for i, ln in enumerate(momfile, 1): match: Match[str] = _matchline.match(ln) #error 1 m: Dict[str, str] = match.groupdict()#error 2 m['ln'] = ln.rstrip() m['ln#'] = i #error 3 yield m #error 4 Depending on how I add types mypy nags on various aspects: error 1: match seems to want type Optional[Match[str]], but why? The regex always has a result (I hope). error 2: when I give in to the above, the result is that it errors on not being able to de a groupdict on a null type. error 3: groupdict results in a type dict[str, str], but I want to add an int. Can I change the type? (In this specific case could change i to str(i) as it has no influence on the program) error 4: result of 3 and output type, type of m and specified output type don't match. Ingo ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] tweeting from python
On 26-5-2019 16:59, nathan tech wrote: > Hi there, > > I was wondering if anyone had some tips that might direct me to answers > to this problem: > > I have a program which one downloads to your computer, and you install. > > Then, I want to add into this program a share button. > > This share button would send a tweet to your twitter saying, "I am > watching episode x of podcast y using program z, check it out at > programurl." > > > Now, I think I am looking at tweepy for this, maybe oauth2, but all the > tutorials on google are for creating one use apps. > Nathan, you may want to give twython a look https://github.com/ryanmcgrath/twython it has Oauth etc. There are tutorials for writing twitter bots with it that you could adapt to your purpose, for example: https://geekswipe.net/technology/computing/code-python-twitter-bot-in-ten-minutes/ Ingo ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] File extension against File content
Many file formats have "magic bytes" that you can use for that purpose. https://en.wikipedia.org/wiki/List_of_file_signatures Ingo On 31-5-2019 12:03, Sunil Tech wrote: > Hi Tutor, > > Is there any way that I can actually programmatically compare the file > extension with its content? > > This is because we can manually save the file in one extension and later > rename the file extension to some other. > > Thanks > - Sunil. G > ___ > Tutor maillist - Tutor@python.org > To unsubscribe or change subscription options: > https://mail.python.org/mailman/listinfo/tutor > ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
[Tutor] path
A user has to type a path in the commandline on Win 10, so just a string. This has to become a path / directory in the file system. Later in the process subdirectories and files are 'appended' by the script. In these files there are also paths, derived from the input path and they have to use forward slashes. The tool that uses these files requires that. # create structure: # /---fullpath <-- input() cmd # | # /--- data # /--- db # |+--- basename.db3<-- pathlib.Path(...).touch() # |+--- basename.read <-- forward slashes inside # /--- ddl # /--- doc # /--- sql # +--- basename.sublime-project <-- forward slashes inside (json) # +--- _FOSSIL_ And here the misery begins... A short excerpt: Python 3.7.0 (v3.7.0:1bf9cc5093, Jun 27 2018, 04:59:51) [MSC v.1914 64 bit (AMD64)] on win32 Type "copyright", "credits" or "license()" for more information. >>> inp = "c:\test\drive\this" >>> import pathlib >>> p=pathlib.PurePath(inp) >>> p PureWindowsPath('c:\test/drive\this') >>> print(pathlib.PurePosixPath(p)) c:/ est/drive his >>> inp 'c:\test\\drive\this' >>> import os >>> print(os.path.normpath(inp)) c: est\drive his >>> print(pathlib.Path(inp)) c: est\drive his >>> how to go from a string to a path, how to append to a path (os.path.join or / with Path), how to turn it into 'posix' TIA, Ingo ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] path
On 29-6-2019 15:52, Mats Wichmann wrote: > Sigh... something dropped my raw string, so that was a really bad sample :( > > inp = r"c:\test\drive\this" > > > On Sat, Jun 29, 2019, at 07:44, Mats Wichmann wrote: >> >> For your example, when you define inp as a string, it needs to be a raw >> string because otherwise Python will interpret the backslash sequences. >> \t means tab, which is why the the results look mangled. >> >> inp = "c:\test\drive\this" >> import pathlib print('input') a=input() b=r"{}".format(a) #does this make sense to create a r'string'? wa = pathlib.PureWindowsPath(a) wb = pathlib.PureWindowsPath(b) pa = pathlib.PurePosixPath(a) pb = pathlib.PurePosixPath(b) ppa = pathlib.PurePosixPath(wa) ppb = pathlib.PurePosixPath(wb) input c:\test\this\path\ >>> print(a) c:\test\this\path\ >>> print(b) c:\test\this\path\ >>> print(wa) c:\test\this\path >>> print(wb) c:\test\this\path >>> print(pa) c:\test\this\path\ >>> print(pb) c:\test\this\path\ >>> print(ppa) c:\/test/this/path >>> print(ppb) c:\/test/this/path What I'm looking for is c:/test/this/path ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] path
On 29-6-2019 16:33, ingo wrote: > > What I'm looking for is c:/test/this/path After further testing, the other tools in the chain accept paths like c:\\test\\dir c:\/test/dir c:/test/dir anything except standard windows, the top two I can generate. Ingo ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] path
On 29-6-2019 15:42, Mats Wichmann wrote: > > Most people don't use pathlib, and that's kind of sad, since it tries to > mitigate the kinds of questions you just asked. Kudos for trying. In the end, it works, Ingo ---%<--%<--%<--- # set up some default directories and files # for starting a new project with SQLite # using Sublime + SQLTools. # # /---fullpath # | # /--- data # /--- db # |+--- basename.db3 # |+--- basename.read # /--- ddl # /--- doc # /--- sql # +--- basename.sublime-project # +--- _FOSSIL_ import pathlib import sys # the last bit of the full path is used as the name for the database # c:\newdatabase\test will create the databse # c:\newdatabase\test\db\test.db3 print('enter full path for new db:') fp = pathlib.Path(input()) fn = fp.name # = os.path.basename() dirs = {} for sub in ['', 'data', 'db', 'ddl', 'doc', 'sql']: dirs[sub] = fp / sub try: dirs[sub].mkdir(parents=False, exist_ok=False) except FileExistsError: print(f'Directory already exists: {dirs[sub]}') sys.exit(1) fdb = dirs['db'] / (fn+'.db3') fdb.touch() fr = dirs['db'] / (fn+'.read') fr.write_text(f""" -- template to build db from tables etc -- using dot commands PRAGMA foreign_keys = OFF; --DROP TABLE IF EXISTS sometable; BEGIN TRANSACTION; --.read {(dirs['ddl'] / 'someddl.ddl').as_posix()} COMMIT; PRAGMA temp_store = 2; PRAGMA foreign_keys = ON; PRAGMA journal_mode = WAL; BEGIN TRANSACTION; --.read {(dirs['sql'] / 'somequery.sql').as_posix()} COMMIT; """ ) fsub = dirs[''] / (fn+'.sublime-project') fsub.write_text(f''' {{ "folders":[ {{ "path": "." }} ], "connections":{{ "Connection SQLite":{{ "database": "{fdb.as_posix()}", "encoding": "utf-8", "type": "sqlite" }} }}, "default": "Connection SQLite" }}''' ) # TODO set up fossil in the fp dir ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
[Tutor] instantiate and name a class from / with a string
With the available classes Root, Channel and SSE I build the following (in CherryPy): root = Root() root.channel = Channel() root.channel.weather = SSE('weather') root.channel.energy = SSE('energy') root.channel.mail = SSE('mail') root.channel.rss = SSE('rss') ... http://example.com/channel/weather/ will then become the emitter of the weather event stream. I'd like create the instances of SSE programmatically by pulling the string 'weather', 'energy' etc. from a database. Ingo ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] instantiate and name a class from / with a string
On 10-8-2019 05:57, Mats Wichmann wrote: > Please be more precise. We're not trying to be argumentative; you have > to help us know enough before we can help you. > Mats, Bob, sorry for not being clear. I have the string and want to create the instances. Bob's name = # get from data base setattr(root.channel, name, SSE(name)) is what I was looking for, time to read up on attributes, Thanks, Ingo ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
[Tutor] two way dictionary
for a little cherrypy app I'm working on I wrote the piece of code below. Goal is to obscure the first part of a directory path(s) the user is browsing. Replacing the first part of the path with an alias works fine, but for the application it has to work both ways. I know how to swap the keys and values of the dict but that's not elegant as it can be changed over time. Is there a dict that works "both ways"? Could a dict be constructed from the dict class that works both ways? Is writing code around a list of tuples more elegant? home = [('one' , "c:\\gnuwin32"), ('two' , "c:\\dell")] Did I miss the completely obvious way to deal with the problem? %<-%<-%<-%<-%<-%<- import os home={ 'one':"c:\\gnuwin32", 'two':"c:\\dell", } def fetch_alias(path, home_dict): path = os.path.normpath(path) pathlist = path.split(os.sep) if pathlist[0] in home_dict: pathlist[0] = home_dict[pathlist[0]] newpath = os.sep.join(pathlist) return os.path.normpath(newpath) else: print "fail" path='two\\truus\\tovert\\konijn' print fetch_alias(path, home) %<-%<-%<-%<-%<-%<- Ingo ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] two way dictionary
On Nov 28, 2007 8:16 PM, Kent Johnson <[EMAIL PROTECTED]> wrote: > ingo janssen wrote: > > Is there a dict that works "both ways"? > > another implementation here: > http://www.radlogic.com/releases/two_way_dict.py > Perfect, never thought to actually search for 'two way dict'. Thanks Kent. Ingo ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
[Tutor] text processing lines variable content
For parsing the out put of the Voro++ program and writing the data to a POV-Ray include file I created a bunch of functions. def pop_left_slice(inputlist, length): outputlist = inputlist[0:length] del inputlist[:length] return outputlist this is used by every function to chop of the required part of the input line. Two examples of the functions that proces a chopped of slice of the line and append the data to the approriate list. def f_vector(outlist): x,y,z = pop_left_slice(line,3) outlist.append(f"<{x},{y},{z}>,") def f_vector_array(outlist, length): rv = pop_left_slice(line, length) rv = [f'<{i[1:-1]}>' for i in rv] #i format is: '(1.234,2.345,3.456)' rv = ",".join(rv) outlist.append(f" //label: {lbl}\n array[{length}]"+"{\n "+rv+"\n }\n") Every line can contain up to 21 data chunks. Within one file each line contains the same amount of chunks, but it varies between files. The types of chunks vary and their position varies. I know beforehand how a line in a file is constructed. I'd like to adapt the order in that the functions are applied, but how? for i, line in enumerate(open("vorodat.vol",'r')): points = i+1 line = line.strip() line = line.split(" ") lbl = f_label(label) f_vector(point) f_value(radius) v=f_number(num_vertex) f_vector_array(rel_vertex,v) f_vector_array(glob_vertex,v) f_value_array(vertex_orders,v) f_value(max_radius) e=f_number(num_edge) f_value(edge_dist) ...etc I thought about putting the functions in a dict and then create a list with the proper order, but can't get it to work. A second question, all this works for small files with hundreds of lines, but some have 10. Then I can get at max 22 lists with 10 items. Not fun. I tried writing the data to a file "out of sequence", not fun either. What would be the way to do this? I thought about writing each data chunk to a proper temporary file instead of putting it in a list first. This would require at max 22 temp files and then a merge of the files into one. TIA, ingo ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] text processing lines variable content
On 06/02/2019 19:07, Mark Lawrence wrote: That's going to a lot of work slicing and dicing the input lists. Perhaps a chunked recipe like this https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.chunked would be better. The length of the text chunks varies from a single character to a list of ~30 3D vectors. I'd like to adapt the order in that the functions are applied, but how? I suspect that you're trying to over complicate things, what's wrong with a simple if/elif chain, a switch based on a dict or similar? You mean create a list with the order=[a,b,e,d...] if a in order: f_vector_array(a, 3) elseif b in order: f_value(max_radius) that would run the proper function, but not in the right order? for i, line in enumerate(open("vorodat.vol",'r')): points = i+1 enumerate takes a start argument so you shouldn't need the above line. points is needed later on in the program and I don't know beforehand how many lines I have. I thought about putting the functions in a dict and then create a list with the proper order, but can't get it to work. Please show us your code and exactly why it didn't work. def f_vector_array(outlist, length): rv = pop_left_slice(line, length) rv = [f'<{i[1:-1]}>' for i in rv] #i format is: '(1.234,2.345,3.456)' rv = ",".join(rv) outlist.append(f" //label: {lbl}\n array[{length}]"+"{\n "+rv+"\n }\n") functions={ 'a':f_number(num_vertex), 'b':f_vector_array(rel_vertex,v) } where rel_vertex is the list where to move the processed data to and v the amount of text to chop of the front of the line. v is not known when defining the dictionary. v comes from an other function v=f_number(num_vertex) that also should live in the dict. then loop order=[a,b,e,d...] for each line I'm not absolutely sure what you're saying here, but would something like the SortedList from http://www.grantjenks.com/docs/sortedcontainers/ help? Maybe this explains it better, assume the split input lines: line1=[a,b,c,d,e,f,...] line2=[a,b,c,d,e,f,...] line3=[a,b,c,d,e,f,...] ... line10=... all data on position a should go to list a a=[a1,a2,a3,...a_n] b=[b1,b2,b3,...b_n] c=[c1,c2,c3,...n_n] etc. this is what for example the function f_vector_array(a, 3) does. All these lists have to be written to a single file, each list contains 10 items. Instead of keeping it all in memory I could write a1 to a temp file A instead of putting it in a list first and b1 to a temp file B etc. in the next loop a2 to file A, b2 to file B etc. When all lines are processed combine the files A,B,C ... to a single file. Or is there a more practical way? Speed is not important. ingo ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] text processing lines variable content
= array[0]{ //label: 0 array[9]{ <-0.377877,0.147157,-0.914086>,<-0.382036,2.8913e-18,-0.924147>,<-0.869981,0,0.493086>,<-0.904528,-0.0477043,0.423738>,<0.75639,-5.72053e-15,0.654121>,<-0,-1,0>,<0.950875,4.0561e-18,-0.309575>,<-5.99268e-17,1,-1.44963e-16>,<-0.849681,0.21476,0.481581> } } #declare Neighbours = array[0]{ //label: 0 array[9]{9205,9105,3062,9946,5786,-3,1483,100,3262}, } #declare Volume = array[0]{11680.5} #declare RelCentroid = array[0]{<-2.00777,-44.9048,-0.428504>} #declare GlobalCentroid = array[0]{<1092.81,-44.8048,581.99>} ingo ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] text processing lines variable content
On 07/02/2019 09:29, Peter Otten wrote: Where will you get the order from? Peter, the order comes from the command line. I intend to call the python program with the same command line options as the Voro++ program. Make the python program call the Voro++ and process its output. one command line option contains a string for formatting the output. That is what I use for order. #all output formatting options order = "%i %q %r %w %p %P %o %m %g %E %s %e %F %a %A %f %t %l %n %v %c %C" order = re.findall("%[a-z]",order,re.M|re.I) for i, line in enumerate(open("vorodat.vol",'r')): points = i line = line.strip() line = line.split(" ") for action in order: if action == "%i": try: lbl = f_label(label) except NameError as e: lbl = f_number(label) label=[lbl] elif action == "%q": try: f_vector(point) except NameError as e: point = [f_vector(point)] elif action == "%r": try: f_value(radius) except NameError as e: radius=[f_value(radius)] etc. order is important as %w tells me how long %p, %P and %o will be. This varies per line. I'll look into what you wrote, thanks, ingo ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] text processing lines variable content
On 07/02/2019 09:58, ingo janssen wrote: On 07/02/2019 09:29, Peter Otten wrote: Where will you get the order from? Ahrg, that should have been: #all output formatting options order = "%i %q %r %w %p %P %o %m %g %E %s %e %F %a %A %f %t %l %n %v %c %C" order = re.findall("%[a-z]",order,re.M|re.I) for i, line in enumerate(open("vorodat.vol",'r')): points = i line = line.strip() line = line.split(" ") for action in order: if action == "%i": try: lbl = f_label(label) except NameError as e: label=[] lbl = f_number(label) elif action == "%q": try: f_vector(point) except NameError as e: point = [] f_vector(point) elif action == "%r": try: f_value(radius) except NameError as e: radius = [] f_value(radius) ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] text processing lines variable content
On 07/02/2019 10:40, Alan Gauld via Tutor wrote: Just saves a little typing is all. Sensei, be lazy, I will study current state of code is at https://gist.github.com/ingoogni/e99c561f23777e59a5aa6b4ef5fe37c8 ingo ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] text processing lines variable content
On 07/02/2019 11:08, Peter Otten wrote: Personally I would avoid the NameError and start with empty lists. If you manage to wrap all branches into functions with the same signature you can replace the sequence of tests with dictionary lookups. Just before I saw your post I put my current code up here: https://gist.github.com/ingoogni/e99c561f23777e59a5aa6b4ef5fe37c8 I will study yours, ingo ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] text processing lines variable content
On 07/02/2019 11:08, Peter Otten wrote: replace the sequence of tests with dictionary lookups updated the gist a few times, now I could pre calculate the slices to be taken per line, but will there be much gain compared to the copping from the left side of the list? ingo ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] text processing lines variable content
On 07/02/2019 18:06, Peter Otten wrote: Sorry, I don't understand the question. after a quick look not unlike what you propose but I have to investigate further, lengths of chunks are known or can be found (sketchy): order= [%i,%q,%r,%w,%p,%P,%o,%m,%g,%E,%s,%e,%F,%a,%A,%f,%t,%l,%n,%v,%c,%C] length=[ 1, 3, 1, 1,%w,%w,%w, 1, 1, 1, 1,%s, 1,%s,%a,%s,%s,%s,%s, 1, 3, 3] from there calculate the slices per line slices={"%i":(0,1), "%q":(1,4), "%r":(4:5)etc modify all functions to accept and deal with the slice tuple, then the action loop gets very simple: for points, line in enumerate(open("vorodat.txt.vol",'r'), 1): line = line.strip() line = line.split(" ") slices = calculate_slices(line) function[action](content[action],slices[action]) thanks for your time and insight, I'll try a few different ways ingo ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
[Tutor] properly propagate problems
One thing I often struggle with is how to deal with exceptions, especially when I have a chain of functions that use each others output and/or long running processes. As the answer will probably be "it depends" take for example this program flow: open a file and read into BytesIO buffer get a FTP connection from pool send buffer to plantuml.jar in memory FTP server render file to image get image from FTP server push the image onto CherryPy bus push (SSE) the image to web browser def read_file(input_file): try: with open(input_file, 'rb') as f: buffer = io.BytesIO(f.read()) except FileNotFoundError as e: print(e) return buffer assume the file is not found, I cannot just kill the whole process. Catching the exception is one thing, but how to deal with it properly, I have to inform the client somehow what went wrong. In this case I could push the error message into the returned buffer and just go from there and the image will show the message. I could also bypass the whole process somehow and push the error message directly on the CherryPy bus. What is wisdom, are there some general rules to follow in such cases ingo ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] properly propagate problems
On 24/03/2019 00:03, Cameron Simpson wrote: Short takeaway: decide what's mechanism and what is policy, and try to put policy further out in higher level code. That, Cameron, was a very insightful answer and an eye opener, as I try to 'fix things' as early as possible. It also answered the question that I thought of right after posting, what if it is a library and knows nothing about its use? I'll certainty have a look at Pfx. Thank you Cameron, ingo ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor