[Tutor] Need Help Modifying a wxPython GUI (scrolling display and logging)
Hello, I am using an open source wxPython GUI that I like very very much. I have ideas about some modifications I need but I cannot be bothering the author too much so I must learn some very specific things about Python in order to make the modification myself. First, I need some help understanding the code behind this GUI. From reading the comments I understand that the part of the program written in C++ sends a Python pickled dictionary to a msg_queue and when Python decodes the pickle the TextCtrl fields (in the wxPython GUI I am using) receive/display the appropriate values. And as new messages are received by Python the GUI fields are cleared to display the new values. For starters I would like the values to be displayed on the GUI in some sort of a scrolling panel as they are cleared from the TextCtrl fields so the user can watch recent activity. I dont know what the best way to do this would be; the wxpython.scrolledPanel widget? I am unclear if this can be put on the same GUI pane as the TextCtrl fields are and I am unclear about if I can take the values from the TextCtrl fields or have to use the pickle or what? I dont see any variables declared (like in Visual Basic) so its not like I can just make a list and a textbox and print it. More importantly I need to save the values from the TextCtrl fields, preferable in a CSV file, for later inspection. From looking at the Logging HOWTO and the Logging Cookbook I see there are alot of loggers available, or ways to log, but they all look like they are orientated towards exception handling and debugging so I am unsure what is the best way to go about this; maybe wxLogTextCtrl ? Utimately I need to log the values from the TextCtrl fields in a row of comma separated values adding a time/date stamp as one of the values. I need this log so the data can easily be worked on in excel or SAS. I need the time/date stamp for time series analysis. I attached the code behind the wxPythoin GUI I am using. Any help will be much appreciated. Thanks in advance -- Matt D #!/usr/bin/env python # -*- coding: utf-8 -*- # # op25_traffic_panel.py # # Copyright 2013 Balint Seeber # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, # MA 02110-1301, USA. # # import wx import cPickle as pickle import gnuradio.gr.gr_threading as _threading wxDATA_EVENT = wx.NewEventType() def EVT_DATA_EVENT(win, func): win.Connect(-1, -1, wxDATA_EVENT, func) class DataEvent(wx.PyEvent): def __init__(self, data): wx.PyEvent.__init__(self) self.SetEventType (wxDATA_EVENT) self.data = data def Clone (self): self.__class__ (self.GetId()) class traffic_watcher_thread(_threading.Thread): def __init__(self, rcvd_pktq, event_receiver): _threading.Thread.__init__(self) self.setDaemon(1) self.rcvd_pktq = rcvd_pktq self.event_receiver = event_receiver self.keep_running = True self.start() def stop(self): self.keep_running = False def run(self): while self.keep_running: msg = self.rcvd_pktq.delete_head() de = DataEvent (msg) wx.PostEvent (self.event_receiver, de) del de # A snapshot of important fields in current traffic # class TrafficPane(wx.Panel): # Initializer # def __init__(self, parent, msgq): wx.Panel.__init__(self, parent) self.msgq = msgq sizer = wx.GridBagSizer(hgap=10, vgap=10) self.fields = {} label = wx.StaticText(self, -1, "DUID:") sizer.Add(label, pos=(1,1)) field = wx.TextCtrl(self, -1, "", size=(144, -1), style=wx.TE_READONLY) sizer.Add(field, pos=(1,2)) self.fields["duid"] = field; label = wx.StaticText(self, -1, "NAC:") sizer.Add(label, pos=(2,1)) field = wx.TextCtrl(self, -1, "", size=(144, -1), style=wx.TE_READONLY) sizer.Add(field, pos=(2,2)) self.fields["nac"] = field; label = wx.StaticText(self, -1, "Source:") sizer.Add(label, pos=(3,1)) field = wx.TextCtrl(self, -1, "", size=(144, -1), style=wx.TE_READONLY) sizer.Add(field, pos=(3,2)) self.fields["source"] = field; label = wx.StaticText(self, -1, "Destination:") sizer.Add(label, pos=(4,1)) field
Re: [Tutor] Need Help Modifying a wxPython GUI (scrolling display and logging)
On 06/10/2013 12:23 PM, Prasad, Ramit wrote: > Matt D wrote: >> Ramit Prasad wrote: >>>>> Scrolled panel is just a graphical container that allows for scrolling >>>>> inside, >>>>> but it is the window that scrolls not widgets inside it. This of it like >>>>> a webpage that scrolls. If you use web email the text widget in the >>>>> email needs to scroll so you can see your full email context and not >>>>> just scroll the page. >>>>> >>>>> You will probably need to create a TextCtrl with the appropriate style >>>>> and append your new data. I have given an example below that should >>>>> automatically scroll with your new data. >>>>> >>>>> #in __init__ >>>>> self.scrolling_widget = wx.TextCtrl( self, wx.ID_ANY, '', size=(-1, 275), >>>> style=wx.TE_AUTO_SCROLL|wx.TE_READONLY|wx.TE_PROCESS_ENTER|wx.TE_WORDWRAP|wx.TE_MULTILINE >>>> ) >>>>> >> Hey, >> I added this the above 3 lines of code to my file and ran it. the box >> shows up on the far left, mostly of the pane, to the left of the current >> text feilds. I am having trouble positioning this textbox under where >> the current text fields are. >> I am not sure but maybe this sets up the grid on the pane: >> sizer = wx.GridBagSizer(hgap=10, vgap=10) >> self.fields = {} >> all the current TextCtrl fields are positioned at (1,1) through (5,5). >> I tried adding: >> sizer.Add(field, pos=(1,6)) >> but it did not move the box to the position? > > Just to make sure, you did call it field and not self.scrolling_widget > (which was in my example)? > > Odd that they don't start at (0,0) when adding to the bag. This is > more a wxpython question and their mailing list might prove more > useful. I could figure it out, but I cannot run the app. Trial > and error here will probably help you the most. You can also > try looking at a different sizer (like BoxSizer and GridSizer). > > Personally, I think BoxSizer is the most intuitive as it matches > my thought process. You just set an orientation (vertical/horizontal) > and add widgets to it. To get something in the other direction, you > create another box sizer with the opposite orientation, add widgets > to that sizer, and then add the new sizer to the original sizer. Not > nearly as pretty as grid/grid bag though (by default), but super simple. > > BoxSizer API > http://wxpython.org/docs/api/wx.BoxSizer-class.html > Some other sizers are listed here: > http://wxpython.org/docs/api/wx.Sizer-class.html > >> >> THanks, >> Matt > > Hey, if i put: self.logfile = open('logfile.csv', 'w') in the .py file, within the 'class TrafficPane', then shouldn't logfile.csv be written to the directory the .py file is in? because its not there after running the program? Where should i look for it? Thanks Matt ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Need Help Modifying a wxPython GUI (scrolling display and logging)
On 06/12/2013 05:59 PM, Dave Angel wrote: > On 06/12/2013 05:32 PM, Matt D wrote: >> On 06/10/2013 12:23 PM, Prasad, Ramit wrote: >>> Matt D wrote: >>>> Ramit Prasad wrote: >>>>>>> Scrolled panel is just a graphical container that allows for >>>>>>> scrolling inside, >>>>>>> but it is the window that scrolls not widgets inside it. This of >>>>>>> it like >>>>>>> a webpage that scrolls. If you use web email the text widget in the >>>>>>> email needs to scroll so you can see your full email context and not >>>>>>> just scroll the page. >>>>>>> >>>>>>> You will probably need to create a TextCtrl with the appropriate >>>>>>> style >>>>>>> and append your new data. I have given an example below that should >>>>>>> automatically scroll with your new data. >>>>>>> >>>>>>> #in __init__ >>>>>>> self.scrolling_widget = wx.TextCtrl( self, wx.ID_ANY, '', >>>>>>> size=(-1, 275), >>>>>> style=wx.TE_AUTO_SCROLL|wx.TE_READONLY|wx.TE_PROCESS_ENTER|wx.TE_WORDWRAP|wx.TE_MULTILINE >>>>>> ) >>>>>>> >>>> Hey, >>>> I added this the above 3 lines of code to my file and ran it. the box >>>> shows up on the far left, mostly of the pane, to the left of the >>>> current >>>> text feilds. I am having trouble positioning this textbox under where >>>> the current text fields are. >>>> I am not sure but maybe this sets up the grid on the pane: >>>> sizer = wx.GridBagSizer(hgap=10, vgap=10) >>>> self.fields = {} >>>> all the current TextCtrl fields are positioned at (1,1) through (5,5). >>>> I tried adding: >>>> sizer.Add(field, pos=(1,6)) >>>> but it did not move the box to the position? >>> >>> Just to make sure, you did call it field and not self.scrolling_widget >>> (which was in my example)? >>> >>> Odd that they don't start at (0,0) when adding to the bag. This is >>> more a wxpython question and their mailing list might prove more >>> useful. I could figure it out, but I cannot run the app. Trial >>> and error here will probably help you the most. You can also >>> try looking at a different sizer (like BoxSizer and GridSizer). >>> >>> Personally, I think BoxSizer is the most intuitive as it matches >>> my thought process. You just set an orientation (vertical/horizontal) >>> and add widgets to it. To get something in the other direction, you >>> create another box sizer with the opposite orientation, add widgets >>> to that sizer, and then add the new sizer to the original sizer. Not >>> nearly as pretty as grid/grid bag though (by default), but super simple. >>> >>> BoxSizer API >>> http://wxpython.org/docs/api/wx.BoxSizer-class.html >>> Some other sizers are listed here: >>> http://wxpython.org/docs/api/wx.Sizer-class.html >>> >>>> >>>> THanks, >>>> Matt >>> >>> >> Hey, >> if i put: >> >> self.logfile = open('logfile.csv', 'w') >> >> in the .py file, within the 'class TrafficPane', then shouldn't >> logfile.csv be written to the directory the .py file is in? because its >> not there after running the program? Where should i look for it? >> Thanks >> Matt >> > > It should put it in the current directory. That *may* be the directory > the script is in, but not necessarily. It's easy to run something like: > > python somdir/myscript.py > > in which case the file would be in the parent directory to myscript.py > > Note that in some environments, the current directory is invisibly set > to some convenient place. For example, when right-clicking on a script > in Windows Explorer, they make the bald assumption that you want to set > the current directory the same as the location of the script. That's > about the worse place for it, but nevermind. > > > Yes, that was my assumption (even thought I am using linux); and right again, it is a very inconvenient place for it to be. however in the interest of speed of testing i figured i would make sure the log was logging the way i want it to and then try to find some sort of wx wigit to let the user of the gui name/save to desired location. meanwhile . . . so you think it saved somewhere right? ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Need Help Modifying a wxPython GUI (scrolling display and logging)
> There are other ways a script might change the current directory. For > example, some naive scripts use os.chdir() > > But how is it you don't know what the current directory was when the > code ran? A simply pwd can tell you, if your prompt doesn't already > reveal it. > > hey i found the logfile. just took a few minutes of looking round. the file is logged all out of order so i have some work to do on that formatting issue. if you have a sec can you take a look at my code please? def update(self, field_values): # logger code--- # first write the CURRENT date/time self.logfile.write('%s,'%(str(strftime("%Y-%m-%d %H:%M:%S", gmtime() # loop through each of the TextCtrl objects for k,v in self.fields.items(): # get the value of the current TextCtrl field f = field_values.get(k, None) if f: #output the value with trailing comma self.logfile.write('%s,'%(str(f))) self.logfile.write('\n') #end logger code #if the field 'duid' == 'hdu', then clear all the fields if field_values['duid'] == 'hdu': self.clear() #loop through all TextCtrl fields storing the key/value pairs in k, v for k,v in self.fields.items(): # get the pickle value for this text control f = field_values.get(k, None) # if the value is empty then set the new value if f: v.SetValue(f) When i open the .csv file the fields are all out of order. what i want is have them all in one row beginning with the date/time. and idea? Thanks! ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
[Tutor] Fwd: Re: Need Help Modifying a wxPython GUI (scrolling display and logging)
Original Message Subject: Re: [Tutor] Need Help Modifying a wxPython GUI (scrolling display and logging) Date: Thu, 13 Jun 2013 00:17:44 -0400 From: Matt D To: Dave Angel On 06/12/2013 09:44 PM, Dave Angel wrote: > On 06/12/2013 09:23 PM, Matt D wrote: >> >>> There are other ways a script might change the current directory. For >>> example, some naive scripts use os.chdir() >>> >>> But how is it you don't know what the current directory was when the >>> code ran? A simply pwd can tell you, if your prompt doesn't already >>> reveal it. >>> >>> >> hey i found the logfile. just took a few minutes of looking round. the >> file is logged all out of order > > Do you have more than one thread? Perhaps you have a race condition. > >> so i have some work to do on that >> formatting issue. if you have a sec can you take a look at my code >> please? >> >> def update(self, field_values): > >> >> # logger code--- >> # first write the CURRENT date/time >> self.logfile.write('%s,'%(str(strftime("%Y-%m-%d %H:%M:%S", >> gmtime() > > The return value of strftime is already a str, so why do you call str() > on it? > >> # loop through each of the TextCtrl objects >> for k,v in self.fields.items(): > > items() returns an unordered list; what order did you actually want? > >> # get the value of the current TextCtrl field >> f = field_values.get(k, None) >> if f: >> #output the value with trailing comma >> self.logfile.write('%s,'%(str(f))) >> self.logfile.write('\n') > > That looks like a newline, not a comma > >> #end logger code >> >> #if the field 'duid' == 'hdu', then clear all the fields >> if field_values['duid'] == 'hdu': >> self.clear() >> #loop through all TextCtrl fields storing the key/value pairs >> in k, v >> for k,v in self.fields.items(): > > Same ordering problem here. If you have a specific order in mind, > you'll need to preserve it in a list, not in a dict. > >> # get the pickle value for this text control >> f = field_values.get(k, None) >> # if the value is empty then set the new value >> if f: >> v.SetValue(f) >> >> >> When i open the .csv file the fields are all out of order. what i want >> is have them all in one row beginning with the date/time. and idea? >> Thanks! >> >> > > A dictionary is unsorted, so those two are probably your problem. As I > mentioned above, you can't count on the items() order. > > Of course, self.items might not really be a dict. This fragment doesn't > prove that one way or another. > > yes the .py file has TextCtrl fields that get there values from a pickled dictionary. Another peice of the code watches a thread for the pickle. this is why i didnt use a list. I have been unable to find a nice way to just make a list with the items i need. would be nice to have that simplicity. What you said is true, the the list is unordered. More importantly the new line comes in at the wrong point. I want all the values in a row starting with time. from there i will look for a way to remove some unwanted items and ordering the others. I attached the .py file for you to see the whole thing hoping this is not too presumptuous. Thanks. #!/usr/bin/env python # -*- coding: utf-8 -*- # # op25_traffic_panel.py # # Copyright 2013 Balint Seeber # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, # MA 02110-1301, USA. # # # the import statements; very similar to #include in c++ # this is the stuff for getting the current time from time import gmtime, strftime import wx import cPickle as pickle import g
Re: [Tutor] Need Help Modifying a wxPython GUI (scrolling display and logging)
On 06/12/2013 09:54 PM, Dave Angel wrote: > On 06/12/2013 09:14 PM, Matt D wrote: >> On 06/12/2013 09:02 PM, Dave Angel wrote: >>> On 06/12/2013 08:46 PM, Matt D wrote: >>>> On 06/12/2013 05:59 PM, Dave Angel wrote: >>>>> On 06/12/2013 05:32 PM, Matt D wrote: >>> >>> >>> >>>>>> Hey, >>>>>> if i put: >>>>>> >>>>>> self.logfile = open('logfile.csv', 'w') >>>>>> >>>>>> in the .py file, within the 'class TrafficPane', then shouldn't >>>>>> logfile.csv be written to the directory the .py file is in? because >>>>>> its >>>>>> not there after running the program? Where should i look for it? >>>>>> Thanks >>>>>> Matt >>>>>> >>>>> >>>>> It should put it in the current directory. That *may* be the >>>>> directory >>>>> the script is in, but not necessarily. It's easy to run something >>>>> like: >>>>> >>>>> python somdir/myscript.py >>>>> >>>>> in which case the file would be in the parent directory to myscript.py >>>>> >>>>> Note that in some environments, the current directory is invisibly set >>>>> to some convenient place. For example, when right-clicking on a >>>>> script >>>>> in Windows Explorer, they make the bald assumption that you want to >>>>> set >>>>> the current directory the same as the location of the script. That's >>>>> about the worse place for it, but nevermind. >>>>> >>>>> >>>>> >>>> Yes, that was my assumption (even thought I am using linux); and right >>>> again, it is a very inconvenient place for it to be. however in the >>>> interest of speed of testing i figured i would make sure the log was >>>> logging the way i want it to and then try to find some sort of wx wigit >>>> to let the user of the gui name/save to desired location. meanwhile . . >>>> . so you think it saved somewhere right? >>> >>> There are other ways a script might change the current directory. For >>> example, some naive scripts use os.chdir() >>> >>> But how is it you don't know what the current directory was when the >>> code ran? A simply pwd can tell you, if your prompt doesn't already >>> reveal it. >>> >>> >> i was assuming the file would write to where the .py file is located. >> of course i know where that is because i put the file there but the >> logfile.csv is not in there. i did a search in the file system and >> nothing so i don't know what is going on here? guess keep looking? its >> gota be here somewhere? no? >> > > So is somebody else running your program? As I said before, how can you > NOT know what your current directory (cwd) is when you ran the script? > And again, the location of the script is irrelevant. > > No more offlist messages. This is a public forum, and I don't do private. > > DaveA > > please pardon mailing you only. when i press reply your email and not the list's email comes in. listen; i run another program in terminal in home directory; that program uses the .py file in the python directory. I already told you i found the file? why would someone else be running the program? ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Need Help Modifying a wxPython GUI (scrolling display and logging)
On 06/13/2013 03:39 AM, Alan Gauld wrote: > On 13/06/13 05:24, Matt D wrote: > >> I already told you i found the file? why would someone else be running >> the program? > > Because it does something useful? > Most pro programmers write programs for other people to use. > Even an amateur may be creating something for their family use. > > If someone other than you were running it that might explain why the > current directory wasn't what you expected since they will have a > different $HOME for example. That's why Dave was asking. > yeah i am not a programmer. just trying to increase the utility of a program (open source) someone else wrote. the file was in the home directory but i have so much pollution there that it took too long for me to spot it. not sure whey the file search didn't work. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Fwd: Re: Need Help Modifying a wxPython GUI (scrolling display and logging)
On 06/13/2013 08:22 AM, Dave Angel wrote: > On 06/13/2013 12:18 AM, Matt D wrote: >> >> >> >>> >>> >> yes the .py file has TextCtrl fields that get there values from a >> pickled dictionary. Another peice of the code watches a thread for the >> pickle. this is why i didnt use a list. I have been unable to find a >> nice way to just make a list with the items i need. would be nice to >> have that simplicity. >> What you said is true, the the list is unordered. More importantly the >> new line comes in at the wrong point. I want all the values in a row >> starting with time. from there i will look for a way to remove some >> unwanted items and ordering the others. >> I attached the .py file for you to see the whole thing hoping this is >> not too presumptuous. Thanks. >> >> > > I don't mind the attached source file. Note that some readers may not > be able to see it (attachments aren't guaranteed to survive), and others > might find it excessive in length. But I'm fine with it. > > I notice you didn't change the newline to a comma, in the place that I > commented earlier. You explicitly separate the fields with newlines, > while commenting that it's done with commas. > > What you presumably want is to change the line inside the loop > > self.logfile.write('\n') > to > self.logfile.write(',') > > and add one of the former lines outside the loop, after writing the last > field. > > About the ordering: Do you have a specific ordering in mind? Who > decides it? The program that creates the pickle? How tightly bound are > the two? Is there a 3rd program that's going to read the csv file? Are > all of these programs written in Python? Will there be multiple > versions, over time? > > If all of these programs have to share the same definition for the csv > file, then at least some of it should be in common code. Simplest is to > have the list/tuple of field names as a real list, defined in a module > that they all import. Then, instead of using self.fields.items(), you > use something like common.FIELD_LIST_NAMES > > common.py might have a line something like: > > #define the tuple of names that will be used for the csv file > FIELD_LIST_NAMES = ("date", "duid", "nac", "source", "dest", "mfid", > "algid", "kid", "mi", "tgid") > > Notice that TrafficPanel.init() might well collapse into a loop, if you > add just a little more information into common.py Then you'd find that > editing the one place adds a new field, both to the csv file but also to > the gui. > > However, then if you add a new field, or remove one, you're obsoleting > any csv files that may still be lying around, with no way to detect > which ones are new and which ones are old. Typically this is managed > with a version field in the first line of the file. > > But another, more standard, way to manage this is to make it a real csv > file, with the field names in the first line (also comma separated). > Python has a csv module, which solves another potential problem your > logic may have: what happens if any of those values has a comma in it? > > > I know I only hinted at the possible implementations, but until you make > some architectural choices clear, I really cannot add much more. > > Here are some other remarks about the code: > > line 53: method Clone() should be lowercase, per Pep8. I don't believe > it does anything useful, but you don't call it anyway. > > line 76: deleting a local just before a method returns does exactly > nothing. When the method ends, the local will go out of scope, and the > effect in either case is to decrement the refcount for the created > DataEvent instance. > > Incidentally, if you happen to be using Thunderbird, you might look for > the Reply-List button. > Hey, line 202: self.logfile.write('%s,'%(str(f))) d does put the comma in properly but, line 203: self.logfile.write('\n') was putting the newline after each value like you said. I moved this back outside of the if statement to see (i am still a little unsure about the indention and i have to test) if it will create a new row only when all the k,v values have been looped through. the ordering: yes this is quite a hole in my understanding of what is going on here. the pickle is created in a collection of pretty complicated C++ code that doesn't explicitly show how the pickle is ordered or whats in it even in the pickle.cc and pickle.h files. the pickle files take in some sort of stream, pic
Re: [Tutor] Fwd: Re: Need Help Modifying a wxPython GUI (scrolling display and logging)
On 06/13/2013 11:23 AM, Dave Angel wrote: > On 06/13/2013 10:37 AM, Matt D wrote: >> On 06/13/2013 08:22 AM, Dave Angel wrote: >>> On 06/13/2013 12:18 AM, Matt D wrote: >>>> >>>> >>>> >>>>> >>>>> >>> >> Hey, >> line 202: self.logfile.write('%s,'%(str(f))) d >> does put the comma in properly but, >> line 203: self.logfile.write('\n') >> was putting the newline after each value like you said. >> I moved this back outside of the if statement to see (i am still a >> little unsure about the indention and i have to test) if it will create >> a new row only when all the k,v values have been looped through. > > Then put it AFTER the loop, not after the if. It should line up with > the for statement. And if you mix spaces with tabs, heaven help you. > Different people have different preferences, but I despise tabs in > source code. Notice that you've done it at least four places: > > #output the value with trailing comma > #if the field 'duid' == 'hdu', then clear all the fields > return 0 > main() > > If your editor let you do that, you aren't using the right settings on > the editor (or the right editor). This didn't affect anything, since > indentation doesn't matter on comments, and the other two lines are > isolated indentations. > > >> >> the ordering: yes this is quite a hole in my understanding of what is >> going on here. the pickle is created in a collection of pretty >> complicated C++ code that doesn't explicitly show how the pickle is >> ordered or whats in it even in the pickle.cc and pickle.h files. the >> pickle files take in some sort of stream, pickle the data, and send it >> to a message queue that the trafficpanel waits on. i need to log this >> pickle or at at least dump it to terminal because i am pretty sure the >> 'source' and 'dest' fields (which currently are not available) are in >> the pickle, albeit in a different data unit. I have read >> "http://www.python.org/doc//current/library/pickle.html"; two times >> already and still cant find a way to print the pickle in human readable >> form. my understanding of pickling stinks. The ordering at this point >> is not so important (not nearly as important as getting the 'source' >> 'dest' fields) because the point of the .csv file is just to import it >> into librecalc and work time series analysis on the data manually. at >> some later point in the development maybe this this task can be >> automated but for now just an unordered file will suffice. > > If you want a consistent ordering, then add the line I described to your > own source code, at module scope. Since you have no access to (control > over) the C++ code, you'll just have to make up your own list, as you've > already effectively done with your GUI. For every field that is NOT in > the dict, you should be outputting a simple comma. > > So your if test is wrong, since it will eat zeros as well as missing > values. And you need an else clause: > > for k,v in FIELD_LIST_NAMES: > # get the value of the current TextCtrl field > f = field_values.get(k, None)2013-06-12 16:28:59,Unknown (0x658), DES-OFB, HDU, 0xa4d5010ca0bbdb0900, 0xfff, Standard MFID (pre-2001), 00x1, > if not f is None: > #output the value with trailing comma > self.logfile.write('%s,'%(str(f))) > else: > self.logfile.write(",") > self.logfile.write("\n") > > And don't forget to add in the header line to your csv file, naming the > fields that are to be used in every line. > as of now the order in the .csv file is like this: 2013-06-12 16:28:59,Unknown (0x658), 00x80, $80 Clear, 0xa4d5010ca0bbdb0900, 0xfff, Standard MFID (pre-2001), 00x1, and keeps repeating this order as long as HUDs are coming in. i am unsure why the date/time is on the same line as NAC? Oh and i have not tested yet with the '\n' newline command out of the if statement. If i have to i can modify the C++ code but was hoping not to have to do that at this stage. the C++ is used for what is computationally intense and the Python is used mainly for the UI. Any idea of a way to write the pickle to file to see what it contains? because it is not explicit in the C++ files, at least not as far as I can tell as of yet. Thanks! ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Fwd: Re: Need Help Modifying a wxPython GUI (scrolling display and logging)
i am trying to figure a way to to use a list to log/print my data: # tmplist = [time, 'nac', 'tgid', 'source', 'dest', 'algid'] is what we want tmplist = [] tmplist.append((str(strftime("%Y-%m-%d %H:%M:%S", localtime( tmplist.append(field_values["nac"]) tmplist.append(field_values["tgid"]) tmplist.append(field_values["source"]) tmplist.append(field_values["dest"]) tmplist.append(field_values["algid"]) When i run the code program dies like this: tmplist.append(field_values["nac"]) ^ SyntaxError: invalid syntax I cant figure why it stops on the third line above? Anyone have an idea? Thanks! ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Fwd: Re: Need Help Modifying a wxPython GUI (scrolling display and logging)
im sorry i dont get it. there is too many brackets in this lin: tmplist.append(field_values["nac"]) Thats where the error is but i dont see too many brackets? On 06/14/2013 08:56 AM, Flynn, Stephen (L & P - IT) wrote: > Not enough closing brackets on the previous line... or actually too many > opening brackets - you don't need all those that you have there already. > > >> # tmplist = [time, 'nac', 'tgid', 'source', 'dest', 'algid'] is what > we >> want >> tmplist = [] >> tmplist.append((str(strftime("%Y-%m-%d %H:%M:%S", > localtime( >> tmplist.append(field_values["nac"]) >> tmplist.append(field_values["tgid"]) >> tmplist.append(field_values["source"]) >> tmplist.append(field_values["dest"]) >> tmplist.append(field_values["algid"]) >> >> When i run the code program dies like this: >> >> tmplist.append(field_values["nac"]) >> ^ >> SyntaxError: invalid syntax >> >> I cant figure why it stops on the third line above? Anyone have an > idea? >> Thanks! > > > This email and any attachment to it are confidential. Unless you are the > intended recipient, you may not use, copy or disclose either the message or > any information contained in the message. If you are not the intended > recipient, you should delete this email and notify the sender immediately. > > Any views or opinions expressed in this email are those of the sender only, > unless otherwise stated. All copyright in any Capita material in this email > is reserved. > > All emails, incoming and outgoing, may be recorded by Capita and monitored > for legitimate business purposes. > > Capita exclude all liability for any loss or damage arising or resulting from > the receipt, use or transmission of this email to the fullest extent > permitted by law. > ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Fwd: Re: Need Help Modifying a wxPython GUI (scrolling display and logging)
On 06/14/2013 10:27 AM, Alan Gauld wrote: > On 14/06/13 14:27, Matt D wrote: >> im sorry i dont get it. there is too many brackets in this lin: >> >> tmplist.append(field_values["nac"]) >> >> Thats where the error is > > No, that's where Python *detected* that an error existed. > The actual error is on the previous line. This is quite > common, especially in cases of mismatched parens or quotes. > > There is a difference between where an error *occurs* and > where an error is *detected*. > got it. the error can be in the previous line. its running now. Thanks guys! ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Fwd: Re: Need Help Modifying a wxPython GUI (scrolling display and logging)
Hey, here is a snip of my code. #logger code-- # first new line #self.logfile.write('\n') # date and time #self.logfile.write('%s,'%(str(strftime("%Y-%m-%d %H:%M:%S", gmtime() # blah = [time, 'nac', 'tgid', 'source', 'dest', 'algid'] is what we want tmplist = [] tmplist.append(str(strftime("%Y-%m-%d %H:%M:%S", localtime( tmplist.append(field_values["nac"]) tmplist.append(field_values["tgid"]) tmplist.append(field_values["source"]) tmplist.append(field_values["dest"]) tmplist.append(field_values["algid"]) #this prints the current row of data to the terminal #print tmplist # this prints the current row of data to the csv file for item in tmplist: self.logfile.write('%s,' % (str(item))) self.logfile.write('\n') # loop through each of the TextCtrl objects #for k,v in self.fields.items(): # get the value of the current TextCtrl field # f = field_values.get(k, None) #if f: # check if k is the field you want # output the value with trailing comma #self.logfile.write('%s,'%(str(f))) # self.logfile.write('\n') # here is where you would put it #end logger code--- i know its ugly. but there is two ways to log here. one makes a list (current) and the other (commented out) loops through the TextCtrls and writes. is their a better way than what i have here? the TextCtrl fields get their values from a pickle. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Fwd: Re: Need Help Modifying a wxPython GUI (scrolling display and logging)
On 06/14/2013 03:14 PM, Dave Angel wrote: > On 06/14/2013 10:48 AM, Matt D wrote: >> Hey, >> here is a snip of my code. >> >> #logger code-- >> # first new line >> #self.logfile.write('\n') >> # date and time >> #self.logfile.write('%s,'%(str(strftime("%Y-%m-%d %H:%M:%S", >> gmtime() >> # blah = [time, 'nac', 'tgid', 'source', 'dest', 'algid'] is >> what we want >> tmplist = [] >> tmplist.append(str(strftime("%Y-%m-%d %H:%M:%S", localtime( >> tmplist.append(field_values["nac"]) >> tmplist.append(field_values["tgid"]) >> tmplist.append(field_values["source"]) >> tmplist.append(field_values["dest"]) >> tmplist.append(field_values["algid"]) >> > > tmplist is an unnecessary complication. Did you look at my sample loop, > which I'll repeat here with a correction: > > > for k in FIELD_LIST_NAMES: > # get the value of the current TextCtrl field > f = field_values.get(k, None) > if not f is None: > #output the value with trailing comma > self.logfile.write('%s,'%(str(f))) > else: > self.logfile.write(",") > self.logfile.write("\n") > > This code preserves your original feature of not crashing when the C++ > program fails to fill in all your expected keys. It also makes sure > there will be unadorned commas for missing fields, making it possible > for a spreadsheet to read the columns correctly. > > If you want to populate a list first, by all means do so, but do it in a > loop, using the FIELD_LIST_NAMES as keys. > > One thing I didn't handle was the date field. I'd do that by adding it > to the field_values dict, or to a copy of it. That way, it's all > consistent, even though one field comes from local and the rest from the > pickle. > > yes acutally this templist business broke my code. the TectCtrls in the traffic panel would were not being populated and the logfile.csv was empty. So should i replace: #logger code--- # first new line self.logfile.write('\n') # date and time self.logfile.write('%s,'%(str(strftime("%Y-%m-%d %H:%M:%S", localtime() # loop through each of the TextCtrl objects for k,v in self.fields.items(): # get the value of the current TextCtrl field f = field_values.get(k, None) if f: # output the value with trailing comma self.logfile.write('%s,'%(str(f))) #end logger code With the code you posted above? I am pretty sure that the reason i don't get the 'source' and 'dest' fields is because of this: #if the field 'duid' == 'hdu', then clear all the fields if field_values['duid'] == 'hdu': self.clear() since the 'source' and 'dest' are in the LUD1 and not the HDU so it doesn't update when the LDU1 comes through (if the LDU1) does actually get serialized. still haven't found a way to get to view the serialized data. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
[Tutor] Need help appending data to a logfile
Hey, I wrote some simple code to write data to a logfile and it works pretty well (thanks guys). Now my problem is that every time i run the program the old logfile.txt is overwritten. I need to be able to stop and start the program without overwriting, or losing, the old data. here is the relavent code: # central part of the program # lays out the GUI panel # omitted lots for our purposes here Class panel(wx.Panel): # open a file named "logfile.txt" in "w" writing mode. # this will create the file if it doesn't exist. self.logfile = open('logfile.txt', 'w') # Updates the TextCtrl field values # and logs TextCtrl field values def update(self, field_values): #logger code--- #first write the CURRENT date/time self.logfile.write('%s,'%(str(strftime("%Y-%m-%d %H:%M:%S", gmtime() # loop through each of the TextCtrl objects for k,v in self.fields.items(): #get the value of the current TextCtrl field f = field_values.get(k, None) if f: #output the value with trailing comma self.logfile.write('%s,'%(str(f))) self.logfile.write('\n') #end logger code In addition to not deleting the old data, it would be awesome to have some sort of wxPython widget that would give the user the ability to 'save as', or name and save the file, from the GUI panel. Thanks! -- Matt D ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Need Help Modifying a wxPython GUI (scrolling display and logging)
> Everything Dave Angel said applies. > > You can sort the keys by doing and sorting the keys and then logging. > That should ensure field order. > > for k in sorted(self.fields): > v = self.fields[k] > > > Also note, that unless you do self.logfile.close() it is not guaranteed > that the data is being written to file. I prefer to use the following > idiom for Python 2.6+ (might be in 2.5, but not sure offhand when it was > added). > > with open('filename.txt', 'a') as f: > # write data > > Thanks! Now with some experience using this logger i have found that the items, while they may not be in an ideal order, are nonetheless always in the same order starting with date/time. In the interest of getting this thing working ASAP the current ordering is acceptable for now; at some later time I may try to arrange into some other desired order. I am testing the 'a' append mode now. hoping this will allow for not overwriting existing data. Where in the program do I put the: self.logfile.close() Is there someway to trigger this from the UI? or even when the program is stopped? ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Need Help Modifying a wxPython GUI (scrolling display and logging)
> A common way to trigger UI actions is a button whose callback calls that. > Or you can bind in an event hook for closing the window. > > in __init__ add this line - > self.Bind(wx.EVT_CLOSE, self.onExit) > > > def onExit(self, event): >'''Run when closing''' >self.logfile.close() >self.Destroy() # Continue closing the app otherwise > # you will not be able to close it via UI > > > Although, exiting the program will probably close the file for you > > > If you are using append mode ('a') you can just do the following > which will append and close the file for you. > with open(filename, 'a' ) as logfile: > logfile.write( data_string ) > > > Make sure you add a newline ('\n') to what you write otherwise all > your output will be on one line with no spaces > > for x in xrange(15): > with open( filename, 'a' ) as f: > f.write( str(x) ) > > filename= > 01234567891011121314 > = > Hey thanks, Currently my data is written like this: 2013-06-19 10:09:53,Unknown,(0x270),Plain,HDU,0x265932377fd55dd000,00x1,Standard MFID (pre-2001),0x666a, 2013-06-19 10:34:43,Unknown,(0x270),Plain,HDU,0xb616ddc0dc3f7d6c00,0xfff,Standard MFID (pre-2001),0x666a, And so on and so on, the new line is only at the end of the '0x666a' in the logfile but here the indentation is forced by my email formatting. so this is good for now. Thanks for the help. But I need to be able to open a text file of choice to write the logfile to. so now i put a button on my panel with the following: btn = wx.Button(self, -1, "Click me") sizer.Add(btn, pos=(7,2)) For some reason i cant not for the life of me figure out how to bind the click event to wxFileDialog. Like if i try: btn.Bind(wx.EVT_BUTTON, self.openfile) I get the AttributeError: object has no attribute 'openfile' Or if i put something like: def openfile(self, event): dlg = wx.FileDialog(self, "Choose a file", os.getcwd(), "", "*.*", wx.OPEN) if dlg.ShowModal() == wx.ID_OK: path = dlg.GetPath() mypath = os.path.basename(path) self.SetStatusText("You selected: %s" % mypath) dlg.Destroy() or anything like it the whole layout of the panel gets destroyed. I have been looking for examples and readying the http://zetcode.com/wxpython/events/ and reading many others but i am having lots of trouble grasping this. The examples are always in a 'wxFrame' or part of an 'app' so the differences from mine (wxPanel) confuse me. THanks! ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
[Tutor] Best Code testing practice?
Hey guys! Is there a fast way test some piece of code? I would like to be able to look at the GUI I am making with out changing the file in dir 'baz' and running the actual program (which is quite extensive). Like if I could just have a .py file with only the smallest amount of code possible to make the GUI run, and I could run the .py file from the interpreter to make the GUI run, this would be beautiful. This allow me to easily/quickly experiment with code and test my code. Do we need an IDE for this sort of thing or can we just use the interpreter? OMG i was up till 3am trying to find some way to do this. Thanks! -- Matt D ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Best Code testing practice?
On 06/20/2013 08:52 AM, Alan Gauld wrote: > On 20/06/13 12:43, Matt D wrote: > >> Is there a fast way test some piece of code? > > There are several testing frameworks for testing Python code. > nose is one example. > But... > >> look at the GUI I am making with out changing the file in dir 'baz' > > In general your GUI should not be tied into one particular file or > folder structure. You should be able to specify that either in a config > file that you read when the GUI starts or as command line arguments. > > That way you can separate your test data from your real production > data. > >> running the actual program (which is quite extensive). > > Hopefully you have separated the presentation logic (the GUI > components) from the processing logic? If so testing the processing > logic is fairly easy using one of the aforementioned frameworks. Testing > the GUI itself is more tricky and usually requires > using some kind of robot to mimic what a user does by sending > dummy event messages into the GUI toolkit. > >> just have a .py file with only the smallest amount of code possible to >> make the GUI run, and I could run the .py file from the interpreter to >> make the GUI run, this would be beautiful. > > That depends on how you built the app. If the GUI is encapsulated > as an an object then you can usually import the GUI code and instantiate > the object. But if you have hard coded links from the GUI event handlers > to your processing logic and from there to the real data/files then its > going to be difficult. > >> Do we need an IDE for this sort of thing > > Almost certainly not, although it may help. > You never *need* an IDE for python, they are only there to help. > > The bottom line is that if your application architecture > has been well designed to separate the GUI and processing > and you have decoupled the data from the logic then testing > should be straightforward. If its all one big organic mashup > then you have a tough time ahead. It may be easier to > re-architect the application and then test it than to > struggle to bolt on tests retrospectively. > all i really want to do is test the the GUI code. i am working on a 'tab' in a notebook of 7 tabs, which is itself part of a large python program which gets all of its computations done in C++. The code for 'tab', or wxPanel really, i am using is in one file. But i dont need to get into all that at this point. all i want to be able to do is make a small sample app that mimics the one tab i am working on and be able to run it to see if the lay out and dialogs are correct. Thanks! ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Best Code testing practice?
On 06/20/2013 10:49 AM, Oscar Benjamin wrote: > On 20 June 2013 15:32, Matt D wrote: >> all i really want to do is test the the GUI code. i am working on a >> 'tab' in a notebook of 7 tabs, which is itself part of a large python >> program which gets all of its computations done in C++. The code for >> 'tab', or wxPanel really, i am using is in one file. >> But i dont need to get into all that at this point. all i want to be >> able to do is make a small sample app that mimics the one tab i am >> working on and be able to run it to see if the lay out and dialogs are >> correct. > > Then make a small app that has just one tab (or has six dummy tabs if > necessary). When this app runs it should use the same function from > your main application to lay out the widgets but but it shouldn't bind > the (same) event handlers. > right make a small sample app. exactly. im sorry if im dense or whatever and obviously i am super new to this process. but i can write the scripts in gedit and then what? how do i run that file and make the gui start? should i be doing something like this?: >>> execfile('filename.py') ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
[Tutor] Writing logfile data to a user opened file
Hey guys! So now my UI panel works well with this: # Open file button click event binded to openfile btn = wx.Button(self, -1, "Click me") sizer.Add(btn, pos=(7,2)) btn.Bind(wx.EVT_BUTTON, self.openFile) #set the panel layout self.SetSizer(sizer) #makes the gui system fit all the controls onto the panel7 self.Layout() self.Fit() EVT_DATA_EVENT(self, self.display_data) #self.watcher = traffic_watcher_thread(self.msgq, self) # openfile defined to start FileDialog def openFile(self, evt): with wx.FileDialog(self, "Choose a file", os.getcwd(), "", "*.*", wx.OPEN) as dlg: if dlg.ShowModal() == wx.ID_OK: path = dlg.GetPath() mypath = os.path.basename(path) So as you guys taught me to do I was opening 'logfile' this way: self.logfile = open('logfile.txt', 'a') And the logger code: #logger code--- # first new line self.logfile.write('\n') # date and time self.logfile.write('%s,'%(str(strftime("%Y-%m-%d %H:%M:%S", localtime() # loop through each of the TextCtrl objects for k,v in self.fields.items(): # get the value of the current TextCtrl field f = field_values.get(k, None) if f: # output the value with trailing comma self.logfile.write('%s,'%(str(f))) #end logger code And that is working well. Now I am trying to think of a way to get what is in the logfile.txt into the file that is opened by the user via the UI. Getting stuck trying to come up with an idea!? maybe something in python like 'user_opened_file = logfile' or 'write logfile to user_opened_file'? I am not able to find standard way to do this. Cheers! -- Matt D ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Best Code testing practice?
> > I suspect that you'd get better answers on a GUI specific mailing list, > like one for wxPython, but I note that you've already asked pretty much > the same question there. > Hey guys! Have decided that it is probably going to be better for my purposes to simply crack open a terminal, cd into the appropriate directory, and do the 'python test_code.py' or whatever the file name is from the command line. I feel it is better for me to learn how to write code in gedit before i use an IDE. Thanks guys! ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Writing logfile data to a user opened file
> > When you open a file the data should be written to that. If you want to > move existing data from logfile.txt into user opened file then you need > to read logfile.txt and then write it to the user opened file. To make > your life simpler, either pass in the file path or open the file save > dialog on __init__. > > > ~Ramit > I got so frustrated try to figure a way to use the logfile.txt that I changed how i log. first i name an array: class TrafficPane(wx.Panel): # Initializer # the class constructor def __init__(self, parent, msgq): wx.Panel.__init__(self, parent) self.msgq = msgq #create the array to put the traffic data in self.log_array = [] Then this is how the array gets into the file the user chooses: # openfile defined to start FileDialog def openFile(self, evt): with wx.FileDialog(self, "Choose a file", os.getcwd(), "", "*.txt*", wx.OPEN) as dlg: if dlg.ShowModal() == wx.ID_OK: path = dlg.GetPath() #mypath = os.path.basename(path) mypath = os.path.abspath(path) f = open(mypath, "rw+") f.writelines(self.log_array) And this is how i get the TextCtrl values into the array: def update(self, field_values): next_line = "" #logger code--- # first new line #self.logfile.write('\n') # date and time #self.logfile.write('%s,'%(str(strftime("%Y-%m-%d %H:%M:%S", localtime() next_line += (str(strftime("%Y-%m-%d %H:%M:%S", localtime( # loop through each of the TextCtrl objects for k,v in self.fields.items(): # get the value of the current TextCtrl field f = field_values.get(k, None) if f: # output the value with trailing comma #self.logfile.write('%s,'%(str(f))) next_line += (str(f) + ',') log_array.append(next_line) #end logger code-- Its running right now. I haven't had the opportunity to test if it works so im keeping my fingers crossed. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Writing logfile data to a user opened file
On 06/21/2013 04:44 PM, Prasad, Ramit wrote: > Matt D wrote: >> [Ramit P wrote:] >>> When you open a file the data should be written to that. If you want to >>> move existing data from logfile.txt into user opened file then you need >>> to read logfile.txt and then write it to the user opened file. To make >>> your life simpler, either pass in the file path or open the file save >>> dialog on __init__. >>> >>> >>> ~Ramit >>> >> I got so frustrated try to figure a way to use the logfile.txt that I >> changed how i log. first i name an array: >> >> class TrafficPane(wx.Panel): >> # Initializer >> # the class constructor >> def __init__(self, parent, msgq): >> wx.Panel.__init__(self, parent) >> self.msgq = msgq >> #create the array to put the traffic data in >> self.log_array = [] >> >> Then this is how the array gets into the file the user chooses: >> >> # openfile defined to start FileDialog >> def openFile(self, evt): >> with wx.FileDialog(self, "Choose a file", os.getcwd(), "", >> "*.txt*", wx.OPEN) as dlg: >> if dlg.ShowModal() == wx.ID_OK: >> path = dlg.GetPath() >> #mypath = os.path.basename(path) >> mypath = os.path.abspath(path) >> f = open(mypath, "rw+") > > Why are you opening the file in "rw+"? > >> f.writelines(self.log_array) > > You should really switch to the "with open() as f:" idiom I keep showing > you. This will automatically close the file for you. > > Also note that your file is only getting written once. You should > probably clear log_array and change the file mode back to append. > >> >> And this is how i get the TextCtrl values into the array: >> >> def update(self, field_values): >> next_line = "" >> #logger code--- >> # first new line >> #self.logfile.write('\n') >> # date and time >> #self.logfile.write('%s,'%(str(strftime("%Y-%m-%d %H:%M:%S", >> localtime() >> next_line += (str(strftime("%Y-%m-%d %H:%M:%S", localtime( >> # loop through each of the TextCtrl objects >> for k,v in self.fields.items(): >> # get the value of the current TextCtrl field >> f = field_values.get(k, None) >> if f: >> # output the value with trailing comma >> #self.logfile.write('%s,'%(str(f))) >> next_line += (str(f) + ',') >> log_array.append(next_line) >> #end logger code-- >> >> Its running right now. I haven't had the opportunity to test if it >> works so im keeping my fingers crossed. > > This is an inefficient string concatenation. It can take large amounts > of memory and time. > > next_line += (str(f) + ',') > > You can use str.join or use the csv module (which I recommend as it > will escape the delimeter (eg. commas ) if it shows up in the data ). > I have sent an example of the csv module already. Also note that > your order of the dictionary is not guaranteed so your data > may end up out of order after each run. > > 'hi', 1 , 5423 > 4255, 'hi', 2 > # instead of > 1, 'hi', 5423 > 2, 'hi', 4255 > > # A str.join example > next_line = [] > for key in sorted( self.fields ): > f = field_values.get(k, None) > if f: > next_line.append(str(f)) > line = ', '.join(next_line) > log_array.append(line) > > > ~Ramit Thanks! so i went with: # openfile defined to start FileDialog def openFile(self, evt): with wx.FileDialog(self, "Choose a file", os.getcwd(), "", "*.txt*", wx.OPEN) as dlg: if dlg.ShowModal() == wx.ID_OK: path = dlg.GetPath() mypath = os.path.basename(path) #mypath = os.path.abspath(path) f = open(mypath, "a") f.writelines(self.log_array) couldn't figure how to use the "with open() as f:" and then down here: # Update the field values # put values in array def update(self, field_values): next_line = "" next_line += strftime("%Y-%m-%d %H:%M:%S") next_line +=
Re: [Tutor] Writing logfile data to a user opened file
> > You should really switch to the "with open() as f:" idiom I keep showing > you. This will automatically close the file for you. > it just occured to me to do this: def openFile(self, evt): with wx.FileDialog(self, "Choose a file", os.getcwd(), "", "*.txt*", wx.OPEN) as dlg: if dlg.ShowModal() == wx.ID_OK: path = dlg.GetPath() mypath = os.path.basename(path) with open(mypath, "a") as f: f.writelines(self.log_array) so thats how i used what you said, "with open() as f:". is this the right way to open the file? ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Writing logfile data to a user opened file
On 06/22/2013 03:47 AM, Alan Gauld wrote: > On 22/06/13 02:42, Matt D wrote: > >> if dlg.ShowModal() == wx.ID_OK: >> path = dlg.GetPath() >> mypath = os.path.basename(path) >> with open(mypath, "a") as f: >> f.writelines(self.log_array) >> >> so thats how i used what you said, "with open() as f:". is this the >> right way to open the file? > > You need to indent the writelines() call so its inside > the 'with' block. > > with open(mypath, "a") as f: >f.writelines(self.log_array) > > This means you don't need to call f.close() each time - which you were > missing from your earlier code. > Thanks! That is how i had it; not sure how the indent got nixed when pasted to my mail. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
[Tutor] Need help printing a pickled data
I have been unable to find a way to write pickled data to text file. My last attempt was to add the last two lines: # the dataevent class -- stores the data that gets transmitted when the event occurs. #it is the data in text fields, stored in self.data as a dictionary, which is basically a c++ map #One of these classes gets created whenever an event occurs. class DataEvent(wx.PyEvent): # the values of the text fields get passed into the constructor inside of data def __init__(self, data): wx.PyEvent.__init__(self) # this line *binds* this class to a certain type of event, wxDATA_EVENT self.SetEventType (wxDATA_EVENT) # and this is the actual data self.data = data with open('mypicklelog.txt','a') as log: log.write(self.data) I cant figure out why these last two line dont write to the .txt file after the program has received the pickled Python dictionary? Very stumped. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Need help appending data to a logfile
On 06/24/2013 05:57 PM, Dave Angel wrote: > On 06/24/2013 05:39 PM, Matt D wrote: >> >>> But what he's doing has nothing to do with logging. He's just using >>> that word. >>> >>> >> Right, I'm not doing a debugging thing. Just trying to create a log of >> data that has been passed into the display of this program. Since I >> started trying to use the array the program is not working. So I am >> wondering if its not the best way to try to save TextCtrl values. >> > > You accidentally posted this offline, direct to me. So it loses all > context. I'll try to get it back into the thread, but it may not "take." > > I don't know what array you're referring to. I VERY seldom use arrays > in Python, since list is much more flexible and easy to use. Nor do I > know what you mean by the program not working. > > I also don't know what you mean by saving TextCtrl values. Aren't they > being appended to the csv file (in random order) with your latest changes? > > > You asked about a "save-as" feature. Why isn't that as simple as > copying the current contents of the saved csv file? Or do you not know > how you would go about copying? > > Be specific with your questions, and you might get some answers. Use > those answers in later versions, and you'll get the same people more > willing to help further. > > Why is it no longer of interest to you to get a predictable order in > your csv file? > > > -- > DaveA > > Well i decided to switch to a .txt file because it is easier for me to lay eyes on it and the commas are there if i want to import it into librecalc or sas or whatever. and values were writing in an acceptable order becuase, while the order was not of my choosing, the order was always the same. so that is fine for importing into another program for later inspection. What you said about the 'save as' is right, i didnt know how to go about copying the logfile.txt (as I had it) to the file the user opened. thats why i decided to try an array like this: # openfile defined to start FileDialog and write the traffic log to the file def openFile(self, evt): with wx.FileDialog(self, "Choose a file", os.getcwd(), "", "*.txt*", wx.OPEN) as dlg: if dlg.ShowModal() == wx.ID_OK: path = dlg.GetPath() mypath = os.path.basename(path) with open(mypath, "a") as f: f.writelines(self.log_array) I name the array up in the initializer self.log_array = [] and this is supposed to get the the values out of the pickle and into the textcrls and into the array: # Update the field values # put values in array def update(self, field_values): next_line = "" next_line += strftime("%Y-%m-%d %H:%M:%S") next_line += ','.join( field_values[k] for k in self.fields.keys() if k in field_values ) log_array.append(next_line) #if the field 'duid' == 'hdu', then clear all the fields if field_values['duid'] == 'hdu': self.clear() #loop through all TextCtrl fields storing the key/value pairs in k, v for k,v in self.fields.items(): # get the pickle value for this TextCtrl f = field_values.get(k, None) # if the value is empty then set the new value if f: v.SetValue(f) But the program is not working, meaning the TextCtrl fields are not getting their values and the log is not being written. I dont know if I am missing a self.something or I some indentation wrong that doest throw the error or what. Very Stuck. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Need help printing a pickled data
On 06/24/2013 06:05 PM, Matt D wrote: > I have been unable to find a way to write pickled data to text file. > My last attempt was to add the last two lines: > > # the dataevent class -- stores the data that gets transmitted when the > event occurs. > #it is the data in text fields, stored in self.data as a dictionary, > which is basically a c++ map > #One of these classes gets created whenever an event occurs. > class DataEvent(wx.PyEvent): > # the values of the text fields get passed into the constructor > inside of data > def __init__(self, data): > wx.PyEvent.__init__(self) > # this line *binds* this class to a certain type of event, > wxDATA_EVENT > self.SetEventType (wxDATA_EVENT) > # and this is the actual data > self.data = data > with open('mypicklelog.txt','a') as log: > log.write(self.data) > > I cant figure out why these last two line dont write to the .txt file > after the program has received the pickled Python dictionary? Very > stumped. > Sorry I forgot to show the error in terminal: File "/usr/local/lib/python2.7/dist-packages/baz/op25_traffic_pane.py", line 53, in __init__ log.write(self.data) TypeError: expected a character buffer object ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Need help printing a pickled data
On 06/24/2013 07:17 PM, Alan Gauld wrote: > On 24/06/13 23:05, Matt D wrote: >> I have been unable to find a way to write pickled data to text file. > > Probably because pickled data is not plain text. > You need to use binary mode. However... > > >> def __init__(self, data): >> wx.PyEvent.__init__(self) >> self.SetEventType (wxDATA_EVENT) >> # and this is the actual data >> self.data = data >> with open('mypicklelog.txt','a') as log: >> log.write(self.data) > > Since you are not using pickle here, all you are really doing > is trying to write whatever data is to a text file that > happens to have 'pickle' in its name. > > When writing to a text file you need to write strings. > You have no guarantee that 'data' is a string. You should > probably convert it before writing it. Thats one of the > advantages of using real pickles - they take care of > that complication for you. > >> I cant figure out why these last two line dont write to the .txt file >> after the program has received the pickled Python dictionary? > > Pickle data has to be unpickled before you can use it. > Before you can write it back again you need to repickle it. > The code you posted does not show you producing and pickled > data nor indeed you reading any pickled data... > > If 'data' is indeed in pickle format you cannot simply write > it to a text file since Pickle is not in a text format. > > im sorry; some more code will clarify i think ; class DataEvent(wx.PyEvent): # the values of the text fields get passed into the constructor inside of data def __init__(self, data): wx.PyEvent.__init__(self) # this line *binds* this class to a certain type of event, wxDATA_EVENT self.SetEventType (wxDATA_EVENT) # and this is the actual data self.data = data with open('mypicklelog.txt','a') as log: log.write(self.data) self.data stores the data as a dictionary. from lower inthe program: # Display the values on the UI def display_data(self,event): #gets the "message" into the event object #message is equal to the "data" parameter in the "DataEvent" class message = event.data # unpickle the string pickled_dict = message.to_string() #separate the string into values for each text control (attrs is a pickle object) attrs = pickle.loads(pickled_dict) #pass this pickle object into update self.update(attrs) The purpose of getting the pickle in file is so that i can make a little program to resend it to this program at will for testing purposes. Clearly I was not aware that we cant put pickled data into a test file. What would be the best way to save the pickle for testing purposes? ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Need help printing a pickled data
On 06/25/2013 07:28 AM, eryksun wrote: > On Mon, Jun 24, 2013 at 6:57 PM, Steven D'Aprano wrote: >> >> You certainly shouldn't be writing pickle data to a log file! Firstly, >> log files are usually opened in text mode, not binary mode, so it >> probably won't work, and secondly even if it did work, you will be >> dumping a load of pickled binary data into the middle of what should be >> a text file. That's a bad idea. And even if it succeeded, what are you >> going to learn from seeing a line like this: > > I don't know which version Matt is using, but 2.x defaults to pickle > protocol 0, which is ASCII. In 3.x you can manually specify protocol > 0: > > >>> pickle.dumps([1,2,3], protocol=0) > b'(lp0\nL1L\naL2L\naL3L\na.' > > With the ASCII protocol you can open the file in either text or binary > mode, so long as you're consistent. But use binary mode if it needs to > be cross-platform. > ___ > Tutor maillist - Tutor@python.org > To unsubscribe or change subscription options: > http://mail.python.org/mailman/listinfo/tutor > I'm using 2.7 for this program. I the docs say what you said about the default being 0 so I didn't know I needed to convert to string. Thanks. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Need help printing a pickled data
> > with open('mypicklelog.txt','ab') as log: # open in binary mode > pickle.dump(self.data, log) # serialize data and write to file > > where pickle.dump(obj, file) converts `obj` to a sequence of bytes before it > is written to `file`. > I put this like this: class DataEvent(wx.PyEvent): # the values of the text fields get passed into the constructor inside of data def __init__(self, data): wx.PyEvent.__init__(self) # this line *binds* this class to a certain type of event, wxDATA_EVENT self.SetEventType (wxDATA_EVENT) # and this is the actual data self.data = data with open('mypicklelog.txt','ab') as log: # open in binary mode pickle.dump(self.data, log) # serialize data and write to file And I still get nothing. This has me super confused, I have trying at this for a long time and I have been reading the docs like http://docs.python.org/2/library/pickle.html And I still don't get it to work. So the C++ makes the pickle, this is from the comments in the C++ : /** * snapshot_du_handler. Writes traffic snapshots to a msg_queue based * on the HDU frame contents. The format used is that of a pickled * python dictionary allowing the other end of the queue to pick only * those fields of interest and ignore the rest. */ Then the python program (attached) program unpickles and repickles it? I attached the python program, if you have a minute or two please take a look, its not too long. #!/usr/bin/env python # -*- coding: utf-8 -*- # # op25_traffic_panel.py # # Copyright 2013 Balint Seeber # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, # MA 02110-1301, USA. # # # MyVersion 1.2 # the import statements; very similar to #include in c++ from time import localtime, strftime import os import wx import cPickle as pickle import gnuradio.gr.gr_threading as _threading # wx is the gui class. it implements a version of "events" -- objects that sit and wait for some data to change, and call a specified function when the change happens wxDATA_EVENT = wx.NewEventType() # this is a function that *sets* what happens when the event is triggered. it takes in an event (win) and a function (func) def EVT_DATA_EVENT(win, func): win.Connect(-1, -1, wxDATA_EVENT, func) # the dataevent class -- stores the data that gets transmitted when the event occurs. #it is the data in text fields, stored in self.data as a dictionary, which is basically a c++ map #One of these classes gets created whenever an event occurs. class DataEvent(wx.PyEvent): # the values of the text fields get passed into the constructor inside of data def __init__(self, data): wx.PyEvent.__init__(self) # this line *binds* this class to a certain type of event, wxDATA_EVENT self.SetEventType (wxDATA_EVENT) # and this is the actual data self.data = data with open('mypicklelog.txt','ab') as log: # open in binary mode pickle.dump(self.data, log) # serialize data and write to file # clone is a python function to make a "deep" copy of an object def Clone (self): self.__class__ (self.GetId()) # thread that waits for new data to be received # when event is triggered new data is passed along # this class inherits from the standard Python class _threading.Thread class traffic_watcher_thread(_threading.Thread): def __init__(self, rcvd_pktq, event_receiver): ## variables are standard values required to set up a thread _threading.Thread.__init__(self) self.setDaemon(1) self.rcvd_pktq = rcvd_pktq self.event_receiver = event_receiver self.keep_running = True self.start() def stop(self): self.keep_running = False def run(self): while self.keep_running: msg = self.rcvd_pktq.delete_head() ## once data is received, an event is "Posted" with PostEvent. ## This is where the DataEvent object gets created de = DataEvent (msg) wx.PostEvent (self.event_receiver, de) del de # A snapshot of important fields in current traffic # this inherits from the gui class Panel (required for all gui programs) class TrafficPane(wx.Panel): # Initializer; class
Re: [Tutor] Need help printing a pickled data
> > Well I think self.data is some kind of container with a pickled string, > given the code to unpickle it is: > Exactly! This is what the C++ file 'pickle.h' creates to send to the Python GUI: /** * A pickled Python dictionary. Used to pass stuff to the UI. */ class pickle { public: /** * pickle constructor. * * \param frame_body A const_bit_queue representing the frame body. */ pickle(); /** * pickle virtual destructor. */ ~pickle(); /** * Add a key/value pair to the pickled dictionary */ void add(std::string key, std::string value); /** * Returns a string describing the Data Unit ID (DUID). */ std::string to_string() const; private: typedef std::map stringmap; stringmap map_; }; #endif /* INCLUDED_PICKLE_H */ I am pretty sure that it sends a C++ map of strings in a form that Python understands as a pickle. I put the code you gave me into the file so ill have to wait and see. meanwhile can you please take a look at this update() and tell me if you see something wrong because ever sense I tried using the array for logging the values from the TextCtrls the program is not updating meaning when the program receives the c++ map, or the pickle, I get nothing in the UI and I get nothing in the log. There got to be something wrong with the loop that I can not see. I attached the file so its not too much code here. I am pretty sure the problem is in the update(0) starts at line 210. #!/usr/bin/env python # -*- coding: utf-8 -*- # # op25_traffic_panel.py # # Copyright 2013 Balint Seeber # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, # MA 02110-1301, USA. # # # MyVersion 1.2 # the import statements; very similar to #include in c++ from time import localtime, strftime import os import wx import cPickle as pickle import gnuradio.gr.gr_threading as _threading # wx is the gui class. it implements a version of "events" -- objects that sit and wait for some data to change, and call a specified function when the change happens wxDATA_EVENT = wx.NewEventType() # this is a function that *sets* what happens when the event is triggered. it takes in an event (win) and a function (func) def EVT_DATA_EVENT(win, func): win.Connect(-1, -1, wxDATA_EVENT, func) # the dataevent class -- stores the data that gets transmitted when the event occurs. #it is the data in text fields, stored in self.data as a dictionary, which is basically a c++ map #One of these classes gets created whenever an event occurs. class DataEvent(wx.PyEvent): # the values of the text fields get passed into the constructor inside of data def __init__(self, data): wx.PyEvent.__init__(self) # this line *binds* this class to a certain type of event, wxDATA_EVENT self.SetEventType (wxDATA_EVENT) # and this is the actual data self.data = data with open('mypicklelog.txt','ab') as log: # open in binary mode pickle.dump(pickle.loads(self.data.to_string()), log) # serialize data and write to file # clone is a python function to make a "deep" copy of an object def Clone (self): self.__class__ (self.GetId()) # thread that waits for new data to be received # when event is triggered new data is passed along # this class inherits from the standard Python class _threading.Thread class traffic_watcher_thread(_threading.Thread): def __init__(self, rcvd_pktq, event_receiver): ## variables are standard values required to set up a thread _threading.Thread.__init__(self) self.setDaemon(1) self.rcvd_pktq = rcvd_pktq self.event_receiver = event_receiver self.keep_running = True self.start() def stop(self): self.keep_running = False def run(self): while self.keep_running: msg = self.rcvd_pktq.delete_head() ## once data is received, an event is "Posted" with PostEvent. ## This is where the DataEvent object gets created de = DataEvent (msg) wx.PostEvent (self.event_receiver, de) del de # A snapshot of important fields in current traffic # this inherits from the gui class Panel (required for all gui programs) class TrafficPane(wx.Panel
Re: [Tutor] Need help printing a pickled data
On 06/25/2013 01:54 PM, Alan Gauld wrote: > On 25/06/13 17:32, Matt D wrote: > >> self.data = data >> with open('mypicklelog.txt','ab') as log: # open in binary mode >> pickle.dump(self.data, log) # serialize data and write to >> file >> >> >> And I still get nothing. > > Define 'nothing'. > > Does the file exist? > Does it have anything in it? > Does self.data exist - what does it look like if you print it? > Are there any error messages? > Yeh nothing as in an empty file. The file is there but there is nothing written in it. self.data exists. that the problem if have is printing what is in it so i dont know what it looks like. No error messages currently in the terminal. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Need help printing a pickled data
> > The real question is why do you want this pickle in a file? I am not sure > it will be easy to pull out and reuse anyway. Given your experience level, > I think this is a lot of work for something that you are unlikely to be able > to easily use. I think it would be more useful to `log.write(repr(attrs))`. > > Once you have the bytes of pickled data, just write that to file. > > with open('mypicklelog.txt','ab') as log: > # All pickles will run together because there is no spacing. > log.write(self.data.to_string()) > > > Again, I think you would be better off with the repr() > with open('mypicklelog.txt','ab') as log: > log.write(repr(pickle.loads(self.data.to_string( > > OK thanks. what i did is went back to the old logger where everything was working. and then i put self.update(attrs) with open('mypicklelog.txt','ab') as log: log.write(repr(attrs)) to try to lay eyes on the pickle. What i really need is a way to test this GUI without waiting so long. I was thinking if i could get a hold of one of the c++ maps i could make a small program that would enable me to send it to the python gui at will; to test the gui. I am not sure if there is a way around this testing difficulty because two of the TextCtrl fields are empty. This is because currently the c++ is only sending in the container the contents of the Header Data Unit (HDU). In digital radio the transmission consists of frames, the HDU is one of them, as the name suggest it is the first. But the Logical Data Unit 1 (LDU1) contains the source and destination id data that ultimately i need to get in the UI. So I am going to have to get into the c++ map stuff and get familiar with how this pickle process works anyway. This is pass/fail thing, it doesn't matter if my code is 'ugly' or whatever and I have time so it shouldn't be impossible. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor