Re: [Tutor] unittest with random population data
On 01/06/15 18:14, Sydney Shall wrote: Finally, I have one comment for the respected Moderator, if he is not out on a walk in the highlands in this cold and wet weather. No, that's on tomorrow's schedule :-) I have taken the liberty of raising my problem here rather than elsewhere, because I have observed that the biological and bio-medical community, who always come late to new notions, is now rapidly discovering python. A great deal of work in these fields involve either stochastic simulations or statistical problems of analysis. The latter are more or less straight-forward, but the simulations are not. I have no problem with this kind of material on tutor because, although the list is for "beginners to Python", that includes those with much experience in other languages and who may expect to use TDD from the beginning. So TDD techniques using standard library modules is a valid topic even though it may be too advanced for those beginners who are new to programming. We need to balance coverage to cater for both categories of "beginner". -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] unittest with random population data
[ I've taken this discussion back to the tutor list. - Cameron ] On 01Jun2015 18:14, Sydney Shall wrote: On 31/05/2015 03:00, Cameron Simpson wrote: You say that your results are "all rather close, consistent with the sigma value I have chosen for the spread of my population". I would advocate making some "contraint" tests that verify this property for _any_ input data set. Then you can run with random and _changing_ input data sets to verify that your code produces the expected _kind_ of results with many data sets. [...] range class which stores contiguous ranges efficiently (a sequence of (low,high) pairs). It has a few add/remove operations which are meant to maintain that sequence on ordered minimal form. cutting and merging adjacent ranges is very easy to get wrong, very sensitive to off-by-one logic errors. So my tests for this class include some random tests which do random unpredictable add/remove operations, and run a consistency check on the object after each operation. This gives me good odds of exercising some tricky sequence which I have not considered explicitly myself. You can see the test suite here: https://bitbucket.org/cameron_simpson/css/src/tip/lib/python/cs/range_tests.py I have studied the material you indicated and it has been most helpful. I think that I have understood the principle involved, but I have had some problem implementing it. The range tests are mostly clear to me but there is one aspect I cannot follow. You use in this suite imports from Range, including Range, overlap, spans and Span. Are these programs that you have written? If so, are they specific to your set up or are they generic? If so, is it possible to obtain these programs? The "cs.range" is a module of my own. As you might imagine, cs.range_tests has the entire purpose of testing cs.range. cs.range is here: https://bitbucket.org/cameron_simpson/css/src/tip/lib/python/cs/range.py Feel free. Since it in turn has some dependencies the easiest way to get it is to use "pip" to install it, as pip will also fetch and insteall its dependencies. But if you just want something to read, fetch and enjoy. I have established a very primitive test suite based on your cs.range >notions and it works fine, but it would be better, I am sure, to do it >properly. There are many other people whose test writing ability surpasses mine. But you're welcome to learn anything you can from mine, and to ask questions. Finally, I have one comment for the respected Moderator, if he is not out on a walk in the highlands in this cold and wet weather. I have taken the liberty of raising my problem here rather than elsewhere, because I have observed that the biological and bio-medical community, who always come late to new notions, is now rapidly discovering python. A great deal of work in these fields involve either stochastic simulations or statistical problems of analysis. The latter are more or less straight-forward, but the simulations are not. You might ask a separate question on the python-l...@python.org about simulations. It has a wider audience than the tutor list and may well include people doing simulation work, or who know where to look. Thanks for all the help. You people are a model of how we could perhaps civilize humanity. Nah. We might all be striving to be a model of how humanity might be when civilised though... Cheers, Cameron Simpson You my man are a danger to society and should be taken out of society for all our sakes. As to what is done to you once removed I couldn't care less. - Roy G. Culley, Unix Systems Administrator ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Trouble using bioread to convert files from .acq to .mat
Laura, that was it! Thank you so much. However, now I am confused about what I need to type into the Command prompt window (on a 64-bit windows computer, using python 2.7) in order to convert a folder of data files. Does anyone know the proper code to do this? On Tue, Jun 2, 2015 at 4:45 AM, Laura Creighton wrote: > In a message of Mon, 01 Jun 2015 15:50:26 -0400, Ila Kumar writes: > >Hello, > > > >I am a new Python user attempting to use bioread ( > >https://pypi.python.org/pypi/bioread/0.9.5) to convert files from > >aqknowledge to matlab. I am using a 64-bit PC, and I have downloaded > >Matlab, Python, numpy, scipy and bioread. Can someone walk me through the > >installation process for this package? I can't seem to get it to work. > > > >Thank you so much for your help!! > >___ > >Tutor maillist - Tutor@python.org > >To unsubscribe or change subscription options: > >https://mail.python.org/mailman/listinfo/tutor > > You are getting something like: > Command python setup.py egg_info failed with error code 1 in > /tmp/pip-build-T3JieI/bioread > Storing debug log for failure in /home/lac/.pip/pip.log > > when you run pip, correct? > > So you go read the package documentation and discover that bioread > uses easy_install (which you are beginning to learn to hate) instead of > pip. > > So then you try that and you get crud like this: > > File "/usr/lib/python2.7/dist-packages/setuptools/sandbox.py", line 43, > in _execfile > exec(code, globals, locals) > File "/tmp/easy_install-PGpger/bioread-0.9.5/setup.py", line 3, in > > ImportError: No module named ez_setup > > Correct? > > If this is your problem, go install this package: > > https://pypi.python.org/pypi/ez_setup > > And then install bioread, still with easy_install > > If this isn't your problem, > write back with more of what is not working, please. > > Hope this is it :) > Laura > > ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Trouble using bioread to convert files from .acq to .mat
On 03/06/15 13:57, Ila Kumar wrote: However, now I am confused about what I need to type into the Command prompt window (on a 64-bit windows computer, using python 2.7) in order to convert a folder of data files. Does anyone know the proper code to do this? You will need to give us a lot more detail than that to help you. Convert what kind of data file? To what kind of output file? Also for most such tasks you won;t be typing it at the prompt you will be writing a script and executing that. Do you have any experience of Python scripting/programming? If not you should find a tutorial and follow that. If you know how to program in another language then the official web site tutorial should be just fine. If you are a complete beginner to programming try one of these: https://wiki.python.org/moin/BeginnersGuide/NonProgrammers Or try mine (see link below)... The code you need could be as simple as # import os def convertFile(fname): # your code to do the conversion goes here dir = /your/folder/path/here for fileName in os.listdir(dir): convertFile(fileName) ## But if there are subfolders to consider, or errors to handle, then it all gets more messy quite quickly. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
[Tutor] dictionary of lists
I am trying to create a dictionary of lists as I read a file. I
envision it looking like: {key: [float_type],[string_type]}
For the first item in the list I am trying to add the value to the
existing value where the key matches but I am getting the following
error:
Resetting execution engine
Running C:\Users\cs062x\Desktop\python\projects\PanHandle\PanHandle\PanHandle.py
The Python REPL process has exited
Traceback (most recent call last):
File
"C:\Users\cs062x\Desktop\python\projects\PanHandle\PanHandle\PanHandle.py",
line 22, in
d[IMEI] += Elapsed_Mins
TypeError: 'float' object is not iterable
Here is the code. My question is how can I keep it list type and just
sumup the first element in the list as I read it?
d = defaultdict(list)
for fname in os.listdir('.'):
with open (fname) as csvfile:
spamreader = csv.reader(csvfile, delimiter=',', quotechar='|')
next(spamreader)
for row in spamreader:
if row[8]:
device = row[36]
Elapsed_Mins = float(row[7])
IMEI = row[8].replace("'", "")
d[IMEI] += Elapsed_Mins ## this is where the error occurs.
Thanks in advance,
Chris
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
[Tutor] FTP GET from Variables Stored in File
I'm fairly new to programming and as a result, Python too. I've been researching the Tutor archives trying to find a solution to my need, however, haven't stumbled across the exact example and are now in information overload. Wondering if I could get some help? I have the need to perform an FTP GET for specific files in a large directory of files. There are many examples out there of FTP'ing from a server after doing the equivalent to a Unix 'ls -la' command and thus, getting all files in the directory or matching on a pattern (i.e., all files ending with *.txt). My problem is I don't have files with a clear pattern of association to allow a pattern match. For example I have 1000 files in a directly with file names like: apple.txt, blue.txt, car.txt, sky.rtf, etc. I know the filenames I want, however it could be say 75 of 1000 files I need to issue a GET on. Inefficient to individually code all 75 explicit file name declarations in a script. So I have the name of each file (i.e., apple.txt, blue.txt) in a separate "seed file" that I want the script to read in and treat each line of the "seed file" as the variable I want to request the FTP GET on. So something like: seedfile.txt contents = apple.com, blue.com. car.txt Open FTP CD Read "seedfile.txt" For each line in "seedfile.txt" use the line as the variable and issue FTP GET on that variable Repeat till end of "sendfile.txt" quit. Thoughts? Help? Thanks. //eric ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
[Tutor] string delimiters
for formatting a string and adding descriptors: test = 'datetimepart1part2part3the_rest' newtest = 'date=' + test[0:4] + ' time=' + test[4:8] + ' part1=' + test[8:13] + ' part2=' + test[13:18] + ' part3=' + test[18:23] + ' the rest=' + test[23:] and while this may be ugly, it does what I want it to do. The question is, if instead of 'the_rest' I have ']the_rest' and sometimes there's not just one. how do I handle that? In other words, this script will iterate over numerous lines in a file, and each one is identical up to the delimiter before the rest, and sometimes there is only one, sometimes there is two, they vary in length. Can I stop using position numbers and start looking for specific characters (the delimiter) and proceed to the end (which is always a constant string btw). regards, Richard -- Windows assumes you are an idiot…Linux demands proof. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] string delimiters
On 03/06/15 20:10, richard kappler wrote: for formatting a string and adding descriptors: test = 'datetimepart1part2part3the_rest' If this is really about parsing dates and times have you looked at the datetime module and its parsing/formatting functions (ie strptime/strftime)? Can I stop using position numbers and start looking for specific characters (the delimiter) and proceed to the end (which is always a constant string btw). The general answer is probably to look at regular expressions. But they get messy fast so usually I'd suggest trying regular string searches/replaces and splits first. But if your pattern is genuinely complex and variable then regex may be the right solution. But if its dates check the strptime() functions first. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] FTP GET from Variables Stored in File
On 03/06/15 17:49, Eric Grey wrote:
declarations in a script. So I have the name of each file (i.e.,
apple.txt, blue.txt) in a separate "seed file" that I want the script to
read in and treat each line of the "seed file" as the variable I want to
request the FTP GET on.
So something like:
seedfile.txt contents = apple.com, blue.com. car.txt
Open FTP
CD
Read "seedfile.txt"
For each line in "seedfile.txt" use the line as the variable and issue FTP
GET on that variable
Repeat till end of "sendfile.txt"
quit.
Thoughts? Help?
Yes that looks like a sensible approach and is almost Python.
What bits of it are you struggling with? Convert your pseudo
code into Python code and show us what breaks. Let me do one
step towards that for you...
Open FTP
FTP.CD
with open("seedfile.txt") as filenames:
for name in filenames:
FTP.GET(name.strip())
Close FTP
Can you fill in the blanks?
--
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] dictionary of lists
On 03/06/15 17:39, Chris Stinemetz wrote:
I am trying to create a dictionary of lists as I read a file. I
envision it looking like: {key: [float_type],[string_type]}
Thats not a dictionary of lists. You maybe mean:
{key: [[float_type],[string_type]]}
Which is a dictionary of lists of lists?
For the first item in the list I am trying to add the value to the
existing value where the key matches
Sorry, I'm sure that made sense to you but not to me.
Which value are you adding to which existing value?
Can you give a before/after example?
Resetting execution engine
Running C:\Users\cs062x\Desktop\python\projects\PanHandle\PanHandle\PanHandle.py
The Python REPL process has exited
That's slightly unusual. How are you running this?
Traceback (most recent call last):
File
"C:\Users\cs062x\Desktop\python\projects\PanHandle\PanHandle\PanHandle.py",
line 22, in
d[IMEI] += Elapsed_Mins
TypeError: 'float' object is not iterable
d = defaultdict(list)
for fname in os.listdir('.'):
with open (fname) as csvfile:
spamreader = csv.reader(csvfile, delimiter=',', quotechar='|')
next(spamreader)
for row in spamreader:
if row[8]:
device = row[36]
Elapsed_Mins = float(row[7])
IMEI = row[8].replace("'", "")
So IMEA is a string and Elapsed_Mins is a float and d is a default dict
that sets its defaults to lists.
d[IMEI] += Elapsed_Mins ## this is where the error occurs.
So this is trying to add a float to a list.
>>> L = []
>>> L += f
Traceback (most recent call last):
File "", line 1, in
TypeError: 'float' object is not iterable
look familiar?
Now, what I don't know, is what you are trying to do.
Are you trying to append the float to the list?
Or to replace the list with the float?
Or to add the float to the value of the first(or last?)
element in the list - if it exists
(and if it doesn't? Then what?)
--
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] string delimiters
I was trying to keep it simple, you'd think by now I'd know better. My
fault and my apology.
It's definitely not all dates and times, the data and character types
vary. This is the output from my log parser script which you helped on the
other day. there are essentially two types of line:
Tue Jun 2 10:22:42 2015SE201506012200310389PS01CT1407166S0011.40009.7.6IN
0018.1LB000258]C10259612019466862270088094]L0223PDF
Tue Jun 2 10:22:43 2015SE0389icdim01307755C0038.20033.20012.0IN10
0032]C10259612804038813568089577
I have to do several things:
the first type can be of variable length, everything after the ] is an
identifier that I have to separate, some lines have one, some have more
than one, variable length, always delimited by a ]
the second type (line 2) doesn't have the internal datetime stamp, so I
just need to add 14 x's to fill in the space where that date time stamp
would be.
and finally, I have to break these apart and put a descriptor with each.
While I was waiting for a response to this, I put together a script to
start figuring things out (what could possibly go wrong?!?!?! :-) )
and I can't post the exact script but the following is the guts of it:
f1 = open('unformatted.log', 'r')
f2 = open('formatted.log', 'a')
for line in f1:
for tag in ("icdm"):
if tag in line:
newline = 'log datestamp:' + line[0:24] # + and so on to format
the lines with icdm in them including adding 14 x's for the missing
timestamp
f2.write(newline) #write the formatted output to the new log
else:
newline = 'log datestamp:' + line[0:24] # + and so on to format
the non-icdm lines
f2.write(newline)
The problems are:
1. for some reason this iterates over the 24 line file 5 times, and it
writes the 14 x's to every file, so my non-icdm code (the else:) isn't
getting executed. I'm missing something basic and obvious but have no idea
what.
2. I still don't know how to handle the differences in the end of the
non-icdm files (potentially more than identifier ] delimited as described
above).
regards, Richard
On Wed, Jun 3, 2015 at 3:53 PM, Alan Gauld
wrote:
> On 03/06/15 20:10, richard kappler wrote:
>
>> for formatting a string and adding descriptors:
>>
>> test = 'datetimepart1part2part3the_rest'
>>
>
> If this is really about parsing dates and times have
> you looked at the datetime module and its parsing/formatting
> functions (ie strptime/strftime)?
>
> Can I stop using position numbers and start looking for specific
>> characters
>> (the delimiter) and proceed to the end (which is always a constant string
>> btw).
>>
>
> The general answer is probably to look at regular expressions.
> But they get messy fast so usually I'd suggest trying regular
> string searches/replaces and splits first.
>
> But if your pattern is genuinely complex and variable then
> regex may be the right solution.
>
> But if its dates check the strptime() functions first.
>
>
> --
> Alan G
> Author of the Learn to Program web site
> http://www.alan-g.me.uk/
> http://www.amazon.com/author/alan_gauld
> Follow my photo-blog on Flickr at:
> http://www.flickr.com/photos/alangauldphotos
>
>
> ___
> Tutor maillist - Tutor@python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
>
--
Windows assumes you are an idiot…Linux demands proof.
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] string delimiters
hold the phone
I have no idea why it worked, would love an explanation, but I changed my
previous test script by eliminating
for tag in ("icdm"):
and changing
if tag in line
to
if 'icdm' in line:
and it works perfectly! It only iterates over the file once, and the else
executes so both types of lines format correctly except for the multiple
identifiers in the non-icdm lines
I could still use some help with that bit, please.
regards, Richard
On Wed, Jun 3, 2015 at 4:13 PM, richard kappler
wrote:
> I was trying to keep it simple, you'd think by now I'd know better. My
> fault and my apology.
>
> It's definitely not all dates and times, the data and character types
> vary. This is the output from my log parser script which you helped on the
> other day. there are essentially two types of line:
>
> Tue Jun 2 10:22:42 2015 name="SE">SE201506012200310389PS01CT1407166S0011.40009.7.6IN
> 0018.1LB000258]C10259612019466862270088094]L0223PDF
> Tue Jun 2 10:22:43 2015 name="SE">SE0389icdim01307755C0038.20033.20012.0IN10
> 0032]C10259612804038813568089577
>
> I have to do several things:
> the first type can be of variable length, everything after the ] is an
> identifier that I have to separate, some lines have one, some have more
> than one, variable length, always delimited by a ]
> the second type (line 2) doesn't have the internal datetime stamp, so I
> just need to add 14 x's to fill in the space where that date time stamp
> would be.
>
> and finally, I have to break these apart and put a descriptor with each.
>
> While I was waiting for a response to this, I put together a script to
> start figuring things out (what could possibly go wrong?!?!?! :-) )
>
> and I can't post the exact script but the following is the guts of it:
>
> f1 = open('unformatted.log', 'r')
> f2 = open('formatted.log', 'a')
>
> for line in f1:
> for tag in ("icdm"):
> if tag in line:
> newline = 'log datestamp:' + line[0:24] # + and so on to
> format the lines with icdm in them including adding 14 x's for the missing
> timestamp
> f2.write(newline) #write the formatted output to the new log
> else:
> newline = 'log datestamp:' + line[0:24] # + and so on to
> format the non-icdm lines
> f2.write(newline)
>
> The problems are:
> 1. for some reason this iterates over the 24 line file 5 times, and it
> writes the 14 x's to every file, so my non-icdm code (the else:) isn't
> getting executed. I'm missing something basic and obvious but have no idea
> what.
> 2. I still don't know how to handle the differences in the end of the
> non-icdm files (potentially more than identifier ] delimited as described
> above).
>
> regards, Richard
>
> On Wed, Jun 3, 2015 at 3:53 PM, Alan Gauld
> wrote:
>
>> On 03/06/15 20:10, richard kappler wrote:
>>
>>> for formatting a string and adding descriptors:
>>>
>>> test = 'datetimepart1part2part3the_rest'
>>>
>>
>> If this is really about parsing dates and times have
>> you looked at the datetime module and its parsing/formatting
>> functions (ie strptime/strftime)?
>>
>> Can I stop using position numbers and start looking for specific
>>> characters
>>> (the delimiter) and proceed to the end (which is always a constant
>>> string
>>> btw).
>>>
>>
>> The general answer is probably to look at regular expressions.
>> But they get messy fast so usually I'd suggest trying regular
>> string searches/replaces and splits first.
>>
>> But if your pattern is genuinely complex and variable then
>> regex may be the right solution.
>>
>> But if its dates check the strptime() functions first.
>>
>>
>> --
>> Alan G
>> Author of the Learn to Program web site
>> http://www.alan-g.me.uk/
>> http://www.amazon.com/author/alan_gauld
>> Follow my photo-blog on Flickr at:
>> http://www.flickr.com/photos/alangauldphotos
>>
>>
>> ___
>> Tutor maillist - Tutor@python.org
>> To unsubscribe or change subscription options:
>> https://mail.python.org/mailman/listinfo/tutor
>>
>
>
>
> --
>
> Windows assumes you are an idiot…Linux demands proof.
>
--
Windows assumes you are an idiot…Linux demands proof.
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] string delimiters
On 03/06/15 21:23, richard kappler wrote:
hold the phone
I have no idea why it worked, would love an explanation, but I changed
my previous test script by eliminating
for tag in ("icdm"):
This loops over the string assigning the characters i,c,d and m to tag
if 'icdm' in line:
This checks if the 4 character string 'icdm' is in the line.
Completely different.
--
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] string delimiters
On 03/06/15 21:13, richard kappler wrote:
I was trying to keep it simple, you'd think by now I'd know better. My
fault and my apology.
It's definitely not all dates and times, the data and character types
vary. This is the output from my log parser script which you helped on
the other day. there are essentially two types of line:
Tue Jun 2 10:22:42 2015name="SE">SE201506012200310389PS01CT1407166S0011.40009.7.6IN
0018.1LB000258]C10259612019466862270088094]L0223PDF
Tue Jun 2 10:22:43 2015name="SE">SE0389icdim01307755C0038.20033.20012.0IN10
0032]C10259612804038813568089577
I have to do several things:
the first type can be of variable length, everything after the ] is an
identifier that I have to separate, some lines have one, some have
more than one, variable length, always delimited by a ]
So why not just split by ']'?
identifiers = line.split(']')[1:] # lose the first one
and finally, I have to break these apart and put a descriptor with each.
Nope. I don't understand that.
Break what apart? and how do you 'put a descriptor with each'?
What is a descriptor for that matter?!
While I was waiting for a response to this, I put together a script to
start figuring things out (what could possibly go wrong?!?!?! :-) )
and I can't post the exact script but the following is the guts of it:
f1 = open('unformatted.log', 'r')
f2 = open('formatted.log', 'a')
for line in f1:
for tag in ("icdm"):
if tag in line:
newline = 'log datestamp:' + line[0:24] # + and so on to
format the lines with icdm in them including adding 14 x's for the
missing timestamp
f2.write(newline) #write the formatted output to the new log
else:
newline = 'log datestamp:' + line[0:24] # + and so on to
format the non-icdm lines
f2.write(newline)
So this checks each line for the 4 tags: i,c,d and m.
if the tag is in the line it does the if clause, including writing to f2
If the tag is not in the line it does the else which also writes to f2.
So you always write 4 lines to f2. Is that correct?
The problems are:
1. for some reason this iterates over the 24 line file 5 times, and it
writes the 14 x's to every file, so my non-icdm code (the else:) isn't
getting executed. I'm missing something basic and obvious but have no
idea what.
That's not what I'd expect. I'd expect it to write 4 lines out for every
input line.
What gets written depending on however many of the 4 tags are found in
the line.
Since we only have partial code we don't know what the formatted lines
look like.
2. I still don't know how to handle the differences in the end of the
non-icdm files (potentially more than identifier ] delimited as
described above).
I'm not clear on this yet either.
I suspect that once you clarify what you are trying to do you will know
how to do it...
--
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] string delimiters
figured that out from your last post, and thank you, now I understand how
that works. I thought I was looking for the entire string, not each
character. That bit all makes sense now.
A descriptor is, for example, for the following part of a string '0032.4'
the descriptor would be weight, so the formatted output would be
weight:0032.4, and so on. each bit of the strings in the post where I
provided the two examples has specific meaning, and I have to parse the
lines so that I add a descriptor (okay, bad word, what should I use?) to
each bit of data from the line.
At the moment I'm doing it by position, which is, I'm sure, a really bad
way to do it, but I need this quickly and don't know enough to know if
there is a better way. I have to parse and output the entire line, but
there are, as I said, two 'types' of string and some are variable in
length. I'm eager for direction. What other information would better help
explain?
regards, Richard
On Wed, Jun 3, 2015 at 4:31 PM, Alan Gauld
wrote:
> On 03/06/15 21:23, richard kappler wrote:
>
>> hold the phone
>>
>> I have no idea why it worked, would love an explanation, but I changed my
>> previous test script by eliminating
>>
>> for tag in ("icdm"):
>>
>
> This loops over the string assigning the characters i,c,d and m to tag
>
> if 'icdm' in line:
>>
>
> This checks if the 4 character string 'icdm' is in the line.
> Completely different.
>
>
> --
> Alan G
> Author of the Learn to Program web site
> http://www.alan-g.me.uk/
> http://www.amazon.com/author/alan_gauld
> Follow my photo-blog on Flickr at:
> http://www.flickr.com/photos/alangauldphotos
>
>
--
Windows assumes you are an idiot…Linux demands proof.
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] string delimiters
On 2015-06-03 12:53, Alan Gauld wrote: ... If this is really about parsing dates and times have you looked at the datetime module and its parsing/formatting functions (ie strptime/strftime)? I asssume strftime gets its name from 'string from time.' What about strptime? How did that get its name? ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] string delimiters
Perhaps the better way for me to have asked this question would have been: How can I find the location within a string of every instance of a character such as ']'? regards, Richard On Wed, Jun 3, 2015 at 5:16 PM, Alex Kleider wrote: > On 2015-06-03 12:53, Alan Gauld wrote: > ... > >> If this is really about parsing dates and times have >> you looked at the datetime module and its parsing/formatting >> functions (ie strptime/strftime)? >> > > I asssume strftime gets its name from 'string from time.' > What about strptime? How did that get its name? > > > ___ > Tutor maillist - Tutor@python.org > To unsubscribe or change subscription options: > https://mail.python.org/mailman/listinfo/tutor > -- Windows assumes you are an idiot…Linux demands proof. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] string delimiters
richard kappler wrote:
> Perhaps the better way for me to have asked this question would have been:
>
> How can I find the location within a string of every instance of a
> character such as ']'?
>>> import re
>>> s = "alpha]beta]gamma]delta"
>>> [m.start() for m in re.finditer(r"]", s)]
[5, 10, 16]
But do you really need these locations? Why not just split() as in
>>> s.split("]")
['alpha', 'beta', 'gamma', 'delta']
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] string delimiters
On 03/06/2015 22:16, Alex Kleider wrote: On 2015-06-03 12:53, Alan Gauld wrote: ... If this is really about parsing dates and times have you looked at the datetime module and its parsing/formatting functions (ie strptime/strftime)? I asssume strftime gets its name from 'string from time.' What about strptime? How did that get its name? 'f' for format, 'p' for parse, having originally come from plain old C. More here https://docs.python.org/3/library/datetime.html#strftime-and-strptime-behavior As Alan has hinted at earlier in this thread, if you're using dates and/or times it's certainly far easier to use the built-in functions rather than try to roll your own. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] string delimiters
On 2015-06-03 15:13, Mark Lawrence wrote: 'f' for format, 'p' for parse, having originally come from plain old C. More here https://docs.python.org/3/library/datetime.html#strftime-and-strptime-behavior So I was wrong about the 'f' as well as having no clue about the 'p'! Thank you very much for clearing that up for me. cheers, Alex ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] dictionary of lists
>
>> Resetting execution engine
>> Running
>> C:\Users\cs062x\Desktop\python\projects\PanHandle\PanHandle\PanHandle.py
>> The Python REPL process has exited
>
>
> That's slightly unusual. How are you running this?
>
I am running it with Microsoft Visual Studio Community 2013 using
Python Tools for Visual Studio
>
> Now, what I don't know, is what you are trying to do.
> Are you trying to append the float to the list?
> Or to replace the list with the float?
> Or to add the float to the value of the first(or last?)
> element in the list - if it exists
> (and if it doesn't? Then what?)
>
>
Although I am certain it is not very efficient I was able to
accomplish what I wanted with the following code I wrote:
import os
import pprint
import csv
from collections import defaultdict
print_map = {'MOU':0, 'Call_Att':1, 'Device':2}
header = ['IMEI','MOUs','Call_Att','Device']
path = 'C:/Users/cs062x/Desktop/Panhandle'
os.chdir(path)
running_MOU = {}
call_attempts = {}
d = defaultdict(list)
for fname in os.listdir('.'):
with open (fname) as csvfile:
spamreader = csv.reader(csvfile, delimiter=',', quotechar='|')
next(spamreader)
for row in spamreader:
if row[8]:
device = row[36]
Elapsed_Mins = float(row[7])
IMEI = row[8].replace("'", "")
if IMEI in running_MOU.keys():
running_MOU[IMEI] += Elapsed_Mins
else:
running_MOU[IMEI] = Elapsed_Mins
if IMEI in call_attempts.keys():
call_attempts[IMEI] += 1
else:
call_attempts[IMEI] = 1
# if key matches append mou else append 0.
d[IMEI] = [running_MOU[IMEI]]
d[IMEI].append([call_attempts[IMEI]])
d[IMEI].append([device])
print ",".join(header)
for k,v in sorted(d.items()):
print k, ",", d[k][print_map['MOU']],",",
d[k][print_map['Call_Att']][0],",", d[k][print_map['Device']][0]
print "complete"
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] string delimiters
On 03/06/15 22:16, Alex Kleider wrote: On 2015-06-03 12:53, Alan Gauld wrote: ... If this is really about parsing dates and times have you looked at the datetime module and its parsing/formatting functions (ie strptime/strftime)? I asssume strftime gets its name from 'string from time.' What about strptime? How did that get its name? f = format - for creating date/time strings p = parse - for extracting date/time fierlds from a string -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] string delimiters
On 03Jun2015 14:16, Alex Kleider wrote: On 2015-06-03 12:53, Alan Gauld wrote: ... If this is really about parsing dates and times have you looked at the datetime module and its parsing/formatting functions (ie strptime/strftime)? I asssume strftime gets its name from 'string from time.' What about strptime? How did that get its name? No, they both come from the standard C library functions of the same names, being "(f)ormat a time as a string" and "(p)arse a time from a string". The shape of the name is because they're "str"ing related functions, hence the prefix. See "man 3 strptime" and "man 3 strftime". Cheers, Cameron Simpson A program in conformance will not tend to stay in conformance, because even if it doesn't change, the standard will. - Norman Diamond ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] string delimiters
On 03Jun2015 17:35, richard kappler wrote:
Perhaps the better way for me to have asked this question would have been:
How can I find the location within a string of every instance of a
character such as ']'?
With the str.find method!
s = 'a]b]c'
pos = s.find(']')
while pos >= 0:
print("pos =", pos)
pos = s.find(']', pos + 1)
Obviously you could recast that as a generator funciton yielding positions for
general purpose use.
Cheers,
Cameron Simpson
Serious error.
All shortcuts have disappeared.
Screen. Mind. Both are blank.
- Haiku Error Messages
http://www.salonmagazine.com/21st/chal/1998/02/10chal2.html
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
[Tutor] line iteration in a file
Figured out the string delimiters problem, thanks for all the help. Now I've run into another. I've used the re.finditer that I think it was Peter suggested. So I have: for line in file: s = line t = [m.start() for m in re.finditer(r"]", s)] q = len(t) which works fine, in testing it finds the number and position of the ]'s in any line I throw at it. I then wrote a series of if/elif statements based on q, in other words if q == 1: do something elif q == 2: do something else elif q == 3: do a third thing else: pass as I looked through enough example to figure out that the most ]'s I can have is 3, but the pass is there just in case. I keep getting a list index out of range error, and my best guess is that it's because t and q are set on the first line read, not each line read, is that right? If not, what might be the problem and either way, how do I fix it? regards, Richard who is proving to his Linux box that he is an idiot pretty regularly -- Windows assumes you are an idiot…Linux demands proof. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] line iteration in a file
On 03Jun2015 22:37, richard kappler wrote: Figured out the string delimiters problem, thanks for all the help. Now I've run into another. I've used the re.finditer that I think it was Peter suggested. So I have: for line in file: s = line t = [m.start() for m in re.finditer(r"]", s)] q = len(t) which works fine, in testing it finds the number and position of the ]'s in any line I throw at it. I then wrote a series of if/elif statements based on q, in other words if q == 1: do something elif q == 2: do something else elif q == 3: do a third thing else: pass as I looked through enough example to figure out that the most ]'s I can have is 3, but the pass is there just in case. I keep getting a list index out of range error, and my best guess is that it's because t and q are set on the first line read, not each line read, is that right? If not, what might be the problem and either way, how do I fix it? Please post a self contained example (i.e. small complete code, not snippets) and a transcribe of the full error message with stack backtrace. What you have above is not enough to figure out what is going wrong. If what you display above is accurate then t and q are set for every line read. Another remark, what is the use of your "else: pass" code? Normally one would put some action here, such as raising an exception for the unhandled value or issuing a warning. Cheers, Cameron Simpson My computer always does exactly what I tell it to do but sometimes I have trouble finding out what it was that I told it to do. - Dick Wexelblat ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
