Re: [Tutor] Networking
On Fri, Oct 15, 2010 at 3:06 PM, Evert Rol wrote: > The very last example on http://docs.python.org/library/socketserver.html may > help you. > Or perhaps the asyncore/asynchat modules. Alternatively Twisted (1) or circuits (2) cheers James 1. http://twistedmatrix.com/trac/ 2. http://bitbucket.org/prologic/circuits/wiki/Home -- -- James Mills -- -- "Problems are solved by method" ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Statistic-Program Problems! Please Help Quickly!
On Fri, 15 Oct 2010 03:09:57 pm Colleen Glaeser wrote: > Now, my functions for X, Y, P, and Q are correct, but I have a couple > of problems when it comes to continuing. First of all, despite what > my teacher has told me, my method for trying to multiply X,Y,P, and > Q's results in the functions for B and M are not working. In case it wasn't clear from my earlier email, this is because the functions X, Y, P and Q don't return anything, they just print their result. > I'm not > sure if there is a way to make functions into variables or how to > solve this problem. > > Second, I am confused as to what my teacher means to do when it comes > to inputting different values of x. The formula for a straight line looks like: y = m*x + b where x and y are variables, and m and b are the slope and intercept of the line. m and b are parameters: you set them once, to choose a line, and then calculate with the variables. When you change the parameters, you change the line. You pick m and b (say, m=2 an b=3) to get a formula y = 2*x + 3, and then you can put in different values of x to get different values of y: x 2*x+3 = y 0 2*0+3 = 3 1 2*1+3 = 5 2 2*2+3 = 7 etc. In this assignment, the values for m and b aren't chosen arbitrarily, but calculated from the data sets using the functions that you called X(), Y(), P() etc. Bringing this back to Python, you first have to calculate your regression parameters m and b: m = ... # something goes here b = ... # something else goes here Then you need to turn them into a function that takes x as an argument and returns y. How to do this? I know earlier I suggested that global variables should be avoided. I stand by that, but in this case I think that here you should use globals. There are alternatives, but they are more advanced techniques, and at this early stage keeping it simple is a good thing. So I might write a function like this: def line(x): return m*x+b and use it like this: y = line(5) # try it with x=5 This returns the value of y that goes with the value of x=5, for the regression line you calculated earlier. Then you move on to the second set of data: m = ... # something goes here b = ... # something else goes here y = line(95) which calculates the y that goes with x=95 for the new regression line. -- Steven D'Aprano ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] join question
Hello,
Solved it.
Problem was that I used fileinfo instead of getinfo.
Thanks for the help.
Roelof
Date: Thu, 14 Oct 2010 18:55:50 -0400
> From: joel.goldst...@gmail.com
> To: tutor@python.org
> Subject: Re: [Tutor] join question
>
>
>
> On Thu, Oct 14, 2010 at 6:43 PM, Alan Gauld
> > wrote:
>
> "Roelof Wobben" > wrote
>
>
>
> print ''.join([zf.getinfo('%s.txt' % p).comment for p in zpp])
>
> So I thought that this would be the same :
>
> for p in zpp:
> test = zf.getinfo(p).comment
> print ''.join(test)
>
> But it seems not to work
>
> Can anyone explain why not ?
>
> Because it's not the same. test in your version is only a single item.
> In the original its a list of items. So to write similar code
> explicitly you need:
>
> lst = []
>
> for p in zpp:
> test = zf.getinfo(p).comment
> lst.append(test)
> print ''.join(lst)
>
> HTH,
>
> Alan G.
>
>
> There are two problems. Alan shows the solution to the joining of each
> element of the list, but also you need to have the correct argument for
> getinfo. In your first example you append ".txt" to p, but in the loop
> example you fail to append it. See my earlier post
>
> --
> Joel Goldstick
>
>
> ___ Tutor maillist -
> Tutor@python.org To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor
>
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Statistic-Program Problems! Please Help Quickly!
On Thu, Oct 14, 2010 at 10:11 PM, Colleen Glaeser wrote: > BTW, the error message my program gives me for the B and M functions is: > > Traceback (most recent call last): > File "I:\Lab 7 wierd stat data.py", line 49, in > B() > File "I:\Lab 7 wierd stat data.py", line 44, in B > > ((Y() * Q()) - (P() * X())) / ((6 * Q()) - (X()**2)) > TypeError: unsupported operand type(s) for *: 'NoneType' and 'NoneType' > > On Thu, Oct 14, 2010 at 11:09 PM, Colleen Glaeser > wrote: > >> Dear tutors, >> >> I am in a beginning-level computer science class in college and am running >> into problems with an assignment. >> >> The assignment is as follows: >> >> Statisticians are fond of drawing regression lines. In statistics and >> other fields where people analyze lots of data, one of the most commonly >> used regression lines is called the “least squares line.” This is the line >> that is supposed to best fit the set of data points, in the sense that it >> minimizes the squared vertical distances between the points and the line. >> Why this should be a good fit is beyond the scope of this assignment. >> >> Presume that you have a collection of n two-dimensional data points. I’ll >> give it as a list of lists, where each of the lists inside represents one >> data point. >> >> Data :[ [x1, y1], [x2, y2], [x3, y3], …, [xn, yn]] >> >> Compute the following >> >> The regression line is then given by >> >> where m and b may be obtained by >> >> and >> >> Your task is to compute the m and b (slope and intercept, respectively) >> for a set of data. You have to analyze the data as given, not count or >> add anything yourself. Your program should do everything, even figure >> out how many data points there are. >> >> Here’s your data: >> >> First set: [ [3, 1], [4, 3], [6, 4], [7, 6], [8, 8], [9, 8] ] >> >> Second set: [ [63, 11], [22, 7.5], [63, 11], [28, 10], [151, 12], [108, >> 10], [18, 8], [115, 10], [31,7], [44, 9] ] >> >> Find m and b, then calculate an estimate for x = 5 using the first data >> set. That is, plug in 5 for x and see what y you get. For the second >> set, try x = 95. >> >> Turn in: code, m, b, and the estimates for both data sets. >> >> >> >> >> *** >> >> There’s an easy way to walk through the data and extract the values you >> need. Use a for loop. Try this: >> >> for item in data: >> >> [x, y] = item >> >> print(x) >> >> >> ~~ >> >> For extra credit: draw a scatter plot of the data, and draw in the least >> squares line. Scale the window to fit, making it a bit wider and higher >> than the data requires, so that some of the points are near but not on the >> edges of the window. Then sketch in the regression line. Note that you >> should calculate the window size based on the data – don’t set them >> yourself; find the max and min values for x and y. You can print the >> scatter plot, or point me toward your web page. In any case, show me the >> code. >> >> >> >> So far, my program is as follows: >> >> Data = [[3,1],[4,3],[6, 4],[7, 6],[8, 8],[9, 8]] >> >> def X(): >> accX = 0 >> for item in Data: >> [x,y] = item >> >> accX = accX + x >> print (accX) >> >> >> def Y(): >> accY = 0 >> for item in Data: >> [x,y] = item >> >> accY = accY + y >> print (accY) >> >> def P(): >> accXY = 0 >> for item in Data: >> [x,y] = item >> >> accXY = accXY + (y*x) >> print (accXY) >> >> def Q(): >> accX2 = 0 >> for item in Data: >> [x,y] = item >> >> accX2 = accX2 + (x**2) >> print (accX2) >> >> X() >> Y() >> P() >> Q() >> >> >> >> def B(): >> ((Y() * Q()) - (P() * X())) / ((6 * Q()) - (X()**2)) >> >> def M(): >> ((Y() * Q()) - (P() * X())) / (X() * Q()) >> >> B() >> M() >> >> Now, my functions for X, Y, P, and Q are correct, but I have a couple of >> problems when it comes to continuing. First of all, despite what my teacher >> has told me, my method for trying to multiply X,Y,P, and Q's results in the >> functions for B and M are not working. I'm not sure if there is a way to >> make functions into variables or how to solve this problem. >> >> Second, I am confused as to what my teacher means to do when it comes to >> inputting different values of x. >> >> Find m and b, then calculate an estimate for x = 5 using the first data >> set. That is, plug in 5 for x and see what y you get. For the second >> set, try x = 95. >> >> Turn in: code, m, b, and the estimates for both data sets. >> >> >> I mean, I know I need to calculate the line of best fit for the data sets >> using B and M, but what in the world is x supposed to do and where does it >> go? How do I program this? This is especially harder since I've never >> taken a proper stat class befo
Re: [Tutor] Converting from unicode to nonstring
Ok, Let me restate and hopefully further clarify. 1. I have a field for a wxpython app using matplotlib to display 2. I have a sqlite3 db which I'm retrieving information from 3. The sqlitle data is returned as unicode: u'field' 4. The portion of the matplotlib code is filled in, in a for x in y: 5. in plot(self.plot), self.plot is the variable I'm using from the unicoded db field comes in from sqlite as u'[1,2,3,4]', which places a string in quotes in that variables place: plot(u'[1,2,3,4]') 6. the plot(eval(self.plot)), changes the variable from the u'[1,2,3,4]' to just [1,2,3,4] 7 As stated somewhere above, the float error has nothing to do with the probel, only the fact that it was used as if I had placed '' around the necessary data from the db field. 8. If anyone has a way better than eval to convert the u'field' when replacing a variable so that self.plot = [1,2,3,4] instead of self.plot = u'[1,2,3,4]' Let me know, meanwhile I'll be reviewing the replies more thoroughly, now that I've had a nap. Thanks, David ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Converting from unicode to nonstring
On Thu, Oct 14, 2010 at 3:02 PM, Sander Sweers wrote: > On 14 October 2010 20:29, David Hutto wrote: >> Actually, I needed it to be converted to something without a string >> attached to it. See a post above, and it was fixed by eval(), > > Using eval is a big security risk and is generally not recommended for > any production code. What do you think eval() return for your string? > A tuple of ints which for your use case serves the same purpose as > using the list comprehension. > > If you really want (you really don't) to use eval() then at least use > the safe one from the ast mode. > from ast import literal_eval literal_eval(u'1,2,3,4') > (1, 2, 3, 4) eval(u'1,2,3,4') > (1, 2, 3, 4) > > As you can see for your use case both return a tuple of ints. > > Greets > Sander > After reading http://bugs.python.org/issue7935, I'll probably use literal_eval. Thanks for pointing literal out. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Converting from unicode to nonstring
On Thu, Oct 14, 2010 at 2:58 PM, Adam Bark wrote:
> On 14/10/10 19:29, David Hutto wrote:
>>
>> On Thu, Oct 14, 2010 at 2:19 PM, Sander Sweers
>> wrote:
>>
>>>
>>> On 14 October 2010 16:14, David Hutto wrote:
>>>
(u'graph1', u'Line', u'222', u'BLUE', u'1,2,3,4', u'True', u'0,5,0,10')
Which is a tuple of unicode strings. From this I
need to place portions of the tuple into other fields,
but not as unicode strings, but literals no ''.
For example if I have the following line:
self.lines = self.newplot.plot([1,2,3,4])
>>>
>>> So you want convert string u'1,2,3,4' to a list of ints [1,2,3,4]?
>>> Then the below will work.
>>>
>>> [int(n) for n in u'1,2,3,4'.replace(',', '')]
>>>
>>>
>>
>> Actually, I needed it to be converted to something without a string
>> attached to it. See a post above, and it was fixed by eval(),
>>
>> Thanks though. And I'm sure at some point this morning in a moment of
>> frustration rather than logic, I tried your approach.
>>
>
> What do you mean by "without a string attached to it"?
> Also using eval could be dangerous unless you're sure the stuff coming out
> of your dbase is safe.
>
At this point I create the db 'manually', so the dat in their is what
I put in. At a later time I'll use other db's, so I guess it's better
to make sure it works under all circumstances.
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Converting from unicode to nonstring
> In which case the code Sanders sent you is a much more sensible way of > recovering your data. Evalling the string u'plot' still doesn't make much > sense to me though. But it's the only thing working correctly though, so until I have a better solution, I'm forced to deal with any bugs in interpreting the data coming in that may occur, and account for those after the fact for now. I think I may have been overestimating the quality of > your code. > Probably so. You should see the whole thing, I'll let yall tear that apart later though =) ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Converting from unicode to nonstring
On Thu, Oct 14, 2010 at 6:56 PM, Alan Gauld wrote:
>
> "David Hutto" wrote
>
>> In other words I needed (1,2,3,4) not u'(1,2,3,4)' to be inserted
>> for variable self.plot
>
> You appear to be making this much more difficult than it needs to be.
> The values you retrieve from the database are strings (forget about
> the unicode aspect its not really relevant here) just as if you had
> used raw_input to read them from the user.
>
> How would you convert a string received from raw_input() to a series
> of numbers? Would you have used eval() or just split the string and
> called int() or float()? Something like:
>
> inp = raw_input("Enter a string of numbers separated by commas")
> nums = [int(n) for n in inp.split(',')]
>
> eval() should always be considered a last resort because of the
> potential for causing damage and the extremely obscure errors
> it can throw up with badly formed input.
>
> I think you are letting the unicode representation spook you into doing
> things in a way you wouldn't if it was normal raw_input you were using.
Not spook, but I couldn't think of another way of placing it into a
variable for a function within the script, without it using the
literal value from the db, (without generating a blank.py file, and
rewriting the whole class each time for the changes in the value), and
use it in the code without it being a literally evaluated u'string'
the way it came from the db
>
> HTH,
>
> --
> Alan Gauld
> Author of the Learn to Program web site
> http://www.alan-g.me.uk/
>
>
> ___
> Tutor maillist - tu...@python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor
>
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Converting from unicode to nonstring
On Thu, Oct 14, 2010 at 7:00 PM, Alan Gauld wrote: > > "David Hutto" wrote > >> it's not necessary to worry about insertion of data other than my own >> inputs. > > But can you be sure that you won't accidentally mistype something > that eval can read as valid code but that does something unexpected > - even if its only throw an apparently bizarre error dump at you... probably not, because I haven't had much experience with using eval. > > Relying on eval() reading your database content as if it were Python > code - which is what it does - is a risky strategy. > > But, heh, it's your data, your project, so as long as you understand > the risks then it's your decision. :-) It's not that I want faulty data, it was the only thing thus far I'd tried that worked under the generalized circumstances I'm testing it under. > > -- > Alan Gauld > Author of the Learn to Program web site > http://www.alan-g.me.uk/ > > > > > ___ > Tutor maillist - tu...@python.org > To unsubscribe or change subscription options: > http://mail.python.org/mailman/listinfo/tutor > ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Converting from unicode to nonstring
On Thu, Oct 14, 2010 at 10:09 PM, Steven D'Aprano wrote: > On Fri, 15 Oct 2010 04:43:46 am David Hutto wrote: > >> Fixed with: >> >> self.lines = self.newplot.plot(eval(self.plot)) > > Anytime you use eval, chances are that it isn't fixed at all, but just > badly and dangerously papered over. > > I really, really wish that eval and exec weren't built-ins. They're > powerful but dangerous and slow, and making them built-ins just > encourages newbies to use them inappropriately. If inappropriately means that it works as expected:). But seriously, I haven't found another way, than in thought, about what alan said about raw_input-running it through their instead, but I haven't tried yet this morning. > > > -- > Steven D'Aprano > ___ > Tutor maillist - tu...@python.org > To unsubscribe or change subscription options: > http://mail.python.org/mailman/listinfo/tutor > ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Converting from unicode to nonstring
On Fri, Oct 15, 2010 at 6:26 AM, David Hutto wrote:
> Ok, Let me restate and hopefully further clarify.
>
> 1. I have a field for a wxpython app using matplotlib to display
>
> 2. I have a sqlite3 db which I'm retrieving information from
>
> 3. The sqlitle data is returned as unicode: u'field'
>
> 4. The portion of the matplotlib code is filled in, in a for x in y:
>
> 5. in plot(self.plot), self.plot is the variable I'm using from the
> unicoded db
> field comes in from sqlite as u'[1,2,3,4]', which places a string in quotes
> in
> that variables place:
>
> plot(u'[1,2,3,4]')
>
> 6. the plot(eval(self.plot)), changes the variable from the u'[1,2,3,4]'
> to just [1,2,3,4]
>
> 7 As stated somewhere above, the float error has nothing to do with
> the probel, only the fact that it was used as if I had placed ''
> around the necessary data from the db field.
>
> 8. If anyone has a way better than eval to convert the u'field' when
> replacing a variable so that
>
> self.plot = [1,2,3,4]
>
> instead of
>
> self.plot = u'[1,2,3,4]'
>
>
> Let me know, meanwhile I'll be reviewing the replies more thoroughly,
> now that I've had a nap.
>
>
> Thanks,
> David
>
This will do it
> So you want convert string u'1,2,3,4' to a list of ints [1,2,3,4]?
> Then the below will work.
>
> [int(n) for n in u'1,2,3,4'.replace(',', '')]
>
> Greets
> Sander
>
or this:
>>> s = '1, 200 , -3,4' # or whatever
>>> [int(x) for x in s.split(',')]
[1, 200, -3, 4]
or this:
To take a string of comma separated integers and convert to a list of ints:
>>> x = u'1,2,3,4'
>>> y = x.split(',')
>>> z = [int(f) for f in y]
>>> z
[1.0, 2.0, 3.0, 4.0]
>>>
--
Joel Goldstick
You can forget about the u' in front. Its a directive to the interpreter
that the following string is in unicode.
In your case u'1,2,3,4' is the same as '1,2,3,4'.
All of the above examples do this:
1. split the string up into 4 different strings in a tuple by removing the
commas :: ('1', '2', '3', '4')
2. convert the individual string vales to integers and put them in a list ::
[1, 2, 3, 4]
--
Steven D'Aprano
Joel Goldstick
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Converting from unicode to nonstring
David Hutto wrote:
> Hey Buddy Pals,
?
> I receive the following output from a sqlite db
>
> (u'graph1', u'Line', u'222', u'BLUE', u'1,2,3,4', u'True', u'0,5,0,10')
How did the string u"1,2,3,4" get into the database in the first place?
The sqlite3 module offers a mechanism to convert data from and to Python
(semi-)transparently:
import sqlite3
import json
sqlite3.register_adapter(list, json.dumps)
sqlite3.register_converter("list", json.loads)
db = sqlite3.connect(":memory:", detect_types=sqlite3.PARSE_DECLTYPES)
cursor = db.cursor()
cursor.execute("create table data (value list)")
cursor.execute("insert into data values (?)", ([11,22,33],))
for row in cursor.execute("select value from data"):
print row
See also:
http://docs.python.org/library/sqlite3.html#converting-sqlite-values-to-
custom-python-types
Peter
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Converting from unicode to nonstring
On 2:59 PM, David Hutto wrote:
Ok, Let me restate and hopefully further clarify.
1. I have a field for a wxpython app using matplotlib to display
2. I have a sqlite3 db which I'm retrieving information from
3. The sqlitle data is returned as unicode: u'field'
4. The portion of the matplotlib code is filled in, in a for x in y:
5. in plot(self.plot), self.plot is the variable I'm using from the unicoded db
field comes in from sqlite as u'[1,2,3,4]', which places a string in quotes in
that variables place:
plot(u'[1,2,3,4]')
Your point #5 shows that you still have the wrong handle on what a
literal string is.
When you write in code
mystring = u"abc"
The u and the quote symbols are *not* in "the variables place". The
"variables place" contains a count, three characters, and some meta
information. The time you need the quotes are when you want to specify
those three characters in your source code. The time you get the quotes
back is when you use repr(mystring) to display it. If you useprint
mystring you don't see any quotes or 'u', nor do you when you call the
function str(mystring).
So I'll try to explain some terminology. When you run the source code
mystring = u"abc"
the interpreter (with help from the compiler) builds an object of type
string, with contents of those three letters. That object is bound to
the name mystring. Nowhere is a u or a quote to be found.
Now, the question is how to turn that string object into a list, since
apparently plot() is looking for a list. Judging from the error
message, it may also be happy to take a float. So it'd be good to find
out just what types of objects it's willing to accept.
One of the first respondents on the thread showed how to convert this
particular string into a list, but he made the implicit assumption that
the values in the list were single-digit integer values. I haven't seen
you ever define what the possible values are. But guessing that they
are supposed to be a list containing a nonzero number of floats,
separated by commas, you could do something like (untested):
result = [float(strvalue) for strvalue in mystring.split(",")]
or, more straightforward:
strvalues = mystring.split(",") # get a list of strings,
split by the comma
result = []
for strvalue in strvalues:
result.append(float(strvalue)) #convert to float
Notice, nothing about quotes, or unicode anywhere in the logic.
DaveA
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Converting from unicode to nonstring
On Fri, 15 Oct 2010 09:26:48 pm David Hutto wrote:
> Ok, Let me restate and hopefully further clarify.
>
> 1. I have a field for a wxpython app using matplotlib to display
> 2. I have a sqlite3 db which I'm retrieving information from
Both of those points are irrelevant.
> 3. The sqlitle data is returned as unicode: u'field'
Semi-relevant. What's important is that you have data as strings. It
could be coming from a text file:
data = open("my data.txt").read()
or from the user:
data = raw_input("Enter some data: ")
or any other source that gives a string. It makes no difference where it
comes from, the only thing that is important is that it is a string.
> 4. The portion of the matplotlib code is filled in, in a for x in y:
I don't really understand this sentence. What do you mean, portion of
code is filled in?
But in any case, it's irrelevant that you are doing something in a for
loop. For loop, while loop, using the data once only, who cares?
> 5. in plot(self.plot), self.plot is the variable I'm using from the
> unicoded db field comes in from sqlite as u'[1,2,3,4]', which places
> a string in quotes in that variables place:
>
> plot(u'[1,2,3,4]')
Your sentence above is a bit convoluted, but I *think* this is what
you're trying to say:
- you take the string (which originally comes from the database, but it
doesn't matter where it comes from) and store it in self.plot;
- you pass self.plot to a function plot(); and
- since self.plot is a string, you are passing a string to the function
plot().
Presumably this is a problem, because plot() expects a list of ints, not
a string.
> 6. the plot(eval(self.plot)), changes the variable from the
> u'[1,2,3,4]' to just [1,2,3,4]
Well, yes it does, so long as the string is perfectly formed, and so
long as it doesn't contain anything unexpected.
It is also slow, and unnecessary, and a bad habit to get into unless you
know exactly what you are doing.
> 7 As stated somewhere above, the float error has nothing to do with
> the probel, only the fact that it was used as if I had placed ''
> around the necessary data from the db field.
Huh?
> 8. If anyone has a way better than eval to convert the u'field' when
> replacing a variable so that
>
> self.plot = [1,2,3,4]
>
> instead of
>
> self.plot = u'[1,2,3,4]'
And now, finally, after a dozen or more emails and 8 points (most of
which are irrelevant!) we finally come to the real problem:
"I have a unicode string that looks like u'[1,2,3,4]' and I want to
convert it into a list [1, 2, 3, 4]. How do I do that?"
There are at least three solutions to this:
(1) eval. The benefit of eval is that it is a built-in, so you don't
have to do any programming. The downsides are that:
- it is very powerful, so it can do too much;
- it is dangerous if you can't trust the source of the data;
- because it does so much, it's a little slow;
- because it does so much, it will happily accept data that *should*
give you an error;
- on the other hand, it's also quite finicky about what it does accept,
and when it fails, the error messages may be cryptic.
(2) ast.literal_eval. The benefit of this is that it is a standard
library function starting in Python 2.6, so all you need do is import
the ast module. The downsides are:
- it is very powerful, so it can do too much;
- because it does so much, it will happily accept data that *should*
give you an error;
- on the other hand, it's also quite finicky about what it does accept,
and when it fails, the error messages may be cryptic.
(3) Write your own converter. The benefit of this is that you can make
it as flexible or as finicky as you like. The downside is that you have
to write it. But that's not actually very hard, and we can make sure
that we get a nice error message in the event of a problem:
def str_to_list(s):
"""Convert a string that looks like a list to a list of ints."""
s = s.strip() # ignore leading and trailing spaces
if not (s.startswith("[") and s.endswith("]")):
raise ValueError("string does not look like a list")
s = s[1:-1] # throw away the [ and ]
s = s.replace(",", " ")
result = []
try:
for word in s.split():
result.append(int(word))
except ValueError:
raise ValueError("item `%s` does not look like an int" % word)
return result
>>> str_to_list(u' [1 , 2, 3, 15 , -3, 26,1, 7 ] ')
[1, 2, 3, 15, -3, 26, 1, 7]
If you pass it faulty data, it gives you a nice error message:
>>> str_to_list( u'{1:2}')
Traceback (most recent call last):
File "", line 1, in
File "", line 5, in str_to_list
ValueError: string does not look like a list
>>> str_to_list(u'[1,2,3,None]')
Traceback (most recent call last):
File "", line 1, in
File "", line 13, in str_to_list
ValueError: item `None` does not look like an int
Compared to eval and ast.literal_eval, both of which do too much:
>>> eval(u'{1:2}')
{1: 2}
>>> eval(u'[1,2,3,None]')
[1, 2, 3, None]
>>>
Re: [Tutor] Converting from unicode to nonstring
hi I faced this problem before and people of tutor helped me to solve it by
only one line
ex.
a=input_raw("your non string")## I input 1,2,3,4 5 6 7 or u[1,2,3,4]
a.replace(",", " ","[","]","u").split() ## you can put any kind of unicode
that you expect inside the " "
a=map(float, a)
print a
[1,2,3,4,5,6,7]
--
From: "Steven D'Aprano"
Sent: Friday, October 15, 2010 9:37 PM
To: "Python Tutor"
Subject: Re: [Tutor] Converting from unicode to nonstring
On Fri, 15 Oct 2010 09:26:48 pm David Hutto wrote:
Ok, Let me restate and hopefully further clarify.
1. I have a field for a wxpython app using matplotlib to display
2. I have a sqlite3 db which I'm retrieving information from
Both of those points are irrelevant.
3. The sqlitle data is returned as unicode: u'field'
Semi-relevant. What's important is that you have data as strings. It
could be coming from a text file:
data = open("my data.txt").read()
or from the user:
data = raw_input("Enter some data: ")
or any other source that gives a string. It makes no difference where it
comes from, the only thing that is important is that it is a string.
4. The portion of the matplotlib code is filled in, in a for x in y:
I don't really understand this sentence. What do you mean, portion of
code is filled in?
But in any case, it's irrelevant that you are doing something in a for
loop. For loop, while loop, using the data once only, who cares?
5. in plot(self.plot), self.plot is the variable I'm using from the
unicoded db field comes in from sqlite as u'[1,2,3,4]', which places
a string in quotes in that variables place:
plot(u'[1,2,3,4]')
Your sentence above is a bit convoluted, but I *think* this is what
you're trying to say:
- you take the string (which originally comes from the database, but it
doesn't matter where it comes from) and store it in self.plot;
- you pass self.plot to a function plot(); and
- since self.plot is a string, you are passing a string to the function
plot().
Presumably this is a problem, because plot() expects a list of ints, not
a string.
6. the plot(eval(self.plot)), changes the variable from the
u'[1,2,3,4]' to just [1,2,3,4]
Well, yes it does, so long as the string is perfectly formed, and so
long as it doesn't contain anything unexpected.
It is also slow, and unnecessary, and a bad habit to get into unless you
know exactly what you are doing.
7 As stated somewhere above, the float error has nothing to do with
the probel, only the fact that it was used as if I had placed ''
around the necessary data from the db field.
Huh?
8. If anyone has a way better than eval to convert the u'field' when
replacing a variable so that
self.plot = [1,2,3,4]
instead of
self.plot = u'[1,2,3,4]'
And now, finally, after a dozen or more emails and 8 points (most of
which are irrelevant!) we finally come to the real problem:
"I have a unicode string that looks like u'[1,2,3,4]' and I want to
convert it into a list [1, 2, 3, 4]. How do I do that?"
There are at least three solutions to this:
(1) eval. The benefit of eval is that it is a built-in, so you don't
have to do any programming. The downsides are that:
- it is very powerful, so it can do too much;
- it is dangerous if you can't trust the source of the data;
- because it does so much, it's a little slow;
- because it does so much, it will happily accept data that *should*
give you an error;
- on the other hand, it's also quite finicky about what it does accept,
and when it fails, the error messages may be cryptic.
(2) ast.literal_eval. The benefit of this is that it is a standard
library function starting in Python 2.6, so all you need do is import
the ast module. The downsides are:
- it is very powerful, so it can do too much;
- because it does so much, it will happily accept data that *should*
give you an error;
- on the other hand, it's also quite finicky about what it does accept,
and when it fails, the error messages may be cryptic.
(3) Write your own converter. The benefit of this is that you can make
it as flexible or as finicky as you like. The downside is that you have
to write it. But that's not actually very hard, and we can make sure
that we get a nice error message in the event of a problem:
def str_to_list(s):
"""Convert a string that looks like a list to a list of ints."""
s = s.strip() # ignore leading and trailing spaces
if not (s.startswith("[") and s.endswith("]")):
raise ValueError("string does not look like a list")
s = s[1:-1] # throw away the [ and ]
s = s.replace(",", " ")
result = []
try:
for word in s.split():
result.append(int(word))
except ValueError:
raise ValueError("item `%s` does not look like an int" % word)
return result
str_to_list(u' [1 , 2, 3, 15 , -3, 26,1, 7 ] ')
[1, 2, 3, 15, -3, 26, 1, 7]
If you pass it faulty data, it gives you a nice error message:
str_to_list( u'{1:2}')
Traceback
[Tutor] 'module' object has no attribute (setting a class attribute)
My intention is to set a class attribute so that any number of instantiations will have this value. the module is tmpl.py. the class is tmpl. if from the script I do this: import tmpl tmpl.tmpl.templatepath = kbLib.templatepath I get error message: 'module' object has no attribute 'templatepath' I ended up getting completely befuddled, and then realized that the problem was the right side of the assignment statement. I.E. I had a typo on the right and python was not informing me of *which* module didn't have the attribute I have two questions regarding this: 1)Am I using the correct method to set a class attribute? 2)Is there a system configuration that would cause the AttributeError exception to print out the module name. (In this cause it was 'kbLib' not 'tmpl'. thanks -- Tim tim at johnsons-web.com or akwebsoft.com http://www.akwebsoft.com ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
[Tutor] Moving my C++ code to Python?
Hi there, I've been using C/C++ for many years (python, just reading about it). I have a software written in C/C++ but considering porting most of it to python, as it seems like it's a better choice for decision making portion of the code. I'm also thinking about having a 'matlab' like interface for reading, processing, and writing. In my current C++ code, I would read data into a vector of structs (which contains other simple vectors, strings, and structs) and it can be as large as 500MB to 2GB. The data gets processed (requires random access). The result is then written out. I would like to make modules for python. The problem is that the vector of structs that is very large. First, is it possible to pass such structures around to and from python and C/C++? What would be the overhead cost of having a large structure that needs to be passed to and from the C/C++ modules? # I imagine I'd use the newly written software this way: >>> import c_stuff # my C/C++ module >>> largestuff = c_stuff.read(file) # read from disk >>> c_stuff.process(largestuff, someparams1) # processing, requires random >>> access to the vector >>> c_stuff.process(largestuff, someparams2) ... >>> c_stuff.process(largestuff, someparams1) # the whole thing may take a >>> few minutes to days >>> >>> import python_stuff # some module written in python to process the data as >>> well >>> >>> python_stuff.process(largestuff, otherparams1) # It's important that this >>> data can be read (and I hope written) by python code >>> python_stuff.process(largestuff, otherparams2) >>> >>> c_stuff.write(largestuff) #write result Thank you in advance, Paul ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Moving my C++ code to Python?
This isn't an answer to your question, but I'm doing about the same in reverse(it sounds like) as you're doing. More directly just processing data for 2-d/3d graphs using matplotlib and wxpython. So, I was wondering if you think it's similar to those ends, and good enough to let someone else look at yet? ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Moving my C++ code to Python?
On Fri, Oct 15, 2010 at 3:07 PM, David Hutto wrote: > This isn't an answer to your question, but I'm doing about the same in > reverse(it sounds like) as you're doing. More directly just processing > data for 2-d/3d graphs using matplotlib and wxpython. So, I was I'm afraid it is a different problem. However, matplotlib and xwpython seem like good tools I may use, once I know my porting is possible (and hopefully easy). > wondering if you think it's similar to those ends, and good enough to > let someone else look at yet? Not sure what you mean by "someone else look at yet?" Thanks, Paul ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Moving my C++ code to Python?
"Paul" wrote I would like to make modules for python. The problem is that the vector of structs that is very large. First, is it possible to pass such structures around to and from python and C/C++? Yes, Try looking at SWIG for one approach to making C/C++ code accesible to Python which may work for you What would be the overhead cost of having a large structure that needs to be passed to and from the C/C++ modules? I'm no expert so can't say for sure but since Pythion tends to operate with references I'd hope not too much. HTH, -- Alan Gauld Author of the Learn to Program web site http://www.alan-g.me.uk/ ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Moving my C++ code to Python?
SWIG supports opaque pointers that you can pass into and out of Python without any problems. Working with SWIG isn't that bad for basic stuff, although it can get complicated if you need to have it interpret exotica like references, structures, or such for Python. Having a good C++ background should cut a lot of the difficulty out of that, though. The documentation is pretty good, although finding what you need can involve a lot of scrolling around in long web pages. http://www.swig.org/ If you are going to use a command line interface, you might check out these modules from the standard library: http://docs.python.org/library/cmd.html http://docs.python.org/library/shlex.html You might also see if numpy can replace some of your C++ code. It's fast, and integrated into Python. http://numpy.scipy.org/ Cheers On Friday 15 October 2010, Paul wrote: > I would like to make modules for python. The problem is that the > vector of structs that is very large. First, is it possible to pass > such structures around to and from python and C/C++? What would be > the overhead cost of having a large structure that needs to be passed > to and from the C/C++ modules? ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] 'module' object has no attribute (setting a class attribute)
On Sat, 16 Oct 2010 08:20:37 am Tim Johnson wrote: > I have two questions regarding this: > 1)Am I using the correct method to set a class attribute? Yes. The way to set a class attribute is the same way that you set an attribute on *any* object: some_object.attribute_name = something In your case, some_object is the class object tmpl.tmpl (Unlike some other languages, classes in Python are themselves objects.) > 2)Is there a system configuration that would cause the > AttributeError exception to print out the module name. > (In this cause it was 'kbLib' not 'tmpl'. No. The right way to fix this problem is to fix the error message, not to create a system configuration option. What would you call it? give_useless_error_messages = OFF *wink* The error messages in Python are not part of the language, but implementation details. Unfortunately CPython (the version you are almost certainly using) does have a few lousy error messages. However, in this case it is fairly straightforward to guess what the problem was. The error message you got was: 'module' object has no attribute 'templatepath' On the left hand side, you had the module object tmpl, but you weren't trying to set tmpl.templatepath (module object . attribute name), you were trying tmpl.tmpl.templatepath (module object . class object . attribute name). If the left hand side was the problem, you would probably have got this error instead: 'module' object has no attribute 'tmpl' (There are other errors you could have got, but they're fairly obscure and/or unusual.) -- Steven D'Aprano ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
