Struggling with struct.unpack() and "p" format specifier

[Geoffrey]

Hope someone can help.
I am trying to read data from a file binary file and then unpack the
data into python variables.  Some of the data is store like this;

xbuffer: '\x00\x00\xb9\x02\x13EXCLUDE_CREDIT_CARD'
# the above was printed using repr(xbuffer).  
# Note that int(0x13) = 19 which is exactly the length of the visible
text
#

In the code I have the following statement;
x = st.unpack('>xxBBp',xbuffer)

This throws out the following error;

x = st.unpack('>xxBBp',xbuffer)
error: unpack str size does not match format

As I read the documentation the "p" format string seems to address
this situation, where the number bytes of the string to read is the
first byte of the stored value but I keep getting this error.

Am I missing something ?
Can the "p" format character be used to unpack this type of data ?

As I mentioned, I can parse the string and read it with multiple
statements, I am just looking for a more efficient solution.

Thanks.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Struggling with struct.unpack() and "p" format specifier

[Geoffrey]

Thanks for your response.

I guess the documentation on the p format wasn't clear to me ... or
perhaps I was just hoping to much for an easy solution !

The data is part of a record structure that is written to a file with
a few "int"'s and "longs" mixed in.  The pattern repeats through the
file with sometime up to 2500 repititions.

Clearly I can create a subroutine to read the records and extract out
the fields.  I was just hoping I could use the "struct" module and
create a pattern like 'LLHpHLpppH' which would unpack the date and
automatically give me the strings without needing to first determine
their lengths as the length is already embedded in the data.

Any suggestion on how to go about proposing the ability to read
variable length strings based on the preceeding byte value to the
struct module ?  It seems it would be a valuable addition, helping
with code clarity, readability and saving quite a few lines of code -
well atleast me anyways !

Thanks again.

Peter Hansen <[EMAIL PROTECTED]> wrote in message news:<[EMAIL PROTECTED]>...
> Geoffrey wrote:
> > As I mentioned, I can parse the string and read it with multiple
> > statements, I am just looking for a more efficient solution.
> 
> This looks like about the best you can do, using the information
> from Tim's reply:
> 
>  >>> buf = '\0\0\xb9\x02\x13EXCLUDE_CREDIT_CARD'
>  >>> import struct
>  >>> x = struct.unpack('>xxBB%sp' % (ord(buf[4])+1), buf)
>  >>> x
> (185, 2, 'EXCLUDE_CREDIT_CARD')
> 
> If you wanted to avoid hard-coding the 4, you would
> be most correct to do this:
> 
> header = '>xxBB'
> lenIndex = struct.calcsize(header)
> x = struct.unpack('%s%dp' % (header, ord(buf[lenIndex])+1), buf)
> 
> ... though that doesn't exactly make it all that readable.
> 
> -Peter
-- 
http://mail.python.org/mailman/listinfo/python-list

Unpacking Binary Data - not using struct module

[Geoffrey]

I am working on a file conversion project that reads data from a one
file format, reformats in and writes in out to another.  The data is
records of informations - names address, account number,statistics.

The numeric values in the original file are stored in what appears to
be a "packed" data format,using a structure that does not use any of
the more standard "C" formats, so I can't use the "struct" module.

As an example, the number "1130" is store using 3 bytes.  The HEX
values of the 3 bytes would be  0x01,0x13,0x0F.In other words, the
hex value value "01130f" is unpacked to become "1130".

I believe the original program is written in a version of "Basic" that
dates from the early 90's if that helps.

Currently, I have no problem reading the data.  I read the three bytes
and use the binascii.hexilify() function convert the hex values to
their corresponding ASCII values, and then slice off the trailing "f".
 In other words;

unpackednumber = int(binascii.hexlify(0x1130f)[:-1])

My questions ...
1) Does anyone recognize this numeric format ?
2) Is there a more efficient module/function I can use, or is the
above the best method?
3) I think that the trailing "f" is sometimes used to indicate the
sign of the number and perhaps even the number of decimal places ...
but am not sure.

Any advice/guidance would be appreciated.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Basic Lotus 1-2-3 worksheet creation class for Python

[geoffrey . eisenbarth]

I just wanted to let you now that as someone whose boss prefers to use WK1 in 
2017, this is going to save my life. Thanks :) 

About to try this out, if I have to do anything to get it to work with Python3 
I'll post the changes here later. 
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: wxGridCellEditor and EVT_CHAR

[Geoffrey Clements]

"lux" <[EMAIL PROTECTED]> wrote in message 
news:[EMAIL PROTECTED]
> Hi to all,
> I need to handle EVT_CHAR in a wxGrid when wxCellEditor is activated,
> but I don't know how.
> Can anyone help me?
>

it's probably better to ask this on comp.soft-sys.wxwindows

-- 
Geoff 


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: logo design

[Geoffrey Summerhayes]

Ken Tilton wrote:
> alex23 wrote:
> > Xah Lee wrote:
> >
> >>No personal offense intended, but human animal's history is what? 3000
> >>years at least in recorded history? And, all you can think of is what,
> >>the view points of a fraction of your personal life span?
> >
> >
> > Thank god evolution spat you out to lead us all to the light, huh?
> >
> > No personal offense intended, but you're a boring, elitist prick.
>
> Actually, no, Xah is pretty cool, give him a chance.

Yeah, this still seems kind of silly though. Implementations,
companies and user groups have logos, not languages.

Picture this: Hey, I'm switching to COBOL because its new
logo looks great on t-shirts and mugs.


Geoff

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Writing and reading variables to/from flat file

[Geoffrey Clements]

"Kevin Walzer" <[EMAIL PROTECTED]> wrote in message 
news:[EMAIL PROTECTED]
>I want to write some variables (user preferences, specifically) to a
> text file and then read the values from that file.
>
> Here is my code to write the data:
>
> verbosemodes= """
>Detailed = "-vv"
>Basic = "-q"
>"""
>
> file = open('prefs', 'w')
>
> file.writelines(verbosemodes)
>
> file.close()
>
> And here is my code, in a separate module, to read the file and display
> the variable values:
>
> readfile = open('prefs').readlines()
>
> for line in readfile:
> print line
>
> print Basic
>
>
> Running the second module yields this error:
>
>Detailed = "-vv"
>
>Basic = "-q"
>
>
> Traceback (most recent call last):
>  File "readprefs.py", line 6, in 
>print Basic
> NameError: name 'Basic' is not defined
>
> Clearly the data is getting read (the lines are being printed), but the
> variable itself ("Basic") is not being initialized properly. I'm not
> sure what I'm doing wrong here--can anyone point me in the right
> direction?  Thanks.
>

All you've done is print two strings with the contentents of the two lines 
from the file, this does not execute the code in these strings.

Try this:
readfile = open('prefs').readlines()

for line in readfile:
 print line
 eval(line)
 print Basic

-- 
Geoff



-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How do I print out in the standard output coloured lines

[Geoffrey Clements]

<[EMAIL PROTECTED]> wrote in message 
news:[EMAIL PROTECTED]
On Feb 2, 1:38 pm, rzed <[EMAIL PROTECTED]> wrote:
> [EMAIL PROTECTED] wrote 
> innews:[EMAIL PROTECTED]:
>
>
>
> > On Feb 2, 1:16 pm, rzed <[EMAIL PROTECTED]> wrote:
> >> [EMAIL PROTECTED] wrote
> >> innews:[EMAIL PROTECTED]:
>
> >> > Hi,
>
> >> >   I'm interested in printing out coloured lines of my
> >> >   application and
> >> > I don't know what to use. Can anybody give me an idea??
>

[snip]

>
> If you're on Linux, you could use the curses module. There may be
> a precompiled Windows version compatible with your Python version,
> or maybe not, but the Windows source is available, and you may be
> able to get it to work with your Python with some effort. Linux
> distros include curses, I think. For Windows curses, take a look
> at . You will understand why
> the phrase "Windows curses" is used, I expect.
>
> --
> rzed
>
> Yes, I'm on a Linux box. I've tried with the curses module, but I
> don't how I could fetch the current use of curses of my shell. I don't
> know if I'm talking about something impossible. I've made some tests
> with the curses module and works fine, but I need to capture the
> current window and change the attributes of texts.

You may find the following useful, it's from Gentoo's portage:
http://sources.gentoo.org/viewcvs.py/portage/main/trunk/pym/portage/output.py?rev=5780&view=markup

-- 
Geoff 


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: MySQLdb "begin()" -

[Geoffrey Clements]

<[EMAIL PROTECTED]> wrote in message 
news:[EMAIL PROTECTED]
> Hi - Feeling a bit weird about this but I cannot find the 'begin'
> method on a connection object of MySQLdb. Can anyone explain why ?
>
> I'm using version 1.2.0 which is pretty recent and I've read that
> 'begin' should be a method of connection but it's not there ! Feeling
> pretty puzzled !
>

What exactly do you expect a begin method to do?  Have you read this:
http://www.python.org/dev/peps/pep-0249/

-- 
Geoff 


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: John Bokma harassment

[Geoffrey Summerhayes]

"Bill Atkins" <[EMAIL PROTECTED]> wrote in message 
news:[EMAIL PROTECTED]
> John Bokma <[EMAIL PROTECTED]> writes:
>
> [snip]
>
>> -- 
>> John   MexIT: http://johnbokma.com/mexit/
>>personal page:   http://johnbokma.com/
>> Experienced programmer available: http://castleamber.com/
>> Happy Customers: http://castleamber.com/testimonials.html
>
> Interesting.  Doesn't your signature contain advertisements for your
> website?  Aren't you posting to five different groups?

Shh! Pointing out ironic hypocrisy never works.

--
Geoff

P.S. You forgot that it's also off-topic for all groups.
P.P.S. Mea culpa


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: John Bokma harassment

[Geoffrey Summerhayes]

"John Bokma" <[EMAIL PROTECTED]> wrote in message 
news:[EMAIL PROTECTED]
> Dra¾en Gemiæ <[EMAIL PROTECTED]> wrote:
>
>> There is a person on USENET, particularly in hr. hierarchy that
>> posts under three different accounts. Sometimes he argues with
>> himself, and sometimes event supports himself :-)
>>
>> Maybe we have the similar case here.
>
> Wouldn't amaze me if some of the buddies of Xah are actually Xah sitting
> in some Internet cafe, enjoying this troll fest, and already thinking up
> the next one.

That's right, we're all Xah, you're the only other one here.

After you kill Navarth, will it be nothing but gruff and deedle
with a little wobbly to fill in the chinks?

--
Geoff


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: John Bokma harassment

[Geoffrey Summerhayes]

"John Bokma" <[EMAIL PROTECTED]> wrote in message news:[EMAIL PROTECTED]
> "Geoffrey Summerhayes" <[EMAIL PROTECTED]> wrote:
>
>> After you kill Navarth, will it be nothing but gruff and deedle
>> with a little wobbly to fill in the chinks?
>
> Comparing Navarth with Xah is a huge insult to Jack Vance. You should be
> ashamed of yourself for even thinking about it, let alone write it down.

Mr. Vance is too intelligent to be insulted by this.
OTOH, Mad Navarth is free to be as insulted as much
as his fictional soul will allow. :)

--
Geoff


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: ?

[Geoffrey Clements]

"urikaluzhny" <[EMAIL PROTECTED]> wrote in message 
news:[EMAIL PROTECTED]
On May 15, 10:06 am, "Terry Reedy" <[EMAIL PROTECTED]> wrote:
> "urikaluzhny" <[EMAIL PROTECTED]> wrote in message
>
> news:[EMAIL PROTECTED]
> | It seems that I rather frequently need a list or iterator of the form
> | [x for x in <> while <>]
>
> I can think of two ways to interpret that.
>> I mean like [x for x in  if ], only that it breaks the loop when
>> the expression  is false.

def gen(a):
for x in a:
if B: break
yield x

a_gen = gen(A)

# now iterate over a_gen

-- 
Geoff 


--
http://mail.python.org/mailman/listinfo/python-list

Re: Video information

[Geoffrey Clements]

"Bill McClain" <[EMAIL PROTECTED]> wrote in message 
news:[EMAIL PROTECTED]
> On 2008-08-09, dusans <[EMAIL PROTECTED]> wrote:
>> Is there a py module, which would get me information of a movie file:
>> - resolution
>> - fps
>
> I don't know of one. I use the transcode utilities for this and parse 
> their
> output.
>

Something like:
from subprocess import Popen, PIPE
probe = Popen(("tcprobe", "-i", ip_file), stdout=PIPE, 
stderr=PIPE).communicate()[0]

or
from subprocess import Popen, PIPE
probe = Popen(("mplayer", "-identify", "-frames", "0", "-ao", "null", 
ip_file), stdout=PIPE, stderr=PIPE).communicate()[0]

First one uses transcode the second one uses mplayer. I normally follow 
these with a regexp search on "probe" for the data I want.
I have found mplayer to be more accurate than transcode which seems to have 
a problem with some files notably MPEG program streams although it seems to 
work fine for elementary streams.

-- 
Geoff 


--
http://mail.python.org/mailman/listinfo/python-list

Re: Programming challenge: wildcard exclusion in cartesian products

[Geoffrey Summerhayes]

Wade Humeniuk wrote:
> [EMAIL PROTECTED] wrote:
> > What I have in mind is the efficient,  generation of the
> > complement S^n/WC(S^n). A good program should initialize, generate, and
> > terminate.
> >
> > T=cartprodex(S,n,WC); //initialize
> > for all i in T do
> >   what you want with i
> >   test to see if any more
> >   terminate if not
> >
> > and it should do this without explicitly generating WC and then
> > complementing. For example, if the cardinality of S is m, and the WC is
> > just '*a*b*', with a != b, then EX(S^n):=S^n\WC(S^n) has cardinality
> > (m-1)^(n-1)*(m+n-1). Specifically, if m=5 and n=10, then |EX|=3670016
> > while |S^10|=9765625, so that |EX|/|S^10| is about 0.3758. In general
> > the program should directly generate EX from arbitrary WC. Of course,
> > in practice the WC should themselves occur in a logically consistent
> > manner, but let's just assume they're a given.
> >
>
> Another attempt.  I have made no special attempt to create an
> exclusion language, just used an anonymous lambda predicate.
>

FWIW, here's my Q-and-D pattern matcher (only partially tested).

(defun match(list pattern &optional (test #'eql))
  "Match a list of atoms against a pattern list
using :all as a 0-to-many wildcard, :single as a
1-to-1 wildcard, a list of elements or a single
element to match a specific place. Optional
argument test for comparing elements (default eql).

Returns: T if match is made, NIL otherwise.

Examples: (match '(0 1 2 3 4 5) '(:all (2 3) 3 :single 5 :all)) => T
  (match '(0 1 2 3 4 5) '(:all (2 3) 3 :single 5 :single)) =>
NIL"
  (let ((current (first pattern))
(next-pattern (rest pattern))
(candidate (first list)))
(cond ((and (null pattern) (null list))
   t)
  ((and (eq :single current) candidate)
   (match (rest list) next-pattern test))
  ((eq :all current)
   (loop for new-list on list
 when (match new-list next-pattern test)
 do (return-from match t))
   (null next-pattern)) ; last case null remainder
  ((if(atom current)
   (funcall test candidate current)
 (member candidate current :test test))
   (match (rest list) next-pattern test)

--
Geoff

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Programming challenge: wildcard exclusion in cartesian products

[Geoffrey Summerhayes]

"Dinko Tenev" <[EMAIL PROTECTED]> wrote in message 
news:[EMAIL PROTECTED]
> [EMAIL PROTECTED] wrote:
>> It would seem that your program is just filtering the full cartesian
>> product, right? The solution I'm looking for generates the elements
>> one-by-one so that it could be used in a loop.
>
> Oops...missed that part.
>
> It took me a while to study the exchange on this topic more thoroughly,
> and I now fully appreciate the fact that the problem calls for a much
> more sophisticated approach.
>
> Sorry for the hasty shot, I'll give it another shortly.

I wouldn't worry about it, Prolog generated the elements one-by-one.
The loop was the print,nl,fail line. Just beefing it up a bit, I
didn't take the time to clean it up though. :-)

gen(_,0,[]).
gen(S,N,[H|T]):- N > 0, N1 is N - 1, member(H,S), gen(S,N1,T).

filter([],[]).
filter([X|T],[X|T1]):- filter(T,T1).
filter([*|T],L):- filter(T,L).
filter([*|T],[_|T1]):- filter([*|T],T1).

filter_list(L,[[and|T]|_]):- filter_and(L,T), !.
filter_list(L,[[or|T]|_]):- filter_list(L,T), !.
filter_list(L,[H|_]):- H \= [and|_], H \= [or|_], filter(H,L),!.
filter_list(L,[H|T]):- H \= [and|_], H \= [or|_], filter_list(L,T).

filter_and(_,[]) :- !.
filter_and(L,[H|T]):- filter_list(L,[H]), filter_and(L,T).

generate_member(X,S,N,[]):-gen(S,N,X).
generate_member(X,S,N,[H|T]):-gen(S,N,X),\+ filter_list(X,[H|T]).

1 ?- generate_member(X,[a,b],3,[[a,*,b],[b,*,a]]).

X = [a, a, a] ;

X = [a, b, a] ;

X = [b, a, b] ;

X = [b, b, b] ;

No
2 ?- generate_member(X,[1,2],3,[[and, [*,2], [or, [2,1,*], [1,2,*).

X = [1, 1, 1] ;

X = [1, 1, 2] ;

X = [1, 2, 1] ;

X = [2, 1, 1] ;

X = [2, 2, 1] ;

X = [2, 2, 2] ;

No

---
Geoff 


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Programming challenge: wildcard exclusion in cartesian products

[Geoffrey Summerhayes]

<[EMAIL PROTECTED]> wrote in message 
news:[EMAIL PROTECTED]
> After the basic fact of generating the exclusion - a considerable
> achievement - the program should be interactive. What if the target set
> has thousands or millions of elements? There should be a  loop-like way
> ('do' in Haskell, for example) to peel off the elements one-by-one and
> then terminate.

There is...(Q&D)

1 ?- generate_member(X,[1,2],3,[[and, [*,2], [or, [2,1,*], [1,2,*),
 write(X),nl,write('Is this the term you were looking for? (y/n):'),
 get(Y), ((Y is 121) -> true; fail). % 121 = 'y'

[1, 1, 1]
Is this the term you were looking for? (y/n):n
[1, 1, 2]
Is this the term you were looking for? (y/n):|: n
[1, 2, 1]
Is this the term you were looking for? (y/n):|: y

Yes
2 ?-

---
Geoff 


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: emptying a list

[Geoffrey Clements]

"lallous"  wrote in message news:ha2htc$u9...@aioe.org...
> Hello
>
> What is faster when clearing a list?
>
> del L[:]
>
> or
>
> L = []
>

Oh, "L = []" definitely, on the basis that there are fewer characters to 
type.


http://docs.python.org/3.1/library/profile.html

-- 
Geoff 


-- 
http://mail.python.org/mailman/listinfo/python-list