[Tutor] Accessing variables from other modules

[Chip Wachob]

Hello,

Hoping that this comes through as text only.  Not sure how to force
that with Gmail.

Very new to Python and trying to follow the instructions I've read on
the tutorial and other places.  But, I'm not meeting with any success.
I have a feeling this is something simple but a search of the archives
didn't provide any useful results.  Perhaps I just don't know the
'lingo' yet.  So I apologize for the likelihood that this is a
duplicate.

I have a background in C and other asm languages so that should give
you some idea of my current understanding related to programming.
Forgive any incorrect use of terms..

I started writing code in Python for a little project I'm working on.
I used examples from the tutorial provided by the DIY board (Adafruit
FT232H Breakout).  Those instructions were the basis for my working
code.

As time went on, I had a _whole_ lot of code and it was getting
unmanageable.  So I looked into breaking it into separate modules
(like I would do in C).  I then used the import statement to 'include'
them into the main.py file.

I ran into a problem in that the code which was moved to the module
could no longer 'see' the ft232h variable (object?).  After many
attempts, I figured out that the best solution seemed to be to put the
ft232h setup code into yet another file.  I then imported that file
into both my main.py and foo.py files.  And, that seemed to work.. on
Friday.

This morning, I came back to continue working on the code, and now the
ft232h variable can no longer be 'seen' by my modules...

When I was running the code on Friday it was being run from a command
line each time.  So, I'm assuming that the dictionary disappears and
that there is a new 'fresh' start each time I execute.  Is this
correct?

I'm running Python 2.7 and it is running on a native Ubuntu machine at
16.04 LTS.

I'll refrain from posting a bunch of code, but here is the 5000 foot view:

main.py - calls various functions from other *.py files (foo.py, etc).
It also creates a console-based menu selection which determines which
of the functions to call.

foo.py (and others) - contain the actual functions for the different
work that I want to get done.

spi.py - contains the setup and initialization code for the Adafruit
board and configures it to function as a SPI peripheral via USB.

execution looks like this:

$ sudo python main.py

So, the important questions are:

- Was I mislead by the fact that there was a power cycle on the
machine and it has now forgotten something that was staying resident
when I tested the code on Friday?  Or, does each 'run' of the script
start fresh?

- What approach do I need to use to be able to initialize the
interface in spi.py and have it be something that is accessible to all
the other modules in my project?

Thank you in advance for your time.
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Tutor] Fwd: Accessing variables from other modules

[Chip Wachob]

My apologies.  I hit reply and not reply to all.

Alan,

I think I answered many of your questions in this message to Steven.




-- Forwarded message --
From: Chip Wachob 
Date: Tue, Sep 4, 2018 at 1:48 PM
Subject: Re: [Tutor] Accessing variables from other modules
To: Steven D'Aprano 


Steven,

Thank you.

Responding to your comments in order:

# module AdafruitInit.py
# from the Adafruit tutorials..
import Adafruit_GPIO.FT232H as FT232H
import Adafruit_GPIO as GPIO

FT232H.use_FT232H()

ft232h = FT232H.FT232H()

# config settings for the SPI 'machine'
spi = FT232.SPI(ft232h, 4, 2, 2, FT232H.MSBFIRST)



# module RSI.py
def write(byte):
   spi.write(byte)

# toggle the latch signal
   ft232h.output(5, GPIO.LOW)
   ft232h.output(5, GPIO.HIGH)



# module main.py
import RSI
import AdafruitInit.py

ft232.setup(5, GPIO.OUT)

write(0xAA)


"write()" tells me that

"global name 'ft232h' is not defined"


Regarding permissions.  I'm not sure about the need for sudo, but that
was used in the example.  The script will fail to run without it.  I
can't say for certain that I understand why sudo is required.  It may
have something to do with disabling the "built in" FTDI driver and
enabling the Adafruit version.  I had hoped to just get things
'working' before worrying about trying to get rid of the sudo
requirement.

I had been working most of the day on Friday and the scripts were
running fine.  I was running the scripts from the command line as
noted above (and related to your question re: sudo).  I powered off
the machine Friday night, and started it back up this morning.

Does some of the FTDI (AdafruitInit.py) remain resident even though
I've exited out of the script.  My menu has an entry that will permit
breaking the while() loop and ending the script.



...



On Tue, Sep 4, 2018 at 1:19 PM, Steven D'Aprano  wrote:
> Hi Chip, and welcome!
>
> On Tue, Sep 04, 2018 at 11:10:36AM -0400, Chip Wachob wrote:
>
>> I'll refrain from posting a bunch of code, but here is the 5000 foot view:
>
> Not posting a mountain of code is a great idea, but from 5000 ft away
> we can't see what is going on.
>
> Try posting *a little bit of code*. This is written for Java programmers
> but the principle is the same for Python:
>
> http://sscce.org/
>
> If I had to make a *wild guess* as to what is going on, it would be that
> you have a couple of modules like this:
>
> # Module spam.py
> x = 1  # global variable
> def foo():
> print(x)
>
>
> # Module eggs.py
> import spam
> from spam import x
> foo()  # prints 1, as you expect
> x = 999
> foo()  # still prints 1, instead of 999
>
>
> Is that what is going on? On something different?
>
>
> A few more comments:
>
>> execution looks like this:
>>
>> $ sudo python main.py
>
> Do you really need this to be run with root permissions?
>
>> So, the important questions are:
>>
>> - Was I mislead by the fact that there was a power cycle on the
>> machine and it has now forgotten something that was staying resident
>> when I tested the code on Friday?  Or, does each 'run' of the script
>> start fresh?
>
> Um, yes no maybe?
>
> Was there a power cycle? How were you running the scripts?
>
>
>> - What approach do I need to use to be able to initialize the
>> interface in spi.py and have it be something that is accessible to all
>> the other modules in my project?
>
>
>
>
>>
>> Thank you in advance for your time.
>> ___
>> Tutor maillist  -  Tutor@python.org
>> To unsubscribe or change subscription options:
>> https://mail.python.org/mailman/listinfo/tutor
>>
> ___
> Tutor maillist  -  Tutor@python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Accessing variables from other modules

[Chip Wachob]

Alan,

Thanks for your comments.

I just forwarded a copy of a message to the list that I inadvertently
sent as a reply only.

So in that message there's information about the code.

Thank you for confirming that once the interpreter is finished running
my script that all the data is gone.

The board was cycled with the PC since it's powered off the USB port.

I believe that when the script closes, the Adafruit driver is removed
and the Ubuntu driver is reinstated (at least that's what the
documentation says).  So, my presumption would be that there would be
no TSR action going on.  The part on the board is 'dumb' and requires
configuration to run in whatever mode is desired.

I realize that the day of the week doesn't have anything to do with
it, but it sure was feeling like it when I wrote.  The only idea I
could come up with was that maybe something was still running in the
background and was 'helping' my code run properly, and today, that
code was not running.  Friday was a day filled with a _lot_ of trial
and error as I attempted to split this code into manageable chunks..
so anything could have happened.  But, I never ran anything in the
interpreter itself.  The script was always run from the command line
as noted in the forwarded message.







On Tue, Sep 4, 2018 at 7:41 PM, Alan Gauld via Tutor  wrote:
> On 04/09/18 16:10, Chip Wachob wrote:
>
>> (like I would do in C).  I then used the import statement to 'include'
>> them into the main.py file.
>
> OK a basically good idea but how did you use the import statement?
> There are many forms:
>
> import foo
> from foo import name1, name2,...
> from foo import *
> import foo as f
> from foo import name1 as n1
>
>
> Now to access name1 (which could be a function,
> class or global variable within foo) you could
> do any of (depending on your import):
>
> x = foo.name1
> x = name1
> x = name1
> x = f.name1
> x = n1
>
> Does your import statememt and access method
> match the above?
>
> The other thing to remember is that local variables
> (defined inside functions) are not visible outside
> the function.
>
>> I ran into a problem in that the code which was moved to the module
>> could no longer 'see' the ft232h variable (object?).  After many
>> attempts, I figured out that the best solution seemed to be to put the
>> ft232h setup code into yet another file.  I then imported that file
>> into both my main.py and foo.py files.  And, that seemed to work.. on
>> Friday.
>
> That may be a good idea, but it shouldn't really
> have been necessary. And it should work any day
> of the week :-)
>
>> This morning, I came back to continue working on the code, and now the
>> ft232h variable can no longer be 'seen' by my modules...
>
> We need to see some actual code to figure that one out.
>
>> When I was running the code on Friday it was being run from a command
>> line each time.  So, I'm assuming that the dictionary disappears and
>> that there is a new 'fresh' start each time I execute.  Is this
>> correct?
>
> Everything disappears at the end of the interpreter session,
> so yes it starts afresh every time. (Unlike if you run your
> modules from inside the interpreter in which case values
> stick around.)
>
>> main.py - calls various functions from other *.py files (foo.py, etc).
>> It also creates a console-based menu selection which determines which
>> of the functions to call.
>>
>> foo.py (and others) - contain the actual functions for the different
>> work that I want to get done.
>>
>> spi.py - contains the setup and initialization code for the Adafruit
>> board and configures it to function as a SPI peripheral via USB.
>
> It is a good idea to keep the hardware specifics in one
> place but for the purpose of this discussion it shouldn't
> make any difference it's just another module.
>
>> - Was I mislead by the fact that there was a power cycle on the
>> machine and it has now forgotten something that was staying resident
>> when I tested the code on Friday?  Or, does each 'run' of the script
>> start fresh?
>
> Each run should start afresh.
> (The only caveat is that if there is some process still
> running on your hardware(I don't know anything about the
> board in question) it may retain memory and pass it back
> to the init code in spi.py on startup. Does the board get
> power cycled between runs?)
>
>> - What approach do I need to use to be able to initialize the
>> interface in spi.py and have it be something that is accessible to all
>> the other modules in my project?
>
> It 

Re: [Tutor] Fwd: Accessing variables from other modules

[Chip Wachob]

Alan,

Once again, thank you for the feedback and comments.

Revised code snips: Sorry they were not complete.  Several typos while
trying to create the SSCCE version.

# module AdafruitInit.py
# from the Adafruit tutorials..
import Adafruit_GPIO.FT232H as FT232H
import Adafruit_GPIO as GPIO

FT232H.use_FT232H()

ft232h = FT232H.FT232H()

# config settings for the SPI 'machine'
spi = FT232.SPI(ft232h, 4, 2, 2, FT232H.MSBFIRST)

---module separator 

# module RSI.py
import AdafruitInit

def write(byte):
   spi.write(byte)

# toggle the latch signal
   ft232h.output(5, GPIO.LOW)
   ft232h.output(5, GPIO.HIGH)

---module separator 

# module main.py
import RSI
import AdafruitInit

ft232h.setup(5, GPIO.OUT)

write(0xAA)




The actual error message for the ft232h... line is:

NameError: global name 'ft232h' is not defined


So, my write line should have read?

RSI.write(0xAA)

Does this mean that I need to call the ft232h like this?

AdafruitInit.ft232h.setup(5, GPIO.OUT)

And, this is why it is not visible?

I feel like I _thought_ I understood the import statements, but I
guess I didn't and that's what is getting me into trouble.

Earlier in the thread you mentioned that there are several ways to
import.  Obviously there's small differences between the methods.  If
you know of a good resource I can use to get a better understanding /
contrast of the different import types I'll read over it and make sure
I'm using the right one for my situation.

And, at this point, I'm pretty clear on the fact that the script that
runs leaves no trace behind when I exit the call to the interpreter
from the command line.

I appreciate your patience with my questions and foibles.




On Wed, Sep 5, 2018 at 4:53 AM, Alan Gauld via Tutor  wrote:
> On 05/09/18 04:12, Chip Wachob wrote:
>
>> # module RSI.py
>> def write(byte):
>>spi.write(byte)
>
> You don't have any import statements here.
> You need to import spi to use it.
>
>> # toggle the latch signal
>>ft232h.output(5, GPIO.LOW)
>>ft232h.output(5, GPIO.HIGH)
>
> And the same for ft232h
>
>>
>> # module main.py
>> import RSI
>> import AdafruitInit.py
>
> Note you do NOT use the .py extension when
> importing, just the base name.
>
>> ft232.setup(5, GPIO.OUT)
>
> But again you have not imported ft232
> Also I note that you use ft232 here but ft232h elsewhere.
> Is that correct?
>
> You must import any external names that
> you intend to use.
>
>> write(0xAA)
>
> And here you need to prefix with RSI:
>
> import RSI
>
> 
>
> RSI.write()
>
> A Python import is very different to a C include.
> In C you actually include the source text in your
> file so everything inside the file becomes visible.
> In a Python import you add names to a dictionary.
> In this case the only name added is RSI. The code
> inside the RSI module is effectively invisible to
> your main.py, only the name of the module is seen.
> So you must prefix the RSI contents before you use it.
>
>
>> "write()" tells me that
>>
>> "global name 'ft232h' is not defined"
>
> Please always post full error texts, never summarize.
>
>> Regarding permissions.  I'm not sure about the need for sudo, but that
>> was used in the example.
>
> I suspect it's needed because you are accessing
> privileged IO ports. It would not normally be
> needed to run a Python script.
>
>> I had been working most of the day on Friday and the scripts were
>> running fine.
>
> That is the real mystery since the above code
> should not have worked.
>
>> Does some of the FTDI (AdafruitInit.py) remain resident
>
> No, it will be deleted.
>
> --
> Alan G
> Author of the Learn to Program web site
> http://www.alan-g.me.uk/
> http://www.amazon.com/author/alan_gauld
> Follow my photo-blog on Flickr at:
> http://www.flickr.com/photos/alangauldphotos
>
>
> ___
> Tutor maillist  -  Tutor@python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Fwd: Accessing variables from other modules

[Chip Wachob]

Thank you!

Okay, I think I'm starting to get a handle on the visibility of
things.  As you said, much different than C.

Just to confirm that I'm understanding correctly:

Even through the import Adafruit_GPIO as GPIO line exists in the
AdafruitInit.py file, which is imported by the import AdafruitInit
line in main.py, main.py does not automatically inherit the
Adafruit_GPIO.  Which is why you indicate that I need to also import
it in main.py





On Wed, Sep 5, 2018 at 9:42 AM, Alan Gauld  wrote:
> On 05/09/18 14:05, Chip Wachob wrote:
>> # module AdafruitInit.py
>> # from the Adafruit tutorials..
>> import Adafruit_GPIO.FT232H as FT232H
>> import Adafruit_GPIO as GPIO
>>
>> FT232H.use_FT232H()
>>
>> ft232h = FT232H.FT232H()
>>
>> # config settings for the SPI 'machine'
>> spi = FT232.SPI(ft232h, 4, 2, 2, FT232H.MSBFIRST)
>>
>
> That last line created a new variable inside the AdafruitInit module.
> It is not visible anywhere else.
>
>> # module RSI.py
>> import AdafruitInit
>>
>> def write(byte):
>>spi.write(byte)
>
> You use spi but it is not visible. It is inside AdafriuitInit
> so you need to prefix the name (I'd suggest importing
> as an alias!)
>
>AdafruitInit.spi.write(byte)
>
>> # toggle the latch signal
>>ft232h.output(5, GPIO.LOW)
>>ft232h.output(5, GPIO.HIGH)
>
> And the same here. ft232h is a variable inside AdafriotInit,
> so you must prefix it.
>
> BUT GPIO is defined in some other module which
> you will also need to import that
>
> import Adafruit_GPIO as GPIO
>
> adafruitInit.ft232h.output(5, GPIO.LOW)
>
>
> Any name that you try to use must be
> - defined in this module or
> - imported or
> - prefixed with the name of a module that you imported.
>
> If you do not define it in this module you must import
> or prefix.
>
>
>> ---module separator 
>>
>> # module main.py
>> import RSI
>> import AdafruitInit
>>
>> ft232h.setup(5, GPIO.OUT)
>>
>> write(0xAA)
>
> Again you need to use the prefixes (and import GPIO)
>
> AdafruitInit.ft232h.setup(...)
> RSI.write(...)
>
>> The actual error message for the ft232h... line is:
>>
>> NameError: global name 'ft232h' is not defined
>
> Please send the full error text, they contain a lot of useful
> information. In this case the summary line is enough but in
> future we really need the full text.
>
> The name error is because your main module cannpt see
> ft232h defined anywhere - it is inside the other module.
>
> So, my write line should have read?
>> RSI.write(0xAA)
>>
>> Does this mean that I need to call the ft232h like this?
>>
>> AdafruitInit.ft232h.setup(5, GPIO.OUT)
>
> Exactly. And import GPIO too.
>
>> Earlier in the thread you mentioned that there are several ways to
>> import.  Obviously there's small differences between the methods.  If
>> you know of a good resource I can use to get a better understanding /
>
> Since you know C already you should probably just read the official
> tutorial.
> It covers all the different styles.
>
> The only guidance i'll add is to avoid the
>
> from foo import *
>
> style since it easily leads to hard to spot name collisions.
> Only use it for experiments in the >>> prompt.
>
>
> --
> Alan G
> Author of the Learn to Program web site
> http://www.alan-g.me.uk/
> http://www.amazon.com/author/alan_gauld
> Follow my photo-blog on Flickr at:
> http://www.flickr.com/photos/alangauldphotos
>
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Fwd: Accessing variables from other modules

[Chip Wachob]

This helps tremendously!

One last question.

In your examples name1 and name2 could be anything that is contained
in that module.. a variable, function, class, etc..  correct?





On Wed, Sep 5, 2018 at 12:58 PM, Alan Gauld via Tutor  wrote:
> On 05/09/18 15:06, Chip Wachob wrote:
>
>> Okay, I think I'm starting to get a handle on the visibility of
>> things.  As you said, much different than C.
>
> Yes. The significant thing is to remember that in
> Python you are importing names. In C you include
> the contents of the file.
>
> #include 
>
> Lets you access everything in the stdio.h file
> as if it were part of your own file.
>
> import sys
>
> Lets you see the sys module but not whats inside it.
> To access whats inside you must use the name as
> a prefix.
>
> The Pytho import idiom
>
> from sys import *
>
> is similar in effect to the C style include
> (although by an entirely different mechanism)
> but is considered bad practice (for the same
> reasons C++ has its scope operator(::))
>
>> Even through the import Adafruit_GPIO as GPIO line exists in the
>> AdafruitInit.py file, which is imported by the import AdafruitInit
>> line in main.py,
>
> The critical conceptual error here is that the
> file is not imported(*), only the name. Importing
> in Python is all about visibility control
>
> (*)In practice although the module is not
> imported into your file it is executed, so any
> new variables, classes and functions are created,
> but their names are not visible in your code
> except via the module name.
>
> Let me revisit the different import styles
> in more detail. Remember we are discussing visibility
> of names not code.
>
>
> ##
> import modulename
>
> This makes modulename visible to the importing module.
> Nothing inside modulename is visible. To access the
> contents you must use modulename as a prefix.
>
> x = modulename.name1
>
> ###
> import modulename as alias
>
> Exactly the same except that you can refer to
> modulename using the (usually shorter) alias.
>
> x = alias.name1
>
> There are a few community standard aliases such as
>
> import numpy as np
> import tkinter as tk
>
> But they are purely conventions, you can use
> any alias you like. For those who enjoy typing
> they could even do
>
> import os as operating_system
>
> And access the functions as
>
> operating_system.listdir('.')
>
> instead of
>
> os.listdir('.')
>
> if they really wanted to...
>
> 
> from module import name1, name2
>
> This imports specific names from within module.
>
> You can now access name1 and name2 directly:
>
> x - name1  # accesses module.name1
>
> But it does NOT import module itself,
> only name1, name2, etc. If you try
>
> x = module.name3
>
> You will get an error about module (not name3!)
> not being recognised.
>
>
> 
> from module import name1 as n1
>
> Same as above but use the alias n1 instead of
> the longer name1.
>
> x = n1   # like x = module.name1
>
> This is very like you doing the following:
>
> from module import name1
> n1 = name1
>
> 
> from module import *
>
> This makes all the names defined in module visible
> within the importing module. Again it does not make
> module itself visible, only the names inside.
>
> This is considered bad practice because if you have
> multiple modules containing the same name (things
> like open() and write() are common then only the
> last name imported will be visible and that can
> lead to unexpected errors.
>
>
> HTH
> --
> Alan G
> Author of the Learn to Program web site
> http://www.alan-g.me.uk/
> http://www.amazon.com/author/alan_gauld
> Follow my photo-blog on Flickr at:
> http://www.flickr.com/photos/alangauldphotos
>
>
> ___
> Tutor maillist  -  Tutor@python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Tutor] Help with building bytearray arrays

[Chip Wachob]

Hello,

I've been struggling with this for the last day or so and I can't seem
to figure out how to make it work.

I'll start out by saying that if there's a better approach, then I'm all ears.

I'm using the Adafruit Breakout board for the FTDI FT232H part.  Along
with this comes the Python libraries and Adafruit libraries which I'm
using.  I don't think that the libraries are the real issue here, but
I thought I should mention it just for the sake of information.

Basically I'm trying to write a block of unsigned bytes to the device
and read back an equal sized block of unsigned bytes.  There's a
function that is provided called transfer(data_to_send, num_of_bytes)
that handles the heavy lifting.  Unfortunately there seems to be a bug
in the part and if I attempt to send the entire block of bytes (64),
the device will lock up.  I've been able to determine that if I send
16 bytes at a time, I'm okay.

So, I take my bytearray(64) and step through it 16 bytes at a time like this:

my function's main pieces are:

def transfer_byte_array():
   MAX_LOOP_COUNT = 64
   slice_size = 16
   read_ary = bytearray(MAX_LOOP_COUNT)
   scratch_ary = bytearray()

   for step in range (0, MAX_LOOP_COUNT, slice_size):
  scratch_ary = transfer(data_to_send, slice_size)

  for bytes in range (0, slice_size):
 read_ary = scratch_ary[bytes]

   return(read_ary)


Ideally, I'd like to take the slice_size chunks that have been read
and concatenate them back togetjer into a long MAX_LOOP_COUNT size
array to pass back to the rest of my code.  Eg:

read_ary = ary_slice[0] + ary_slice[1] + ary_slice[2] + ary_slice[3]

I know that the + version doesn't work (or didn't for me) but it is
just my attempt at illustrating the overall goal.

The problem that I repeatedly run into is with the line:

read_ary = scratch_ary[bytes]  (or variants thereof)

The traceback is this:

Traceback (most recent call last):
  File "SW8T_5.py", line 101, in 
loop_size = RSI_size_the_loop(Print)
  File "/home/temp/Python_Scratch/examples/RSI.py", line 350, in
RSI_size_the_loop
read_ary.append(scratch_ary[singles])
TypeError: an integer or string of size 1 is required

or, one of the other common ones that I've seen is

TypeError: can't concat bytearray to list

This one is confusing because both of the operands are bytearry
types.. or at least I thought they should be...

when I try to replace the same line with :

read_ary += scratch_ary[bytes]

or

read_ary.append(scratch[bytes])

or

read_ary = read_ary + scratch_ary[bytes]


I'm obviously missing something fundamental here.  Problem is I can't
seem to find any examples of people asking this question before on the
inter-webs..

Thank you in advance to taking time to read.
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Tutor] Additionally

[Chip Wachob]

Sorry admin, I don't know how to append a message I already sent:


the transfer function expects an input of a bytearray and returns the same:

def transfer(self, data):
"""Full-duplex SPI read and write.  The specified array of bytes will be
clocked out the MOSI line, while simultaneously bytes will be read from
the MISO line.  Read bytes will be returned as a bytearray object.
"""
# Build command to read and write SPI data.
command = 0x30 | (self.lsbfirst << 3) | (self.read_clock_ve <<
2) | self.write_clock_ve
logger.debug('SPI transfer with command {0:2X}.'.format(command))
# Compute length low and high bytes.
# NOTE: Must actually send length minus one because the MPSSE engine
# considers 0 a length of 1 and  a length of 65536
length = len(data)
len_low  = (length-1) & 0xFF
len_high = ((length-1) >> 8) & 0xFF
# Send command and length.
self._assert_cs()
self._ft232h._write(str(bytearray((command, len_low, len_high
self._ft232h._write(str(bytearray(data)))
self._ft232h._write('\x87')
self._deassert_cs()
# Read response bytes.
return bytearray(self._ft232h._poll_read(length))
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Tutor] Help with building bytearray arrays

[Chip Wachob]

Admin, please remove my earlier messages.

This message is a properly 'self contained' message.

Hello,

I've been struggling with this for the last day or so and I can't seem
to figure out how to make it work.

I'll start out by saying that if there's a better approach, then I'm all ears.

I'm using the Adafruit Breakout board for the FTDI FT232H part.  Along
with this comes the Python libraries and Adafruit libraries which I'm
using.  I don't think that the libraries are the real issue here, but
I thought I should mention it just for the sake of information.

Basically I'm trying to write a block of unsigned bytes to the device
and read back an equal sized block of unsigned bytes.  There's a
function that is provided called transfer(data_to_send, num_of_bytes)
that handles the heavy lifting.  Unfortunately there seems to be a bug
in the part and if I attempt to send the entire block of bytes (64),
the device will lock up.  I've been able to determine that if I send
16 bytes at a time, I'm okay.

So, I take my bytearray(64) and step through it 16 bytes at a time like this:

my function's main pieces are:

def transfer_byte_array():
   MAX_LOOP_COUNT = 64
   slice_size = 16
   read_ary = bytearray(MAX_LOOP_COUNT)
   scratch_ary = bytearray()

   for step in range (0, MAX_LOOP_COUNT, slice_size):
  scratch_ary = transfer(data_to_send, slice_size)

  for bytes in range (0, slice_size):
 read_ary = scratch_ary[bytes]

   return(read_ary)


Ideally, I'd like to take the slice_size chunks that have been read
and concatenate them back togetjer into a long MAX_LOOP_COUNT size
array to pass back to the rest of my code.  Eg:

read_ary = ary_slice[0] + ary_slice[1] + ary_slice[2] + ary_slice[3]

I know that the + version doesn't work (or didn't for me) but it is
just my attempt at illustrating the overall goal.

The problem that I repeatedly run into is with the line:

read_ary = scratch_ary[bytes]  (or variants thereof)

The traceback is this:

Traceback (most recent call last):
  File "SW8T_5.py", line 101, in 
loop_size = RSI_size_the_loop(Print)
  File "/home/temp/Python_Scratch/examples/RSI.py", line 350, in
RSI_size_the_loop
read_ary.append(scratch_ary[singles])
TypeError: an integer or string of size 1 is required

or, one of the other common ones that I've seen is

TypeError: can't concat bytearray to list

This one is confusing because both of the operands are bytearry
types.. or at least I thought they should be...

when I try to replace the same line with :

read_ary += scratch_ary[bytes]

or

read_ary.append(scratch[bytes])

or

read_ary = read_ary + scratch_ary[bytes]


I'm obviously missing something fundamental here.  Problem is I can't
seem to find any examples of people asking this question before on the
inter-webs..

the transfer function expects an input of a bytearray and returns the same:

def transfer(self, data):
"""Full-duplex SPI read and write.  The specified array of bytes will be
clocked out the MOSI line, while simultaneously bytes will be read from
the MISO line.  Read bytes will be returned as a bytearray object.
"""
# Build command to read and write SPI data.
command = 0x30 | (self.lsbfirst << 3) | (self.read_clock_ve <<
2) | self.write_clock_ve
logger.debug('SPI transfer with command {0:2X}.'.format(command))
# Compute length low and high bytes.
# NOTE: Must actually send length minus one because the MPSSE engine
# considers 0 a length of 1 and  a length of 65536
length = len(data)
len_low  = (length-1) & 0xFF
len_high = ((length-1) >> 8) & 0xFF
# Send command and length.
self._assert_cs()
self._ft232h._write(str(bytearray((command, len_low, len_high
self._ft232h._write(str(bytearray(data)))
self._ft232h._write('\x87')
self._deassert_cs()
# Read response bytes.
return bytearray(self._ft232h._poll_read(length))


Thank you in advance to taking time to read.
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Help with building bytearray arrays

[Chip Wachob]

Point taken on 'bytes'..  thanks.

the

scratch_ary = bytearray()

was my attempt at 'setting' the type of the variable.  I had hoped
that it would help resolve the error messages telling me that the
types didn't go together.

Coming from a 'C' background, I find the lack of typing in Python to
be confusing.  I'm used to working with bytes / words signed and
unsigned for a reason.

So, if I'm understanding the transfer() function correctly, the
function takes and returns a bytearray type.  You mentioned about
constructing a bytearray if I need one.  Can you expand on how I
approach that?

I'm going to try the experiment you mentioned in hopes of it giving me
a better understanding of the 'types' and what happens with the
variables.

Thank you,


On Fri, Sep 7, 2018 at 6:23 PM, Cameron Simpson  wrote:
> On 07Sep2018 15:45, Chip Wachob  wrote:
>>
>> Basically I'm trying to write a block of unsigned bytes to the device
>> and read back an equal sized block of unsigned bytes.  There's a
>> function that is provided called transfer(data_to_send, num_of_bytes)
>> that handles the heavy lifting.  Unfortunately there seems to be a bug
>> in the part and if I attempt to send the entire block of bytes (64),
>> the device will lock up.  I've been able to determine that if I send
>> 16 bytes at a time, I'm okay.
>>
>> So, I take my bytearray(64) and step through it 16 bytes at a time like
>> this:
>>
>> my function's main pieces are:
>>
>> def transfer_byte_array():
>>   MAX_LOOP_COUNT = 64
>>   slice_size = 16
>>   read_ary = bytearray(MAX_LOOP_COUNT)
>>   scratch_ary = bytearray()
>>
>>   for step in range (0, MAX_LOOP_COUNT, slice_size):
>>  scratch_ary = transfer(data_to_send, slice_size)
>>
>>  for bytes in range (0, slice_size):
>> read_ary = scratch_ary[bytes]
>>
>>   return(read_ary)
>>
>>
>> Ideally, I'd like to take the slice_size chunks that have been read
>> and concatenate them back togetjer into a long MAX_LOOP_COUNT size
>> array to pass back to the rest of my code.  Eg:
>>
>> read_ary = ary_slice[0] + ary_slice[1] + ary_slice[2] + ary_slice[3]
>
>
> Minor remark: don't use the name "bytes" for a variable, it is a builtin
> type name and you're shadowing it.
>
> It looks to me like "transfer" hands you back a buffer with the read data,
> so this:
>
>  scratch_ary = bytearray()
>
> don't do anything (it gets discarded).
>
> If you're getting back a bytes or bytearray object from transfer, just
> gather them all up in an list:
>
>  returned_buffers = []
>  for ..
>  response = transfer(data_to_send, slice_size)
>  returned_buffers.append(response)
>  ...
>  read_ary = b''.join(returned_buffers)
>
> Note that that makes a new bytes object for read_ary to refer to. You don't
> need the earlier initialisation of read_ary.
>
> Also note that the bytes object is read only; if that is a problem you'll
> need to construct a bytearray instead.
>
> [...]
>>
>> The problem that I repeatedly run into is with the line:
>>
>> read_ary = scratch_ary[bytes]  (or variants thereof)
>>
>> The traceback is this:
>>
>> Traceback (most recent call last):
>>  File "SW8T_5.py", line 101, in 
>>loop_size = RSI_size_the_loop(Print)
>>  File "/home/temp/Python_Scratch/examples/RSI.py", line 350, in
>> RSI_size_the_loop
>>read_ary.append(scratch_ary[singles])
>> TypeError: an integer or string of size 1 is required
>
>
> Yeah I thought that looked weird to me too.
>>
>> or, one of the other common ones that I've seen is
>>
>> TypeError: can't concat bytearray to list
>>
>> This one is confusing because both of the operands are bytearry
>> types.. or at least I thought they should be...
>
>
> No, one will be a list :-) putting a bunch of:
>
>  print(repr(foo))
>
> replacing "foo" with relevant variables will be illuminating to you; you can
> see immediately where this are not what you expected.
>
>> I'm obviously missing something fundamental here.  Problem is I can't
>> seem to find any examples of people asking this question before on the
>> inter-webs..
>
>
> You have the opposite of my problem. I can often find people asking the same
> question, but less often an answer. Or a decent answer, anyway.
>
> Cheers,
> Cameron Simpson 
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Help with building bytearray arrays

[Chip Wachob]

Cameron, et al.

First off, thank you for being patient with me.  I'm not used to the
email list communication style.

Since Cameron's response was the one that raised the most questions /
comments, I'm going to reply to it.

Inline.. now that I know that this is the preferred method...

Before I jump in, the 1000 foot view is I have to send an array of 512
bytes down the SPI loop, and read back 512 bytes that were latched in
from a control interface.  Unfortunately, there's a glitch in the FTDI
part and I can't just send the 512 bytes.. the part times out and and
causes the script to terminate early...  So, my solution was to
'chunk' the data into smaller groups which the part _can_ handle.
This works fine until I come to the point where I concatenate the
'chunks' into my 512 byte array..  which other functions will then
process.

The libraries that I'm using are from the project link below.  I'm
learning from all of you how much code to post, I didn't want to post
the entire library, but the way it's set up it is hard not to.. as has
been pointed out.. methods of classes and then there's other files
that contain some of the low-level workings..

https://github.com/adafruit/Adafruit_Python_GPIO

Anyhow, on with the 'show'..


On Sat, Sep 8, 2018 at 6:49 AM, Cameron Simpson  wrote:
> On 08Sep2018 20:01, Cameron Simpson  wrote:
>>>
>>> So, if I'm understanding the transfer() function correctly, the
>>> function takes and returns a bytearray type.
>>
>>
>> It would be good to see the specification for the transfer function. They
>> we can adhere to its requirements. Can you supply a URL?
>
>
> I see you supplied the whole transfer function in your first message :-(

Yes, I realize now that I left out important information.  I have my
own transfer function which is the supplied transfer function with a
bunch of GPIO wrapped around it, and the 'chunking'.  This way I can
call it when I need to send and receive data from the loop, which is
often.  In the end, I will have to talk with several different SPI
devices, all of which will have to have different GPIO line wiggling
going on before and after they call the spi.transfer.

You're walk-through is very helpful.

>
> I'll recite it here and walk through its contents to explain:
>
>def transfer(self, data):
>
> Ok, this is a method, a particular type of function associated with a class
> instance. (All objects are class instances, and the class is their type.)
>
> So to call this you would normally have a control object of some kind. [...]
> Ah, it looks like you should have an SpiDev instance, inferring from this
> code:
>
>
> https://github.com/adafruit/Adafruit_Python_GPIO/blob/master/Adafruit_GPIO/SPI.py
>
> So suppose you've got such an object and a variable referring to it, let's
> say it is named "spi". You'd normally call the transfer function like this:
>
>  spi.transfer(some_data)
>
> instead of calling transfer() "bare", as it were.

Again, I was generalizing and also being somewhat lazy and not typing
it all... my bad.

>
> When you call a Python method, the instance itself is implicitly passed as
> the parameter "self" (well, the first parameter - in Python we always call
> this "self" like C++ uses "this").

Ah, the one 'thorn' in my side is "this".  I have considerable
difficulty with the 'this' concept.  That probably means that my code
could be 'better', but I tend to avoid "this" like the plague.

>
>"""Full-duplex SPI read and write.  The specified array of bytes will
> be
>clocked out the MOSI line, while simultaneously bytes will be read
> from
>the MISO line.  Read bytes will be returned as a bytearray object.
>"""
>
> This is a docstring. It is like a comment but it gets attached to the
> function and can be inspected. Not your problem.
>
># Build command to read and write SPI data.
>command = 0x30 | (self.lsbfirst << 3) | (self.read_clock_ve << 2) |
> self.write_clock_ve
>
> This constructs a value just as you would in C.
>
>logger.debug('SPI transfer with command {0:2X}.'.format(command))
>
> Write a debugging message.

I saw this, but I don't know how to turn it 'on' so I could see the
log messages.  I wouldn't mind doing this with my code as well.
Perhaps creating my own log?  Right now, I have a verbose flag that I
pass to all my functions.. called disp.  Then for each place where I
want to see a value, etc, I have a line that reads:  if(disp): print "
What data is this ", data

>
># Compute length low and high bytes.
># NOTE: Must actually send length minus one because the MPSSE engine
># considers 0 a length of 1 and  a length of 65536
>length = len(data)
>len_low  = (length-1) & 0xFF
>len_high = ((length-1) >> 8) & 0xFF
>
> All just like C.
>
># Send command and length.
>self._assert_cs()
>
> I would guess that this raises a control signal. Ah, RS-232? So
> clear-to-send then.

This is actually a chip sel

Re: [Tutor] Help with building bytearray arrays

[Chip Wachob]

On Sat, Sep 8, 2018 at 9:14 PM, Cameron Simpson  wrote:
> On 08Sep2018 11:40, Alan Gauld  wrote:
>>
>> On 08/09/18 03:15, Chip Wachob wrote:
>>>
>>> Ideally, I'd like to take the slice_size chunks that have been read
>>> and concatenate them back togetjer into a long MAX_LOOP_COUNT size
>>> array to pass back to the rest of my code.  Eg:
>>
>>
>> You need to create a list of read_ary
>>
>> results = []
>>
>> then after creating each read_ary value append it
>> to results.
>>
>> results.append(read_ary)
>>
>> Then, at the very end, return the summation of all
>> the lists in results.
>>
>> return sum(results,[])
>

Cameron is correct (Gold Star for you Cameron)

>
> Actually he's getting back bytearray instances from transfer and wants to
> join them up (his function does a few small transfers to work around an
> issue with one big transfer). His earlier code is just confused. So he
> wants:
>
>  bytearray().join(results)
>
> Hacked example:
>
>  >>> bytearray().join( (bytearray("foo"),bytearray("bah")) )
>  bytearray(b'foobah')

I understand this example and I can replicate it in the interpreter..

But, I'm still missing something here.

I presume that I need to instantiate an array of slice_size-sized bytearrays.

So, when I'm looping through, I can do:

for i in range (0, slice_count):
   results[i] = spi.transfer(data_out)

Then I can :

all_together = butearray().join(results)

But I can't seem to be able to find the proper syntax to create the
initial array.

And any attempt at filling the arrays to test some stand-alone code
only give me errors.

Here's my code (all of it)

#
#
#

import sys

slice_size = 16# in bytes
MAX_LOOP_COUNT = 64# in bytes

slice_count = MAX_LOOP_COUNT / slice_size

print " slice size = ", slice_size
print " MLC = ", MAX_LOOP_COUNT
print " slice count = ", slice_count

#results = bytearray( (bytearray(slice_size)*slice_count) )
results = bytearray(slice_size)# why isn't this 16 bytes long?

res_list = (results)*slice_count

print " results ", [results]
print " results size ", sys.getsizeof(results)
print " res_list ", [res_list]
print " res_list size ", sys.getsizeof(res_list)
print " type = ", type(results)

results[0] = ([1,3,5,7,9])

all_together = bytearray().join(results)

But I'm getting:

$ python merge_1.py
 slice size =  16
 MLC =  64
 slice count =  4
 results  
[bytearray(b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00')]
 results size  65
 res_list  
[bytearray(b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00')]
 res_list size  113
 type =  
Traceback (most recent call last):
  File "merge_1.py", line 27, in 
results[0] = ([1,3,5,7,9])
TypeError: an integer or string of size 1 is required

Again, I feel like I'm circling the target, but not able to divine the
proper syntax

I hope this makes sense.


>
> And he's working with bytearrays because the target library is Python 2,
> where there's no bytes type.
>
> Cheers,
> Cameron Simpson 
>
> ___
> Tutor maillist  -  Tutor@python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Help with building bytearray arrays

[Chip Wachob]

Cameron,

Thank you again for the insight.

Yes, data_out is an equivalently-sized 'chunk' of a larger array.

I'm 'getting' this now..

So, without all the fluff associated with wiggling lines, my function
now looks like this:

def RSI_size_the_loop():
   results = []
   all_together = []   # not certain if I need this, put it in in an
attempt to fix the incompatibility if it existed

   for x in range (0, MAX_LOOP_COUNT, slice_size):
  results.append(my_transfer(disp, data_out, slice_size)

  print " results ", x, " = ", results  # show how results grows
on each iteration

   all_together = bytearray().join(results)

   print " all together ", all_together


I can observe results increasing in size and the last time through the loop:

 results  48  =
[[bytearray(b'\xff\xff\xff\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00')],
[bytearray(b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00')],
[bytearray(b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00')],
[bytearray(b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00')]]

So, now when I hit the line:

all_together = bytearray().join(results)

I'm getting the Traceback :

Traceback (most recent call last):
  File "SW8T_5.py", line 101, in   # this is my main script
loop_size = RSI_size_the_loop(Print)
  File "/home/temp/Python_Scratch/examples/RSI.py", line 359, in
RSI_size_the_loop
all_together = bytearray().join(results)
TypeError: can only join an iterable of bytes (item 0 has type 'list')

I looked this up, and there's very few search engine results on this
type of TypeError.

I wanted to clarify my understanding of iterable, and looked that up here:

https://infohost.nmt.edu/tcc/help/pubs/python/web/iterable.html

And, as far as I can tell I'm using a compatible sequence type.

I've even added in print statements for the types, so I could double
check, and I get:

results returns 
all_together returns 

So both are type 'list' which is referred to here :

https://infohost.nmt.edu/tcc/help/pubs/python/web/sequence-types.html

as a valid sequence type but apparently there's a detail I'm still missing...


On Mon, Sep 10, 2018 at 5:22 AM, Cameron Simpson  wrote:
> On 09Sep2018 23:00, Chip Wachob  wrote:
>>
>> On Sat, Sep 8, 2018 at 9:14 PM, Cameron Simpson  wrote:
>>>
>>> Actually he's getting back bytearray instances from transfer and wants to
>>> join them up (his function does a few small transfers to work around an
>>> issue with one big transfer). His earlier code is just confused. So he
>>> wants:
>>>
>>>  bytearray().join(results)
>>>
>>> Hacked example:
>>>
>>>  >>> bytearray().join( (bytearray("foo"),bytearray("bah")) )
>>>  bytearray(b'foobah')
>>
>>
>> I understand this example and I can replicate it in the interpreter..
>>
>> But, I'm still missing something here.
>> I presume that I need to instantiate an array of slice_size-sized
>> bytearrays.
>
>
> But no!
>
>> So, when I'm looping through, I can do:
>> for i in range (0, slice_count):
>>   results[i] = spi.transfer(data_out)
>
>
> Python lists are a variable size data structure, so our example goess:
>
>  results = []
>
> which allocates an empty list. Then:
>
>  for i in range(slice_count):
>results.append(spi.transfer(data_out))
>
> suitably adjusted (data_out will be different pieces of the larger data,
> yes?) which grows the array by one item with each append. Your spi.transfer
> function allocates a new bytearray for each return value, so you end up with
> a list of distinct bytearrays.
>
>> Then I can :
>>
>> all_together = butearray().join(results)
>
>
> Yes.
>
>> But I can't seem to be able to find the proper syntax to create the
>> initial array.
>> And any attempt at filling the arrays to test some stand-alone code
>> only give me errors.
>
>
> You _can_ allocate a presized list, but there's almost no benefit and it
> isn't what people normally do.
>
>> Here's my code (all of it)
>
> [...]
>>
>> #results = bytearray( (bytearray(slice_size)*slice_count) )
>> results = bytearray(slice_size)# why isn't this 16 bytes long?
>
>
> It is 16 bytes long when I do it:
>
>>>> bs=bytearray(16)
>>>> bs
>
> bytearray(b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00')
>
>> res_list = (results)*slice_count
>
>
> This is a single 64 byte bytea

Re: [Tutor] Help with building bytearray arrays

[Chip Wachob]

Peter,

I see that clue "[[".

The thread history pretty much sums up what is going on up to this point.

I'll cover it once more:


I'm using Adafruit FT232H Breakout board and Adafruit's library.

https://github.com/adafruit/Adafruit_Python_GPIO

Per Adafruit's example code, I create an SPI interface:

https://learn.adafruit.com/adafruit-ft232h-breakout?view=all

I can access this interface with the spi.transfer() spi.write() and
spi.read() functions.

The transfer() function (see link above) accepts and returns bytearray
objects data.

My application requires me to send a large payload down the SPI port
to my circuit, and read back the same in full duplex.

Due to a limitation of the part, I can't send my whole bytearray at
one time.  It crashes out.

So, my solution was to send as large of a chunk of data at one time as
I can inside a loop until I get to the end of the data to be sent.

Meanwhile, the transfer() function is returning 'chunks' of bytearrays.

Earlier discussions here indicated that the best way was to :

results = []

for i in range (0, slice_size):
   results.append(transfer(data_out))

Then, concatenate the results into a long bytearray so I can use it
elsewhere in my code.

all_together = bytearray().join(results)

I _thought_ that this was going to create a concatenated list of all
the returned results:

all_together = [results[0] + results[1] + results [2] + results[3]]
-- for example

but, as you point out, I got:

all_together = [[results[0]], [results[1]], [results[2]]. [results[3]]
 -- I'm sure that the brackets and braces are not syntactically
correct, but I think you get the idea.

So I see why my .join() isn't working.  I'm not sure how to fix it though.


Related to this, but I'm not yet at that point until I get this
resolved, I need to walk through the all_together data and find where
the data changes from one value to another..

My background is in C and other 'historical' languages, so I'm trying
to get a hold of the way Python handles arrays, which is different
that the way I've thought for 20+ years.. :)

I hope this helps.

I'm beginning to wonder if Python was the right choice for this
project.. but it's too late for me to switch now.

Thanks to everyone for your comments and patience.



On Mon, Sep 10, 2018 at 1:42 PM, Peter Otten <__pete...@web.de> wrote:
> Chip Wachob wrote:
>
>> Cameron,
>>
>> Thank you again for the insight.
>>
>> Yes, data_out is an equivalently-sized 'chunk' of a larger array.
>>
>> I'm 'getting' this now..
>>
>> So, without all the fluff associated with wiggling lines, my function
>> now looks like this:
>>
>> def RSI_size_the_loop():
>>results = []
>>all_together = []   # not certain if I need this, put it in in an
>> attempt to fix the incompatibility if it existed
>>
>>for x in range (0, MAX_LOOP_COUNT, slice_size):
>>   results.append(my_transfer(disp, data_out, slice_size)
>>
>>   print " results ", x, " = ", results  # show how results grows
>> on each iteration
>>
>>all_together = bytearray().join(results)
>>
>>print " all together ", all_together
>>
>>
>> I can observe results increasing in size and the last time through the
>> loop:
>>
>>  results  48  =
>>
> [[bytearray(b'\xff\xff\xff\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00')],
>
> Note that there are two '[' at the start of the list. This means that the
> first list item is another list. In fact you seem to have a list of single
> item lists like
>
> [["foo"], ["bar"], ...]
>
> when you need
>
> ["foo", "bar", ...]
>
> Of course join will fail with that:
>
>>>> "".join(["foo", "bar"])
> 'foobar'
>>>> "".join([["foo"], ["bar"]])
> Traceback (most recent call last):
>   File "", line 1, in 
> TypeError: sequence item 0: expected string, list found
>
> I think the error message is pretty clear ;)
>
> Have a look into your my_transfer() function to find out why it returns a
> list (or show us the code).
>
> ___
> Tutor maillist  -  Tutor@python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Tutor] Best solution to modifying code within a distributed library

[Chip Wachob]

Hello once again,

I'm sure this is probably way outside my 'pay grade' but I would like
to try an experiment and I'm not sure how to go about it.

I'm using the Adafruit FT232 libraries found here:

https://github.com/adafruit/Adafruit_Python_GPIO/blob/master/Adafruit_GPIO/SPI.py

I'm experiencing some wiggling of the IO lines when I configure the IO
pin direction.

I've looked through the code in the FT232H.py file and found what I
believe to be the culprit.

I would like to comment out line 340 (self.mpsse_write_gpio()) to
prove that this is what is causing glitches that I do not want.

Using the .__file__ inside the interpreter I learned that the file is
located here on my machine:

/usr/local/lib/python2.7/dist-packages/Adafruit_GPIO-1.0.3-py2.7.egg/Adafruit_GPIO/FT232H.pyc

But obviously, this is a binary file.

If I understand enough about Python, I believe that I need to edit the
FT232H.py file in the .egg file to implement the change.  There is
also likely some sort of compilation that needs to be done once the
change is made...

BUT

As I've also learned from our friend Google, one should not be editing
.egg files, etc.

So, can the experts out there point me to the proper method of
implementing the change?

IF, this ends up resolving my issues, then I presume that I will need
to create my own .egg file.  Is this accurate?

As always thank you in advance for your time,
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Best solution to modifying code within a distributed library

[Chip Wachob]

Mats,

Silly question here..

But after using the git clone command, I've got a directory of the
Adafruit project in the same directory as my project.

When I import the library, will I get the 'installed' library, or do I
get the library that is in the project directory?

If I have to specify which library to use, how is that done?

Thanks,


On Wed, Sep 19, 2018 at 7:51 PM, Mats Wichmann  wrote:
> On 09/19/2018 03:47 PM, Chip Wachob wrote:
>> Hello once again,
>>
>> I'm sure this is probably way outside my 'pay grade' but I would like
>> to try an experiment and I'm not sure how to go about it.
>>
>> I'm using the Adafruit FT232 libraries found here:
>>
>> https://github.com/adafruit/Adafruit_Python_GPIO/blob/master/Adafruit_GPIO/SPI.py
>>
>> I'm experiencing some wiggling of the IO lines when I configure the IO
>> pin direction.
>>
>> I've looked through the code in the FT232H.py file and found what I
>> believe to be the culprit.
>>
>> I would like to comment out line 340 (self.mpsse_write_gpio()) to
>> prove that this is what is causing glitches that I do not want.
>>
>> Using the .__file__ inside the interpreter I learned that the file is
>> located here on my machine:
>>
>> /usr/local/lib/python2.7/dist-packages/Adafruit_GPIO-1.0.3-py2.7.egg/Adafruit_GPIO/FT232H.pyc
>>
>> But obviously, this is a binary file.
>>
>> If I understand enough about Python, I believe that I need to edit the
>> FT232H.py file in the .egg file to implement the change.  There is
>> also likely some sort of compilation that needs to be done once the
>> change is made...
>>
>> BUT
>>
>> As I've also learned from our friend Google, one should not be editing
>> .egg files, etc.
>
> You'll want to get the original and work from there. You already know
> where it is - you've included the github link.
>
> It's hard to know how much needs to be explained here... roughly, in
> your project you want to clone the git tree, and make sure that's what
> your experiment is picking up.  That would start as:
>
> git clone https://github.com/adafruit/Adafruit_Python_GPIO.git
>
> or of you want to start with something you might want to create a github
> pull request to the maintainer, make sure you have a github account,
> click the fork button on the github page, then in your own account find
> the URL to give to "git clone" for your fork, and start from there.
>
> to do an experiment, the former ought to be enough, but "there are more
> details", depending on what you're familiar with as far as these tools.
>
> Do write back with more questions if you go down this path...
>
>
>
> ___
> Tutor maillist  -  Tutor@python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Best solution to modifying code within a distributed library

[Chip Wachob]

David,

I should have pointed out that I've already posted to the forums there
and there is only crickets.

So, I've taken it upon myself to attempt to solve it for myself.  But,
as I noted, I'm very very new to Python and the whole .egg, pip, git
thing, and that's what lead to my query here.

Mats,

Thanks for the pointer on the clone.  I think I'll take that approach
for now.  Since this is an experiment, this seems like the right path
to take.

Thanks to all for the information.


On Wed, Sep 19, 2018 at 10:35 PM, David Rock  wrote:
>
>> On Sep 19, 2018, at 18:51, Mats Wichmann  wrote:
>>
>> On 09/19/2018 03:47 PM, Chip Wachob wrote:
>>> Hello once again,
>>>
>>> I would like to comment out line 340 (self.mpsse_write_gpio()) to
>>> prove that this is what is causing glitches that I do not want.
>>>
>>
>> You'll want to get the original and work from there. You already know
>> where it is - you've included the github link.
>>
>
> There’s another option… Ask Adafruit directly.
>
> They are amazingly helpful and would be more than happy to hear if there’s a 
> potential issue with their code.
>
>
> —
> David Rock
> da...@graniteweb.com
>
>
>
>
> ___
> Tutor maillist  -  Tutor@python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Best solution to modifying code within a distributed library

[Chip Wachob]

Looks like I'm golden in this regard.  My first path element is ''
which is what I'd want if I'm including a modified library.

Thank you,



On Thu, Sep 20, 2018 at 6:14 PM, Mats Wichmann  wrote:
> On 09/19/2018 09:59 PM, Chip Wachob wrote:
>> Mats,
>>
>> Silly question here..
>>
>> But after using the git clone command, I've got a directory of the
>> Adafruit project in the same directory as my project.
>>
>> When I import the library, will I get the 'installed' library, or do I
>> get the library that is in the project directory?
>>
>> If I have to specify which library to use, how is that done?
>
> you look at, and possibly modify, sys.path
>
>
>   >>> import sys
>   >>> sys.path
>   ['', '/usr/lib64/python36.zip', '/usr/lib64/python3.6',
> '/usr/lib64/python3.6/lib-dynload',
> '/usr/lib64/python3.6/site-packages', '/usr/lib/python3.6/site-packages']
>
>
> the first element '' is the local directory, so with my sys.path, it
> would pick the local one first.
>
>
> if you wanted the opposite, that is be _sure_ you get the installed one,
> you could use a stanza something like this:
>
>
>savepath = sys.path
>sys.path = [path for path in sys.path if path.strip('.')]
>import foo
>sys.path = savepath
>
>
> but this is actually kind of tricky stuff, trying to deal with possibly
> two modules of the same name, so tread carefully.
>
>
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Tutor] Shifting arrays as though they are a 'word'

[Chip Wachob]

Hello,

I was not able to find any answers in the archive on this one.

I'm wondering if this task can be done in a better way than what I've
attempted..

I have an array of bytes.  Up to 64, which makes for 512 bits.

I am reading these bytes in serially, and once I have a collection of
them, I want to shift them by 'n' bits.  The size of the array and the
number of bits are both variable up to the limit of 64/512.

Now, I've played around with looping through the bytes and taking the
LSByte and shifting it by 'n' bits using >> or << and that works for
the first byte.  But then I need to take the next byte in the sequence
and shift it in the opposite direction by 8-n bits using << or >>
(opposite the LSByte direction), respectively.  Then I OR the two
bytes and save them into the location of the LSByte and then move to
the next byte in the sequence and so on.  While this works most of the
time, I sometimes get strange behavior at the 'fringes' of the bytes.
Sometimes I end up with zero, or the shift seems to 'roll over'.

I'm thinking that maybe there's a way to treat the array / list and
shift allowing the bits to transfer from byte to byte as needed.
Maybe by concatenating the bytes into one huge word and then breaking
it apart again?

I'm thinking that you folks out there know of a built-in function, or,
an easier and more predictable way to accomplish the same.

Thank you,
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] [spoiler] Re: Shifting arrays as though they are a 'word'

[Chip Wachob]

e read 
= ", hex(curr_read_ary[num_mod])
print " num_mod = ", 
num_mod
print " stop change = 
", hex(stop_chng_ary[num_mod])
print " Stop change = 
", ((num_mod * 8) + stop_chng_ary[num_mod])
print " Stop State = ", 
bit_state_human_read

# update array with current state of 
stops
last_read_ary = curr_read_ary[:]
# copy ary to ary

except KeyboardInterrupt:   # watches for ctrl-c
# provides a clean exit
print "\n\n Test Complete \n\n"

if(disp): print " -- Report Complete"

return



On 10/6/18, Peter Otten <__pete...@web.de> wrote:
> Chip Wachob wrote:
>
>> Hello,
>>
>> I was not able to find any answers in the archive on this one.
>>
>> I'm wondering if this task can be done in a better way than what I've
>> attempted..
>>
>> I have an array of bytes.  Up to 64, which makes for 512 bits.
>>
>> I am reading these bytes in serially, and once I have a collection of
>> them, I want to shift them by 'n' bits.  The size of the array and the
>> number of bits are both variable up to the limit of 64/512.
>>
>> Now, I've played around with looping through the bytes and taking the
>> LSByte and shifting it by 'n' bits using >> or << and that works for
>> the first byte.  But then I need to take the next byte in the sequence
>> and shift it in the opposite direction by 8-n bits using << or >>
>> (opposite the LSByte direction), respectively.  Then I OR the two
>> bytes and save them into the location of the LSByte and then move to
>> the next byte in the sequence and so on.  While this works most of the
>> time, I sometimes get strange behavior at the 'fringes' of the bytes.
>> Sometimes I end up with zero, or the shift seems to 'roll over'.
>>
>> I'm thinking that maybe there's a way to treat the array / list and
>> shift allowing the bits to transfer from byte to byte as needed.
>> Maybe by concatenating the bytes into one huge word and then breaking
>> it apart again?
>>
>> I'm thinking that you folks out there know of a built-in function, or,
>> an easier and more predictable way to accomplish the same.
>
> Here are two ways to implement the left shift:
>
> def bitshift(b, n, byteorder="big"):
> size = len(b) + (n + 7) // 8
> shifted = int.from_bytes(b, byteorder) << n
> return shifted.to_bytes(size, byteorder)
>
>
> def bitshift2(b, n):
> nbytes, nbits = divmod(n, 8)
> if nbits:
> a = [0]
> for bb in b:
> hi, lo = divmod(bb << nbits, 256)
> a[-1] |= hi
> a.append(lo)
> b = bytes(a)
> return b + b"\x00" * nbytes
>
>
> assert bitshift(b"\xaa\xbb", 12) == b"\x0a\xab\xb0\x00"
> assert bitshift2(b"\xaa\xbb", 12) == b"\x0a\xab\xb0\x00"
>
>
> ___
> Tutor maillist  -  Tutor@python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
>
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] [spoiler] Re: Shifting arrays as though they are a 'word'

[Chip Wachob]

First,

Steven,

Thank you for your insight.



On 10/8/18, Steven D'Aprano  wrote:
> On Mon, Oct 08, 2018 at 09:16:16AM -0400, Chip Wachob wrote:
>
>> - What follows is a mini version, the array could be any size up to 64
>> bytes
>>
>> input:  10010010 . 1110 . 01010011
>>
>> shift 'x' (for example, I'll use 1)
>>
>> output: 01001001 . 01110111 . 10101001
>
> The first two seem to be simple right bit-shift:
>
> 10010010 -> 01001001
> 1110 -> 01110111
>
> but the third seems to be something completely different, neither a left
> nor a right shift:
>
> 01010011 -> 10101001

I _think_ that you got the input and output confused on the left-most
byte.  I checked to make sure that I had not made a typo, but I think
that it should be

10101001 => 01010011 which is a right shift of one bit.



>
> A left-bit shift would give this:  10100110
> and a right shift would give this: 00101001
>
> So now I don't know what you want.
>
> [...]
>> The above are not actual data that have been manipulated by the
>> software but illustrations of observations.  The real data is 'random'
>> so I'm doing the best to recall from memory what happens.
>
> Seriously, "this is what I vaguely remember happening" is no way to
> debug software. Show us *actual data* with *actual results* and maybe we
> can help, otherwise we're just pissing into the wind here.
>

I want to say also that I realize that my ethereal results were not
very helpful.   I'm having trouble actually capturing real data as it
seems to be 'intermittent'.  I fully expected the Brain Trust to come
back with A) There's a much better way to do what you're doing, or B)
Hey, there's a function for that, etc.

I'm going to keep digging into my problem using these suggestions and
get to the bottom of it.

I'm sorry if you felt that I was wasting your time.

>
> [...]
>> Ideally, I'd love to be able to say :
>>
>> # here's the array / list
>> rx_data = []
>>
>> # read through all the bytes
>> # stash the results into the list
>> for x in range (bytes_of_data):
>>rx_data[x] = read_data()[0]
>>
>> # bit shift the entire lot
>> rx_data = rx_data >> bits_to_shift.
>>
>> Which would provide me with the output described above.
>
> Take your byte-array returned from read_data, the *lot* of it, not just
> the first byte. Convert to an int, and shift the int.

I would be happy to take the 'lot' of it, except that the function
returns only one byte at a time.  It's a library / driver provided by
the manufacturer.

>
> py> b = bytearray(4)  # work with four bytes
> py> b[1] = 255  # fill in some non-zero values
> py> b[2] = 127
> py> b
> bytearray(b'\x00\xff\x7f\x00')
> py> n = int.from_bytes(b, 'big')

I'm not familiar with this int.from_bytes.

Being new to Python, I don't have a mental library of these nifty
functions.  How / where does one find a library of these wonderful
functions so I can search to find something that might be really
handy?

Could I use the int.from_bytes on a 64-byte array, or would that end
up breaking things?  Then, I could shift the whole mess, but I would
have to break it back apart to use the data.  Which, I presume I could
slice out of the larger 'word'.


> py> hex(n)
> '0xff7f00'
> py> bin(n >> 1)
> '0b0111000'
> py> bin(n << 1)
> '0b01110'
>
> Does that help?
>

It does help somewhat.

>
>
>
> --
> Steve
> ___
> Tutor maillist  -  Tutor@python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
>
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] [spoiler] Re: Shifting arrays as though they are a 'word'

[Chip Wachob]

Mats,

Thanks for the suggestion.

Even if I did mask, I would still have to shift the data to line it up
properly.  And the shifting seems to be where I'm introducing my
error(s).

This bit stream is complicated in that I have to reverse the bytes as
well since I'm reading off the 'end' of a bunch of registers.  So, the
MSByte comes to me first and goes into the [0] location and the extra
bits get added to the '0th' end of the LSByte.  It is a very unhandy
hardware setup but one that I don't have any control over.

All good insight.

On 10/8/18, Mats Wichmann  wrote:
> On 10/08/2018 07:16 AM, Chip Wachob wrote:
>> All,
>>
>> Sorry for not being more clear.  I think I was nearing my fill of
>> Python for the day when I wrote.
>>
>> Now, refreshed, let me see if I can fill in the blanks a bit more.
>>
>> - Bits that are shifted in either direction off the end of the
>> 64-byte(or 'n' size) collection are of no use.  They are actually
>> phantom bits created by the hardware that I'm trying to get rid of by
>> using this shifting.
>
> if you have stray bits, masking is usually a better solution than shifting.
> ___
> Tutor maillist  -  Tutor@python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
>
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] [spoiler] Re: Shifting arrays as though they are a 'word'

[Chip Wachob]

Alan,

Thank you.

Another related question.  How do I know I want to do dir(int) vs.. I
don't know dir(long)?

And, when I do dir(int) I get the following results.  I must not be
seeing the from_bytes in this list...

>>> dir(int)
['__abs__', '__add__', '__and__', '__class__', '__cmp__',
'__coerce__', '__delattr__', '__div__', '__divmod__', '__doc__',
'__float__', '__floordiv__', '__format__', '__getattribute__',
'__getnewargs__', '__hash__', '__hex__', '__index__', '__init__',
'__int__', '__invert__', '__long__', '__lshift__', '__mod__',
'__mul__', '__neg__', '__new__', '__nonzero__', '__oct__', '__or__',
'__pos__', '__pow__', '__radd__', '__rand__', '__rdiv__',
'__rdivmod__', '__reduce__', '__reduce_ex__', '__repr__',
'__rfloordiv__', '__rlshift__', '__rmod__', '__rmul__', '__ror__',
'__rpow__', '__rrshift__', '__rshift__', '__rsub__', '__rtruediv__',
'__rxor__', '__setattr__', '__sizeof__', '__str__', '__sub__',
'__subclasshook__', '__truediv__', '__trunc__', '__xor__',
'bit_length', 'conjugate', 'denominator', 'imag', 'numerator', 'real']





On 10/9/18, Alan Gauld via Tutor  wrote:
> On 09/10/18 02:48, Chip Wachob wrote:
>
>>> bytearray(b'\x00\xff\x7f\x00')
>>> py> n = int.from_bytes(b, 'big')
>>
>> I'm not familiar with this int.from_bytes.
>>
>> Being new to Python, I don't have a mental library of these nifty
>> functions.  How / where does one find a library of these wonderful
>> functions so I can search to find something that might be really
>> handy?
>
> The interactive prompt.
>>>> dir(int)
>
> And the help() function
>
>>>> help(int)
>
> And finally the online documentation and search engine.
> But dir/help are usually faster.
>
> --
> Alan G
> Author of the Learn to Program web site
> http://www.alan-g.me.uk/
> http://www.amazon.com/author/alan_gauld
> Follow my photo-blog on Flickr at:
> http://www.flickr.com/photos/alangauldphotos
>
>
> ___
> Tutor maillist  -  Tutor@python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
>
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] [spoiler] Re: Shifting arrays as though they are a 'word'

[Chip Wachob]

I'm sorry that my explanations are obtuse...

Hopefully this will clarify things.  Again, not knowing how much
information is too much...  I know I can be too verbose and that's not
always a good thing.


I have a bank of registers that I read via SPI.   The SPI API is from
the IC manufacturer.

The maximum size of that bank is 512 bits.  Or, 64 bytes.

The register bank can be any valid integer between 1 and 512 long and
changes each time the program is run.  There is nothing that limits it
to byte-sized counts.

To determine the size of the bank, I can saturate the register with
ones, followed by a stream of 0x00's.  When I see the reads from the
bank come back as something other than 0xFF, then I know I've 'pushed'
zeros through the entire length of the registers.  I keep a running
count of bytes that contain 0xFF when I do this and if the last byte
is something other than 0xFF or 0x00 then I add one to my byte count.
This is the byte count for this register until the program is run
again.  The next time, the bit count is likely to be different.

The input to the register array is at the least significant bit.  The
output is connected to the most significant bit.

When I read a register, the SPI pumps out the MSByte first, which goes
into my '0'th location in my list (array).  (The data is read as MSbit
first BTW, which is accurate given this configuration).  When the read
is complete, I have something that looks like this:

data_in[0] = MSByte
data_in[1] = MSByte -1
...
data_in[n-1] = LSByte -1
data_in[n] = LSByte*

*The data_in[n] (LSByte) is the byte that can have extra '1's in it's
least significant bit positions due to the pull-up on the data line..
Only the LSByte has the 'extra' bits.  The other bytes are 'normal'.

To handle the data in what I'll call a 'normal' order, I use the
list.reverse() function that I discovered somewhere on the web.  That
puts the bytes in what I would call the 'proper' order.  So then I
have :

data_in[0] = LSByte
data_in[1] = LSByte - 1
data_in[2] = LSByte - 2
...
data_in[n-1] = MSByte -1
data_in[n] = MSByte

I still have the extra bits that were added to the lower bits of
data[0], I need to right shift to remove the 'added' ones.  If this
were only a byte sized register, I would do the shift and that would
be sufficient but that is not likely.

So, in the example of just one additional bit that needs removing:

data_in[0] >>= 1

but, I need the least significant bit from data_in[1] to move into the
most significant bit of the now-shifted data_in[0].  This of course
has a domino affect down the line until I reach data_in[n].  Since I
know the length of the register from my initialization process, I can
ignore / mask the most significant bit of data_in[n] in my other
calculations.

My initial solution was to take data_in[1] and _left_ shift it by the
amount of 8 - shift_count.

scratch = data_in[n] <<= (8 - #)

This moved the LSbit to the MSbit location of scratch.  I then OR the
data_in[0] shifted result and the scratch result to get my final
data_in[0] value.

data_in[0] = data_in[0] | scratch

I really hope that this makes sense.

The API is providing me with a bytearray-formatted result, but the API
only reads one whole byte, or eight bits, at a time.  So, I've been
using only the [0] of the bytearray for my data.  I hope that this
answers your question about octets.  I'm not entirely sure I follow
the question well enough to answer.





On 10/9/18, Alan Gauld via Tutor  wrote:
> On 09/10/18 02:55, Chip Wachob wrote:
>
>> This bit stream is complicated in that I have to reverse the bytes as
>> well since I'm reading off the 'end' of a bunch of registers.  So, the
>> MSByte comes to me first and goes into the [0] location and the extra
>> bits get added to the '0th' end of the LSByte.  It is a very unhandy
>> hardware setup but one that I don't have any control over.
>
>
> I'm not sure I understand this fully.
> Do you mean that each byte has extra
> bits added or that only the last (LSByte)
> has extra bits?
>
> If it's the latter then it becomes a fairly
> trivial exercise of slicing the data off
> rather than shifting. But then you get left
> with a non-byte.
>
> If it's the former then I don't understand
> how you can determine "byte" boundaries!
>
> Also when you say you have to "reverse" the
> bytes do you mean the bits within each byte
> need to be reversed? Or merely that the bytes
> themselves come in the reverse order? If
> its the latter then the bytearray.reverse()
> method should suffice to fix that issue.
>
> Finally, just to clarify what's happening.
> It sounds like you have an API that delivers
> 8 bits(octets) at a time? (Not ne

Re: [Tutor] [spoiler] Re: Shifting arrays as though they are a 'word'

[Chip Wachob]

Peter,

Thank you for taking the time to create the example.

I'm new to Python, and, as a result, about a quarter of your example
makes sense to me.  The remainder I will have to start googling to see
if I can get my head around it.


On 10/9/18, Peter Otten <__pete...@web.de> wrote:
> Chip Wachob wrote:
>
>> All,
>>
>> Sorry for not being more clear.  I think I was nearing my fill of
>> Python for the day when I wrote.
>>
>> Now, refreshed, let me see if I can fill in the blanks a bit more.
>
> You have posted code sprinkled with prints and irrelevant details.
> Please rewrite the relevant part into a simple function,
>
> def bitshift_right(data, nshift):
>...
>
> just the pure algorithm without any prints. Then find input data that
> produces an AssertionError in the snippet below:
>
> input = ...
> rshift = ...
> wanted_output = ...
> assert bitshift_right(input, rshift) == wanted_output
>
> I'm sure then someone is able and willing to point out the error in your
> implementation.
>
> In the mean time here's my effort which also provides an idea on how to wrap
>
> the shifting into a simple class:
>
> $ cat bitshift_oo.py
> format_bits = "{:08b}".format
> try:
> format_bits(42)
> except ValueError:
> # python 2
> def format_bits(b):
> return bin(b)[2:].zfill(8)
>
>
> def to_bits(s, sep=""):
> return sep.join(map(format_bits, s))
>
>
> def bitshift_right(b, n):
> nbytes, nbits = divmod(n, 8)
> length = len(b)
> if nbits:
> nbits = 8 - nbits
> a = [0]
> for bb in reversed(b):
> hi, lo = divmod(bb << nbits, 256)
> a[-1] |= lo
> a.append(hi)
> b = bytearray(a[::-1])
> return (bytearray(nbytes) + b)[:length]
>
>
> class Bytes(bytearray):
> def __rshift__(self, n):
> return Bytes(bitshift_right(self, n))
>
> def __str__(self):
> return to_bits(self, " ")
>
>
> if __name__ == "__main__":
> SAMPLE = Bytes(b"\xaa\xbb")
> for i in range(12):
> print(SAMPLE >> i)
> $ python bitshift_oo.py
> 10101010 10111011
> 01010101 01011101
> 00101010 10101110
> 00010101 01010111
> 1010 10101011
> 0101 01010101
> 0010 10101010
> 0001 01010101
>  10101010
>  01010101
>  00101010
>  00010101
> $
>
> PS: An easier approach would be to match the to_bits() function with a
> to_bytes() function. Then you can implement the shift as a simple string
> manipulation:
>
> def bitshift_right(b, n):
> bits = to_bits(b)
> shifted = ("0" * n + bits)[:len(bits)]
> return to_bytes(shifted)
>
>
> ___
> Tutor maillist  -  Tutor@python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
>
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] [spoiler] Re: Shifting arrays as though they are a 'word'

[Chip Wachob]

Alan,

Yes, 2.7 here.  I wanted to use 3, but the API only supports 2.7.
And, I'm not about to get into rewriting an API. :)

So, octets it is.  Thank you for the clarification.

Yes, my background is C, let me look into your suggested method.

It is the shifting across those byte boundaries that is really messing
me up.  If only they were all just bytes..  but then that wouldn't be
any 'fun' would it?

Thank you,


On 10/9/18, Alan Gauld via Tutor  wrote:
> On 09/10/18 13:45, Chip Wachob wrote:
>
>> Another related question.  How do I know I want to do dir(int) vs.. I
>> don't know dir(long)?
>
> Because you want to convert the byte array into an int.
> And there is no long in Python...
>
>> And, when I do dir(int) I get the following results.  I must not be
>> seeing the from_bytes in this list...
>>
>>>>> dir(int)
>> ['__abs__', '__add__', '__and__', '__class__', '__cmp__',
>
> Ah, I assume this means you are using Python v2.7?
> I assumed you were using v3. The bytes operations were,
> I think, added in v3.
>
> That complicates things, I think you are probably best
> converting your byte list into a string and then using
> the struct module to convert your bytearray into an int.
> (struct is a bit like C's sscanf function if
> that helps - I seem to recall you know C?)
>
> --
> Alan G
> Author of the Learn to Program web site
> http://www.alan-g.me.uk/
> http://www.amazon.com/author/alan_gauld
> Follow my photo-blog on Flickr at:
> http://www.flickr.com/photos/alangauldphotos
>
>
> ___
> Tutor maillist  -  Tutor@python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
>
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Tutor] Displaying Status on the Command Line

[Chip Wachob]

Hello,

I'm sure that this is simple and my searches have just not used the
correct words.

What I would like to do is display, on a single line, in the terminal
/ command line a progress percentage, or, simply a sequence of - / -
\, etc.. or even, accumulating period characters.

What would the escape codes be, or is there a better way to handle this?

Note that I need this to be platform agnostic.

Thank you in advance for your insight.
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Displaying Status on the Command Line

[Chip Wachob]

Wow!

Thank you!

Lots of things for me to try.

I should have mentioned that I'm working with Python 2, but I think I
can parse my way through these examples.

Best,

On 11/7/18, Cameron Simpson  wrote:
> On 08Nov2018 10:00, Steven D'Aprano  wrote:
>>> Note that I need this to be platform agnostic.
>>
>>That's hard, even on a single platform like Linux.
>
> Most, nearly all, terminal honour carriage return and backspace. That is
> technically enough. Even terminals with a destructive backspace (rare -
> it is normally just a cursor motion) can get by (backspace, overwrite
> the new text).
>
>>Most xterminal windows use either the xterm or vt1000 set of commands,
>>which are broadly similar, but that's not guaranteed. If somebody
>>happens to be running a different terminal type, they'll see something
>>weird.
>>
>>And I have no idea what happens on Windows.
>
> I'd sort of expect Windows terminals, even cmd.exe, to accept the ANSI
> sequences, which is what vt100 and xterms use. But that is expectation,
> not knowledge.
>
> Cheers,
> Cameron Simpson 
> ___
> Tutor maillist  -  Tutor@python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
>
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Tutor] Putting a Bow on It

[Chip Wachob]

Hello,

I've been off working on other projects, but I'm finally back to the
project that so many here have helped me work through.  Thank you to the
group at large.

So, this leads me to my question for today.

I'm not sure what the "correct" term is for this, but I want to create what
I'll call a Package.

I want to bundle all my scripts, extra libraries, etc into one file.  If I
can include a copy of Python with it that would be even better.

I'm looking for the simplest approach for the user to install.  Eg:
libraries will install themselves in the correct directories, etc, so that
there is minimal pain on the part of the user.

I would need to do this for both Windows and Linux so something that is
platform agnostic would be great.

Precise Questions:

A) What sort of term is the proper term for this process that I want to do?
B) Good link suggestions where I can go read / learn about it?
C) Can I just include the .pyc files and the main .py file in the
'release'?  I'm thinking about protecting accidental editing of the .py
files leading to malfunctions.
D) Any other tips or pointers that you would offer as part of attempting
this process?

Again, thank you for all the assistance along the way.

Best,
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Putting a Bow on It

[Chip Wachob]

Thanks.  These are both great helps to get me started.

The little bit of searching does leave me a little bit confused, but the
reference to the book is somewhat helpful / encouraging.

I see a lot of people saying that certain approaches have been depreciated,
then re-appreciated (?) then depreciated once more and so on..  that sure
makes it confusing to me.  Unfortunately since I'm using someone's pre-made
libraries, and that requires 2.7, I'm sort of locked at that version, but
it seems like most, if not all, of these options will work for any version
of Python.

These posts give me some keywords that should help me narrow the field a
bit.

I realize that choosing a tool is always a case of personal preference.  I
don't want to start a 'this is better than that' debate.

If the 'pros' out there have more input, I'm all ears.

Best,






On Sun, Feb 10, 2019 at 7:11 AM Albert-Jan Roskam 
wrote:

>
>
> On 8 Feb 2019 19:18, Chip Wachob  wrote:
>
> Hello,
>
> I've been off working on other projects, but I'm finally back to the
> project that so many here have helped me work through.  Thank you to the
> group at large.
>
> So, this leads me to my question for today.
>
> I'm not sure what the "correct" term is for this, but I want to create
> what
> I'll call a Package.
>
> I want to bundle all my scripts, extra libraries, etc into one file.  If I
> can include a copy of Python with it that would be even better.
>
>
>
> ==》 Hi, also check out pyscaffold or the similar cookiecutter to generate
> "package skeletons". Directory structure, default setup.py, Readme.md file
> template, etc etc. I've never used them, but py2exe or cx_freeze might also
> interest you.
>
>
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Putting a Bow on It

[Chip Wachob]

Mats,

You put just the right words to my difficulties.  Thank you.

Since I last posted, I attempted to use Setuptools, and got a handful of
files that were less than 1kB.  I also attempted to use py2exe (I know this
is only for Windoze, but I wanted to find some sliver of success) and
py2exe does not like the fact that I have Python 2.7.15 installed (which I
am locked to).  I tried using pip to install py2exe==0.6.9 (a version that
says it supports Python 2.7) but pip is telling me that it can't find any
version of Python 2.7.

I'm trying to make the installation of the script / executable as simple as
possible because I know those who will be using it will NOT be Python savvy
in the remotest way.

Thanks for confirming that I'm not simply going mad...

Best,


On Mon, Feb 11, 2019 at 11:30 AM Mats Wichmann  wrote:

> On 2/11/19 6:48 AM, Chip Wachob wrote:
> > Thanks.  These are both great helps to get me started.
> >
> > The little bit of searching does leave me a little bit confused, but the
> > reference to the book is somewhat helpful / encouraging.
> >
> > I see a lot of people saying that certain approaches have been
> depreciated,
> > then re-appreciated (?) then depreciated once more and so on..  that sure
> > makes it confusing to me.  Unfortunately since I'm using someone's
> pre-made
> > libraries, and that requires 2.7, I'm sort of locked at that version, but
> > it seems like most, if not all, of these options will work for any
> version
> > of Python.
> >
> > These posts give me some keywords that should help me narrow the field a
> > bit.
> >
> > I realize that choosing a tool is always a case of personal preference.
> I
> > don't want to start a 'this is better than that' debate.
> >
> > If the 'pros' out there have more input, I'm all ears.
>
> I'm having the same problems, everybody seems to have an idea of what is
> state of the art, and they don't often agree. And sadly, people do not
> always date their blog entries so you can eliminate what is too old to
> be useful to a "newbie" (a category I fall into with packaging)
>
> There are really two classes of solution:
>
> base tools for manually packaging.  The Python Packaging Authority is
> supposed to be definitive for what the state of these is.
>
> smart systems which automate some or all of the steps.  These are often
> labeled with some sort of hypelabel - Python packaging finally done
> right or some such.  (I've tried a couple and they have failed utterly
> for the project I want to redo the packaging on. My impression is these
> will usually fail if your project is not meant to be imported)
>
>
> ___
> Tutor maillist  -  Tutor@python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
>
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Putting a Bow on It

[Chip Wachob]

Oscar,

Thanks for your full understanding of my situation.  And putting it into
better words than I did.

The code that I've written is entirely Python.  There are necessary
libraries that go along with that, and, due to my inexperience, I am not
100% certain they are pure Python or not.  Some of the drivers from the IC
manufacturer (FTDI) are .dll files that get installed on the machine, and
I'm sure that's going to have to be a separate step.

I've been tooling around with PyInstaller over the last couple of days, and
it seems to be getting me closer to what I would like.  Unfortunately, I
seem to have hundreds of 'missing' modules.  I'm sure that something must
be missing because I can't launch the .exe file that is created.  It looks
like it is going to run, then it comes up and says it can't execute the
script (not the exact words, but you get the idea).  I'm just not sure how
to cull the 'necessary' modules from the ancillary ones.

I fear that once this is 'in the wild' that folks who don't have much / any
experience with Python will have a really hard time getting all the pieces
together.  Maybe I'm over-concerned, but I've always liked to err on the
side of caution when it comes to 'anyone' being able to do something.

I'm also certain that the 'pros' out there are sitting and saying, 'this
isn't hard to do...  he's being a ninny', but I'm really new to the
language, and feel fortunate that I've gotten to the point where I can
actually 'release' something.

One of the other posts, which I don't think I answered, asked about the
Python version.  Since I have libraries that are locked at 2.7, that is
where I have to stay, and that means that some tools are already off the
table since they are for 3.3, etc.

I really appreciate the input from everyone here.  With the pieces of
information you've offered, I've been able to make a little progress.

Best,



On Mon, Feb 18, 2019 at 8:27 AM Oscar Benjamin 
wrote:

> On Mon, 11 Feb 2019 at 17:30, Alan Gauld via Tutor 
> wrote:
> >
> > Three is a lot of work going on in Python land but no universal
> > solution. Some things work better on particular platforms.
> > And building library packages is easier than complete applications.
>
> I think this is the basic problem. There has been a lot of progress in
> distributing libraries with pip, wheel etc. For an experienced
> programmer it's easy to set something up with pip and venv that gets
> you all the libraries you want. It's also pretty straight-forward to
> write a setup.py and use setuptools to build the wheels that pop can
> then install.
>
> Distributing applications is still problematic though and that's what
> Chip wants to do. In particular people often want to be able to do
> this in a single file hence py2exe, pyinstaller etc.
>
> But note that most professional software doesn't work that way.
> Normally a user can download a single file but that file will be the
> installer that then puts all the other files in position.
>
> > But for now, if you have more than a pure Python application,
> > you are pretty much stuck with building multiple solutions with
> > multiple tools.
>
> Chip is yours a pure Python application?
>
> If it is then my suggestion would be to ask your users (if using
> Windows) to install Python from the normal Python downloads page:
> https://www.python.org/downloads/
>
> Then you can give them a zipapp of your application:
> https://docs.python.org/3/library/zipapp.html
>
> I think if your zipapp has the .py extension and the shebang then it
> should be possible to double-click the file to run it on Windows (it's
> possible this depends on options ticked/unticked in the Windows
> installer - might need to be installed for all users).
>
> The advantage of this approach is that Python already has a
> well-maintained installer so you don't have to bother with that part
> yourself. It does mean that it's a two step-process for your users but
> once they have done the first step you can send them as many zipapps
> as you like and it should be fairly easy from there.
>
> I haven't actually tested all of these steps together properly though
> so there's a good chance if you try this that you will quickly
> encounter some difficulty...
>
> --
> Oscar
> ___
> Tutor maillist  -  Tutor@python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
>
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Putting a Bow on It

[Chip Wachob]

Yes, the .dll files will have to be installed separately.  So that's one
step.

I'd like to package the Python code together and have it all in one
directory (which it currently is).  Trying to limit human intervention
(error) as much as possible.

It does look like I'm going to have to have them pip install some modules
though.  Things like numpy, etc.  I wish I could avoid that, but I don't
see a way around it.

I'll try the Hello World first and see what happens.  Thanks for the
reminder.  Sometimes you get so far into the forest you can't find a tree
to save your behind.

Thank you,


On Mon, Feb 18, 2019 at 4:08 PM Oscar Benjamin 
wrote:

> On Mon, 18 Feb 2019 at 13:41, Chip Wachob  wrote:
> >
> > The code that I've written is entirely Python.  There are necessary
> libraries that go along with that, and, due to my inexperience, I am not
> 100% certain they are pure Python or not.  Some of the drivers from the IC
> manufacturer (FTDI) are .dll files that get installed on the machine, and
> I'm sure that's going to have to be a separate step.
>
> Do you mean that the users will already need to run a separate
> installer for the drivers which is already available somewhere? Or do
> you mean that you want to bundle those DLLs yourself?
>
> If you are bundling the DLLs then it's no longer pure Python and the
> zipapp approach won't work (DLLs cannot be used from inside a zip
> file). In that case py2exe/pyinstaller would be the only possible one
> file solutions I know of - they also basically zip up your code but in
> a self-extracting exe.
>
> > I've been tooling around with PyInstaller over the last couple of days,
> and it seems to be getting me closer to what I would like.  Unfortunately,
> I seem to have hundreds of 'missing' modules.  I'm sure that something must
> be missing because I can't launch the .exe file that is created.  It looks
> like it is going to run, then it comes up and says it can't execute the
> script (not the exact words, but you get the idea).  I'm just not sure how
> to cull the 'necessary' modules from the ancillary ones.
>
> Yeah, you can spend a long time going through that. I guess you've
> probably already read this:
>
> https://pyinstaller.readthedocs.io/en/v3.3.1/operating-mode.html#analysis-finding-the-files-your-program-needs
> The suggestion there is to get everything working in one folder mode
> before trying one file mode.
>
> Before that though: have you got it working with a hello world type
> Python script (no dependencies)?
>
> --
> Oscar
> ___
> Tutor maillist  -  Tutor@python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
>
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Tutor] ANSI / VT100 Escape Codes in Windows 7 Environment

[Chip Wachob]

Hello again,

As always, this list has been very helpful, and thank you.

So, I thought I was good to go until I attempted to run my code on a
Windows 7 vintage machine this morning.  The application is intended to be
run in the command window in Windows, from the terminal in Linux...

In the code I had included the ANSI escape characters so I could invert the
text and change the color.  Basically, I wanted to make the warning / error
messages stand out from the crowd.

Eg:
# there's an error in the loop, report it
# format reverse with yellow background
print "\033[7m\033[33m"
print " ERROR IN LOOP SIZE"
.
.
.
# clear formatting
print "\033[m"

As I have now learned, Windows 7 does not support this functionality.
Development had been done on a Windows 10 (and Ubuntu) machine where this
worked just fine.

I've been spending the morning looking at different solutions.  Many of
them seem to involve either including yet aother module, or, worse (IMHO) a
C library that will 'emulate' the ANSI escape sequences, API calls, etc.

Before I go tearing into my currently working code, I'm looking for
feedback from those much more experienced.

I need this code to run correctly on both Windows 7 and 10 (future Linux)
and I'm stuck with Python 2.7 due to other module limitations.

What do the experts out there suggest as the path of least pain and
suffering?

Again, thank you for your time and guidance.
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] ANSI / VT100 Escape Codes in Windows 7 Environment

[Chip Wachob]

Wow!  What a treasure of information.

>From the looks of it, I'm going to give colorama a try.  It seems like it
will do what I want to do, and, if I'm reading the documentation correctly,
it will work cross platform.  Ie: if I am in Linux, it will simply be
ignored.  And, it appears to interpret the ANSI codes (\033[xxxm) when it
comes to Windows...

As an aside, I enjoyed the trip down memory lane.  I still have my Hercules
graphics card and my AMBER monitor for my 286 machine.  You know, the
machine that has two full-height HDD with an MFM interface and provides a
combined 20Mbyte of storage space! And of course a math co-processor...  I
no longer recall how many Kbytes of onboard memory I had, maybe 1024...   :)

Thank you so much for sharing your insights.

Best,


On Wed, Feb 27, 2019 at 9:40 PM boB Stepp  wrote:

> On Wed, Feb 27, 2019 at 6:50 PM Chip Wachob  wrote:
> >
> > Hello again,
> >
> > As always, this list has been very helpful, and thank you.
> >
> > So, I thought I was good to go until I attempted to run my code on a
> > Windows 7 vintage machine this morning.  The application is intended to
> be
> > run in the command window in Windows, from the terminal in Linux...
> >
> > In the code I had included the ANSI escape characters so I could invert
> the
> > text and change the color.  Basically, I wanted to make the warning /
> error
> > messages stand out from the crowd.
>
> As I have been on a "curses" kick lately, I wonder if it would work
> for you?  Of course this means another module, but it is in the
> standard lib on all non-Windows Python versions.  For Windows I found
> this thread on stackoverflow:
>
>
> https://stackoverflow.com/questions/32417379/what-is-needed-for-curses-in-python-3-4-on-windows7
>
> The checked answer gives a link to binaries for Windows, which seems
> to support all Python versions through 3.7, including 2.7.
>
> Just a thought...
>
> --
> boB
> ___
> Tutor maillist  -  Tutor@python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
>
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Tutor] Exception not working as expected?

[Chip Wachob]

Hello,

Python 2.7 & Windows and also Linux are the platforms being used.  Running
the code from the command line / terminal as   python except.py.  Note that
it does work properly in Linux.  So I'm guessing I need to test for a
different exception along with the KeyboardInterrupt??

So, the code below is my example.

When I run it, everything is fine until I attempt to test the
KeyboardInterrupt exception.

When I do this, the traceback is as follows:
Traceback (most recent call last):
  File "except.py", line 71, in 
start_menu()


What I was expecting was the message in the exception to be displayed ("
Keyboard Interrupt - Exiting "), then for the program to exit gracefully.

I'm sure that this is a NOOB mistake, but I can't seem to figure out why
the exception isn't being trapped and handled the way I had hoped.

Can someone shed some light on this for me?

Thank you,




# import standard libraries
import os
import time
import sys

DISPLAY_TIME = 5
g_user_num_items = -1

def start_menu():

start_quit = False

os.system('cls' if os.name == 'nt' else 'clear')

print "\n\n\n\n Welcome to Test.\n"
time.sleep(DISPLAY_TIME)

try:
while not start_quit:

os.system('cls' if os.name == 'nt' else 'clear')

print "\n +++ Test Start Menu +++"
print "Enter the number of items to test,"
print "or, 0 for individual testing."
print "Then press \n"

# wait for it...
num_items = int(raw_input(": "))

if num_items == 0: # individual testing
print "\nIndividual testing requested"
g_user_num_items = num_items
start_quit = True

elif(num_items >= 1 and num_items <= 512):   # multi testing
g_user_num_items = num_items
print "\nItem count of ", g_user_num_items, ", accepted!"
start_quit = True

else:   # unexpected input eg A, >, ? etc
print " Invalid number of items, please re-enter"

time.sleep(DISPLAY_TIME / 2)

# end item count input

# detect ctrl-c
except KeyboardInterrupt:
# provides a clean exit
print "\n\n Keyboard Interrupt - Exiting \n\n"
time.sleep(5)

# trap no choice, just Enter key cases
except ValueError:
# handles non-entry selections like 'Enter'
print "Invalid selection please try again"
time.sleep(DISPLAY_TIME / 2)
start_menu()

return
# end main menu


##
# Main entry point into program
#

if __name__ == "__main__":

start_menu()

print "End of Script\n"
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Exception not working as expected?

[Chip Wachob]

Alan,

Thanks.  Once again, a case of not finding a tree in the forest...  One of
these days I'll remember to KISS and try some SIMPLE experimentation...


So I did your experiment with just the single line raw_input('>'), ran it,
and used Ctrl-C to exit.  Traceback shown below.


C:\Temp_Python\Exception>python rawsimple.py
>Traceback (most recent call last):
  File "rawsimple.py", line 1, in 
raw_input('>')


C:\Temp_Python\Exception>



Note that I ran the same code on a Linux machine and did get the name of
the exception as you expected (KeyboardInterrupt).  And I was able to make
a slightly longer script (again, as suggested) and got the desired results..

Open to suggestions on how to capture the exception who's name isn't
displayed...

I'm still attempting to get my head around Eryk's post and perhaps that
explains why there's no exception listed.


On Thu, Feb 28, 2019 at 8:34 PM Alan Gauld via Tutor 
wrote:

> On 28/02/2019 21:03, Chip Wachob wrote:
>
> > it does work properly in Linux.  So I'm guessing I need to test for a
> > different exception along with the KeyboardInterrupt??
>
> Don't guess, test.
>
> Write a single line script
>
> raw_input('> ')
>
> Run it in a console.
> Hit Ctrl-C while it waits for input.
> See what the stack trace says is the exception
> Edit your script
>
> try:
>   raw_input('> ')
> except :
>   print "I got it that time!"
>
> run the new script.
> Did it work? Hooray! Transfer to the main script.
> If not come back here armed with tracebacks...
>
>
> --
> Alan G
> Author of the Learn to Program web site
> http://www.alan-g.me.uk/
> http://www.amazon.com/author/alan_gauld
> Follow my photo-blog on Flickr at:
> http://www.flickr.com/photos/alangauldphotos
>
>
> ___
> Tutor maillist  -  Tutor@python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
>
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Exception not working as expected?

[Chip Wachob]

Eryk

Thank you for the in-depth explanation and code sample.

I was able to use the sample code to create a functional script.

Now, I'm still not completely following what you mean in the last couple of
paragraphs about the except ValueError.

I realized that I am never actually trapping on the else statement in my
original code.  When I enter inappropriate values, the exception is always
trapping, not the else statement.  Which seems sort of odd to me.

I'm not sure what you mean by putting the except close to the int().

Generally, I feel like I'm getting a handle on this, but that actually
leads me to a related question.

As you can tell, this is just the top of a whole bunch of menu tree paths
that the user can take.  Below is a very rough textual representation of
the layout of the menus.  I understand how the sys.exit() works in the case
of the initial menu, but I don't follow how I can use that same approach to
navigate 'back' up the menu tree.  Each major Test (Type A, Type B, etc) is
a .py file unto itself.

Something about this is feeling like maybe it should be a class, or some
other function that I call throughout my program.

How can I best handle this situation and, how do I trap those invalid
entries and, as you say, "continue" the try statements?


Main Menu
|
|-- Test Type A Items
||-- 1. Test A1
||-- 2. Test A2
||-- 3. Test A3
||-- 0. Back to Main
|
|-- Test Type B Items
||-- 1. Test B1
||-- 2. Test B2
||-- 3. Test B3
||-- 0. Back to Main
|
|-- Test Individual Item
||-- 1. Test I1
||-- 2. Test I2
||-- 3. Test I3
||-- 0. Back to Main
|
|-- Quit

Thank you,


On Thu, Feb 28, 2019 at 9:31 PM eryk sun  wrote:

> On 2/28/19, Chip Wachob  wrote:
> >
> > Python 2.7 & Windows and also Linux are the platforms being used.
> Running
> > the code from the command line / terminal as   python except.py.  Note
> that
> > it does work properly in Linux.  So I'm guessing I need to test for a
> > different exception along with the KeyboardInterrupt??
>
> When reading from the console, we depend on a particular error being
> set in order to distinguish an interrupted read from end-of-file
> (EOF), but this error doesn't get set in Windows 8+ due to a bug in
> the Windows API. I created an issue with Microsoft's console team, but
> thus far they've ignored it.
>
> Due to this bug, Python 2 code that calls raw_input() or input() also
> needs to handle EOFError, which should probably be handled anyway.
> However, it's not enough to separately handle KeyboardInterrupt and
> EOFError, since the KeyboardInterrupt may get raised while handling
> EOFError. This can happen because Windows calls the console control
> handler asynchronously in a new thread. We need a bit of a kludge that
> checks for a delayed KeyboardInterrupt. For example, the following
> shows a case that uses a common handler for EOF and Ctrl+C:
>
> import os
> import sys
> import time
>
> def main():
> try:
> os.system('cls' if os.name == 'nt' else 'clear')
> print "\n\n\n\n Welcome to Test.\n"
> time.sleep(DISPLAY_TIME)
> start_menu()
> except (KeyboardInterrupt, EOFError):
> try:
> time.sleep(0.1)
> except KeyboardInterrupt:
> pass
> # provides a clean exit
> print "\n\n Keyboard Interrupt or EOF - Exiting \n\n"
> time.sleep(5)
> print "End of Script\n"
> return 0
>
> if __name__ == "__main__":
> sys.exit(main())
>
> > # trap no choice, just Enter key cases
> > except ValueError:
> > # handles non-entry selections like 'Enter'
> > print "Invalid selection please try again"
> > time.sleep(DISPLAY_TIME / 2)
> > start_menu()
>
> ValueError should be handled near the int() conversion that raises it,
> and the exception handler should `continue` the loop instead of
> recursively calling start_menu().
> ___
> Tutor maillist  -  Tutor@python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
>
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Tutor] Reading .csv data vs. reading an array

[Chip Wachob]

Hello,

Quick background on what I'm trying to achieve.

I have a data set from a digital storage oscilloscope.  It includes sampled
data points for several electrical events that I'd like to break down and
decode.

The scope generates a single file with all of the events concatenated.  The
data set is a comma-delimited file with time and volts.  There's a header
between events that will allow me to recognize the beginning of a new
event.  This 'raw' file is roughly 4 GBytes of data.  Too much for any
editor to handle.

So, I've written a script that will go through and 'chunk' out each event
and save it to a .csv file.  Now, I have smaller files to work with but
they are still a bit too large for most applications.  These files are
roughly 50 MByte.

In order to extract the desired information from the files, I ended up
reading through each row in the .csv file and finding the events of
interest (rising edges) and saved off data on either side of the event into
arrays, which I saved to .csv files.  I then wrote a script that further
processed the information to generate actual bit-concatenated words..

So, here's where it gets interesting.  And, I'm presuming that someone out
there knows exactly what is going on and can help me get past this hurdle.

When I read through the .csv file and collect the events and the bits, I
get the expected result.  A 16-bit output

To improve efficiency, I then took this snippet of code and included it
into a larger script that will help skip a few manual steps and automate
the process on the original 4 GByte file.  In this larger script, I save
the data that would have normally gone to the .csv files into an array and
I work on the array within the script.  Everything should be the same.. or
so I thought.

When I use the same code, except reading an array, I get results that are
basically 10x the size of the correct output.

I've checked the contents of the array against the contents of the .csv
file (the sources in each of these cases) and they are identical to each
other.  Same size, dimensions and data.

My guess, at this point, is that the way a loop reading a .csv file and the
way a loop reads an array are somehow slightly different and my code isn't
accounting for this.

The other possibility is that I've become code-blind to a simple mistake
which my brain keeps overlooking...

Thank you in advance for your time,



Example code when reading from file:

risingdetected = False

for row in csvReader:
voltage = float(row[1])
# print("test  v ", voltage)

if(voltage > (avg + triglevel) and not(risingdetected)):
# we've found an edge
risingdetected = True
edgearray.append([float(row[0]), float(row[1])])
# print(edgearray)

elif(voltage < (avg + triglevel) and risingdetected):
# print(voltage)
# we've found the falling edge of the signal
risingdetected = False

print("edge array: ", edgearray)# displays the array

arraysize = len(edgearray)# roughly a count <= 33
print("edge array size: ", arraysize)# display size


Example code when reading array inside a script:
(note that I simplified things since it didn't make sense to have a bunch
of repeated math going on for what ended up being the same result.
Specifically, triggervolts, which is the avg + the trigger level)

[I also added in the else statement to see if there were cases where it was
falling through.. which there are.  But that should be the case in both
instances]:

risingdetected = False

for row in range (len(TrigWind)):
# grab the voltage from this entry
voltage = float(TrigWind[row][1])

# if we've not already detected a rising edge
# and we're over the trigger level
if((voltage > triggervolts) and not(risingdetected)):
# set the rising edge detected to help control flow
risingdetected = True
# write the edge entry into the edge array
edgearray.append([float(TrigWind[row][0]),
float(TrigWind[row][1])])

# We've detected a rising edge, now we're looking for a falling
edge
elif((voltage < triggervolts) and risingdetected):
# we're done with the pulse, time to wait for the next one..
risingdetected = False

else:
print("Error")

print("Edge array: ", edgearray)# display the array results

arraysize = len(edgearray)# ends up being about twice the size
of the .csv version
print("Size of edagearray: ", arraysize)# show the size of the
array
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Reading .csv data vs. reading an array

[Chip Wachob]

Oscar and Mats,

Thank you for your comments and taking time to look at the snips.

Yes, I think I had commented that the avg+trigger was = triggervolts in my
original post.

I did find that there was an intermediary process which I had forgotten to
comment out that was adversely affecting the data in one instance and not
the other.  So it WAS a case of becoming code blind.  But I didn't give
y'all all of the code so you would not have known that.  My apologies.

Mats, I'd like to get a better handle on your suggestions about improving
the code.  Turns out, I've got another couple of 4GByte files to sift
through, and they are less 'friendly' when it comes to determining the
start and stop points.  So, I have to basically redo about half of my code
and I'd like to improve on my Python coding skills.

Unfortunately, I have gaps in my coding time, and I end up forgetting the
details of a particular language, especially a new language to me, Python.

I'll admit that my 'C' background keeps me thinking as these data sets as
arrays.. in fact they are lists, eg:

[
[t0, v0],
[t1, v1],
[t2, v2],
.
.
.
[tn, vn]
]

Time and volts are floats and need to be converted from the csv file
entries.

I'm not sure that follow the "unpack" assignment in your example of:

for row in TrigWind:
time, voltage = row  # unpack

I think I 'see' what is happening, but when I read up on unpacking, I see
that referring to using the * and ** when passing arguments to a function...

I tried it anyhow, with this being an example of my source data:

"Record Length",202,"Points",-0.005640001706,1.6363
"Sample Interval",5e-09,s,-0.005639996706,1.65291
"Trigger Point",1128000,"Samples",-0.005639991706,1.65291
"Trigger Time",0.341197,s,-0.005639986706,1.60309
,,,-0.005639981706,1.60309
"Horizontal Offset",-0.00564,s,-0.005639976706,1.6363
,,,-0.005639971706,1.65291
,,,-0.005639966706,1.65291
,,,-0.005639961706,1.6363
.
.
.

Note that I want the items in the third and fourth column of the csv file
for my time and voltage.

When I tried to use the unpack, they all came over as strings.  I can't
seem to convert them selectively..

Desc1, Val1, Desc2, TimeVal, VoltVal = row

TimeVal and VoltVal return type of str, which makes sense.

Must I go through yet another iteration of scanning TimeVal and VoltVal and
converting them using float() by saving them to another array?


Thanks for your patience.

Chip









On Sat, Jul 13, 2019 at 9:36 AM Mats Wichmann  wrote:

> On 7/11/19 8:15 AM, Chip Wachob wrote:
>
> kinda restating what Oscar said, he came to the same conclusions, I'm
> just being a lot more wordy:
>
>
> > So, here's where it gets interesting.  And, I'm presuming that someone
> out
> > there knows exactly what is going on and can help me get past this
> hurdle.
>
> Well, each snippet has some "magic" variables (from our point of view,
> since we don't see where they are set up):
>
> 1: if(voltage > (avg + triglevel)
>
> 2: if((voltage > triggervolts)
>
> since the value you're comparing voltage to gates when you decide
> there's a transition, and thus what gets added to the transition list
> you're building, and the list size comes out different, and you claim
> the data are the same, then guess where a process of elimination
> suggests the difference is coming from?
>
> ===
>
> Stylistic comment, I know this wasn't your question.
>
> > for row in range (len(TrigWind)):
>
> Don't do this.  It's not a coding error giving you wrong results, but
> it's not efficient and makes for harder to read code.  You already have
> an iterable in TrigWind.  You then find the size of the iterable and use
> that size to generate a range object, which you then iterate over,
> producing index values which you use to index into the original
> iterable.  Why not skip all that?  Just do
>
> for row in TrigWind:
>
> now row is actually a row, as the variable name suggests, rather than an
> index you use to go retrieve the row.
>
> Further, the "row" entries in TrigWind are lists (or tuples, or some
> other indexable iterable, we can't tell), which means you end up
> indexing into two things - into the "array" to get the row, then into
> the row to get the individual values. It's nicer if you unpack the rows
> into variables so they can have meaningful names - indeed you already do
> that with one of them. Lets you avoid code snips like  "x[7][1]"
>
> Conceptually then, you can take this:
>
> for row in range(len(Trigwind)):
> voltage = float(TrigWind[row][1])
> ...
> edgearray.append([float(

Re: [Tutor] Reading .csv data vs. reading an array

[Chip Wachob]

Mats,

Thank you!

So I included the QUOTE_NONNUMERIC to my csv.reader() call and it almost
worked.

Now, how wonderful that the scope's csv file simply wrote an s for seconds
and didn't include quotes.  Now Python tells me it can't create a float of
s.  Of course I can't edit a 4G file in any editor that I have installed,
so I have to work with the fact that there is a bit of text in there that
isn't quoted.

Which leads me to another question related to working with these csv
files.

Is there a way for me to tell the reader to skip the first 'n' rows?  Or,
for that matter, skip rows in the middle of the file?

A this point, I think it may be less painful for me to just skip those few
lines that have text.  I don't believe there will be any loss of accuracy.

But, since row is not really an index, how does one conditionally skip a
given set of row entries?

I started following the link to iterables but quickly got lost in the
terminology.

Best,


On Mon, Jul 15, 2019 at 3:03 PM Mats Wichmann  wrote:

> On 7/15/19 12:35 PM, Chip Wachob wrote:
> > Oscar and Mats,
> >
> > Thank you for your comments and taking time to look at the snips.
> >
> > Yes, I think I had commented that the avg+trigger was = triggervolts in
> > my original post.
> >
> > I did find that there was an intermediary process which I had forgotten
> > to comment out that was adversely affecting the data in one instance and
> > not the other.  So it WAS a case of becoming code blind.  But I didn't
> > give y'all all of the code so you would not have known that.  My
> apologies.
> >
> > Mats, I'd like to get a better handle on your suggestions about
> > improving the code.  Turns out, I've got another couple of 4GByte files
> > to sift through, and they are less 'friendly' when it comes to
> > determining the start and stop points.  So, I have to basically redo
> > about half of my code and I'd like to improve on my Python coding skills.
> >
> > Unfortunately, I have gaps in my coding time, and I end up forgetting
> > the details of a particular language, especially a new language to me,
> > Python.
> >
> > I'll admit that my 'C' background keeps me thinking as these data sets
> > as arrays.. in fact they are lists, eg:
> >
> > [
> > [t0, v0],
> > [t1, v1],
> > [t2, v2],
> > .
> > .
> > .
> > [tn, vn]
> > ]
> >
> > Time and volts are floats and need to be converted from the csv file
> > entries.
> >
> > I'm not sure that follow the "unpack" assignment in your example of:
> >
> > for row in TrigWind:
> > time, voltage = row  # unpack
> >
> > I think I 'see' what is happening, but when I read up on unpacking, I
> > see that referring to using the * and ** when passing arguments to a
> > function...
>
> That's a different aspect of unpacking.  This one is sequnce unpacking,
> sometimes called tuple (or seqeucence) assignment.  In the official
> Python docs it is described in the latter part of this section:
>
> https://docs.python.org/3/tutorial/datastructures.html#tuples-and-sequences
>
>
> > I tried it anyhow, with this being an example of my source data:
> >
> > "Record Length",202,"Points",-0.005640001706,1.6363
> > "Sample Interval",5e-09,s,-0.005639996706,1.65291
> > "Trigger Point",1128000,"Samples",-0.005639991706,1.65291
> > "Trigger Time",0.341197,s,-0.005639986706,1.60309
> > ,,,-0.005639981706,1.60309
> > "Horizontal Offset",-0.00564,s,-0.005639976706,1.6363
> > ,,,-0.005639971706,1.65291
> > ,,,-0.005639966706,1.65291
> > ,,,-0.005639961706,1.6363
> > .
> > .
> > .
> >
> > Note that I want the items in the third and fourth column of the csv
> > file for my time and voltage.
> >
> > When I tried to use the unpack, they all came over as strings.  I can't
> > seem to convert them selectively..
>
> That's what the csv module does, unless you tell it not to. Maybe this
> will help:
>
> https://docs.python.org/3/library/csv.html#csv.reader
>
> There's an option to convert unquoted values to floats, and leave quoted
> values alone as strings, which would seem to match your data above quite
> well.
>
> > Desc1, Val1, Desc2, TimeVal, VoltVal = row
> >
> > TimeVal and VoltVal return type of str, which makes sense.
> >
> > Must I go through yet another iteration of scanning TimeVal and VoltVal
> > and converting them using float() by savi