[Tutor] Help with building bytearray arrays

2018-09-08 Thread Chip Wachob
Admin, please remove my earlier messages.

This message is a properly 'self contained' message.

Hello,

I've been struggling with this for the last day or so and I can't seem
to figure out how to make it work.

I'll start out by saying that if there's a better approach, then I'm all ears.

I'm using the Adafruit Breakout board for the FTDI FT232H part.  Along
with this comes the Python libraries and Adafruit libraries which I'm
using.  I don't think that the libraries are the real issue here, but
I thought I should mention it just for the sake of information.

Basically I'm trying to write a block of unsigned bytes to the device
and read back an equal sized block of unsigned bytes.  There's a
function that is provided called transfer(data_to_send, num_of_bytes)
that handles the heavy lifting.  Unfortunately there seems to be a bug
in the part and if I attempt to send the entire block of bytes (64),
the device will lock up.  I've been able to determine that if I send
16 bytes at a time, I'm okay.

So, I take my bytearray(64) and step through it 16 bytes at a time like this:

my function's main pieces are:

def transfer_byte_array():
   MAX_LOOP_COUNT = 64
   slice_size = 16
   read_ary = bytearray(MAX_LOOP_COUNT)
   scratch_ary = bytearray()

   for step in range (0, MAX_LOOP_COUNT, slice_size):
  scratch_ary = transfer(data_to_send, slice_size)

  for bytes in range (0, slice_size):
 read_ary = scratch_ary[bytes]

   return(read_ary)


Ideally, I'd like to take the slice_size chunks that have been read
and concatenate them back togetjer into a long MAX_LOOP_COUNT size
array to pass back to the rest of my code.  Eg:

read_ary = ary_slice[0] + ary_slice[1] + ary_slice[2] + ary_slice[3]

I know that the + version doesn't work (or didn't for me) but it is
just my attempt at illustrating the overall goal.

The problem that I repeatedly run into is with the line:

read_ary = scratch_ary[bytes]  (or variants thereof)

The traceback is this:

Traceback (most recent call last):
  File "SW8T_5.py", line 101, in 
loop_size = RSI_size_the_loop(Print)
  File "/home/temp/Python_Scratch/examples/RSI.py", line 350, in
RSI_size_the_loop
read_ary.append(scratch_ary[singles])
TypeError: an integer or string of size 1 is required

or, one of the other common ones that I've seen is

TypeError: can't concat bytearray to list

This one is confusing because both of the operands are bytearry
types.. or at least I thought they should be...

when I try to replace the same line with :

read_ary += scratch_ary[bytes]

or

read_ary.append(scratch[bytes])

or

read_ary = read_ary + scratch_ary[bytes]


I'm obviously missing something fundamental here.  Problem is I can't
seem to find any examples of people asking this question before on the
inter-webs..

the transfer function expects an input of a bytearray and returns the same:

def transfer(self, data):
"""Full-duplex SPI read and write.  The specified array of bytes will be
clocked out the MOSI line, while simultaneously bytes will be read from
the MISO line.  Read bytes will be returned as a bytearray object.
"""
# Build command to read and write SPI data.
command = 0x30 | (self.lsbfirst << 3) | (self.read_clock_ve <<
2) | self.write_clock_ve
logger.debug('SPI transfer with command {0:2X}.'.format(command))
# Compute length low and high bytes.
# NOTE: Must actually send length minus one because the MPSSE engine
# considers 0 a length of 1 and  a length of 65536
length = len(data)
len_low  = (length-1) & 0xFF
len_high = ((length-1) >> 8) & 0xFF
# Send command and length.
self._assert_cs()
self._ft232h._write(str(bytearray((command, len_low, len_high
self._ft232h._write(str(bytearray(data)))
self._ft232h._write('\x87')
self._deassert_cs()
# Read response bytes.
return bytearray(self._ft232h._poll_read(length))


Thank you in advance to taking time to read.
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor


Re: [Tutor] Help with building bytearray arrays

2018-09-08 Thread Chip Wachob
Point taken on 'bytes'..  thanks.

the

scratch_ary = bytearray()

was my attempt at 'setting' the type of the variable.  I had hoped
that it would help resolve the error messages telling me that the
types didn't go together.

Coming from a 'C' background, I find the lack of typing in Python to
be confusing.  I'm used to working with bytes / words signed and
unsigned for a reason.

So, if I'm understanding the transfer() function correctly, the
function takes and returns a bytearray type.  You mentioned about
constructing a bytearray if I need one.  Can you expand on how I
approach that?

I'm going to try the experiment you mentioned in hopes of it giving me
a better understanding of the 'types' and what happens with the
variables.

Thank you,


On Fri, Sep 7, 2018 at 6:23 PM, Cameron Simpson  wrote:
> On 07Sep2018 15:45, Chip Wachob  wrote:
>>
>> Basically I'm trying to write a block of unsigned bytes to the device
>> and read back an equal sized block of unsigned bytes.  There's a
>> function that is provided called transfer(data_to_send, num_of_bytes)
>> that handles the heavy lifting.  Unfortunately there seems to be a bug
>> in the part and if I attempt to send the entire block of bytes (64),
>> the device will lock up.  I've been able to determine that if I send
>> 16 bytes at a time, I'm okay.
>>
>> So, I take my bytearray(64) and step through it 16 bytes at a time like
>> this:
>>
>> my function's main pieces are:
>>
>> def transfer_byte_array():
>>   MAX_LOOP_COUNT = 64
>>   slice_size = 16
>>   read_ary = bytearray(MAX_LOOP_COUNT)
>>   scratch_ary = bytearray()
>>
>>   for step in range (0, MAX_LOOP_COUNT, slice_size):
>>  scratch_ary = transfer(data_to_send, slice_size)
>>
>>  for bytes in range (0, slice_size):
>> read_ary = scratch_ary[bytes]
>>
>>   return(read_ary)
>>
>>
>> Ideally, I'd like to take the slice_size chunks that have been read
>> and concatenate them back togetjer into a long MAX_LOOP_COUNT size
>> array to pass back to the rest of my code.  Eg:
>>
>> read_ary = ary_slice[0] + ary_slice[1] + ary_slice[2] + ary_slice[3]
>
>
> Minor remark: don't use the name "bytes" for a variable, it is a builtin
> type name and you're shadowing it.
>
> It looks to me like "transfer" hands you back a buffer with the read data,
> so this:
>
>  scratch_ary = bytearray()
>
> don't do anything (it gets discarded).
>
> If you're getting back a bytes or bytearray object from transfer, just
> gather them all up in an list:
>
>  returned_buffers = []
>  for ..
>  response = transfer(data_to_send, slice_size)
>  returned_buffers.append(response)
>  ...
>  read_ary = b''.join(returned_buffers)
>
> Note that that makes a new bytes object for read_ary to refer to. You don't
> need the earlier initialisation of read_ary.
>
> Also note that the bytes object is read only; if that is a problem you'll
> need to construct a bytearray instead.
>
> [...]
>>
>> The problem that I repeatedly run into is with the line:
>>
>> read_ary = scratch_ary[bytes]  (or variants thereof)
>>
>> The traceback is this:
>>
>> Traceback (most recent call last):
>>  File "SW8T_5.py", line 101, in 
>>loop_size = RSI_size_the_loop(Print)
>>  File "/home/temp/Python_Scratch/examples/RSI.py", line 350, in
>> RSI_size_the_loop
>>read_ary.append(scratch_ary[singles])
>> TypeError: an integer or string of size 1 is required
>
>
> Yeah I thought that looked weird to me too.
>>
>> or, one of the other common ones that I've seen is
>>
>> TypeError: can't concat bytearray to list
>>
>> This one is confusing because both of the operands are bytearry
>> types.. or at least I thought they should be...
>
>
> No, one will be a list :-) putting a bunch of:
>
>  print(repr(foo))
>
> replacing "foo" with relevant variables will be illuminating to you; you can
> see immediately where this are not what you expected.
>
>> I'm obviously missing something fundamental here.  Problem is I can't
>> seem to find any examples of people asking this question before on the
>> inter-webs..
>
>
> You have the opposite of my problem. I can often find people asking the same
> question, but less often an answer. Or a decent answer, anyway.
>
> Cheers,
> Cameron Simpson 
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor


Re: [Tutor] Additionally

2018-09-08 Thread Steven D'Aprano
On Fri, Sep 07, 2018 at 10:16:38PM -0400, Chip Wachob wrote:
> Thank you.
> 
> I sent a message with the contents of both previous messages.
> 
> Hopefully the admin can just delete the two previous messages
> 
> Thanks for bearing with me.  I'm used to posting on forum boards, not via 
> email.

No problem. 

And if I come across as a bit short tempered, its because I'm suffering 
a lot of pain and discomfort at the moment due to illness. I'm trying 
not to let it show, but sometimes it might.


-- 
Steve

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor


Re: [Tutor] Help with building bytearray arrays

2018-09-08 Thread Cameron Simpson
Please try to adopt the inline reply style; we prefer it here. It lets us reply 
point by point and makes messages read like conversations. Anyway...


On 07Sep2018 23:57, Chip Wachob  wrote:

Point taken on 'bytes'..  thanks.

the
scratch_ary = bytearray()

was my attempt at 'setting' the type of the variable.  I had hoped
that it would help resolve the error messages telling me that the
types didn't go together.
Coming from a 'C' background, I find the lack of typing in Python to
be confusing.  I'm used to working with bytes / words signed and
unsigned for a reason.


Ok. Variables aren't typed. Variables are references to objects, which _are_ 
typed. So pointing a variable at a bytearray doesn't set its type in any 
persistent sense.


Thus:

 x = 1
 x = "a string"
 x = bytearray()

All just point "x" at different objects. If you'd like a C metaphor, variables 
are _like_ (void*) pointers: they can reference any kind of object.


Aside: they're not pointers, avoid the term - people will complain. As far as 
the language spec goes they're references. Which may in a particular 
implementation be _implemented internally_ using pointers.



So, if I'm understanding the transfer() function correctly, the
function takes and returns a bytearray type.


It would be good to see the specification for the transfer function. They we 
can adhere to its requirements. Can you supply a URL?



You mentioned about
constructing a bytearray if I need one.  Can you expand on how I
approach that?


Well, you were getting several bytes or bytearray objects back from transfer 
and wanted to join them together. The efficient way is to accumulate them in a 
list and then join them all together later using the bytearry (or bytes) .join 
method:


 https://docs.python.org/3/library/stdtypes.html#bytearray.join

So:

 data_list = []
 for 
   # send some data, get some data back
   data = transfer(...)
   # add that buffer to the data_list
   data_list.append(data)
 # make a single bytes object being the concatenation of all the smaller data 
 # chunks

 all_data = bytes.join(data_list)

It looks to me like your transfer() function is handed a bytes or bytearray and 
returns one. Normally that would be a separate bytes or bytearray.


Aside: bytes and bytearray objects are generally the way Python 3 deals with 
chunks of bytes. A "bytes" is readonly and a bytearray may have its contents 
modified. From a C background, they're like an array of unsigned chars.



I'm going to try the experiment you mentioned in hopes of it giving me
a better understanding of the 'types' and what happens with the
variables.


Sounds good.

Cheers,
Cameron Simpson 
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor


Re: [Tutor] Help with building bytearray arrays

2018-09-08 Thread Alan Gauld via Tutor
On 08/09/18 04:57, Chip Wachob wrote:

> was my attempt at 'setting' the type of the variable.  

A variable in Python is just a name. It has no type.
Only values have types. So you can set (or change)
the type of a value but not of a variable.

> Coming from a 'C' background, I find the lack of typing in Python to
> be confusing.  

There is no shortage of typing in Python its just
applied to the values(objects) not to their labels.

> I'm used to working with bytes / words signed and
> unsigned for a reason.

The same applies in Python. You just have to separate
the concepts of variable name and variable value.
In C those two concepts get kind of squished together
but in Python the name/value duality is strong.

You can always check the type of an object by using
the type() function or if you think a type error is
likely catch it when it happens with:

try:...
except TypeError:


-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor


Re: [Tutor] Help with building bytearray arrays

2018-09-08 Thread Alan Gauld via Tutor
On 08/09/18 03:15, Chip Wachob wrote:
> Admin, please remove my earlier messages.

No can do, once the emails are sent by the server
they are out there on the net, stored in people's
mailboxes and in various internet archives.

When using a mailing list always check before
sending, there's no going back...

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor


Re: [Tutor] Help with building bytearray arrays

2018-09-08 Thread Alan Gauld via Tutor
On 08/09/18 03:15, Chip Wachob wrote:

> my function's main pieces are:

It would probably be better to post the entire function,
a partial code sample like this just gives us an inkling
of what you are trying to do but not enough to be sure
of where the errors lie.

> def transfer_byte_array():
>MAX_LOOP_COUNT = 64
>slice_size = 16
>read_ary = bytearray(MAX_LOOP_COUNT)
>scratch_ary = bytearray()
> 
>for step in range (0, MAX_LOOP_COUNT, slice_size):
>   scratch_ary = transfer(data_to_send, slice_size)

You pass in two arguments here but in your description
of transfer below it is a method of an object that
only takes one (self being the object). Something is
broken here.

I would expect, given the code below, to see something
like
scratch_ary = someObject.transfer(data_to_send)

>   for bytes in range (0, slice_size):
>  read_ary = scratch_ary[bytes]

This is replacing rad_ary each time with a single value from scratch_ary.
At the end of theloop read_ary wil;l cpontain the last valuye in
scratch_ary. You can achieve the same result without a loop:

raed_ary = scratch_ary[-1]

But I suspect that's not what you want.
I think what you really want is the first
slice_size elements of scratch_ary:

read_ary = scratch_ary[:slice_size]   # use slicing

> Ideally, I'd like to take the slice_size chunks that have been read
> and concatenate them back togetjer into a long MAX_LOOP_COUNT size
> array to pass back to the rest of my code.  Eg:

You need to create a list of read_ary

results = []

then after creating each read_ary value append it
to results.

results.append(read_ary)

Then, at the very end, return the summation of all
the lists in results.

return sum(results,[])

> def transfer(self, data):

The self parameter indicates that this is a method
definition from a class. That implies that to use
it you must create an instance of the class and
call the method on that instance.

HTH

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor


Re: [Tutor] Help with building bytearray arrays

2018-09-08 Thread Cameron Simpson

On 08Sep2018 20:01, Cameron Simpson  wrote:

So, if I'm understanding the transfer() function correctly, the
function takes and returns a bytearray type.


It would be good to see the specification for the transfer function. 
They we can adhere to its requirements. Can you supply a URL?


I see you supplied the whole transfer function in your first message :-(

I'll recite it here and walk through its contents to explain:

   def transfer(self, data):

Ok, this is a method, a particular type of function associated with a class 
instance. (All objects are class instances, and the class is their type.)


So to call this you would normally have a control object of some kind. [...] 
Ah, it looks like you should have an SpiDev instance, inferring from this code:


 
https://github.com/adafruit/Adafruit_Python_GPIO/blob/master/Adafruit_GPIO/SPI.py

So suppose you've got such an object and a variable referring to it, let's say 
it is named "spi". You'd normally call the transfer function like this:


 spi.transfer(some_data)

instead of calling transfer() "bare", as it were.

When you call a Python method, the instance itself is implicitly passed as the 
parameter "self" (well, the first parameter - in Python we always call this 
"self" like C++ uses "this").


   """Full-duplex SPI read and write.  The specified array of bytes will be
   clocked out the MOSI line, while simultaneously bytes will be read from
   the MISO line.  Read bytes will be returned as a bytearray object.
   """

This is a docstring. It is like a comment but it gets attached to the function 
and can be inspected. Not your problem.


   # Build command to read and write SPI data.
   command = 0x30 | (self.lsbfirst << 3) | (self.read_clock_ve << 2) | 
self.write_clock_ve

This constructs a value just as you would in C.

   logger.debug('SPI transfer with command {0:2X}.'.format(command))

Write a debugging message.

   # Compute length low and high bytes.
   # NOTE: Must actually send length minus one because the MPSSE engine
   # considers 0 a length of 1 and  a length of 65536
   length = len(data)
   len_low  = (length-1) & 0xFF
   len_high = ((length-1) >> 8) & 0xFF

All just like C.

   # Send command and length.
   self._assert_cs()

I would guess that this raises a control signal. Ah, RS-232? So clear-to-send 
then.


   self._ft232h._write(str(bytearray((command, len_low, len_high

Ok, it looks like this is Python 2, not Python 3. Let's unpack it.

"(command, len_low, len_high)" is a tuple of 3 values. A tuple is like a read 
only list. We're passing that to the bytearray() constructor, which will accept 
an iterable of values and make a bytes buffer. And we want to write that to 
"self._ft232h". Which expects a str, which is why I think this is Python 2. In 
Python 2 there's no "bytes" type and str is an immutable array of 8-bit 
character values. So writing bytes uses str.


So this tabkes a tuple of values, to be bytes, makes those into a bytearray and 
makes that into a str, and writes that str to the serial line.


   self._ft232h._write(str(bytearray(data)))

We write that data itself.

   self._ft232h._write('\x87')
   self._deassert_cs()

And here we lower the control signal.

   # Read response bytes.
   return bytearray(self._ft232h._poll_read(length))

This calls the _poll_read method, presumably requesting "length" bytes, the 
same length as the supplied data. I presume we get a str back: we make a new 
bytearray with those bytes in it and return it.


So you do get a new bytearray back from each call, so we can accumulate them 
into a list and then join them together later to make a single bytearray.


Why this faffing about with str and bytearray? Probably for Python 2/3 
compatibility, and because you want to deal with bytes (small ints) instead of 
characters. Ignore it: we're dealing with bytes.


Cheers,
Cameron Simpson 
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor


Re: [Tutor] No module named uno in virtual environment

2018-09-08 Thread Mats Wichmann
On 09/07/2018 03:10 PM, Jim wrote:
> Mint 18.1
> System python3 3.5.2
> 
> Python3-uno is available to the system python3. How can I make it
> available to python 3.6.5 in a virtual environment I installed using venv?

with your virtualenv activated, just install it.

python -m pip install uno

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor


Re: [Tutor] No module named uno in virtual environment

2018-09-08 Thread Jim

On 09/08/2018 05:26 PM, Mats Wichmann wrote:

On 09/07/2018 03:10 PM, Jim wrote:

Mint 18.1
System python3 3.5.2

Python3-uno is available to the system python3. How can I make it
available to python 3.6.5 in a virtual environment I installed using venv?


with your virtualenv activated, just install it.

python -m pip install uno



My bad, I did not explain what python3-uno does. It is a system package 
that lets python interact with libreoffice. The uno that pip installs is 
a html generator.


I used apt to install python3-uno on my system. System python3 can use 
it and the python 3.5 in one of my virtual environments can use it. 
Python 3.6 in my other virtual environment cannot access it.


So to re-phrase my question. Python3-uno is installed on my system what 
do I need to do to get python 3.6 in a virtual environment to use it?


Maybe I am missing something obvious. With all the advice given to use 
virtual environments I would think that this problem has come up, but 
googling did not produce a solution.


Regards,  Jim

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor


Re: [Tutor] Help with building bytearray arrays

2018-09-08 Thread Cameron Simpson

On 08Sep2018 11:40, Alan Gauld  wrote:

On 08/09/18 03:15, Chip Wachob wrote:

Ideally, I'd like to take the slice_size chunks that have been read
and concatenate them back togetjer into a long MAX_LOOP_COUNT size
array to pass back to the rest of my code.  Eg:


You need to create a list of read_ary

results = []

then after creating each read_ary value append it
to results.

results.append(read_ary)

Then, at the very end, return the summation of all
the lists in results.

return sum(results,[])


Actually he's getting back bytearray instances from transfer and wants to join 
them up (his function does a few small transfers to work around an issue with 
one big transfer). His earlier code is just confused. So he wants:


 bytearray().join(results)

Hacked example:

 >>> bytearray().join( (bytearray("foo"),bytearray("bah")) )
 bytearray(b'foobah')

And he's working with bytearrays because the target library is Python 2, where 
there's no bytes type.


Cheers,
Cameron Simpson 
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor


Re: [Tutor] No module named uno in virtual environment

2018-09-08 Thread Mats Wichmann
On September 8, 2018 5:46:46 PM MDT, Jim  wrote:
>On 09/08/2018 05:26 PM, Mats Wichmann wrote:
>> On 09/07/2018 03:10 PM, Jim wrote:
>>> Mint 18.1
>>> System python3 3.5.2
>>>
>>> Python3-uno is available to the system python3. How can I make it
>>> available to python 3.6.5 in a virtual environment I installed using
>venv?
>> 
>> with your virtualenv activated, just install it.
>> 
>> python -m pip install uno
>> 
>
>My bad, I did not explain what python3-uno does. It is a system package
>
>that lets python interact with libreoffice. The uno that pip installs
>is 
>a html generator.
>
>I used apt to install python3-uno on my system. System python3 can use 
>it and the python 3.5 in one of my virtual environments can use it. 
>Python 3.6 in my other virtual environment cannot access it.
>
>So to re-phrase my question. Python3-uno is installed on my system what
>
>do I need to do to get python 3.6 in a virtual environment to use it?
>
>Maybe I am missing something obvious. With all the advice given to use 
>virtual environments I would think that this problem has come up, but 
>googling did not produce a solution.
>
>Regards,  Jim
>
>___
>Tutor maillist  -  Tutor@python.org
>To unsubscribe or change subscription options:
>https://mail.python.org/mailman/listinfo/tutor

ok, awkward naming issue. try openoffice-python instead?
-- 
Sent from a mobile device with K-9 Mail. Please excuse my brevity.
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor