Re: Embedding Python in C

[Barry Scott]

> On 17 Jul 2019, at 19:39, Jesse Ibarra  wrote:
> 
> On Wednesday, July 17, 2019 at 11:55:28 AM UTC-6, Barry Scott wrote:
>>> On 17 Jul 2019, at 16:57,  wrote:
>>> 
>>> I am using Python3.6:
>>> 
>>> [jibarra@redsky ~]$ python3.6
>>> Python 3.6.8 (default, Apr 25 2019, 21:02:35) 
>>> [GCC 4.8.5 20150623 (Red Hat 4.8.5-36)] on linux
>>> Type "help", "copyright", "credits" or "license" for more information.
>>> 
>>> 
>>> I am 
>>> referencing:https://docs.python.org/3.6/extending/embedding.html#beyond-very-high-level-embedding-an-overview
>>> 
>>> Is there a way to call a shared C lib using PyObjects?
>> 
>> If what you want to call is simple enough then you can use the ctypes library
>> that ships with python.
>> 
>> If the code you want to call is more complex you will want to use one of a 
>> number of libraries to help
>> you create a module that you can import.
>> 
>> I use PyCXX for this purpose that allows me to write C++ code that can call 
>> C++ and C libs and interface
>> easily with python. Home page http://cxx.sourceforge.net/ 
>>  the source kit contains demo code that you 
>> shows
>> how to cerate a module, a class and function etc. 
>> 
>> Example code: 
>> https://sourceforge.net/p/cxx/code/HEAD/tree/trunk/CXX/Demo/Python3/simple.cxx
>>  
>> 
>> 
>> Barry
>> PyCXX maintainer
>> 
>>> 
>>> Please advise.
>>> 
>>> Thank you.
>>> -- 
>>> https://mail.python.org/mailman/listinfo/python-list
>>> 
> 
> My options seem rather limited, I need to make a Pipeline from (Smalltalk -> 
> C -> Python) then go back (Smalltalk <- C <- Python). Since Smalltalk does 
> not support Python directly I have to settle with the C/Python API 
> (https://docs.python.org/3.6/extending/embedding.html#beyond-very-high-level-embedding-an-overview
>  
> ).
>  Any suggestions?

1. Run a new python process for each "call" you need to make.
2. Start a subprocess running python that talks via pipes to smalltalk and send 
messages back and forth to get the work done.
3. Write a C (I'd use C++ myself and PyCXX to avoid the complexity of the 
Python C API) extension to Smalltalk that initialises
a python interpreter and bridge calls from Smalltalk into Python on that 
intepreter.

Depending on the performance you need and the amount of data involved will help 
decide what is a reasonable design to choose.

Barry



> -- 
> https://mail.python.org/mailman/listinfo/python-list 
> 
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: join and split with empty delimiter

[Ben Bacarisse]

Irv Kalb  writes:

> I have always thought that split and join are opposite functions.  For
> example, you can use a comma as a delimiter:
>
 myList = ['a', 'b', 'c', 'd', 'e']
 myString = ','.join(myList)
 print(myString)
> a,b,c,d,e
>
 myList = myString.split(',')
 print(myList)
> ['a', 'b', 'c', 'd', 'e']
>
> Works great.

Note that join and split do not always recover the same list:

>>> ','.join(['a', 'b,c', 'd']).split(',')
['a', 'b', 'c', 'd']

You don't even have to have the delimiter in one of the strings:

>>> '//'.join(['a', 'b/', 'c']).split('//')
['a', 'b', '/c']

> But i've found a case where they don't work that way.  If
> I join the list with the empty string as the delimiter:
>
 myList = ['a', 'b', 'c', 'd']
 myString = ''.join(myList)
 print(myString)
> abcd
>
> That works great.  But attempting to split using the empty string
> generates an error:
>
 myString.split('')
> Traceback (most recent call last):
>   File "", line 1, in 
> myString.split('')
> ValueError: empty separator
>
> I know that this can be accomplished using the list function:
>
 myString = list(myString)
 print(myString)
> ['a', 'b', 'c', 'd']
>
> But my question is:  Is there any good reason why the split function
> should give an "empty separator" error?  I think the meaning of trying
> to split a string into a list using the empty string as a delimiter is
> unambiguous - it should just create a list of single characters
> strings like the list function does here.

One reason might be that str.split('') is not unambiguous.  For example,
there's a case to be made that there is a '' delimiter at the start and
the end of the string as well as between letters.  '' is a very special
delimiter because every string that gets joined using it includes it!
It's a wild version of ','.join(['a', 'b,c', 'd']).split(',').

Of course str.split('') could be defined to work the way you expect, but
it's possible that the error is there to prompt the programmer to be
more explicit.

-- 
Ben.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: join and split with empty delimiter

[Richard Damon]

On 7/18/19 6:27 AM, Ben Bacarisse wrote:
> One reason might be that str.split('') is not unambiguous.  For example,
> there's a case to be made that there is a '' delimiter at the start and
> the end of the string as well as between letters.  '' is a very special
> delimiter because every string that gets joined using it includes it!
> It's a wild version of ','.join(['a', 'b,c', 'd']).split(',').
>
> Of course str.split('') could be defined to work the way you expect, but
> it's possible that the error is there to prompt the programmer to be
> more explicit.

One thought I am having is that if the split is explicitly on '', then
it is easy enough to use list(), but if it isn't a hard coded '', then
the gotchas described above  are significant, that unless the join list
was built of exactly 1 character strings, the split won't match, and
that seems somewhat special case for a variable delimiter.

-- 
Richard Damon

-- 
https://mail.python.org/mailman/listinfo/python-list

Extending an Enum type

[Antoon Pardon]

Hi, I am experimenting with writing an Earley Parser. Now I would like
to have the non-terminals from the grammer I am reading in, be
represented bye an enum like type. So that if the grammer contains the
following production: Term -> Term '+' Factor I can reprensent the right
hand side with a list that gets printed something like the following:
[, '+', ] I am a bit at a
loss right now on how to start. Can someone point me in the right
direction? -- Antoon Pardon

-- 
https://mail.python.org/mailman/listinfo/python-list

Extendable Enum like Type?

[Antoon Pardon]

Something seems to have gone wrong with the formatting of my latest 
contribution,
so let me try again.

I am experimenting with writing an Earley Parser. Now I would like to
have the non-terminals from the grammer I am reading in, be represented
bye an enum like type. So that if the grammer contains the following
production: Term -> Term '+' Factor I can reprensent the right hand side
with a list that gets printed something like the following:
[, '+', ] I am a bit at a
loss right now on how to start. Can someone point me in the right
direction? -- Antoon Pardon

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Embedding Python in C

[Jesse Ibarra]

On Wednesday, July 17, 2019 at 2:20:51 PM UTC-6, Christian Gollwitzer wrote:
> Am 17.07.19 um 20:39 schrieb Jesse Ibarra:
> > My options seem rather limited, I need to make a Pipeline from (Smalltalk 
> > -> C -> Python) then go back (Smalltalk <- C <- Python). Since Smalltalk 
> > does not support Python directly I have to settle with the C/Python API 
> > (https://docs.python.org/3.6/extending/embedding.html#beyond-very-high-level-embedding-an-overview).
> >  Any suggestions?
> > 
> 
> Ah, now you finally tell us your problem!
> 
> Depending on, how complete / advanced / efficient the bridge needs to 
> be, it can be easy or hard.
> 
> What level of integration do you want to achieve? Do you want
> 
> a) to call Python functions from Smalltalk
> b) call Smalltalk functions from Python
> c) pass callbacks around, e.g. use a Smalltalk function within a Python 
> list comprehension, and if so, which way
> d) integrate the class systems - derive a Python class from a Smalltalk 
> base or the other way round
> 
> e) ?
> 
> 
> The most basic thing is a), but even getting that right might be 
> non-trivial, since both C APIs will have different type systems which 
> you need to match. I don't speak Smalltalk, so can't comment in detail 
> on this - but in practice it will also depend on the implementation you 
> are using.
> 
> Best regards,
> 
>   Christian

For right now, I need to call a .so file from Smalltalk. I can't explicitly use 
Python libraries since Smalltalk does not support Python. I need to use the 
C/Python API (in Smalltalk) to create a bridge where I can call a .so and a 
function in that .so file with a PyObject (This will be called back to 
Smalltalk from the .so file). I can't figure out a way to do that since the 
C/API can't call C or .so files.

sorry for the confusion on my part
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Extendable Enum like Type?

[Ethan Furman]

On 07/18/2019 06:04 AM, Antoon Pardon wrote:


I am experimenting with writing an Earley Parser. Now I would like to
have the non-terminals from the grammer I am reading in, be represented
bye an enum like type. So that if the grammer contains the following
production: Term -> Term '+' Factor I can reprensent the right hand side
with a list that gets printed something like the following:
[, '+', ] I am a bit at a
loss right now on how to start. Can someone point me in the right
direction?


The basic method is:

from enum import Enum # `from aenum` [1][2] if less than Python 3.4

Class NonTerminal(Enum):
Term = 1
Factor = 2
...

The docs [3] also have a lot of information.

Does that answer your question?

--
~Ethan~


[1] https://pypi.org/project/aenum/
[2] I am the author of Enum, aenum, and the enum34 backport.
[3] https://docs.python.org/3/library/enum.html
--
https://mail.python.org/mailman/listinfo/python-list

Re: join and split with empty delimiter

[Danilo Coccia]

Il 18/07/2019 12:27, Ben Bacarisse ha scritto:
> Irv Kalb  writes:
> 
>> I have always thought that split and join are opposite functions.  For
>> example, you can use a comma as a delimiter:
>>
> myList = ['a', 'b', 'c', 'd', 'e']
> myString = ','.join(myList)
> print(myString)
>> a,b,c,d,e
>>
> myList = myString.split(',')
> print(myList)
>> ['a', 'b', 'c', 'd', 'e']
>>
>> Works great.
> 
> Note that join and split do not always recover the same list:
> 
 ','.join(['a', 'b,c', 'd']).split(',')
> ['a', 'b', 'c', 'd']
> 
> You don't even have to have the delimiter in one of the strings:
> 
 '//'.join(['a', 'b/', 'c']).split('//')
> ['a', 'b', '/c']
> 
>> But i've found a case where they don't work that way.  If
>> I join the list with the empty string as the delimiter:
>>
> myList = ['a', 'b', 'c', 'd']
> myString = ''.join(myList)
> print(myString)
>> abcd
>>
>> That works great.  But attempting to split using the empty string
>> generates an error:
>>
> myString.split('')
>> Traceback (most recent call last):
>>   File "", line 1, in 
>> myString.split('')
>> ValueError: empty separator
>>
>> I know that this can be accomplished using the list function:
>>
> myString = list(myString)
> print(myString)
>> ['a', 'b', 'c', 'd']
>>
>> But my question is:  Is there any good reason why the split function
>> should give an "empty separator" error?  I think the meaning of trying
>> to split a string into a list using the empty string as a delimiter is
>> unambiguous - it should just create a list of single characters
>> strings like the list function does here.
> 
> One reason might be that str.split('') is not unambiguous.  For example,
> there's a case to be made that there is a '' delimiter at the start and
> the end of the string as well as between letters.  '' is a very special
> delimiter because every string that gets joined using it includes it!
> It's a wild version of ','.join(['a', 'b,c', 'd']).split(',').
> 
> Of course str.split('') could be defined to work the way you expect, but
> it's possible that the error is there to prompt the programmer to be
> more explicit.
> 

It is even more ambiguous if you consider that any string starts with an
infinite number of empty strings, followed by a character, followed by
an infinite number of empty strings, followed by ...
The result wouldn't fit on screen, or in memory for that!

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Embedding Python in C

[Christian Gollwitzer]

Am 18.07.19 um 16:18 schrieb Jesse Ibarra:

On Wednesday, July 17, 2019 at 2:20:51 PM UTC-6, Christian Gollwitzer wrote:

What level of integration do you want to achieve? Do you want

a) to call Python functions from Smalltalk
b) call Smalltalk functions from Python
c) pass callbacks around, e.g. use a Smalltalk function within a Python
list comprehension, and if so, which way
d) integrate the class systems - derive a Python class from a Smalltalk
base or the other way round





For right now, I need to call a .so file from Smalltalk. I can't explicitly use 
Python libraries since Smalltalk does not support Python. I need to use the 
C/Python API (in Smalltalk) to create a bridge where I can call a .so and a 
function in that .so file with a PyObject (This will be called back to 
Smalltalk from the .so file). I can't figure out a way to do that since the 
C/API can't call C or .so files.

sorry for the confusion on my part



I still don't get it, sorry. To me it is unclear which part of the 
integration you manage to do so far, and which part is the problem.


Which Smalltalk interpreter are you using? The answer to the following 
will heavily depend on that.



Suppose I'd give you a C file with the following simple function:


double add(double a, double b) {
return a+b;
}

Do you know how to compile this code and make it usable from Smalltalk?

One level up, consider a C function working on an array:

double arrsum(int n, double *arr) {
double sum=0.0;
for (int i=0; iHow would you compile and link this with your Smalltalk implementation, 
such that you can pass it a Smalltalk array?


Once you can do this, you can proceed to call a Python function, which 
in C means that you invoke the function PyObject_CallObject(). A basic 
example is shown here:


https://docs.python.org/2/extending/embedding.html#pure-embedding

Christian




--
https://mail.python.org/mailman/listinfo/python-list

Re: join and split with empty delimiter

[Ben Bacarisse]

Danilo Coccia  writes:

> Il 18/07/2019 12:27, Ben Bacarisse ha scritto:
>> Irv Kalb  writes:
>> 
>>> I have always thought that split and join are opposite functions.  For
>>> example, you can use a comma as a delimiter:
>>>
>> myList = ['a', 'b', 'c', 'd', 'e']
>> myString = ','.join(myList)
>> print(myString)
>>> a,b,c,d,e
>>>
>> myList = myString.split(',')
>> print(myList)
>>> ['a', 'b', 'c', 'd', 'e']
>>>
>>> Works great.
>> 
>> Note that join and split do not always recover the same list:
>> 
> ','.join(['a', 'b,c', 'd']).split(',')
>> ['a', 'b', 'c', 'd']
>> 
>> You don't even have to have the delimiter in one of the strings:
>> 
> '//'.join(['a', 'b/', 'c']).split('//')
>> ['a', 'b', '/c']
>> 
>>> But i've found a case where they don't work that way.  If
>>> I join the list with the empty string as the delimiter:
>>>
>> myList = ['a', 'b', 'c', 'd']
>> myString = ''.join(myList)
>> print(myString)
>>> abcd
>>>
>>> That works great.  But attempting to split using the empty string
>>> generates an error:
>>>
>> myString.split('')
>>> Traceback (most recent call last):
>>>   File "", line 1, in 
>>> myString.split('')
>>> ValueError: empty separator
>>>
>>> I know that this can be accomplished using the list function:
>>>
>> myString = list(myString)
>> print(myString)
>>> ['a', 'b', 'c', 'd']
>>>
>>> But my question is:  Is there any good reason why the split function
>>> should give an "empty separator" error?  I think the meaning of trying
>>> to split a string into a list using the empty string as a delimiter is
>>> unambiguous - it should just create a list of single characters
>>> strings like the list function does here.
>> 
>> One reason might be that str.split('') is not unambiguous.  For example,
>> there's a case to be made that there is a '' delimiter at the start and
>> the end of the string as well as between letters.  '' is a very special
>> delimiter because every string that gets joined using it includes it!
>> It's a wild version of ','.join(['a', 'b,c', 'd']).split(',').
>> 
>> Of course str.split('') could be defined to work the way you expect, but
>> it's possible that the error is there to prompt the programmer to be
>> more explicit.
>
> It is even more ambiguous if you consider that any string starts with an
> infinite number of empty strings, followed by a character, followed by
> an infinite number of empty strings, followed by ...
> The result wouldn't fit on screen, or in memory for that!

Right, but that can be finessed by saying that two delimiters can't
overlap, which is the usual rule.  A reasonable interpretation of "not
overlapping" might well exclude having more the one delimiter in the
same place.

-- 
Ben.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Embedding Python in C

[Chris Angelico]

On Fri, Jul 19, 2019 at 5:51 AM Christian Gollwitzer  wrote:
> Once you can do this, you can proceed to call a Python function, which
> in C means that you invoke the function PyObject_CallObject(). A basic
> example is shown here:
>
> https://docs.python.org/2/extending/embedding.html#pure-embedding
>

Or, better:

https://docs.python.org/3/extending/embedding.html#pure-embedding

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: join and split with empty delimiter

[MRAB]

On 2019-07-18 20:52, Ben Bacarisse wrote:

Danilo Coccia  writes:


Il 18/07/2019 12:27, Ben Bacarisse ha scritto:

[snip]

Of course str.split('') could be defined to work the way you expect, but
it's possible that the error is there to prompt the programmer to be
more explicit.


It is even more ambiguous if you consider that any string starts with an
infinite number of empty strings, followed by a character, followed by
an infinite number of empty strings, followed by ...
The result wouldn't fit on screen, or in memory for that!


Right, but that can be finessed by saying that two delimiters can't
overlap, which is the usual rule.  A reasonable interpretation of "not
overlapping" might well exclude having more the one delimiter in the
same place.

The delimiters wouldn't be overlapping, they'd be adjacent, but it does 
seem reasonable not to split on an empty delimiter more than once at a 
certain position. That's what a regex split (usually) does (in the re 
module since Python 3.7, and in other regex implementations).

--
https://mail.python.org/mailman/listinfo/python-list