Re: Importing module from another subdirectory
On Tue, 23 Apr 2019, dieter wrote: I use "virtualenv" (for "VIRTUAL ENVironmet") to separate projects. Dieter, I know about virtualenv and tried using them. Found conflicting information and didn't know if I really needed them. I'll re-learn how to activate and use them. One project is for my own use and I understand now that a virtualenv with its own sys.path appendices would work. The other project is intended to be available for installation and use by clients (most of whom work in a Windows environment) so I need to learn how development in a virtual environment affects ultimate distribution via github. If you are ready to use "virtual environment"s and ready to learn about advanved Python packaging facilities, then read the setuptools documentation (--> "https://setuptools.readthedocs.io/en/latest/";). This URL will likely teach me what I need to learn for both projects. Thanks very much. Best regards, Rich -- https://mail.python.org/mailman/listinfo/python-list
Re: Better ways for implementing two situations
Às 19:42 de 21/04/19, Stefan Ram escreveu: > Paulo da Silva writes: >> I have a list of objects and want to split it in a list of groups. >> "equal objects" is based on an id we can get from the object. > > main.py > > input = [ 'abc', 'ade', 'bcd' ] > > for group, list in \ > __import__( 'itertools' ).groupby( input, lambda x: x[ 0 ] ): > print( group, *list ) > Yes, seems to be the best way to go. I need to sort data first, however. Thanks -- https://mail.python.org/mailman/listinfo/python-list
Re: Better ways for implementing two situations
Às 20:10 de 21/04/19, MRAB escreveu: > On 2019-04-21 19:23, Paulo da Silva wrote: >> Hi all. >> ... > Have you compared the speed with an implementation that uses > defaultdict? Your code always creates an empty list for each item, even > though it might not be needed. I never used defaultdict. I'll take a look at it. Thanks -- https://mail.python.org/mailman/listinfo/python-list
Re: Better ways for implementing two situations
Às 20:41 de 21/04/19, DL Neil escreveu: > Olá Paulo, > ... > > Given that we're talking "big data", which Python Data Science tools are > you employing? eg NumPy. Sorry. I misused the term "big data". I should have said a big amount of data. It is all about objects built of text and some numbers. Some properties of some objects are got on demand and when needed by reading and processing data from the hard drive. Thanks anyway. -- https://mail.python.org/mailman/listinfo/python-list
Re: Better ways for implementing two situations
Às 22:21 de 21/04/19, Paul Rubin escreveu:
> Paulo da Silva writes:
>> splitter={}
>> for f in Objs:
>> splitter.setdefault(f.getId1,[]).append(f)
>> groups=[gs for gs in splitter.values() if len(gs)>1]
>
> It's easiest if you can sort the input list and then use
> itertools.groupby.
Yes, sort and groupby seems the best way to go.
Thanks.
--
https://mail.python.org/mailman/listinfo/python-list
How to catch a usefull error message ?
Hi,
In a CPython lib I have an _init() method wich take one argument, a file
name.
char *fname;
if (!PyArg_ParseTuple(args, "s", &fname))
return NULL;
So, if I instanciate my object with a bad argument I've a good error
message:
tif = ImgProc(123)
TypeError: argument 1 must be str, not int
(followed by the traceback)
But if I do:
try:
tif = ImgProc(123)
except Exception as why:
print("Error:", why)
I get just:
Error: returned a result with an error set
Without traceback. That's not very usefull.
I prefer to keep the instanciation of this object into a try-except bloc
but how to read the error message ?
Vincent
--
https://mail.python.org/mailman/listinfo/python-list
Re: How to catch a usefull error message ?
On Wed, Apr 24, 2019 at 3:18 AM Vincent Vande Vyvre
wrote:
>
> Hi,
>
> In a CPython lib I have an _init() method wich take one argument, a file
> name.
>
> char *fname;
>
> if (!PyArg_ParseTuple(args, "s", &fname))
> return NULL;
>
> So, if I instanciate my object with a bad argument I've a good error
> message:
>
> tif = ImgProc(123)
> TypeError: argument 1 must be str, not int
> (followed by the traceback)
>
> But if I do:
> try:
> tif = ImgProc(123)
> except Exception as why:
> print("Error:", why)
>
> I get just:
>
> Error: returned a result with an error set
>
It looks like there's an internal problem in the C function. Are you
sure it's hitting the PyArg_ParseTuple and then immediately returning
NULL? Post a bit more of your code; this error looks like something is
leaving an error state but then carrying on with the code.
ChrisA
--
https://mail.python.org/mailman/listinfo/python-list
Re: How to catch a usefull error message ?
On 2019-04-23 10:56, Vincent Vande Vyvre wrote:
Hi,
In a CPython lib I have an _init() method wich take one argument, a file
name.
char *fname;
if (!PyArg_ParseTuple(args, "s", &fname))
return NULL;
So, if I instanciate my object with a bad argument I've a good error
message:
tif = ImgProc(123)
TypeError: argument 1 must be str, not int
(followed by the traceback)
But if I do:
try:
tif = ImgProc(123)
except Exception as why:
print("Error:", why)
I get just:
Error: returned a result with an error set
Without traceback. That's not very usefull.
I prefer to keep the instanciation of this object into a try-except bloc
but how to read the error message ?
Have a look ta the 'traceback' module.
Example:
import traceback
try:
1/0
except Exception as ex:
print('Error:', ex)
traceback.print_exc()
--
https://mail.python.org/mailman/listinfo/python-list
Re: How to catch a usefull error message ?
Le 23/04/19 à 19:23, Chris Angelico a écrit :
On Wed, Apr 24, 2019 at 3:18 AM Vincent Vande Vyvre
wrote:
Hi,
In a CPython lib I have an _init() method wich take one argument, a file
name.
char *fname;
if (!PyArg_ParseTuple(args, "s", &fname))
return NULL;
So, if I instanciate my object with a bad argument I've a good error
message:
tif = ImgProc(123)
TypeError: argument 1 must be str, not int
(followed by the traceback)
But if I do:
try:
tif = ImgProc(123)
except Exception as why:
print("Error:", why)
I get just:
Error: returned a result with an error set
It looks like there's an internal problem in the C function. Are you
sure it's hitting the PyArg_ParseTuple and then immediately returning
NULL? Post a bit more of your code; this error looks like something is
leaving an error state but then carrying on with the code.
ChrisA
Into the lib:
static int
ImgProc_init(ImgProc *self, PyObject *args, PyObject *kwds)
{
PyObject *tmp;
char *fname;
if (!PyArg_ParseTuple(args, "s", &fname))
return NULL;
tmp = self->src;
self->src = PyUnicode_FromString(fname);
Py_XDECREF(tmp);
return 0;
}
If i do:
try:
tif = ImgProc(123)
except Exception as why:
print(sys.exc_info())
raise
I get:
(, SystemError("
returned a result with an error set",), 0x7f3bcac748c8>)
TypeError: argument 1 must be str, not int
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/vincent/oqapy-3/trunk/filters/ui_lenscorrection.py", line
104, in on_main_cursor_changed
self.prepare_preview_process()
File "/home/vincent/oqapy-3/trunk/filters/ui_lenscorrection.py", line
137, in prepare_preview_process
self.main.process_on_preview(params)
File "/home/vincent/oqapy-3/trunk/filters/lenscorrection.py", line
56, in process_on_preview
tif = ImgProc(123)
SystemError: returned a result with an error set
Why a SystemError ?
I'm using Python 3.6.7 and 3.7.3
Vincent
--
https://mail.python.org/mailman/listinfo/python-list
Re: How to catch a usefull error message ?
Le 23/04/19 à 19:27, MRAB a écrit :
On 2019-04-23 10:56, Vincent Vande Vyvre wrote:
Hi,
In a CPython lib I have an _init() method wich take one argument, a file
name.
char *fname;
if (!PyArg_ParseTuple(args, "s", &fname))
return NULL;
So, if I instanciate my object with a bad argument I've a good error
message:
tif = ImgProc(123)
TypeError: argument 1 must be str, not int
(followed by the traceback)
But if I do:
try:
tif = ImgProc(123)
except Exception as why:
print("Error:", why)
I get just:
Error: returned a result with an error set
Without traceback. That's not very usefull.
I prefer to keep the instanciation of this object into a try-except bloc
but how to read the error message ?
Have a look ta the 'traceback' module.
Example:
import traceback
try:
1/0
except Exception as ex:
print('Error:', ex)
traceback.print_exc()
try:
tif = ImgProc(123)
except Exception as why:
traceback.print_exc()
print("always alive !")
TypeError: argument 1 must be str, not int
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/vincent/oqapy-3/trunk/filters/lenscorrection.py", line
56, in process_on_preview
tif = ImgProc(123)
SystemError: returned a result with an error set
always alive !
-
This is better of nothing and the try-except works.
Vincent
--
https://mail.python.org/mailman/listinfo/python-list
Re: How to catch a usefull error message ?
On Wed, Apr 24, 2019 at 4:47 AM Vincent Vande Vyvre
wrote:
>
> Le 23/04/19 à 19:23, Chris Angelico a écrit :
> > On Wed, Apr 24, 2019 at 3:18 AM Vincent Vande Vyvre
> > wrote:
> >> Hi,
> >>
> >> In a CPython lib I have an _init() method wich take one argument, a file
> >> name.
> >>
> >> char *fname;
> >>
> >> if (!PyArg_ParseTuple(args, "s", &fname))
> >> return NULL;
> >>
> >> So, if I instanciate my object with a bad argument I've a good error
> >> message:
> >>
> >> tif = ImgProc(123)
> >> TypeError: argument 1 must be str, not int
> >> (followed by the traceback)
> >>
> >> But if I do:
> >> try:
> >> tif = ImgProc(123)
> >> except Exception as why:
> >> print("Error:", why)
> >>
> >> I get just:
> >>
> >> Error: returned a result with an error set
> >>
> > It looks like there's an internal problem in the C function. Are you
> > sure it's hitting the PyArg_ParseTuple and then immediately returning
> > NULL? Post a bit more of your code; this error looks like something is
> > leaving an error state but then carrying on with the code.
> >
> > ChrisA
>
> Into the lib:
>
> static int
> ImgProc_init(ImgProc *self, PyObject *args, PyObject *kwds)
> {
> PyObject *tmp;
> char *fname;
>
> if (!PyArg_ParseTuple(args, "s", &fname))
> return NULL;
>
> tmp = self->src;
> self->src = PyUnicode_FromString(fname);
> Py_XDECREF(tmp);
> return 0;
> }
>
> If i do:
> try:
> tif = ImgProc(123)
> except Exception as why:
> print(sys.exc_info())
> raise
> I get:
> (, SystemError("
> returned a result with an error set",), 0x7f3bcac748c8>)
> TypeError: argument 1 must be str, not int
>
> The above exception was the direct cause of the following exception:
>
> Traceback (most recent call last):
>File "/home/vincent/oqapy-3/trunk/filters/ui_lenscorrection.py", line
> 104, in on_main_cursor_changed
> self.prepare_preview_process()
>File "/home/vincent/oqapy-3/trunk/filters/ui_lenscorrection.py", line
> 137, in prepare_preview_process
> self.main.process_on_preview(params)
>File "/home/vincent/oqapy-3/trunk/filters/lenscorrection.py", line
> 56, in process_on_preview
> tif = ImgProc(123)
> SystemError: returned a result with an error set
>
> Why a SystemError ?
The SystemError means that you're using the Python C API in a way that
doesn't make sense to the interpreter. You're leaving a marker saying
"hey, I need you to throw an exception" but then you're also returning
a value. You'll need to figure out where that's happening and exactly
what is being called. How are you setting up your class?
ChrisA
--
https://mail.python.org/mailman/listinfo/python-list
Re: How to catch a usefull error message ?
On Wed, Apr 24, 2019 at 4:51 AM Vincent Vande Vyvre
wrote:
>
> Le 23/04/19 à 19:27, MRAB a écrit :
> > On 2019-04-23 10:56, Vincent Vande Vyvre wrote:
> >> Hi,
> >>
> >> In a CPython lib I have an _init() method wich take one argument, a file
> >> name.
> >>
> >> char *fname;
> >>
> >> if (!PyArg_ParseTuple(args, "s", &fname))
> >> return NULL;
> >>
> >> So, if I instanciate my object with a bad argument I've a good error
> >> message:
> >>
> >> tif = ImgProc(123)
> >> TypeError: argument 1 must be str, not int
> >> (followed by the traceback)
> >>
> >> But if I do:
> >> try:
> >> tif = ImgProc(123)
> >> except Exception as why:
> >> print("Error:", why)
> >>
> >> I get just:
> >>
> >> Error: returned a result with an error set
> >>
> >> Without traceback. That's not very usefull.
> >>
> >> I prefer to keep the instanciation of this object into a try-except bloc
> >> but how to read the error message ?
> >>
> > Have a look ta the 'traceback' module.
> > Example:
> >
> > import traceback
> >
> > try:
> > 1/0
> > except Exception as ex:
> > print('Error:', ex)
> > traceback.print_exc()
>
> try:
> tif = ImgProc(123)
> except Exception as why:
> traceback.print_exc()
> print("always alive !")
>
> TypeError: argument 1 must be str, not int
>
> The above exception was the direct cause of the following exception:
>
> Traceback (most recent call last):
>File "/home/vincent/oqapy-3/trunk/filters/lenscorrection.py", line
> 56, in process_on_preview
> tif = ImgProc(123)
> SystemError: returned a result with an error set
> always alive !
> -
>
> This is better of nothing and the try-except works.
>
The try/except is NOT your problem. It's just a symptom.
ChrisA
--
https://mail.python.org/mailman/listinfo/python-list
Re: How to catch a usefull error message ?
On 2019-04-23 19:21, Vincent Vande Vyvre wrote:
Le 23/04/19 à 19:23, Chris Angelico a écrit :
On Wed, Apr 24, 2019 at 3:18 AM Vincent Vande Vyvre
wrote:
Hi,
In a CPython lib I have an _init() method wich take one argument, a file
name.
char *fname;
if (!PyArg_ParseTuple(args, "s", &fname))
return NULL;
So, if I instanciate my object with a bad argument I've a good error
message:
tif = ImgProc(123)
TypeError: argument 1 must be str, not int
(followed by the traceback)
But if I do:
try:
tif = ImgProc(123)
except Exception as why:
print("Error:", why)
I get just:
Error: returned a result with an error set
It looks like there's an internal problem in the C function. Are you
sure it's hitting the PyArg_ParseTuple and then immediately returning
NULL? Post a bit more of your code; this error looks like something is
leaving an error state but then carrying on with the code.
ChrisA
Into the lib:
static int
ImgProc_init(ImgProc *self, PyObject *args, PyObject *kwds)
{
PyObject *tmp;
char *fname;
if (!PyArg_ParseTuple(args, "s", &fname))
return NULL;
tmp = self->src;
self->src = PyUnicode_FromString(fname);
Py_XDECREF(tmp);
return 0;
}
[snip]
That function returns an int.
If PyArg_ParseTuple fails, your function returns NULL, which is cast to
an int, 0.
If PyArg_ParseTuple succeeds, your function returns 0.
Either way, it returns 0.
So how does the caller know whether the function was successful? Does it
check PyErr_Occurred?
--
https://mail.python.org/mailman/listinfo/python-list
Re: need help understanding: converting text to binary
In comp.lang.python, Chris Angelico wrote: > Have you checked to see if Python can already do this? You mention I'm sure there's a library already. I'm trying to mix library usage with my own code to get practice writing in python. In this case, I want code to deal with MIME encoding in email headers. I turned to a library for the base64 part and translated C code I once wrote for the quoted-printable part. I was struck by how complicated it seems to be to generate binary blobs in python, which makes me think I'm not getting something. >> Is there a more python-esque way to convert what should be plain ascii > What does "plain ASCII" actually mean, though? ASCII encoded binary data. ASCII is code points that fit in 7-bits comprising the characters found on a typical 1970s US oriented typewriter plus a few control characters. >> into a binary "bytes" object? In the use case I'm working towards the >> charset will not be ascii or UTF-8 all of the time, and the charset >> isn't the responsibility of the python code. Think "decode this if >> charset matches user-specified value, then output in that same charset; >> otherwise do nothing." > I'm not sure what this means, If the terminal expects this encoding, then decode the ASCII transport encoding and show the raw stream. If the terminal doesn't expect this encoding, do not decode. Python should be treating it as a a binary stream, and doesn't need to understand the encoding itself. > but I would strongly recommend just > encoding and decoding regardless. Use text internally and bytes at the > outside. That feels entirely wrong. I don't know what b'\x9A' means without knowing the character set and character encoding. If the encoding is a multibyte one, b'\x9A' doesn't mean anything on its own. That's why I want to treat it as binary. Elijah -- thinking of an array of "unsigned char" not of characters -- https://mail.python.org/mailman/listinfo/python-list
Re: need help understanding: converting text to binary
In comp.lang.python, Paul Rubin wrote:
> Eli the Bearded <*@eli.users.panix.com> writes:
>> # decode a single hex digit
>> def hord(c): ...
>
>def hord(c): return int(c, 16)
That's a good method, thanks.
> > # decode quoted printable, specifically the MIME-encoded words
> > # variant which is slightly different than the body text variant
> > def decodeqp(v): ...
>
> I think this is supposed to mean:
>
> from itertools import islice
>
> def getbytes(v):
> cs = iter(bytes(v,'ascii'))
> for c in cs:
> if c == ord('='):
> h1,h2 = islice(cs,2)
> yield int(chr(h1)+chr(h2), 16)
> else: yield c
>
> def decodeqp(v):
> return bytes(getbytes(v))
>
> print (decodeqp('=21_yes')) # prints "b'!_yes'"
But that's not the output my sample produced.
def getbytes(v):
cs = iter(bytes(v,'ascii'))
for c in cs:
if c == ord('='):
h1,h2 = islice(cs,2)
yield int(chr(h1)+chr(h2), 16)
elif c == ord('_'):
yield ord(' ')
else: yield c
That's certainly a lot cleaner that what I had.
Elijah
--
and shorter than the one in /usr/lib/python3.5/quopri.py
--
https://mail.python.org/mailman/listinfo/python-list
Re: need help understanding: converting text to binary
On 23Apr2019 20:35, Eli the Bearded <*@eli.users.panix.com> wrote: In comp.lang.python, Chris Angelico wrote: Is there a more python-esque way to convert what should be plain ascii What does "plain ASCII" actually mean, though? ASCII encoded binary data. ASCII is code points that fit in 7-bits comprising the characters found on a typical 1970s US oriented typewriter plus a few control characters. [...] but I would strongly recommend just encoding and decoding regardless. Use text internally and bytes at the outside. That feels entirely wrong. I don't know what b'\x9A' means without knowing the character set and character encoding. If the encoding is a multibyte one, b'\x9A' doesn't mean anything on its own. That's why I want to treat it as binary. If you don't know the encoding then you don't know you're looking at a hex digit. OTOH, if the binary data contain ASCII data then you do know the encoding: it is ASCII. If that is mixed with other data then you need to know where it starts/stops in order to pull it out to be decoded. The overall data may be a mix, but the bit you're pulling out is encoded text, which you could decode. Cheers, Cameron Simpson -- https://mail.python.org/mailman/listinfo/python-list
Re: How to catch a usefull error message ?
Vincent Vande Vyvre wrote:
static int
ImgProc_init(ImgProc *self, PyObject *args, PyObject *kwds)
{
PyObject *tmp;
char *fname;
if (!PyArg_ParseTuple(args, "s", &fname))
return NULL;
You should be returning -1 here, not NULL.
--
Greg
--
https://mail.python.org/mailman/listinfo/python-list
Re: need help understanding: converting text to binary
Cameron Simpson wrote: If you don't know the encoding then you don't know you're looking at a hex digit. OTOH, if the binary data contain ASCII data then you do know the encoding: it is ASCII. Not necessarily, it could be a superset of ASCII such as latin-1 or utf-8. You do need to know that it is such an encoding, however. If the encoding is completely unknown, you can't make any assumptions. -- Greg -- https://mail.python.org/mailman/listinfo/python-list
Re: need help understanding: converting text to binary
In comp.lang.python, Cameron Simpson wrote: > On 23Apr2019 20:35, Eli the Bearded <*@eli.users.panix.com> wrote: >> That feels entirely wrong. I don't know what b'\x9A' means without >> knowing the character set and character encoding. If the encoding is a >> multibyte one, b'\x9A' doesn't mean anything on its own. That's why I >> want to treat it as binary. > If you don't know the encoding then you don't know you're looking at a > hex digit. OTOH, if the binary data contain ASCII data then you do know > the encoding: it is ASCII. Hmmm. Maybe I'm not making myself clear. ASCII "=9a" should decode to b'\x9A' and it is that binary byte for which I don't know the meaning and why I don't want to use "text internallly" for as suggested upthread. > If that is mixed with other data then you need to know where it > starts/stops in order to pull it out to be decoded. The overall data may > be a mix, but the bit you're pulling out is encoded text, which you > could decode. I do want to decode it, and possibly compare it for an exact match. And because there are different possible encodings of the same source data (consider the trivial case of "=9A" versus "=9a", I don't want to just keep it in raw form). Elijah -- not to mention QP versus b64 -- https://mail.python.org/mailman/listinfo/python-list
Re: Importing module from another subdirectory
Rich Shepard writes: > On Tue, 23 Apr 2019, dieter wrote: > ... > One project is for my own use and I understand now that a virtualenv with > its own sys.path appendices would work. Those are two separate approaches: With a "virtualenv", there is usually no need to tweak "sys.path" -- you simply install everything your project needs into the "virtualenv". "sys.path" tweaks are typically employed with a "central" Python installation, to have it look at non-standard places for module/packages under specific circumstances. > The other project is intended to be > available for installation and use by clients (most of whom work in a > Windows environment) so I need to learn how development in a virtual > environment affects ultimate distribution via github. A "virtual environment" behaves like a typical Python installation (it is just somewhat isolated). Thus, there is no problem to install a package developped in a "virtual environment" into any Python environment -- aport from potential name conflicts (choose appropriate names to reduce this risk). -- https://mail.python.org/mailman/listinfo/python-list
