[Python-Dev] os.path.join failure mode
If a function (or other non-string object) is accidentally passed as an
argument to os.path.join() the result is an AttributeError:
In [3]: os.path.join(fn, "path")
> ---
> AttributeErrorTraceback (most recent call last)
> /home/tom/ in ()
> > 1 os.path.join(fn, "path")
> /usr/lib/python2.7/posixpath.pyc in join(a, *p)
> 66 if b.startswith('/'):
> 67 path = b
> ---> 68 elif path == '' or path.endswith('/'):
> 69 path += b
> 70 else:
> AttributeError: 'function' object has no attribute 'endswith'
It's relatively easy to run into this if you mean to pass the return value
of a function (fn()) as the argument but accidentally forget to append
parens (()) to the callable, thus passing the function itself instead.
Would it not be more helpful to raise a TypeError("Argument must be a
string") than the slightly more mysterious AttributeError?
It's not the most difficult error in the world to figure out, I admit, but
a TypeError just seems like the more correct thing to do here.
Thanks,
Tom
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] os.path.join failure mode
On Sat, 09 Feb 2013 09:59:13 +, Thomas Scrace wrote:
> If a function (or other non-string object) is accidentally passed as an
> argument to os.path.join() the result is an AttributeError:
>
>
> In [3]: os.path.join(fn, "path")
> > ---
> > AttributeErrorTraceback (most recent call last)
> > /home/tom/ in ()
> > > 1 os.path.join(fn, "path")
> > /usr/lib/python2.7/posixpath.pyc in join(a, *p)
> > 66 if b.startswith('/'):
> > 67 path = b
> > ---> 68 elif path == '' or path.endswith('/'):
> > 69 path += b
> > 70 else:
> > AttributeError: 'function' object has no attribute 'endswith'
>
>
>
> It's relatively easy to run into this if you mean to pass the return value
> of a function (fn()) as the argument but accidentally forget to append
> parens (()) to the callable, thus passing the function itself instead.
>
> Would it not be more helpful to raise a TypeError("Argument must be a
> string") than the slightly more mysterious AttributeError?
>
> It's not the most difficult error in the world to figure out, I admit, but
> a TypeError just seems like the more correct thing to do here.
We don't generally do that kind of type checking in Python. Sometimes the
error that results is obscure enough that adding an early check of some
sort is worth it, but this one is very clear, so I don't see that an
additional check would add much value here.
The reason we avoid such type checks is that we prefer to operate via
"duck typing", which means that if an object behaves like the expected
input, it is accepted. Here, if we did an explicit type check for str,
it would prevent join from working on an "act alike" object that had
just enough str like methods to work correctly in os.join (for example,
some specialized object that was among other things a filename proxy).
Granted, this is pretty far-fetched for os.path.join, but the general
principle applies :)
--David
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Force to use Py_CLEAR and Py_VISIT in example for Python extending
For now
http://docs.python.org/3/extending/newtypes.html#supporting-cyclic-garbage-collection
at first the doc demonstrate long way than note Py_CLEAR and Py_VISIT macroses.
I like to remove code similar to
if (self->first) {
vret = visit(self->first, arg);
if (vret != 0)
return vret;
}
and
tmp = self->first;
self->first = NULL;
Py_XDECREF(tmp);
and replace those to Py_VISIT and Py_CLEAN.
I think we have to demonstrate best practices in our examples.
Let's py2.7 branch live untouched, change only docs for python 3.x
Any objections?
--
Thanks,
Andrew Svetlov
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Force to use Py_CLEAR and Py_VISIT in example for Python extending
2013/2/9 Andrew Svetlov :
> For now
> http://docs.python.org/3/extending/newtypes.html#supporting-cyclic-garbage-collection
> at first the doc demonstrate long way than note Py_CLEAR and Py_VISIT
> macroses.
>
> I like to remove code similar to
>
> if (self->first) {
> vret = visit(self->first, arg);
> if (vret != 0)
> return vret;
> }
>
> and
>
> tmp = self->first;
> self->first = NULL;
> Py_XDECREF(tmp);
>
> and replace those to Py_VISIT and Py_CLEAN.
>
> I think we have to demonstrate best practices in our examples.
> Let's py2.7 branch live untouched, change only docs for python 3.x
>
> Any objections?
IMO, it's fine if you change 2.7, too.
--
Regards,
Benjamin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] os.path.join failure mode
On Sat, Feb 9, 2013 at 11:31 PM, R. David Murray wrote: > The reason we avoid such type checks is that we prefer to operate via > "duck typing", which means that if an object behaves like the expected > input, it is accepted. Here, if we did an explicit type check for str, > it would prevent join from working on an "act alike" object that had > just enough str like methods to work correctly in os.join (for example, > some specialized object that was among other things a filename proxy). > Granted, this is pretty far-fetched for os.path.join, but the general > principle applies :) I occasionally ponder the possibility of a time where we have a string.Text ABC, and can stop hardcoding checks for isinstance(str) everywhere :P (No, I'm not volunteering to write it. And if anyone does want to discuss it further, please start a new thread on python-ideas, as python-dev isn't the right venue) Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Force to use Py_CLEAR and Py_VISIT in example for Python extending
On Sat, Feb 9, 2013 at 11:52 PM, Andrew Svetlov wrote: > I think we have to demonstrate best practices in our examples. > Let's py2.7 branch live untouched, change only docs for python 3.x > > Any objections? For debugging purposes, I think extension developers do need to understand what those macros are doing under the covers. However, the order of presentation should definitely be reversed to "here is how you should write it", followed by an explanation of what that actually means with the macros expanded. For Py_CLEAR, it would be OK to just link to the macro's entry in the C API docs, but for Py_VISIT, I think it's worth expanding out the full example. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] os.path.join failure mode
R. David Murray bitdance.com> writes: > The reason we avoid such type checks is that we prefer to operate via > "duck typing", which means that if an object behaves like the expected > input, it is accepted. Here, if we did an explicit type check for str, > it would prevent join from working on an "act alike" object that had > just enough str like methods to work correctly in os.join (for example, > some specialized object that was among other things a filename proxy). I see, that makes sense. Thanks. I guess this actually goes to the heart of the flexibility of dynamic/weakly-typed languages. If we wanted to strictly enforce the type of a function's arguments we would use a strong type system. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] os.path.join failure mode
On Sat, 09 Feb 2013 14:35:33 +, Thomas Scrace wrote: > R. David Murray bitdance.com> writes: > > > The reason we avoid such type checks is that we prefer to operate via > > "duck typing", which means that if an object behaves like the expected > > input, it is accepted. Here, if we did an explicit type check for str, > > it would prevent join from working on an "act alike" object that had > > just enough str like methods to work correctly in os.join (for example, > > some specialized object that was among other things a filename proxy). > > I see, that makes sense. Thanks. I guess this actually goes to the heart of > the > flexibility of dynamic/weakly-typed languages. If we wanted to strictly > enforce > the type of a function's arguments we would use a strong type system. No, it is more the difference between *statically* typed and dynamically typed. Python is a strongly typed language (every object has a specific type). --David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] os.path.join failure mode
On 9 February 2013 17:15, R. David Murray wrote: > > No, it is more the difference between *statically* typed and dynamically > typed. Python is a strongly typed language (every object has a specific > type). > > > Yes, sorry, I think I probably have my terminology confused. What I really meant by 'strongly typed' was something like "a language where values passed to functions must be of a specified type". That is to say, something more along the lines of C, where function definitions explicitly specify the type of each argument, and where it is an error to pass to a function an argument whose type does not match that in the definition. Sorry to turn this into programming design 101, I know it's not really the place for it. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] os.path.join failure mode
On 2/9/2013 8:31 AM, R. David Murray wrote:
On Sat, 09 Feb 2013 09:59:13 +, Thomas Scrace wrote:
If a function (or other non-string object) is accidentally passed as an
argument to os.path.join() the result is an AttributeError:
In [3]: os.path.join(fn, "path")
---
AttributeErrorTraceback (most recent call last)
/home/tom/ in ()
> 1 os.path.join(fn, "path")
/usr/lib/python2.7/posixpath.pyc in join(a, *p)
66 if b.startswith('/'):
67 path = b
---> 68 elif path == '' or path.endswith('/'):
69 path += b
70 else:
AttributeError: 'function' object has no attribute 'endswith'
It's relatively easy to run into this if you mean to pass the return value
of a function (fn()) as the argument but accidentally forget to append
parens (()) to the callable, thus passing the function itself instead.
Would it not be more helpful to raise a TypeError("Argument must be a
string") than the slightly more mysterious AttributeError?
It's not the most difficult error in the world to figure out, I admit, but
a TypeError just seems like the more correct thing to do here.
I agree. Since the exception type is not documented and since no one
should intentionally pass anything but strings, and therefore should not
be writing
try:
os.path.join(a,b)
except AttributeError:
barf()
I think it would be acceptable to make a change in 3.4.
We don't generally do that kind of type checking in Python. Sometimes the
error that results is obscure enough that adding an early check of some
sort is worth it, but this one is very clear, so I don't see that an
additional check would add much value here.
Changing AttributeError to TypeError only requires try-except, which is
cheap if there is no error, not an early type check.
The reason we avoid such type checks is that we prefer to operate via
"duck typing", which means that if an object behaves like the expected
input, it is accepted.
And catching the existing AttributeError allows work-alikes. I agree
that an isinstance check is bad as well as unnecessary.
As it is, the core code for path, given above, is *already* wrapped in
try: except TypeError, (to give a more friendly error message!, as
TypeError is most like from a unicode-bytes mixing). So there would be
no extra cost for correct calls and all that is needed is another except
clause.
except AttributeError as e:
bad = e.args[0].split()[0]
msg = "all components must be strings; one is a {}".format(bad)
raise TypeError("all components must be strings")
Thomas, unless David or someone else shoots down this idea here, I say
go ahead and open an enhancement issue and add terry.reedy as nosy.
--
Terry Jan Reedy
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] os.path.join failure mode
On Sat, 09 Feb 2013 13:57:41 -0500, Terry Reedy wrote:
> On 2/9/2013 8:31 AM, R. David Murray wrote:
> Changing AttributeError to TypeError only requires try-except, which is
> cheap if there is no error, not an early type check.
>
> > The reason we avoid such type checks is that we prefer to operate via
> > "duck typing", which means that if an object behaves like the expected
> > input, it is accepted.
>
> And catching the existing AttributeError allows work-alikes. I agree
> that an isinstance check is bad as well as unnecessary.
>
> As it is, the core code for path, given above, is *already* wrapped in
> try: except TypeError, (to give a more friendly error message!, as
> TypeError is most like from a unicode-bytes mixing). So there would be
> no extra cost for correct calls and all that is needed is another except
> clause.
>
> except AttributeError as e:
> bad = e.args[0].split()[0]
> msg = "all components must be strings; one is a {}".format(bad)
> raise TypeError("all components must be strings")
>
> Thomas, unless David or someone else shoots down this idea here, I say
> go ahead and open an enhancement issue and add terry.reedy as nosy.
Especially given that there is already a try/except there, this seems
fine to me. I think perhaps the error text would be better if it
referred only to the type that is invalid, not to str. So something
like:
TypeError("object of type {} is not valid as a path
"component".format(type(bad)))
--David
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] os.path.join failure mode
On 9 February 2013 20:08, R. David Murray wrote:
>
>
> Especially given that there is already a try/except there, this seems
> fine to me. I think perhaps the error text would be better if it
> referred only to the type that is invalid, not to str. So something
> like:
>
> TypeError("object of type {} is not valid as a path
> "component".format(type(bad)))
>
> --David
I agree that this is a better error message. I have raised an
enhancement issue as Terry suggested:
http://bugs.python.org/issue17174
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] os.path.join failure mode
Terry Reedy wrote: I agree. Since the exception type is not documented and since no one should intentionally pass anything but strings, and therefore should not be writing try: os.path.join(a,b) except AttributeError: barf() I think it would be acceptable to make a change in 3.4. Why should we go out of our way to address this particular case, and not any of the infinitely many other situations where you could forget to add parens to a function call? -- Greg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] os.path.join failure mode
On 2/9/2013 6:23 PM, Greg Ewing wrote: > Terry Reedy wrote: >> I agree. Since the exception type is not documented and since no one >> should intentionally pass anything but strings, and therefore should >> not be writing >> >> try: >> os.path.join(a,b) >> except AttributeError: >> barf() >> >> I think it would be acceptable to make a change in 3.4. > > Why should we go out of our way to address this particular > case, and not any of the infinitely many other situations > where you could forget to add parens to a function call? > I agree we do not want to go down this slippery slope. The number of such places in the standard library is huge. The existing error message is one that a user should be able to understand, since it's so common. -- Eric. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] os.path.join failure mode
And this is not just with Python. Try any other dynamic language (Ruby), send a function in place of a string and see failure msg. And if the question is really about path joins and path manipulations, then I believe PEP 428 ( http://www.python.org/dev/peps/pep-0428/) would be better candidate to address these scenarios. On Sat, Feb 9, 2013 at 3:45 PM, Eric V. Smith wrote: > On 2/9/2013 6:23 PM, Greg Ewing wrote: >> Terry Reedy wrote: >>> I agree. Since the exception type is not documented and since no one >>> should intentionally pass anything but strings, and therefore should >>> not be writing >>> >>> try: >>> os.path.join(a,b) >>> except AttributeError: >>> barf() >>> >>> I think it would be acceptable to make a change in 3.4. >> >> Why should we go out of our way to address this particular >> case, and not any of the infinitely many other situations >> where you could forget to add parens to a function call? >> > > I agree we do not want to go down this slippery slope. The number of > such places in the standard library is huge. The existing error message > is one that a user should be able to understand, since it's so common. > > -- > Eric. > ___ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/senthil%40uthcode.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
