Re: [Python-Dev] Should there be a way or API for retrieving from a code object a loader method and package file where the code comes from?

2008-12-23 Thread R. Bernstein
Paul Moore writes:
 > 2008/12/23  :
 > > What is wanted is a uniform way get and describe a file location
 > > from a code object that takes into account the file might be a member
 > > of an archive.
 > 
 > But a code object may not have come from a file. 

Right. That's why I mentioned for example "eval" and "exec" that you
cite below. So remove the "file" in what is cited above. Replace with:
"a unform way to get information (not necessarily just the source
text) about the location/origin of code from a code object.

 > Ignoring the
 > interactive prompt (not because it's unimportant, just because people
 > have a tendency to assume it's the only special case :-)) you need to
 > consider code loaded via a PEP302 importer from (say) a sqlite
 > database, or code created using compile(), or possibly even more
 > esoteric means.
 > 
 > So I'm not sure your request is clearly specified.

Is the above any more clear? 

 > 
 > > Are there even guidelines for saying what string goes into a code
 > > object's co_filename? Clearly it should be related to the source code
 > > that generated the code, and there are various conventions that seem
 > > to exist when the code comes from an "eval" or an "exec".
 > 
 > I'm not aware of guidelines - the documentation for compile() says
 > "The filename argument should give the file from which the code was
 > read; pass some recognizable value if it wasn't read from a file
 > ('' is commonly used)" which is pretty non-commital.
 > 
 > > But empirically it seems as though there's some variation. It could be
 > > an absolute file or a file with no root directory specified. (But is
 > > it possible to have things like "." and ".."?). And in the case of a
 > > member of a package what happens? Should it be just the member without
 > > the package? Or should it include the package name like
 > >   /usr/lib/python2.5/site-packages/tracer-0.1.0-py2.5.egg/tracer.py ?
 > >
 > > Or be unspecified? If left unspecified as I gather it is now, it makes
 > > it more important to have some sort of common routine to be able to
 > > pick out the archive part in a filesystem from the member name inside
 > > the archive.
 > 
 > I think you need to be clear on *why* you want to know this
 > information. Once it's clear what you're trying to achieve, it will be
 > easier to say what the options are.

This is what I wrote originally (slightly modified):
  
  A use case here I am thinking of here is in a stack trace or a
  debugger, or a tool which wants to show in great detail, information
  from a code object obtained possibly via a frame object.

I find it kind of sucky to see in a traceback: "" as opposed
to the text (or prefix of the text) of the actual string that was
passed. Or something that has been referred to as a "pseudo-file" like
/usr/lib/python2.5/site-packages/tracer-0.1.0-py2.5.egg/foo/bar.py
when it is really member foo/bar.py of zipped egg
/usr/lib/python2.5/site-packages/tracer-0.1.0-py2.5.egg.

(As a separate issue, it seems that zipimporter file locations inside
setuptools may have a problem.)

Inside a debugger or an IDE, it is conceivable a person might want
loader, and module information, and if the code is part of an archive
file, then member information. (If part of an eval string then, the
eval string.)

 > 
 > It sounds like you're trying to propose a stronger convention, to be
 > enforced in the future. 

Well, I wasn't sure if there was one. But I gather from what you write,
there isn't. :-)

Yes, I would suggest a stronger convention. Or a more up-front
statement that none is desired/forthcoming.

 > (At least, your suggestion of producing stack
 > traces implies that you want stack trace code not to have to deal with
 > the current situation). When PEP 302 was being developed, we were
 > looking at similar issues. That's why I pointed you at get_source() -
 > it was the best we could do with all the various conflicting
 > requirements, and the fact that it's optional is because we had to
 > cater for cases where there simply wasn't a meaningful answer.
 > Frankly, backward compatibility requirements kill a lot of the options
 > here.
 > 
 > Maybe what you want is a *pair* of linked conventions:
 > 
 > - co_filename (or a replacement) returns a (notionally opaque, but
 > in practice a filename for file-based cases) token representing "the
 > file or other object the code came from"

This would be nice.

 > -  xxx.get_source_code(token) is a function (I don't know where,
 > xxx is a placeholder for some "suitable" module) which, given such a
 > token, returns the source, or None if there's no viable concept of
 > "the source".

There always is a viable concept of a source. It's whatever was done
to get the code. For example, if it was via an eval then the source
was the eval function and a string, same for exec. If it's via
database access, well that then and some summary info about what's
known about that. 

 > 
 > Or maybe you want a (possibly separate) a

Re: [Python-Dev] Should there be a way or API for retrieving from a code object a loader method and package file where the code comes from?

2008-12-23 Thread R. Bernstein
Nick Coghlan writes:
 > 3. Do what a number of standard library APIs (e.g. linecache) that
 > accept filenames do and also accept an optional "module globals"
 > argument. 

Actually, I did this and committed a change (to pydb) before posting
any of these queries. ;-)

If "a number of standard library APIs" are doing the *same* thing,
then shouldn't this exposed as a common routine?

If on the other hand, by "a number" you mean "one" as in linecache --
1 *is* a number too! -- then perhaps the relevant code that is buried
inside the "updatecache" should be exposed on its own.  (As a side
benefit that code can be tested separately too!)

Should I file a feature request for this? 
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Should there be a way or API for retrieving from a code object a loader method and package file where the code comes from?

2008-12-23 Thread R. Bernstein
Nick Coghlan writes:
 > R. Bernstein wrote:
 > > Nick Coghlan writes:
 > >  > 3. Do what a number of standard library APIs (e.g. linecache) that
 > >  > accept filenames do and also accept an optional "module globals"
 > >  > argument. 
 > > 
 > > Actually, I did this and committed a change (to pydb) before posting
 > > any of these queries. ;-)
 > > 
 > > If "a number of standard library APIs" are doing the *same* thing,
 > > then shouldn't this exposed as a common routine?
 > > 
 > > If on the other hand, by "a number" you mean "one" as in linecache --
 > > 1 *is* a number too! -- then perhaps the relevant code that is buried
 > > inside the "updatecache" should be exposed on its own.  (As a side
 > > benefit that code can be tested separately too!)
 > > 
 > > Should I file a feature request for this? 
 > 
 > The reason for my slightly odd phrasing is that all of the examples I
 > was originally going to mention (traceback, pdb, doctest, inspect)
 > actually all end up calling linecache to do the heavy lifting.
 > 
 > So it is possible that linecache.getlines() actually *is* the common
 > routine you're looking for 

I never asked about getting the text lines for the source code, no
matter how many times people suggest that as an alternative. :-)

Instead, I was asking about a common way to get information about the
source location for say a frame or traceback object (which might
include package name and type) and suggest that there should be a more
unambiguous way to display this information than seems to be in use at
present.

Part of work to retrieve or displaying that information has to do the
some of the same things that is inside of linecache.updatecache()
*before* it retrieves the lines of the source code (when
possible). And possibly parts of it include parts of what's done in
pieces of the inspect module.

 > - it just needs to be added to the
 > documentation and the __all__ attribute for linecache to be officially
 > supported. Currently, only the single line getline() function is
 > documented and exposed via __all__, but I don't see any reason for that
 > restriction - linecache.getlines() has been there with a stable API
 > since at least Python 2.5.
 > 
 > For cases where you have an appropriate Python object (i.e. a module,
 > function, method, class, traceback, frame or code object) rather than a
 > pseudo-filename, then inspect.getsource() actually jumps through a lot
 > of hoops to try to find the actual source code for that object - in
 > those cases, using the appropriate inspect function is generally a much
 > better idea than trying to interpret __file__ yourself.
 > 
 > Cheers,
 > Nick.

Thanks for the information. I will keep in mind those inspect routines. 

They probably will be a helpful for another problem I had been
wondering about -- how one can determine if there is no code
associated at a given a line and file. (In other words and invalid
location for a debugger line breakpoint, such as because the line
part of a comment or the interior line of a string that spans many
lines)

 > 
 > -- 
 > Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
 > ---
 > ___
 > Python-Dev mailing list
 > Python-Dev@python.org
 > http://mail.python.org/mailman/listinfo/python-dev
 > Unsubscribe: 
 > http://mail.python.org/mailman/options/python-dev/rocky%40gnu.org
 > 
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com