Smart quote character replacement for those with deficient sed-fu.
Use case: === 1. OSX 10.7, bunch of .tex files culled from a blog via some cruddy old Perl script. .tex files are utf-8 encoded, which means the quotes and apostrophes drop out going through pdflatex. 2. Recommend using git to manage the .tex files, which are all in a /src directory in the project. If you're going to do batch edits, You Will Need to branch, realize you gooned up, and roll back. 3. Understood, sed is the more classical tool here, except that I could not it to work in my OSX terminal. Invocation: === python charfix.py `ls src/*.tex` charfix.py code: === # -*- coding: utf-8 -*- import fileinput def process(line): print line.replace( "’", "'" ).replace( "“", '"' ).replace( "”", '"' ) if __name__=="__main__": for line in fileinput.input( inplace=1 ): process(line) === Discussion: === Thank you, python! -- http://mail.python.org/mailman/listinfo/python-list
Re: Choosing a Python IDE. what is your Pythonish recommendation? I do not know what to choose.
On Monday, January 2, 2017 at 6:39:03 AM UTC-5, Antonio Caminero Garcia wrote: > Hello, I am having a hard time deciding what IDE or IDE-like code editor > should I use. This can be overwhelming. > > So far, I have used Vim, Sublime, Atom, Eclipse with PyDev, Pycharm, IntelliJ > with Python plugin. > > The thing with the from-the-scratch full featured IDEs (Eclipse, IntelliJ, > Pycharm) is that they look like a space craft dashboard and that unwarranted > resources consumption and the unnecessary icons. I want my IDE to be > minimalistic but powerful. My screen should be mostly “made of code” as > usually happens in Vim, Sublime or Atom. However, Pycharm is really cool and > python oriented. > > The problem with Vim is the learning curve, so I know the very basic stuff, > but obviously not enough for coding and I do not have time to learn it, it is > a pity because there are awesome plugins that turns Vim into a lightweight > powerful IDE-like. So now it is not an option but I will reconsider it in the > future, learning little by little. Also, I am not very fan GUI guy if the > task can be accomplished through the terminal. However, I don’t understand > why people underrate GUIs, that said I normally use shortcuts for the most > frequent tasks and when I have to do something that is not that frequent then > I do it with the mouse, for the latter case in vim you would need to look for > that specific command every time. > > Sublime is my current and preferred code editor. I installed Anaconda, Git > integration and a couple of additional plugins that make sublime very > powerful. Also, what I like about sublime compared to the full featured IDEs, > besides the minimalism, is how you can perform code navigation back and forth > so fast, I mean this is something that you can also do with the others but > for some subjective reason I specifically love how sublime does it. The code > completion in sublime I do not find it very intelligence, the > SublimeCodeIntel is better than the one that Anaconda uses but the > completions are not as verbose as in the IDEs. > > Now, I am thinking about giving a try to Visual Studio Code Edition (take a > look, it sounds good > https://marketplace.visualstudio.com/items?itemName=donjayamanne.python). I > need an editor for professional software development. What would you > recommend to me? I am told that means other than Emacs exist to edit code and interact with systems, but I don't worry about them. Happy New Year, Chris -- https://mail.python.org/mailman/listinfo/python-list
codecs / subprocess interaction: utf help requested
The first print statement does what you'd expect. The second print statement has rather a lot of rat in it. The goal here is to write a function that will return the man page for some command (mktemp used as a short example here) as text to client code, where the groff markup will be chopped to extract all of the command options. Those options will eventually be used within an emacs mode, all things going swimmingly. I don't know what's going on with the piping in the second version. It looks like the output of p0 gets converted to unicode at some point, but I might be misunderstanding what's going on. The 4.8 codecs module documentation doesn't really offer much enlightment, nor google. About the only other place I can think to look would be the unit test cases shipped with python. Sort of hoping one of the guru-level pythonistas can point to illumination, or write something to help out the next chap. This might be one of those catalytic questions, the answer to which tackles five other questions you didn't really know you had. Thanks, Chris --- #!/usr/bin/python import subprocess p = subprocess.Popen(["bzip2", "-c", "-d", "/usr/share/man/man1/mktemp. 1.bz2"] , stdout=subprocess.PIPE) stdout, stderr = p.communicate() print stdout p0 = subprocess.Popen(["cat","/usr/share/man/man1/mktemp.1.bz2"], stdout=subprocess.PIPE) p1 = subprocess.Popen(["bzip2"], stdin=p0.stdout, stdout=subprocess.PIPE) stdout, stderr = p1.communicate() print stdout --- -- http://mail.python.org/mailman/listinfo/python-list
Re: codecs / subprocess interaction: utf help requested
On Jun 10, 6:10 pm, John Machin <[EMAIL PROTECTED]> wrote: > On Jun 11, 7:17 am, smitty1e <[EMAIL PROTECTED]> wrote: > > > The first print statement does what you'd expect. > > The second print statement has rather a lot of rat in it. > > The goal here is to write a function that will return the man page for > > some command (mktemp used as a short example here) as text to client > > code, where the groff markup will be chopped to extract all of the > > command options. Those options will eventually be used within an > > emacs mode, all things going swimmingly. > > I don't know what's going on with the piping in the second version. > > It looks like the output of p0 gets converted to unicode at some > > point, > > Whatever gave you that idea? > > > but I might be misunderstanding what's going on. The 4.8 > > codecs module documentation doesn't really offer much enlightment, > > nor google. About the only other place I can think to look would be > > the unit test cases shipped with python. > > Get your head out of the red herring factory; unicode, "utf" (which > one?) and codecs have nothing to do with your problem. Think about > looking at your own code and at the bzip2 documentation. > > > > > Sort of hoping one of the guru-level pythonistas can point to > > illumination, or write something to help out the next chap. This > > might be one of those catalytic questions, the answer to which tackles > > five other questions you didn't really know you had. > > Thanks, > > Chris > > --- > > #!/usr/bin/python > > import subprocess > > > p = subprocess.Popen(["bzip2", "-c", "-d", "/usr/share/man/man1/mktemp. > > 1.bz2"] > > , stdout=subprocess.PIPE) > > stdout, stderr = p.communicate() > > print stdout > > > p0 = subprocess.Popen(["cat","/usr/share/man/man1/mktemp.1.bz2"], > > stdout=subprocess.PIPE) > > p1 = subprocess.Popen(["bzip2"], stdin=p0.stdout, > > stdout=subprocess.PIPE) > > stdout, stderr = p1.communicate() > > print stdout > > --- > > You left out the command-line options for bzip2. The "rat" that you > saw was the result of compressing the already-compressed man page. > Read this:http://www.bzip.org/docs.html > which is a bit obscure. The --help output from my copy of an antique > (2001, v1.02) bzip2 Windows port explains it plainly: > """ >If invoked as `bzip2', default action is to compress. > as `bunzip2', default action is to decompress. > as `bzcat', default action is to decompress to stdout. > >If no file names are given, bzip2 compresses or decompresses >from standard input to standard output. > """ > > HTH, > John Don't I feel like the biggest dork on the planet. I had started with >cat /usr/share/man/man1/paludis.1.bz2 | bunzip2 then proceeded right to a self-foot-shoot when I went to python. *sigh* Thanks for the calibration, sir. Rm C -- http://mail.python.org/mailman/listinfo/python-list
.py to sqlite translator [1 of 2]
Disclaimer(s): the author is nobody's pythonista. This could probably be done more elegantly. The driver for the effort is to get PyMacs to work with new-style classes. This rendering stage stands alone, and might be used for other purposes. A subsequent post will show using the resulting file to produce (I think valid) .el trampoline signatures for PyMacs. If nothing else, it shows some python internals in an interesting way. Tested against version 2.5.1 Maybe "lumberjack.py" would be a better name, since "It cuts down trees, goes real slow, and uses disk galore. Wishes it'd been webfoot[1], just like its dear author". Cheers, Chris [1] Author was born in Oregon. #A sample file: class sample( object ): """fairly trivial sample class for demonstration purposes. """ def __init__( self , some_string ): self.hold_it = some_string def show( self ): print self.hold_it #Invocation: # ./pysqlrender.py -f sample.py -o output #Script: #!/usr/bin/python """Script to dump the parse tree of an input file to a SQLite database. """ from optparse import OptionParser import os import parser import pprint import re import sqlite3 import symbol import token import types from types import ListType \ , TupleType target_table = """CREATE TABLE tbl_parse_tree( parse_tree_id INTEGER PRIMARY KEY AUTOINCREMENT , parse_tree_symbol_id , parse_tree_indent , parse_tree_value );""" target_insert = """INSERT INTO tbl_parse_tree( parse_tree_symbol_id , parse_tree_indent , parse_tree_value ) VALUES (%s, %s, '%s' );""" symbol_table = """CREATE TABLE tlp_parse_tree_symbol ( parse_tree_symbol_id INTEGER PRIMARY KEY , parse_tree_symbol_val );""" symbol_insert = """INSERT INTO tlp_parse_tree_symbol ( parse_tree_symbol_id , parse_tree_symbol_val ) VALUES ( %s, '%s' );""" class symbol_manager( object ): """ Class to merge symbols and tokens for ease of use. """ def __init__( self , c): for k in symbol.sym_name: sql = symbol_insert % ( k, symbol.sym_name[k] ) try: c.execute( sql ) except sqlite3.IntegrityError: pass for k in token.tok_name: sql = symbol_insert % ( k, token.tok_name[k] ) try: c.execute( sql ) except sqlite3.IntegrityError: pass def get_symbol( self , key ): ret = -1 if symbol.sym_name.has_key(key): ret = symbol.sym_name[key] elif token.tok_name.has_key(key) : ret = token.tok_name[ key] return ret def recurse_it( self, tester ): """Check to see if dump_tup should recurse """ if self.get_symbol(tester) > 0: return True return False class stocker( object ): """Remembers the depth of the tree and effects the INSERTs into the output file. """ def __init__( self ): self.cur_indent = 0 def do_symbol( self , c , symbol_value , val = "" ): """Stuff something from the parse tree into the database table. """ if symbol_value==5: self.cur_indent += 1 elif symbol_value==6: self.cur_indent -= 1 try: sql = target_insert\ % ( symbol_value , self.cur_indent , re.sub( "'", "`", str(val) )) c.execute( sql ) except AttributeError: print "connection bad in lexer" except sqlite3.OperationalError: print "suckage at indent of %s for %s" \ % (self.cur_indent, sql) def dump_tup( tup , sym , c , stok ): """Recursive function to descend TUP and analyze its elements. tup parse tree of a file, rendered as a tuple sym dictionary rendered from symbol module c live database cursor stok output object effect token storage """ for node in tup: typ = type( node ) r = getattr( typ , "__repr__" , None ) if (issubclass(typ, tuple) and r is tuple.__repr__): if token.tok_name.has_key( node[0] ): stok.do_symbol( c , node[0] , node[1] ) elif sym.recurse_it( node[0] ): stok.do_symbol( c , node[0] , '__py__' ) #If you say node[1] here, # the sqlite file is fat
sqlite to .el translator [2 of 2]
Since I haven't yet been shot for the earlier post of a .py to sqlite
rendering script, here is another script that takes the previous
output and does something useful.
Note that this is not the full integration with PyMacs--I rather hope
to spark some interest and save some time headbanging on a) getting
this right, and b) getting it integrated.
What sort of a poetic reference can we get here? "Inflammation of the
source code reminds me of your script..."
Cheers,
Chris
#invocation:
[EMAIL PROTECTED] ~/proj/mddl4/test $ ./elisp_extractor.py -f output -o
sample.el -p sample
#output:
[EMAIL PROTECTED] ~/proj/mddl4/test $ cat sample.el
(sample-sample---init-- (self some-string))
(sample-sample-show (self))
#script:
#!/usr/bin/python
from optparse import OptionParser
import re
import sqlite3
CLASS_OFFSET = 4
ALL_CLASSES = """SELECT parse_tree_id
, parse_tree_value
FROM tbl_parse_treeA
INNER JOIN
( SELECTMAX(parse_tree_id) AS
parse_tree_id_
FROM tbl_parse_tree
UNION ALL
SELECTparse_tree_id + %s --
offset to get name
FROM tbl_parse_tree
WHERE parse_tree_symbol_id=328 --
classdefs
AND parse_tree_indent = 0 --
but not nested
ORDER BY 1 DESC) B
ON A.parse_tree_id = B.parse_tree_id_;""" %
CLASS_OFFSET
FUNC_OFFSET = 4
CLASS_FUNCS = """SELECT parse_tree_id
, parse_tree_value
FROM
tbl_parse_tree A
INNER JOIN
( SELECTparse_tree_id + %s AS
parse_tree_id_
FROM tbl_parse_tree
WHERE parse_tree_id BETWEEN %s AND
%s
AND parse_tree_symbol_id = 261
--funcdefs
AND parse_tree_indent= 1 ) B
ON A.parse_tree_id=B.parse_tree_id_;
--no nested funcs"""
FUNC_ARGS = """SELECT parse_tree_value
FROM tbl_parse_tree
WHERE parse_tree_id BETWEEN %s AND
(SELECT MIN(parse_tree_id)
FROM tbl_parse_tree
WHERE parse_tree_id> %s
AND parse_tree_symbol_id = 8)
AND parse_tree_symbol_id = 1;"""
def extract_emacs_representation_of_python_classes( module_name
, file_name
, target_name ):
"""module_name prefixes all of the functions
file_nameis an sqlite database file specification
target_name is an output .el file specification
"""
conn = sqlite3.connect( file_name )
conn.isolation_level = None
c= conn.cursor()
c.execute( ALL_CLASSES )
buf = 0
class_ranges = []
targ_file= open( target_name, 'w' )
#build up a list of class data
for row in c:
if (buf != 0):
class_ranges.append( (row[0],buf-1,row[1]) )
buf = row[0]
for cur_class in class_ranges:
c.execute( CLASS_FUNCS % ( FUNC_OFFSET,
cur_class[0],cur_class[1]) )
func_args = []
for row in c:
func_args.append( (row[0]+1,row[1]) )
#print report
cur_prefix = "%s-%s" % (module_name,
re.sub("_","-",cur_class[2]))
for row in func_args:
c.execute( FUNC_ARGS % (row[0], row[0]) )
args = []
for arg in c:
args.append( arg[0] )
targ_file.write( "(%s-%s (%s))\n" \
% ( cur_prefix
, re.sub( "_", "-", row[1])
, re.sub( "_", "-", " ".join(args
targ_file.close()
def main():
usage= "usage: %prog [options] arg"
parser = OptionParser(usage)
parser.add_option("-f", "--file", dest="filename"
, action="store", type="string"
, help ="connect to sqlite file FILENAME")
parser.add_option("-o", "--output",dest="output"
, action="store", type="string"
, help ="name of .el output file")
parser.add_option("-p", "--prefix",dest="prefix"
, action="store", type="string"
, help ="prefix for use in elisp")
(options, args) = parser.parse_args()
extract_emacs_representation_of_python_classes( options.prefix
, options.filename
, options.output )
if __name__ =
unified command line args, environment variables, .conf file settings.
Just a fun exercise to unify some of the major input methods for a
script into a single dictionary.
Here is the output, given a gr.conf file in the same directory with
the contents stated below:
[EMAIL PROTECTED] ~/proj/mddl4/test $ ./inputs.py
{'source_db': '/home/sweet/home.db'}
[EMAIL PROTECTED] ~/proj/mddl4/test $ source_db="test_env" ./inputs.py
{'source_db': 'test_env'}
[EMAIL PROTECTED] ~/proj/mddl4/test $ ./inputs.py -f "test_cli"
{'source_db': 'test_cli'}
For the file
=
#!/usr/bin/env python
#Goal: unification of environment variables, command line, and
# configuration file settings.
# The .conf file is the specification.
# Optional environment variables can override.
# Optional command-line inputs trump all.
# Example is a file spec for a source database for an application.
# .conf has a contents like:
#
# [CONF]
# source_db=/home/sweet/home.db
#TODO:
# 1. Decide (make an option?) to retain the simple dictionary or build
a
# class from the options.
# 2. Allow for multiple .conf locations, trying to be cool like
# xorg.conf
from ConfigParser import SafeConfigParser
from optparse import OptionParser
import os
CONF = "CONF" #section in .conf file
CONF_FILE = "gr.conf" #name of config file
PLACEBO = "placebo" #mindless default that we don't want
gconf = {} #all your config are belong to here
parser= OptionParser()
parser.add_option( "-f", "--file"
, dest= "source_db"
, help= "source database"
, default = PLACEBO )
(cl_opts, args) = parser.parse_args()
config= SafeConfigParser()
config.read(CONF_FILE)
file_conf = dict(config.items(CONF))
for k in file_conf:
gconf[k]=file_conf[k]
if os.environ.has_key(k):
gconf[k] = os.environ[k]
if cl_opts.__dict__.has_key(k):
if cl_opts.__dict__[k]!=PLACEBO:
gconf[k] = cl_opts.__dict__[k]
print gconf
--
http://mail.python.org/mailman/listinfo/python-list
Re: unified command line args, environment variables, .conf file settings.
On May 2, 11:29 pm, Ben Finney <[EMAIL PROTECTED]>
wrote:
> smitty1e <[EMAIL PROTECTED]> writes:
> > Just a fun exercise to unify some of the major input methods for a
> > script into a single dictionary.
> > Here is the output, given a gr.conf file in the same directory with
> > the contents stated below:
>
> > [EMAIL PROTECTED] ~/proj/mddl4/test $ ./inputs.py
> > {'source_db': '/home/sweet/home.db'}
> > [EMAIL PROTECTED] ~/proj/mddl4/test $ source_db="test_env" ./inputs.py
> > {'source_db': 'test_env'}
> > [EMAIL PROTECTED] ~/proj/mddl4/test $ ./inputs.py -f "test_cli"
> > {'source_db': 'test_cli'}
>
> A good start. However, you need to account for two conventions with
> configuration of programs via environment variables:
[snip]
> This is probably best done by a mapping specifically for the
> environment variable names:
>
> config_env_names = {
> 'source_db': 'GR_SOURCE_DB',
> }
>
> and then referencing that dictionary when inspecting the environment:
>
> for key in config.keys():
> # ...
> environ_name = config_env_names[key]
> environ_value = os.environ.get(environ_name)
> if environ_value is not None:
> config[key] = environ_value
> #...
Hmmm...
I kinda think your excellent points are more in the realm of policy
than mechanism.
What I was going for was to assert that there a designated section (of
many possible ones)
in a .conf file that represents the set of possible program inputs,
and then
offering an example of letting environment and command line arguments
override that
.conf section.
Your points for reducing collision are well taken. Possibly having a
removable prefix
(which could be a "magic" .conf entry) would allow for a de-conflictor
prefix like
MY_PROGRAM_REALLY_ROCKSsource_db without having to propagate such an
eyesore throughout
the program code.
In any case, it stands as a fun little worked example.
R,
C
--
http://mail.python.org/mailman/listinfo/python-list
