Re: Use global, or not
DL Neil writes: > On 29/06/19 1:44 AM, Cecil Westerhof wrote: >> I have written a GUI program where I have quit a few global variables. >> I did not like this, so I now use one global dict. Something like: >> global global_dict > ... > >> Is that an acceptable way to do this? > > > If it works, isn't that the largest part of "acceptable"? Depends on your way of thinking. ;-) I think it is always best when you write in a language to do it in that languages way. In this case Python. I also think this is a program that could be interesting to share. Then it becomes even more important to do it the Python way. And my experience is that when asking this type of question you often get a tip that helps you make your program better. > In each case, (previously discussed and/or below) you are taking > advantage of "namespaces" to keep one 'lot' of values separated and > distinct from others - as well as finding a means of 'passing them > around'. The other half of the considerations is how the values are > being retrieved/utilised within the mainline code. > > An alternative might be to use a class. Then accessing a single value > becomes instance.attribute, but when you have need for several > attribute's values or to compute a value from several attributes, an > instance.method() may simplify things. That was the other possibility I was thinking about. And that would be maybe better. Because I now do things like: global_dict['messages']['created'].format(len(filepathArr)) much better would be: instance.created(len(filepathArr)) -- Cecil Westerhof Senior Software Engineer LinkedIn: http://www.linkedin.com/in/cecilwesterhof -- https://mail.python.org/mailman/listinfo/python-list
Do I need a parser?
Dear all, I'm sure that this subject has been addressed many times before on this forum, but my poor knowledge of English and of computer jargon and concepts results on not being able to find the answer i'm looking for when I search the forum. So here is my problem: I have this open source project for the scientific community were i want to duplicate an old MS-DOS application written in Fortran. I don't have the source code. The idea is to re-write the software in Python. Originally, the old application would would need to input files: one config file, written with a specific format (see below) and a second one, the so-called scrip file, that defines the sequence of operations to be performed by the main software, also written in a specific format. To make the transition to the new application as painless as possible to the users, because most of them have their collection of scrips (and settings) developed over the years and are not willing to learn a new script language, I would like to make the new app 100% compatible with the old input files. The operation of the new software would be like this: From the shell, run "my_new_software old_script_file.***". The new software would load the old_script, parse it (?), set the internal variables, load the script and run it. So, to get to my questions: - To load and read the config file I need a parser, right? Is their a parser library where we can define the syntax of the language to use? Are there better (meaning easier) ways to accomplish the same result? - For the interpretation of the script file, I don't have any clue how to this... One important thing, the script language admits some simple control flow statements like do-wile, again written using a specific sintax. Thanks a lot for the help and sorry for the long post. Mariano Example of a config (settings) file . CONDAD -11 BURAD2 4 SALT1 1.0 KNO3 ELEC5 -2.0 mV 400 58 0. 0 . Example of a script === !Conductivity titration cmnd bur1 f set vinit 100 set endpt 2000 set mvinc 20 set drftim 1 set rdcrit cond 0.5 per_min set dosinc bur1 0.02 1000 set titdir up titratc cond bur1 -- https://mail.python.org/mailman/listinfo/python-list
Re: Do I need a parser?
On 29/06/2019 14:39, josé mariano wrote:
Dear all,
I'm sure that this subject has been addressed many times before on this forum,
but my poor knowledge of English and of computer jargon and concepts results on
not being able to find the answer i'm looking for when I search the forum.
So here is my problem: I have this open source project for the scientific
community were i want to duplicate an old MS-DOS application written in
Fortran. I don't have the source code. The idea is to re-write the software in
Python. Originally, the old application would would need to input files: one
config file, written with a specific format (see below) and a second one, the
so-called scrip file, that defines the sequence of operations to be performed
by the main software, also written in a specific format.
Is there any way you can get the source code? Can you track down the
original author? Are there old backups?
That might eliminate the need for rewriting, and, in any case, would
make it so much easier to be sure you're doing the right thing and not
missing something.
To make the transition to the new application as painless as possible to the
users, because most of them have their collection of scrips (and settings)
developed over the years and are not willing to learn a new script language, I
would like to make the new app 100% compatible with the old input files.
Obviously. Make sure you have tests, tests, and more tests. If there's
documentation, use it, but don't trust it.
That's assuming there are loads of old scripts, that just continuing to
use the old program is not an option. DOSbox to the rescue?
Another option might be to write a script that parses the old files and
converts them to something more friendly to your infrastructure, such as
YAML config files and Python scripts. This has two benefits:
(1) a human can easily check the result. If there are some
incompatibilities, they'll be easier to spot. If the script misses
something, the user can add it in.
(2) it might be easier to add new features later
It is a more complex and less user-friendly solution, though.
The operation of the new software would be like this: From the shell, run
"my_new_software old_script_file.***". The new software would load the
old_script, parse it (?), set the internal variables, load the script and run it.
So, to get to my questions:
- To load and read the config file I need a parser, right? Is their a parser
library where we can define the syntax of the language to use? Are there better
(meaning easier) ways to accomplish the same result?
You need to parse the file, obviously. Python is good for this sort of
thing. str.split() and the re module are your friends. The format looks
reasonably simple, so I'd just parse it into a simple data structure
with the basic tools Python provides.
- For the interpretation of the script file, I don't have any clue how to
this... One important thing, the script language admits some simple control
flow statements like do-wile, again written using a specific sintax.
From the look of it, it's one instruction per line, and the first word
of the line is, in some sense, a command? In this case, one way I can
think of to run it would be an interpreter structured something like this:
class ScriptRunner:
def __init__(self, config, script):
self._variables = {}
self._config = config
self._script_lines = []
for line in script.split('\n'):
line = line.strip()
if line.startswith('!'):
#comment
continue
cmd, *args = line.split()
self._script_lines.append((cmd.lower(), args))
def run_script(self):
self._script_iter = iter(self._script_lines)
while True:
try:
cmd, args = next(self._script_iter)
self.dispatch(cmd, args)
except StopIteration:
return
def dispatch(self, cmd, args):
method_name = f'_cmd_{cmd}'
method = getattr(self, method_name)
method(args)
def _cmd_set(self, args):
varname, value = args
self._variables[varname] = value
def _cmd_while(self, loop_args):
# check condition or something
loop_body = []
while True:
try:
cmd, args = next(self._script_iter)
# MAGIC
if cmd == 'endwhile':
break
else:
loop_body.append((cmd, args))
except StopIteration:
raise RuntimeError('loop not ended')
while condition_is_met:
for cmd, args in loop_body:
# otherwise, just carry on executing the loop body
self.dispatch(cmd, args)
In any case, there are three things you need to keep track of:
variables, where in the script you are, and what control
Re: Handle foreign character web input
On 28/06/2019 22:25, Tobiah wrote: A guy comes in and enters his last name as RÖnngren. With a capital Ö in the middle? That's unusual. So what did the browser really give me; is it encoded in some way, like latin-1? Does it depend on whether the name was cut and pasted from a Word doc. etc? Should I handle these internally as unicode? Right now my database tables are latin-1 and things seem to usually work, but not always. If your database is using latin-1, German and French names will work, but Croatian and Polish names often won't. Not to mention people using other writing systems. So Günther and François are ok, but Bolesław turns into Boles?aw and don't even think about anybody called Владимир or محمد. Also, what do people do when searching for a record. Is there some way to get 'Ronngren' to match the other possible foreign spellings? -- https://mail.python.org/mailman/listinfo/python-list
Re: change spacing to two instead of four with pep8 or flake8?
> I've also taken to having my files auto-formatted with yapf on save ... @Cameron: Have you considered black at all and if so, what are your thoughts? Thanks, Malcolm -- https://mail.python.org/mailman/listinfo/python-list
Re: Handle foreign character web input
On 6/29/19 3:19 AM, Thomas Jollans wrote: > On 28/06/2019 22:25, Tobiah wrote: >> A guy comes in and enters his last name as RÖnngren. > With a capital Ö in the middle? That's unusual. >> >> So what did the browser really give me; is it encoded >> in some way, like latin-1? Does it depend on whether >> the name was cut and pasted from a Word doc. etc? >> Should I handle these internally as unicode? Right >> now my database tables are latin-1 and things seem >> to usually work, but not always. > > > If your database is using latin-1, German and French names will work, > but Croatian and Polish names often won't. Not to mention people using > other writing systems. > > So Günther and François are ok, but Bolesław turns into Boles?aw and > don't even think about anybody called Владимир or محمد. I would say that currently, the only real reason to use an encoding other than Unicode (normally UTF-8) would be historical inertia. Maybe a field that will only ever have plain ASCII characters could use ASCII (such a field would never have real natural language words, but only computer generated codes). All the various 'codepages' were useful in their day, when machines were less capable, and Unicode hadn't been invented or wasn't supported well or was too expensive to use. Now (as I understand it), all Python (3) 'Strings' are internally Unicode, if you need something with a different encoding it needs to be in Bytes. -- Richard Damon -- https://mail.python.org/mailman/listinfo/python-list
Re: Do I need a parser?
I might be able to help. I'd need to understand a bit more about the configuration and scripting languages. Do you have any reference material? Or some way to look up the titration device on the internet? -- Bob Gailer -- https://mail.python.org/mailman/listinfo/python-list
Re: Handle foreign character web input
On 6/28/19 4:25 PM, Tobiah wrote:
A guy comes in and enters his last name as RÖnngren.
So what did the browser really give me; is it encoded
in some way, like latin-1? Does it depend on whether
the name was cut and pasted from a Word doc. etc?
Should I handle these internally as unicode? Right
now my database tables are latin-1 and things seem
to usually work, but not always.
Also, what do people do when searching for a record.
Is there some way to get 'Ronngren' to match the other
possible foreign spellings?
The first thing I'd want to do is to produce a front-end to discover the
character set (latin-1, whatever) and convert it to a standard UTF-8. e.g.:
data.decode('latin1').encode('utf8')
That gets rid of character set variations in the data, simplifying
things before any of the hard work has to be done.
Then you have a choice - store and index everything as utf-8, or
transliterate some or all strings to 7 bit US ASCII. You may have to
perform the same processing on input search strings.
I have not used it myself but there is a Python port of a Perl module
by Sean M. Burke called Unidecode. It will transliterate non-US ASCII
strings into ASCII using reasonable substitutions of non-ASCII
sequences. I believe that there are other packages that can also do this.
The easy way to use packages like this is to transliterate entire
records before putting them into your database, but then you may perplex
or even offend some users who will look at a record and say "What's
this? That's not French!" You'll also have to transliterate all input
search strings.
A more sophisticated way is to leave the records in Unicode, but add
transliterated index strings for those index strings that wind up
containing utf-8 non-ASCII chars.
There are various ways to do this that tradeoff time, space, and
programming effort. You can store two versions of each record, search
one and display the other. You can just process index strings and add
the transliterations to the record. What to choose depends on your
needs and resources.
And of course all bets are off if some of your data is Chinese,
Japanese, Hebrew, or maybe even Russian or Greek.
Sometimes I think, Why don't we all just learn Esperanto? But we all
know that that isn't going to happen.
Alan
--
https://mail.python.org/mailman/listinfo/python-list
Re: Handle foreign character web input
On 2019-06-28, Chris Angelico wrote: > On Sat, Jun 29, 2019 at 6:31 AM Tobiah wrote: >> A guy comes in and enters his last name as RÖnngren. >> >> So what did the browser really give me; is it encoded >> in some way, like latin-1? Does it depend on whether >> the name was cut and pasted from a Word doc. etc? >> Should I handle these internally as unicode? Right >> now my database tables are latin-1 and things seem >> to usually work, but not always. > > Definitely handle them as Unicode. You'll receive them in some > encoding, probably UTF-8, and it depends on the browser. You can basically assume it is the encoding that the page the form was on was using - which is a good reason to always explicitly specify utf-8 encoding on HTML pages. >> Also, what do people do when searching for a record. >> Is there some way to get 'Ronngren' to match the other >> possible foreign spellings? > > Ehh... probably not. That's a human problem, not a programming > one. Best of luck. And yet there are many programs which attempt to solve it. The Python module 'unidecode' will do a decent stab of it if the language is vaguely European. Certainly, storing the UTF-8 string and also the 'unidecoded' ASCII string and searching on both is unlikely to hurt and will often help. Additionally using Metaphone or similar will probably also help. -- https://mail.python.org/mailman/listinfo/python-list
Re: Do I need a parser?
On 6/29/19 8:39 AM, josé mariano wrote: Dear all, I'm sure that this subject has been addressed many times before on this forum, but my poor knowledge of English and of computer jargon and concepts results on not being able to find the answer i'm looking for when I search the forum. So here is my problem: I have this open source project for the scientific community were i want to duplicate an old MS-DOS application written in Fortran. I don't have the source code. The idea is to re-write the software in Python. Originally, the old application would would need to input files: one config file, written with a specific format (see below) and a second one, the so-called scrip file, that defines the sequence of operations to be performed by the main software, also written in a specific format. To make the transition to the new application as painless as possible to the users, because most of them have their collection of scrips (and settings) developed over the years and are not willing to learn a new script language, I would like to make the new app 100% compatible with the old input files. The operation of the new software would be like this: From the shell, run "my_new_software old_script_file.***". The new software would load the old_script, parse it (?), set the internal variables, load the script and run it. So, to get to my questions: - To load and read the config file I need a parser, right? Is their a parser library where we can define the syntax of the language to use? Are there better (meaning easier) ways to accomplish the same result? - For the interpretation of the script file, I don't have any clue how to this... One important thing, the script language admits some simple control flow statements like do-wile, again written using a specific sintax. Thanks a lot for the help and sorry for the long post. Mariano Example of a config (settings) file . CONDAD -11 BURAD2 4 SALT1 1.0 KNO3 ELEC5 -2.0 mV 400 58 0. 0 . Example of a script === !Conductivity titration cmnd bur1 f set vinit 100 set endpt 2000 set mvinc 20 set drftim 1 set rdcrit cond 0.5 per_min set dosinc bur1 0.02 1000 set titdir up titratc cond bur1 I'll just add a general comment here. Yes, you do need a parser and that parser should be a separate module or separate class from the rest of your program. As Thomas Jollans wrote, str.split() might be enough to do all of the string twiddling for you. If you have a separate class (or group of classes) that produces a configuration object and a script object then, if you discover examples of configuration or script files that you weren't aware of when you wrote the code, then may you only need to modify your parser code and may not have to modify your script execution logic. Finally, I want to say that I wish everyone in the U.S. had as much command of English as you do. Si pudiera hablar español tan bien como usted habla inglés, estaría muy feliz. (You should have seen what that looked like before I applied Google Translate :) Alan -- https://mail.python.org/mailman/listinfo/python-list
Re: Plumbing behind super()
Thanks for the replies from everybody. It looks like I should double check super_init and see what truck is coming from that which will hit me with a gotcha later. I'm very naively right now plucking the class from my locals and I was able to proceed in the very, very short term. I think I would have run into something like this earlier but I was doing something else incorrectly with self references in general. I was having my byte code push the object reference on the stack for method calls instead of using a naive one. For example: m.change_a(2) Disregarding unrelated code, it disassembles to this in a 3.6 intepreter: 3 6 LOAD_FAST0 (m) 8 LOAD_ATTR1 (change_a) 10 LOAD_CONST 1 (2) 12 CALL_FUNCTION1 I have been doing an oopsies of trying to push the self reference on the stack for the method. So I'm doing something like: 3 6 LOAD_FAST0 (m) 8 LOAD_ATTR1 (change_a) X LOAD_FAST0 (m) 10 LOAD_CONST 1 (2) 12 CALL_FUNCTION2 Whoops. Now I need to figure out how the interpreter knows that change_a is a method and knows what self to feed it. I'm assuming that's in the cell variables similar to what super()'s doing as explained here. I haven't implemented cell variables so this is where I'm stuck in a sand pit. -- https://mail.python.org/mailman/listinfo/python-list
Re: change spacing to two instead of four with pep8 or flake8?
On 29Jun2019 10:19, Malcolm Greene wrote: I've also taken to having my files auto-formatted with yapf on save ... @Cameron: Have you considered black at all and if so, what are your thoughts? I did consider black. Its core selling point was its total inflexibility. Use this and stop _caring_ about the formatting: it will be PEP8 compliant and readable. It is like "go fmt" in that regard (Which I use with Go, when I use Go - you can see that in the "format" script I posted). The target here is: all code uses the same style, the chosen style is good (readable, reasonable, acceptable to most people), let's just use it and spent our time worrying about coding instead of style. However, I want flexibility. Like the OP, I use 2 spece indentation and have my own foibles. Here is the .style.yapf file I use in my personal code: [style] based_on_style = pep8 blank_line_before_module_docstring = True blank_line_before_nested_class_or_def = True blank_lines_around_top_level_definition = 1 dedent_closing_brackets = True #indent_dictionary_value = True indent_width = 2 split_before_expression_after_opening_paren = True split_before_first_argument = True split_complex_comprehension = True space_between_ending_comma_and_closing_bracket = False split_before_dot = true use_tabs = False This comes remarkably close to the style I was hand applying before biting the bullet and putting in the work required to get my editor to autoformat on save. [ Aside: another nice thing about autoformatting, over the "nice!" feeling one gets seeing the code shuffle around in front of one's face, is that if there's a syntax error the code _doesn't_ shuffle around and one can go "whoops" and eyeball the code more closely. ] Cheers, Cameron Simpson -- https://mail.python.org/mailman/listinfo/python-list
Re: Plumbing behind super()
On 30/06/2019 01:15, [email protected] wrote: Whoops. Now I need to figure out how the interpreter knows that change_a is a method and knows what self to feed it. I'm assuming that's in the cell variables similar to what super()'s doing as explained here. I haven't implemented cell variables so this is where I'm stuck in a sand pit. Look up descriptors. Actually, carefully (re)read all of https://docs.python.org/3/reference/datamodel.html -- https://mail.python.org/mailman/listinfo/python-list
Re: Use global, or not
On 29/06/19 11:42 PM, Cecil Westerhof wrote:
DL Neil writes:
On 29/06/19 1:44 AM, Cecil Westerhof wrote:
I have written a GUI program where I have quit a few global variables.
I did not like this, so I now use one global dict. Something like:
global global_dict
...
Is that an acceptable way to do this?
If it works, isn't that the largest part of "acceptable"?
Depends on your way of thinking. ;-)
You detected that I had my tongue firmly stuck in my cheek!
I think it is always best when you write in a language to do it in
that languages way. In this case Python.
I also think this is a program that could be interesting to share.
Then it becomes even more important to do it the Python way.
And my experience is that when asking this type of question you often
get a tip that helps you make your program better.
Here's hoping that you have/will!
Speaking personally, I avoid using "global". Earlier, the topic of
"namespaces" was broached. The danger of nesting one namespace inside
another is that the local scope may (accidentally) override THE name, or
that one forgets the "global variable_name" line-of-code. Added to
which, are OOP virtues which talk about separation and encapsulation, etc...
The pythonic way includes contentions:
1 There should be one-- and preferably only one --obvious way to do it.
- although you might not like the next line, until it is understood to
be a joke at Guido's expense...
2 "we are all adults here".
In each case, (previously discussed and/or below) you are taking
advantage of "namespaces" to keep one 'lot' of values separated and
distinct from others - as well as finding a means of 'passing them
around'. The other half of the considerations is how the values are
being retrieved/utilised within the mainline code.
An alternative might be to use a class. Then accessing a single value
becomes instance.attribute, but when you have need for several
attribute's values or to compute a value from several attributes, an
instance.method() may simplify things.
That was the other possibility I was thinking about. And that would be
maybe better. Because I now do things like:
global_dict['messages']['created'].format(len(filepathArr))
much better would be:
instance.created(len(filepathArr))
+1
Scanning my projects directory, there seems to be quite a mix of classes
and modules to handle this situation. ("one way"..."obvious"???)
The last example might recommend an investigation of "property". I'm
slightly reluctant to suggest this to someone coming from another, more
structured, language, eg Java, because it is so easy to make a
literal-translation between languages and (as you have said) miss a
pythonic advantage, eg direct access to instance.attribute (cf the need
to code "getters", such as "instance.get_attribute()").
Most (Python) "property" tutorials seem to center on attempts to
reproduce "private variables". (you can't really/easily and (again)
aren't we "all adults...") However, I like using @property whenever
there is some computation-required involving attributes, eg
class name():
...
@property
def filepath_length( self ):
return len( filepath_arr )
...
instance.filepath_length
Again, speaking personally, the advantages relate to readability and
separation of concerns (I'd like the answer delivered, not the need to
compute something which the object should do as part of 'itself' - IMHO)
Warning: there's a few statements and opinions 'here' which others may
care to argue - and like you I'm happy to (hope to) learn something
useful...
Refs:
The "Zen of Python": start python and import this
https://docs.python.org/3.6/howto/descriptor.html?highlight=property
https://docs.python.org/3.6/library/functions.html?highlight=property#property
https://docs.python.org/3.6/tutorial/classes.html
--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list
Re: Use global, or not
On 06/29/2019 05:42 AM, Cecil Westerhof wrote: > That was the other possibility I was thinking about. And that would be > maybe better. Because I now do things like: > global_dict['messages']['created'].format(len(filepathArr)) > > much better would be: > instance.created(len(filepathArr)) Sure. But if you put all your globals in a class, you're most likely only ever going to instantiate it once, so it's really a singleton. And the standard Python implementation of a singleton is usually a module. So just create a file called globals.py and place all your globals in there. Then import it into each module that needs it and refer to it as globals.whatever. This is what I do and I've seen others do this also. So it must be right! haha. -- https://mail.python.org/mailman/listinfo/python-list
Re: How do you insert an item into a dictionary (in python 3.7.2)?
On 06/28/2019 09:06 AM, CrazyVideoGamez wrote:
> How do you insert an item into a dictionary? For example, I make a dictionary
> called "dictionary".
>
> dictionary = {1: 'value1', 2: 'value3'}
>
> What if I wanted to add a value2 in the middle of value1 and value3?
How about:
dict[1.5] = 'value2'
In seriousness, though, it looks like you really want to be using a list
instead of a dict. You can insert things into a list using the "insert"
method of the list.
a = ['value1', 'value3']
print (a[0])
a.insert(1,'value2')
print (a)
--
https://mail.python.org/mailman/listinfo/python-list
