Timezone and ISO8601 struggles with datetime and xml.utils.iso8601.parse
Hello, I am trying to convert a local time into UTC ISO8601, then parse it back into local time. I tried the following: -- #!/usr/bin/python import time import datetime import xml.utils.iso8601 year = 2005 month = 7 day= 22 hour = 10 # This is localtime minute = 30 mydatetime = datetime.datetime(year, month, day, hour, minute) strtime= mydatetime.isoformat() print "Time: " + strtime # Localtime too mytimestamp = xml.utils.iso8601.parse(strtime) -- How can I convert this into UTC? Commonsense would have me guess that the date is converted into UTC on construction of the datetime object, hovever, this doesn't seem to be the case. I also found the astimezone(tz) method, but where can I obtain the concrete tz object? The second problem has to do with the ISO8601 parser, which raises the following error: -- Traceback (most recent call last): File "./timetest.py", line 16, in ? mytimestamp = xml.utils.iso8601.parse(strtime) File "/usr/lib/python2.4/site-packages/_xmlplus/utils/iso8601.py", line 22, in parse raise ValueError, "unknown or illegal ISO-8601 date format: " + `s` ValueError: unknown or illegal ISO-8601 date format: '2005-07-22T10:30:00' -- Why does it fail to parse the value returned by the datetime object, and how can I create a parseable time from the datetime object? Thanks, -Samuel -- http://mail.python.org/mailman/listinfo/python-list
Re: Timezone and ISO8601 struggles with datetime and xml.utils.iso8601.parse
> Take a look at the utcoffset method of datetime objects. This returns 0. However, meanwhile I figured out a way to do this: Every datetime object by default does not handle timezones at all, and as such "isoformat" does not return an offset in the ISO8601 string. The only way around this appears to be passing the tzinfo to the constructor every time (datetime.tzinfo is not writeable). I am not aware of a python-provided implementation for a conrete tzinfo, so I copied this code: from datetime import * import time as _time STDOFFSET = timedelta(seconds = -_time.timezone) if _time.daylight: DSTOFFSET = timedelta(seconds = -_time.altzone) else: DSTOFFSET = STDOFFSET DSTDIFF = DSTOFFSET - STDOFFSET class LocalTimezone(tzinfo): def utcoffset(self, dt): if self._isdst(dt): return DSTOFFSET else: return STDOFFSET def dst(self, dt): if self._isdst(dt): return DSTDIFF else: return ZERO def tzname(self, dt): return _time.tzname[self._isdst(dt)] def _isdst(self, dt): tt = (dt.year, dt.month, dt.day, dt.hour, dt.minute, dt.second, dt.weekday(), 0, -1) stamp = _time.mktime(tt) tt = _time.localtime(stamp) return tt.tm_isdst > 0 from the Python documentation into my program. (I am sure there must be a better way to do this though.) Then, when passing the tz.LocalTimezone instance to datetime, isoformat() returns the string with an offset appended (e.g. +02:00). The resulting string can then also successfully be parsed with xml.utils.iso8601.parse(). Thanks for your help! -Samuel -- http://mail.python.org/mailman/listinfo/python-list
No newline using printf
Hello, I have been searching for an answer for almost two hours now and have not found an answer. I want this code: for i in range(3): print i # or whatever To produce this output: 012 How can I print a word without appending a newline character? Appending a "," to the print statement only substitutes the newline for a space, which is not what I am looking for. Any hints? Thanks, -Samuel -- http://mail.python.org/mailman/listinfo/python-list
Re: No newline using printf
Thanks for your help, Guys. This works of course. -Samuel -- http://mail.python.org/mailman/listinfo/python-list
Creating database structures in a portable way
Hi, I am looking for a library that takes an XML file that specifies a table structure, and generates the CREATE/DROP/ALTER SQL statements to create the tables in the database. In particular, I am trying to port a PHP application that currently uses the AdoDB XML schema: > http://phplens.com/lens/adodb/docs-datadict.htm#xmlschema Is there a way to do something similar with the DB-API modules in Python? Thanks, -Samuel -- http://mail.python.org/mailman/listinfo/python-list
Re: Creating database structures in a portable way
> FWIW, there's a Python port of adodb: > http://phplens.com/lens/adodb/adodb-py-docs.htm > > and parsing XML in Python is quite easy. So you could as well port the > AdoDB XML to Python too. That is exactly what I am trying to avoid. While implementing the parser might be easy, you have to translate things into different database flavors, caring about a bunch of things that I'd rather not have to think about. > OTOH, there are other - possibly better (YMMV) - DB abstraction layers > in Python, like SQLAlchemy. SQLAlchemy looks pretty good, but unfortunately apparently requires shell access for installation (or at least, I have not found any other solution covered in the docs), which I can not use. I need a solution that can be shipped in a software package, and installed by simply copying it to the server. > And since the above solution requires > (re)writing the xml-parsing part, it might be worth rewriting it so it > knows how to generate SQLAlchemy schemas instead. Rewriting the schema is possible (I only want to keep it separated from the code), so using SQLAlchemy's built in solution for generating tables seems just fine. It's only the installation part. Anyway, I am wondering; Python seems to include database adapters for almost any important database, all with a unified API. Why would you need another database abstraction on top of that? Thanks, -Samuel -- http://mail.python.org/mailman/listinfo/python-list
Re: Creating database structures in a portable way
> > SQLAlchemy looks pretty good, but unfortunately apparently requires > > shell access for installation (or at least, I have not found any other > > solution covered in the docs), which I can not use. > > It doesn't use binaries AFAIK, so just copying should work as well. Indeed, from browsing the package it seems like a pure Python solution. I installed it (but didn't test it yet because I have no code to drive it yet). > > Rewriting the schema is possible (I only want to keep it separated from > > the code), > > Why ? Isn't your code supposed to use it ? It often makes sense to use meta languages to enforce clean separation of logic. In my experience it is helpful to separate the database logic from the application logic. > A first point is that the DB-API doesn't hide the differences between > various SQL dialects. I see. > A second point is that DB-API requires you to > embed SQL statements as strings, while SQLAlchemy allow you to build > your SQL queries in pure Python. I'd rather not rewrite every single Sql statement into Python, since it would 1. replace code that has already proven to work by something untested. 2. add yet another abstraction, at the cost of performance. Luckily, SQLAlchemy also lets me use SQL statements directly. > (and FWIW, if using an existing DB, you > don't even have to describe the schema a second time to use it). Well, > just start playing with it, and I really doubt you'll want to come back > to the embedded hand-written SQL !-) If I ever write something from scratch I'll use it. Thanks for your comments, this was very helpful. -Samuel -- http://mail.python.org/mailman/listinfo/python-list
Instantiating an object when the type is only known at runtime
Hi, I am trying to replace the eval() in the following code: def myfunc(type, table): module = __import__(type) type = 'module' + '.' + type obj = eval(type) return obj(row[table.c.name], row[table.c.handle]) I am out of ideas. Any hints? Thanks, -Samuel -- http://mail.python.org/mailman/listinfo/python-list
Re: Instantiating an object when the type is only known at runtime
Thanks, that's what I was looking for. > >>> m = __import__( "StringIO" ) > >>> x = getattr( m, "StringIO" )() > >>> x > > >>> For the records: If the module is already loaded, this also works: if my_type_is_not_yet_loaded: module = __import__(type) obj= getattr(module, type) else: obj= globals().get(type) resource = obj(my_arg1, my_arg2) Thanks again, -Samuel -- http://mail.python.org/mailman/listinfo/python-list
Running scripts in a sandbox
Hi, I currently have two files: core.py Runs the application, provides APIs to plugins untrusted-plugin.py Executes code using the API from core.py I would like to run the code of untrusted-plugin.py in a sandbox, but still provide it with some hooks to the core API. My environment does not allow me to apply any changes OS-wise (standard web hoster), so I have no way to run the plugin in another process. I found a reference to rexec using Google, but apparently it was removed due to security issues in Python 2.3. Is there antoher way to do this? Thanks, -Samuel -- http://mail.python.org/mailman/listinfo/python-list
Automating a telnet session with an echo to stdout
Hi, I am trying to automate a telnet session (currently using Python's telnetlib) and would like to echo any response of the remote host to stdout, as soon as it arrives on telnetlib's open TCP socket. Currently I print the return value of the read_some() method (or other read* methods) to the screen. However, that way of doing it means that the response gotten from the remote host is printed only periodically. Can anyone point me to how to enable the immediate echo? I know that this is possible using interact(), but I do not want to hand the session over to the user, I just want him to see what's going on. -Samuel -- http://mail.python.org/mailman/listinfo/python-list
Re: Automating a telnet session with an echo to stdout
On Sat, 02 Jun 2007 17:41:01 +, Samuel wrote: > I am trying to automate a telnet session (currently using Python's > telnetlib) and would like to echo any response of the remote host to > stdout, as soon as it arrives on telnetlib's open TCP socket. For the records: Because I did not find any other solution, I modified telnetlib to provide a callback that let's me see what is going on. The patch is here: http://sourceforge.net/tracker/index.php?func=detail&aid=1730959&group_id=5470&atid=305470 -Samuel -- http://mail.python.org/mailman/listinfo/python-list
Re: which "GUI module" you suggest me to use?
On Wed, 06 Jun 2007 00:22:40 +, Grant Edwards wrote: >> I know that WxPython work only under Windows and PyGTK work only under >> Linux... > > You 'know' wrong. > > wxPython works fine under Windows, Linux and OSX. wxPython emulates Gtk (though using some native widgets, it also uses some of its own) and in many cases it looks non-native compared to Gtk. If your target platform includes Unix systems, you'll have to decide whether inconsistencies with the look and feel of the desktop are an issue for you. > PyGTK works under Linux and Windows, but doens't use native widgets > under Windows, so it won't look like a "normal" windows app. Gtk on Win32 can be themed to looked like Windows, AFAIK the Win32 installer does this by default since a couple of months. -Samuel -- http://mail.python.org/mailman/listinfo/python-list
Re: which "GUI module" you suggest me to use?
On Wed, 06 Jun 2007 14:43:35 +, Grant Edwards wrote: > On 2007-06-06, Samuel <[EMAIL PROTECTED]> wrote: >> On Wed, 06 Jun 2007 00:22:40 +, Grant Edwards wrote: >>> wxPython works fine under Windows, Linux and OSX. >> >> wxPython emulates Gtk > > What? On some platforms (Linux), wxPython _uses_ Gtk. I don't see how > you could say it emulates it. That may be true in some cases, but in fact, most widgets show some sort of difference. Take for example the HPaned - it looks totally different (the slider is a lot slimmer, plus moving it makes a line appear. The behavior is different as well). Even simple widgets show differences. Try triple-clicking into entry boxes, it's different from Gtk. >> (though using some native widgets, it also uses some of its own) and in >> many cases it looks non-native compared to Gtk. > > How can that be the case when wxPython is using Gtk? Obviously, it does *not* always use Gtk. >> If your target platform includes Unix systems, you'll have to decide >> whether inconsistencies with the look and feel of the desktop are an >> issue for you. > > wxPython looks completely native on Unix, because it's using a native > widget set (Gtk). Same as above. -Samuel -- http://mail.python.org/mailman/listinfo/python-list
Re: which "GUI module" you suggest me to use?
On Wed, 06 Jun 2007 09:45:48 -0500, Chris Mellon wrote: > On 6/6/07, Samuel <[EMAIL PROTECTED]> wrote: > In the general case, wxWidgets wraps (not emulates) Gtk. I don't believe > that there are any common controls left which are still emulated (maybe > the list control? I'm not sure - I don't follow the bleeding edge of wx > anymore). See my other response. Also, the fact that Gtk widgets are used does not necessarily mean that the behavior is the same - you can still do all kinds of weird stuff by hooking into the callbacks. Some of the differences to Gtk were probably introduced to wx in order to have more consistent behavior over different platforms. > wxPython (as opposed to wxWidgets, the C++ core) has a sizeable library > of custom controls as part of its standard lib. Most of these are > owner-drawn for various reasons and often won't appear native (Andrea > Gavin, probably the most prolific custom control author, works primarily > on Windows). This affects the C++ core as well. aMule (which is written in CPP) is one particular complex example that exposes many of those problems when used on Linux. >> > PyGTK works under Linux and Windows, but doens't use native widgets >> > under Windows, so it won't look like a "normal" windows app. >> >> Gtk on Win32 can be themed to looked like Windows, AFAIK the Win32 >> installer does this by default since a couple of months. >> >> > That stretches the truth rather significantly. While the win32 theme > does use the windows theme apis for drawing, it still has slightly > different colors (especially window backgrounds and menus), and (more > importantly) vastly and notably different behavior. Well, I have only tried Gaim on windows and could not tell the difference. But then, I am not a regular Windows user. > Shortcuts are > different, renderings are different, the Gtk drawing model is used > instead of the windows one (leads to quite jarring repainting > differences), different fonts, etc, etc. It looks okay in a screenshot > but is clearly non-native and foreign in use. Sounds bad. It also sounds much like what I experienced with wx on Linux. I guess there is no perfect solution, you always have to target one primary platform. (Well, there's SWT, but the Gtk emulation too has it's drawbacks...) -Samuel -- http://mail.python.org/mailman/listinfo/python-list
Trivial string substitution/parser
Hi, How would you implement a simple parser for the following string: --- In this string $variable1 is substituted, while \$variable2 is not. --- I know how to write a parser, but I am looking for an elegant (and lazy) way. Any idea? -Samuel -- http://mail.python.org/mailman/listinfo/python-list
Re: Trivial string substitution/parser
On Sun, 17 Jun 2007 11:00:58 +, Duncan Booth wrote: > The elegant and lazy way would be to change your specification so that $ > characters are escaped by $$ not by backslashes. Then you can write: > >>>> from string import Template >>>> ... Thanks, however, turns out my specification of the problem was incomplete: In addition, the variable names are not known at compilation time. I just did it that way, this looks fairly easy already: --- import re def variable_sub_cb(match): prepend = match.group(1) varname = match.group(2) value = get_variable(varname) return prepend + value string_re = re.compile(r'(^|[^\\])\$([a-z][\w_]+\b)', re.I) input = r'In this string $variable1 is substituted,' input += 'while \$variable2 is not.' print string_re.sub(variable_sub_cb, input) --- -Samuel -- http://mail.python.org/mailman/listinfo/python-list
Retrieving a stacktrace from an exception object / forwarding an exception
Hi,
I am trying to wrap a function that throws an exeption in such a way
that the stacktrace is logged into a file and the exception is
forwarded after that. For example:
---
def my_func():
raise Exception("hello")
def wrapper():
try:
my_func()
except Exception, e:
f = open("logfile", 'a')
f.write(e.stacktrace())
raise e
wrapper() # should throw the exception with a stacktrace showing
my_func()
---
Any idea if and how this can be done?
-Samuel
--
http://mail.python.org/mailman/listinfo/python-list
Re: Retrieving a stacktrace from an exception object / forwarding an exception
Thanks, Guys, this works as expected. -Samuel -- http://mail.python.org/mailman/listinfo/python-list
ANNOUNCE: Exscript 0.9 (Initial Release)
Introduction - Exscript is a scripting language for automating Telnet or SSH sessions. It supports a wide range of features, such as parallelization, AAA authentication methods, TACACS, and a very simple template language. Exscript itself is written in Python, and it should run on any platform where Python supports OS threads. This is the initial public release. Dependencies - Python 2.2 or better Python Crypto module Download Exscript -- Release: http://exscript.googlecode.com/files/exscript-0.9.tgz SVN instructions: http://code.google.com/p/exscript/source Links -- Exscript project page: http://code.google.com/p/exscript/ Mailing list: http://groups.google.com/group/exscript Bug tracker: http://code.google.com/p/exscript/issues/list Browse the source: http://exscript.googlecode.com/svn/trunk/ -- http://mail.python.org/mailman/listinfo/python-list
Debugging "broken pipe" (in telnetlib)
Hi, When using telnetlib, the connection sometimes breaks with the following error: "error: (32, 'Broken pipe')" where the traceback points to self.sock.send(buffer) in telnetlib.py. The problem is unreproducible, but happens fairly often (approx. 5% of the time). Any idea how to debug such a problem? How can I find out what broke the pipe? -Samuel -- http://mail.python.org/mailman/listinfo/python-list
Re: Debugging "broken pipe" (in telnetlib)
On Jul 3, 3:03 pm, Jean-Paul Calderone <[EMAIL PROTECTED]> wrote: > EPIPE results when writing to a socket for which writing has been shutdown. > This most commonly occurs when the socket has closed. You need to handle > this exception, since you can't absolutely prevent the socket from being > closed. The exception is already caught and logged, but this is really not good enough. By "handling this exception", do you mean that there is a way to handle it such that the connection still works? I found some code that attempts to retry when SIGPIPE was received, but this only results in the same error all over again. Why can this not be prevented (in the general case)? Unless something fancy happened, what can cause the socket to close? Looking at the raw data received by the connected host, the connection gets lost in mid- stream; I can not see anything that might cause the remote side to close the connection (in which case I'd expect a "connection reset by peer" or something). > There might be some other change which would be appropriate, though, > if it is the case that something your application is doing is causing the > socket to be closed (for example, sending a message which the remote side > decides is invalid and causing it to close the socket explicitly from its > end). The program is doing the same thing repeatedly and it works 95% of the time, so I am fairly sure that nothing special is sent. > It's difficult to make any specific suggestions in that area without > knowing exactly what your program does. Unfortunately the application is rather complex and a simple test case is not possible. Basically, it creates a number of daemon threads, each of which creates a (thread local, non-shared) instance of telnetlib and connects to a remote host. Are there any special conditions that must be taken care of when opening a number of sockets in threads? (The code runs on AIX 4.1, where Python supports native OS threads.) -Samuel -- http://mail.python.org/mailman/listinfo/python-list
Re: Debugging "broken pipe" (in telnetlib)
Thanks for your comments, Jean-Paul. -- http://mail.python.org/mailman/listinfo/python-list
Components for a client/server architecture
Hi, I am looking for some recommendations for client/server technologies and for the communication involved. I am currently planning to port a Perl application that has grown out of proportion to another language and architecture. For this purpose, I am investigating technologies that best fit the requirements. The current plan for the architecture of the ported software includes: - A client/server model, where the server provides a service for configuring network devices and the client provides a user interface that runs on Unix. - Java (and possibly Jython) or Mono/C# (or possibly IronPython) on the server. Requirements are: A strong and fair threading model. This is actually what drove me away from Perl and what essentially prevents using a normal Python interpreter on the server. I don't know whether the GIL and starvation issues also apply to IronPython or Jython. Sharing non thread-aware objects of arbitrary type between threads without any restriction is absolutely necessary (this is not possible in Perl). - Perl or Python on the client side. Clients have to provide a UI (a web application is not an option). Unfortunately, I have very little experience with client/server architectures and the protocols involved, so I would like to collect your recommendations as a starting point for further research: - Are there any standard protocols that make it easy to share objects between Java/Jython/IronPython and Python or Perl over the network? I am thinking of a portable, language independent object (de-)serialization, if such a thing exists. Having a defined protocol/interface between the client and the server that makes it easy to switch technologies on either side of the architecture is a strong requirement. - How does bi-directional communication in such a protocol work? I am assuming that the client should not poll the server for callbacks, OTOH, leaving a TCP connection open also does not seem right. - What is the security model of that protocol? If my description of the project is inaccurate or too incomplete I apologize; the problem is that I still find it hard to tell which requirements actually matter. If you have any pointers that might be of interest for such an architecture, please let me know. -Samuel -- http://mail.python.org/mailman/listinfo/python-list
Re: Components for a client/server architecture
On Mon, 21 May 2007 12:06:50 +0200, Diez B. Roggisch wrote: > I'm not sure which configuration you want to change how often. But I'm > not convinced that the python threading limitations really do make a > difference here. Do you really benefit from multi-core capabilities in > this scenario? The threading issues are not bound to multi cpu systems. The problem is that some of Python's blocking functions require holding the global lock. "Not all built-in functions that may block waiting for I/O allow other threads to run." "It is not possible to interrupt the acquire() method on a lock" http://docs.python.org/lib/module-thread.html I also found that IronPython does not have a global lock, so far it seems well suited for solving the problems I am trying to avoid. I am still looking for a good comparison between IronPython, Python, and Jython. > Sounds like CORBA to me. CORBA has a very mature and good implementation > for Python called OmniORB, and interoperability with other orbs (the > ones available for e.g. Java) is very good - as CORBA as standard is > mature. I have worked with Bonobo (an implementation of CORBA) before, though not on the network - it is fairly complex. But I did not think of using it for this purpose, it might actually make sense. I'll have to look into the transport protocol more. > And from what I know about SOAP it's certainly not better > suited (besides the fact that it sucks big time anyway) SOAP seems well suited for web services. But it comes with quite some overhead, I tend to say that it's not a perfect fit for our purpose. Thank you for your comment! -Samuel -- http://mail.python.org/mailman/listinfo/python-list
Re: Components for a client/server architecture
On May 22, 3:10 am, Josiah Carlson <[EMAIL PROTECTED]> wrote: > That snippet of code shows that acquiring a lock does release the GIL. Of course, that does not mean that the (possible) issues no longer apply. However, I decided to hack up a quick prototype and see how it goes. If it doesn't work it shouldn't be too hard to switch to IronPython. > You should also consider XML-RPC. Setting up and using XML-RPC in > Python is quite easy (see the recipes in the Python cookbook), both as a > server and client. It's nice and simple, but more in the SOAP domain of things. As for CORBA, I had a closer look at Omniorb and it looks powerful, pretty much exactly what I was looking for. Thanks! -Samuel -- http://mail.python.org/mailman/listinfo/python-list
Simple omniORBpy example throws exception with error 0x41540002
Hi, I am trying to get the files from this tutorial to work: http://www.grisby.org/presentations/py10code.html Direct link to the files: http://www.grisby.org/presentations/py10code/adder.idl http://www.grisby.org/presentations/py10code/adderServer.py It produces the following error: $ omniidl -bpython adder.idl && python adderServer.py Traceback (most recent call last): File "adderServer.py", line 23, in nameRoot = nameRoot._narrow(CosNaming.NamingContext) File "/usr/lib/python2.5/site-packages/omniORB/CORBA.py", line 667, in _narrow return _omnipy.narrow(self, dest._NP_RepositoryId) omniORB.CORBA.TRANSIENT: Minor: 0x41540002, COMPLETED_NO. According to Google this might mean "Connect failed", however, I don't understand why the server would open a connection. I expected it to simply listen on a socket, but I probably just don't understand how it works. Can anyone please explain this? -Samuel -- http://mail.python.org/mailman/listinfo/python-list
omniORBpy: Error 0x41540002
Hi, I am trying to get the files from this tutorial to work: http://www.grisby.org/presentations/py10code.html Direct link to the files: http://www.grisby.org/presentations/py10code/adder.idl http://www.grisby.org/presentations/py10code/adderServer.py It produces the following error: $ omniidl -bpython adder.idl && python adderServer.py Traceback (most recent call last): File "adderServer.py", line 23, in nameRoot = nameRoot._narrow(CosNaming.NamingContext) File "/usr/lib/python2.5/site-packages/omniORB/CORBA.py", line 667, in _narrow return _omnipy.narrow(self, dest._NP_RepositoryId) omniORB.CORBA.TRANSIENT: Minor: 0x41540002, COMPLETED_NO. According to Google this might mean "Connect failed", however, I don't understand why the server would open a connection. I expected it to simply listen on a socket, but I probably just don't understand how it works. Can anyone please explain this? -Samuel -- http://mail.python.org/mailman/listinfo/python-list
Re: Simple omniORBpy example throws exception with error 0x41540002
On May 22, 1:54 pm, "Diez B. Roggisch" <[EMAIL PROTECTED]> wrote: > It indeed does open a connection - because it wants to register with a > NameServer. Ah, I see now how this works. I happen to run Ubuntu here, so I tried the following: - sudo apt-get install orbit-name-server-2 - orbit-name-server-2 & - Add to /etc/omniORB4.cfg: InitRef = NameService=IOR:01002b00... (where everything starting from "IOR:" is the output given by orbit- name-server-2. However, this does not seem to change the behavior. Any hints? Thanks, -Samuel -- http://mail.python.org/mailman/listinfo/python-list
Re: omniORBpy: Error 0x41540002
On May 22, 2:53 pm, "Diez B. Roggisch" <[EMAIL PROTECTED]> wrote: > Please see my answer to your first post. Gaa, Google's web client reported an error where there was none. Sorry about the repost. -Samuel -- http://mail.python.org/mailman/listinfo/python-list
Re: Simple omniORBpy example throws exception with error 0x41540002
On May 22, 1:54 pm, "Diez B. Roggisch" <[EMAIL PROTECTED]> wrote: > It indeed does open a connection - because it wants to register with a > NameServer. Ah, I see now how this works. I happen to run Ubuntu here, so I tried the following: - sudo apt-get install orbit-name-server-2 - orbit-name-server-2 & - Add to /etc/omniORB4.cfg: InitRef = NameService=IOR:01002b00... (where everything starting from "IOR:" is the output given by orbit- name-server-2. However, this does not seem to change the behavior. Any hints? Thanks, -Samuel -- http://mail.python.org/mailman/listinfo/python-list
Re: Simple omniORBpy example throws exception with error 0x41540002
On May 22, 6:53 pm, Duncan Grisby <[EMAIL PROTECTED]> wrote: > I think ORBit is configured to only listen on its proprietary Unix > domain socket protocol by default, not TCP, so omniORB doesn't know > how to talk to it. You should either tell ORBit to listen on TCP > (Google for how), or use omniORB's naming service, which listens on > TCP by default. Yay, after replacing orbit2-nameserver by omniorb4-nameserver everything works fine. Thanks a lot for your comment! -Samuel -- http://mail.python.org/mailman/listinfo/python-list
Re: PyGTK and HTML rendering?
On Sat, 26 May 2007 23:23:19 +0200, Ivan Voras wrote: > Is there an easy off-the-shelf way to get HTML formatting inside the > TextArea widget? Gtk does not support this currently, but they would love to see this feature added into Gtk: > http://bugzilla.gnome.org/show_bug.cgi?id=59390 It shouldn't be too hard to do, sounds like a nice project? :-) -Samuel -- http://mail.python.org/mailman/listinfo/python-list
ANNOUNCE: Spiff Workflow 0.0.1 (Initial Release)
Introduction Spiff Workflow is a library implementing a framework for workflows. It is based on http://www.workflowpatterns.com and implemented in pure Python. Supported workflow patterns include (at least) the following: 1. Sequence 2. Parallel Split 3. Synchronization 4. Exclusive Choice 5. Simple Merge 6. Multi-Choice 7. Structured Synchronizing Merge 8. Multi-Merge 9. Structured Discriminator 10. Arbitrary Cycles Spiff Workflow is part of the Spiff platform, which aims to produce a number of generic libraries generally needed in enterprise applications. This release is the initial release and SHOULD NOT BE USED IN PRODUCTION ENVIRONMENTS - this release is meant for development only. Spiff Workflow is free software and distributed under the GNU LGPLv2.1. Dependencies - Python >= 2.4 Download - The release page is here: http://pypi.python.org/pypi/Spiff%20Workflow/0.0.1 You can also check the code out of SVN: svn checkout http://spiff.googlecode.com/svn/trunk/libs/Workflow/ Links: --- Spiff project page: http://code.google.com/p/spiff/ Mailing list: http://groups.google.com/group/spiff-devel Bug tracker: http://code.google.com/p/spiff/issues/list Documentation: http://spiff.googlecode.com/svn/trunk/libs/Workflow/README Browse the source: http://spiff.googlecode.com/svn/trunk/libs/Workflow/ If you have any questions, please do not hesitate to ask. -Samuel -- http://mail.python.org/mailman/listinfo/python-list
Re: python video editing libs
On Sat, 18 Aug 2007 12:55:47 +1200, DavidM wrote: > Does anything like this exist? GStreamer/GNonLin might possibly support what you are looking for, though I have not tested this. http://docs.programmers.ch/index.php/Using_Gnonlin_with_GStreamer_and_Python -Samuel -- http://mail.python.org/mailman/listinfo/python-list
Re: Hot subject: a good python editor and/or IDE?
On Sun, 19 Aug 2007 11:47:03 +0200, Sébastien wrote: > Hi folks, > > I am currently using Eclipse+PyDev when developping Python projects but > I lack a fast, simple editor for tiny bit of scripts. So here is my > question: what is, for you, the current best ( but still kind of light! > ) Python editor/IDE ? A tiny precision, I am on Ubuntu so I am looking > for a linux compatible editor. Vim with SnippetsEMU works great with Python. I made a demo of this in action here: http://debain.org/?p=198 Installation/configuration example on Ubuntu: --- $ sudo apt-get install vim $ mkdir -p $HOME/.vim/ftplugin/ $ mkdir -p $HOME/.vim/after/ftplugin/ $ wget http://www.vim.org/scripts/download_script.php?src_id=6951 -O se.vba $ vim se.vba :so % :wq $ echo "setlocal sw=4 setlocal ts=4 noremap py o/**/ " >> ~/.vim/ftplugin/python.vim $ wget http://code.google.com/p/snippetsemu/issues/attachment?aid=-6063627743376712928&name=python_snippets.vim $ cp python_snippets.vim $HOME/.vim/after/ftplugin/ $ echo "syntax on set sw=2 set ts=2 set nu set nuw=3 set autoindent set expandtab" >> $HOME/.vimrc --- (not tested, but it should work) -Samuel -- http://mail.python.org/mailman/listinfo/python-list
Re: Hot subject: a good python editor and/or IDE?
On Sun, 19 Aug 2007 13:08:35 +, Samuel wrote: > $ sudo apt-get install vim I just realized, this should be $ sudo apt-get install vim-python or $ sudo apt-get install vim-full -Samuel -- http://mail.python.org/mailman/listinfo/python-list
ANNOUNCE: Spiff Workflow 0.0.2
Introduction Spiff Workflow is a library implementing a framework for workflows. It is based on http://www.workflowpatterns.com and implemented in pure Python. Spiff Workflow is part of the Spiff platform, which aims to produce a number of generic libraries generally needed in enterprise applications. This release SHOULD NOT BE USED IN PRODUCTION ENVIRONMENTS - it is meant for development only. Spiff Workflow is free software and distributed under the GNU LGPLv2.1. New In This Release --- This release is a huge step forward and an essential rewrite that comes with many new features, bugfixes and improvements. The API is now even simpler to use, and support for the following workflow patterns was added (check the Workflow Patterns page for what they mean): 11. Implicit Termination 12. Multiple Instances without Synchronization 13. Multiple Instances with a Priori Design-Time Knowledge 14. Multiple Instances with a Priori Run-Time Knowledge 15. Multiple Instances without a Priori Run-Time Knowledge 23. Transient Trigger 28. Blocking Discriminator 29. Cancelling Discriminator 30. Structured Partial Join 31. Blocking Partial Join 32. Cancelling Partial Join 33. Generalized AND-Join 37. Acyclic Synchronizing Merge 38. General Synchronizing Merge 41. Thread Merge 42. Thread Split Spiff Workflow now also supports persistence using Python's pickle module. The unit test suite was extremely improved and extended - every pattern now comes with a test. The full ChangeLog is here: http://spiff.googlecode.com/svn/trunk/libs/Workflow/ChangeLog Dependencies - Python >= 2.4 Download - Please check out the code from SVN: svn checkout http://spiff.googlecode.com/svn/trunk/libs/Workflow/ You can also get a tarball from the release page: http://pypi.python.org/pypi/Spiff%20Workflow/ Links: --- Example XML: http://spiff.googlecode.com/svn/trunk/libs/Workflow/tests/xml/patterns/ Spiff project page: http://code.google.com/p/spiff/ Mailing list: http://groups.google.com/group/spiff-devel Bug tracker: http://code.google.com/p/spiff/issues/list Documentation: http://spiff.googlecode.com/svn/trunk/libs/Workflow/README Browse the source: http://spiff.googlecode.com/svn/trunk/libs/Workflow/ If you have any questions, please do not hesitate to ask. -Samuel -- http://mail.python.org/mailman/listinfo/python-list
Python object <-> XML
Hi, Say you have the following XML: item 1 item 2 my group Is there an easy way (i.e. without writing a sax/dom parser) to load this into a (number of) Python object(s), manipulate the instance, and save the result back to XML? -Samuel -- http://mail.python.org/mailman/listinfo/python-list
Re: Python object <-> XML
On Sep 3, 10:19 pm, Jarek Zgoda <[EMAIL PROTECTED]> wrote:
> > Is there an easy way (i.e. without writing a sax/dom parser) to load
> > this into a (number of) Python object(s), manipulate the instance, and
> > save the result back to XML?
>
> Yea, use ElementTree and you'd get a bunch of nested lists of very
> simple objects.
Sorry for being unclear. By "load this into a number of Python
objects" I mean, filling already existing objects with data. In other
words:
class Item(object):
def __init__(self, ref, name):
self.ref = ref
self.name = name
class Group(object):
def __init__(self, ref, name, item = []):
self.ref = ref
self.name = name
self.item = item
mapper = Mapper()
objects = mapper.load('data.xml')
print objects['1']
(Obviously, in my example the mapper could not always know where a
list is required, but an existing mapper will surely have a solution
implemented.)
I guess what I am looking for is pretty much an ORM that also works
with XML.
-Samuel
--
http://mail.python.org/mailman/listinfo/python-list
ImportError depending on the calling module
Hi, Given the following directory structure: - |-- Obj.py |-- __init__.py |-- foo | |-- FooTest.py | `-- __init__.py `-- start1.py - With the following content: - $ cat Obj.py class Obj(object): pass $ cat __init__.py import Obj __all__ = ['Obj'] $ cat foo/FooTest.py from Obj import Obj class FooTest(object): pass $ cat foo/__init__.py from FooTest import FooTest __all__ = ['FooTest'] $ cat start1.py from foo import FooTest x = FooTest() print "Works!" [EMAIL PROTECTED]:~/test$ - The test works: - $ python start1.py Works! $ - However, if the module is imported elsewhere, e.g. in the following directory structure: - test |-- Obj.py |-- __init__.py |-- foo | |-- FooTest.py | `-- __init__.py `-- start1.py test2 `-- start2.py - Where start2.py has the following content: - $ cd test2 $ cat start2.py import sys, os.path sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), '..'))) import test import test.foo print 'Works!' - It does not work: - $ python start2.py Traceback (most recent call last): File "start2.py", line 4, in import test.foo File "/home/sab/test/foo/__init__.py", line 1, in from FooTest import FooTest File "/home/sab/test/foo/FooTest.py", line 1, in from Obj import Obj ImportError: No module named Obj - How would you import the foo module correctly? -Samuel -- http://mail.python.org/mailman/listinfo/python-list
Re: ImportError depending on the calling module
On Sep 6, 5:44 pm, Gabriel Genellina <[EMAIL PROTECTED]> wrote: > On 6 sep, 08:47, Samuel <[EMAIL PROTECTED]> wrote: > > Given the following directory structure: > > > - > > |-- Obj.py > > |-- __init__.py > > |-- foo > > | |-- FooTest.py > > | `-- __init__.py > > `-- start1.py > > - > > The container looks like a package (you didn't tell us the name). > Should be placed somewhere accessible thru sys.path But it is. I added the path to sys.path. > They should import the package as any other > client code, without doing any import tricks nor playing with > sys.path. Why does it matter whether I install it in sys.path or whether sys.path is modified? What's the difference? What I am doing is I ship two modules in one tarball with my software. The software is only unpacked and ran. It has to work without installation. -Samuel -- http://mail.python.org/mailman/listinfo/python-list
Re: ImportError depending on the calling module
On Sep 6, 6:13 pm, Samuel <[EMAIL PROTECTED]> wrote: > Why does it matter whether I install it in sys.path or whether > sys.path is modified? What's the difference? > > What I am doing is I ship two modules in one tarball with my software. > The software is only unpacked and ran. It has to work without > installation. I see now how this works; when a script is ran, only the top-level of the current module is searched. Now that's just ugly, but well... at least I now know how to work around this. Thanks, -Samuel -- http://mail.python.org/mailman/listinfo/python-list
Re: ImportError depending on the calling module
On Sep 6, 6:44 pm, Gabriel Genellina <[EMAIL PROTECTED]> wrote: > On 6 sep, 13:13, Samuel <[EMAIL PROTECTED]> wrote: > > > On Sep 6, 5:44 pm, Gabriel Genellina <[EMAIL PROTECTED]> wrote: > > > They should import the package as any other > > > client code, without doing any import tricks nor playing with > > > sys.path. > > > Why does it matter whether I install it in sys.path or whether > > sys.path is modified? What's the difference? > > Because it's more simple, less error prone, easier to test, more > efficient... > > > What I am doing is I ship two modules in one tarball with my software. > > The software is only unpacked and ran. It has to work without > > installation. > > That's fine, and a good requirement. Place start2.py inside the > container directory, and your package beneath it. Then you don't have > to modify sys.path to find the package - "import packagename" just > works. And it still works if the user installs the package (into site- > packages, by example). > > start1.py > start2.py > packagename/ > |-- Obj.py > |-- __init__.py > |-- foo/ > |-- FooTest.py > `-- __init__.py > > Packages are... well, packages, boxes. Don't try to import a module > inside a package as it were a standalone module But the start2.py script *is* part of the library. It should be in the package. However, I just found that Python 2.5 introduces relative imports. This sounds like an attempt to remedy the situation. -Samuel -- http://mail.python.org/mailman/listinfo/python-list
ANNOUNCE: Exscript 0.9.8
Introduction - Exscript is a scripting language for automating Telnet or SSH sessions. It supports a wide range of features, such as parallelization, AAA authentication methods, TACACS, and a very simple template language. Python developers can easily extend Exscript templates using standard Python modules. This release comes with many new features, and the documentation was also greatly improved. Please refer to the project page for documentation. Release 0.9.8 is now feature complete and expected to be the last release before 1.0 stable. New since Exscript 0.9 --- * Full support for SSH was added. * Greatly improved the prompt detection. This should now work by default with Unix, IOS, JunOS and many more. * Support for assigning variables in the Exscript template. * URL-style hostnames are now supported. * The exscript interpreter now checks regular expressions for correctness at compiled time, not only at runtime. * The "loop" statement now supports iterating over multiple lists in one run. * A deadlock when using TACACS authentication was fixed. * Character noise from telnet control characters is now properly filtered. * Exscript no longer deadlocks if an error occurs while writing to a logfile. * Performance was improved when many hosts are loaded from a text file. Dependencies - Python 2.2 or better Python Crypto module Python pexpect module For SSH support, the "ssh" command line utility (OpenSSH). Download Exscript -- Release: http://exscript.googlecode.com/files/exscript-0.9.8.tgz SVN instructions: http://code.google.com/p/exscript/source Links -- Exscript project page: http://code.google.com/p/exscript/ Mailing list: http://groups.google.com/group/exscript Bug tracker: http://code.google.com/p/exscript/issues/list Browse the source: http://exscript.googlecode.com/svn/trunk/ -- http://mail.python.org/mailman/listinfo/python-list
CGI handler: Retrieving POST and GET at the same time
Hi, I have the following form: and would like to retrieve both fields, "one" and "two". However, the following does not work: form_data = cgi.FieldStorage() for key in form_data: print key + ":", form_data[key].value It prints out: two: 2 How can I retrieve the GET variables in that context? -Samuel -- http://mail.python.org/mailman/listinfo/python-list
Re: CGI handler: Retrieving POST and GET at the same time
On Mar 10, 8:45 pm, "Pierre Quentel" <[EMAIL PROTECTED]> wrote: > To get the key-value dictionary : > > cgi.parse_qs(os.environ["QUERY_STRING"]) Thanks a lot, it works! -Samuel -- http://mail.python.org/mailman/listinfo/python-list
Converting a list to a dictionary
Hi,
is there a short version for this?
res_dict = {}
for resource in res_list:
res_dict[resource.get_id()] = resource
This does not work:
res_dict = dict([r.get_id(), r for r in res_list])
-Samuel
--
http://mail.python.org/mailman/listinfo/python-list
Re: Converting a list to a dictionary
On Mar 14, 9:52 pm, Bruno Desthuilliers <[EMAIL PROTECTED]> wrote: > res_dict = dict((r.get_id(), r) for r in res_list) > > or if you have to be compatible with older python versions: > > res_dict = dict([(r.get_id(), r) for r in res_list]) Yep, that works. Strange, I was sure I had tested the latter, but I must have been doing something wrong. Thanks for your help! -Samuel -- http://mail.python.org/mailman/listinfo/python-list
Re: Converting a list to a dictionary
On Mar 14, 9:32 pm, "Drew" <[EMAIL PROTECTED]> wrote: > I'm using Python2.5 and it seems that this only gives me a hash with > the first id and first record. Am I doing something wrong? Try this instead: >>> class Person(): ... def __init__(self): ... self.id = 5 ... >>> mylist = [] >>> for i in range(100): ... p = Person() ... p.id = i ... mylist.append(p) ... >>> mydict = dict((r.id,r) for r in mylist) >>> mydict What this does is it maps the id to the object. In your case, you only have one id. -Samuel -- http://mail.python.org/mailman/listinfo/python-list
Re: Converting a list to a dictionary
On Mar 14, 9:48 pm, "Drew" <[EMAIL PROTECTED]> wrote: > This is interesting behavior, but may not be what the original poster > intended. I am the original poster :). > If I understand correctly, this means that if more than one > object shares the same id, only one copy will be created in the dict. > Is this correct? Yes. Dictionaries are just hashes; you can't have the same key twice. Bye, -Sam -- http://mail.python.org/mailman/listinfo/python-list
MIME Magic
Hi, How can I determine the type of a file from "magic bytes", similar to what the "file" command on unix does? I found http://docs.python.org/lib/module-mimetypes.html but this only seems to use the filename (extension) for finding the type. Any hints? -Samuel -- http://mail.python.org/mailman/listinfo/python-list
Re: MIME Magic
On Mar 18, 12:30 am, Jorge Godoy <[EMAIL PROTECTED]> wrote: > I'd start by taking a look at "file"'s code. :-) > > The thing is being able to identify the signatures of several different types > of files. Here's some help for > you:http://www.garykessler.net/library/file_sigs.html > > And here's an example of an authoritative source for that > table:http://www.libpng.org/pub/png/spec/1.2/PNG-Structure.html Actually, I am looking for an existing solution because I do not want to write it from scratch... I hope I am not out of options. -Samuel -- http://mail.python.org/mailman/listinfo/python-list
ANNOUNCE: Exscript 0.9.9
Introduction - Exscript is a scripting language for automating Telnet or SSH sessions. It supports a wide range of features, such as parallelization, AAA authentication methods, TACACS, and a very simple template language. This release comes with many new features, and the documentation was also greatly improved. Please refer to the project page for documentation. A lot of changes were made since release 0.9.8, so 0.9.9 was added before the final 1.0. Hopefully this will be the last release before 1.0 stable. New since 0.9.8 -- * The SSH adapter was completely reworked and should now be on par with the Telnet adapter. * The telnet adapter now supports negotiation. This means that a connection should now automatically choose a script-friendly terminal type. (e.g. one that uses no color control characters). * Exscript is now fully supported on Python >= 2.2. * SSH support is now optional. The protocol adapters are now loaded dynamically, such that SSH does not have to be installed if only Telnet is used. * The error handling of the parser was greatly improved, and more errors should now be detected at compile time. * Prompt recognition was further improved. * It is now possible to use \r and \n in strings. * Strings may now be concatenated using the "." operator. * Added "in" and "not in" operators for searching through a list. * Fixed: Escaping of some characters outside of code context did not work. * Fixed: Escaping quotes in a string would not work. * Added support for using $-prefixed variable names in strings. * Added the builtin variable "response", that now includes the response of the last command. * #-prefixed lines are now comments. * Fixed a bug that some operations would not work when accessing the iterator variable in the body of a loop tag. * Fix: In several cases accessing a variable would fail. * Added support for "while" and "until" loops. * Added support for "into" keyword. * Use friendlier thread names. Dependencies - * Python 2.2 or greater * Python-crypto * Python-pexpect (optional, for SSH support) * ssh (optional, for SSH support) Download Exscript -- Release: http://exscript.googlecode.com/files/exscript-0.9.9.tgz SVN instructions: http://code.google.com/p/exscript/source Links -- Exscript project page: http://code.google.com/p/exscript/ Mailing list: http://groups.google.com/group/exscript Bug tracker: http://code.google.com/p/exscript/issues/list Browse the source: http://exscript.googlecode.com/svn/trunk/ -- http://mail.python.org/mailman/listinfo/python-list
Announcing: Spiff Guard (Generic Access Lists for Python)
Introduction Spiff Guard is a library for implementing access lists in Python. It provides a clean and simple API and was implemented with performance and security in mind. It was originally inspired by phpGACL (http://phpgacl.sourceforge.net/), but features an API that is significantly cleaner and easier to use. Spiff Guard is the first library published as part of the Spiff platform. The Spiff platform aims to produce a number of generic libraries generally needed in enterprise (web) applications. Spiff Guard is free software and distributed under the GNU GPLv2. Dependencies - sqlalchemy (http://www.sqlalchemy.org/) Download - Please check out the code from SVN: svn checkout http://spiff.googlecode.com/svn/trunk/libs/Guard/ Links: --- Spiff project page: http://code.google.com/p/spiff/ Bug tracker: http://code.google.com/p/spiff/issues/list Documentation: http://spiff.googlecode.com/svn/trunk/libs/Guard/README Browse the source: http://spiff.googlecode.com/svn/trunk/libs/Guard/ Any questions, please ask (or file a bug). -Samuel -- http://mail.python.org/mailman/listinfo/python-list
Distutils: Python version information
In Python cheese shop, there is a field that shows the Python version that is required by a listed package. I have not found this covered in the distutils documentation: How do you add the Python version information into the setup script of a package? Unfortunately, the keyword list: http://docs.python.org/dist/meta-data.html is incomplete (for example, the "provides" and "requires" keywords are not listed, even though covered in the previous chapters of the documentation). -Samuel -- http://mail.python.org/mailman/listinfo/python-list
ANNOUNCE: Exscript 0.9.11
Introduction - Exscript is a scripting language for automating Telnet or SSH sessions. It supports a wide range of features, such as parallelization, AAA authentication methods, TACACS, and a very simple template language. Please refer to the project page for updated documentation (see the links at the bottom of this announcement). New since 0.9.10 -- * Support for Huawei routers was added. * It is now possible to use a different authentication and authorization passwords when using AAA. (The -a switch was added.) * It is now possible to skip the authentication procedure all-together. (The -n switch was added.) * The --no-prompt switch was added. * The priority of the "not" operator was decreased. Please refer to the documentation. * Support for regular expression modifiers was added. * The operating system and device detection mechanism was improved. * The default timeout was increased to 30 seconds (previous was 10s). * The "extract" keyword now supports a "from" operator for extracting a string from a variable. * The interpreter now has a --version flag. * Windows newline characters are now supported in exscripts. * The performance of prompt matching was greatly improved. * Cisco router detection was improved. * Fix: The "is not" operator was broken. * Fix: Escaping of "/" characters would not work. * Fix: "while" and "until" loops would sometimes not work. * Fix: Arithmetic operations with 0 no longer fail. Dependencies - * Python 2.2 or greater * Python-crypto * Python-pexpect (optional, for SSH support) * ssh (optional, for SSH support) Download Exscript -- Release: http://exscript.googlecode.com/files/exscript-0.9.11.tgz SVN instructions: http://code.google.com/p/exscript/source Links -- Exscript project page: http://code.google.com/p/exscript/ Mailing list: http://groups.google.com/group/exscript Bug tracker: http://code.google.com/p/exscript/issues/list Browse the source: http://exscript.googlecode.com/svn/trunk/ -- http://mail.python.org/mailman/listinfo/python-list
Listing subtypes
Hi, I remember seeing an easy way to list all subtypes of a specific type but I haven't been able to find it anymore. What I am trying to do is this: Given a class, get a list of all classes that derive from it. Pretty much like __mro__ but downwards rather then upwards. Any ideas? -Samuel -- http://mail.python.org/mailman/listinfo/python-list
Re: Listing subtypes
On Nov 29, 1:54 am, "Diez B. Roggisch" <[EMAIL PROTECTED]> wrote: > print A.__subclasses__() Ah, I knew I had seen this before. Thanks! -Samuel -- http://mail.python.org/mailman/listinfo/python-list
Detecting mod_rewrite via CGI
Hi,
I am using an Apache webserver with mod_rewrite and the cgi module. Is
there a way to do the following that isn't completely evil?
def mod_rewrite_enabled():
try:
file = open(os.path.join(os.environ['DOCUMENT_ROOT'],
'.htaccess'))
except:
return False
for line in file:
if line.startswith('RewriteEngine on'):
file.close()
return True
file.close()
return False
Unfortunately, not using .htaccess isn't an option.
-Samuel
--
http://mail.python.org/mailman/listinfo/python-list
ANNOUNCE: Spiff Guard 1.9.0
Introduction
Spiff Guard is a library for implementing access lists in Python. It
provides a clean and simple API and was implemented with performance
and security in mind. It was originally inspired by phpGACL (http://
phpgacl.sourceforge.net/), but features an API that is significantly
cleaner and easier to use.
Spiff Guard is free software and distributed under the GNU GPLv2.
Changes since 1.4.0:
-
The bad:
o This release breaks API, hard.
The good:
o Spiff Guard is now type-aware. That means that you can create your
own types and store them in the database; Spiff Guard will create
an instance of the same type when you retrieve the object later.
o Sections are now obsolete - instead, just use types as a section.
o Spiff Guard makes now extensive use of caching.
o The API is now a lot easier to use - if that is even possible.
Example Code:
--
guard = Guard(db_connection)
group = ResourceGroup("My Group")
user= Resource("My User")
website = ResourceGroup("My Website")
view= Action("View")
write = Action("Edit")
guard.grant(group, view, website)
guard.grant(user, edit, website)
if guard.has_permission(user, view, website):
print 'Permission granted.'
Dependencies
-
sqlalchemy (http://www.sqlalchemy.org/)
Download
-
Tarball:
http://pypi.python.org/packages/source/S/Spiff%20Guard/Spiff%20Guard-1.9.0.tar.gz#md5=a81ca3f310899ca8471d26ffbb58a83a
SVN:
svn checkout http://spiff.googlecode.com/svn/trunk/libs/Guard/
Links:
---
Documentation: http://spiff.googlecode.com/svn/trunk/libs/Guard/README
Example: http://spiff.googlecode.com/svn/trunk/libs/Guard/tests/DBTest.py
Spiff project page: http://code.google.com/p/spiff/
Mailing list: http://groups.google.com/group/spiff-devel
Bug tracker: http://code.google.com/p/spiff/issues/list
Browse the source: http://spiff.googlecode.com/svn/trunk/libs/Guard/
If you have any questions, please do not hesitate to ask or file a
bug.
-Samuel
--
http://mail.python.org/mailman/listinfo/python-list
Generating API documentation as a textfile
Does anyone know an easy way to extract the API documentation that is embedded into a Python file as a text file? -Samuel -- http://mail.python.org/mailman/listinfo/python-list
Re: Generating API documentation as a textfile
On Mon, 03 Dec 2007 06:45:45 -0800, Giampaolo Rodola' wrote: > dir.__doc__ This contains only the docstring one object (module, class, function, ...). I was thinking more of the complete API documentation that can be found in a file, and formatted in a readable way. -Samuel -- http://mail.python.org/mailman/listinfo/python-list
Re: highscores list
On Sat, 08 Dec 2007 13:32:08 -0500, Shawn Minisall wrote: > I'm writing a game that uses two functions to check and see if a file > called highScoresList.txt exists in the main dir of the game program. If > it doesn, it creates one. That part is working fine. The problem is > arising when it goes to read in the high scores from the file when I > play again. Not your original problem, but you should want to look at this instead of creating yet another parser: http://docs.python.org/lib/RawConfigParser-objects.html -Samuel -- http://mail.python.org/mailman/listinfo/python-list
Ruby's Template Engine for Python
Hi,
Following the masses I was drawn to RoR's website today and I saw the
following HTML template in one of the screencasts:
--
<% form_remote_tag :url => { :action => :search },
:update => :results,
:complete => visual_effect(:blind_down, 'image')
:before => %(Element.show('image'))
:after=> %(Element.hide('image')) %>
<%= text_field_tag 'Search' %>
<%= submit_tag 'Search' %>
<% end %>
--
I have to admit that a template engine that integrates with AJAX like
that looks inherently cool. Is there anything like that for Python,
optimally a stand-alone library? Django seems to handle this differently,
from what I can see.
-Samuel
--
http://mail.python.org/mailman/listinfo/python-list
Re: Ruby's Template Engine for Python
On Sat, 08 Dec 2007 13:06:15 -0800, Steve Howell wrote: > This is what I came up with: > > http://pylonshq.com/pastetags/form_remote_tag I see that Pylons uses a standard templating systems with all the JS renderers hooked into it as standard template function calls. That's great, now I just have to find the piece of code that provides those callbacks. Ah, here it is: > http://pylonshq.com/WebHelpers/module-webhelpers.html And there I go, easily adding the same functionality to any template system of my choice. That was easier than I expected. Thanks for your help ;-)! -Samuel -- http://mail.python.org/mailman/listinfo/python-list
Re: a strange SyntaxError
On Sun, 09 Dec 2007 12:35:46 -0800, CoolGenie wrote: > OK, sorry, this was about indents. Stupid VIM! $ mkdir -p $HOME/.vim/ftplugin/ $ echo "setlocal sw=4 setlocal ts=4 noremap py o/**/ " >> ~/.vim/ftplugin/python.vim $ echo "syntax on set sw=2 set ts=2 set nu set nuw=3 set autoindent set expandtab" >> $HOME/.vimrc --- Problem solved, never to be found again. Bye, -Sam -- http://mail.python.org/mailman/listinfo/python-list
ANNOUNCE: Spiff Integrator 0.1
Introduction Spiff Integrator is a small but powerful library for adding plugin support to Python applications. It is in some ways comparable to Debian's dpkg and handles packing/unpacking, installation/updates/removal, and dependency management and provides a bus over which plugins may communicate. Spiff Integrator is free software and distributed under the GNU GPLv2. This is the initial release. Dependencies - sqlalchemy (http://www.sqlalchemy.org/) Spiff Guard (http://pypi.python.org/pypi/Spiff%20Guard) Download - Tarball: http://pypi.python.org/packages/source/S/Spiff%20Integrator/Spiff%20Integrator-0.1.tar.gz SVN: svn checkout http://spiff.googlecode.com/svn/trunk/libs/Integrator/ Links: --- Documentation: http://spiff.googlecode.com/svn/trunk/libs/Integrator/README API Documentation: http://spiff.debain.org/static/docs/spiff_integrator/index.html Example: http://spiff.googlecode.com/svn/trunk/pkg.py Spiff project page: http://code.google.com/p/spiff/ Mailing list: http://groups.google.com/group/spiff-devel Bug tracker: http://code.google.com/p/spiff/issues/list Browse the source: http://spiff.googlecode.com/svn/trunk/libs/Integrator/ If you have any questions, please do not hesitate to ask or file a bug. -Samuel -- http://mail.python.org/mailman/listinfo/python-list
ANN:proxysocket(socks4,socks5)v0.1
http://code.google.com/p/proxysocket/downloads/list -- http://mail.python.org/mailman/listinfo/python-list
Re: ANN:proxysocket(socks4,socks5)v0.1
On 1月18日, 下午3时04分, Tim Roberts <[EMAIL PROTECTED]> wrote: > Samuel <[EMAIL PROTECTED]> wrote: > > >http://code.google.com/p/proxysocket/downloads/list > > Allow me to introduce you to the concept of comments. Python allows you to > include descriptive sentences in your program that explain what the > functions do, what your intentions were, what the variables do, what the > states mean, etc. It's easy to do; you just start the text with # signs. > > # This function allows you to ... > > # These variables define the connection state as the connection is > # made. > > They're great. You should try them. > -- > Tim Roberts, [EMAIL PROTECTED] > Providenza & Boekelheide, Inc. My English writing ability is bad, could you join this project? -- http://mail.python.org/mailman/listinfo/python-list
Re: ANN:proxysocket(socks4,socks5)v0.1
v0.2 http://proxysocket.googlecode.com/files/ProxySocket.py On 1月18日, 下午3时04分, Tim Roberts <[EMAIL PROTECTED]> wrote: > Samuel <[EMAIL PROTECTED]> wrote: > > >http://code.google.com/p/proxysocket/downloads/list > > Allow me to introduce you to the concept of comments. Python allows you to > include descriptive sentences in your program that explain what the > functions do, what your intentions were, what the variables do, what the > states mean, etc. It's easy to do; you just start the text with # signs. > > # This function allows you to ... > > # These variables define the connection state as the connection is > # made. > > They're great. You should try them. > -- > Tim Roberts, [EMAIL PROTECTED] > Providenza & Boekelheide, Inc. -- http://mail.python.org/mailman/listinfo/python-list
Re: Protocol for thread communication
On Tue, 04 Mar 2008 22:12:00 -0700, Michael Torrie wrote: > Does anyone have any recommended ideas/ways of implementing a proper > control and status protocol for communicating with threads? I have a > program that spawns a few worker threads, and I'd like a good, clean way > of communicating the status of these threads back to the main thread. > Each thread (wrapped in a very simple class) has only a few states, and > progress levels in those states. And sometimes they can error out, > although if the main thread knew about it, it could ask the thread to > retry (start over). How would any of you do this? A callback method > that the thread can call (synchronizing one-way variables isn't a > problem)? A queue? How would the main thread check these things? I implemented a generic library that could easily support these things: http://code.google.com/p/exscript/source/browse/trunk/lib/WorkQueue/ Basically, it's an asynchronous queue into which the jobs are added. The "MainLoop" class manages jobs (like, making sure that n jobs are running at the same time, and providing signals when jobs are done). One way to add a status to each job is by adding a status attribute to the Job class, in addition, a Job may emit a signal (using a simple signal/event mechanism such as this one: http://code.google.com/p/exscript/source/browse/trunk/lib/Exscript/ Trackable.py ) whenever a status changes, and have the MainLoop class fetch that. -Samuel -- http://mail.python.org/mailman/listinfo/python-list
Apache/mod_python: Registering a request handler dynamically
Hi,
Is there a way to dynamically overwrite the request handler from within
mod_python scripts? Something along those lines:
---
from mod_python import apache
def myhandler(request):
request.content_type = 'text/plain'
request.write('Hello world')
apache.set_default_handler(myhandler)
---
I specifically want to avoid changing the Apache directive, as this code
is supposed to function in a place where the user has no permission to
override the Apache directive.
The reason is that I am trying to hide the difference between different
environments (such as mod_python or CGI) from the developer, such that
the following is possible:
---
#!/usr/bin/python
import os, os.path
os.chdir(os.path.dirname(__file__))
from PleaseHideMyEnvironment import RequestHandler
def index(request):
request.write('Hello World')
RequestHandler(index)
---
So at the time at which RequestHandler() is created, I need a way to make
sure that mod_python calls to the RequestHandler instead of the normal
handler, whenever a new request is made.
Any idea?
-Samuel
--
http://mail.python.org/mailman/listinfo/python-list
Re: Resizing Tif's to smaller gifs adds pixels
Scott,
I appreciate the quick response, but I need this in a GIF format.
Samuel
On Oct 21, 3:46 pm, Scott David Daniels <[EMAIL PROTECTED]> wrote:
> [EMAIL PROTECTED] wrote:
> > ...I need to scale a TIFF image from 1925x588 px to a GIF of 600xnnn px.
> > I've tried the following code, but it leads to a lot of extra odd
> > colored pixels being inserted into the resulting image.
> > img = "tmp/tmsho20080901.tif"
> > im = Image.open("tmp/tmsho20080901.tif")
> > w, h = im.size
> > im.thumbnail((600, h * 600 / w), Image.ANTIALIAS)
>
> This line is rather silly, making a result that you drop on the floor.>
> newimg = im.resize((600, int(h * (600.0 / w))), Image.ANTIALIAS)
> > newimg.save("tmsho20080901.gif")
>
> ...
>
> > Using ImageMagick's convert I would do this...
>
> > convert -colors 256 -resize 600 -colorspace RGB -black-threshold 100 -
> > contrast -intent Perceptual tmp/tmsho20080901.tif tmsho20080901.gif
>
> > I think it may have something to do with the palette or the number of
> > colors alotted for the resulting image, but I'm really not a graphics
> > guy.
>
> Yep, you are right. The issue is trying to reduce to a 256-color
> pallette. Try using '.png' format (portable network graphics), that
> allows a full palette for exact pixel images.
>
> Try something like this:
> im = Image.open("tmp/tmsho20080901.tif")
> w, h = im.size
> im.thumbnail((600, h * 600 // w),
> Image.ANTIALIAS).save("tmp/sho20080901.png")
>
> --Scott David Daniels
> [EMAIL PROTECTED]
--
http://mail.python.org/mailman/listinfo/python-list
Re: 2 + 2 = 5
On Friday, July 6, 2012 8:39:58 AM UTC+10, Andrew Cooper wrote: > On 05/07/2012 22:46, Evan Driscoll wrote: > > On 01/-10/-28163 01:59 PM, Alexander Blinne wrote: > >> 5+0 is actually 4+0, because 5 == 4, so 5+0 gives 4. > >> 5+1 is actually 4+1, which is 5, but 5 is again 4. > >> 5+2 is 4+2 which is 6. > > > > Now all I can think is "Hoory for new math, new-hoo-hoo math" > :-) > > > > Evan > > It wont do you a bit of good to read new math! > > (My mind was on exactly the same track) > > ~Andrew +1 -- http://mail.python.org/mailman/listinfo/python-list
Force Python ast to emit lines no longer than $length
I've written a library that works at the ast level. Sometimes the generated output goes over the linter line length limit. "foo_bar_can_haz_foo_bar_can_haz_foo_bar_can_haz_foo_bar_can_haz_foo_bar_can_haz_foo_bar_can_haz" How do I generate this kind of code instead? "foo_bar_can_haz_foo_bar_can_haz_foo_bar_can_haz_" "foo_bar_can_haz_foo_bar_can_haz_foo_bar_can_haz" (also happy with the \ and parenthesised variants) [cross-posted: stackoverflow.com/q/65800797] The only thing I can think of doing—retaining support for 3.6, 3.7, 3.8, 3.9, and 3.10a4—is to contribute to both astor and the builtin ast.unparse… -- https://mail.python.org/mailman/listinfo/python-list
Re: Force Python ast to emit lines no longer than $length
I ended up adding word-wrap support directly to my code-generation: https://github.com/SamuelMarks/doctrans/commit/6147b21e168b66623aa1be95cb38b1969daa5147 Samuel Marks Charity <https://sydneyscientific.org> | consultancy <https://offscale.io> | open-source <https://github.com/offscale> | LinkedIn <https://linkedin.com/in/samuelmarks> On Wed, Jan 20, 2021 at 2:05 PM Samuel Marks wrote: > I've written a library that works at the ast level. Sometimes the > generated output goes over the linter line length limit. > > > "foo_bar_can_haz_foo_bar_can_haz_foo_bar_can_haz_foo_bar_can_haz_foo_bar_can_haz_foo_bar_can_haz" > > How do I generate this kind of code instead? > > "foo_bar_can_haz_foo_bar_can_haz_foo_bar_can_haz_" > "foo_bar_can_haz_foo_bar_can_haz_foo_bar_can_haz" > > (also happy with the \ and parenthesised variants) [cross-posted: > stackoverflow.com/q/65800797] > > The only thing I can think of doing—retaining support for 3.6, 3.7, > 3.8, 3.9, and 3.10a4—is to contribute to both astor and the builtin > ast.unparse… > -- https://mail.python.org/mailman/listinfo/python-list
Re: Running python from pty without prompt
Just in case it's not clear, this is running on a (virtual) PTY. -- https://mail.python.org/mailman/listinfo/python-list
Re: Running python from pty without prompt
Michael, yes. FYI, I found out why this works. Pressing Ctrl-D flushes the input buffer. If you do this on an empty line, it causes read(...) to return 0 which Ruby considers end of input for the script, but the pipe is not closed. -- https://mail.python.org/mailman/listinfo/python-list
Re: Running python from pty without prompt
Here are some examples of different languages: https://github.com/ioquatix/script-runner/blob/master/examples/python-eot.py -- https://mail.python.org/mailman/listinfo/python-list
Using SSML in Python script/program
I need to use SSML (Synthesized Speech Markup Language) to play an audio file with the tag in my Alexa Skill (as per Amazon's instructions). Problem is, I don't know how to use SSML with Python. I know I can use it with Java but I want to build my skills with Python. I've looked all over, but haven't found any working examples of SSML in a Python script/program - does anyone know? Any help is much appreciated! -- https://mail.python.org/mailman/listinfo/python-list
Re: [poliastro-dev] ANN: poliastro 0.13.0 released 🚀
Le mar. 6 août 2019 à 08:33, Samuel Dupree a écrit : > Juan Luis Cano, > > When will poliastro ver. 0.13.0 become available from Anaconda? At the > time of this note, only ver. 0.12.0 is available. > > Lastly what is the recommended procedure to update poliastro from vers. > 0.12.0 to 0.13.0? > > Sam Dupree > A pull request was automatically created by a bot: https://github.com/conda-forge/poliastro-feedstock/pull/29 and Juan Luis merged it yesterday, so that the new version is available since yesterday, see the version badge on the README at https://github.com/conda-forge/poliastro-feedstock Clicking on that badge sends to: https://anaconda.org/conda-forge/poliastro which has the installation instructions. Use `conda upgrade` instead of `conda install` if you want to upgrade. -- https://mail.python.org/mailman/listinfo/python-list
ABC with abstractmethod: kwargs on Base, explicit names on implementation
After a discussion on #python on Freenode, I'm here. The gist of it is: > Falc - Signature of method 'Pharm.foo()' does not match signature of base > method in class 'Base' What's the right way of specialising the children and leaving the Base pretty much empty? Specifically I want: • All implementers to be forced to implement Base's method • Base to be able to do very basic things with kwargs, namely log it (to a file like stdout or a db) • Support for [at least] Python 3.6–3.9 (I don't think `Protocol` has been backported?) • Implementors to have access to the proper name, rather than having to unpack kwargs Should I be using a Base class? - Metaclass? - Something else entirely? - I know giving `b` a default value would resolve the [PyCharm] linter error… but I want it to be a required argument. Full example: from abc import ABC, abstractmethod class Base(ABC): @abstractmethod def foo(self, a, **kwargs): """ :param a: var :type a: ```int``` :param **kwargs: keyword args :type **kwargs: ``dict`` """ print(a) class Pharm(Base): def foo(self, a, b): """ :param a: var :type a: ```int``` :param b: var :type b: ```int``` """ super(Pharm, self).foo(a=a) Thanks, Samuel Marks Charity <https://sydneyscientific.org> | consultancy <https://offscale.io> | open-source <https://github.com/offscale> | LinkedIn <https://linkedin.com/in/samuelmarks> -- https://mail.python.org/mailman/listinfo/python-list
Re: ABC with abstractmethod: kwargs on Base, explicit names on implementation
The main thing I want is type safety. I want Python to complain if the callee uses the wrong argument types, and to provide suggestions on what's needed and info about it. Without a base class I can just have docstrings and type annotations to achieve that. What can I use that will require all implementers to have a minimum of the same properties and arguments, but also allow them to add new properties and arguments? Preferably I would like this all to happen before compile/interpret time. This also opens it up to AST driven stub generation; as can be found in various IDEs (and also a little project I wrote that `import ast`). What are my options here? - It doesn't seem like the metaclass or decorator approaches will help here… Samuel Marks Charity <https://sydneyscientific.org> | consultancy <https://offscale.io> | open-source <https://github.com/offscale> | LinkedIn <https://linkedin.com/in/samuelmarks> Samuel Marks wrote at 2020-8-24 18:24 +1000: >After a discussion on #python on Freenode, I'm here. > >The gist of it is: >> Falc - Signature of method 'Pharm.foo()' does not match signature of base >> method in class 'Base' > >What's the right way of specialising the children and leaving the Base >pretty much empty? > >Specifically I want: >• All implementers to be forced to implement Base's method >• Base to be able to do very basic things with kwargs, namely log it >(to a file like stdout or a db) >• Support for [at least] Python 3.6–3.9 (I don't think `Protocol` has >been backported?) >• Implementors to have access to the proper name, rather than having >to unpack kwargs > >Should I be using a Base class? - Metaclass? - Something else >entirely? - I know giving `b` a default value would resolve the >[PyCharm] linter error… but I want it to be a required argument. > >Full example: > >from abc import ABC, abstractmethod > > >class Base(ABC): >@abstractmethod >def foo(self, a, **kwargs): > ... >class Pharm(Base): >def foo(self, a, b): > ... Python make a distinction between positional and keyword arguments. A positional argument is identified by its position in the parameter list; a keyword argument is identified by its name. `**` introduces arbitrary keyword arguments. In a call, all those arguments must be passed as "name=value". In your case above, `b` is not a keyword argument and thus is not matched by `**kwargs`. The error you observe is justified. You can try: class Base(ABC): @abstractmethod def foo(self, a, *args, **kwargs): ... class Pharm(Base): def foo(self, a, b, *args, **kwargs): ... Note that the base method signature allows arbitrary positional and keyword arguments. As a consequence, derived methods must do the same. If this is not what you want, you might want to explore the use of a decorator or a meta class rather than a base class. -- https://mail.python.org/mailman/listinfo/python-list
Re: ABC with abstractmethod: kwargs on Base, explicit names on implementation
So there is no way to get a AOT error when the two functions aren't implemented, if the two functions have different required arguments on implementors? To paint this picture for why I need this, say the first is: class Base(ABC): @abstractclass def train(self, epochs): asset epochs is not None and epochs > 0 …and the implementation is: class TensorFlow(Base): def train(self, learning_rate, epochs, callbacks, metrics, loss, optimizer): super(Base, self).__init__(epochs=epochs) [+ docstrings and PEP484 types; excluded here for concision] So how do I enable this use-case? - Obviously it doesn't make sense to include these on the base class, and giving them default values probably doesn't make sense either. You're saying I shouldn't be using ABC for this. So what should I be using? The requirement I'm trying to enforce is that each implementor has a `train` callable attached. Preferably with one required field (but this is just a nice-to-have). BTW: I have enabled an argument inheritance use-case through creating a library with the builtin `ast` module. So now you can have `def train_c(self, config)` with config referring to an instance of a class which inherits from a base config class. See here for the implementation https://github.com/SamuelMarks/doctrans However if I only have functions which accept an instance of a class as the argument, then that will make it less user-friendly to the API caller. So I really am looking for handling both interfaces in a straightforward manner. Thanks for your suggestions, Samuel Marks Charity <https://sydneyscientific.org> | consultancy <https://offscale.io> | open-source <https://github.com/offscale> | LinkedIn <https://linkedin.com/in/samuelmarks> On Fri, Aug 28, 2020 at 3:51 AM Dieter Maurer wrote: > > Samuel Marks wrote at 2020-8-27 15:58 +1000: > >The main thing I want is type safety. I want Python to complain if the > >callee uses the wrong argument types, and to provide suggestions on > >what's needed and info about it. > > > >Without a base class I can just have docstrings and type annotations > >to achieve that. > > > >What can I use that will require all implementers to have a minimum of > >the same properties and arguments, but also allow them to add new > >properties and arguments? > > A main paradigm of object oriented programming is the > ability to use a derived class instance with knowledge only > about the base class. This implies that in many cases, you > need not know the concrete class because any instance of a derived > class must have the essential behavior of the base class instances. > > This paradigm imposes limitations on the allowable signature changes. > An overriding method may add further parameters but all those > must have default values - otherwise, the use with base class knowledge > only would cause errors. > > > Preferably I would like this all to happen before compile/interpret > time. > > Use a "lint" version to check compatibilty. -- https://mail.python.org/mailman/listinfo/python-list
Re: ABC with abstractmethod: kwargs on Base, explicit names on implementation
Putting Liskov substitution principal to one side, I had a suggestion to follow PEP3102, and do `def train(self, *, epochs):`… It's a rather simple suggestion that I just might take aboard. In response to Dieter: My purpose for using a base class is so that the highest level interface—say a CLI, GUI, REST and RPC API—can all be configured in one place, but enable the greatest possible divergence from a parameter standpoint. An alternative here [in this domain] would be to create a new Keras/scikit.learn. I.e., one consistent interface that is to be implemented in full by each framework. The reason I don't want to take this alternative is manyfold, that I don't need to get in here (happy to start a new thread if you're interested). Two major advantages of this approach are: 0. Implementers can be [pretty much] as divergent as they want, with different default and required parameters; and even semantics [though I do have `assert`s to force `method0` to be executed before `method1`]; 1. Implementers don't need to create their own CLIs, GUIs, REST and RPC APIs Two major disadvantages: 0. Parameters aren't known ahead of time, so you can do the whole `Animal` [duck type] trick where `Animal` could actually be `Dog` or `Horse` [making the obvious Base `ABC` & `abstractmethod` approach 'wrong'] 1. Each implementer framework can maintain wildly different internal APIs, making more hardcore integrations—in a multi-ML sense—far more difficult Samuel Marks Charity <https://sydneyscientific.org> | consultancy <https://offscale.io> | open-source <https://github.com/offscale> | LinkedIn <https://linkedin.com/in/samuelmarks> On Sat, Aug 29, 2020 at 8:24 PM Dieter Maurer wrote: > > Samuel Marks wrote at 2020-8-29 19:14 +1000: > >So there is no way to get a AOT error when the two functions aren't > >implemented, if the two functions have different required arguments on > >implementors? > > > >To paint this picture for why I need this, say the first is: > > > >class Base(ABC): > >@abstractclass > >def train(self, epochs): > >asset epochs is not None and epochs > 0 > > > >…and the implementation is: > > > >class TensorFlow(Base): > >def train(self, learning_rate, epochs, callbacks, metrics, loss, > > optimizer): > >super(Base, self).__init__(epochs=epochs) > > > >[+ docstrings and PEP484 types; excluded here for concision] > > > >So how do I enable this use-case? - Obviously it doesn't make sense to > >include these on the base class, and giving them default values > >probably doesn't make sense either. > > > >You're saying I shouldn't be using ABC for this. So what should I be using? > > What is your purpose to use a base class in the first place -- > and especially one where `train` has this signature? > > You signal with this base class, that it makes sense to > call `train` with a single positional argument. > But `train` in the derived class cannot be called in this way. > > Base classes are there to capture common features of several > related classes. In your example above, this is not the case. > Do not use base classes in this way. > > >The requirement I'm trying to enforce is that each implementor has a > >`train` callable attached. > >Preferably with one required field (but this is just a nice-to-have). > > Read the Python documentation about metaclasses and the special methods > related to class derivation. Both approaches would allow you > to check whatever you want. > > > ... > >However if I only have functions which accept an instance of a class > >as the argument, then that will make it less user-friendly to the API > >caller. So I really am looking for handling both interfaces in a > >straightforward manner. > > Any system makes some things easy and other things difficult. > Try to stay with the essential paradigms underlaying the system -- > in order to use the easy things whenever possible. > > In your case, this means that the signature of an overriding > method must be quite similar to that of the overridden method. -- https://mail.python.org/mailman/listinfo/python-list
CLI parsing—with `--help` text—`--foo bar`, how to give additional parameters to `bar`?
Previously I have solved related problems with explicit `-}` and `-{`
(or `-b`) as in `nginxctl`:
```
$ python -m nginxctl serve --temp_dir '/tmp' \
-b 'server' \
--server_name 'localhost' --listen '8080' \
-b location '/' \
--root '/tmp/wwwroot' \
-} \
-}
nginx is running. Stop with: /usr/local/bin/nginx -c /tmp/nginx.conf -s stop
```
To illustrate the issue, using `ml-params` and ml-params-tensorflow:
```
$ python -m ml_params --engine 'tensorflow' train --help
usage: python -m ml_params train [-h]
--optimizer
{Adadelta,Adagrad,Adam,Adamax,Ftrl,Nadam,RMSprop}
Run the training loop for your ML pipeline.
optional arguments:
-h, --helpshow this help message and exit
--optimizer {Adadelta,Adagrad,Adam,Adamax,Ftrl,Nadam,RMSprop}
The optimizer
```
Idea: preprocess `sys.argv` so that this syntax would work
`--optimizer Adam[learning_rate=0.01]`*
*square rather than round so as not to require escape characters or
quoting in `sh`
Unfortunately this approach wouldn't give nice `--help` text.
What's the right solution here?
Thanks,
Samuel Marks
Charity <https://sydneyscientific.org> | consultancy
<https://offscale.io> | open-source <https://github.com/offscale> |
LinkedIn <https://linkedin.com/in/samuelmarks>
--
https://mail.python.org/mailman/listinfo/python-list
Re: CLI parsing—with `--help` text—`--foo bar`, how to give additional parameters to `bar`?
Yes it’s my module, and I’ve been using argparse https://github.com/SamuelMarks/ml-params No library I’ve found provides a solution to CLI argument parsing for my use-case. So I’ll write one. But what should it look like, syntactically and semantically? On Fri, 16 Oct 2020 at 3:14 am, Dieter Maurer wrote: > Samuel Marks wrote at 2020-10-15 20:53 +1100: > > ... > >To illustrate the issue, using `ml-params` and ml-params-tensorflow: > > ... > >What's the right solution here? > > While Python provides several modules in its standard library > to process parameters (e.g. the simple `getopt` and the flexible > `argparse`), > it is up to the "application" whether it uses such a module (and which one) > or whether it handle arguments on its own. > > Apparently, `ml_param` is not a staudard Python module. > Is it a package of your own? Then I suggest to check `argparse` whether > it supports your use case (I know, it can be customized to do it, > but maybe, it does it already out of the box). > > If `ml_param` is a third party module, then the question > is actually an `ml_param` question. Ask its support mailing lists > or have a look at its source. > > -- Samuel Marks Charity <https://sydneyscientific.org> | consultancy <https://offscale.io> | open-source <https://github.com/offscale> | LinkedIn < https://linkedin.com/in/samuelmarks> -- https://mail.python.org/mailman/listinfo/python-list
Re: CLI parsing—with `--help` text—`--foo bar`, how to give additional parameters to `bar`?
--optimizer Adam,learning_rate=0.01,something_else=3 That syntax isn’t so bad! =] How would you suggest the help text for this looks? (don’t worry about implementation, just what goes to stdout/stderr) PS: Yeah I used square brackets for my Bash arrays On Fri, 16 Oct 2020 at 10:26 am, Cameron Simpson wrote: > One other thing: > > On 15Oct2020 20:53, Samuel Marks wrote: > >Idea: preprocess `sys.argv` so that this syntax would work > >`--optimizer Adam[learning_rate=0.01]`* > > > >*square rather than round so as not to require escape characters or > >quoting in `sh` > > Square brackets are also shell syntax, introducing glob character > ranges. You're only not noticing probably because an unmatched glob is > normally let through unchanged. > > For example: > >somecmd x[b] > > would go through as "x[b]" _unless_ you had a matching local filename > (in this case a file named "xb") in which case the shell would match the > glob and pass through the match (in this case "xb"). > > You'll find this issue a fair bit: if you want some punctuation, someone > else has probably already wanted that punctuation for something. You'll > still need to quote things for a lot of stuff. > > Commas are not special. You could try: > > --optimizer Adam,learning_rate=0.01,something_else=3 > > which would work for a fair amount of stuff. > > >Unfortunately this approach wouldn't give nice `--help` text. > >What's the right solution here? > > To get nice help text you need your command line parser to be able to > compose that help text in some way. Usually the help text in argparse is > supplied when you define the option. > > Cheers, > Cameron Simpson > -- Samuel Marks Charity <https://sydneyscientific.org> | consultancy <https://offscale.io> | open-source <https://github.com/offscale> | LinkedIn < https://linkedin.com/in/samuelmarks> -- https://mail.python.org/mailman/listinfo/python-list
Re: CLI parsing—with `--help` text—`--foo bar`, how to give additional parameters to `bar`?
Hi Dan, The feature that existing CLI parsers are missing is a clean syntax for specifying options on the second parameter (the "value"), where there may be different options available depending on which you choose. For example: https://www.tensorflow.org/api_docs/python/tf/keras/optimizers/Adam has `learning_rate`, `beta_1`, `beta_2`, `epsilon`, and `amsgrad`* Whereas https://www.tensorflow.org/api_docs/python/tf/keras/optimizers/RMSprop has `learning_rate`, `rho`, `momentum`, `epsilon`, `centered`* *with clipnorm, clipvalue, decay hidden behind kwargs So the question is how to expose this as CLI options. `--optimizer Adam` is a good first step, but it should error if I try and give it `momentum`. The comma syntax is my favourite so far. I guess I'll just have to write a validator outside the CLI parser to handle this… Samuel Marks Charity <https://sydneyscientific.org> | consultancy <https://offscale.io> | open-source <https://github.com/offscale> | LinkedIn <https://linkedin.com/in/samuelmarks> On Fri, Oct 16, 2020 at 11:19 AM <[email protected]> wrote: > > On 2020-10-16 at 10:20:40 +1100, > Cameron Simpson wrote: > > > On 16Oct2020 10:09, Samuel Marks wrote: > > >Yes it’s my module, and I’ve been using argparse > > >https://github.com/SamuelMarks/ml-params > > > > > >No library I’ve found provides a solution to CLI argument parsing for my > > >use-case. > > Out of curiosity, what do your command line interfaces require that > argparse and other libraries don't have? > > Yes, if my only tool is a hammer, then every problem looks like a nail, > but I've yet to have a requirement that the POSIX rules don't cover. > > > >So I’ll write one ... > > Been there. Done that. :-) > > > > [...] But what should it look like, syntactically and semantically? > > [...] > > > In particular, I would not invent yet another command line syntax. > > I agree. The POSIX Utility Syntax and Guidelines: > > https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap12.html > > (I believe argparse supports a superset of the POSIX syntax.) > > I haven't found a definitive definition of GNU long options, only > examples. > > HTH, > Dan > -- > https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: Re: CLI parsing—with `--help` text—`--foo bar`, how to give additional parameters to `bar`?
Yeah I've played with custom actions before https://github.com/offscale/argparse-utils/tree/master/argparse_utils/actions But this would only help in one phase, the important phase of providing help text will need to be provided out-of-argparse and thrown in (like my trivial absl alternative, exposing a function which takes an argparse instance and returns an argparse instance) The only hack remaining is that I have to pass through `sys.argv` at least once before giving it to argparse. I wonder if there's a way to not need to explicitly go through it at all… https://github.com/SamuelMarks/ml-params/blob/d1fb184/ml_params/__main__.py#L89 [I didn't know `getopt` was exposed otherwise I'd use that , but there has to be a solution just using argparse?] Samuel Marks Charity <https://sydneyscientific.org> | consultancy <https://offscale.io> | open-source <https://github.com/offscale> | LinkedIn <https://linkedin.com/in/samuelmarks> Samuel Marks Charity | consultancy | open-source | LinkedIn On Fri, Oct 16, 2020 at 3:47 PM Dieter Maurer wrote: > > Samuel Marks wrote at 2020-10-16 10:09 +1100: > >Yes it’s my module, and I’ve been using argparse > >https://github.com/SamuelMarks/ml-params > > > >No library I’ve found provides a solution to CLI argument parsing for my > >use-case. > > Do you know that with `argparse` you can specify how many arguments an option > expects? Thus, it should be quite easily possible to > have --opt ... > Do you know that you can define new `Action`s for `argparse`? > This way, you could properly process `--opt ,, ...`. -- https://mail.python.org/mailman/listinfo/python-list
Re: CLI parsing—with `--help` text—`--foo bar`, how to give additional parameters to `bar`?
Yeah, but the risk with config files is you need a website—and/or full JSON schema output—to figure out what’s needed. (Although I should mention that with my doctrans project you can generate a config class—and class method—from/to your argparse parser; enabling the config file scenario rather cleanly) Also there is a project which takes your CLI and turns it into a GUI (Gooey). At some future point doctrans will be “complete”, and then you can provide inputs via: - CLI arguments - Config file - RPC* - REST* *TODO, will also optionally generate ORM classes for persistence On Fri, 16 Oct 2020 at 6:08 pm, Karen Shaeffer wrote: > Passing a lot of command line parameters is very error prone. Opening a > file and specifying flags is much more user friendly, because you have any > necessary help documented right there. In my eyes, the command line is only > useful for simple scripts. > > Karen > > > On Oct 15, 2020, at 11:44 PM, Karen Shaeffer > wrote: > > > > In my work, for complex ML modeling code, I never pass anything through > the command line. I implement absl.flags in each module, specifying all the > defaults. At runtime I instantiate the flags as class variables of a > ModelConfiguration class that gets passed into the model class. Absl > supports all I need for this scheme. Of course, for a package, I can have a > configuration module for all these configuration classes. > > > > This scheme is easy to maintain. It can be dynamically customized at run > time. It’s very easy to document and maintain and to use. In one project, I > even supported console inputs that dynamically managed configurations of > pipelines. Nothing was ever passed in on command lines. I wonder why you > need to play on the command line. > > > > And yea, I’ve written a command line parsing function in C a long time > ago. I thought that really cool at the time. I wouldn’t want to do it now. > > > > Karen. > > > >> On Oct 15, 2020, at 9:58 PM, Samuel Marks > wrote: > >> > >> Yeah I've played with custom actions before > >> > https://github.com/offscale/argparse-utils/tree/master/argparse_utils/actions > >> > >> But this would only help in one phase, the important phase of > >> providing help text will need to be provided out-of-argparse and > >> thrown in > >> > >> (like my trivial absl alternative, exposing a function which takes an > >> argparse instance and returns an argparse instance) > >> > >> The only hack remaining is that I have to pass through `sys.argv` at > >> least once before giving it to argparse. I wonder if there's a way to > >> not need to explicitly go through it at all… > >> > https://github.com/SamuelMarks/ml-params/blob/d1fb184/ml_params/__main__.py#L89 > >> > >> [I didn't know `getopt` was exposed otherwise I'd use that >> sometimes do in C>, but there has to be a solution just using > >> argparse?] > >> > >> Samuel Marks > >> Charity <https://sydneyscientific.org> | consultancy > >> <https://offscale.io> | open-source <https://github.com/offscale> | > >> LinkedIn <https://linkedin.com/in/samuelmarks> > >> > >> > >> Samuel Marks > >> Charity | consultancy | open-source | LinkedIn > >> > >> > >> On Fri, Oct 16, 2020 at 3:47 PM Dieter Maurer > wrote: > >>> > >>> Samuel Marks wrote at 2020-10-16 10:09 +1100: > >>>> Yes it’s my module, and I’ve been using argparse > >>>> https://github.com/SamuelMarks/ml-params > >>>> > >>>> No library I’ve found provides a solution to CLI argument parsing for > my > >>>> use-case. > >>> > >>> Do you know that with `argparse` you can specify how many arguments an > option > >>> expects? Thus, it should be quite easily possible to > >>> have --opt ... > >>> Do you know that you can define new `Action`s for `argparse`? > >>> This way, you could properly process `--opt ,, ...`. > >> -- > >> https://mail.python.org/mailman/listinfo/python-list > > > > -- Samuel Marks Charity <https://sydneyscientific.org> | consultancy <https://offscale.io> | open-source <https://github.com/offscale> | LinkedIn < https://linkedin.com/in/samuelmarks> -- https://mail.python.org/mailman/listinfo/python-list
Static typing—with annotations—function name & arguments, or result of call, or string
From my understanding, `ast.arguments` and `inspect.Signature` are the
two builtin ways of defining the function name and arguments in a
structured way.
What I am creating is a way of configuring… well let's be specific to
my use-case. I am building a way to configure TensorFlow.
One which is very type-driven, and will error as early-as-possible
when incorrect types or lack of required parameters are provided.
I can dynamically infer things like so:
```
import inspect
import tensorflow as tf
sig = inspect.signature(tf.keras.optimizers.Adam)
tuple({
"default": sig.parameters[param].default,
"name": param,
"typ": type(sig.parameters[param].default).__name__
if sig.parameters[param].default is not inspect._empty
and sig.parameters[param].annotation is inspect._empty
else sig.parameters[param].annotation,
}
for param in sig.parameters if param != 'name'
)
```
I can also parse the docstring, as I do in my doctrans library and tool.
Which will give me the options I can provide the class. So there's an
obvious solution, to generate these classes:
```
class TensorFlowConfig(object):
optimizer: Optional[Optimizer] = None
# Or maybe a `Protocol`?
class Optimizer(object): pass
class AdamConfig(Optimizer):
learning_rate: float = 0.001
beta_1: float = 0.9
beta_2: float = 0.999
epsilon: float = 1e-07
amsgrad: bool = False
kwargs: dict = {}
TensorFlowConfig().optimizer = AdamConfig()
```
But, keeping in mind the goal to expose everything in all interfaces,
the question then becomes how to generate an argparse parser from
this. And how to generate a function from this. And how to ensure that
whatever code-completion interface one uses in Python, that it can
figure out what the correct parameters are.
So I should get an error about incorrect type when I:
```
# CLI
--optimizer 'Adam' 'learning_rate = True'
# class*
TensorFlowConfig().optimizer = tf.keras.optimizers.Adam(learning_rate=True)
TensorFlowConfig().optimizer = AdamConfig(learning_rate=True)
# Function*
MyTensorFlowClass.train(optimizer=tf.keras.optimizers.Adam(learning_rate=True))
MyTensorFlowClass.train(optimizer=AdamConfig(learning_rate=True))
* both of these next two lines—after the heading—should be equivalent
```
Syntax like this should also be valid
```
TensorFlowConfig().optimizer = 'tf.keras.optimizers.Adam'
TensorFlowConfig().optimizer = 'Adam'
MyTensorFlowClass.train(optimizer='tf.keras.optimizers.Adam')
MyTensorFlowClass.train(optimizer='Adam')
```
Do I have huge annotations like this*, with an `Any` lopped on for
further analysis down the line? - So it can easily generate into
`choices` for argparse? [which will need to be subclassed in order to
enable that "--optimizer 'Adam' 'learning_rate = True'" syntax]
*
https://github.com/SamuelMarks/ml-params-tensorflow/blob/1d48502/ml_params_tensorflow/ml_params/trainer.py#L213-L215
What makes sense?
Thanks for your insights,
Samuel Marks
Charity <https://sydneyscientific.org> | consultancy
<https://offscale.io> | open-source <https://github.com/offscale> |
LinkedIn <https://linkedin.com/in/samuelmarks>
PS: This is the `doctrans` project I referenced
https://github.com/SamuelMarks/doctrans
--
https://mail.python.org/mailman/listinfo/python-list
Change directory not successfully done
Hi, guys, This should be a simple problem, but I just can not resolve it. I just want to use a python script to change my working directory. see my following code: # mycd.py 1) destdir = "" 2) command = "cd "+ destdir 3) os.system(command) 4) os.chdir(destdir) But neither 3) nor 4) is used, I just can not change the directory after execute mycd.py. This remind me of bash script. If you change directory in your bash script file, it will only impact the sub process of that script, except that you invoke that bash script by ./script_file_name. But what should I do in the case of python script? Thanks and Regards Samuel Yin -- http://mail.python.org/mailman/listinfo/python-list
Re: Change directory not successfully done
Thanks for your answer. I know why the why os.system or os.chdir failed change my directory. But Sorry for my un-clear description of my problem. Currently I work in window platform, use cmd.exe instead of bash. I mentioned bash just as a example to illustrate my problem. Thanks and Regards Samuel Yin Mike Meyer 写道: Samuel Yin <[EMAIL PROTECTED]> writes: Hi, guys, This should be a simple problem, but I just can not resolve it. I just want to use a python script to change my working directory. see my following code: # mycd.py 1) destdir = "" 2) command = "cd "+ destdir 3) os.system(command) 4) os.chdir(destdir) But neither 3) nor 4) is used, I just can not change the directory after execute mycd.py. This remind me of bash script. If you change directory in your bash script file, it will only impact the sub process of that script, except that you invoke that bash script by ./script_file_name. But what should I do in the case of python script? Actually, one solution is a level of indirection worse than the bash script. Doing a cd changes the current directory of the process that executes the cd system call. In a bash script, it's the shell executing the script. In your python script, os.system launches another process to run the command, and it's *that* process that has it's directory changed. The os.chdir changes the shell of the python interpreter, which still doesn't do you any good. One solution is to switch to a shell that understands Python, and have that execfile your script. There is a Python environment that can be configured to be used as a shell, but I can't remeber it's name. If you want to stay with bash, your solutions are the same as they are for setting an environment variable in the parent shell. You can google for that for a long discussion of the issues. The solution I liked from that thread was an alias: In your bash do: alias mycd="eval $(python mycd.py)" mycd.py looks like: destdir = 'xx' command = 'os ' + destdir print command At the bash prompt you enter the command "mycd", your python script builds a command for the shell to execute and prints it, the eval reads that output and executes it in your shell. If you want to pass arguments to the python script, you'll need to use a shell function instead of an alias. -- http://mail.python.org/mailman/listinfo/python-list
Re: Unexpected behavior of read only attributes and super
Even more strangeness If I define the class to use slots class SD(dict): __slots__ = ['a','b'] s = SD() >>> s.__iter__ >>> s.__iter__ = 5 Traceback (most recent call last): File "", line 1, in ? AttributeError: 'SD' object attribute '__iter__' is read-only Then I get the read only exception but not because it was read only but because I defined slots. This looks like a logic error in how setattr is working in in the superclass dict ****** Samuel M. Smith Ph.D. 2966 Fort Hill Road Eagle Mountain, Utah 84043 801-768-2768 voice 801-768-2769 fax ** "The greatest source of failure and unhappiness in the world is giving up what we want most for what we want at the moment" ** -- http://mail.python.org/mailman/listinfo/python-list
Re: Logging
"Neil Benn" <[EMAIL PROTECTED]> schrieb im Newsbeitrag
news:[EMAIL PROTECTED]
Neil Benn wrote:
Hello,
I'm running a test and having issues with logging, if I call
logging.shutdown() and then want to start the logging going again
then I get a problem as if I call shutdown, I can't get the root
logger again, such as :
.
Previous code missed out a line :
.>>> import logging
.>>> objTestLogger = logging.getLogger()
.>>> objTestLogger.setLevel(logging.INFO)
.>>> objTestLogger.addHandler(logging.FileHandler('c:\\test.log'))
.>>> objTestLogger.info("THIS IS A TEST")
.>>> logging.shutdown()
.>>> objTestLogger = logging.getLogger()
.>>> objTestLogger.setLevel(logging.INFO)
.>>> objTestLogger.addHandler(logging.FileHandler('c:\\test.log'))
.>>> objTestLogger.info("THIS IS A TEST")
.Traceback (most recent call last):
. File "c:\program files\python23\lib\logging\__init__.py", line 679,
in emit
.self.stream.write("%s\n" % msg)
.ValueError: I/O operation on closed file
.>>>
logging.shutdown() doesn't clean up the dict of handlers at the module
level and the list of handlers maintained and at the level of the
manager of loggers. So after calling logging.getLogger() etc. for the
second time, you still have the closed filehandle in the list of
loghandlers. The new FileHandler with the open filehandle is appended to
this list of handlers. Then when you call ...info("...") this call is
dispatched to all handlers in the handlerlist, i.e. also to the one with
the closed filehandle.
I don't know if this has to be considered a bug in the logging module or
if your use case is simply not covered.
If you add something like
.>>> logging._handlers.clear()
.>>> logging.root.handlers = []
.>>> for l in logging.Logger.manager.loggerDict.values():
.>>> l.handlers = []
after logging.shutdown() and before getting the new logger, your script
will probably run without errors.
hth,
Samuel
--
http://mail.python.org/mailman/listinfo/python-list
Dependencies from git aren't being installed? (setuptools, pip)
I'm sure I've just misconfigured my setup.py, requirements.txt and/or project structure, but I've tried: * install_requires * requirements.txt * install_requires + dependency_links With (for install_requires): * bettertutors_sql_models==dev * bettertutors_sql_models For requirements.txt, dependency_links: * git+https://github.com/bettertutors/sql-models#egg=bettertutors_sql_models * git+https://github.com/bettertutors/sql-models#egg=bettertutors_sql_models * https://github.com/bettertutors/sql-models/archive/master.zip#egg=bettertutors_sql_models * https://github.com/bettertutors/sql-models/archive/master.zip * https://github.com/bettertutors/sql-models * https://github.com/bettertutors/sql-models/tarball/master#egg=sql-models Also asked here: http://stackoverflow.com/q/28540839 At one point I even tried writing my own install_deps function() to be run before setup(): https://gist.github.com/SamuelMarks/45a998a83dd60ddbadbc But nothing--apart from `cd`ing to my sql-models dir and running `pip install .`--is working for me. I'm using Python 2.7.*, and have experienced identical issues on Windows and Linux (with/without virtualenv). How am I meant to write in this dependency? -- https://mail.python.org/mailman/listinfo/python-list
How to read a directory path from a txt file
I have created a txt file with various paths to directories. The paths look like this /home/wachkama/Desktop/api/genshi /home/wachkama/Desktop/www/portal/schedule /home/wachkama/Desktop/show/help.genshi How do i read this paths in python ? -- https://mail.python.org/mailman/listinfo/python-list
Re: How to read a directory path from a txt file
On Monday, June 2, 2014 10:18:25 AM UTC-4, Samuel Kamau wrote: > I have created a txt file with various paths to directories. The paths look > like this > > /home/wachkama/Desktop/api/genshi > > /home/wachkama/Desktop/www/portal/schedule > > /home/wachkama/Desktop/show/help.genshi > > > > How do i read this paths in python ? I have permission issues with my web server. So I have written part of my script to read and write the current paths in the web servers with its username and group name into a txt file. I want my script to replace the new permissions on my web server with the permissions written on my txt file. So The last part of my script is the tricky part. How to read the txt file and go to all the paths and rewrite the permissions as per what is on the txt file. -- https://mail.python.org/mailman/listinfo/python-list
