[Python-Dev] A proposal: configuring logging using dictionaries
A little while ago, I posted here a suggestion about a new way to configure logging, using dictionaries. This received some positive and no negative feedback, so I have thought some more about the details of how it might work. I present below the results of that thinking, in a PEP-style format. I don't know if an actual PEP is required for a change of this type, but I felt that it's still worth going through the exercise to try and achieve a reasonable level of rigour. (I hope I've succeeded.) I would welcome all your feedback on this proposal. If I hear no negative feedback, I propose to implement this feature as suggested. I thought about posting this on comp.lang.python as well, but possibly it's a little too much information for most of the folks there. I think it would be useful to get feedback from the wider community, though, and welcome any suggestions on how best to achieve this. Thanks and regards, Vinay Sajip --- PEP: XXX Title: Dictionary-Based Configuration For Logging Version: $Revision$ Last-Modified: $Date$ Author: Vinay Sajip Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 15-Oct-2009 Python-Version: 2.7 and 3.2 Post-History: Abstract This PEP describes a new way of configuring logging using a dictionary to hold configuration information. Rationale = The present means for configuring Python's logging package is either by using the logging API to configure logging programmatically, or else by means of ConfigParser-based configuration files. Programmatic configuration, while offering maximal control, fixes the configuration in Python code. This does not facilitate changing it easily at runtime, and, as a result, the ability to flexibly turn the verbosity of logging up and down for different parts of a using application is lost. This limits the usability of logging as an aid to diagnosing problems - and sometimes, logging is the only diagnostic aid available in production environments. The ConfigParser-based configuration system is usable, but does not allow its users to configure all aspects of the logging package. For example, Filters cannot be configured using this system. Furthermore, the ConfigParser format appears to engender dislike (sometimes strong dislike) in some quarters. Though it was chosen because it was the only configuration format supported in the Python standard at that time, many people regard it (or perhaps just the particular schema chosen for logging's configuration) as 'crufty' or 'ugly', in some cases apparently on purely aesthetic grounds. Recent versions of Python inlude JSON support in the standard library, and this is also usable as a configuration format. In other environments, such as Google App Engine, YAML is used to configure applications, and usually the configuration of logging would be considered an integral part of the application configuration. Although the standard library does not contain YAML support at present, support for both JSON and YAML can be provided in a common way because both of these serialization formats allow deserialization of Python dictionaries. By providing a way to configure logging by passing the configuration in a dictionary, logging will be easier to configure not only for users of JSON and/or YAML, but also for users of bespoke configuration methods, by providing a common format in which to describe the desired configuration. Another drawback of the current ConfigParser-based configuration system is that it does not support incremental configuration: a new configuration completely replaces the existing configuration. Although full flexibility for incremental configuration is difficult to provide in a multi-threaded environment, the new configuration mechanism will allow the provision of limited support for incremental configuration. Specification = The specification consists of two parts: the API and the format of the dictionary used to convey configuration information (i.e. the schema to which it must conform). Naming -- Historically, the logging package has not been PEP-8 conformant. At some future time, this will be corrected by changing method and function names in the package in order to conform with PEP-8. However, in the interests of uniformity, the proposed additions to the API use a naming scheme which is consistent with the present scheme used by logging. API --- The logging.config module will have the following additions: * A class, called ``DictConfigurator``, whose constructor is passed the dictionary used for configuration, and which has a ``configure()`` method. * A callable, called ``dictConfigClass``, which will (by default) be set to ``DictConfigurator``. This is provided so that if desired, ``DictConfigurator`` can be replaced with a suitable user-defined implementation. * A function, called ``dictConfig()``, which takes a single argument - the dictionary holding the configuration. This function will cal
Re: [Python-Dev] A proposal: configuring logging using dictionaries
Vinay Sajip wrote: > A little while ago, I posted here a suggestion about a new way to configure > logging, using dictionaries. This received some positive and no negative > feedback, so I have thought some more about the details of how it might work. > I > present below the results of that thinking, in a PEP-style format. I don't > know > if an actual PEP is required for a change of this type, but I felt that it's > still worth going through the exercise to try and achieve a reasonable level > of > rigour. (I hope I've succeeded.) Since part of the purpose of this new configuration option is to allow other projects to use it as a target for specifying a logging configuration, maintaining it as a formal PEP seems like a good idea to me. This is already done for a few other standard-ish interfaces (distutils metadata, DB-API, WSGI). > I would welcome all your feedback on this proposal. If I hear no negative > feedback, I propose to implement this feature as suggested. My only real objection is to the "incremental" flag in the dictionary. I would suggest having two different API methods instead - initDictConfig and updateDictConfig. Or is there some specific reason for having the choice of incremental update vs replacement configuration in the hands of the config author rather than the application developer? My other question is whether or not it would be worth having the logging module able to export it's *current* configuration in dictionary form. I don't have a use case other than that it might be useful for debugging logging configuration errors when attempting to combine multiple sources of logging data, but it's worth considering. > So, for example, consider the following YAML snippet:: > > handers: > h1: #This is an id > # configuration of handler with id h1 goes here > h2: #This is another id > # configuration of handler with id h2 goes here Typo in the header here: s/handers/handlers/ > Configuration Errors > > > If an error is encountered during configuration, the system will raise a > ``ValueError`` or a ``TypeError`` with a suitably descriptive message. The > following is a (possibly incomplete) list of conditions which will raise an > error: > > * A ``level`` which is not a string or which is a string not corresponding to > an actual logging level > > * A ``propagate`` value which is not a Boolean > > * An id which does not have a corresponding destination > > * An invalid logger name You may want to allow yourself AttributeError and ImportError here. Alternatively, you could commit to converting those to an appropriate ValueError that provides more detail on the config setting that contained the problematic reference. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia --- ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] PY_SSIZE_T_CLEAN in py3k
Hello, It turns out (*) that even in py3k, not all modules are PY_SSIZE_T_CLEAN. Should we try to remedy that, and make PY_SSIZE_T_CLEAN the default in future 3.x versions? As far I know, there's no reason not to be PY_SSIZE_T_CLEAN, except for having to convert old code. (*) http://bugs.python.org/issue7080 Regards Antoine. PS : no, I'm not volunteering to do it all myself :) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] A proposal: configuring logging using dictionaries
Nick Coghlan gmail.com> writes: > Since part of the purpose of this new configuration option is to allow > other projects to use it as a target for specifying a logging > configuration, maintaining it as a formal PEP seems like a good idea to > me. This is already done for a few other standard-ish interfaces > (distutils metadata, DB-API, WSGI). I'm happy with this. I've subscribed to the PEPs list and once the subscription comes through, will post there linking to the PEP draft. > My only real objection is to the "incremental" flag in the dictionary. I > would suggest having two different API methods instead - initDictConfig > and updateDictConfig. Or is there some specific reason for having the > choice of incremental update vs replacement configuration in the hands > of the config author rather than the application developer? I was thinking that in some environments, configuration changes might be sent as pickled dicts over the wire. In such cases it might be useful to have the "incremental" flag in the payload. But I haven't thought that through in detail, as it's somewhat peripheral to this feature. > My other question is whether or not it would be worth having the logging > module able to export it's *current* configuration in dictionary form. I > don't have a use case other than that it might be useful for debugging > logging configuration errors when attempting to combine multiple sources > of logging data, but it's worth considering. Another fair point - perhaps a getConfig() which returns a dict can be added. > You may want to allow yourself AttributeError and ImportError here. > Alternatively, you could commit to converting those to an appropriate > ValueError that provides more detail on the config setting that > contained the problematic reference. Perhaps TypeError, too. The idea is to definitely pinpoint where in the configuration the error occurred. I'll think about the detail of this a bit further down the line, but I just wanted to establish that I wasn't planning on introducing any logging-related exception classes. Thanks for the quick response and detailed feedback. Regards, Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] A proposal: configuring logging using dictionaries
Vinay Sajip yahoo.co.uk> writes: > Perhaps TypeError, too. Sorry, brainfade there. I already mentioned TypeError in the original post :-) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] transitioning from % to {} formatting
Michael Foord wrote: Benjamin Peterson wrote: 2009/10/5 Nick Coghlan : So I would agree that method invocation on literals (particularly string literals) is an already established language idiom. And who hasn't ever used 4.56.as_integer_ratio()? :) > I've tried 4.__add__ a few times (not for a while now though). That's a Syntax Error >>> 4.__add__ File "", line 1 4.__add__ ^ SyntaxError: invalid syntax however this works: >>> 4 .__add__ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
