Re: Scrapy/XPath help

[donarb]

On Friday, December 21, 2012 1:58:47 PM UTC-8, Always Learning wrote:
> The errors I get are
> >>File 
> >>"C:\python27\lib\site-packages\scrapy-0.16.3-py2.7.egg\scrapy\selector\lxmlsel.py",
> >> line 47, in select
> 
> >>raise ValueError("Invalid XPath: %s" % xpath)
> 
> >>exceptions.ValueError: Invalid XPath: /span[@class='team-name'/text()
> 


You're missing a right bracket in the xpath expression:

/span[@class='team-name']/text()
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Evaluate postgres boolean field

[donarb]

On Friday, January 4, 2013 10:08:22 AM UTC-8, andyd...@gmail.com wrote:
> Hi,
> 
> I'm hoping for some help on a python script I need to query an api. I'm not a 
> (Python) programmer ordinarily, but do plan to improve!
> 
> Specifically I have a for loop evaluating a database row, which I think I can 
> treat as a list. My [4] is a postgres boolean field, and I'm temporarily 
> stuck on how to evaluate this to determine if I use the values in [1].
> 
> Could I have some advice on what to change? Also do let me know if you can 
> recommend a good beginners python book.
> 
> Data example:
> 
> [13, 'Barbican Station', 'Barbican Station, London Underground Ltd., 
> Aldersgate St, London, EC1A 4JA', 
> '0101E0E6108851AB9E9803B9BF5BB6972294C24940',
>  True]
> 
> 
> Code:
> 
> #!/usr/bin/python
> import psycopg2
> #note that we have to import the Psycopg2 extras library!
> import psycopg2.extras
> import sys
>  
> def main():
> conn_string = "host='localhost' dbname='gisdb' user='postgres' 
> password='#'"
> # print the connection string we will use to connect
> print "Connecting to database\n->%s" % (conn_string)
>  
> conn = psycopg2.connect(conn_string)
>  
> # HERE IS THE IMPORTANT PART, by specifying a name for the cursor
> # psycopg2 creates a server-side cursor, which prevents all of the
> # records from being downloaded at once from the server.
> cursor = conn.cursor('cursor_tube', 
> cursor_factory=psycopg2.extras.DictCursor)
> cursor.execute('SELECT * FROM tubestations LIMIT 1000')
>  
> # Because cursor objects are iterable we can just call 'for - in' on
> # the cursor object and the cursor will automatically advance itself
> # each iteration.
> # This loop should run 1000 times, assuming there are at least 1000
> # records in 'my_table'
> row_count = 0
> for row in cursor:
> row_count += 1
> if row[4] = True
> print row[1]
> #print "row: %s%s\n" % (row_count, row)
>  
> if __name__ == "__main__":
> main()
> 
> Thanks!
> 
> 
> Andy

Your code is pretty close to working, you just need to make a couple 
modifications. You are using the equals sign as an assignment, not a 
comparison, although the comparison and value are unnecessary since the field's 
value is either true or false. And you're missing a colon at the end of the 
condition. Note also that since you are using a DictCursor you can use column 
names to reference your row's fields, I guessed on the field names, but you 
should get the idea.


for row in cursor:
row_count += 1
if row['active']:
print row['name']
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Is there a more elegant way to handle determing fail status?

[donarb]

On Tuesday, January 15, 2013 3:24:44 PM UTC-8, J wrote:
> Ok, so I have a diagnostic tool, written by someone else. That tool
> 
> runs a series of small tests defined by the user and can simplified
> 
> summary output that can be one of the following:
> 
> 
> 
> FAILED_CRITICAL
> 
> FAILED_HIGH
> 
> FAILED_MEDIUM
> 
> FAILED_LOW
> 
> PASSED
> 
> 
> 
> I also have a wrapper script I wrote to run these tests, summarize the
> 
> results of all tests aggregated and then fail based on a particular
> 
> fail level.
> 
> 
> 
> The idea is that if I run 3 tests with the diagnostic tool and it
> 
> tells me the following:
> 
> 
> 
> testA: PASSED
> 
> testB: FAILED_MEDIUM
> 
> testC: PASSED
> 
> 
> 
> AND I told the wrapper to only fail on HIGH or above, the wrapper will
> 
> tell me that I had a medium failure, but the wrapper will still exit
> 
> with a 0 (success)
> 
> 
> 
> if I get the same results as above, but tell the wrapper to fail on
> 
> LOW, then it will tell me I had that medium failure, but the wrapper
> 
> will exit with a 1 (failure).
> 
> 
> 
> The problem is that my exit determination looks like this:
> 
> 
> 
> if fail_priority == fail_levels['FAILED_CRITICAL']:
> 
> if critical_fails:
> 
> return 1
> 
> if fail_priority == fail_levels['FAILED_HIGH']:
> 
> if critical_fails or high_fails:
> 
> return 1
> 
> if fail_priority == fail_levels['FAILED_MEDIUM']:
> 
> if critical_fails or high_fails or medium_fails:
> 
> return 1
> 
> if fail_priority == fail_levels['FAILED_LOW']:
> 
> if critical_fails or high_fails or medium_fails or low_fails:
> 
> return 1
> 
> 
> 
> return 0
> 
> 
> 
> So, to explain the above... the fail level can be set by the user when
> 
> running the wrapper using -f (or it defaults to 'high')
> 
> the wrapper assigns a number to each level using this:
> 
> 
> 
> # Set correct fail level
> 
> args.fail_level = 'FAILED_%s' % args.fail_level.upper()
> 
> 
> 
> # Get our failure priority and create the priority values
> 
> fail_levels = {'FAILED_CRITICAL':4,
> 
>'FAILED_HIGH':3,
> 
>'FAILED_MEDIUM':2,
> 
>'FAILED_LOW':1}
> 
> fail_priority = fail_levels[args.fail_level]
> 
> 
> 
> the variables critical_fails, high_fails, medium_fails, low_fails are
> 
> all counters that are etiher None, or the number of tests that were
> 
> failed.
> 
> 
> 
> So using this output from the diagnostic tool:
> 
> 
> 
> testA: PASSED
> 
> testB: FAILED_HIGH
> 
> testC: PASSED
> 
> testD: FAILED_MEDIUM
> 
> testE: PASSED
> 
> 
> 
> critical_fails would be None
> 
> high_fails would be 1
> 
> medium_fails would be 1
> 
> low_fails would be None.
> 
> 
> 
> The exit code determination above works, but it just feels inelegant.
> 
> It feels like there's a better way of implementing that, but I can't
> 
> come up with one that still honors the fail level properly (e.g. other
> 
> solutions will fail on medium, but won't fail properly on medium OR
> 
> higher).
> 
> 
> 
> I can provide the full script if necessary, if the above isn't enough
> 
> to point me in a direction that has a better way of doing this...
> 
> 
> 
> Thanks for looking,
> 
> 
> 
> Jeff

My back of the envelope coding would do it this way. Use an array of 
fail_counters, with PASSED as the first element all the way up to 
FAILED_CRITICAL as the last element. Then use a simple loop starting from index 
fail_priority to the end of the list looking for errors. Like this:

# Array of fail counters
fail_counters = [
0,  # PASSED
0,  # LOW
0,  # MEDIUM
0,  # HIGH
0   # CRITICAL
]

... run tests, accumulate error counts in fail_counters

for i in range(fail_priority, len(fail_counters)):
if fail_counters[i]:
return 1
return 0


-- 
http://mail.python.org/mailman/listinfo/python-list

PyWart: Exception error paths far too verbose

[donarb]

Done

https://github.com/erikrose/tracefront

This module displays traces with shortened paths, and will even prepend your 
editor of choice and line number to the path, making a shortcut to jumping to 
the source in error via copy/paste.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Thorough Python 2.7.3 Windows Build Documentation?

[donarb]

On Jan 17, 7:29 am, "Leonard, Arah" 
wrote:
> Hello fellow Python programmers,
>
> I'm building a 32-bit CPython 2.7.3 distro for Windows using the MS Visual 
> Studio Professional 2008 SP1 (and all hotfixes) MSVC 9 compiler.  My build 
> works, technically, but it also happens to benchmark over 30% slower than the 
> precompiled binaries in the distributed Python 2.7.3 MSI.  Can anyone point 
> me in the direction of some thoroughly detailed build documentation so that I 
> can figure out how to get that 30% back with my build?  The only 
> documentation that I can find just says MSVC 9, period.  There's no mention 
> of SP1 or not, hotfixes, nor of any specific compiler/linker optimizations 
> used to build the official distro.  Something, somewhere, has to be 
> significantly different between our builds for a 30% performance difference, 
> and it'd probably be handy for the Python community to know how to avoid the 
> same pitfall that cost me performance so that we can all get the most out of 
> Python.  Any and all help will be greatly appreciated.  Thanks.
>
> Sincerely,
> Arah Leonard
>
> Arah Leonard
> Software Development Engineer
>
> Bruker AXS Inc.
> 5465 East Cheryl Parkway
> Madison, WI 53711
> US       Phone: +1 608-276-3812
>  Phone: +1 800-234-XRAY(9729)
>  Fax:
>
>   arah.leon...@bruker-axs.com
>  www.bruker.com
>
> 
>
> The information contained in this email is confidential. It is intended 
> solely for the addressee. Access to this email by anyone else is 
> unauthorized. If you are not the intended recipient, any form of disclosure, 
> reproduction, distribution or any action taken or refrained from in reliance 
> on it, is prohibited and may be unlawful. Please notify the sender 
> immediately.

Try dumping the build configuration parameters:

   >>> import pprint, sysconfig
   >>> pprint.pprint(sysconfig.get_config_vars())

Then you can compare the existing version with yours.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Output JSON-schema from bottle application?

[donarb]

On Thursday, February 27, 2014 7:49:07 PM UTC-8, Alec Taylor wrote:
> Are there libraries for doing this?
> 
> I would like to autogenerate JSON-schema for use inside an API explorer.
> 
> However whenever there is a schema change; I would only like to change
> the schema in one place (where possible).
> 
> E.g.: For use here - https://github.com/salesking/json-schema-browser
> 
> How do I do this?
> 
> Thanks for all suggestions,
> 
> Alec Taylor

Found this on PyPi, can't say if it's suitable for your requirements.

https://pypi.python.org/pypi/jsonschema
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: modification time in Python - Django: datetime != datetime :-(

[donarb]

On Monday, March 3, 2014 6:28:21 AM UTC-8, Jaap van Wingerde wrote:
> Op  schreef Chris Angelico  
> in bericht 
> :
> 
> > See if ls is actually giving you ctime rather than mtime - compare the
> > results if you ask for os.path.getctime.
> 
> jaap@liakoster:~$ python
> Python 2.7.3 (default, Jan  2 2013, 13:56:14) 
> [GCC 4.7.2] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
> >>> import os, time
> >>> from time import gmtime
> >>> time.strftime('%Y-%m-%dT%H:%m:%SZ',gmtime(os.path.getmtime('/var/django/test2/art/templates/art_index.html')))
> '2014-03-02T19:03:55Z'
> >>> time.strftime('%Y-%m-%dT%H:%m:%SZ',gmtime(os.path.getctime('/var/django/test2/art/templates/art_index.html')))
> '2014-03-02T19:03:55Z'
> >>> quit()
> jaap@liakoster:~$ ls --full-time 
> /var/django/test2/art/templates/art_index.html
> -rwxrwx--- 1 lia www-data 2456 2014-03-02 19:16:55.568139590 + 
> /var/django/test2/art/templates/art_index.html
> jaap@liakoster:~$ 
> 
> ls is giving me the modified time.

You're using the months format '%m' when you should be using minutes '%M'.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: modification time in Python - Django: datetime != datetime :-(

[donarb]

Note that it's bad form to post the same question to different forums, you also 
posted this question to django-users. By posting to multiple forums, you run 
the risk of not having the question answered or followed up in one of the 
forums. This frustrates other users who may one day have a similar problem and 
find your orphaned question.

Remember, these forums are not just about you, they are also about those who 
follow later.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to parse JSON passed on the command line?

[donarb]

On Wednesday, November 6, 2013 10:09:49 PM UTC-8, yupeng zhang wrote:
> 
> Hi Anthony Papillion.
> 
> I'm fresh to Python, but I do love its short simple and graceful.
> I've solved your problem. You could try the code below:
> 
> getargfromcli.py "\"{'url':'http://www.google.com'}\""
> 
> AS command line will strip your ".
> 
> From the Python document, we could get the info as:
> json.loads('["foo", {"bar":["baz", null, 1.0, 2]}]')
> the json.loads' argument should be string. Try it:)

That's not going to work, JSON strings must use double quotes, which you've 
rewritten as single quotes. The correct way (as shown previously) is to wrap 
the entire string in single quotes, thus preserving the double quotes inside.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: OT: How to tell an HTTP client to limit parallel connections?

[donarb]

On Friday, November 8, 2013 9:25:30 AM UTC-8, Grant Edwards wrote:
> Yes, this off-topic, but after a fair amount of Googling and searching
> in the "right" places, I'm running out of ideas.
> 
> I've got a very feeble web server.  The crypto handshaking involved in
> opening an https: connection takes 2-3 seconds.  That would be fine if
> a browser opened a single connection and then sent a series of
> requests on that connection to load the various elements on a page.
> 
> But that's not what browsers do.  They all seem to open whole handful
> of connections (often as many as 8-10) and try to load all the page's
> elements in parallel.  That turns what would be a 3-4 second page load
> time (using a single connection) into a 20-30 second page load time.
> Even with plaintext http: connections, the multi-connection page load
> time is slower than the single-connection load time, but not by as
> large a factor.
> 
> Some browsers have user-preference settings that limit the max number
> of simultaneous connections to a single server (IIRC the RFCs suggest
> a max of 4, but most browsers seem to default to a max of 8-16).
> 
> What I really need is an HTTP header or meta-tag or something that I
> can use to tell clients to limit themselves to a single connection.
> 
> I haven't been able to find such a thing, but I'm hoping I've
> overlooked something...
> 

There's an Apache module called mod_limitipconn that does just what you are 
asking. I've never used it and can't vouch for it. I'm not aware of anything 
like this for other servers like Nginx.

http://dominia.org/djao/limitipconn2.html
-- 
https://mail.python.org/mailman/listinfo/python-list