Override path to python during installation (setuptools)

[Yegor Yefremov]

I'm trying to add lava-test package to Buildroot. After "cross-compilation" I 
get following path in the /usr/bin/lava:

#!/home/user/MyProjects/versioned/buildroot/output/host/usr/bin/python

instead of #!/usr/bin/python

I use following command to install lava_test package:

+define LAVA_TEST_INSTALL_TARGET_CMDS
+  (cd $(@D); 
PYTHONPATH="$(TARGET_DIR)/usr/lib/python$(PYTHON_VERSION_MAJOR)/site-packages"\
+  $(HOST_DIR)/usr/bin/python setup.py install --prefix=$(TARGET_DIR)/usr)
+endef

How can I override this auto-generated path?

The whole patch can be seen here: http://patchwork.ozlabs.org/patch/155498/

Regards,
Yegor
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: new to Python - modules to leverage Perl scripts?

[Peter Otten]

Rogelio wrote:

> I've got quite a few Perl scripts that I would like to leverage, and
> I'd like to make some Python wrapper scripts for them.
> 
> The Perl scripts shell into various network appliances, run certain
> commands, and then output those commands into a file.
> 
> I recently found out about the subprocess modules (including "call")
> and am wondering what other Python libraries and modules I should
> check out.  Specifically, I need functions that will do the
> following...
> 
> 1) Take a big IP file and break it up into smaller chunks (e.g. 20 IP
> addresses per file)

itertools.islice()

> 2) Run the Perl script on those smaller IP files

subprocess. If the script can read from stdin you may not need the 
intermediate file; use 

subprocess.Popen(..., stdin=PIPE).communicate(inputdata_as_a_string)

instead.

> 3) Run the Perl commands in parallel (to take less time)

multiprocessing? Though your problem may be I/O-bound.

> 4) Take the little logs and concatenate the files

itertools.chain.from_iterable()

> 5) Look for existence of certain strings in the logs

with open(logfile) as f:
for line in f:
if some_string in line:
# whatever

If you know Perl you will not be completely puzzled when you have to switch 
to a regex. You'll rather make that switch to early ;)
 
> I'm still playing with the subprocess.call command to make sure that
> it calls everything okay.  But I'm hoping someone can point me where
> to look for functions that will make this an effective wrapper script.

You sound like you are not completely satisfied with subprocess. What are 
you missing?

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: new to Python - modules to leverage Perl scripts?

[Cameron Simpson]

On 07May2012 09:16, Peter Otten <__pete...@web.de> wrote:
| Rogelio wrote:
| > I've got quite a few Perl scripts that I would like to leverage, and
| > I'd like to make some Python wrapper scripts for them.
| > 
| > The Perl scripts shell into various network appliances, run certain
| > commands, and then output those commands into a file.
| > 
| > I recently found out about the subprocess modules (including "call")
| > and am wondering what other Python libraries and modules I should
| > check out.  Specifically, I need functions that will do the
| > following...
[...]
| > 2) Run the Perl script on those smaller IP files
| 
| subprocess. If the script can read from stdin you may not need the 
| intermediate file; use 
| 
| subprocess.Popen(..., stdin=PIPE).communicate(inputdata_as_a_string)
| 
| instead.
| 
| > 3) Run the Perl commands in parallel (to take less time)
| 
| multiprocessing? Though your problem may be I/O-bound.

If he stuffs the inputs into TemporaryFiles he can attach them to
subprocess.Popen() stdin and just not wait. Dispatch them all and collect
afterwards. Only tempfile and subprocess needed; no need for
communicate.

Cheers,
-- 
Cameron Simpson  DoD#743
http://www.cskk.ezoshosting.com/cs/

One of the most important things you learn from the internet is that there is
no 'them' out there. It's just an awful lot of 'us'. - Douglas Adams
-- 
http://mail.python.org/mailman/listinfo/python-list

Scrapy - importing files from local, rather than www

[nbw]

Hi everyone, I'm new to Python (loving it!) and Scrapy. I have a
question I just can't seem to get my head around. I can get a simple
Scrapy spider to pick up URLs and download them fine, but the HTML
files I have are stored locally. The reason for this, is for some
reason when I "Save As" the pages I get everything, whereas if Scrapy
runs over them it seems to miss certain areas where there's
Javascript.

So, I have them sitting in a directory (C:/scrapy_test) but can't for
the life of me get Scrapy to find them. Is there anyone who's had this
problem and solved it, or can help?

Any help is much appreciated.
Kind regards,
nbw
-- 
http://mail.python.org/mailman/listinfo/python-list

problem in event handling on change of variable value.

[Nadhiya A]

Hi,

I want know how can I control Canoe tool from Python. I want to use Python as 
the mediator for linking CANoe and Matlab.
Please give me some inputs regarding this.

Thanks & Regards
Nadhiya.A


Larsen & Toubro Limited

www.larsentoubro.com

This Email may contain confidential or privileged information for the intended 
recipient (s). If you are not the intended recipient, please do not use or 
disseminate the information, notify the sender and delete it from your system.

 Earth Day. Every Day.
-- 
http://mail.python.org/mailman/listinfo/python-list

Workshop on Design Patterns and Advanced OOPS, Bangalore

[Exam Aya]

Workshop on

Design Patterns and Advanced OOPS

Date: Sunday , 13th May, 2012

At Bangalore

Hurry Register Online Now
https://docs.google.com/spreadsheet/viewform?formkey=dGlMMnEtYmJVY2M3bEhNZG5WY3BHTkE6MA#gid=0


About the workshop:

For anyone working on object oriented programing languages like C++,
Java, C#, Python, PHP, etc. this workshop is a must. It goes through
the set of design patterns that started it all - Go4 - (Gang of 4)
patterns. The workshop will very interactive and each design pattern
will be explored rather than taught or lectured on. A problem
statement will be provided and the group will move towards why the
design pattern is required. This will not only drill into you why use
a pattern in the first place but also which pattern to use in what
scenario.


Course Contents (Day 1)

1. The Pit Falls of OOPS
- Refresher on OOPS
- When does OOPS get confusing

2. Design Pattern Intro
- What are Design Patterns
- What are its uses and applications

3. Design Patterns
- Strategy Pattern
- Observer Pattern


Course Contents (Day 1)
- Decorator Pattern
- Factory Pattern
- Singleton Pattern
- Command Pattern
- Adapter Pattern
- Facade Pattern
- Template Pattern
- Iterator Pattern
- Composite Pattern
- State Pattern


Time : 09:30 am to 05:30 pm

Venue:

1910, 8th A Cross (road connects 4th Main and 6th Main), HAL 3rd Stage
(New Thippasandira), Bangalore - 75


Email eve...@examaya.com

Website: www.examaya.com


Course Fees: Rs. 750

Cost of Course Materials, Refreshments and Lunch

Hurry Register Online Now!
https://docs.google.com/spreadsheet/viewform?formkey=dGlMMnEtYmJVY2M3bEhNZG5WY3BHTkE6MA#gid=0

About the Speaker:

Sethuraman is a passionate Java programmer who is currently working in
Thoughtworks. He has a total of 8 years of experience in the industry
and is a very "hands on guy". He is known in the professional circle
for his excellent communication
-- 
http://mail.python.org/mailman/listinfo/python-list

return respective values when mutiple keys are passed in dictionary

[Nikhil Verma]

HI All

I was clearing my concepts on dictionary and stuck in this problem.
I have a dictionary which i have formed by using zip function on two list
so that one list (which i have hardcoded) becomes the keys and the other
list
becomes its values.

Now i want to know how can i get the values of keys at once if i pass the
keys in a dictionary.

Let say I have a dictionary

mydict = {'a':'apple' , 'b':'boy' ,'c' : 'cat', 'd':'duck','e':'egg'}

Now if i do :-

mydict.get('a')
'apple'

What i want is some i pass keys in get and in return i should have all the
values of those keys which i pass.

##
mydict.get('a','b','c')###demo for what i want
'apple','boy','cat'### Output i want
#

-- 
Regards
Nikhil Verma
+91-958-273-3156
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: return respective values when mutiple keys are passed in dictionary

[Виталий Волков]

You can try to use map(mydict.get, ('a', 'b', 'c')) and then make join
On May 7, 2012 2:33 PM, "Nikhil Verma"  wrote:

> HI All
>
> I was clearing my concepts on dictionary and stuck in this problem.
> I have a dictionary which i have formed by using zip function on two list
> so that one list (which i have hardcoded) becomes the keys and the other
> list
> becomes its values.
>
> Now i want to know how can i get the values of keys at once if i pass the
> keys in a dictionary.
>
> Let say I have a dictionary
>
> mydict = {'a':'apple' , 'b':'boy' ,'c' : 'cat', 'd':'duck','e':'egg'}
>
> Now if i do :-
>
> mydict.get('a')
> 'apple'
>
> What i want is some i pass keys in get and in return i should have all the
> values of those keys which i pass.
>
> ##
> mydict.get('a','b','c')###demo for what i want
> 'apple','boy','cat'### Output i want
> #
>
> --
> Regards
> Nikhil Verma
> +91-958-273-3156
>
>
> --
> http://mail.python.org/mailman/listinfo/python-list
>
>
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: [SOLVED] for-loop: weird behavior

[ferreirafm]

The problem has been solved opening the file for iteration in the second
loop. I didn't realized such Python behavior.
Any comments are appreciated.
Fred  

--
View this message in context: 
http://python.6.n6.nabble.com/SOLVED-for-loop-weird-behavior-tp4953214p4957597.html
Sent from the Python - python-list mailing list archive at Nabble.com.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PyTextile Question

[dinkypumpkin]

On Friday, 4 May 2012 04:39:47 UTC+1, Josh English  wrote:
> However, when I convert my multiple-paragraph text object with textile, my 
> original line breaks are preserved. Since I'm going to HTML, I d'nt want my 
> line breaks preserved.

I think any Textile implementation will preserve line breaks within a paragraph 
by converting them to BR tags.  Both RedCloth and PyTextile do, anyway.

> I've tried several ways of pre-processing the text in the node, but pytextile 
> still gives me line breaks.

Below is a test script that shows one way I've dealt with this issue in the 
past by reformatting paragraphs to remove embedded line breaks.  YMMV.

import re, textile
print "INPUT1:"
s1 = """This is a long multi-line description 
with several paragraphs and hopefully, eventually, 
proper HTML P-tags. 

This is a new paragraph. It should be surrounded 
by its own P-tag. 

Hopefully (again), I won't have a bunch of unwanted 
BR tags thrown in."""
print(s1)
print "OUTPUT1:"
html1 = textile.textile(s1)
print(html1)
print "INPUT2:"
s2 = re.sub(r'[ \t]*\n[ \t]*(\n?)[ \t]*', r' \1\1', s1, flags=re.MULTILINE)
print(s2)
print "OUTPUT2:"
html2 = textile.textile(s2)
print(html2)

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: return respective values when mutiple keys are passed in dictionary

[Chris Angelico]

On Mon, May 7, 2012 at 9:31 PM, Nikhil Verma  wrote:
> mydict = {'a':'apple' , 'b':'boy' ,'c' : 'cat', 'd':'duck','e':'egg'}
>
> Now if i do :-
>
> mydict.get('a')
> 'apple'
>
> What i want is some i pass keys in get and in return i should have all the
> values of those keys which i pass.
>
> ##
> mydict.get('a','b','c')    ###demo for what i want
> 'apple','boy','cat'    ### Output i want
> #

Presumably you want to get back a list or tuple, so a list
comprehension is your best bet.

[mydict.get(i) for i in ('a','b','c')]

Incidentally, are you aware that dictionaries can be subscripted? The
get() method is good when you want a default value for anything that
doesn't exist, otherwise you can simply use:

mydict['a']

ChrisA
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: return respective values when mutiple keys are passed in dictionary

[Arnaud Delobelle]

On 7 May 2012 12:31, Nikhil Verma  wrote:
> HI All
>
> I was clearing my concepts on dictionary and stuck in this problem.
> I have a dictionary which i have formed by using zip function on two list so
> that one list (which i have hardcoded) becomes the keys and the other list
> becomes its values.
>
> Now i want to know how can i get the values of keys at once if i pass the
> keys in a dictionary.
>
> Let say I have a dictionary
>
> mydict = {'a':'apple' , 'b':'boy' ,'c' : 'cat', 'd':'duck','e':'egg'}
>
> Now if i do :-
>
> mydict.get('a')
> 'apple'

mydict['a'] is the usual way to get the value associated with a key.
The difference is that it will throw an exception if the key doesn't
exist, which is most of the time the sanest thing to do.

> What i want is some i pass keys in get and in return i should have all the
> values of those keys which i pass.
>
> ##
> mydict.get('a','b','c')    ###demo for what i want
> 'apple','boy','cat'    ### Output i want
> #

1. You can use a list comprehension

>>> [mydict[k] for k in 'a', 'b', 'c']
['apple', 'boy', 'cat']

2. You can use map (for python 3.X, you need to wrap this in list(...))

>>> map(mydict.__getitem__, ['a', 'b', 'c'])
['apple', 'boy', 'cat']

3. You can use operator.itemgetter

>>> from operator import itemgetter
>>> itemgetter('a', 'b', 'c')(mydict)
('apple', 'boy', 'cat')

-- 
Arnaud
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: key/value store optimized for disk storage

[Steve Howell]

On May 6, 10:21 pm, John Nagle  wrote:
> On 5/4/2012 12:14 AM, Steve Howell wrote:
>
> > On May 3, 11:59 pm, Paul Rubin  wrote:
> >> Steve Howell  writes:
> >>>      compressor = zlib.compressobj()
> >>>      s = compressor.compress("foobar")
> >>>      s += compressor.flush(zlib.Z_SYNC_FLUSH)
>
> >>>      s_start = s
> >>>      compressor2 = compressor.copy()
>
>     That's awful. There's no point in compressing six characters
> with zlib.  Zlib has a minimum overhead of 11 bytes.  You just
> made the data bigger.
>

The actual strings that I'm compressing are much longer than six
characters. Obviously, "foobar" was just for example purposes.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Workshop on Design Patterns and Advanced OOPS, Bangalore

[Terry Reedy]

On 5/7/2012 6:23 AM, Exam Aya wrote:


For anyone working on object oriented programing languages like C++,
Java, C#, Python, PHP, etc. this workshop is a must. It goes through
the set of design patterns that started it all - Go4 - (Gang of 4)
patterns. The workshop will very interactive and each design pattern
will be explored rather than taught or lectured on. A problem
statement will be provided and the group will move towards why the
design pattern is required.


I hope someone who really knows Python attends, so people can discover 
how easy some of the patterns are in Python, which does not have the 
straightjacket of statically typed names. For instance, heterogeneous 
sets and lists, used for instance for composites consisting of items and 
sub-composites, are part of Python rather than a tricky add-on.



Course Contents (Day 1)

1. The Pit Falls of OOPS
- Refresher on OOPS
- When does OOPS get confusing

2. Design Pattern Intro
- What are Design Patterns
- What are its uses and applications

3. Design Patterns
- Strategy Pattern
- Observer Pattern


Course Contents (Day 1)
- Decorator Pattern
- Factory Pattern
- Singleton Pattern
- Command Pattern
- Adapter Pattern
- Facade Pattern
- Template Pattern
- Iterator Pattern
- Composite Pattern
- State Pattern


Time : 09:30 am to 05:30 pm

Venue:

1910, 8th A Cross (road connects 4th Main and 6th Main), HAL 3rd Stage
(New Thippasandira), Bangalore - 75


If I lived in Bangalore, I might attend. This announcement is, however, 
a bit OT for this list. On the other hand, it is encouraging that Python 
is now routinely listed as a major language in such things. Not so when 
I learned it 16 years ago ;-).


--
Terry Jan Reedy

--
http://mail.python.org/mailman/listinfo/python-list

Re: sorting 1172026 entries

[Cameron Simpson]

On 07May2012 11:02, Chris Angelico  wrote:
| On Mon, May 7, 2012 at 10:31 AM, Cameron Simpson  wrote:
| > I didn't mean per .append() call (which I'd expect to be O(n) for large
| > n), I meant overall for the completed list.
| >
| > Don't the realloc()s make it O(n^2) overall for large n? The list
| > must get copied when the underlying space fills. I confess to being
| > surprised at how quick it went for me though. I suppose reallocing in
| > chunks (eg to double the available size) might conceal this for a while.
| > It should still be well over O(n) overall (not amortised per .append()
| > call).
| 
| I haven't checked the CPython code, but the usual way to do
| reallocations is to double the size

My example above:-)

| (or add 50% or something) each
| time, meaning that as n increases, the frequency of reallocations
| decreases - hence the O(1) amortized time.

Hmm, yes. But it is only O(1) for doubling. If one went with a smaller
increment (to waste less space in the end case where one stops growing the
array) then there are more copies and less .append efficiency, trading
potential memory bloat for compute time in .append(). If you went all
the way to adding only one item the cost goes to O(n) for .append()
and O(n^2) overall, with varying costs in between.

| In any case, it's the
| recommended way to build large strings:
| 
| s = ""
| while condition:
| s += "some_value"
| 
| versus:
| 
| s = []
| while condition:
| s.append("some_value")
| s = "".join(s)
| 
| so I would be very much surprised if this common wisdom is merely
| replacing one O(n*n) operation with another O(n*n) operation.

It sort of is (except that list.append may be specially tuned to realloc
in big chunks). The join-lots-of-strings standard example is based on
the length of the strings in characters/bytes being far more than the
number of strings. So you do a lot of tiny listappends instead, copying
nothing, then a big allocate-the-correct-size-and-copy-once with s.join.

Also, this:

  s += "some_value"

is not a list extension. Because strings are immutable (and anyway in
general for "+" in python) you're making a new string and copying two
other string contents into it. Inherently a copy of the whole of "s" on
every "+" operation. Compared with list.append() or list.extend(), which
are in-place modifications to the original list.

So these are really apples and oranges here.
-- 
Cameron Simpson  DoD#743
http://www.cskk.ezoshosting.com/cs/

We are in great haste to construct a magnetic telegraph from Maine to Texas;
but Maine and Texas, it may be, have nothing important to communicate.
- H. D. Thoreau
-- 
http://mail.python.org/mailman/listinfo/python-list

try/except KeyboardInterrupt vs signal handler for SIGINT

[Rafael Durán Castañeda]

Hi,

I was wondering which approach is better to stop several servers from 
just on python application when 'ctrl-c' is pressed, a try/except 
KeyboardInterrupt or just registering a SIGINT handler and exit when the 
signal is triggered. Any advantage/disadvantage from one approach over 
the other?


P.S.: Since several servers are started just one 'ctrl-c' doesn't work 
without any extra change and I have some concerns about the signal handler.


Thank you for your answers.
--
http://mail.python.org/mailman/listinfo/python-list

RELEASED: Pymacs 0.25

[François Pinard]

Hello to everybody, and Emacs users in the Python community.

Pymacs 0.25 is now available.  There has been a while, so I advise current 
Pymacs
users to switch with caution.

- Python 3 is now supported.  This required new installation mechanics,
  and a Python pre-processor written for the circumstance (named **).

- Pymacs now installs a single Python file instead of a Python module.

Nice thanks to Pymacs contributors.  It surely has been fun working with you 
all!

--

Pymacs is a powerful tool which, once started from Emacs, allows
both-way communication between Emacs Lisp and Python.  Pymacs aims
Python as an extension language for Emacs rather than the other way
around, and this asymmetry is reflected in some design choices.  Within
Emacs Lisp code, one may load and use Python modules.  Python functions
may themselves use Emacs services, and handle Emacs Lisp objects kept in
Emacs Lisp space.

For more information, see http://pymacs.progiciels-bpi.ca/ .  You may
fetch the distribution as one of:

  - https://github.com/pinard/Pymacs/tarball/v0.25

  - https://github.com/pinard/Pymacs/zipball/v0.25

-- 
François Pinard   http://pinard.progiciels-bpi.ca
-- 
http://mail.python.org/mailman/listinfo/python-list

How do I run a python program from an internet address?

[Albert]

I have a small text based python program that I want to make available
to people who might be behind a firewall or can't install python on
their office computers, but can access the internet.  It is just an
algorithm that makes a handful of straightforward calculations on some
input that the user provides and spits out some text as output that
they might want to print out on a printer.  I can program python on my
local machine, but don't know how to make the code accessible from a
browser.

What would be the best way to figure out how to do this?  I looked at
Google app engine tutorial, but can't figure out how that will help we
get the code into the cloud so I can access it from any browser.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How do I run a python program from an internet address?

[Chris Angelico]

On Tue, May 8, 2012 at 9:20 AM, Albert  wrote:
> I have a small text based python program that I want to make available
> to people who might be behind a firewall or can't install python on
> their office computers, but can access the internet.  It is just an
> algorithm that makes a handful of straightforward calculations on some
> input that the user provides and spits out some text as output that
> they might want to print out on a printer.  I can program python on my
> local machine, but don't know how to make the code accessible from a
> browser.

Probably the easiest way is to make your program into a web server itself:

http://docs.python.org/py3k/library/http.server.html

Put it on a high port (eg 8000) and then direct people to
http://your.machine.ip.address:8000/ to access it.

There are other ways, too, such as CGI scripting, but the http.server
module is pretty convenient.

ChrisA
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How do I run a python program from an internet address?

[Steven D'Aprano]

On Mon, 07 May 2012 16:20:35 -0700, Albert wrote:

> I have a small text based python program that I want to make available
> to people who might be behind a firewall or can't install python on
> their office computers, but can access the internet.  It is just an
> algorithm that makes a handful of straightforward calculations on some
> input that the user provides and spits out some text as output that they
> might want to print out on a printer.  I can program python on my local
> machine, but don't know how to make the code accessible from a browser.
> 
> What would be the best way to figure out how to do this?

Try googling "python web app how to". For example, the very first link 
found here:

https://duckduckgo.com/html/?q=python%20web%20app%20how%20to

is an introduction to making Python applications available to run over 
the Internet.



-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

indexed property? Can it be done?

[Charles Hixson]

class Node:

def__init__(self, nodeId, key, value, downRight, downLeft, parent):
dirty=True
dlu=utcnow()
self.node=[nodeId, downLeft, [key], [value], 
[downRight], parent, dirty, dlu]


Note that node[3] is a list of keys (initially 1) and node[3] is a list 
of values, etc.


What I'd like to do is to be able to address them thusly:
k = node.key[2]
v = node.value[2]
but if there's a way to do this, I haven't been able to figure it out.  
Any suggestions?


--
Charles Hixson

--
http://mail.python.org/mailman/listinfo/python-list

Re: indexed property? Can it be done?

[Chris Rebert]

On Mon, May 7, 2012 at 8:15 PM, Charles Hixson
 wrote:
> class Node:
>
>    def    __init__(self, nodeId, key, value, downRight, downLeft, parent):
>        dirty    =    True
>        dlu    =    utcnow()
>        self.node    =    [nodeId, downLeft, [key], [value], [downRight],
> parent, dirty, dlu]

Why are you using a single opaque list instead of separate, meaningful
attributes for each datum?

Cheers,
Chris
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Creating a directory structure and modifying files automatically in Python

[John Nagle]

On 5/6/2012 9:59 PM, Paul Rubin wrote:

Javier  writes:

Or not... Using directories may be a way to do rapid prototyping, and
check quickly how things are going internally, without needing to resort
to complex database interfaces.


dbm and shelve are extremely simple to use.  Using the file system for a
million item db is ridiculous even for prototyping.


   Right.  Steve Bellovin wrote that back when UNIX didn't have any
database programs, let alone free ones.

John Nagle

--
http://mail.python.org/mailman/listinfo/python-list

Open Source: you're doing it wrong - the Pyjamas hijack

[Steven D'Aprano]

A.k.a. "we had to destroy the project in order to save it".

http://technogems.blogspot.com.au/2012/05/pyjamas-hijacked.html


Seriously, this was a remarkably ham-fisted and foolish way to "resolve" 
a dispute over the direction of an open source project. That's the sort 
of thing that gives open source a bad reputation.

(The sad thing is, when closed source software developers do this sort of 
thing, it gets blamed on "bad apples"; when open source developers do it, 
it gets used as an indictment on the entire FOSS community.)



-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: indexed property? Can it be done?

[Dan Sommers]

On Mon, 07 May 2012 20:15:36 -0700
Charles Hixson  wrote:

> class Node:
> 
>  def__init__(self, nodeId, key, value, downRight, downLeft,
> parent): dirty=True
>  dlu=utcnow()
>  self.node=[nodeId, downLeft, [key], [value], 
> [downRight], parent, dirty, dlu]
> 
> Note that node[3] is a list of keys (initially 1) and node[3] is a
> list of values, etc.
> 
> What I'd like to do is to be able to address them thusly:
> k = node.key[2]
> v = node.value[2]
> but if there's a way to do this, I haven't been able to figure it
> out. Any suggestions?

Untested:

def __init__(self, nodeId, key, value, downRight, downLeft, parent):
dirty = True
dlu = utcnow()
self.node = [nodeId, downLeft, dict(key=value),
 [downRight], parent, dirty, dlu]

Now you can use self.node[2][key] to get/set value.

But why not make the elements of node their own attributes?

Untested:

def __init__(self, nodeId, key, value, downRight, downLeft, parent):
self.dirty = True
self.dlu = utcnow()
self.nodeId = nodeId
self.downLeft = downLeft
self.downRight = downRight
self.values = dict(key=value)
self.parent = parent

And then you don't have to remember that node[2] is the key/value pairs
(note the typo (the two "3"s) in your original post).  With each
attribute in its own, well, attribute, you can always use
node.values[key] to access the value associated with a particular key.

HTH,
Dan
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Creating a directory structure and modifying files automatically in Python

[Steve Howell]

On May 7, 8:46 pm, John Nagle  wrote:
> On 5/6/2012 9:59 PM, Paul Rubin wrote:
>
> > Javier  writes:
> >> Or not... Using directories may be a way to do rapid prototyping, and
> >> check quickly how things are going internally, without needing to resort
> >> to complex database interfaces.
>
> > dbm and shelve are extremely simple to use.  Using the file system for a
> > million item db is ridiculous even for prototyping.
>
>     Right.  Steve Bellovin wrote that back when UNIX didn't have any
> database programs, let alone free ones.
>

It's kind of sad that the Unix file system doesn't serve as an
effective key-value store at any kind of nontrivial scale.  It would
simplify a lot of programming if filenames were keys and file contents
were values.




-- 
http://mail.python.org/mailman/listinfo/python-list

Re: indexed property? Can it be done?

[Charles Hixson]

On 05/07/2012 08:33 PM, Chris Rebert wrote:

On Mon, May 7, 2012 at 8:15 PM, Charles Hixson
  wrote:
   

class Node:

def__init__(self, nodeId, key, value, downRight, downLeft, parent):
dirty=True
dlu=utcnow()
self.node=[nodeId, downLeft, [key], [value], [downRight],
parent, dirty, dlu]
 

Why are you using a single opaque list instead of separate, meaningful
attributes for each datum?

Cheers,
Chris

   
Because that's the most reasonable implementation.  The various list 
items would not be the same from instance to instance.  I could pull the 
top-level list items off as separate variables, but this would leave the 
problem exactly where it is, and ordinary properties allow me to address 
the entries like id, and dlu without problem.  But the list variables 
are a separate problem, and not so readily soluble.


FWIW, if I must I can operate with only a list, and define functions to 
do the access and manipulation.  I'd rather not, which is why I'm 
exploring whether an indexed property is feasible.


Note that I *could* return, e.g., the entire keys list, but I'd rather 
use a bit more data protection than that.  Which is what an indexed 
setter property would allow.  And node.key[3] is as fully informative a 
name as I can construct for that entry.  But the number of items in the 
keys list varies from node to node, so defining a key3 property is not 
reasonable.  (Besides, that's so ugly that I'd rather use a function, 
even though indexing in Python really should be done with brackets 
rather than parens.)


--
Charles Hixson

--
http://mail.python.org/mailman/listinfo/python-list

Re: indexed property? Can it be done?

[Charles Hixson]

On 05/07/2012 08:44 PM, Dan Sommers wrote:

On Mon, 07 May 2012 20:15:36 -0700
Charles Hixson  wrote:

   

class Node:

  def__init__(self, nodeId, key, value, downRight, downLeft,
parent): dirty=True
  dlu=utcnow()
  self.node=[nodeId, downLeft, [key], [value],
[downRight], parent, dirty, dlu]

Note that node[3] is a list of keys (initially 1) and node[3] is a
list of values, etc.

What I'd like to do is to be able to address them thusly:
k = node.key[2]
v = node.value[2]
but if there's a way to do this, I haven't been able to figure it
out. Any suggestions?
 

Untested:

def __init__(self, nodeId, key, value, downRight, downLeft, parent):
 dirty = True
 dlu = utcnow()
 self.node = [nodeId, downLeft, dict(key=value),
  [downRight], parent, dirty, dlu]

Now you can use self.node[2][key] to get/set value.

But why not make the elements of node their own attributes?

Untested:

def __init__(self, nodeId, key, value, downRight, downLeft, parent):
 self.dirty = True
 self.dlu = utcnow()
 self.nodeId = nodeId
 self.downLeft = downLeft
 self.downRight = downRight
 self.values = dict(key=value)
 self.parent = parent

And then you don't have to remember that node[2] is the key/value pairs
(note the typo (the two "3"s) in your original post).  With each
attribute in its own, well, attribute, you can always use
node.values[key] to access the value associated with a particular key.

HTH,
Dan
   
Did you notice that the node list contained sublists?  I can access the 
top level node items through ordinary properties, and that is my 
intent.  So I don't need to remember what top level index represents 
what item.  But the second level items are variable in length, so I 
really want to do an indexed access to them.


Yes, I admit that in the snipped defined by the __init__ method those 
secondary lists only received one entry.  Other method would extend 
their length, to a variable amount for different class instances.


A part of the reason that the class retains that top level list is so 
that if I can't create an indexed property to get and set them, I can 
revert to an alternative that is a bit uglier than this, but might be 
more efficient.  (It would have methods that operated directly on the 
list rather than using properties for ANY of the approach...and avoid 
creating a class that they handle.  Not as clean as what I'm hoping for, 
but so far I haven't come up with any way except functions that doesn't 
directly expose the data...and if I must use that approach, then the 
class doesn't buy me anything for the overhead.)


--
Charles Hixson

--
http://mail.python.org/mailman/listinfo/python-list

Re: Creating a directory structure and modifying files automatically in Python

[John Nagle]

On 5/7/2012 9:09 PM, Steve Howell wrote:

On May 7, 8:46 pm, John Nagle  wrote:

On 5/6/2012 9:59 PM, Paul Rubin wrote:


Javierwrites:

Or not... Using directories may be a way to do rapid prototyping, and
check quickly how things are going internally, without needing to resort
to complex database interfaces.



dbm and shelve are extremely simple to use.  Using the file system for a
million item db is ridiculous even for prototyping.


 Right.  Steve Bellovin wrote that back when UNIX didn't have any
database programs, let alone free ones.



It's kind of sad that the Unix file system doesn't serve as an
effective key-value store at any kind of nontrivial scale.  It would
simplify a lot of programming if filenames were keys and file contents
were values.


   You don't want to go there in a file system.  Some people I know
tried that around 1970.  "A bit is a file.  An ordered collection of 
files is a file".  Didn't work out.


   There are file models other than the UNIX one.  Many older systems
had file versioning.  Tandem built their file system on top of their
distributed, redundant database system.  There are backup systems
where the name of the file is its hash, allowing elimination of
duplicates.  Most of the "free online storage" sites do that.

John Nagle
--
http://mail.python.org/mailman/listinfo/python-list

Re: Open Source: you're doing it wrong - the Pyjamas hijack

[Chris Angelico]

On Tue, May 8, 2012 at 1:54 PM, Steven D'Aprano
 wrote:
> A.k.a. "we had to destroy the project in order to save it".
>
> http://technogems.blogspot.com.au/2012/05/pyjamas-hijacked.html

Great summary, very handily peppered with links to appropriate posts.

> Seriously, this was a remarkably ham-fisted and foolish way to "resolve"
> a dispute over the direction of an open source project. That's the sort
> of thing that gives open source a bad reputation.

I'd probably be on the side of the dissidents in terms of philosophy -
freedom is there to be used, but if it costs you too much (effort,
quality, etc) to use all-free-software, the cart's involved in equine
artistry. You want a wiki? Throw down MySQL and MediaWiki. Want
hosting? GitHub is fine. I don't restrict my hardware purchases to
"free BIOS or no sale".

But a backstabbing takeover is not doing anyone any good. Especially
not the reputation of the project. Here at work we have some
familiarity with Python, and my boss is just starting to learn
Javascript (after our main JS developer left); but there's no way that
I'm going to consider introducing pyjamas / pyjs until this is
resolved.

> (The sad thing is, when closed source software developers do this sort of
> thing, it gets blamed on "bad apples"; when open source developers do it,
> it gets used as an indictment on the entire FOSS community.)

It's not quite as mixed-standards as that. If you see Microsoft or
Apple charging a fortune for trivial upgrades and/or bug fixes, you
blame it on corporate development. And some low-quality software in
the FOSS market is acknowledged as "you get what you pay for",
although that one can backfire too. But yes, it's a harsh reality that
one open-source community's actions reflect badly on another. (Which
is why I want to be really REALLY careful of using the term "open
source" here at work. Just because we let people have the source code
to certain scripts etc does not mean we should use that term. Just
sayin'.)

I hope that pyjamas can be restored at some point to a single live
project. Whether that's headed by Luke Leighton or C Anthony Risinger
(neither of whom I know at all and thus I can't speak to either's
merits) or someone else, I don't particularly care, but frankly, I
don't think there's need in the world for a fork of such a project.
Aside from philosophical disagreements, what would be the differences
between the Luke fork and the Anthony fork? Could anyone explain, to a
prospective user, why s/he should pick one or the other? If not, the
projects need to merge, or else one will die a sad death of
stagnation.

ChrisA
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Open Source: you're doing it wrong - the Pyjamas hijack

[alex23]

On May 8, 1:54 pm, Steven D'Aprano  wrote:
> Seriously, this was a remarkably ham-fisted and foolish way to "resolve"
> a dispute over the direction of an open source project. That's the sort
> of thing that gives open source a bad reputation.

The arrogance and sense of entitlement was so thick you could choke on
it. Here's a sampling from the circle jerk of self-justification that
flooded my inbox over the weekend:

"i did not need to consult Luke, nor would that have be productive"

No, it's generally _not_ productive to ask someone if you can steal
their project from them.

"i have retired Luke of the management duties, particularly, *above*
the source"

Who is this C Anthony Risinger asshole and in what way did he _hire_
the lead developer?

"What I have wondered is, what are effects of having the project
hostage to the whims of an individuals often illogically radical
software libre beliefs which are absolutely not up for discussion at
all with anyone."

What I'm wondering is: how is the new set up any different? Why were
Luke Leighton's philosophies/"whims" any more right or wrong than
those held by the new Gang of Dicks?

"Further more, the reason I think it's a bad idea to have this drawn
out discussion is that pretty much the main reason for this fork is
because of Luke leadership and project management decisions and
actions. To have discussions of why the fork was done would invariably
lead to quite a bit of personal attacks and petty arguments."

Apparently it's nicer to steal someone's work than be mean to them.

"I agree, Lex - this is all about moving on.  This is a software
project, not a cult of personality."

Because recognising the effort of the lead developer is cult-like.

"My only quibble is with the term "fork."  A fork is created when you
disagree with the technical direction of a project.  That's not the
issue here.  This is a reassignment of the project administration only
- a shuffling of responsibility among *current leaders* of the
community.  There is no "divine right of kings" here."

My quibble is over the term "fork" too, as this is outright theft. I
don't remember the community acknowledging _any other leadership_ over
Luke Leighton's.

"I suspect Luke will be busy with other projects and not do much more
for Pyjamas/pyjs, Luke correct me if you see this and I am wrong."

How about letting the man make his own fucking decisions?

"All of you spamming the list with your unsubscribe attempts: Anthony
mentioned in a previous email that he's using mailman now"

Apparently it's the responsibility of the person who was subscribed
without their permission to find out the correct mechanism for
unsubscribing from that list.

"apparantly a bunch of people were marked as "POSTING" in the DB, but
not receiving mail (?)"

Oh I see, the sudden rush of email I received was due to an error in
the data they stole...

"Nobody wins if we spend any amount of time debating the details of
this transition, what's done is done."

Truly the justification of assholes.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: indexed property? Can it be done?

[Ian Kelly]

On Mon, May 7, 2012 at 9:15 PM, Charles Hixson
 wrote:
> class Node:
>
>    def    __init__(self, nodeId, key, value, downRight, downLeft, parent):
>        dirty    =    True
>        dlu    =    utcnow()
>        self.node    =    [nodeId, downLeft, [key], [value], [downRight],
> parent, dirty, dlu]
>
> Note that node[3] is a list of keys (initially 1) and node[3] is a list of
> values, etc.
>
> What I'd like to do is to be able to address them thusly:
> k = node.key[2]
> v = node.value[2]
> but if there's a way to do this, I haven't been able to figure it out.  Any
> suggestions?

http://code.activestate.com/recipes/577703-item-properties/
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Open Source: you're doing it wrong - the Pyjamas hijack

[alex23]

Even worse, here's what Risinger had to say when Leighton asked them
to stop sending him email:

"probably best not to feed the troll, Pascal -- especially one
overwrought and lost in high dudgeon -- they tend to brickwall common
reason and simple social advices."

"Luke has made his decision -- and burned all ties -- by pitching a
snit of hollow threats"

"i full-heartily recommend that everyone do exactly as Luke requests,
i.e. "cease and desist from all communications" with him, regarding
this project's past or future ..."

There's being an asshole, and then there's being an absolute fucking
asshole. It seems pretty clear which category this behaviour falls in.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: indexed property? Can it be done?

[Chris Angelico]

On Tue, May 8, 2012 at 2:18 PM, Charles Hixson
 wrote:
> Not as clean as what I'm hoping for, but so far I haven't come up with any
> way except functions that doesn't directly expose the data...and if I must
> use that approach, then the class doesn't buy me anything for the overhead.)

C++ and Java teach us to hide everything and use trivial getters and
setters to make things available:

class Foo
{
private:
int value;
public:
int getValue() {return value;}
int setValue(int newval) {return value=newval;}
};

Python lets us happily access members directly, but change to
getter/setter if we need to: (example lifted from
http://docs.python.org/library/functions.html#property )

class C(object):
def __init__(self):
self._x = None

def getx(self):
return self._x
def setx(self, value):
self._x = value
def delx(self):
del self._x
x = property(getx, setx, delx, "I'm the 'x' property.")

Not fundamentally different, except that with Python, you can skip all
this in the simple case:

class C(object):
def __init__(self,val):
self.value = val

In short, data hiding isn't such a great thing after all. Just use the
members directly until such time as you find you need to change that
(and be honest, how often have you had thin getter/setter methods and
then changed their functionality?).

ChrisA
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: sorting 1172026 entries

[Ian Kelly]

On Mon, May 7, 2012 at 3:52 PM, Cameron Simpson  wrote:
> | (or add 50% or something) each
> | time, meaning that as n increases, the frequency of reallocations
> | decreases - hence the O(1) amortized time.
>
> Hmm, yes. But it is only O(1) for doubling. If one went with a smaller
> increment (to waste less space in the end case where one stops growing the
> array) then there are more copies and less .append efficiency, trading
> potential memory bloat for compute time in .append(). If you went all
> the way to adding only one item the cost goes to O(n) for .append()
> and O(n^2) overall, with varying costs in between.

It's O(1) amortized as long as the increment is exponential.  IIRC
Python actually grows the list by a factor of 1/8 in the limit, which
is still exponential and still O(1) amortized.

Cheers,
Ian
-- 
http://mail.python.org/mailman/listinfo/python-list