Re: [Numpy-discussion] C-compiler options

This problem has nothing to do with numpy -- somehow, gcc binaries themselves were overwritten by a process that installed just 32-bit versions, ... so the problem is with the c compiler itself, and has been resolved. thanks! Dan On Thu, Nov 11, 2010 at 7:48 AM, Dan Yamins wrote: > > &g

Re: [Numpy-discussion] C-compiler options

> Yes, it should. This was fixed a couple of days ago in > commit:435c7262592e94c8519f (master) and commit:8346ba04a5c574441304 > (1.5.x). I guess you used an older revision, can you please update and > try again? > Did this, but am having the same problem.This is the relevant output at the be

Re: [Numpy-discussion] C-compiler options

> Are you sure that the python executable that you are using is the > 64-bit python executable that you think it is? > Hm ... Well, I think so ... It certain is 64bit, or at least: In [1]: import platform In [2]: platform.architecture() Out[2]: ('64bit', '') Should I be checking something else?

[Numpy-discussion] C-compiler options

Hi: I'm trying to build numpy with python27, 64-bit, on OSX 10.5.8. When I run python setup.py build, only 32-bit binaries get built. I can see what is happening is that not 64-bit flags are getting passed to the c compiler (I don't know about the fortran compiler. So I have two questions;

Re: [Numpy-discussion] Numpy/Scipy for EC2

On Sat, Dec 5, 2009 at 1:32 AM, David Goldsmith wrote: > Dan- > > I almost hate to ask - after what you've already provided, which is > substantial: did you ever try out what alestic had to offer? If not, I may > be that guinea pig. ;-) > > David -- I'm posting my response to your question to

[Numpy-discussion] Type Inference

Sometimes I need to convert object-type arrays to their "natural, real" type, without a priori knowing what that type is, e.g. the equivalent of: >>> Y = np.array(X.tolist()) where X is the object array. If X is naturally an array of ints, Y will be an int array, if X is naturally strings, then

Re: [Numpy-discussion] Bug in rec.fromarrays ; plus one other possible bug

>('\x00est', ''), ('\x00est', ''), ('\x00est', ''), ('\x00est', > > ''), > >('\x00est', ''), ('\x00est', '')], > > dtype=[('A', '|S4'), ('B', '|S0')]) > > That certainly looks like a bug -- where does the \0 appear in front of > all but the first string? > Sorry, I'm not sure

[Numpy-discussion] Bug in rec.fromarrays ; plus one other possible bug

Hi, I'm writing to report what looks like a two bugs in the handling of strings of length 0. (I'm using 1.4.0.dev7746, on Mac OSX 10.5.8. The problems below occur both for python 2.5 compiled 32-bit as well as python2.6 compiled 64-bit). Bug #1: A problem arises when you try to create a record

Re: [Numpy-discussion] Problems casting object arrays to string type on Ubuntu

> I suspect you're running into this bug: > > http://projects.scipy.org/numpy/ticket/1235 > > This has been fixed in SVN r7514, (and thus 1.4 pre releases), so it's > not a platform difference. > Great, thanks for the reply! best, Dan > > > wrote: > > > > Hi all,

Re: [Numpy-discussion] Problems casting object arrays to string type on Ubuntu

Really, no idea about this? (Sorry if my original email was unclear.) On Sat, Nov 21, 2009 at 3:27 PM, Dan Yamins wrote: > Hi all, > > I'm having some issues casting object arrays to string type, especially on > my Ubuntu installation. (Ubuntu Jaunty, 9.04, with Numpy v. 1.3

[Numpy-discussion] Problems casting object arrays to string type on Ubuntu

Hi all, I'm having some issues casting object arrays to string type, especially on my Ubuntu installation. (Ubuntu Jaunty, 9.04, with Numpy v. 1.3.) With small arrays, the conversion is just wrong. With large arrays, there seems to be some memory corruption. Conversion to int or float (when ap

Re: [Numpy-discussion] Numpy/Scipy for EC2

Hi all: I'm just writing to report on my experience using Starcluster, which enables the use of NumPy and Scipy in the Amazon EC2 cloud computing environment. The purpose of my email is to extol Starcluster's qualities, and suggest that the NumPy community be aware of its development.I suspect

Re: [Numpy-discussion] Automatic string length in recarray

On Tue, Nov 3, 2009 at 11:43 AM, David Warde-Farley wrote: > On 2-Nov-09, at 11:35 PM, Thomas Robitaille wrote: > > > But if I want to specify the data types: > > > > np.rec.fromrecords([(1,'hello'),(2,'world')],dtype=[('a',np.int8), > > ('b',np.str)]) > > > > the string field is set to a length o

Re: [Numpy-discussion] Numpy/Scipy for EC2

I haven't used it, but this seems to provide a good environment for your > needs. > > http://web.mit.edu/stardev/cluster/ > > Robert Kern to the rescue again! StarCluster looks great. And thanks Dorian as well, I'm also checking out Alestic. Dan > -- > Robert Kern > > "I have come to b

[Numpy-discussion] Numpy/Scipy for EC2

Hi all: I'm gearing up to build an Amazon Machine Instance (AMI) for use in doing Numpy/Scipy computations on the Amazon EC2 cloud. I'm writing to ask if anyone has any advice for which (if any) publicly available AMI I should start with. If any one has any specific AMI's that they think are goo

Re: [Numpy-discussion] Tabular data package

> > I didn't see any explicit nan handling. Are missing values allowed > e.g. in the constructor? > No, this is a valid point. We don't handle this as explicitly as we should. Are you mostly talking about nan handling in loading from delimited text files? (Or are you talking about something mo

[Numpy-discussion] __eq__ method for recarray returns recarray

If I have two recarrays with the same len and column headers, the __eq__ method returns the rich comparison, which is great. E.g. In [20]: x = np.rec.fromrecords([(1,2,'dd',.3),(33,2,'y',2.2),(2,3,'a',21.4),(3,4,'b',33.2)],names=['A','B','C','D']) In [21]: y = np.rec.fromrecords([(1,2,'dd',.3),(

Re: [Numpy-discussion] ndarray from column data

> > What's wrong with recarrays? In any case, if you need a true ndarray > object > you can always do: > > ndarr = recarr.view(np.ndarray) > > and you are done. > I have a question about this though. The object "ndarr" will consist of "records", e.g.: In [96]: type(ndarr[0]) Out[96]: If

Re: [Numpy-discussion] Rec array: numpy.rec vs numpy.array with complex dtype

Pierre, thanks for your response. I have some follow up questions. Short answer: > a np.recarray is a subclass of ndarray with structured dtype, where > fields can be accessed has attributes (as in 'yourarray.yourfield') > instead of as items (as in yourarray['yourfield']). Is this the only su

[Numpy-discussion] Rec array: numpy.rec vs numpy.array with complex dtype

Dear Numpy list: We've been using the numpy.rec classes to make record array objects. We've noticed that in more recent versions of numpy, record-array like objects can be made directly with the numpy.ndarray class, by passing a complex data type. However, it looks like the numpy.rec class is st

[Numpy-discussion] building 64bit

Hi, I'm trying to build scipy 64bit binaries. I'm on OS10.5.6, and using python2.6 built as 64bit universal from macports. When I run: python2.6 setup.py install the build doesn't fail along the way. However, when I then go into python and try something, e.g. "from scipy import linalg", I

Re: [Numpy-discussion] Seg fault from numpy.rec.fromarrays

> > > Then I attempt > >>>> A = numpy.rec.fromarrays(L,names = ['Aggregates','__color__']) > > So what happens when you set the dtype here? > > Since you have variable lengths of strings, numpy probably has guessed > incorrectly. I would also check that Col1 and Col2 are what you expect, > esp

Re: [Numpy-discussion] Seg fault from numpy.rec.fromarrays

> >> Can someone explain why this might be happening, and how I can fix it >> (without having to use the pickling hack)? >> > > What architecture/operating system is this? > Sorry, I should have included this information before. it's OS 10.5.6. the is a 64-bit intel core-2 duo, but the python i

[Numpy-discussion] Seg fault from numpy.rec.fromarrays

Hi all, I'm having a seg fault error from numpy.rec.fromarrays. I have a python list L = [Col1, Col2] where Col1 and Col2 are python lists of short strings (the max length of Col1 strings is 4 chars and max length of Col2 is 7 chars). The len of Col1 and Col2 is about 11500. Then I attempt

Re: [Numpy-discussion] Record arrays

> > In [12]: r2.foo > Out[12]: array([1, 1, 1]) > > > One downside of this is that the attribute access feature slows down > all field accesses, even the r['foo'] form, because it sticks a bunch > of pure Python code in the middle. Much code won't notice this, but if > you end up having to iterate

Re: [Numpy-discussion] Record arrays

On Thu, Jun 26, 2008 at 3:34 PM, Gael Varoquaux < [EMAIL PROTECTED]> wrote: > On Thu, Jun 26, 2008 at 11:48:06AM -0500, John Hunter wrote: > > I personally think they are the best thing since sliced bread, and > > everyone here who uses them becomes immediately addicted to them. I > > would like

Re: [Numpy-discussion] Sparse Matrices in Numpy -- (with eigenvalue algorithms if possible)

On Tue, Jun 24, 2008 at 2:12 AM, Charles R Harris <[EMAIL PROTECTED]> wrote: > Hi Dan, > > Did you finally get numpy installed as 64 bits? > > Chuck > > Hey, thanks for asking. I did in fact get it installed -- at least, I think so. First, I had to built python in 64bit. I did this using exact

Re: [Numpy-discussion] Sparse Matrices in Numpy -- (with eigenvalue algorithms if possible)

> > > > > The next version of SciPy will include two sparse eigensolvers: ARPACK > and LOBPCG > > > http://scipy.org/scipy/scipy/browser/trunk/scipy/sparse/linalg/eigen > OK, so I then I have three questions: 1) Does your commend imply that the current version of scipy does _not_ have an eigen

[Numpy-discussion] Sparse Matrices in Numpy -- (with eigenvalue algorithms if possible)

I'm wondering if there is a currently-available Sparse-Matrix package for numpy? If so, how do I get it?And, if there is a good sparse-matrix package, does it include an eigenvalue-computation algorithm? How would a sparse-matrix package interact with something like numpy.linalg.eig, or for t

Re: [Numpy-discussion] Building 64-bit numpy on OSX

. What I think matters is "no matching architecture in > > universal wrapper". Hmmm. I wonder if you and Michael have the same > > versions of OS X? > > > And why is dlopen looking for a universal library? One would hope that > > distutils would have taken care of that. > I think you're right

Re: [Numpy-discussion] Building 64-bit numpy on OSX

> > I'm forced to run it as su. (Is this a bad idea?) Anyhow, when I > > do run "sudo python setup.py install" in the numpy-1.1.0 directory I > > downloaded from SciPy website, the build apparently works. > If you were able to install python without sudo you should not need to > use sudo to bu

Re: [Numpy-discussion] Building 64-bit numpy on OSX

>./configure --disable-toolbox-glue --prefix=/Users/mabshoff/64bitnumpy/python-2.5.2-bin --with-gcc="gcc -m64" > Let's build numpy 1.1.0: > > bsd:64bitnumpy mabshoff$ tar xf numpy-1.1.0.tar.gz > bsd:64bitnumpy mabshoff$ cd numpy-1.1.0 > bsd:numpy-1.1.0 mabshoff$ python setup.py install > > > Mi

Re: [Numpy-discussion] A memory problem: why does mmap come up in numpy.inner?

What Charles pointed out was that while the inner product is very big, > it seems to fit into memory on his 32-bit Linux machine; is it > possible that OSX is preventing your python process from using even > the meager 2-3 GB that a 32-bit process ought to get? Yes -- I think this is what is hap

Re: [Numpy-discussion] A memory problem: why does mmap come up in numpy.inner?

> Try > > In [3]: numpy.dtype(numpy.uintp).itemsize > Out[3]: 4 > > which is the size in bytes of the integer needed to hold a pointer. The > output above is for 32 bit python/numpy. > > Chuck > Check, the answer is 4, as you got for the 32-bit. What would the answer be on a 64-bit architecture?

Re: [Numpy-discussion] A memory problem: why does mmap come up in numpy.inner?

> > Hey Dan. Now, that you mention you are using OS X, I'm fairly > confident that the problem is that you are using a 32-bit version of > Python (i.e. you are not running in full 64-bit mode and so the 4GB > limit applies). > > The most common Python on OS X is 32-bit python. I think a few peop

Re: [Numpy-discussion] A memory problem: why does mmap come up in numpy.inner?

On Wed, Jun 4, 2008 at 10:07 PM, David Cournapeau < [EMAIL PROTECTED]> wrote: > On Wed, 2008-06-04 at 21:38 -0400, Dan Yamins wrote: > > > > > Anne, thanks so much for your help. I still a little confused. If > > your scenario about the the memory allocation is w

Re: [Numpy-discussion] A memory problem: why does mmap come up in numpy.inner?

On Wed, Jun 4, 2008 at 9:06 PM, Charles R Harris <[EMAIL PROTECTED]> wrote: > > > On Wed, Jun 4, 2008 at 6:42 PM, Dan Yamins <[EMAIL PROTECTED]> wrote: > >> I'm using python 2.5.2 on OS X, with 8 GB of ram, and a 64-bit processor. >> In >> this, s

Re: [Numpy-discussion] A memory problem: why does mmap come up in numpy.inner?

I don't know much about OSX, but I do know that many malloc() > implementations take advantage of a modern operating system's virtual > memory when allocating large blocks of memory. For small blocks, > malloc uses memory arenas, but if you ask for a large block malloc() > will request a whole bunc

[Numpy-discussion] A memory problem: why does mmap come up in numpy.inner?

I'm using python 2.5.2 on OS X, with 8 GB of ram, and a 64-bit processor. In this, setting, I'm working with large arrays of binary data. E.g, I want to make calls like: Z = numpy.inner(a,b) where and b are fairly large -- e.g. 2 rows by 100 columns. However, when such a call