Re: [Numpy-discussion] searching binary data

2010-09-22 Thread David Cournapeau
On Wed, Sep 22, 2010 at 11:25 PM, Neal Becker wrote: > David Cournapeau wrote: > >> On Wed, Sep 22, 2010 at 11:10 PM, Neal Becker wrote: >>> A colleague of mine posed the following problem.  He wants to search >>> large files of binary data for sequences. >>> >> >> Is there a reason why you canno

Re: [Numpy-discussion] searching binary data

2010-09-22 Thread Robert Kern
On Wed, Sep 22, 2010 at 09:10, Neal Becker wrote: > A colleague of mine posed the following problem.  He wants to search large > files of binary data for sequences. > > I thought of using mmap (to avoid reading all data into memory at once) and > then turning this into a numpy array (using buffer=

Re: [Numpy-discussion] searching binary data

2010-09-22 Thread Neal Becker
David Cournapeau wrote: > On Wed, Sep 22, 2010 at 11:10 PM, Neal Becker wrote: >> A colleague of mine posed the following problem. He wants to search >> large files of binary data for sequences. >> > > Is there a reason why you cannot use one of the classic string search > algorithms applied to

Re: [Numpy-discussion] searching binary data

2010-09-22 Thread David Cournapeau
On Wed, Sep 22, 2010 at 11:10 PM, Neal Becker wrote: > A colleague of mine posed the following problem.  He wants to search large > files of binary data for sequences. > Is there a reason why you cannot use one of the classic string search algorithms applied to the bytestream ? David ___

[Numpy-discussion] searching binary data

2010-09-22 Thread Neal Becker
A colleague of mine posed the following problem. He wants to search large files of binary data for sequences. I thought of using mmap (to avoid reading all data into memory at once) and then turning this into a numpy array (using buffer=). But, how to then efficiently find a sequence? Note t