2008/10/2 Bill Broadley <[EMAIL PROTECTED]>
<...>
Why hardware? I have some python code that managed 10MB/sec per CPU (or
> 80MB
> on 8 CPUs if you prefer) that compresses with zlib, hashes with sha256, and
> encrypts with AES (256 bit key). Assuming the compression you want isn't
> substantial
Currently I generate nearly one TB data every few days and I need to pass it
Bill's right - 6 MB/s is really not much to ask from even a complex WAN.
I think the first thing you should do is find the bottleneck. to me it
sounds like you have a sort of ropey path with a 100 Mbps hop somewhere.
Xu, Jerry wrote:
Hello,
Currently I generate nearly one TB data every few days and I need to pass it
along enterprise network to the storage center attached to my HPC system, I am
thinking about compressing it (most tiff format image data)
tiff uncompressed, or tiff compressed files? If unc
Hi Jerry
I think HDF5 can help you in some way... - http://www.hdfgroup.org/HDF5/
Rodrigo
2008/10/2 Xu, Jerry <[EMAIL PROTECTED]>
> Hello,
>
> Currently I generate nearly one TB data every few days and I need to pass
> it
> along enterprise network to the storage center attached to my
John Hearns wrote:
> Damn Small (DSL) is, Puppy is not. I believe Puppy is it's own beast.
>
> The FAQ on the site says its a Slackware derivative, if I'm not wrong.
> What goes around comes around I guess :-) Maybe those kipper ties from
> the 70s will be back too.
First question on the page
Hi,
Am 02.10.2008 um 22:09 schrieb Xu, Jerry:
Currently I generate nearly one TB data every few days and I need
to pass it
along enterprise network to the storage center attached to my HPC
system, I am
thinking about compressing it (most tiff format image data)
is it plain tiff or alread
On Thu, Oct 02, 2008 at 04:09:36PM -0400, Xu, Jerry wrote:
>
> Currently I generate nearly one TB data every few days and I need to pass it
> along enterprise network to the storage center attached to my HPC system, I am
> thinking about compressing it (most tiff format image data) as much as I c
On Thu, Oct 02, 2008 at 05:40:31PM -0400, Joe Landman wrote:
> I have heard of some "xml accelerators" in the
> past (back when XML was considered a good buzzword) that did on-the-fly
> compression.
Well, given how wordy the tags are, simply compressing those is
inexpensive and is a big win,
Xu, Jerry wrote:
Hello,
Currently I generate nearly one TB data every few days and I need to pass it
along enterprise network to the storage center attached to my HPC system, I am
thinking about compressing it (most tiff format image data) as much as I can, as
fast as I can before I send it cr
On Wed, Oct 01, 2008 at 04:07:39PM -0500, Alan Louis Scheinine wrote:
> NiftyOMPI Mitch wrote
>> QDR is interesting... in all likelyhood the
>> QDR game will be optical for any link further away than a single rack.
>> Once IB goes optical there will be a lot of reason to install IB in
> > machine r
Hello,
Currently I generate nearly one TB data every few days and I need to pass it
along enterprise network to the storage center attached to my HPC system, I am
thinking about compressing it (most tiff format image data) as much as I can, as
fast as I can before I send it crossing network ...
> When I tried to ressurect my thing a couple years ago, I realized my
> original code was all wrong in trading time for space (plenty of time on the
> 386, then SunOS servers; not enough space, but new machine had plenty of
> unused RAM). I thought some about redesigning to reverse the trade-off
>
> Damn Small (DSL) is, Puppy is not. I believe Puppy is it's own beast.
>
> The FAQ on the site says its a Slackware derivative, if I'm not wrong.
What goes around comes around I guess :-) Maybe those kipper ties from the
70s will be back too.
Actually, and here I toss in a handgrenade, if we a
In the article:
"What if a high level programing description language was developed.
Note I did not say programming language. This description language would
allow you to “describe” what you needed to do and not how to do it (as
discussed before)."
I would ask then, how does one "describe what
stephen mulcahy wrote:
> John Hearns wrote:
>> H can I forsee Puppy Linux HPC Edition
>> http://www.puppylinux.org/
>>
>>
>> Being half serious here, is it worth trying to get one of these
>> slimmed-down distros to the state where it will run an HPC job?
>> Oh, and in addition to
John,
When I tried to ressurect my thing a couple years ago, I realized my
original code was all wrong in trading time for space (plenty of time on the
386, then SunOS servers; not enough space, but new machine had plenty of
unused RAM). I thought some about redesigning to reverse the trade-off,
wh
2008/10/2 Peter St. John <[EMAIL PROTECTED]>
> John,
> After I first thought up my nutty GA scheme, I was then astonished by the
> John Holland Scientific American artifcle. I was aghast that anyone could
> even imagine thinking along those lines with 1960's hardware, as he had
> done.
>
> I belie
John,
After I first thought up my nutty GA scheme, I was then astonished by the
John Holland Scientific American artifcle. I was aghast that anyone could
even imagine thinking along those lines with 1960's hardware, as he had
done.
I got my first working version done on a 386 (daughtercard on a 28
2008/10/2 Bogdan Costescu <[EMAIL PROTECTED]>:
> On Wed, 1 Oct 2008, Eric Thibodeau wrote:
>
>> the NFS root approach only does changes on the head node and changed files
>> don't need to be propagated and are accessed on a as-needed basis, this
>> might have significant impacts on large deployment
On Thu, 2 Oct 2008, Lux, James P wrote:
If one considers that a single wire is about 1 uH/meter (typical electrical
wiring will be much less, because it's a pair, with currents flowing
opposite directions), the series L might be a few tens of uH. At, say, 20
A, there's just not much energy stor
Rgb wrote:
>
> I understand inductive surge when powering up, I understand in detail
> browning out a primary power transformer, but I think those are
> different issues and irrelevant here.
Inductive surge -> magnetizing current in large iron core inductors (depends
on where you are in line frequ
To comment a bit on the article:
>What if a high level programing description language was developed.
Note I did not say programming language.
>This description language would allow you to “describe” what you
needed to do and not how to do it (as
>discussed before). This draft description wou
John Hearns wrote:
H can I forsee Puppy Linux HPC Edition
http://www.puppylinux.org/
Being half serious here, is it worth trying to get one of these
slimmed-down distros to the state where it will run an HPC job?
Oh, and in addition to a barebones install for our contemplative
2008/10/1 Donald Becker <[EMAIL PROTECTED]>
>
>
> It's foreseeable that holding an 8GB install
> image in memory will be trivial, but that will be a few years in the
> future, not today. And we will need better VM and PTE management to make
> it efficient.
>
>
>
H can I forsee Puppy Lin
On Wed, 1 Oct 2008, Eric Thibodeau wrote:
the NFS root approach only does changes on the head node and changed
files don't need to be propagated and are accessed on a as-needed
basis, this might have significant impacts on large deployments
NFS-root doesn't scale too well, the implementation
On Wed, 1 Oct 2008, Donald Becker wrote:
That's correct. Our model is that a "cluster" is a single system --
and a single install.
That's the idea that I've also started with, almost 10 years ago ;-)
Not using Beo*/bproc, but NFS-root which allowed a single install in
the node "image" to be
Vincent Diepeveen wrote:
[...]
Supporting a thing like openmosix requires a lot more than 1 guy who in
order to modify 3 bytes needs 1000 dollar.
Hello, Vincent.
I've been involved in the openMosix project from the start, and I still
use it. My comment about the $1,000 for 'updates' concerne
I just read Douglas Eadline's article on Linux Magazine, entitled "What He
Said"
http://www.linux-mag.com/id/7087
Very thought provoking article, and took me back to thinking about genetic
algorithms,
a subject I flirted with 20 years ago. I didn't find it worthwhile on a
Sparc1 system with a whop
28 matches
Mail list logo
