Hi Jon, They now have Lustre through FSx or what ever AWS have called it. I am not sure you guys have heard about the capital one data breach but at times im still rather weary of the cloud.
Regards, Jonathan From: Jonathan Engwall <engwalljonathanther...@gmail.com> Sent: Wednesday, 31 July 2019 01:03 To: Douglas Eadline <deadl...@eadline.org> Cc: Jonathan Aquilina <jaquil...@eagleeyet.net>; Beowulf Mailing List <Beowulf@beowulf.org>; Chris Samuel <ch...@csamuel.org> Subject: Re: [Beowulf] Lustre on google cloud AWS has a host of free tier sercives you should blend together. Elastic Beanstalk and Lambda (AWS proprietary lambda) can move lots of data below a cost level. Your volume will automatically cause billing obviously. I have a friend at AWS. Maybe something new is going on, I can check up with him. On Mon, Jul 29, 2019, 11:24 AM Douglas Eadline <deadl...@eadline.org<mailto:deadl...@eadline.org>> wrote: > What would be the reason for getting such large data sets back on premise? > Why not leave them in the cloud for example in an S3 bucket on amazon or > google data store. I think this touches on the ownership issue I have seen some people mention (I think Addison Snell or i360). That is, you own the data but not the infrastructure. To use the "data lake" analogy, you start out creating a swimming pool in the cloud. You own the water, but it is in someone else's pool. Manageable. At some point your little pool becomes a big lake. Moving the lake, for any number of reasons, become a really big issue and possibly unmanageable. "For any number of reasons" can be cost, performance, access, etc. and the issues you never imagined (a black swan as it were) Just like everything else, it all depends ... (and how risk adverse you are). -- Doug > > Regards, > Jonathan > > -----Original Message----- > From: Beowulf > <beowulf-boun...@beowulf.org<mailto:beowulf-boun...@beowulf.org>> On Behalf > Of Chris Samuel > Sent: Sunday, 28 July 2019 03:36 > To: beowulf@beowulf.org<mailto:beowulf@beowulf.org> > Subject: Re: [Beowulf] Lustre on google cloud > > On Friday, 26 July 2019 4:46:56 AM PDT John Hearns via Beowulf wrote: > >> Terabyte scale data movement into or out of the cloud is not scary in >> 2019. >> You can move data into and out of the cloud at basically the line rate >> of your internet connection as long as you take a little care in >> selecting and tuning your firewalls and inline security devices. >> Pushing 1TB/day etc. >> into the cloud these days is no big deal and that level of volume is >> now normal for a ton of different markets and industries. > > Whilst this is true as Chris points out this does not mean that there > won't be data transport costs imposed by the cloud provider (usually for > egress). > > All the best, > Chris > -- > Chris Samuel : http://www.csamuel.org/ : Berkeley, CA, USA > > > > _______________________________________________ > Beowulf mailing list, Beowulf@beowulf.org<mailto:Beowulf@beowulf.org> > sponsored by Penguin Computing > To change your subscription (digest mode or unsubscribe) visit > https://beowulf.org/cgi-bin/mailman/listinfo/beowulf > _______________________________________________ > Beowulf mailing list, Beowulf@beowulf.org<mailto:Beowulf@beowulf.org> > sponsored by Penguin Computing > To change your subscription (digest mode or unsubscribe) visit > https://beowulf.org/cgi-bin/mailman/listinfo/beowulf > -- Doug _______________________________________________ Beowulf mailing list, Beowulf@beowulf.org<mailto:Beowulf@beowulf.org> sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit https://beowulf.org/cgi-bin/mailman/listinfo/beowulf
_______________________________________________ Beowulf mailing list, Beowulf@beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit https://beowulf.org/cgi-bin/mailman/listinfo/beowulf
