The best success I've seen is a mix of strategies: some amount storage that's considered "permanent" with aggressive quota limits (per user or per project), plus a larger no-quota space that's cleaned up based on some retention policy. I've used this basic scheme in small environments, and also seen it used successfully in large institutions (where I was not directly responsible for storage).
That said, there's no way to make everyone happy with this -- people will complain when a file is deleted unexpectedly, or when they hit their quota, or both. Time-based retention policies that work by automatically checking file modification time also invite users to write "touch" scripts to try to fool the cleaner. It can turn into a fun game. But I've also used these kinds of policies to justify new storage purchases, and make users into allies in funding them. On a lighter note, this is a good excuse for me to link my favorite James Cuff post ;-) - http://blog.jcuff.net/2011/12/cautionary-tale-about-storage-and.html Cheers, Adam On Mon, Feb 19, 2018 at 6:12 AM, Richter, Brian J {BIS} <brian.j.rich...@pepsico.com> wrote: > Hey All, > > > > I was hoping to get some recommendations for Storage. Last year we set up > our first HPC and I’m looking for a good strategy moving forward for > Storage. We set up a dedicated space on the cluster for Storage that has 5.5 > TB of space. This space can be quickly chewed up depending on the project > the business is working on. Do you guys typically do a retention policy on > the cluster to ensure there is always enough space? I was thinking anything > older than a month should be cleaned up. > > > > Thanks! > > > > Brian J. Richter > > Global R&D Senior Analyst • Information Technology > > 617 W Main St, Barrington, IL 60010 > Office: 847-304-2356 • Mobile: 847-305-6306 > > brian.j.rich...@pepsico.com > > > > > > > _______________________________________________ > Beowulf mailing list, Beowulf@beowulf.org sponsored by Penguin Computing > To change your subscription (digest mode or unsubscribe) visit > http://www.beowulf.org/mailman/listinfo/beowulf > _______________________________________________ Beowulf mailing list, Beowulf@beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
