Re: Moving /tmp to tmpfs makes it useless
On Fri, May 25, 2012 at 06:08:34AM +0300, Serge wrote: > It dosn't always work in practice. Among the problems I faced myself... > I wasn't able to watch a web presentation (from something like > vimeo/youtube), because there was not enough free space in /tmp for flash > player to download and show it. I thought those were streaming video sites? Do you mean flash player or something else? > Ok, I agree. There're two main /tmp usages. Some applications use it to > save memory (databases, archive managers, cd burners, image processors, > science software, the flash player after all). Other apps use it for > communication (browsers, mail clients). But in both cases these files may > be large and should be still stored on real disk, not on tmpfs. > And having /tmp on tmpfs makes it useless for both usages. Why is it useless for the second case? > The matter is that exceptions are *not rare*. A simple 2-hours-long > video presentation may cause you problems. Pressing Enter in `mc` on > linux-kernel-src.tar.bz2 may cause you problems. How much RAM do you have / how big is your /tmp(fs)? The fact this caused you trouble suggests to me that they must be very small. -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20120525072204.GC11341@debian
Re: Moving /tmp to tmpfs makes it useless
On Fri, May 25, 2012 at 12:41:44PM +0600, Andrey Rahmatullin wrote: > On Fri, May 25, 2012 at 02:22:24AM +0300, Serge wrote: > > What's a temporary file? Really, why would applications temporarily store > > its data in a file? They do that to *free some memory*. Placing those files > > back to memory renders the whole process of writing the file useless. > > If the files are small and can stay in memory why would application save it > > to file? > What if a file is large but still can fit into the memory, but app authors > didn't assume that's possible? Said app authors need to learn about swap. -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20120525072220.GD11341@debian
Re: Moving /tmp to tmpfs makes it useless
Le vendredi 25 mai 2012 à 02:22 +0300, Serge a écrit : > I've read across different debates about whether using tmpfs is good or bad > but I could not find the most important reason, so here it is... > > Reason > == > What's a temporary file? Really, why would applications temporarily store > its data in a file? They do that to *free some memory*. Placing those files > back to memory renders the whole process of writing the file useless. > If the files are small and can stay in memory why would application save it > to file? This is complete bullshit. If you know nothing about how virtual memory works, please refrain from making a fool of yourself. Files which are written on a regular filesystem stay on memory. This is called the buffer cache. Whenever they are not used and/or the system needs to reclaim memory, they are trashed. Files which are written on a tmpfs stay on memory. Whenever they are not used and/or the system needs to reclaim memory, they are swapped. See? No difference. Oh stop, there is a difference: in a tmpfs the system doesn’t need to commit the data on disk, and therefore can write it to disk whenever it likes, especially when the disk is not too used. There is no need to keep a journal nor to access the disk several times to update metadata. Only unused data is written to disk. Which means a *huge* performance improvement. Do the measurements yourself, it works with basically anything that makes heavy use of /tmp. And if you find /tmp too small, it’s really simple: increase your swap size. That’s all. -- .''`. Josselin Mouette : :' : `. `' `- -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/1337930993.14895.308.camel@pi0307572
Re: Moving /tmp to tmpfs makes it useless
Le 25/05/2012 05:03, Russell Coker a écrit : > On Fri, 25 May 2012, Serge wrote: >> Q: /tmp on tmpfs increases apps performance. >> A: What apps? Real apps don't write files during performance-critical >>operations. Even if they do, they write large files. And large files are >>written faster when they're written on real disk, rather then swapped >>out and slow down the entire system (see the "Who uses /tmp" part). >>The apps that can really benefit from tmpfs are too rare. And we're >>talking about default settings and most common cases. > > Any application which writes synchronously (through fsync(), fdatasync(), or > opening with O_SYNC) will get a massive performance benefit from using tmpfs. If some kind of sync is required by the application, I think this is because the application want to ensure the data are really written to the disk so that their state remains coherent even in case of crash. If the application is ok to have this kind of data written to tmpfs (ie in memory), I do not see the interest of using sync at first. Can someone shows me a valid use case of sync on tmpfs? Regards, Vince -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4fbf38dd.4000...@free.fr
Re: Moving /tmp to tmpfs makes it useless
On Fri, May 25, 2012 at 08:22:20AM +0100, Jon Dowland wrote: > > > What's a temporary file? Really, why would applications temporarily store > > > its data in a file? They do that to *free some memory*. Placing those > > > files > > > back to memory renders the whole process of writing the file useless. > > > If the files are small and can stay in memory why would application save > > > it > > > to file? > > What if a file is large but still can fit into the memory, but app authors > > didn't assume that's possible? > > Said app authors need to learn about swap. We all know how bad can Linux behave in some conditions involving heavy swap usage. Creating a file manually seems easier and safer, but when I have several Gb of free memory I want to use it instead. -- WBR, wRAR signature.asc Description: Digital signature
Re: Moving /tmp to tmpfs is fine
On Fri, 25 May 2012 02:22:24 +0300 Serge wrote: > What's a temporary file? Really, why would applications temporarily store > its data in a file? They do that to *free some memory*. Placing those files > back to memory renders the whole process of writing the file useless. Most of the time when I'm writing stuff to use /tmp (or $TMPDIR wherever that is), I'm using it only because I want to unpack something which is already a file into a temporary location which is fast and easy to remove without permission problems or having to setup anything in advance. RAM is ideal for that. $TMPDIR is ideal when a program needs to download something (or often a collection of somethings) and then needs to inspect them, checksum them, analyse, process and report on stuff about them and then throw them away and move on to the next collection of somethings. Now that may include processing them to upload them to somewhere else but still the local temporary copy needs to be cleaned up too. $TMPDIR is also important for control sockets and other clever pipe type stuff. > If the files are small and can stay in memory why would application save it > to file? Because it started out as a file and the program just wants to use some of the stuff inside the file? > Moving /tmp to tmpfs is effectively the same as suggesting to delete /tmp, > because there's no use for it as a temporary files storage any more. Not true. Temporary file storage is exactly why I need /tmp and temporary files don't get much more temporary than when they only exist in RAM. Having tmpfs to make RAM look like a filesystem is ideal. > FHS > === > Filesystem Hierarchy Standard defines two directories for temporary files: > /var/tmp — for files that should be preserved between reboots > /tmp — for files that should not be preserved between reboots > It's simple and clear. Yes and it is just what my programs need - space for files which need to be retrieved as files, processed as files but then blown away trivially. It means that the programs need to be run on devices with reasonable amounts of RAM - I don't really care. The programs need to be run on devices with a lot of disc space too, a fast processor and a fast network connection. That's life. I also write programs which have to work on tiny SSD drives, in minimal amounts of RAM and no network at all. You cannot expect to mix those two worlds and for things to "just work". If program A is too resource-hungry, find (or write) program B. > Since it's only reasonable to store large data sets in temporary files, > standard sets no size limits for these files. So if application's author > had actually read FHS he should expect these directories to handle > large files. Temporary files can be large files. Large files may only need to exist for a few seconds, it doesn't matter. Match the program to the resources available on that device and if one truly doesn't exist, write it. > Let's check the real world and see what applications actually use /tmp. > When you copy files in `mc` they're copied over /tmp/mc-username (to > handle some complex cases, like copying from inside iso-image to ssh). > When you click on a file in Firefox and select "Open with", Firefox stores > that file in /tmp. You cannot assume these files to be small. When you > watch large videos, adobe flash stores downloaded part of it as something > like /tmp/FlashXXG49VWF. Archive managers may unpack archives to /tmp. > CD burners store iso-files there. Image processing software was already > mentioned in this list. It's beginning to sound like your particular machines need either more RAM or to use a different temporary location which is on a permanent location. Just add some rules to clean it all up at reboot. That does NOT mean that Debian should change the default just to suit low memory devices. (Other stuff I write is for v.low memory devices which don't have swap, /tmp is on tmpfs in RAM and we just have to be careful what applications we use.) > Suggestion > == > Do not mount /tmp as tmpfs by default. Instead... No. The default is fine and sane but no default will ever satisfy every possible device. Low memory devices have many many more problems than just where /tmp is mounted. -- Neil Williams = http://www.linux.codehelp.co.uk/ pgpFujBBNhc3Q.pgp Description: PGP signature
Re: Moving /tmp to tmpfs makes it useless
On Fri, May 25, 2012 at 09:46:37AM +0200, Vincent Danjean wrote: > If some kind of sync is required by the application, I think this is > because the application want to ensure the data are really written to > the disk so that their state remains coherent even in case of crash. > If the application is ok to have this kind of data written to > tmpfs (ie in memory), I do not see the interest of using sync at > first. Can someone shows me a valid use case of sync on tmpfs? I'd agree with your analysis, it's buggy for them to aggressively sync for writes to /tmp. -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20120525084938.GB18308@debian
Re: Moving /tmp to tmpfs makes it useless
On 05/25/2012 03:22 PM, Jon Dowland wrote: > How much RAM do you have / how big is your /tmp(fs)? The fact this caused > you trouble suggests to me that they must be very small. > What if we're installing Debian on a very small system, and that we need operations with big files in /tmp? How much RAM he has is totally irrelevant, the issue will be the same if our users have less RAM, or bigger files to handle. Thomas -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4fbf4c1a.70...@debian.org
Re: Moving /tmp to tmpfs makes it useless
> We all know how bad can Linux behave in some conditions involving heavy > swap usage. Apparently not everybody has experienced that, and this would explain why they are so happy about the idea of paging out a couple of Gigabytes. -- Salvo Tomaselli -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/201205251104.12631.tipos...@tiscali.it
Re: Moving /tmp to tmpfs makes it useless
> This is complete bullshit. If you know nothing about how virtual memory > works, please refrain from making a fool of yourself. :-( > Files which are written on a regular filesystem stay on memory. This is > called the buffer cache. Whenever they are not used and/or the system > needs to reclaim memory, they are trashed. > Files which are written on a tmpfs stay on memory. Whenever they are not > used and/or the system needs to reclaim memory, they are swapped. > > See? No difference. You seem to forget that memory is not an unlimited resource, the system might need it for other things, and in that case a large tmpfs causes severe slowdown (and even complete freeze). Bye -- Salvo Tomaselli -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20120525.06639.tipos...@tiscali.it
Re: Moving /tmp to tmpfs is fine
> It's beginning to sound like your particular machines need either more > RAM or to use a different temporary location which is on a permanent > location. Just add some rules to clean it all up at reboot. Perhaps there are a couple of thousand users with the same use case, I don't know if it is the case but should be investigated rather than discarded. > That does NOT mean that Debian should change the default just to suit > low memory devices. So let's put minimum requirements unnecessarily high so a few people with super expensive laptops can have a 0.3μs speedup? (And people with cheaper hardware might never find out why the hell "linux" freezes if they click on a large tar archive). > No. The default is fine and sane but no default will ever satisfy every > possible device. Low memory devices have many many more problems than > just where /tmp is mounted. But with tmpfs on disk, more devices would work by default (the ones with a lot of memory and disk, and the ones without much memory but with disk space). Bye -- Salvo Tomaselli -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/201205251119.15851.tipos...@tiscali.it
Re: Moving /tmp to tmpfs makes it useless
On Vi, 25 mai 12, 17:08:42, Thomas Goirand wrote: > On 05/25/2012 03:22 PM, Jon Dowland wrote: > > How much RAM do you have / how big is your /tmp(fs)? The fact this caused > > you trouble suggests to me that they must be very small. > > > What if we're installing Debian on a very small system, and that > we need operations with big files in /tmp? > > How much RAM he has is totally irrelevant, the issue will be the > same if our users have less RAM, or bigger files to handle. ,[ /usr/share/doc/initscripts/NEWS.Debian.gz ] | sysvinit (2.88dsf-23) unstable; urgency=low | | Changes to the configuration and defaults of tmpfs filesystems: | | [...] | | In order to improve the default size limits of tmpfs filesystems, | it is now possible to configure the size as a percentage of the | total virtual memory. The default for /tmp and /run/shm is now | 20%VM rather than 20% (RAM). A tmpfs will only be mounted on | /tmp on systems with more than 64MiB RAM. | | -- Roger Leigh Wed, 18 Apr 2012 23:30:37 +0100 ` Kind regards, Andrei -- Offtopic discussions among Debian users and developers: http://lists.alioth.debian.org/mailman/listinfo/d-community-offtopic signature.asc Description: Digital signature
Re: Moving /tmp to tmpfs makes it useless
Hi all, like many, I also felt like I had something to say in that thread. But as discussed in the “making debian-devel useful” thread(s), I will wait a bit to see if others will not say it better before me. Most importantly, it has been proposed many times that pepole limit themselves to roughly one message per thread and per day when possible. I think that it would be a great way to make this thread constructive. Conversly, people who have time to post many messages could perhaps spare some to send a summary ? It is not that a decision will be taken that weekend and that we need to rush giving our opinions. Also, I invite everybody to read Roger's proposition in #630615 if they do not have time to read the whole bug discussion. http://bugs.debian.org/630615#89 Have a nice and relaxed week-end. -- Charles Plessy Tsurumi, Kanagawa, Japan -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20120525092913.gb2...@falafel.plessy.net
Re: Moving /tmp to tmpfs makes it useless
On 25/05/12 11:08, Thomas Goirand wrote: On 05/25/2012 03:22 PM, Jon Dowland wrote: How much RAM do you have / how big is your /tmp(fs)? The fact this caused you trouble suggests to me that they must be very small. What if we're installing Debian on a very small system, and that we need operations with big files in /tmp? Increase your swap? If it is not possible, then your RAM won't help much neither. If you really have many RAM and less disk, then this is very a specific case and we shouldn't bother making it "a general case". Regards, -- Mehdi Dogguy مهدي الدڤي http://dogguy.org/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4fbf51d6.4070...@dogguy.org
Re: Moving /tmp to tmpfs makes it useless
Le vendredi 25 mai 2012 à 11:11 +0200, Salvo Tomaselli a écrit : > You seem to forget that memory is not an unlimited resource, the system might > need it for other things, and in that case a large tmpfs causes severe > slowdown (and even complete freeze). Then increase your swap size. When you run out of swap, the system fills memory with things that could be swapped out, and has to reduce the buffer cache. Which then gives poor performance, regardless of the filesystem your /tmp is mounted on. -- .''`. Josselin Mouette : :' : `. `' `- -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/1337938747.14895.387.camel@pi0307572
Re: Moving /tmp to tmpfs makes it useless
Le vendredi 25 mai 2012 à 11:04 +0200, Salvo Tomaselli a écrit : > Apparently not everybody has experienced that, and this would explain why > they > are so happy about the idea of paging out a couple of Gigabytes. Because paging out a couple Gigabytes is veery different from writing a couple Gigabytes to disk, of course. -- .''`. Josselin Mouette : :' : `. `' `- -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/1337938886.14895.389.camel@pi0307572
Re: Moving /tmp to tmpfs makes it useless
> Because paging out a couple Gigabytes is veery different from > writing a couple Gigabytes to disk, of course. Yes because writing that on disk will only block the thread performing the write, not every thread that tries to allocate memory. -- Salvo Tomaselli -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/201205251149.49965.tipos...@tiscali.it
Re: Moving /tmp to tmpfs is fine
Am 2012-05-25 11:19, schrieb Salvo Tomaselli: It's beginning to sound like your particular machines need either more RAM or to use a different temporary location which is on a permanent location. Just add some rules to clean it all up at reboot. Perhaps there are a couple of thousand users with the same use case, I don't know if it is the case but should be investigated rather than discarded. That does NOT mean that Debian should change the default just to suit low memory devices. So let's put minimum requirements unnecessarily high so a few people with super expensive laptops can have a 0.3μs speedup? (And people with cheaper hardware might never find out why the hell "linux" freezes if they click on a large tar archive). Doing that on inferior hardware is just plain stupid. If you have plenty of disk space, just unpack the tar archive. No. The default is fine and sane but no default will ever satisfy every possible device. Low memory devices have many many more problems than just where /tmp is mounted. But with tmpfs on disk, more devices would work by default (the ones with a lot of memory and disk, and the ones without much memory but with disk space). And those with lots of RAM but not so much disk space (SD card or USB driver or even with no hard drive at all)? There's no solution that works for everyone in all situations. However, tmpfs at least works for many of them. If you KNOW that this default does not fit your use-case, why don't you simply change the configuration? HS -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/d77602e05a6b6ce8cd0072a717bf8...@mail.hendrik-sattler.de
Re: Moving /tmp to tmpfs makes it useless
In data Friday 25 May 2012 11:39:07, Josselin Mouette ha scritto: > Le vendredi 25 mai 2012 à 11:11 +0200, Salvo Tomaselli a écrit : > > You seem to forget that memory is not an unlimited resource, the system > > might need it for other things, and in that case a large tmpfs causes > > severe slowdown (and even complete freeze). > > Then increase your swap size. I've never seen an almost full swap partition. But the lack of memory has forced me to do several hardware resets nevertheless. And somehow the OOM killer was not even triggered, i kept ending up with a system where i had a working shell but could not run certain commands (such as kill), but i could see the output of "free" for example. > When you run out of swap, the system fills memory with things that could > be swapped out, and has to reduce the buffer cache. > > Which then gives poor performance, regardless of the filesystem > your /tmp is mounted on. True, an insufficient swap is a cause of problems too. Not the only one tho. Bye -- Salvo Tomaselli -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/201205251154.57460.tipos...@tiscali.it
Re: Moving /tmp to tmpfs makes it useless
On Fri, 25 May 2012, Thomas Goirand wrote: > for small files, and in that case, it's faster. In reality, it's > not that much faster, thanks to Linux caching of the filesystem, Under heavy filesystem IO load, yes it is. By several orders of magnitude. > the point. Maybe we should add a /small-files-on-tmpfs (choose > a better name, of course...) folder so that apps know what to do, > that'd be a lot more graceful than just switching to whole /tmp > to tmpfs without any app knowing about it. Nice idea, but it would be worthless. In fact it is the other way. We have /var/tmp for the large file since about forever, and important platforms that have /tmp in memory since the early 2000's (Solaris) And that STILL wasn't enough for people to not screw it up and dump large files in /tmp. -- "One disk to rule them all, One disk to find them. One disk to bring them all and in the darkness grind them. In the Land of Redmond where the shadows lie." -- The Silicon Valley Tarot Henrique Holschuh -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20120525101420.ga6...@khazad-dum.debian.net
Re: Moving /tmp to tmpfs makes it useless
On Friday 25 May 2012 09:46:37 Vincent Danjean wrote: > If some kind of sync is required by the application, I think this is > because the application want to ensure the data are really written to > the disk so that their state remains coherent even in case of crash. > If the application is ok to have this kind of data written to > tmpfs (ie in memory), I do not see the interest of using sync at > first. Agreed. > Can someone shows me a valid use case of sync on tmpfs? Unlikely. The last time I checked sync is a no-op on ramfs and tmpfs anyway (shmem.c). Also, tmpfs would not write out to swap on sync. Likewise, as there is no point of syncing anything if a on-disk filesystem is mounted read-only, hence kernel does not even bother to (sync.c). -- pub 4096R/0E4BD0AB -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/201205251214.43368.danc...@spnet.net
Re: Moving /tmp to tmpfs makes it useless
On Fri, 25 May 2012, Jon Dowland wrote: > On Fri, May 25, 2012 at 09:46:37AM +0200, Vincent Danjean wrote: > > If some kind of sync is required by the application, I think this is > > because the application want to ensure the data are really written to > > the disk so that their state remains coherent even in case of crash. > > If the application is ok to have this kind of data written to > > tmpfs (ie in memory), I do not see the interest of using sync at > > first. Can someone shows me a valid use case of sync on tmpfs? > > I'd agree with your analysis, it's buggy for them to aggressively sync for > writes > to /tmp. Hmm, no. If the application Does It Right (emacs, vim...) they will break hardlinks on open, and fsync properly on close/write by default. That doesn't make them buggy, just because someone decided to tell them to act on a file hosted in a tmpfs. -- "One disk to rule them all, One disk to find them. One disk to bring them all and in the darkness grind them. In the Land of Redmond where the shadows lie." -- The Silicon Valley Tarot Henrique Holschuh -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20120525101648.gb6...@khazad-dum.debian.net
Re: Moving /tmp to tmpfs is fine
> Doing that on inferior hardware is just plain stupid. If you have > plenty of > disk space, just unpack the tar archive. Double-click on a .tar causes it to be unpacked in /tmp/something. I suppose a lot of not so skilled users do that instead of tar -xf > And those with lots of RAM but not so much disk space (SD card or USB > driver or > even with no hard drive at all)? Can you link me one such device? Nothing pops in my mind. And how widespread they are? Normal desktop/laptop configurations have at least 100 times more disk space than ram. For servers.. it depends on the taks they are supposed to do, but usually servers have an administrator who is more or less aware of what he is doing. > There's no solution that works for everyone in all situations. True. But experts can change the defaults more effectively than non experts. > However, tmpfs at least works for many of them. If you KNOW > that this default does not fit your use-case, why don't you simply > change the configuration? The point is that the desktop user is often clueless and doesn't know it. He will just notice the problem and will not have a clue of how to solve it. So I would suggest that defaulting to a more safe configuration would be better, and of course those who want tmpfs can always change the default. But those who don't even know about what /tmp is would not end up with something that doesn't work so well on their hardware. Bye -- Salvo Tomaselli -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/201205251220.11062.tipos...@tiscali.it
Re: Moving /tmp to tmpfs makes it useless
On Fri, 25 May 2012, Salvo Tomaselli wrote: > > Because paging out a couple Gigabytes is veery different from > > writing a couple Gigabytes to disk, of course. > > Yes because writing that on disk will only block the thread performing the > write, not every thread that tries to allocate memory. What IO scheduler are you using? It must not be CFQ. CFQ will happly screw it up and stall both readers and writers when the number of dirty pages gets too large (which will happen if you write a gigabyte file to disk). Either that, or you're cheating and your backend devices are simply "fast enough" that it doesn't matter (fast RAID or fast SSD). The real question is: what does it better under CFQ+IO contention? Several threads doing filesystem IO, or the swapper? Hint: it is not an easy question to answer because it depends on the load, type of load, and backend device. -- "One disk to rule them all, One disk to find them. One disk to bring them all and in the darkness grind them. In the Land of Redmond where the shadows lie." -- The Silicon Valley Tarot Henrique Holschuh -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20120525102037.gc6...@khazad-dum.debian.net
Re: Moving /tmp to tmpfs makes it useless
On Fri, 25 May 2012, Salvo Tomaselli wrote: > nevertheless. And somehow the OOM killer was not even triggered, i kept > ending > up with a system where i had a working shell but could not run certain > commands (such as kill), but i could see the output of "free" for example. Switch to the deadline IO scheduler (and forget about any attempts at fairness for desktop loads, but at least it never screws up as badly as CFQ), and make proper use of cgroups. And disable memory overcommit, which actually *forces* you to have quite a lot of both core and swap for things to even run... you have no idea how much of a memory pig most crap nowadays is until you do. -- "One disk to rule them all, One disk to find them. One disk to bring them all and in the darkness grind them. In the Land of Redmond where the shadows lie." -- The Silicon Valley Tarot Henrique Holschuh -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20120525102451.gd6...@khazad-dum.debian.net
Re: Moving /tmp to tmpfs is fine
On 25/05/2012 18:20, Salvo Tomaselli wrote: > Double-click on a .tar causes it to be unpacked in /tmp/something. > I suppose a lot of not so skilled users do that instead of tar -xf That doesn't seem to happen with file-roller. Perhaps you need to file a bug with your graphical archiver program. -- Kind regards, Loong Jin signature.asc Description: OpenPGP digital signature
Re: Moving /tmp to tmpfs makes it useful
Henrique de Moraes Holschuh dixit: >On Fri, 25 May 2012, Thomas Goirand wrote: >> for small files, and in that case, it's faster. In reality, it's >> not that much faster, thanks to Linux caching of the filesystem, > >Under heavy filesystem IO load, yes it is. By several orders of magnitude. Not just that. It’s faster because the files don’t even get written to disc if they’re kept for, say, five minutes, and aren’t just short-lived. If you’ve got a slow hard disc (say low-end spec or old system), RAM’s still cheaper for tmpfs: no matter whether tmpfs or a real on-disc filesystem, buffer cache would still get it, AND the writes would be very slow (especially for short-lived files, _if_ the system is otherwise busy and would not wait before writing them to disc), and the system could concentrate on getting the files we want to disc instead of the temporary ones. And then there’s the CF card usecase. Or SSD, nowadays, I guess (my IBM X40 just got two CF cards to replace the HDD). There, you’d absolutely want to minimise disc writes. And just the chance that it could not end up there (except as part of swap) is better then the chance it could end up there (plus journal writes). >In fact it is the other way. We have /var/tmp for the large file since >about forever, and important platforms that have /tmp in memory since the >early 2000's (Solaris) Right. /tmp always had size limitations, it used to be no bigger than a few MiB, and people were ALWAYS tought to use /var/tmp for files that are large, should not be removed across reboots or from a cronjob pruning /tmp files older than 7 days, or a combination of these. (Uglities like /usr/tmp notwithstanding, although I’ve mostly seen that as compat symlink to /var/tmp later.) I’m absolutely for /tmp as tmpfs by default. Even, no, especially on low-end systems. Heck, be lucky it’s tmpfs and not Classical BSDs’ mfs (basically a variable-size 4.2FFS ramdisc, not just an object store in the buffer cache). Every tool either supports setting TMPDIR=/var/tmp before running it or is buggy. Go fix these instead, and then just run them with that if you need them to process files you don’t want in tmpfs. bye, //mirabilos -- [...] if maybe ext3fs wasn't a better pick, or jfs, or maybe reiserfs, oh but what about xfs, and if only i had waited until reiser4 was ready... in the be- ginning, there was ffs, and in the middle, there was ffs, and at the end, there was still ffs, and the sys admins knew it was good. :) -- Ted Unangst über *fs -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/pine.bsm.4.64l.1205251057460.4...@herc.mirbsd.org
Re: gitpkg with a quilt export hook
On Fri, May 25, 2012 at 1:32 AM, Brian May wrote: > On 23 May 2012 23:31, Bastien ROUCARIES wrote: >> 9. git checkout -b debian-patches/6.7.7.0-1 >> 10. git checkout -b debian/6.7.7.0-1 # create new debian branch > > Just curious here, why do you create a new branch for every Debian > revision? Wouldn't it be better to create a branch for every upstream > revision, so you can track changes made between Debian revisions? If you see the workflow, you could do this limiting the diff to debian dir, and doing merging allow that debian sub branch (check with gitk or qgit) Branch are really really cheap Moreover one branch one version allow me to easilly release to stable/testing from experimental. If you use only one debian branch you could not do this easilly. Moreover you have a branch for every upstream revision used on debian called upstream/version. Bastien > I think there are trade-offs here, just curious what your reasoning is . > > Thanks > -- > Brian May -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/cae2spaz_vpfyvceslxaxmzdxfparjwqglgtu_hpu5nfgdib...@mail.gmail.com
Re: Moving /tmp to tmpfs makes it useless
On Fri, May 25, 2012 at 02:22:24AM +0300, Serge wrote: > I've read across different debates about whether using tmpfs is good or bad > but I could not find the most important reason, so here it is... I haven't got anything particularly new to add to the discussion here. But I would like to refer you to the previous discussion on the topic. I am well aware that the default does not satisfy all use cases, but that's simply not possible for this problem. The best we can do is have a good default. That could be by improving the tmpfs default sizes and behaviour (my preferred solution). It could be by defaulting to not using a tmpfs. However, the majority of software which finds the tmpfs too small has unreasonable expectations of what can be expected to be available (by default). http://lists.debian.org/debian-devel/2011/04/msg00554.html http://lists.debian.org/debian-devel/2011/11/msg00281.html http://lists.debian.org/debian-devel/2012/02/msg00473.html http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=630615 http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=630615#50 http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=630615#89 http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=665406 http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=674517 http://fedoraproject.org/w/index.php?title=Features/tmp-on-tmpfs Also see sysvinit in experimental. > And the most important thing: file managers, browsers, image editors, > cd burners — these are not some rare scientific stuff, but a common > programs, that most people use every day. Putting them on a small tmpfs > will break them. Is this because the tmpfs is too small, or because those applications are broken? Please see the discussion above first. > Do not mount /tmp as tmpfs by default. Instead... > Debian already allows custom partitioning during the system install. For > example it's possible to mount /tmp on a separate partition. The suggestion > is to extend partitioner with a new option "Configure tmpfs partitions". > That option should allow to mount anything as tmpfs (not just /tmp, but > also /var/run, /media, /opt or whatever the user might want). It would be > nice to have the `size` option there as well. I did want to have this for wheezy (#633299). But I lacked the time and familiarity with the d-i code, and the d-i developers also have higher priorities. I'm sure support for tmpfs in the partitioner would be welcome if you want to add this functionality. Regards, Roger -- .''`. Roger Leigh : :' : Debian GNU/Linuxhttp://people.debian.org/~rleigh/ `. `' schroot and sbuild http://alioth.debian.org/projects/buildd-tools `-GPG Public Key F33D 281D 470A B443 6756 147C 07B3 C8BC 4083 E800 -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20120525114403.gm22...@codelibre.net
Re: Moving /tmp to tmpfs makes it useless
Jon Dowland dixit: >On Fri, May 25, 2012 at 09:46:37AM +0200, Vincent Danjean wrote: >> If some kind of sync is required by the application, I think this is >> because the application want to ensure the data are really written to >> the disk so that their state remains coherent even in case of crash. >> If the application is ok to have this kind of data written to >> tmpfs (ie in memory), I do not see the interest of using sync at >> first. Can someone shows me a valid use case of sync on tmpfs? > >I'd agree with your analysis, it's buggy for them to aggressively sync for >writes >to /tmp. No. An application might not know it’s writing to tmpfs (for example, if it wasn’t even written for an operating system with tmpfs in the first place). And it might want to use sync writes. The user of such application might want to tell it to write into a mounted tmpfs, for speed, if they don’t care about data loss – and since sync is nop on tmpfs, this is A Good Thing. bye, //mirabilos -- ch: good, you corrected yourself. ppl tend to tweet such news immediately, sth. like "grml devs seem to be buyable" dileks: we _are_. if you throw enough money in our direction, things will happen everyone is buyable, it's just a matter of priceand now comes [mira] and uses this as a signature ;0 -- they asked for it… -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/pine.bsm.4.64l.1205251142500.4...@herc.mirbsd.org
Re: Moving /tmp to tmpfs is fine
Chow Loong Jin dixit: >On 25/05/2012 18:20, Salvo Tomaselli wrote: >> Double-click on a .tar causes it to be unpacked in /tmp/something. >> I suppose a lot of not so skilled users do that instead of tar -xf > >That doesn't seem to happen with file-roller. Perhaps you need to file a bug Hm. mc does put things into /tmp as does debdiff, but the latter at least honours TMPDIR (and --no-unpack-tarballs). But in the very most cases, I *do* want them to be extracted in /tmp as they “usually” fit. So I’d rather have a heuristic put into the file manager whether to set TMPDIR before calling the extraction utility (or which tmpdir to use if it designates the extraction place by itself). mc maintainers, are you listening? bye, //mirabilos -- "Using Lynx is like wearing a really good pair of shades: cuts out the glare and harmful UV (ultra-vanity), and you feel so-o-o COOL." -- Henry Nelson, March 1999 -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/pine.bsm.4.64l.1205251155530.7...@herc.mirbsd.org
Re: Moving /tmp to tmpfs is fine
2012/5/25 Neil Williams wrote: > You cannot expect to mix those two worlds and for things to "just > work". Easy. Let's leave /tmp on a real disk and both world will "just work". > If program A is too resource-hungry, find (or write) program B. Or fix the program A, right? And here we go... By default the program Debian is too memory-hungry (with large tmpfs) or breaking apps (with small tmpfs). Let's fix that? ;) > The default is fine and sane but no default will ever satisfy every > possible device. Low memory devices have many many more problems than > just where /tmp is mounted. Every system becomes "Low memory" with these defaults. Assuming you've set your tmpfs size to 20% you need 2.5GB memory just to "temporarily" unpack kernel sources and check for some files. Or it's supposed to be unpacked not in /tmp? And other programs should not be using /tmp to write large files? Then what *real* programs should use /tmp now? Is it useless for popular programs? > That does NOT mean that Debian should change the default just to suit > low memory devices. [...] we just have to be careful what applications > we use. So instead of fixing the defaults you suggest everybody to drop the programs they use (mc, firefox, mysql)? ;) -- Serge -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/CAOVenEpEOyAyOVKszAJvdJg2Urf-LqvNzC1=kvvrs04kcne...@mail.gmail.com
Re: Moving /tmp to tmpfs makes it useless
2012/5/25 Henrique de Moraes Holschuh wrote: > Switch to the deadline IO scheduler > [...] and make proper use of cgroups. > [...] And disable memory overcommit Ehm, so you assume that heavy swapping is not a bad thing, and instead of fixing a single default option you suggest every debian user to tweak and/or rebuild the kernel? Are you serious? ;) -- Serge -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/caoveneowsedle2g-uapwxpaumteh2agsgthleppta50gr9q...@mail.gmail.com
I just broke the bts (Fw: /bin/htpasswd exists in apache2-utils ans mini-httpd)
Hi there, this Bugreport (see below) seems to break the bts [1] just for the record. kind regards, Michael [1] http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=674515 Beginn der weitergeleiteten Nachricht: Datum: Fri, 25 May 2012 09:22:18 +0200 Von: Michael Stummvoll An: sub...@bugs.debian.org Betreff: /bin/htpasswd exists in apache2-utils ans mini-httpd Package: mini-httpd,apache2-utils Severity: normal Dear Maintainers, two different binaries with the same name /bin/htpasswd are provided by apache2 and mini-httpd. If you install mini-httpd after apache2 the htpasswd from mini-httpd will override the file from apache2. The file from apache2 is downward-compatible to the from mini-httpd. This means it provides all the options like mini-httpd, but mini-httpd does not provide all options from apache2. So it doesn't make any sense the the htpasswd from mini-httpd overrides that from apache2. I think thats a bug. It could be resolved by let the packages conflict or let mini-httpd provide something like mini-htpasswd (which could get linked to htpasswd as long as apache2 does not exists) kind regards, Michael Stummvoll -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20120525144027.371954b5@eddie
Re: Moving /tmp to tmpfs makes it useful
2012/5/25 Thorsten Glaser wrote: > It’s faster because the files don’t even get written to disc if > they’re kept for, say, five minutes, and aren’t just short-lived. Theoretically, yes. But I'm not asking to forbid everybody to use tmpfs. We're talking about defaults and about the most used test cases. Can you name some popular real-world program that write only small files and become noticeably faster when /tmp is on tmpfs? > Every tool either supports setting TMPDIR=/var/tmp before running > it or is buggy. Go fix these instead Do I understand you right? You suggest every tool that need large tmp files to use /var/tmp instead? Ok, then... what popular tools should use /tmp now? > And then there’s the CF card usecase. Or SSD, nowadays, I guess > (my IBM X40 just got two CF cards to replace the HDD). There, you’d > absolutely want to minimise disc writes. /tmp on tmpfs won't help in that case. You do not reduce number of disk writes, you just move them to other directories. As you suggested, all the programs will still use files i.e. in /var/tmp. Or they'll write to swap if tmpfs is large enough. Or they'll just break, if it's small. But you'll get the same number of SSD writes (or maybe even more, because of other applications being heavily swapped as well). Again, I'm not asking to drop this feature. I'm asking to have it disabled *by default* but supported by debian installer, so really smart people, that know what may be broken by it, but really need it, could enable it. -- Serge -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/CAOVenEp+mr3yNi=4obcg2qtztfjuaj_svcus1y-7ij5mnhj...@mail.gmail.com
Re: Moving /tmp to tmpfs makes it useless
On Fri, May 25, 2012 at 11:11:06AM +0200, Salvo Tomaselli wrote: > > Files which are written on a regular filesystem stay on memory. This is > > called the buffer cache. Whenever they are not used and/or the system > > needs to reclaim memory, they are trashed. > > Files which are written on a tmpfs stay on memory. Whenever they are not > > used and/or the system needs to reclaim memory, they are swapped. > > > > See? No difference. > > You seem to forget that memory is not an unlimited resource, the > system might need it for other things, and in that case a large > tmpfs causes severe slowdown (and even complete freeze). So what? If you write to a normal file system, it goes into the page cache, which is pretty much the same as writing into tmpfs. In both cases if you have swap configured, the data will get pushed to disk; either to the file system or to swap, as memory requirements dictate. The main advantage of tmpfs is that it gets wiped on reboot, and so it prevents people and applications from thinking that they can keep stuff in /tmp forever. It's also faster because a file system has to do extra work to make sure the files are preserved after a reboot. - Ted -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20120525125238.ga5...@thunk.org
Re: I just broke the bts (Fw: /bin/htpasswd exists in apache2-utils ans mini-httpd)
Hi, On 2012-05-25 13:40, Michael Stummvoll wrote: Hi there, this Bugreport (see below) seems to break the bts [1] just for the record. ow...@bugs.debian.org is the place to report this. Thanks, -- Jonathan Wiltshire j...@debian.org Debian Developer http://people.debian.org/~jmw 4096R: 0xD3524C51 / 0A55 B7C5 1223 3942 86EC 74C3 5394 479D D352 4C51 -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/8472db3abfc2bd8f55307710e6075...@hogwarts.powdarrmonkey.net
Re: Moving /tmp to tmpfs makes it useful
On Fri, May 25, 2012 at 03:50:31PM +0300, Serge wrote: > /tmp on tmpfs won't help in that case. You do not reduce number of disk > writes, you just move them to other directories. As you suggested, all > the programs will still use files i.e. in /var/tmp. Or they'll write to > swap if tmpfs is large enough. Or they'll just break, if it's small. Why do you assume tmpfs contents is always written to the swap? -- WBR, wRAR signature.asc Description: Digital signature
Re: I just broke the bts (Fw: /bin/htpasswd exists in apache2-utils ans mini-httpd)
> ow...@bugs.debian.org is the place to report this. > > Thanks, > I just send the mail there, thanks for that info. Kind regards, Michael -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20120525145735.3fc77a31@eddie
Re: Moving /tmp to tmpfs makes it useless
Josselin Mouette wrote: > Files which are written on a regular filesystem stay on memory. This is > called the buffer cache. Whenever they are not used and/or the system > needs to reclaim memory, they are trashed. > Files which are written on a tmpfs stay on memory. Whenever they are not > used and/or the system needs to reclaim memory, they are swapped. There is one significant difference though. When you read data back to memory from swap, the kernel does not remember that it already exists on disk; when the data is evicted from memory again, it is unnecessarily rewritten to disk rather than just dropped. Thus, if you do multiple read iterations through a large set of data (which does not fit in memory) on tmpfs, each iteration does disk read AND write rather than just read. -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/1337950893.1831.25.camel@glyph.nonexistent.invalid
Re: Moving /tmp to tmpfs makes it useless
Le vendredi 25 mai 2012 à 16:01 +0300, Uoti Urpala a écrit : > There is one significant difference though. When you read data back to > memory from swap, the kernel does not remember that it already exists on > disk; when the data is evicted from memory again, it is unnecessarily > rewritten to disk rather than just dropped. Thus, if you do multiple > read iterations through a large set of data (which does not fit in > memory) on tmpfs, each iteration does disk read AND write rather than > just read. I don’t know enough of the kernel innards, but it sounds to me that if a previously swapped page hasn’t been modified, it should be kept on swap *and* memory as long as possible. If it does not, it sounds like a bug, but it would indeed lead to the behavior you describe for this specific usage pattern. -- .''`. Josselin Mouette : :' : `. `' `- -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/1337952290.14895.407.camel@pi0307572
Re: Moving /tmp to tmpfs makes it useful
On 25/05/12 13:50, Serge wrote: Again, I'm not asking to drop this feature. I'm asking to have it disabled *by default* but supported by debian installer, so really smart people, that know what may be broken by it, but really need it, could enable it. +1 On 25/05/12 13:52, Ted Ts'o wrote: So what? If you write to a normal file system, it goes into the page cache, which is pretty much the same as writing into tmpfs. In both cases if you have swap configured, the data will get pushed to disk; That's not at all the same, the page cache is more temporary, it's getting flushed to disk pretty quick if memory is tight (presumably) but in the same situation using tmpfs going to swap is surely going to be more disruptive? I won't pretend to know the details half as well as other commentators but it seems only logical that you'd end up pushing more memory from other running processes onto disk as well as (or instead of) the tmpfs memory, which is going to have to get reloaded at some point. And anyway, not everybody uses swap, in which case this "default" is not entirely viable. I, for one, had no idea this had become default for Debian and I think it's likely to be one of those things that jumps out to bite people who weren't expecting it at some inconvenient moment. I'm sure the project veterans and more attentive readers of this list are tired of recurring arguments like this, but usually if something is recurring it is for a reason. Given my general "no swap" preference, I'm glad this has come up again so that I'm aware of it this time. The tmpfs setup seems far more appropriate as a performance tweak for admins than as a default. Where there is plenty of RAM, buffer cache makes the difference largely negligible. But where there isn't an abundance of RAM, it could needlessly cause problems (especially without swap). Not big problems perhaps, and not likely to many, but that could still be a few thousand people with a project like Debain, so I just don't see the issue with leaving /tmp on disk, why complicate the matter? -Will -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4fbf8dda.9070...@willdaniels.co.uk
Re: Moving /tmp to tmpfs makes it useful
2012/5/25 Andrey Rahmatullin wrote: > Why do you assume tmpfs contents is always written to the swap? Most programs, using /tmp for temporary files, may write *large* files there. So most programs fill tmpfs with large files. So tmpfs grows and swaps out (probably swapping out other applications as well). I really can't think of any popular application that write a lot of small-only files and can benefit from /tmp being on tmpfs. And I know a lot of *popular* apps that will have troubles because of that. If default settings cause trouble in a lot of cases and cause no benefits, than it's a bad default, and it should be fixed. :) -- Serge -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/caoveneqkgq7pud1xenervfxwzrh3qxk_xd9eweuz7nkjdt2...@mail.gmail.com
Re: Moving /tmp to tmpfs makes it useful
On Fri, May 25, 2012 at 05:08:37PM +0300, Serge wrote: > > Why do you assume tmpfs contents is always written to the swap? > > Most programs, using /tmp for temporary files, may write *large* files > there. So most programs fill tmpfs with large files. So tmpfs grows and > swaps out (probably swapping out other applications as well). > > I really can't think of any popular application that write a lot of > small-only files and can benefit from /tmp being on tmpfs. And I > know a lot of *popular* apps that will have troubles because of that. /tmp 8,0G 60M 8,0G1% /tmp Does this count as "large files"? As "a lot of small-only files"? -- WBR, wRAR signature.asc Description: Digital signature
Re: Moving /tmp to tmpfs makes it useless
On Fri, May 25, 2012 at 11:44:15AM +, Thorsten Glaser wrote: > No. An application might not know it’s writing to tmpfs (for > example, if it wasn’t even written for an operating system > with tmpfs in the first place). And it might want to use sync > writes. The user of such application might want to tell it to > write into a mounted tmpfs, for speed, if they don’t care about > data loss – and since sync is nop on tmpfs, this is A Good Thing. Sorry, you (and hmh) are right. I think my point was, you tend to do syncs if you care about the data being written, but /tmp is a volatile filesystem whether it's a ramdisk or not: many long-running timeshares will reap files from /tmp at times other than boot up. (Please don't CC me, I'm on the list.) -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20120525143420.GC31519@debian
Re: Moving /tmp to tmpfs makes it useless
On Fri, May 25, 2012 at 05:08:42PM +0800, Thomas Goirand wrote: > On 05/25/2012 03:22 PM, Jon Dowland wrote: > > How much RAM do you have / how big is your /tmp(fs)? The fact this caused > > you trouble suggests to me that they must be very small. > > > What if we're installing Debian on a very small system, and that > we need operations with big files in /tmp? > > How much RAM he has is totally irrelevant, the issue will be the > same if our users have less RAM, or bigger files to handle. It's relevant for me to try to understand the specific problem this fellow has, rather than generalisations. The reply with Roger's changelog snippet seems to address the general case, though. -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20120525143526.GD31519@debian
Re: Moving /tmp to tmpfs makes it useless
]] Josselin Mouette > Le vendredi 25 mai 2012 à 11:11 +0200, Salvo Tomaselli a écrit: > > You seem to forget that memory is not an unlimited resource, the > > system might need it for other things, and in that case a large > > tmpfs causes severe slowdown (and even complete freeze). > > Then increase your swap size. > > When you run out of swap, the system fills memory with things that > could be swapped out, and has to reduce the buffer cache. Once swap usage hits ~2x physical memory size, you'll run into trouble with kswapd on Linux consuming large amounts of CPU, though. On the other side, if this means we end up getting that bug fixed, I wouldn't complain. :-) -- Tollef Fog Heen UNIX is user friendly, it's just picky about who its friends are -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/87mx4wxqem@xoog.err.no
Re: Moving /tmp to tmpfs makes it useless
Serge writes: > Suggestion > == > Do not mount /tmp as tmpfs by default. Instead... > Debian already allows custom partitioning during the system install. For > example it's possible to mount /tmp on a separate partition. The suggestion > is to extend partitioner with a new option "Configure tmpfs partitions". > That option should allow to mount anything as tmpfs (not just /tmp, but > also /var/run, /media, /opt or whatever the user might want). It would be > nice to have the `size` option there as well. You still haven't said what the new default should be. I think having / and /tmp share the same file system is a bad idea, because then writing lots of stuff to /tmp would potentially fill up the root file system (that typically also includes /var) and then cause a lot of breakage. However, if I put /tmp in a separate (on-disk) file system, I have to decide how much disk space to I want to permanently allocate for temporary data, in addition to the disk space permanently allocated for swapping. I think separating these two types of storage really doesn't serve any purpose, and it'd be much better to just allocate one partition for swapping and temporary files and have the space assigned dynamically where its needed. Which is exactly what tmpfs allows me to do. Best, -Nikolaus -- »Time flies like an arrow, fruit flies like a Banana.« PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6 02CF A9AD B7F8 AE4E 425C -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/87d35s2wb6@inspiron.ap.columbia.edu
Re: Moving /tmp to tmpfs is fine
On Fri, 25 May 2012 15:25:58 +0300 Serge wrote: > 2012/5/25 Neil Williams wrote: > > > You cannot expect to mix those two worlds and for things to "just > > work". > > Easy. Let's leave /tmp on a real disk and both world will "just work". Do you not use swap? > > If program A is too resource-hungry, find (or write) program B. > > Or fix the program A, right? And here we go... By default the program > Debian is too memory-hungry (with large tmpfs) or breaking apps (with > small tmpfs). Let's fix that? ;) Write program C which does only the bits of what program A does but does it by using a lot less resources (generally there's no point differentiating between machines with low RAM and low storage space). e.g. netsurf in place of iceweasel. GPE instead of XFCE/Gnome. Yes you lose functionality but functionality takes up resources, so something has to give. > > The default is fine and sane but no default will ever satisfy every > > possible device. Low memory devices have many many more problems than > > just where /tmp is mounted. > > Every system becomes "Low memory" with these defaults. Huh? Not true. The vast majority of systems have large amounts of swap, so tmpfs never runs out until the swap space is full - it just gets very, very much slower. > Assuming you've > set your tmpfs size to 20% you need 2.5GB memory just to "temporarily" > unpack kernel sources and check for some files. And? The machines I use to unpack and repack kernel packages handle that quite nicely. > > That does NOT mean that Debian should change the default just to suit > > low memory devices. [...] we just have to be careful what applications > > we use. > > So instead of fixing the defaults you suggest everybody to drop the > programs they use (mc, firefox, mysql)? ;) On machines which don't have the resources for those programs, yes. There are alternatives out there - change to a less hungry program or expand the hardware. I write packages for both types of machines - I use powerful servers with lots of RAM and lots and lots of disc space for the repetitive tasks of creating smaller packages which can be more easily handled by low resource devices which have no swap, less than 1Gb of SSD and only 512Mb RAM. Then I write lots of new code on nice shiny workstations with lots of RAM and lots of disc space and deploy those packages on the units which gives a full user interface environment using (currently) ~21% of the available 512MB of RAM. rootfs1020M 757M 264M 75% / tmpfs 62M 0 62M 0% /lib/init/rw No problem with tmpfs being in RAM, it's all about being rational in the selection of packages. That way, the units can run on minimal power for days when the original desktop would drain the UPS in 10 minutes. Different hardware -> different software selection. When a package requires too many resources for a particular device, you simply choose another package, write another package or expand the hardware. As the proverb goes, you can't fit a gallon into a pint pot. http://www.emdebian.org/ -- Neil Williams = http://www.linux.codehelp.co.uk/ pgpKlMn11btUk.pgp Description: PGP signature
Re: Moving /tmp to tmpfs makes it useless
> > Switch to the deadline IO scheduler > > [...] and make proper use of cgroups. > > [...] And disable memory overcommit [Serge] > instead of fixing a single default option you suggest every debian > user to tweak and/or rebuild the kernel? Are you serious? ;) What?!? "and/or rebuild the kernel" is FUD. /proc/sys/vm/overcommit_memory has been tweakable at runtime for almost as long as I can remember (at least Linux 2.4, possibly earlier). The tweak is: # echo vm.overcommit_memory=2 >> /etc/sysctl.conf # sysctl -p -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20120525152520.ga3...@p12n.org
Re: Moving /tmp to tmpfs makes it useless
2012/5/25 Jon Dowland wrote: >> I wasn't able to watch a web presentation (from something like >> vimeo/youtube), because there was not enough free space in /tmp for >> flash player to download and show it. > > I thought those were streaming video sites? You mean, those sites don't store temporary files? They do. Just opened youtube url and watched a growing /tmp/FlashXX2pKrFq (new flash player unlinks those files after open(), you may not notice them, but they're still there, check `lsof | grep Flash`). > Do you mean flash player or something else? Yes, it was a flash player powered by adobe flash plugin. >> Ok, I agree. There're two main /tmp usages. Some applications use it to >> save memory (databases, archive managers, cd burners, image processors, >> science software, the flash player after all). Other apps use it for >> communication (browsers, mail clients). But in both cases these files >> may be large and should be still stored on real disk, not on tmpfs. >> And having /tmp on tmpfs makes it useless for both usages. > > Why is it useless for the second case? Because programs like browsers or mail clients may also store large files. You can just click on a 300MB archive or 1GB video file in Firefox and select "Open with" to view it. And I've seen real users doing that (it's just two clicks to view a file, why not). >> The matter is that exceptions are *not rare*. A simple 2-hours-long >> video presentation may cause you problems. Pressing Enter in `mc` on >> linux-kernel-src.tar.bz2 may cause you problems. > > How much RAM do you have / how big is your /tmp(fs)? The fact this > caused you trouble suggests to me that they must be very small. On that machine I had 1GB of ram and about 250MB of /tmp in tmpfs. I configured that manually more than a year ago and dropped that setting later when realized that its mostly useless and just slows me down having me to manually untar archives (instead of single Enter in mc) and copying links to curl/wget (instead of 2-clicks in firefox) in order to have them stored not in /tmp. -- Serge -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/CAOVenEr9-0vAvtAYe7ks5V6d=ethxqjpsc30k9zsrz9mv4n...@mail.gmail.com
Re: Moving /tmp to tmpfs makes it useful
Serge dixit: >cases. Can you name some popular real-world program that write >only small files and become noticeably faster when /tmp is on tmpfs? Most shell scripts using << (here documents). mc, out of all things, as long as the temporary files (e.g. of a tarball) fit inside it. And countless others. The majority, in any case. What you want is to optimise for a corner case. >> Every tool either supports setting TMPDIR=/var/tmp before running >> it or is buggy. Go fix these instead > >Do I understand you right? You suggest every tool that need large >tmp files to use /var/tmp instead? I do not suggest it. I say that every tool that doesn’t honour when the user sets TMPDIR=/var/tmp is broken (but that’s nothing new, as it has always been like that), and that the user should set that in some corner cases like yours. >Ok, then... what popular tools should use /tmp now? /tmp (on tmpfs) should still be the default, as that’s the common case, and one that can be optimised for well. >/tmp on tmpfs won't help in that case. You do not reduce number of disk >writes, you just move them to other directories. As you suggested, all No. You reduce the number of disc writes quite a bit, as all current filesystems use journalling and write more additional metadata than paging would ever use. (And, from the other mail, the read accesses could be fixed (for the slow HDD case) or don’t matter that much (for the CF/SSD case).) >the programs will still use files i.e. in /var/tmp. Or they'll write to >swap if tmpfs is large enough. Or they'll just break, if it's small. Ever heard of statfs? >But you'll get the same number of SSD writes (or maybe even more, >because of other applications being heavily swapped as well). No. The common case wouldn’t swap them a lot, besides read-only things (.text and .rodata, plus some bits) would not need to be written anyway. So you’d still get less. >Again, I'm not asking to drop this feature. I'm asking to have it disabled >*by default* but supported by debian installer, so really smart people, >that know what may be broken by it, but really need it, could enable it. And I’m proving that it should be *enabled by default*, so that really smart people like you, who need it disabled in corner cases, can do that. And I’m suggesting that file managers that extract archives apply some heuristics (size of the archive vs. size and current usage of /tmp) to determine whether they should use /tmp or /var/tmp. >Most programs, using /tmp for temporary files, may write *large* files No. The absolute majority writes *small* files there. >there. So most programs fill tmpfs with large files. So tmpfs grows and >swaps out (probably swapping out other applications as well). You can always limit the size of your tmpfs? I think in Debian it is already sized to a quite small (IMHO) portion of the main memory by default. >I really can't think of any popular application that write a lot of >small-only files and can benefit from /tmp being on tmpfs. And I $GUI_BROWSER_DU_JOUR with XTaran’s unburden-home-dir, for example. bye, //mirabilos -- "Using Lynx is like wearing a really good pair of shades: cuts out the glare and harmful UV (ultra-vanity), and you feel so-o-o COOL." -- Henry Nelson, March 1999 -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/pine.bsm.4.64l.1205251615020.20...@herc.mirbsd.org
Re: Moving /tmp to tmpfs makes it useless
2012/5/25 Nikolaus Rath wrote: > You still haven't said what the new default should be. The suggestion was to just leave it as it was configured in partitioner. If it was configured as a part of root fs leave it as it is. If it's on a separate partition leave it there. Just no automatic tmpfs. If you ask what I prefer personally... Well, for regular users I would suggest put everything on a single filesystem. Regular users use to fill entire disk space and then have headaches resizing filesystems or worse, start saving files in some strange directories (/usr, /var) because other partitions are already full. This is especially true for notebooks, where you can't easily upgrade HDD. For advanced users, or users with 500+GB HDD I would probably put /home on a separate partition. As for servers it's pretty usual to have all the /home, /var, /tmp, /usr and /opt on separate partitions. But that's IMHO, and that's a question for a different topic. :) -- Serge -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/caoveneqnjle6czlmh3azaob7rj8z1qomtnco97q9rnpkjyv...@mail.gmail.com
Re: Moving /tmp to tmpfs is fine
2012/5/25 Neil Williams wrote: > Do you not use swap? I use it for suspend-to-disk. > Yes you lose functionality but functionality takes up resources, so > something has to give. Which functionality will I lose by placing /tmp on the real disk? > The vast majority of systems have large amounts of swap, so tmpfs never > runs out until the swap space is full - it just gets very, very much > slower. Right, system will become unusable and user will press Reset button far before that. >> Assuming you've set your tmpfs size to 20% you need 2.5GB memory just >> to "temporarily" unpack kernel sources and check for some files. > And? The machines I use to unpack and repack kernel packages handle that > quite nicely. Since we're talking about default settings, you assume default debian system to have at least 2.5GB just to be able to unpack kernels? > Different hardware -> different software selection. I don't understand your point. I could understand it if we were choosing among benefits that most users get from /tmp being on disk and /tmp being on tmpfs. But there're NO benefits in having /tmp on tmpfs. It works (not works better, just works somehow) only on systems with a lot of RAM. And nobody still named any popular programs, that can definitely benefit from that. While there're many programs that either break or may render system unstable. Yes, /tmp on tmpfs works in many cases. But /tmp on disk works always. Why would we select the worst of these two options? I don't understand, do you suggest to break some applications and make system less stable for nothing? What's a good part? -- Serge -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/CAOVenEr24CyHFALpn5J0mnfqLk+S3ovHqL+=bMR6qr-rVH=9...@mail.gmail.com
Re: Moving /tmp to tmpfs makes it useless
On Fri, 2012-05-25 at 10:02 -0400, Nikolaus Rath wrote: > I think having / and /tmp share the same file system is a bad idea, > because then writing lots of stuff to /tmp would potentially fill up the > root file system (that typically also includes /var) and then cause a > lot of breakage. If only ext*fs supported quotas... (Snark aside, does tmpfs support quotas yet/will it ever?) Weldon -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/1337967711.26842.3.ca...@minerva.trit.us
Re: Moving /tmp to tmpfs makes it useless
Serge writes: > 2012/5/25 Nikolaus Rath wrote: > >> You still haven't said what the new default should be. > > The suggestion was to just leave it as it was configured in partitioner. > If it was configured as a part of root fs leave it as it is. If it's on > a separate partition leave it there. Just no automatic tmpfs. I don't think /tmp as part of the rootfs is a good idea for the reasons outlined in the rest of my mail. That leaves you with using tmpfs or the installer having to ensure that there's a separate partition for /tmp. If you do the later, however, you may just as well ensure that there is a big enough swap partition and stick with tmpfs. Best, -Nikolaus -- »Time flies like an arrow, fruit flies like a Banana.« PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6 02CF A9AD B7F8 AE4E 425C -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/877gw02md7@inspiron.ap.columbia.edu
Re: Moving /tmp to tmpfs makes it useless
> So what? If you write to a normal file system, it goes into the page > cache, which is pretty much the same as writing into tmpfs. tmpfs will make it stay forever in the RAM, caches are flushed to disk and their space can be used for new things. - Ted -- Salvo Tomaselli -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/201205252118.32729.tipos...@tiscali.it
Re: Moving /tmp to tmpfs makes it useful
On 05/25/2012 10:20 PM, Andrey Rahmatullin wrote: > /tmp 8,0G 60M 8,0G1% /tmp > Does this count as "large files"? > As "a lot of small-only files"? > Exactly how is this a practical explanation and example? :/ Are you saying that in *your case* /tmp is almost unused? That doesn't even remotely help to write this. Serge made a much better point with common apps. Thomas -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4fbfdc52.20...@debian.org
Re: Moving /tmp to tmpfs makes it useful
On 05/26/2012 12:21 AM, Thorsten Glaser wrote: > What you want is > to optimise for a corner case. > Ah... So Mysql, flash videos, mc, image editors, cd burners, and all the (very common) examples he gave are just "corner cases" in your book? We must have very different readings... Thomas -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4fbfdd03.3050...@debian.org
Re: Moving /tmp to tmpfs makes it useless
Salvo Tomaselli dixit: >tmpfs will make it stay forever in the RAM, caches are flushed to disk and >their space can be used for new things. For sane values of “forever”, usually a second of so. As long as there’s a way to change it¹, please don’t optimise for the uncommon behaviour. ① tmpfs can be changed. Dropping file-rc compatibility or SCC architectures to optimise for Poettering crap or amd64 cannot be changed that easily, and thus my “please” doesn’t apply to those. Just to have that said as well. bye, //mirabilos -- I believe no one can invent an algorithm. One just happens to hit upon it when God enlightens him. Or only God invents algorithms, we merely copy them. If you don't believe in God, just consider God as Nature if you won't deny existence. -- Coywolf Qi Hunt -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/pine.bsm.4.64l.1205251921280.20...@herc.mirbsd.org
Re: Moving /tmp to tmpfs makes it useless
On 05/25/2012 05:33 PM, Mehdi Dogguy wrote: >> What if we're installing Debian on a very small system, and that we >> need operations with big files in /tmp? >> > > Increase your swap? So, in this case, we will have the following scenario: - An app writes in /tmp - There's not enough space, so the system starts swapping, including some apps. - The file gets written to /tmp, then gets read - Finally, the file gets deleted - Then we have randomly very sloppy reaction of apps that were swapped out so that the file could be written in /tmp. So, when the user is just opening a big file form any app that is writing to /tmp to give it to another one (let's say, a viewer...), he is *expecting* that operation to take some time, and most of the time, that's not really a problem to wait for some file to open. But a user rightly does *not* expect to have *random* waits for his $application to respond, which frankly, is *extremely* annoying. And that's what you are proposing by increasing the swap space. Cheers, Thomas -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4fbfe095.7070...@debian.org
Re: Moving /tmp to tmpfs makes it useless
Le 25/05/2012 04:00, Steve Langasek a écrit : > On Fri, May 25, 2012 at 03:57:28AM +0300, Serge wrote: >>> Every file that exists in /tmp on the system from which I'm writing this >>> exists there not because the application is saving memory but because the >>> application needs to share that file with other applications. That >>> includes a bunch of Kerberos ticket caches, several X server IPC >>> rendezvous points, gnupg-agent and ssh-agent data, and a bunch of UNIX >>> domain sockets for ORBit. > >> I agree, not everybody read FHS, some software may have bugs. > >> According to FHS these should go to /var/run (or /run, if you like). >> I mean, if you want to fix this, you should move those files to /run, >> you should not turn /tmp into /run because of them. > > Absolutely not. /run is a root-only directory, suitable only for > system-wide data/sockets. It absolutely must not be used for non-root data. > For normal users, shouldn't the applications use ~/.cache/ or whatever the xdg vars point to? -- Jean-Christophe Dubacq signature.asc Description: OpenPGP digital signature
Re: Moving /tmp to tmpfs makes it useless
In data Friday 25 May 2012 21:23:28, Thorsten Glaser ha scritto: > Salvo Tomaselli dixit: > >tmpfs will make it stay forever in the RAM, caches are flushed to disk and > >their space can be used for new things. > > For sane values of “forever”, usually a second of so. I've things in my /tmp that were created 5 days ago (aptitude's stuff) Yes if you don't run anything, no process will ever write on /tmp. -- Salvo Tomaselli -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/201205252203.57364.tipos...@tiscali.it
Re: Moving /tmp to tmpfs makes it useless
Weldon Goree writes: > On Fri, 2012-05-25 at 10:02 -0400, Nikolaus Rath wrote: >> I think having / and /tmp share the same file system is a bad idea, >> because then writing lots of stuff to /tmp would potentially fill up the >> root file system (that typically also includes /var) and then cause a >> lot of breakage. > > If only ext*fs supported quotas... Aren't quotas only for non-root and per file system? I think we're already safe from non-root filling up / because of the reserved 5%. Best, -Nikolaus -- »Time flies like an arrow, fruit flies like a Banana.« PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6 02CF A9AD B7F8 AE4E 425C -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/874nr42hz0@inspiron.ap.columbia.edu
Re: Moving /tmp to tmpfs makes it useless
Nikolaus Rath wrote: >I think having / and /tmp share the same file system is a bad idea, >because then writing lots of stuff to /tmp would potentially fill up >the >root file system (that typically also includes /var) and then cause a >lot of breakage. no, by default ext3/4 reserves 5% of the disk space for root user. so a regular user cannot fill up a partition -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/660adf11-f1e2-4b8f-8ab0-2940cdea8...@email.android.com
Re: Moving /tmp to tmpfs is fine
On Fri, May 25, 2012 at 08:14:10PM +0300, Serge wrote: > 2012/5/25 Neil Williams wrote: > > Different hardware -> different software selection. > > I don't understand your point. I could understand it if we were choosing > among benefits that most users get from /tmp being on disk and /tmp being > on tmpfs. But there're NO benefits in having /tmp on tmpfs. It works (not > works better, just works somehow) only on systems with a lot of RAM. This is plain wrong. NO benefits for tmpfs? "just works somehow"? Whatever other arguments you had, the statement above tells me you only look at _your_ use case and dismiss all others, or that you don't understand the different behaviours of fsync() (with enough memory, that is) on tmpfs, HDDs and SSDs. And no, "I really can't think of any popular application" is not a valid discussion point. iustin, happily using /tmp on tmpfs since many, many years ago, and configuring it as such on all Debian machines he installs (of various roles). -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20120525202638.ga19...@teal.hq.k1024.org
Re: Moving /tmp to tmpfs makes it useful
Serge writes: >> Every tool either supports setting TMPDIR=/var/tmp before running >> it or is buggy. Go fix these instead > > Do I understand you right? You suggest every tool that need large > tmp files to use /var/tmp instead? That's not a new suggestion, it's been standard practice for nigh-on _decades_... -miles -- Do not taunt Happy Fun Ball. -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/87zk8vdbnx@catnip.gol.com
Re: Moving /tmp to tmpfs makes it useless
Le Fri, May 25, 2012 at 12:44:03PM +0100, Roger Leigh a écrit : > > I haven't got anything particularly new to add to the discussion here. > But I would like to refer you to the previous discussion on the > topic. I am well aware that the default does not satisfy all use > cases, but that's simply not possible for this problem. The best we > can do is have a good default. That could be by improving the > tmpfs default sizes and behaviour (my preferred solution). It could > be by defaulting to not using a tmpfs. However, the majority of > software which finds the tmpfs too small has unreasonable expectations > of what can be expected to be available (by default). Dear Roger, thanks for your patient work. As you explained, with latest versions of sysvinit (currently un experimetnal), 20% of the whole memory (RAM + swap) will be used for /tmp. This will give a couple of gigabytes in systems with 8 Gb RAM + 8 Gb swap, which may be borderline when working with big files. On my system at work, I often need to sort big files, and I think that even coreutil's sort program does not check in advance if there will be enough space in /tmp. I often lose some time for forgetting to set TMPDIR before running a command. How about doing the reverse and defind the amount of memory that /tmp will not use ? This limit could be in the range of what is necessary to avoid a GNOME session to be killed, and fall back to a percentage if the memory is lower than that treshold ? Thhe current system of using a percentage of the memory, reduces the impact to the recommendation of increasing the swap size. For the moment, if one wants to give 1 Gb more to /tmp, he would have to increase the swap by 5 Gb. Have a nice week-end. PS: I think that it would be great if people could reduce the speed at which they send messages. We are about to release a "diversity statement", but the way this list is used, only males fluent in English and with a strong opintion express themselves. PPS: With the advent of virtualisation and cloud computing, we will definitely benefit stopping the applications to use /tmp as if it were a directory with a large amount of space. One of the reasons I do not set TMPDIR by default in my environment is that I would lose the benefit of having cruft cleaned at reboot. What we are perhaps missign here is a couple of sound recommendations, plus some facilities such as a temporary directory ready for the user somewhere, which would function like /tmp does. Something like getting XDG_SESSION_TMPDIR for free after login, without having to think on how to implement it. -- Charles Plessy Tsurumi, Kanagawa, Japan -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20120526003657.ga20...@falafel.plessy.net
Re: Moving /tmp to tmpfs makes it useful
2012/5/25 Thorsten Glaser wrote: Ok, we have guesses, let's do some TESTS... >> Can you name some popular real-world program that write >> only small files and become noticeably faster when /tmp is on tmpfs? > Most shell scripts using << (here documents). mc, out of all things, > as long as the temporary files (e.g. of a tarball) fit inside it. > And countless others. The majority, in any case. What you want is > to optimise for a corner case. Do they become faster? My tests show they don't (see below). I'm asking for *popular* apps, that create files in /tmp, *never put large files* there, and become *noticeably* faster with /tmp on tmpfs? And remember that there're popular apps that break because of that, so to make sure it's still a good thing all 3 conditions should be met. We're talking about defaults, not about corner cases after all... > I say that every tool that doesn’t honour when the user sets > TMPDIR=/var/tmp is broken (but that’s nothing new, as it has always > been like that), and that the user should set that in some corner > cases like yours. Browsers, filemanagers, flash video, cd burners... Do you basically suggest every debian user to think whether it's a corner case or not and manually select TMPDIR *every time* they start browser? ;) > You reduce the number of disc writes quite a bit, as all current > filesystems use journalling and write more additional metadata than > paging would ever use. I believe you're not talking about some small short-lived temporary files, because they never actually get to disk, kernel filesystem cache is smart enough for that (unless you've changed dirty_writeback_centisecs and dirty_expire_centisecs to some low values). So you expect to reduce number of disk writes for *large* files "quite a bit". I'm curious how much is that? I wanted to test: # time cp * /mnt/temp/ # mount tmpfs /mnt/temp -t tmpfs -o size=10G # time cp * /mnt/temp/ watching `iostat -k 1` but failed. My 2GB RAM was filled pretty quickly and the system almost died. Anybody feels the power to compare time and number of kB_wrtn by swap and extfs for the file that's at least twice as large as physical RAM? > And I’m suggesting that file managers that extract archives apply > some heuristics (size of the archive vs. size and current usage of > /tmp) to determine whether they should use /tmp or /var/tmp. So you suggest every file/archive manager developers to change their code (with a not so trivial change) and ignore FHS because debian decided to change some variable from "no" to "yes" by default? And what's this all for? What's the benefit? It does not make them faster: # mkdir /mnt/temp # mount tmpfs /mnt/temp -t tmpfs -o size=10G # cd /mnt/temp # time tar xf ~/linux-3.4.tar.bz2 real0m27.100s user0m24.327s sys 0m2.100s # cd .. ; umount temp ; cd temp # time tar xf ~/linux-3.4.tar.bz2 real0m28.072s user0m22.103s sys 0m4.767s It varies from 26 to 31 seconds between runs, but the result is obvious — no noticeable difference. User won't notice the speed, but will notice problems when system becomes heavily swapping or when some apps break because of not enough space in /tmp. > ... apply some heuristics ... Just curious, what heuristics? I've attached a small archive to this email, can you estimate its unpacked size without unpacking it? ;) >>I really can't think of any popular application that write a lot of >>small-only files and can benefit from /tmp being on tmpfs. > $GUI_BROWSER_DU_JOUR with XTaran’s unburden-home-dir, for example. `unburden-home-dir` is a bad example: (a) It's a real corner case (it's the first time I've heard about it) (b) It can store cache in any directory, not just /tmp (c) Those files may be really large, so it may also break or make your system unstable because of tmpfs. And, finally, (d) it still does not benefit from /tmp being on tmpfs. It's not making anything noticeably faster because of tmpfs, is it? PS: there's a major difference between "most programs will still work" and "most programs will become faster". You're telling that in many cases apps will still work somehow with /tmp being on tmpfs. And YOU'RE RIGHT, they will. And if they won't there're workarouns like TMPDIR. That's true. What I'm saying is that most apps will work *better* with /tmp being on real disk, system will use less RAM and be more stable, user won't have to use any workarounds and *won't lose anything*. Thanks for reading this HUGE message. -- Serge archive.bz2.gz Description: GNU Zip compressed data
Re: Moving /tmp to tmpfs makes it useful
Serge writes: > I'm asking for *popular* apps, that create files in /tmp, *never put > large files* there, and become *noticeably* faster with /tmp on tmpfs? Is that even the issue, for the most part? My impression is that using tmpfs is just logistically simpler and safer -- it can be mounted very early, doesn't muck with the (possibly small or r/o) root fs, makes it simple to limit temp space without a separate partition, etc. -miles -- 「寒いね」と話しかければ「寒いね」と答える人のいるあったかさ [俵万智]
Re: Moving /tmp to tmpfs makes it useful
On Fri, May 25, 2012 at 02:49:14PM +0100, Will Daniels wrote: > On 25/05/12 13:52, Ted Ts'o wrote: > >So what? If you write to a normal file system, it goes into the page > >cache, which is pretty much the same as writing into tmpfs. In both > >cases if you have swap configured, the data will get pushed to disk; > > That's not at all the same, the page cache is more temporary, it's > getting flushed to disk pretty quick if memory is tight (presumably) > but in the same situation using tmpfs going to swap is surely going > to be more disruptive? There will be some, but really, not that much difference between going from tmpfs to swap compared to files written to a filesystem (in both cases the data is stored in the page cache, whether it's a tmpfs file or an ext2/3/4 or xfs or btrfs file) in many cases. The major difference is that tmpfs pages only get written out to swap when the system is under memory pressure. In contrast, pages which are backed by a filesystem will start being written to disk after 30 seconds _or_ if the system is under memory pressure. So if you have a super-fast networking connection, it won't matter; the download fill the memory very quickly, at which point it will get written to swap and/or the file location on disk at the same rate, since you'll be able to download faster than you can save to disk, so the network connection will get throttled due to TCP/IP backpressure to roughly the same rate as you are writing to the HDD. If you have a slow networking connection, it's possible that the 30 second writeback timer will kick off before you start filling the memory --- but in that case, it's not going to be that disrupting in the tmpfs case, either. You'll hit memory pressure, and at that point you'll start writing to disk perhaps a bit later than the 30 second writeback timer. But at the same time, the download is coming in slowly enough that you're probably not overwhelming the speed at which you can write to the HDD or SSD. The place where it will make a difference is if you have a very large amount of memory, *and* you are downloading a really big file to /tmp (substantially bigger than your physical memory), *and* your effective end-to-end download speed is faster than your HDD write speed, but slow enough that it takes substantially longer than 30 seconds to download enough to fill your free physical memory. But that's actually a pretty narrow window. > And anyway, not everybody uses swap, in which case this "default" is > not entirely viable. I, for one, had no idea this had become default > for Debian and I think it's likely to be one of those things that > jumps out to bite people who weren't expecting it at some > inconvenient moment. Well, it's all about defaults, right? It's easy enough to set up a swap partition, or even just a swap file by default in the installer. You can set up a swap file on the fly, so it's not that hard to deal with it after the fact. > I'm sure the project veterans and more attentive readers of this > list are tired of recurring arguments like this, but usually if > something is recurring it is for a reason. Given my general "no > swap" preference, I'm glad this has come up again so that I'm aware > of it this time. > > The tmpfs setup seems far more appropriate as a performance tweak > for admins than as a default. Where there is plenty of RAM, buffer > cache makes the difference largely negligible. But where there isn't > an abundance of RAM, it could needlessly cause problems (especially > without swap). If you're worried about installations which don't have much memory (i.e., the 512mb netbook), then swap is absolutely mandatory, I would think! And if you consider how much memory most desktop/laptops have, and how often people **really** are downloading multi-gigabyte files to /tmp (note that browsers tend to default downloads to ~/Downloads), I really think the people who are agitating against tmpfs are really making a much more theoretical argument than one that seems likely to be hit by an unsophisticated user --- and a sophistcated user can easily decide whether to use /tmp on disk or not. - Ted -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20120526015655.ga5...@thunk.org
Re: Moving /tmp to tmpfs makes it useless
Thomas Goirand writes: > On 05/25/2012 05:33 PM, Mehdi Dogguy wrote: >>> What if we're installing Debian on a very small system, and that we >>> need operations with big files in /tmp? >>> >> >> Increase your swap? > > So, in this case, we will have the following scenario: > - An app writes in /tmp > - There's not enough space, so the system starts swapping, > including some apps. > - The file gets written to /tmp, then gets read > - Finally, the file gets deleted > - Then we have randomly very sloppy reaction of apps > that were swapped out so that the file could be written > in /tmp. I believe tmpfs memory is swapped out preferentially, so your scenario doesn't have to play out like that. However, paging being a complex process, it's not impossible either. Is that something people are actually seeing? Because I haven't encountered this. Best, -Nikolaus -- »Time flies like an arrow, fruit flies like a Banana.« PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6 02CF A9AD B7F8 AE4E 425C -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/87d35rkabc@vostro.rath.org
Debian documentation permalinks
Hi there. First, here's what I'm talking about - http://en.wikipedia.org/wiki/Permalink Unfortunately Wikipedia doesn't offer permalinks itself, so hopefully the above link won't "rot". This was initially in reference to the recent spat of comments/opinions to "Re: Moving /tmp to tmpfs makes it useful". It's sometimes amusing to watch as helpful enthusiasts try to unravel a problem. I nearly deleted the entire thread without giving it a moments thought - I get a lot of message list email. What I noticed by its absence was that no-one linked to official Debian policy detailing the choices made and their justification. Then it struck me that if such a document existed, it would be subject to change as Debian policy itself evolved, making any old links nonsensical or misleading. So what I'm requesting is that Debian documentation be "permalink friendly" i.e. that Debian documentation pages provide permalinks. It would be even better if they also mentioned the version(s) of Debian they apply to. This could be done using the "wayback machine" - here's an interesting example. http://wayback.archive.org/web/*/http://en.wikipedia.org/wiki/Wikipedia:Using_the_Wayback_Machine From this it's possible to specify a permalink URL even to sites without permalinks, but note that a version being referenced might be missed or duplicated, so this isn't an ideal solution. Regards, Philip Ashmore -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4fc039da.5090...@philipashmore.com
Re: Moving /tmp to tmpfs makes it useless
2012/5/25 Nikolaus Rath wrote: > I don't think /tmp as part of the rootfs is a good idea for the reasons > outlined in the rest of my mail. Forgot to answer that part. Sorry. > I think having / and /tmp share the same file system is a bad idea, > because then writing lots of stuff to /tmp would potentially fill up the > root file system (that typically also includes /var) and then cause a > lot of breakage. There's another feature, TMP_OVERFLOW_LIMIT to help just for that case, it prints a warning and mounts /tmp on tmpfs if / partition gets filled. Brilliant idea, I like it. :) > That leaves you with using tmpfs or the installer having to ensure that > there's a separate partition for /tmp. If you do the later, however, you > may just as well ensure that there is a big enough swap partition and > stick with tmpfs. There's a difference between /tmp on a separate partition and /tmp on tmpfs: /tmp on disk can't DoS the system, eat all the memory and make it swapping. And regular user must not worry about that. Imagine that you're helping your girlfriend to install debian by phone. You can tell her "Just use the default", or you'll have to tell "make sure you have enough swap space... and carefully unpack large archives or your system may become slow... and don't run programs that write large files to /tmp without configuring them... what programs? cd burners and archive managers... and don't watch youtube videos larger than 250MB... how can you find that out... well... just don't watch videos longer than 20 minutes..." etc. IMHO, that's what "default" settings are about. :) -- Serge -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/CAOVenEpTFQjFA92kELdMbj_dE_izb=yu38zu1w8fauherrf...@mail.gmail.com
Exported (ba)sh functions in the environment
Hi there. I recently had cause to search for an environment variable to see if it was being set. As a result I noticed that the environment has become a bit of a dumping ground for installed programs where configuration files would have been a cleaner option. Looking for an override in the environment is of course a good idea, but setting default values in the environment that could be 1. hard coded in the programs as defaults 2. stored in a configuration file 3. sourced from a configuration script is a bit sloppy. On my machine running "set > set.txt && ls -lsa set.txt" reveals that my environment contains 225517 of "stuff" - some of it is even being taken up by exported function definitions! That's 225517 bytes that needs to be copied every time a script runs. Regards, Philip Ashmore -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4fc044da.90...@philipashmore.com
Re: Exported (ba)sh functions in the environment
On Sat, May 26, 2012 at 03:50:02AM +0100, Philip Ashmore wrote: [...] > On my machine running "set > set.txt && ls -lsa set.txt" reveals that my > environment contains 225517 of "stuff" - some of it is even being > taken up by > exported function definitions! > > That's 225517 bytes that needs to be copied every time a script runs. No. 'env' shows the environment variables. Ben. -- Ben Hutchings We get into the habit of living before acquiring the habit of thinking. - Albert Camus -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20120526025656.gz4...@decadent.org.uk
Re: Exported (ba)sh functions in the environment
On 26/05/12 03:50, Philip Ashmore wrote: That's 225517 bytes that needs to be copied every time a script runs. Yeah that should read "every time a script or program runs." Philip -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4fc0472e.70...@philipashmore.com
Re: Exported (ba)sh functions in the environment
On 26/05/12 03:59, Philip Ashmore wrote:
> On 26/05/12 03:50, Philip Ashmore wrote:
>>
>> That's 225517 bytes that needs to be copied every time a script runs.
>
> Yeah that should read "every time a script or program runs."
>
> Philip
>
Sorry Ben, our emails collided.
According to "man sh" (which links to the dash man page)
set [{ -options | +options | -- }] arg ...
The set command performs three different functions.
With no arguments, it lists the values of all shell variables.
So are these copied every time a script runs?
Philip
--
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/4fc0495d.6070...@philipashmore.com
Re: Exported (ba)sh functions in the environment
Philip Ashmore writes:
> According to "man sh" (which links to the dash man page)
> set [{ -options | +options | -- }] arg ...
> The set command performs three different functions.
> With no arguments, it lists the values of all shell variables.
> So are these copied every time a script runs?
Only exported shell variables become part of the environment. The rest
are local to the specific shell. "env" will show just the environment.
I'm curious why even your set of shell variables is so large, though. My
environment is only 1699 bytes on a system I logged onto via ssh, and 1998
on my desktop (running Xfce). One of the biggest chunks of that is
LS_COLORS.
--
Russ Allbery (r...@debian.org)
Re: Exported (ba)sh functions in the environment
On 26/05/12 04:14, Russ Allbery wrote:
> I'm curious why even your set of shell variables is so large, though. My
> environment is only 1699 bytes on a system I logged onto via ssh, and
1998
> on my desktop (running Xfce). One of the biggest chunks of that is
> LS_COLORS.
>
Here's where I wish I was a shell script guru:
for var in `cat set.txt`; do
{ if in env discard }
done
{ sort offenders by decending size }
Here's a summary of the ones that caught my eye. Sorry if I missed
anyone out!
_xspecs
__expand_tilde_by_ref()
__gdbus()
__get_cword_at_cursor_by_ref()
__git_complete_remote_or_refspec()
( lots more git functions )
__grub_dir()
_a2dismod()
_apt_file()
_debconf_show()
_dkms()
_hg()
_inkscape()
_insserv()
_module_assistant()
_openvpn()
_pbuilder()
_stg()
_svn()
_update_initramfs()
_valgrind()
_xinetd_services()
Regards,
Philip
--
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/4fc04e32.3010...@philipashmore.com
Re: Exported (ba)sh functions in the environment
Hi, On Fri, May 25, 2012 at 08:14:28PM -0700, Russ Allbery wrote: > Philip Ashmore writes: > I'm curious why even your set of shell variables is so large, though. My > environment is only 1699 bytes on a system I logged onto via ssh, and 1998 > on my desktop (running Xfce). One of the biggest chunks of that is > LS_COLORS. It was like that for me until I installed the bash-completion package - programmable completion for the bash shell. $ set|wc 7355 19255 218488 HUGE! Osamu -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20120526033204.GA6570@localhost
Re: Exported (ba)sh functions in the environment
Philip Ashmore writes:
> Here's where I wish I was a shell script guru:
>for var in `cat set.txt`; do
> { if in env discard }
>done
>{ sort offenders by decending size }
> Here's a summary of the ones that caught my eye. Sorry if I missed
> anyone out!
Oh. This is smelling like artifacts of bash-completion; perhaps set is
dumping your completion settings. I'm pretty sure those are not copied
into subshells and are only loaded in interactive shells.
--
Russ Allbery (r...@debian.org)
Re: Exported (ba)sh functions in the environment
On 26/05/12 04:34, Russ Allbery wrote:
> Philip Ashmore writes:
>
>> Here's where I wish I was a shell script guru:
>>for var in `cat set.txt`; do
>> { if in env discard }
>>done
>>{ sort offenders by decending size }
>> Here's a summary of the ones that caught my eye. Sorry if I missed
>> anyone out!
> Oh. This is smelling like artifacts of bash-completion; perhaps set is
> dumping your completion settings. I'm pretty sure those are not copied
> into subshells and are only loaded in interactive shells.
>
In my original email I ran "set > set.txt && ls -lsa set.txt".
Putting the same into a script:
#!/bin/sh
set > set.txt && ls -lsa set.txt
...got me 2353 bytes - I'll be a donkeys ass!
It looks like the fact that I typed it made it interactive, even though it
didn't require/allow any interaction.
Thanks,
Philip
--
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/4fc051da.6060...@philipashmore.com
Re: Moving /tmp to tmpfs makes it useful
2012/5/26 Miles Bader wrote: >> I'm asking for *popular* apps, that create files in /tmp, *never put >> large files* there, and become *noticeably* faster with /tmp on tmpfs? > Is that even the issue, for the most part? My impression is that > using tmpfs is just logistically simpler and safer -- it can be > mounted very early, doesn't muck with the (possibly small or r/o) root > fs, makes it simple to limit temp space without a separate partition, > etc. We're talking about *default* settings. That what makes it an issue. It won't be an issue if most debian installations were on a small read-only root and most of them wanted /tmp to be tmpfs because of that. But it's not. I doubt major part of debians use r/o root and even if they do, not all of them would like to use tmpfs for that case. Most servers dedicate separate partitions to /tmp. I had it mount-bound to /home/tmp in that case. There're many solutions that are fast, use fewer memory, support quotas... Why tmpfs? What makes things worse: there're popular programs and popular usecases that break because of that. It may slow down or even freeze the system because of heavy swapping. So there must be a strong good reason surpassing these problems. I assume people supporting /tmp on tmpfs actually use it that way. What I don't understand is why they're arguing that "it's not that bad" instead of just saying why it's good? Does it makes something faster? What? How many seconds faster? Does it reduces disks operations? In what cases and how much? If so many people tested RAMTMP and found no good reasons for it, then maybe let's change it back to "no" by default? That's what testing is for anyway? -- Serge -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/caovenep+aeoxvtyomjksxrsdasno3j80w4byzsb0esvcant...@mail.gmail.com
Re: Moving /tmp to tmpfs makes it useful
Excerpts from Ted Ts'o's message of 2012-05-25 18:56:55 -0700: > On Fri, May 25, 2012 at 02:49:14PM +0100, Will Daniels wrote: > > On 25/05/12 13:52, Ted Ts'o wrote: > > >So what? If you write to a normal file system, it goes into the page > > >cache, which is pretty much the same as writing into tmpfs. In both > > >cases if you have swap configured, the data will get pushed to disk; > > > > That's not at all the same, the page cache is more temporary, it's > > getting flushed to disk pretty quick if memory is tight (presumably) > > but in the same situation using tmpfs going to swap is surely going > > to be more disruptive? > > There will be some, but really, not that much difference between going > from tmpfs to swap compared to files written to a filesystem (in both > cases the data is stored in the page cache, whether it's a tmpfs file > or an ext2/3/4 or xfs or btrfs file) in many cases. > > The major difference is that tmpfs pages only get written out to swap > when the system is under memory pressure. In contrast, pages which > are backed by a filesystem will start being written to disk after 30 > seconds _or_ if the system is under memory pressure. > On laptops and other power sensitive devices, this is pretty critical. Hypothetical: I have 2GB of RAM, and I want to watch a 50MB video file on a connection that will take, say, 10 minutes to cache the whole thing (and its a 10 minute video). With a regular filesystem hosting /tmp, Every 30 seconds I will wake up the hard disk, and write data to it. I doubt most spinning disks will go to sleep in < 30 seconds, so this is more than 10 minutes solid of hard disk spinning. With tmpfs, there is no memory pressure, so my disk never even spins up to write anything to it. If I do run into memory pressure, yes, I need to use swap at that point. But at that point I've got a lot more than just the disk draining power. -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/1338010047-sup-1...@fewbar.com
Re: Moving /tmp to tmpfs makes it useless
On 05/26/2012 09:20 AM, Nikolaus Rath wrote: > I believe tmpfs memory is swapped out preferentially, so your scenario > doesn't have to play out like that. However, paging being a complex > process, it's not impossible either. I haven't read the actual kernel code, just making assumption from what I generally know about swapping algorithms (eg: a long long time ago, I worked on something like that on Atari computers...). What's going to go to the swap space are pages that have a lower rank. There are many ways to rank a page. But these pages are generally what hasn't been used often, and hasn't been used for a while. So let's say you start an app, and don't use at all some of its functions, it gets swapped out (if needed) until you start using it. This is where one can complain about the randomness of swap access: it's not really predictable from the user's point of view, while having a file written to /tmp is. If you access often files in your tmpfs, of course, they wont go to the swap, they will have a higher rank, and the kernel will decide to swap some pages that are less often accessed. And that's when it becomes very annoying, whatever is the swap algorithm: things go in and out of the swap space depending on how often you use them, when all your app cared was writing a (potentially) big file to disk. Here, we don't care if things are going faster (benches are useless), we care that it "feels" right, and we care our system is reactive to what the user asks for. Now, what happens if your /tmp gets full? Hum... generally, not much. The application fails, and if written correctly, displays an error message. If written badly, it simply crashes. In all cases, that's not a big deal. But if your swap gets full (or used really a lot), then it's a disaster. Your system becomes unresponsive to users commands, and sometimes even crashes (eg: the OOM killer starts acting funnily and kills random processes...). No way anyone wants this on his system. Even less a user with not a lot of experience, who just happen to choose the installer's default, and that wont understand what's happening. Thomas -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4fc0716b.4040...@debian.org
Re: Debian documentation permalinks
On 05/25/2012 10:03 PM, Philip Ashmore wrote: > Hi there. > > First, here's what I'm talking about - > http://en.wikipedia.org/wiki/Permalink > Unfortunately Wikipedia doesn't offer permalinks itself, so > hopefully the above link won't "rot". And here's the permalink to the above article, as it was when the preceding post was made: http://en.wikipedia.org/w/index.php?title=Permalink&oldid=483438630 Or, if you prefer links that don't indicate where they're really going: http://en.wikipedia.org/w/index.php?oldid=483438630 -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4fc06fcd.3010...@gmail.com
