Hi Damien
(apologies for top-posting, handicapped mail client).
Actually, "/" (by /tmp) is filling up, and clearing very rapidly due to temp
files being created and removed at high speed. We ca only see this
by doing:
---
#!/usr/bin/perl
while(1){
$timestamp = localtime();
system("echo $timestamp `df -h /tmp` >> /home/traianow/dfstats.txt");
system("echo $timestamp `du -sh /tmp` >> /home/traianow/dfstats.txt");
sleep 1;
}
---
We're seeing this fast-changing disk space usage patterns like this, repeating
every few tens of seconds:
----
Mon Jun 20 11:41:54 2011 844M /tmp
Mon Jun 20 11:41:55 2011 Filesystem Size Used Avail Capacity Mounted on
/dev/amrd0s1a 989M 987M -76M 108% /
Mon Jun 20 11:41:55 2011 849M /tmp
Mon Jun 20 11:41:56 2011 Filesystem Size Used Avail Capacity Mounted on
/dev/amrd0s1a 989M 987M -76M 108% /
Mon Jun 20 11:41:56 2011 849M /tmp
Mon Jun 20 11:41:57 2011 Filesystem Size Used Avail Capacity Mounted on
/dev/amrd0s1a 989M 987M -76M 108% /
Mon Jun 20 11:41:57 2011 849M /tmp
Mon Jun 20 11:41:58 2011 Filesystem Size Used Avail Capacity Mounted on
/dev/amrd0s1a 989M 987M -76M 108% /
Mon Jun 20 11:41:58 2011 849M /tmp
Mon Jun 20 11:42:01 2011 Filesystem Size Used Avail Capacity Mounted on
/dev/amrd0s1a 989M 987M -76M 108% /
Mon Jun 20 11:42:01 2011 849M /tmp
Mon Jun 20 11:42:02 2011 Filesystem Size Used Avail Capacity Mounted on
/dev/amrd0s1a 989M 141M 769M 15% /
Mon Jun 20 11:42:02 2011 3.2M /tmp
Mon Jun 20 11:42:03 2011 Filesystem Size Used Avail Capacity Mounted on
/dev/amrd0s1a 989M 142M 768M 16% /
Mon Jun 20 11:42:03 2011 4.8M /tmp
Mon Jun 20 11:42:04 2011 Filesystem Size Used Avail Capacity Mounted on
/dev/amrd0s1a 989M 145M 765M 16% /
Mon Jun 20 11:42:04 2011 7.7M /tmp
Mon Jun 20 11:42:06 2011 Filesystem Size Used Avail Capacity Mounted on
/dev/amrd0s1a 989M 148M 762M 16% /
Mon Jun 20 11:42:06 2011 10M /tmp
Mon Jun 20 11:42:07 2011 Filesystem Size Used Avail Capacity Mounted on
/dev/amrd0s1a 989M 150M 760M 16% /
----
What I'm trying to determine is what caused the change in temp file writing
behaviour on the server, and if this is the kind behaviour likely on a heavily
loaded box with cpu running at 100% (which this system is). i.e, do processes
like cvs that write tmp files suddenly start writing more temp files when
starved for cpu, leading to this kind of behaviour?
Thanks,
Traiano
________________________________________
From: [email protected] [[email protected]]
on behalf of Damien Fleuriot [[email protected]]
Sent: Monday, June 20, 2011 12:01 PM
To: [email protected]
Subject: Re: (no subject)
On 6/20/11 10:13 AM, Traiano Welcome wrote:
> Hi List
>
> We have a FreeBSD 6.2-STABLE #0 server running as a general unix shell
> server. Recently the system has been running at high load (average 8, and cpu
> 100%), and even more recently we've started seeing the following types of
> error when we do cvs commits on the system. The system has between 150 to 200
> users on it during the day.
>
> ---
> "/: write failed, filesystem is full"
> Error: /tmp/file.commit.72971.tmp: No space left on device;
> /tmp/file.commit.72971.tmp: WARNING: FILE TRUNCATED
> ---
>
> The disks are definitely not full (this shows up in df -hi), both in terms of
> storage space and inode utilisation. However the cpu utilisation is
> permanently at 100%, and we're aware of which processes are causing the
> utilisation. My question is: Is it possible, under some circumstances that
> cpu starvation could result in the type of "filesystem is full" errors we're
> seeing above?
>
> Thanks in Advance,
> Traiano Welcome
>
> _______________________________________________
> [email protected] mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-questions
> To unsubscribe, send any mail to "[email protected]"
Are you really sure your file system is not full ?
1/ sync
2/ df -h
3/ df -i
_______________________________________________
[email protected] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "[email protected]"
_______________________________________________
[email protected] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "[email protected]"
