i have a lot of data coming/pouring in from this:
my ($rout, $pid) = $ssh->pipe_out($cmd);
while (my $line = <$rout>) {
print filehandle $line;
}
I want to stop writing after certain size is written (say 1gb).
so i can try this: (it is working). But I am worried I am doing too many stat
(and i am imagining it would be very expensive operation). there are about
500,000 lines coming in for 500mb data file, so below is doing stat 500
thousand times!!!
my ($rout, $pid) = $ssh->pipe_out($cmd);
while (my $line = <$rout>) {
my
($dev,$ino,$mode,$nlink,$uid,$gid,$rdev,$size,$atime,$mtime,$ctime,$blksize,$blocks)=
stat(filehandle);
if ($size < $huge_limit ){
print $tmp_fh $line;
} else {
seek($tmp_fh, 0, 0);
print $tmp_fh "file size limit reached\n";
kill TERM => $pid;
last;
}
} ###end while
so i am 'thinking' to try this: storing output in local temp array and checking
when it reaches 10mb ($huge_limit) then save it to file and increase counter,
when it happens total of $huge_limit_counter times (50 for 500mb limit). I
close the channel and proceed. Here I am thinking, checking size of array
500,000 times (a 500k line array) is somehow 'more efficient' then checking
file-size using stat.
my $counter=1;my @tmp_arr;
my ($rout, $pid) = $ssh->pipe_out($cmd);
while (my $line = <$rout>) {
push(@tmp_arr,$line);
if (size(\@tmp_arr) > $huge_limit){
$counter++;
if ($counter >= $huge_limit_counter){
kill TERM => $pid;
last;
} else {
print $tmp_fh @tmp_arr;
undef(@tmp_arr);
}
}
}
if(tell($rout) != -1){close $rout;}
Please advice any suggestion for increasing efficiency of this code.
thank you.
Rajeev
