[PHP] File Write Operation Slows to a Crawl....
Hi: Newbie here. This is my first attempt at PHP scripting. I'm trying to find an alternative to Lotus Domino's domlog.nsf for logging web transactions. Domino does create an Apache compatible text file of the web transactions, and this is what Im trying to parse. I started off using a code snibbet I found on the web. I modified it a little bit to suit my needs. It was working fine with the small 600k test log file I was using, but since Ive moved to the larger 18Mb production log file heres what happens: Ive modified the code and added an echo statement to echo each loop that gets processed. Initially it starts off very fast but then performance becomes very slow, to a point where I can count each loop as its being processed. Its taking a little over 3 hours to parse the entire file. I figured it was a disk cache thing, so I created a ram drive. This has improved the performance, but is still taking an hour to parse. Here is the PHP script Im using: This is running on a Toshiba Tecra A4 Laptop with FreeBSD 7.0 Release. Plenty of RAM and HDD space. The PHP Version is: PHP 5.2.5 with Suhosin-Patch 0.9.6.2 (cli) (built: Feb 11 2009 09:28:47) Copyright (c) 1997-2007 The PHP Group Zend Engine v2.2.0, Copyright (c) 1998-2007 Zend Technologies What should I do to get this script to run faster? Any help is appreciated . Regards, Fred Schnittke Powered by Execulink Webmail http://www.execulink.com/ -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] File Write Operation Slows to a Crawl....
Thanks Paul and Shawn:
I can't answer the Why's in your posts, as this is literally my first
attempt at PHP, but I will investigate your response and refine my code
accordingly.
What I did find is that by replacing the following code:
$fhandle = fopen("/ramdrive/import_file.txt", "w");
foreach($new_format as $data) {
fputs($fhandle, "$data");
}
fclose($fhandle);
with this code:
$fp = fopen('data.txt', 'a');
fwrite($fp, $new_format[$each_rec]);
fclose($fp);
The parsing finished literally, within seconds. I don't have to worry
about anyone else opening this file, so I think this should work ok.
Thanks for your help
Fred Schnittke
Powered by Execulink Webmail
http://www.execulink.com/
--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] File Write Operation Slows to a Crawl....
Hi Shawn:
Yes, good idea, I agree
Thanks
Fred
> fschnit...@execulink.com wrote:
>> Thanks Paul and Shawn:
>>
>> I can't answer the Why's in your posts, as this is literally my first
>> attempt at PHP, but I will investigate your response and refine my code
>> accordingly.
>>
>> What I did find is that by replacing the following code:
>>
>> $fhandle = fopen("/ramdrive/import_file.txt", "w");
>> foreach($new_format as $data) {
>> fputs($fhandle, "$data");
>> }
>> fclose($fhandle);
>>
>>
>> with this code:
>>
>> $fp = fopen('data.txt', 'a');
>> fwrite($fp, $new_format[$each_rec]);
>> fclose($fp);
>>
>> The parsing finished literally, within seconds. I don't have to worry
>> about anyone else opening this file, so I think this should work ok.
>>
>> Thanks for your help
>>
>>
>> Fred Schnittke
>>
>>
>>
>> Powered by Execulink Webmail
>> http://www.execulink.com/
>>
> Good job! I posted a minute ago with 2 options, but for your fix I
> would recommend the following:
>
> // move this before your while()
> $fp = fopen('data.txt', 'w'); //notice the 'w' now
>
> // move this after the } that ends your while
> fclose($fp);
>
> Should save even more time because you're not opening and closing the
> file each time through the while().
>
> --
> Thanks!
> -Shawn
> http://www.spidean.com
>
> --
> PHP General Mailing List (http://www.php.net/)
> To unsubscribe, visit: http://www.php.net/unsub.php
>
>
Powered by Execulink Webmail
http://www.execulink.com/
--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php
