ID: 48507 Comment by: pahan at hubbitus dot spb dot su Reported By: krynble at yahoo dot com dot br Status: Verified Bug Type: Filesystem function related Operating System: Unix PHP Version: 5.* New Comment:
Sorry for duplicate (#50456 is my), but in it, additionally to there described problem in fgetcsv I also suggest fix fputcvs to allow [force] enclosing single words in field. Off course it does *not* solve this problem of incorrect fgetcsv parsing, because RFC allow not quoted values ( http://www.faqs.org/rfcs/rfc4180.html , section 2.5 ), but, it is make pair fputcsv/fgetcsv as minimum compatible in PHP implementation. Previous Comments: ------------------------------------------------------------------------ [2009-12-12 01:33:51] j...@php.net See also bug #50456 ------------------------------------------------------------------------ [2009-09-22 15:09:20] phofstetter at sensational dot ch below you'll find a small script which shows how to implement a user filter that can be used to on-the-fly utf8-encode the data so that fgetcsv is happy and returns correct output even if the first character in a field has its high-bit set and is not valid utf-8: Remember: This is a workaround and impacts performance. This is not a valid fix for the bug. I didn't yet have time to deeply look into the C implementation for fgetcsv, but all these calls to php_mblen() feel suspicious to me. I'll try and have a look into this later today, but for now, I'm just glad I have this workaround (quickly hacked together - keep that in mind): <?php class utf8encode_filter extends php_user_filter { function is_utf8($string){ return preg_match('%(?: [\xC2-\xDF][\x80-\xBF] # non-overlong 2-byte |\xE0[\xA0-\xBF][\x80-\xBF] # excluding overlongs |[\xE1-\xEC\xEE\xEF][\x80-\xBF]{2} # straight 3-byte |\xED[\x80-\x9F][\x80-\xBF] # excluding surrogates |\xF0[\x90-\xBF][\x80-\xBF]{2} # planes 1-3 |[\xF1-\xF3][\x80-\xBF]{3} # planes 4-15 |\xF4[\x80-\x8F][\x80-\xBF]{2} # plane 16 )+%xs', $string); } function filter($in, $out, &$consumed, $closing) { while ($bucket = stream_bucket_make_writeable($in)) { if (!$this->is_utf8($bucket->data)) $bucket->data = utf8_encode($bucket->data); $consumed += $bucket->datalen; stream_bucket_append($out, $bucket); } return PSFS_PASS_ON; } } /* Register our filter with PHP */ stream_filter_register("utf8encode", "utf8encode_filter") or die("Failed to register filter"); $fp = fopen($_SERVER['argv'][1], "r"); /* Attach the registered filter to the stream just opened */ stream_filter_prepend($fp, "utf8encode"); while($data = fgetcsv($fp, 0, ';', '"')) print_r($data); fclose($fp); ------------------------------------------------------------------------ [2009-09-22 14:45:22] phofstetter at sensational dot ch I was looking into this (after having been bitten by it) and I can add another tidbit that might help tracking this down: The bug doesn't happen if the file fgetcsv() is reading is in UTF-8-format. I have created a test-file in ISO-8859-1 and then used file_put_contents(utf8encode(file_get_contents())) to create the UTF8-version of it (explaining this here because I'm not sure whether this would write a BOM or not - probably not though). That version could be read correctly. I'm now writing a stream filter that does the UTF-8 conversion on the fly to hook that in between the file and fgetcsv() - while I would lose a bit of performance, in my case, this is the cleanest workaround. ------------------------------------------------------------------------ [2009-09-21 18:11:47] dmulryan at calendarwiz dot com Note: Previous comment has error where URL is shown in array element. This is not a bug but my error in the example. Bug is in special characters. ------------------------------------------------------------------------ [2009-09-21 18:07:42] dmulryan at calendarwiz dot com Similar problem when parsing the following line: 0909211132,1,ØÊááàÑ,äÆæç,CForm,Y,1,1,1,97.95.176.240,2530 which produces empty array elements for fields with special characters: Array ( [0] => 0909211132 [1] => 1 [2] => [3] => [4] => URL [5] => Y [6] => 1 [7] => 1 [8] => 1 [9] => 97.95.176.240 [10] => 2530 ) ------------------------------------------------------------------------ The remainder of the comments for this report are too long. To view the rest of the comments, please view the bug report online at http://bugs.php.net/48507 -- Edit this bug report at http://bugs.php.net/?id=48507&edit=1
