From: Operating system: Any PHP version: 5.3.5 Package: Unknown/Other Function Bug Type: Bug Bug description:token_get_all with regards to __halt_compiler is not binary safe
Description: ------------ A. token_get_all() eats some characters which are not allowed in plain PHP code and trigger a "Unexpected character in input" warning. B. after a T_HALT_COMPILER, the tokens are still identified as if they were not after this T_HALT_COMPILER, when in reality they are just random data. So, when using token_get_all on code which contains a T_HALT_COMPILER, the data after that is corrupted because of A. Test script: --------------- <?php $code = "<?php __halt_compiler();\x01?>\x02"; $tokens = token_get_all($code); $reconstructed_code = ''; foreach ($tokens as $t) { $reconstructed_code .= isset($t[1]) ? $t[1] : $t; } var_dump($code); var_dump($reconstructed_code); Expected result: ---------------- string(28) "<?php __halt_compiler();?>" string(28) "<?php __halt_compiler();?>" Actual result: -------------- PHP Warning: Unexpected character in input: '' (ASCII=1) state=0 on line 5 string(28) "<?php __halt_compiler();?>" string(27) "<?php __halt_compiler();?>" -- Edit bug report at http://bugs.php.net/bug.php?id=54089&edit=1 -- Try a snapshot (PHP 5.2): http://bugs.php.net/fix.php?id=54089&r=trysnapshot52 Try a snapshot (PHP 5.3): http://bugs.php.net/fix.php?id=54089&r=trysnapshot53 Try a snapshot (trunk): http://bugs.php.net/fix.php?id=54089&r=trysnapshottrunk Fixed in SVN: http://bugs.php.net/fix.php?id=54089&r=fixed Fixed in SVN and need be documented: http://bugs.php.net/fix.php?id=54089&r=needdocs Fixed in release: http://bugs.php.net/fix.php?id=54089&r=alreadyfixed Need backtrace: http://bugs.php.net/fix.php?id=54089&r=needtrace Need Reproduce Script: http://bugs.php.net/fix.php?id=54089&r=needscript Try newer version: http://bugs.php.net/fix.php?id=54089&r=oldversion Not developer issue: http://bugs.php.net/fix.php?id=54089&r=support Expected behavior: http://bugs.php.net/fix.php?id=54089&r=notwrong Not enough info: http://bugs.php.net/fix.php?id=54089&r=notenoughinfo Submitted twice: http://bugs.php.net/fix.php?id=54089&r=submittedtwice register_globals: http://bugs.php.net/fix.php?id=54089&r=globals PHP 4 support discontinued: http://bugs.php.net/fix.php?id=54089&r=php4 Daylight Savings: http://bugs.php.net/fix.php?id=54089&r=dst IIS Stability: http://bugs.php.net/fix.php?id=54089&r=isapi Install GNU Sed: http://bugs.php.net/fix.php?id=54089&r=gnused Floating point limitations: http://bugs.php.net/fix.php?id=54089&r=float No Zend Extensions: http://bugs.php.net/fix.php?id=54089&r=nozend MySQL Configuration Error: http://bugs.php.net/fix.php?id=54089&r=mysqlcfg