Todd, The tokenizer was a component that already existed in ParseKit. So when I > began work on the new PEG-based static packrat generator a few weeks ago, I > took a shortcut. I used the existing tokenizer for token recognition. I may > unify parser/tokenizer recognition in the future for the PEG-based > generator (like ANTLR). But either way, the existing tokenizer will remain > as a useful stand-alone component for other applications.
ANTLR is not PEG! It is ALLSTAR (aLL[*]). The reason I migrated away from ANTLR is because of the tokenizer, which cannot be parser-driven (not without a huge amount of "glue" code). Quite unlike PEG. My way out of it was this: https://bitbucket.org/apalala/grako I can explain further, if you care. Cheers, -- Juancarlo *Añez*
_______________________________________________ PEG mailing list [email protected] https://lists.csail.mit.edu/mailman/listinfo/peg
