[Python-Dev] Re: compiled python3.10 is unable to find _ssl
Solved! The astute reader will notice that while I built the latest version of python3.10 2022-05-24 11:38, the one I was testing was built May 15 2022, 12:44:05. This was left over from an earlier iteration, and not cleaned up with a subsequent make clean that was run after another ./configure with a different --prefix value. ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/HQUMR3RSUWXC5FVQTTVCOW6GV7DLY3UK/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] spling1...@gmail.com
Sent from Mail for Windows ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/HOH2QERFVDIIEN26L2HGOU2O7LITLHOT/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Is it possible to view tokenizer output?
Hi, I'm just getting into the CPython codebase just for fun, and I've just started messing around with the tokenizer and the grammar. I was wondering, is there a way to just print out the results of the tokenizer (as in just the stream of tokens it generates) in a human readable format? It would be really helpful for debugging. Hope the question's not too basic. Cheers. ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/2ZTZBAN5H2ET2IB7EXTKD27R5T6QVHZB/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Is it possible to view tokenizer output?
Le 30/05/2022 à 00:59, Jack a écrit : Hi, I'm just getting into the CPython codebase just for fun, and I've just started messing around with the tokenizer and the grammar. I was wondering, is there a way to just print out the results of the tokenizer (as in just the stream of tokens it generates) in a human readable format? It would be really helpful for debugging. Hope the question's not too basic. python -m tokenize file.py ? See https://docs.python.org/3/library/tokenize.html#command-line-usage Cheers, Jean ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/AKIHN3EVNBRJCOLR4ABXV7OADYKXKKUU/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Is it possible to view tokenizer output?
Thanks! I didn't even know about that module. Does this take into account your local changes to the tokenizer, though? I've added a new token type to Grammar/Tokens, and some code to tokenizer.c to return that token type in appropriate circumstances. I've stepped through the tokenizer in the debugger, so I /think /it's working. When I run -m tokenize as you suggest, I don't see my custom token type. The devguide mentions that "|Lib/tokenize.py| needs changes to match changes to the tokenizer.", so I'm guessing I would have to manually repeat my changes in tokenize.py to see them, right? But what I want to see is what tokenizer.c is producing when my newly built Python binary actually reads a file. On 30/05/2022 00:09, Jean Abou Samra wrote: Le 30/05/2022 à 00:59, Jack a écrit : Hi, I'm just getting into the CPython codebase just for fun, and I've just started messing around with the tokenizer and the grammar. I was wondering, is there a way to just print out the results of the tokenizer (as in just the stream of tokens it generates) in a human readable format? It would be really helpful for debugging. Hope the question's not too basic. python -m tokenize file.py ? See https://docs.python.org/3/library/tokenize.html#command-line-usage Cheers, Jean ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/ZCG44GZJH4FWPC66TI7GNLYZTHGLQ6NM/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Is it possible to view tokenizer output?
python -m tokenize < file-to-parse.py See the comment at the top of tokenize.py. IIRC, it re-implements the tokenizer, it does not call the one used for python code. Eric On 5/29/2022 6:59 PM, Jack wrote: Hi, I'm just getting into the CPython codebase just for fun, and I've just started messing around with the tokenizer and the grammar. I was wondering, is there a way to just print out the results of the tokenizer (as in just the stream of tokens it generates) in a human readable format? It would be really helpful for debugging. Hope the question's not too basic. Cheers. ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/2ZTZBAN5H2ET2IB7EXTKD27R5T6QVHZB/ Code of Conduct: http://python.org/psf/codeofconduct/ ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/EZ3E7VMT6WAT23EWWLV3O4CXWXYUGJWB/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Is it possible to view tokenizer output?
Well, I just stuck a print statement in _PyTokenizer_Get() and it's done the job for me, right now. Thanks, Jack On 30/05/2022 00:36, Eric V. Smith wrote: python -m tokenize < file-to-parse.py See the comment at the top of tokenize.py. IIRC, it re-implements the tokenizer, it does not call the one used for python code. Eric On 5/29/2022 6:59 PM, Jack wrote: Hi, I'm just getting into the CPython codebase just for fun, and I've just started messing around with the tokenizer and the grammar. I was wondering, is there a way to just print out the results of the tokenizer (as in just the stream of tokens it generates) in a human readable format? It would be really helpful for debugging. Hope the question's not too basic. Cheers. ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/2ZTZBAN5H2ET2IB7EXTKD27R5T6QVHZB/ Code of Conduct: http://python.org/psf/codeofconduct/ ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/PLOALW2GXI3MHWEEA6L2KDLGHZDP2NC6/ Code of Conduct: http://python.org/psf/codeofconduct/
