[PATCH] D152416: add -dump-tokens-bg option for clang

Anarion-zuo added a comment. I see that plugin is a more elegant way of doing this. Do you still want this as a part of LLVM? Repository: rG LLVM Github Monorepo CHANGES SINCE LAST ACTION https://reviews.llvm.org/D152416/new/ https://reviews.llvm.org/D152416 _

[PATCH] D152416: add -dump-tokens-bg option for clang

Anarion-zuo added a comment. We intend to build a deep learning model with code as input, then noticed clang does not currently have a convenient way of dumping tokens in huge projects. The dumped tokens are supposed to be converted to vectors, then given to a deep learning model. Repository:

[PATCH] D152416: add -dump-tokens-bg option for clang

Anarion-zuo added a comment. It's my first time here. I don't know if I got everything right. CodeOwners.rst says you guys are relevant somewhat. Thanks! @ChuanqiXu @jansvoboda11 Repository: rG LLVM Github Monorepo CHANGES SINCE LAST ACTION https://reviews.llvm.org/D152416/new/ https://r

[PATCH] D152416: add -dump-tokens-bg option for clang

Anarion-zuo created this revision. Herald added a project: All. Anarion-zuo requested review of this revision. Herald added a project: clang. Herald added a subscriber: cfe-commits. Repository: rG LLVM Github Monorepo https://reviews.llvm.org/D152416 Files: clang/include/clang/Driver/Options