Hi, I have a Tokenizer that ensures emails are kept whole, but I then run the token stream through a set of filters that inevitably breaks up the email back into its components. Some of these filters are proprietary, and I can't modify them.
It seems hacky to include an EmailRecoveryFilter at the end of my stream to take component tokens and reconstruct an email: and frankly, that could result in incorrect reconstruction in some cases. Ideally, I'd have a way to mark a Token as KEEP_WHOLE such that downstream TokenFilter's simply ignore it. Is there a precedent for this, or do I need to implement that myself? And if so, would you suggest I use the Token type, the Payload, or use a wrapper around Token itself to store the flag? Thanks! Tavi
