How to remove duplicate tokens from solr

[Rajdeep Sahoo]

Hi team,
 Is there any way to remove duplicate tokens from solr. Is there any filter
for this.