Pros and Cons of Using Deduplication of Solr at Huge Data Indexing

[Furkan KAMACI]

I use Solr 4.2.1 as SolrCloud. I crawl huge data with Nutch and index them
with SolrCloud. I wonder about Solr's deduplication mechanism. What exactly
it does and does it results with a slow indexing or is it beneficial for my
situation?