On Wed, Apr 28, 2021 at 11:45:59AM -0400, Stefan Monnier wrote: > > Here's a sort of example I just ran into [...]
> FWIW, I'm pretty sure that such anectodal evidence is of no importance > because you can also come up with examples where the situation > is reversed. Still it might tell us something by "backscatter" :) For example, I forced now DDG's hand by doing some "foo site:blah.blah". Tomorrow I'll check again and see whether that site is in the index. > This is simply because the subset of the internet that is indexed by the > two search engines is not simply in a subset relation. So the question > is not whether such things happen, but how often they happen for your > use-case one way compared to how it happens for your use-case the > other way. Yes, of course. Nobody can index the whole Internet these days ;-) > BTW, as far as I know, DDB doesn't do its own indexing but it relies > internally on Bing, so in the above is explained by the difference > between Bing and Google. Technically they could probably just as > well rely on Google (or on both). They do have an own crawler [1]. No idea of how much it contributes to their data set, though. Cheers [1] https://help.duckduckgo.com/duckduckgo-help-pages/results/duckduckbot/ - t
signature.asc
Description: Digital signature
