Re: [Development] Search engine indexing of qt-project.org

2013-06-12 Thread Jonas Gastal
On Tue, Jun 11, 2013 at 1:30 PM, Jonas Gastal wrote: > Interestingly, doing wget I get the same as Giuseppe, using the > browser(chromium) I get the same as Robin. > > On further analysis the reason I was getting the disallow everything robots.txt in chromium was because of https everywhere. I don

Re: [Development] Search engine indexing of qt-project.org

2013-06-11 Thread Jonas Gastal
Interestingly, doing wget I get the same as Giuseppe, using the browser(chromium) I get the same as Robin. On Tue, Jun 11, 2013 at 12:30 PM, Giuseppe D'Angelo wrote: > CCing w...@qt-project.org... > > On 11 June 2013 17:13, Robin Burchell wrote: > > Hi, > > > > I've noticed that the quality of