On 10th and 11th January 2023, I accessed the top 10,000 websites accessed from the UK[1] <#_ftn1> using a domestic internet connection and the Chrome web browser. I recorded the following information from the response.
1. The presence of programmatic advertising. 2. Enablement of User Agent Client Hints (UA-CH) via the presence of the Accept-CH HTTP header. Of the 10,000 websites accessed 4,392 utilise advertising of some form. Of these 46 have enabled UA-CH. ~1% of the top 4,392 advertising funded websites accessed from the UK are ready for User Agent Reduction as at 10th/11th January 2023. >From 1st February Demand Side Platforms (DSPs) that compete with Google will no longer receive the information they need to identify the model of device and inform their fraud detection algorithms from the top 99% of the websites that incorporate advertising of some kind and are accessed from the UK. The spend from advertisers that would have previously been directed towards these websites and DSPs will likely be directed elsewhere including to Google’s competing advertising services. Thus the User Agent Reduction change will directly benefit Google financially to the detriment of competitors until these websites have implemented the necessary replacement technology - User Agent Client Hints. More work is needed to inform websites that incorporate advertising about the User Agent Reduction change prior to making further User Agent Reduction changes. [1] <#_ftnref1> A list provided by SEMRush on 10th January was used to establish the top 10k websites visited from the UK. https://www.semrush.com/analytics/ranks/rank/?db=uk <https://www.semrush.com/analytics/ranks/rank/?db=uk-> On Friday, 3 June 2022 at 22:12:23 UTC+1 James Rosewell wrote: > Dear Google, > > > > Adding Matthew Hancox and David Verroken in their role as Monitoring > Trustee > <https://www.gov.uk/cma-cases/investigation-into-googles-privacy-sandbox-browser-changes#monitoring-trustee-report> > > of Google’s commitments with the CMA. > > > > Google make the following statements in their May 2022 report > <https://assets.publishing.service.gov.uk/media/62835bfee90e071f6af1457e/Privacy_Sandbox_Progress_Report_to_the_CMA_2022_Q1.pdf> > > in relation to this Intent to Deprecate and Freeze notice. > > > > *API / Technology * > > * Feedback Theme * > > * (Ranked by * > > * Prevalence) * > > * Questions and * > > * Concerns * > > * Summary * > > * Chrome Response * > > *User Agent Reduction * > > *Performance * > > *There are concerns about the latency of getting hints via Critical-CH (on > the first page load). * > > *Chrome is investigating ways to improve performance. * > > *User-Agent * > > *Reduction / * > > *User-Agent Client * > > *Hints * > > *Anti-Fraud / Anti-Abuse concerns * > > *Having as much information as possible is * > > *important when debugging certain types of attacks, including Denial of > Service. Losing some info from the UA string may pose challenges. * > > *Chrome is in discussions and evaluating ways to maintain privacy while > providing sufficient information that will be useful for debugging. * > > *User Agent Reduction * > > *Confusion around OT setup * > > *Multiple Origin Trial participants recommended improving documentation > with examples of how to enroll in the Origin Trial. * > > *The Reduced UA Origin Trial is ending, but Chrome intends to improve the > **instructions > for the* > <https://developer.chrome.com/blog/user-agent-reduction-deprecation-trial/> > <https://developer.chrome.com/blog/user-agent-reduction-deprecation-trial/> > > *Deprecation Trial* > <https://developer.chrome.com/blog/user-agent-reduction-deprecation-trial/> * > (including > making the example demo more prominent). * > > *User Agent Reduction * > > *Concern about values of specific hint * > > *Questions around * > > *if the * > > *Sec-CH-UA-Model * > > *is the same as < deviceModel> in the User-Agent string. * > > *Sec-CH-UA-Model is the same as <deviceModel> in the User-Agent string. > Chrome will try to make this more clear in future documentation. * > > *User-Agent Reduction * > > *Concern about * > > *enrolling in Deprecation * > > *Trial * > > *Questions around h ow to enroll a large number of domains into the * > > *Deprecation Trial. * > > *Chrome has considered centralized approaches when designing the > Deprecation Trial, but Chrome believes the existing Origin Trial is the > best option as it gives all control to developers ( since they can choose > to s end the header or not). * > > *User-Agent Client * > > *Hints * > > *Concerns around * > > *prescriptive nature of * > > *UA-CH * > > *There is a concern that UA-CH is * > > *overly prescriptive when compared to the flexibility the * > > *User-Agent header offers, as de ned by rfc7231. * > > *Chrome sees the prescriptive nature of UA-CH headers as an * > > *important improvement * > > *over the flexibility of the UA string, both from the point of view of > eventual cross-browser interoperability and user privacy protection (by > preventing arbitrary additions of high-entropy identifiers). * > > *However the issue remains open in case others also share this concern and > would like to provide feedback.* > > *User-Agent Client * > > *Hints * > > *Concerns that the API is being used to block certain browsers * > > *Concern that a site is using the API to look for “Google Chrome” or > “Microsoft Edge” and blocking all other browsers. * > > *The concept of a brand list was designed to handle this case - a browser > can send “Google Chrome” in addition to their own brands. * > > *User-Agent Client * > > *Hints * > > *Request for a method * > > *to enumerate all supported hints * > > *Interest in having a * > > *programmatic way to know all supported hints for a browser. * > > *Chrome is evaluating the feature request. * > > *User-Agent * > > *Reduction / * > > *User-Agent Client * > > *Hints * > > *Anti-Fraud / Anti-Abuse concerns * > > *Client hints are not available on first load for HTTP1 * > > *One of the Client Hints * > > *Reliability APIs * > > *(ACCEPT_CH) is only available over HTTP2 and H TTP3. For servers who are > still served over HTTP1, they will need to rely solely on Critical-CH. * > > *User-Agent Reduction * > > *Impact on Chrome for Android * > > *Questions on how this impacts Chrome on Android in particular. * > > *UA Reduction as well as UA-CH will ship on Chrome on Android, in addition > to Desktop. For Chrome on Android, the changes will only take place in > “Phase 6”, currently scheduled for Chrome 110. * > > *Gnatcatcher + * > > *User-Agent * > > *Reduction * > > *Reducing signals for anti-fraud * > > *Anti-fraud impact of concurrently reducing IP and U A access. * > > *Expecting Willful IP * > > *Blindness anti-fraud policy stipulations (to allow use of I P for > anti-fraud use cases) will resolve defensibility concerns around IP > proxying. * > > > > *Latency* > > > > Google acknowledged Privacy Sandbox is part of their “Ad Systems” in the > commitments to the CMA. User Agent Reduction and User Agent Client Hints > are part of Privacy Sandbox. > > > > Any delay in retrieving the UACH values will delay the population of the Open > RTB Structured User Agent > <https://iabtechlab.com/wp-content/uploads/2022/04/OpenRTB-2-6_FINAL.pdf> > (SUA) data thus delaying the request for advertising. Those websites that > are bundled with the web browser via defaults, or are well-known and > visited frequently, are likely to benefit over those that are visited for > the first time. Therefore the latency issue is material to the impact on > advertising and the economics of the Open Web. > > > > *Complexity* > > > > Any increase in complexity associated with obtaining information (for > example the permissions policy) will be easier for companies with more > engineers to implement than companies with fewer engineers. The web became > the web in part due to simplicity over alternatives. As companies with more > engineers tend to be larger, complexity benefits the largest companies in > markets, and becomes a “tax” for smaller companies. > > > > *Other Analysis of Report* > > > > Movement for an Open Web (MOW) provide further analysis of the full first > quarter report here > <https://movementforanopenweb.com/in-depth-analysis-of-googles-first-quarterly-report/> > > including the lack of information on training of Google employees > concerning their obligations. > > > > *Need to Pause* > > > > There is now sufficient justification for Google to pause the deprecation > of the User Agent as currently planned to enable the following to be > achieved. > > > > 1. Publish via this forum, W3C, IETF and privacysandbox.com the > information concerning latency. This will have a particular impact on > Google’s “Ad Systems” which are a particular focus of the commitments. > 2. Align the draft proposal to the agreed privacy standard of GDPR as > required under the commitments, removing references to concepts such as > “entropy”, “first party”, and “third party” which have no meaning under > GDPR. > 3. Update the draft proposal to enable DNS records to be used to > provide the information transmitted in the Accept-CH and Critical-CH > headers. This will enable web site operates to avoid the latency issues > described. > 4. Update the draft proposal to remove the additional headers and > complexity and follow the work already deployed by Facebook to append the > information to the exist User Agent string. See issue 200 > <https://github.com/WICG/ua-client-hints/issues/200> from WICG. > 5. Provide information concerning the core problem > <https://github.com/WICG/ua-client-hints/issues/215> being addressed > and justification as required under the commitments. > 6. Gain consensus on the draft proposal before deployment to support > the claim that this can become a standard that will support cross-browser > interoperability and will not fragment the web increasing complexity for > participants. > > If Google are not minded to adopt the above please can you provide your > justification or an alternative remedy? Given the current timeline > advertised for reduction and deprecation the industry needs this guidance > in advance of the next quarterly report in August 2022. > > > > Regards, > > > > James Rosewell > > > > *From:* 'James Rosewell - 51Degrees' via blink-dev <[email protected]> > *Sent:* 15 January 2020 18:51 > *To:* blink-dev <[email protected]> > *Cc:* [email protected]; [email protected]; [email protected] > *Subject:* Re: [blink-dev] Intent to Deprecate and Freeze: The User-Agent > string > > > > Hi Yoav, > > > > Whilst the change may be sensible from an engineering perspective - the > User-Agent string is not efficient - it's going to be a breaking change for > many industries and services that don't operate to browser provider "dog > year" timescales. > > > > Any technology being "retired" or regulatory change would typically be > acompanied with a consultaiton period and two years notice. Consider GDPR > or mobile networks. > > > > Here's some examples: > > > > 1. Programmatic advertising. It's many players - including Google - have > spent years developing message formats that have the User-Agent embedded in > them. Upgrading them to client hints will require a major version change. > The last major version change to AdCom / OpenRTB was released in November > 2019 after many years consultation. > > > > 2. Analytics solutions will need to rush through changes to support client > hints. Their users will need to migrate their deployments otherwise false > assumptions will be formed wasting effort and causing confusion. > > > > 3. Performance improvements that require instant knowledge of the user > agent to minimise data and improve render time for certain devices would be > compromised due to the handshake. Consider people living in India where > average device profiles are very different to western europe or north > america. > > > > 4. Any implementation that removes the informaiton provided from the > majority, whilst enabling Google due to it's size, influence or breadth of > services (play store, android, search), to become the defacto single source > of information about browser, operating system and device model usage > globally risks being anti competitive. The implementaiton details don't > appear to be clear enough to form a conclusion on this. > > > > If the core problem is "we are broadcasting a lot of information about our > users, in clear text, to all servers" then a staged approach might be to > strengthen the warning around non SSL secure web sites and thrid parties to > increase users control and awareness. Such a change could be accompanied > with user experience monitoring (I opt in) and the severity of the problem > better understood. Adding an SSL certificate is a change many web site > operators have already made or are planning to make. > > > > I'm interested in the subject as my business (51Degrees) provides device > detection services utilising User-Agent for web traffic. TAC and app keys > are used for non web. We've come a long way from "sniffing" using machine > learning and other techniques to support analytics, optimisation and > problem diagnosis. Switching to client hints helps us from an engineering > perspective. Rolling out the change to client hints - in parallel with > User-Agent - to gather the extra evidence fields is a substantial change. > > > > Regards, > > > > James > > -- > You received this message because you are subscribed to a topic in the > Google Groups "blink-dev" group. > To unsubscribe from this topic, visit > https://groups.google.com/a/chromium.org/d/topic/blink-dev/-2JIRNMWJ7s/unsubscribe > . > To unsubscribe from this group and all its topics, send an email to > [email protected]. > To view this discussion on the web visit > https://groups.google.com/a/chromium.org/d/msgid/blink-dev/9c0ada91-754d-48f5-a76d-f0a9fc5363a5%40chromium.org > > <https://groups.google.com/a/chromium.org/d/msgid/blink-dev/9c0ada91-754d-48f5-a76d-f0a9fc5363a5%40chromium.org?utm_medium=email&utm_source=footer> > . > This email and any attachments are confidential and may also be > privileged. If you are not the named recipient, please notify the sender > immediately and do not disclose, use, store or copy the information > contained herein. This is an email from 51Degrees.mobi Limited, Davidson > House, Forbury Square, Reading, RG1 3EU. T: +44 118 328 7152 > <+44%20118%20328%207152>; E: [email protected]; 51Degrees.mobi Limited > t/as 51Degrees. > -- You received this message because you are subscribed to the Google Groups "blink-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/a5cdb329-fa46-4592-b326-4f94855dcf4bn%40chromium.org.
