#36339: BrokenLinkEmailsMiddleware fires when Referer is invalid and URL
redirects
--------------------------+-----------------------------------------
Reporter: Xeekoo4u | Type: Uncategorized
Status: new | Component: Uncategorized
Version: 5.2 | Severity: Normal
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
--------------------------+-----------------------------------------
Hi all,
I discovered some behavior in the BrokenLinkEmailsMiddleware that I don't
fully understand: Our Website is scraped by bots on a regular basis and
one of the bots likes to use ''https://my.web.site/wp-admin'' as the
referrer. The bot requests several URLs that do not exist (e.g. ''/wp-
content'') which are part of the ''IGNORABLE_404_URLS'' and are ignored
properly, but it also requests the ''/login'' URL that returns a 301
redirect to ''/login/''. For some reason, this request results in an email
being sent if the referrer is an invalid URL (even though the actually
requested URL does not return a 404).
It confuses me that I cannot find the part of the code that validates
whether the referrer is valid or not. I discovered issues from almost a
decade ago (https://code.djangoproject.com/ticket/26059,
https://code.djangoproject.com/ticket/25971) that face a similar issue,
but in their case the referrer was (almost) equal to the URL.
Is there any advice how I could handle this issue? Why is the referrer
even validated and why doesn't the validation respect the
''IGNORABLE_404_URLS''? I'd be glad to remove some email spam :)
--
Ticket URL: <https://code.djangoproject.com/ticket/36339>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion visit
https://groups.google.com/d/msgid/django-updates/010701965026eae1-8e6518e9-6ab9-4c5c-8a40-38c9a05d9630-000000%40eu-central-1.amazonses.com.
