Re: Trac spam attack / spam filter reactivated

[akki]

Hi

Recently, I would say since the past week, many of my activities on Trac 
are being reported as spam. Even trying to add myself to cc takes me to an 
error page.

I got the following message when I last tried to modify a ticket. I was 
changing the summary of one of the tickets:

>
>
> Submission rejected as potential spam
>
>- Akismet says content is spam
>
>
>- SpamBayes determined spam probability of 93.46%
>
>
I sometimes also get a page with a captcha but without a submit button and 
it redirects me to another TracError page if I submit it anyways by 
pressing the return key.


Please take the necessary steps to mitigate this problem and let me know if 
there is something I could help with.

Thanks

On Thursday, 7 July 2016 08:03:54 UTC+5:30, Tim Graham wrote:
>
> In the past couple hours, code.djangoproject.com experienced a spam 
> attack of new tickets and wiki pages. After running without the spam filter 
> for at least a couple months (I forget exactly when I deactivated it but it 
> was sometime after we switched to requiring authenticated users to file a 
> ticket), I've reactivated it. If you find your submissions inappropriately 
> marked as spam, let me know so we can tune the settings.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/deedad00-3ba7-4da3-ad9b-5c98efd82f02%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Need help with MySQL 5.7 crashing on Django's Jenkins

[gilberto dos santos alves]

great!

2016-08-05 19:19 GMT-03:00 Tim Graham :

> I spun up a new machine and haven't seen this issue on it, so the problem
> may have been either hardware related or something that was fixed with a
> fresher install of Ubuntu 16.04.
>
> On Wednesday, August 3, 2016 at 11:07:54 AM UTC-4, gilberto dos santos
> alves wrote:
>>
>> hummm! thanks. i will continue search for more precise aswers and
>> solutions. regards.
>>
>> 2016-08-02 17:27 GMT-03:00 Tim Graham :
>>
>>> I'm not sure exceeding max_connections is the issue. We have up to 8
>>> executors on each machine and the tests aren't running in parallel, so I
>>> think we wouldn't have more than 8 connections unless running the tests can
>>> open more than 1 connection? Also, we don't expect 8 all executors to all
>>> be running MySQL builds at the same time.
>>>
>>> In looking at innodb_buffer_pool_instances, I see that it only takes
>>> effect if innodb_buffer_pool_size is greater than 1GB (defaults to 128MB).
>>> Is the buffer pool stored on disk? We moved the datadir from /var/lib/mysql
>>> to /mnt/mysql_tempfs which is a 2GB tmpfs, so I'm wondering if we need to
>>> increase that tmpfs accordingly.
>>>
>>> MySQL docs say, "For best efficiency, specify a combination of
>>> innodb_buffer_pool_instances and innodb_buffer_pool_size so that each
>>> buffer pool instance is at least 1GB." We have 14GB of RAM on the Jenkins
>>> machine -- if we have to dedicate 8 GB to MySQL, that seems too much.
>>>
>>> https://dev.mysql.com/doc/refman/5.6/en/innodb-multiple-buff
>>> er-pools.html
>>>
>>> Here's the result of the queries you mentioned:
>>>
>>> mysql> show variables like '%buffer%';
>>> +-++
>>> | Variable_name   | Value  |
>>> +-++
>>> | bulk_insert_buffer_size | 8388608|
>>> | innodb_buffer_pool_chunk_size   | 134217728  |
>>> | innodb_buffer_pool_dump_at_shutdown | ON |
>>> | innodb_buffer_pool_dump_now | OFF|
>>> | innodb_buffer_pool_dump_pct | 25 |
>>> | innodb_buffer_pool_filename | ib_buffer_pool |
>>> | innodb_buffer_pool_instances| 1  |
>>> | innodb_buffer_pool_load_abort   | OFF|
>>> | innodb_buffer_pool_load_at_startup  | ON |
>>> | innodb_buffer_pool_load_now | OFF|
>>> | innodb_buffer_pool_size | 134217728  |
>>> | innodb_change_buffer_max_size   | 25 |
>>> | innodb_change_buffering | all|
>>> | innodb_log_buffer_size  | 16777216   |
>>> | innodb_sort_buffer_size | 1048576|
>>> | join_buffer_size| 262144 |
>>> | key_buffer_size | 536870912  |
>>> | myisam_sort_buffer_size | 8388608|
>>> | net_buffer_length   | 16384  |
>>> | preload_buffer_size | 32768  |
>>> | read_buffer_size| 131072 |
>>> | read_rnd_buffer_size| 262144 |
>>> | sort_buffer_size| 262144 |
>>> | sql_buffer_result   | OFF|
>>> +-++
>>>
>>> mysql> show variables like '%connec%';
>>> +---+-+
>>> | Variable_name | Value   |
>>> +---+-+
>>> | character_set_connection  | utf8|
>>> | collation_connection  | utf8_general_ci |
>>> | connect_timeout   | 10  |
>>> | disconnect_on_expired_password| ON  |
>>> | init_connect  | |
>>> | max_connect_errors| 100 |
>>> | max_connections   | 151 |
>>> | max_user_connections  | 0   |
>>> | performance_schema_session_connect_attrs_size | 512 |
>>> +---+-+
>>>
>>>
>>>
>>> On Tuesday, August 2, 2016 at 11:49:07 AM UTC-4, gilberto dos santos
>>> alves wrote:
>>>
 hi. IMO vars for django may uses this values inside [MYSQLD], cause
 max_connections default is 100

 innodb_buffer_pool_instances=8
 max_connections=255

 you could verify your environment using console command

 mysql -u your-user -p[your-password-whit-nospace]

 show variables like '%connec%';

 show variables like '%buffer%';


 for example my env show:
 mysql> show variables like '%buffer%';
 +--+--+
 | Variable_name   

Re: Trac spam attack / spam filter reactivated

[Tim Graham]

Yes, the bayesian spam filter is giving some false positives and the 
weighting is such that even if you submit the captcha, your comment still 
might be considered spam (spammers were completing the captchas to submit 
their content). You could look into if there's some other spam prevention 
measures in Trac that might be more effective.

On Saturday, August 6, 2016 at 7:55:18 AM UTC-4, akki wrote:
>
> Hi
>
> Recently, I would say since the past week, many of my activities on Trac 
> are being reported as spam. Even trying to add myself to cc takes me to an 
> error page.
>
> I got the following message when I last tried to modify a ticket. I was 
> changing the summary of one of the tickets:
>
>>
>>
>> Submission rejected as potential spam
>>
>>- Akismet says content is spam
>>
>>
>>- SpamBayes determined spam probability of 93.46%
>>
>>
> I sometimes also get a page with a captcha but without a submit button and 
> it redirects me to another TracError page if I submit it anyways by 
> pressing the return key.
>
>
> Please take the necessary steps to mitigate this problem and let me know 
> if there is something I could help with.
>
> Thanks
>
> On Thursday, 7 July 2016 08:03:54 UTC+5:30, Tim Graham wrote:
>>
>> In the past couple hours, code.djangoproject.com experienced a spam 
>> attack of new tickets and wiki pages. After running without the spam filter 
>> for at least a couple months (I forget exactly when I deactivated it but it 
>> was sometime after we switched to requiring authenticated users to file a 
>> ticket), I've reactivated it. If you find your submissions inappropriately 
>> marked as spam, let me know so we can tune the settings.
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/c8812758-ecbe-41c1-a582-234e8241a413%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Trac spam attack / spam filter reactivated

[akki]

This might be a quite obvious solution but people seem to be happy with the 
performance of Bayesian filter when training it properly. Here are two 
articles I found explaining some good points to keep in mind while training 
the filter - edgewall-ticket-10314 
 and trac-wiki-spamfilters ; 
just wanted to make sure we are not being bitten due to bad training.

Also, if the spammers, assuming they are bots, are able to solve recaptcha, 
we can try keycaptchas  
supported 
by Trac which are relatively harder to solve. If they are being hit 
manually by someone, the ip-throttling 
 technique might 
tackle such a situation (but then they'll use proxies, sigh!).

Well, this is a tricky experimenting business but I hope something out of 
this works.


On Saturday, 6 August 2016 18:26:34 UTC+5:30, Tim Graham wrote:
>
> Yes, the bayesian spam filter is giving some false positives and the 
> weighting is such that even if you submit the captcha, your comment still 
> might be considered spam (spammers were completing the captchas to submit 
> their content). You could look into if there's some other spam prevention 
> measures in Trac that might be more effective.
>
> On Saturday, August 6, 2016 at 7:55:18 AM UTC-4, akki wrote:
>>
>> Hi
>>
>> Recently, I would say since the past week, many of my activities on Trac 
>> are being reported as spam. Even trying to add myself to cc takes me to an 
>> error page.
>>
>> I got the following message when I last tried to modify a ticket. I was 
>> changing the summary of one of the tickets:
>>
>>>
>>>
>>> Submission rejected as potential spam
>>>
>>>- Akismet says content is spam
>>>
>>>
>>>- SpamBayes determined spam probability of 93.46%
>>>
>>>
>> I sometimes also get a page with a captcha but without a submit button 
>> and it redirects me to another TracError page if I submit it anyways by 
>> pressing the return key.
>>
>>
>> Please take the necessary steps to mitigate this problem and let me know 
>> if there is something I could help with.
>>
>> Thanks
>>
>> On Thursday, 7 July 2016 08:03:54 UTC+5:30, Tim Graham wrote:
>>>
>>> In the past couple hours, code.djangoproject.com experienced a spam 
>>> attack of new tickets and wiki pages. After running without the spam filter 
>>> for at least a couple months (I forget exactly when I deactivated it but it 
>>> was sometime after we switched to requiring authenticated users to file a 
>>> ticket), I've reactivated it. If you find your submissions inappropriately 
>>> marked as spam, let me know so we can tune the settings.
>>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/a359b711-8101-46ca-a91d-ae1b59cf7997%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Fellow Report - August 6, 2016

[Tim Graham]

Triaged

---

https://code.djangoproject.com/ticket/26977 - Instantiating an abstract 
model with a string ForeignKey fails with TypeError: isinstance() arg 2 
must be a class, type, or tuple of classes and types (accepted)

https://code.djangoproject.com/ticket/26989 - Provide a way of specifying 
ON DELETE and On UPDATE properties in model DDL (duplicate)

https://code.djangoproject.com/ticket/26990 - Support MySQL FULLTEXT INDEX 
in the model DDL (wontfix)

https://code.djangoproject.com/ticket/26987 - Document using .get() without 
args to return an object from a single-row queryset (accepted)

https://code.djangoproject.com/ticket/26994 - Remove blank line after 
template have been rendered (duplicate)

https://code.djangoproject.com/ticket/26995 - Windows Error 10053 after 
changing database engine (duplicate)

https://code.djangoproject.com/ticket/26979 - Using an 
admin.RelatedOnlyFieldListFilter in admin does not working if a type of a 
field is ForeignKey and value to_field is not pk (in my case it is UUID). 
(needsinfo)

https://code.djangoproject.com/ticket/26999 - Add changes to 
``model_to_dict`` in Django 1.10 to the release notes (fixed)

https://code.djangoproject.com/ticket/27000 - Django manage Command.usage() 
is broken (fixed)

https://code.djangoproject.com/ticket/27006 - Hints how to update code 
automatically to support Python3 (wontfix)

https://code.djangoproject.com/ticket/27011 - Django Middleware 1.10 does 
not run template_context_processors for exceptions thrown from middleware 
(invalid)

Authored



https://github.com/django/django/pull/7004 - Fixed #26991 -- Fixed a crash 
in MySQL where SQL_AUTO_IS_NULL doesn't return a result.

https://github.com/django/django/pull/7005 - Fixed #26988 -- 
Improved/clarified User.is_authenticated/anonymous comparisons.

https://github.com/django/django/pull/7016 - Fixed #27005 -- Fixed crash if 
request.META[''CONTENT_LENGTH']=''.

https://github.com/django/django/pull/7018 - Fixed #27009 -- Made 
update_session_auth_hash() rotate the session key. (and clarified docs)

Reviewed/committed

--

https://github.com/django/django/pull/6960 - Fixed #26927 -- Made subwidget 
iteration pass disabled and required attributes.

https://github.com/django/django/pull/7011 - Fixed #27001 -- Fixed a query 
count regression in ModelChoiceField with RadioSelect.

https://github.com/django/django/pull/7009 - Fixed #26981 -- Added 
DiscoverRunner.get_test_runner_kwargs().

https://github.com/django/django/pull/6961 - Fixed #26928 -- Changed forms' 
checked attribute to HTML5 boolean style.

https://github.com/django/django/pull/6638 - Fixed #26517, 26433 -- Fixed 
annotations with empty predicates.

https://github.com/django/django/pull/7029 - Fixed #27023 -- Prevented 
possibility of shell loading ~/.pythonrc.py twice.

https://github.com/django/django/pull/7010 - Edited multi-db topic guide 
for grammar and clarity.

https://github.com/django/django/pull/6725 - Fixed #26706 -- Made 
RelatedManager modification methods clear prefetch_related() cache.

https://github.com/django/django/pull/6857 - Fixed #26808 -- Added 
Meta.indexes for class-based indexes.

https://github.com/django/django/pull/7013 - Fixed #27004 -- Made 
migrations consistency check ignore unapplied squashed migrations.

https://github.com/django/django/pull/7034 - Fixed #27027 -- Restored 
Client.force_login() using the first auth backend.
https://github.com/django/django/pull/7033 - Fixed #27026 -- Fixed state 
initialization of bulk_create() objects if can_return_ids_from_bulk_insert.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/61102096-b69b-447d-9bec-80023b6ecbf0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.