#36551: Race condition in savepoint ID generation causes duplicate identifiers
-------------------------------------+-------------------------------------
Reporter: Mijo Kristo | Owner: (none)
Type: Bug | Status: new
Component: Database layer | Version:
(models, ORM) |
Severity: Normal | Resolution:
Keywords: threading savepoint | Triage Stage:
race condition thread safety | Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by Natalia Bidart):
* easy: 1 => 0
Old description:
> **Problem**
>
> The BaseDatabaseWrapper.savepoint() method contains a race condition
> where the operation self.savepoint_state += 1 is not atomic. This can
> lead to duplicate savepoint IDs in multithreaded environments.
>
> **Location**
>
> django/db/backends/base/base.py, line ~608
> **
> Problematic Code**
>
> thread_ident = _thread.get_ident()
> tid = str(thread_ident).replace("-", "")
>
> self.savepoint_state += 1 # Race condition here
> sid = "s%s_x%d" % (tid, self.savepoint_state)
>
> **Impact**
>
> - Multiple threads can read the same savepoint_state value
> - Results in duplicate savepoint IDs (e.g., "s123_x5", "s123_x5")
> - Causes database errors when rolling back to savepoints
> - Can lead to data corruption in high-concurrency applications
> - Affects applications using nested transaction.atomic() blocks
>
> **Reproduction**
>
> The race condition can be reproduced with this simple test:
>
> import threading, time
>
> counter = 0
>
> def buggy_increment():
> global counter
> temp = counter
> time.sleep(0.001)
> counter = temp + 1
> print(f'Thread got: {counter}')
>
> threads = [threading.Thread(target=buggy_increment) for _ in range(5)]
> for t in threads: t.start()
> for t in threads: t.join()
>
> print(f'Final: {counter} (should be 5)')
>
> Expected output: Final: 5
> Actual output: Final: 1 (lost updates due to race condition)
>
> **Real-world scenarios**
>
> This bug can manifest in production applications as:
> - "savepoint does not exist" database errors
> - Transaction rollback failures
> - Silent data corruption in e-commerce/banking applications
> - Inventory overselling in high-traffic scenarios
>
> **Proposed Fix
> **
> Wrap the increment operation in the existing _thread_sharing_lock:
>
> thread_ident = _thread.get_ident()
> tid = str(thread_ident).replace("-", "")
>
> with self._thread_sharing_lock:
> self.savepoint_state += 1
> sid = "s%s_x%d" % (tid, self.savepoint_state)
>
> This solution:
> - Uses Django's existing thread safety infrastructure
> - Has minimal performance impact
> - Maintains backward compatibility
> - Fixes the race condition completely
>
> **Testing**
>
> The fix can be verified by running the reproduction case above - with the
> fix, all threads will get unique values.
>
> Django's existing test suite should continue to pass as this change only
> affects the thread safety of savepoint generation without changing the
> API.
>
> **Additional Notes**
>
> - This is an ideal "easy pickings" issue for new contributors
> - The bug is timing-dependent and may not reproduce consistently
> - High-concurrency production environments are most affected
> - The fix leverages existing Django patterns for thread safety
New description:
**Problem**
The BaseDatabaseWrapper.savepoint() method contains a race condition where
the operation self.savepoint_state += 1 is not atomic. This can lead to
duplicate savepoint IDs in multithreaded environments.
**Location**
django/db/backends/base/base.py, line ~608
**Problematic Code**
{{{
thread_ident = _thread.get_ident()
tid = str(thread_ident).replace("-", "")
self.savepoint_state += 1 # Race condition here
sid = "s%s_x%d" % (tid, self.savepoint_state)
}}}
**Impact**
- Multiple threads can read the same savepoint_state value
- Results in duplicate savepoint IDs (e.g., "s123_x5", "s123_x5")
- Causes database errors when rolling back to savepoints
- Can lead to data corruption in high-concurrency applications
- Affects applications using nested transaction.atomic() blocks
**Reproduction**
The race condition can be reproduced with this simple test:
{{{#!python
import threading, time
counter = 0
def buggy_increment():
global counter
temp = counter
time.sleep(0.001)
counter = temp + 1
print(f'Thread got: {counter}')
threads = [threading.Thread(target=buggy_increment) for _ in range(5)]
for t in threads: t.start()
for t in threads: t.join()
print(f'Final: {counter} (should be 5)')
}}}
Expected output: Final: 5
Actual output: Final: 1 (lost updates due to race condition)
**Real-world scenarios**
This bug can manifest in production applications as:
- "savepoint does not exist" database errors
- Transaction rollback failures
- Silent data corruption in e-commerce/banking applications
- Inventory overselling in high-traffic scenarios
**Proposed Fix **
Wrap the increment operation in the existing _thread_sharing_lock:
{{{#!python
thread_ident = _thread.get_ident()
tid = str(thread_ident).replace("-", "")
with self._thread_sharing_lock:
self.savepoint_state += 1
sid = "s%s_x%d" % (tid, self.savepoint_state)
}}}
This solution:
- Uses Django's existing thread safety infrastructure
- Has minimal performance impact
- Maintains backward compatibility
- Fixes the race condition completely
**Testing**
The fix can be verified by running the reproduction case above - with the
fix, all threads will get unique values.
Django's existing test suite should continue to pass as this change only
affects the thread safety of savepoint generation without changing the
API.
**Additional Notes**
- This is an ideal "easy pickings" issue for new contributors
- The bug is timing-dependent and may not reproduce consistently
- High-concurrency production environments are most affected
- The fix leverages existing Django patterns for thread safety
--
Comment:
Hello Mijo Kristo, thank you for your ticket report. Can you please
confirm:
* If any part of this report was generated or assisted by an AI/LLM?
* Does this come from a real-world use case, or is it primarily a
theoretical scenario?
This will help us understand and prioritize the issue better.
--
Ticket URL: <https://code.djangoproject.com/ticket/36551#comment:1>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion visit
https://groups.google.com/d/msgid/django-updates/01070198a8fe5635-ddf489c7-c074-48cd-94d0-8f70ae9fc222-000000%40eu-central-1.amazonses.com.
