Here comes the result of benchmark with different databases. I pushed the test code to the benchmark-databases: https://github.com/bdrung/apport/tree/benchmark-databases You can execute the test file or use `run_all` to execute them all.
## Creation and query time Results on jammy Contents with a Ryzen 7 5700G: ``` $ ./contents_leveldb.py -jd Creation duration: 39.186 s Access duration: 1.529 ms Total duration: 39.566 s $ ./contents_lmdb.py -jd Creation duration: 42.202 s Access duration: 0.292 ms Total duration: 42.746 s $ ./contents_sqlite_v1.py -jd Creation duration: 47.238 s Access duration: 1.090 ms Total duration: 47.734 s $ ./contents_sqlite_v2.py -jd Creation duration: 47.416 s Access duration: 1.232 ms Total duration: 48.882 s $ ./contents_sqlite_v3.py -jds Creation duration: 54.331 s Entries in packages table: 72,036 Entries in directories table: 92,679 Total entries: 8,037,545 Access duration: 1.537 ms Total duration: 54.657 s $ ./contents_sqlite_v3b.py -jds Creation duration: 53.860 s Entries in packages table: 72,036 Entries in directories table: 30,219 Total entries: 8,037,545 Access duration: 1.606 ms Total duration: 54.120 s $ ./contents_ndbm.py -jd Creation duration: 1 min 41.623 s (= 101.623 s) Access duration: 0.274 ms Total duration: 1 min 43.074 s (= 103.074 s) $ ./contents_dbm.py -jd Creation duration: 2 min 6.741 s (= 126.741 s) Access duration: 0.364 ms Total duration: 2 min 7.168 s (= 127.168 s) $ ./contents_pickle.py -jd Creation duration: 36.603 s Save duration: 4.301 s Access duration: 0.017 ms Total duration: 41.456 s $ ./contents_unoptimized_pickle.py -jd Creation duration: 59.445 s Save duration: 4.354 s Access duration: 0.018 ms Total duration: 1 min 4.296 s (= 64.296 s) ``` Results on a Raspberry Pi 2W on 2024-07-26 with jammy Contents (dbm and ndbm were too slow): ``` $ ./contents_leveldb.py -dsj Creation duration: 28 min 38.402 s (= 1718.402 s) Total entries: 7,911,114 Access duration: 153.075 ms Total duration: 32 min 17.500 s (= 1937.500 s) $ ./contents_lmdb.py -dsj Creation duration: 35 min 15.257 s (= 2115.257 s) Total entries: 7,911,114 Access duration: 508.313 ms Total duration: 36 min 47.932 s (= 2207.932 s) $ ./contents_sqlite_v1.py -dsj Creation duration: 14 min 32.801 s (= 872.801 s) Total entries: 7,911,114 Access duration: 39.372 ms Total duration: 16 min 0.188 s (= 960.188 s) $ ./contents_sqlite_v2.py -dsj Creation duration: 14 min 45.833 s (= 885.833 s) Total entries: 7,911,114 Access duration: 37.519 ms Total duration: 15 min 55.706 s (= 955.706 s) $ ./contents_sqlite_v3.py -dsj Creation duration: 16 min 20.365 s (= 980.365 s) Entries in packages table: 72,036 Entries in directories table: 92,679 Total entries: 8,037,545 Access duration: 45.323 ms Total duration: 18 min 6.704 s (= 1086.704 s) $ ./contents_pickle.py -dsj Creation duration: 24 min 36.127 s (= 1476.127 s) Save duration: 4 h 9 min 1.947 s (= 14941.947 s) Total entries: 7,911,114 Access duration: 33.207 ms Total duration: 4 h 37 min 24.957 s (= 16644.957 s) ``` Results on a Raspberry Pi 2W on 2024-07-25 with noble (dbm and ndbm were too slow): ``` $ ./run_all -ds $ ./contents_leveldb.py -ds Creation duration: 8 min 15.134 s (= 495.134 s) Total entries: 4,267,770 Access duration: 28.648 ms Total duration: 8 min 43.739 s (= 523.739 s) $ ./contents_lmdb.py -ds Creation duration: 10 min 5.755 s (= 605.755 s) Total entries: 4,267,770 Access duration: 55.521 ms Total duration: 11 min 42.699 s (= 702.699 s) $ ./contents_sqlite_v1.py -ds Creation duration: 4 min 37.866 s (= 277.866 s) Total entries: 4,267,770 Access duration: 38.377 ms Total duration: 5 min 14.372 s (= 314.372 s) $ ./contents_sqlite_v2.py -ds Creation duration: 4 min 41.446 s (= 281.446 s) Total entries: 4,267,770 Access duration: 40.402 ms Total duration: 5 min 14.173 s (= 314.173 s) $ ./contents_sqlite_v3.py -ds Creation duration: 5 min 27.911 s (= 327.911 s) Entries in packages table: 66,602 Entries in directories table: 79,371 Total entries: 4,333,447 Access duration: 51.763 ms Total duration: 5 min 56.019 s (= 356.019 s) $ ./contents_sqlite_v3b.py -ds Creation duration: 5 min 34.859 s (= 334.859 s) Entries in packages table: 66,602 Entries in directories table: 26,298 Total entries: 4,333,447 Access duration: 46.310 ms Total duration: 5 min 59.153 s (= 359.153 s) ``` ## Results on low memory systems Benchmark results on a Raspberry Pi 2W with swap disabled (so only 512 MB): ``` $ ./contents_leveldb.py -dsj Killed ./contents_leveldb.py failed with 137 $ ./contents_lmdb.py -dsj Killed ./contents_lmdb.py failed with 137 $ ./contents_sqlite_v1.py -dsj Creation duration: 14 min 42.957 s (= 882.957 s) Total entries: 8,037,545 Access duration: 36.858 ms Total duration: 16 min 11.612 s (= 971.612 s) $ ./contents_sqlite_v2.py -dsj Creation duration: 15 min 2.569 s (= 902.569 s) Entries in packages table: 72,036 Total entries: 8,037,545 Access duration: 38.994 ms Total duration: 16 min 12.557 s (= 972.557 s) $ ./contents_sqlite_v3.py -dsj Creation duration: 16 min 26.629 s (= 986.629 s) Entries in packages table: 72,036 Entries in directories table: 92,679 Total entries: 8,037,545 Access duration: 51.080 ms Total duration: 17 min 6.754 s (= 1026.754 s) $ ./contents_ndbm.py -dsj # never ending $ ./contents_pickle.py Killed ./contents_pickle.py failed with 137 $ ./contents_unoptimized_pickle.py Killed ./contents_unoptimized_pickle.py failed with 137 ``` Only SQLite did not get OOM killed. ## Disk comsumption ``` $ du -s contents_cache/*jammy-amd* | sort -n 633772 contents_cache/contents_mapping-jammy-amd64.leveldb 637344 contents_cache/contents_mapping-jammy-amd64_disqualified3.sqlite3 710408 contents_cache/contents_mapping-jammy-amd64_v3.sqlite3 772936 contents_cache/contents_mapping-jammy-amd64.pickle_old 776824 contents_cache/contents_mapping-jammy-amd64.pickle 815944 contents_cache/contents_mapping-jammy-amd64_v3b.sqlite3 1125732 contents_cache/contents_mapping-jammy-amd64.ndbm.db 1164144 contents_cache/contents_mapping-jammy-amd64.dbm 1313036 contents_cache/contents_mapping-jammy-amd64_v2.sqlite3 1511080 contents_cache/contents_mapping-jammy-amd64_v1.sqlite3 2397832 contents_cache/contents_mapping-jammy-amd64.lmdb ``` ``` $ du -s contents_cache/*noble-amd* | sort -n 80596 contents_cache/contents_mapping-noble-amd64.leveldb 365908 contents_cache/contents_mapping-noble-amd64.pickle_old 372064 contents_cache/contents_mapping-noble-amd64.pickle 399664 contents_cache/contents_mapping-noble-amd64_v3.sqlite3 562756 contents_cache/contents_mapping-noble-amd64.dbm 673468 contents_cache/contents_mapping-noble-amd64_v2.sqlite3 679064 contents_cache/contents_mapping-noble-amd64.ndbm.db 738288 contents_cache/contents_mapping-noble-amd64_v1.sqlite3 871544 contents_cache/contents_mapping-noble-amd64.lmdb ``` -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2073787 Title: apport-retrace needs more than 1 GB To manage notifications about this bug go to: https://bugs.launchpad.net/apport/+bug/2073787/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
