Sorry, I am a newbie in the community. Next, I will try to describe the issue I encountered and the investigation process.
I tried to build a virtual hypervisor platform based on iser in debian 12, and the software platform is based on libvirt/qemu. In my test, the target is located in a debian 12 virtual machine, providing services through an mxc5 vf virtual NIC, and the service software is based on LIO. The initiator is located on the host and connected to the target through mxc5 pf NIC. The initiator enable iser transport of libvirt/qemu/libiscsi. In this environment, the initiator (qemu) will have random crashes. Then I tried to upgrade libiscsi to master (by compiling), and the probability of crashes was reduced. Then I tried to compile with -fsanitize=address, and saw the following information from the crashed process: > ==110292==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! > ================================================================= > ==110292==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x60200032249a at pc 0x7faf98a47681 bp 0x7fad65173060 sp 0x7fad65172810 > WRITE of size 48 at 0x60200032249a thread T2 > #0 0x7faf98a47680 in __interceptor_memset ../../../../src/libsanitizer/sanitizer_common/sanitizer_common_interceptors.inc:799 > #1 0x7faf96570fb0 in iscsi_szmalloc /work/libiscsi/lib/init.c:133 > #2 0x7faf9657f30b in iscsi_allocate_pdu /work/libiscsi/lib/pdu.c:204 > #3 0x7faf9658747e in iscsi_scsi_command_async /work/libiscsi/lib/iscsi-command.c:206 > #4 0x55a0ef5ef485 in iscsi_aio_ioctl /home/builder/debian/qemu-7.2+dfsg/bin/debug/native/../../../block/iscsi.c:1099 > #5 0x55a0ef2b93be in bdrv_co_ioctl /home/builder/debian/qemu-7.2+dfsg/bin/debug/native/../../../block/io.c:3210 > #6 0x55a0ef42936f in raw_co_ioctl /home/builder/debian/qemu-7.2+dfsg/bin/debug/native/../../../block/raw-format.c:423 > #7 0x55a0ef2b9289 in bdrv_co_ioctl /home/builder/debian/qemu-7.2+dfsg/bin/debug/native/../../../block/io.c:3208 > #8 0x55a0ef250249 in blk_co_do_ioctl /home/builder/debian/qemu-7.2+dfsg/bin/debug/native/../../../block/block-backend.c:1655 > #9 0x55a0ef2504d6 in blk_aio_ioctl_entry /home/builder/debian/qemu-7.2+dfsg/bin/debug/native/../../../block/block-backend.c:1676 > #10 0x55a0ef9cfee1 in coroutine_trampoline /home/builder/debian/qemu-7.2+dfsg/bin/debug/native/../../../util/coroutine-ucontext.c:177 > #11 0x7faf950709bf (/lib/x86_64-linux-gnu/libc.so.6+0x519bf) > > 0x60200032249a is located 0 bytes to the right of 10-byte region [0x602000322490,0x60200032249a) > allocated by thread T2 here: > #0 0x7faf98ab89cf in __interceptor_malloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:69 > #1 0x7faf96570b5f in iscsi_malloc /work/libiscsi/lib/init.c:84 > #2 0x7faf965c14a4 in iser_prepare_read_cmd /work/libiscsi/lib/iser.c:773 > #3 0x7faf965c205d in iser_send_command /work/libiscsi/lib/iser.c:890 > #4 0x7faf965c2584 in iscsi_iser_send_pdu /work/libiscsi/lib/iser.c:940 > #5 0x7faf965c2c3b in iscsi_iser_queue_pdu /work/libiscsi/lib/iser.c:1001 > #6 0x7faf965853d7 in iscsi_queue_pdu /work/libiscsi/lib/pdu.c:943 > #7 0x7faf96587b20 in iscsi_scsi_command_async /work/libiscsi/lib/iscsi-command.c:283 > #8 0x55a0ef5ef485 in iscsi_aio_ioctl /home/builder/debian/qemu-7.2+dfsg/bin/debug/native/../../../block/iscsi.c:1099 > #9 0x55a0ef2b93be in bdrv_co_ioctl /home/builder/debian/qemu-7.2+dfsg/bin/debug/native/../../../block/io.c:3210 > #10 0x55a0ef42936f in raw_co_ioctl /home/builder/debian/qemu-7.2+dfsg/bin/debug/native/../../../block/raw-format.c:423 > #11 0x55a0ef2b9289 in bdrv_co_ioctl /home/builder/debian/qemu-7.2+dfsg/bin/debug/native/../../../block/io.c:3208 > #12 0x55a0ef250249 in blk_co_do_ioctl /home/builder/debian/qemu-7.2+dfsg/bin/debug/native/../../../block/block-backend.c:1655 > #13 0x55a0ef2504d6 in blk_aio_ioctl_entry /home/builder/debian/qemu-7.2+dfsg/bin/debug/native/../../../block/block-backend.c:1676 > #14 0x55a0ef9cfee1 in coroutine_trampoline /home/builder/debian/qemu-7.2+dfsg/bin/debug/native/../../../util/coroutine-ucontext.c:177 > #15 0x7faf950709bf (/lib/x86_64-linux-gnu/libc.so.6+0x519bf) > > Thread T2 created by T0 here: > #0 0x7faf98a49726 in __interceptor_pthread_create ../../../../src/libsanitizer/asan/asan_interceptors.cpp:207 > #1 0x55a0ef94820b in qemu_thread_create /home/builder/debian/qemu-7.2+dfsg/bin/debug/native/../../../util/qemu-thread-posix.c:545 > #2 0x55a0ef13a2cb in iothread_init /home/builder/debian/qemu-7.2+dfsg/bin/debug/native/../../../iothread.c:215 > #3 0x55a0ef032eac in event_loop_base_complete /home/builder/debian/qemu-7.2+dfsg/bin/debug/native/../../../event-loop-base.c:86 > #4 0x55a0ef02f2e0 in user_creatable_complete /home/builder/debian/qemu-7.2+dfsg/bin/debug/native/../../../qom/object_interfaces.c:28 > #5 0x55a0ef02fd48 in user_creatable_add_type /home/builder/debian/qemu-7.2+dfsg/bin/debug/native/../../../qom/object_interfaces.c:125 > #6 0x55a0ef030314 in user_creatable_add_qapi /home/builder/debian/qemu-7.2+dfsg/bin/debug/native/../../../qom/object_interfaces.c:157 > #7 0x55a0edd64ecb in object_option_foreach_add /home/builder/debian/qemu-7.2+dfsg/bin/debug/native/../../../softmmu/vl.c:1718 > #8 0x55a0edd60be2 in qemu_create_early_backends /home/builder/debian/qemu-7.2+dfsg/bin/debug/native/../../../softmmu/vl.c:1896 > #9 0x55a0edd564c5 in qemu_init /home/builder/debian/qemu-7.2+dfsg/bin/debug/native/../../../softmmu/vl.c:3547 > #10 0x55a0eefe2d91 in main /home/builder/debian/qemu-7.2+dfsg/bin/debug/native/../../../softmmu/main.c:47 > #11 0x7faf95046249 in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58 > > SUMMARY: AddressSanitizer: heap-buffer-overflow ../../../../src/libsanitizer/sanitizer_common/sanitizer_common_interceptors.inc:799 in __interceptor_memset > Shadow bytes around the buggy address: > 0x0c048005c440: fa fa fd fd fa fa fd fa fa fa fd fa fa fa fd fa > 0x0c048005c450: fa fa fd fa fa fa fd fa fa fa fd fa fa fa fd fa > 0x0c048005c460: fa fa fd fd fa fa fd fd fa fa fd fd fa fa fd fa > 0x0c048005c470: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fd fd > 0x0c048005c480: fa fa fd fa fa fa fa fa fa fa fa fa fa fa fa fa > =>0x0c048005c490: fa fa 00[02]fa fa fa fa fa fa fa fa fa fa fa fa > 0x0c048005c4a0: fa fa fa fa fa fa fd fa fa fa 00 02 fa fa fd fd > 0x0c048005c4b0: fa fa fd fd fa fa fa fa fa fa 00 00 fa fa 00 02 > 0x0c048005c4c0: fa fa fd fd fa fa fd fd fa fa fd fa fa fa fd fa > 0x0c048005c4d0: fa fa fd fa fa fa fd fa fa fa fd fa fa fa fd fd > 0x0c048005c4e0: fa fa fd fd fa fa fd fa fa fa fd fa fa fa fd fa > Shadow byte legend (one shadow byte represents 8 application bytes): > Addressable: 00 > Partially addressable: 01 02 03 04 05 06 07 > Heap left redzone: fa > Freed heap region: fd > Stack left redzone: f1 > Stack mid redzone: f2 > Stack right redzone: f3 > Stack after return: f5 > Stack use after scope: f8 > Global redzone: f9 > Global init order: f6 > Poisoned by user: f7 > Container overflow: fc > Array cookie: ac > Intra object redzone: bb > ASan internal: fe > Left alloca redzone: ca > Right alloca redzone: cb > ==110292==ABORTING Therefore, it is determined that the allocation of iscsi_pdu->indata.data is iscsi_malloc, and based on an uncertain size, it may be released through iscsi_sfree. From the code analysis, the size of the memory released by iscsi_sfree should be equal to iscsi->smalloc_size, not an uncertain value. Therefore, a judgment was added to my patch to use iscsi_smalloc to allocate memory with data_size <= iscsi->smalloc_size. The actual size of the memory allocated by iscsi_smalloc is equal to iscsi->smalloc_size. I have submitted a pull request to the above, refer to https://github.com/sahlberg/libiscsi/pull/431, and the issue refers to https://github.com/sahlberg/libiscsi/issues/430. Michael Tokarev <m...@tls.msk.ru> 于2024年11月11日周一 04:29写道： > 10.11.2024 23:00, IriKa Qiu wrote: > > I don't understand what you mean? > > I mean that sending patch for an issue without mentioning a word > what the issue actually IS - this sounds a bit strange, don't you find? > > I don't have iscsi targets to test anymore, so don't have an (easy) way > to test if a change fixes something or not, but again, without knowing > what the problem is about, it's hardly possible to test it. > > I'd suggest to poke the upstream bugtracker too. > > Thanks, > > /mjt >
