Public bug reported: At the end of dd-ing an image to a microSD card pretty often a lot I/O errors occur. For a brand new microSD card it started happening after 3 flashes of the UC24 image. When it started happening nothing was able to fix it. I tried different USB ports, different cables, SDWires and even commercial microSD card readers. Zapper reboot or power cycle didn’t help at all. The same microSD card and the same readers, including SDWire, worked just fine on my laptop. Flashing the image successfully on my laptop didn’t make the I/O errors on Zapper go away. It was stable and 100% reproducible. Switching Zapper image from Jammy to Noble fixed the problem. The same thing was noticed By Kevin Yeh. He reflashed all Zappers connected to RPI DUTs. It looks like it helped a lot and all the provision fails I saw after that were caused by UC image and RPI firmware mismatch.
I did more experiments with flashing microSD cards and it’s not always very quick to reproduce but when it starts happening it’s 100% reproducible. I tried different microSD cards and different USB card readers. For some cards it starts to happen immediately and for other a few flashes are needed. I tried to reproduce the same I/O errors for USB sticks but I couldn’t - it looks like there is some difference between mass storage device for pure USB sticks and microSD card readers. And it looks like there's something "written" to the ailing card because I wrote earlier when it starts to happen it happens always. (venv) ubuntu@ubuntu:~/vcs/zapper/zapper-control$ uname -a Linux ubuntu 5.15.0-1075-raspi #78-Ubuntu SMP PREEMPT Wed Mar 19 05:28:45 UTC 2025 aarch64 aarch64 aarch64 GNU/Linux (venv) ubuntu@ubuntu:~/vcs/zapper/zapper-control$ cat /etc/os-release PRETTY_NAME="Ubuntu 22.04.3 LTS" NAME="Ubuntu" VERSION_ID="22.04" VERSION="22.04.3 LTS (Jammy Jellyfish)" VERSION_CODENAME=jammy ID=ubuntu ID_LIKE=debian HOME_URL="https://www.ubuntu.com/"; SUPPORT_URL="https://help.ubuntu.com/"; BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"; PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"; UBUNTU_CODENAME=jammy Sample image used to flash to the card: ubuntu-core-24-arm64+raspi.img Command: sudo dd if=ubuntu-core-24-arm64+raspi.img of=/dev/sda bs=4M status=progress Sample dmesg logs: (also attached as a dmesg.txt file) [ 743.784231] Buffer I/O error on dev sda, logical block 884180, lost async page write [ 743.792142] Buffer I/O error on dev sda, logical block 884181, lost async page write [ 743.800021] Buffer I/O error on dev sda, logical block 884182, lost async page write [ 743.807893] Buffer I/O error on dev sda, logical block 884183, lost async page write [ 743.815823] Buffer I/O error on dev sda, logical block 884184, lost async page write [ 743.823718] Buffer I/O error on dev sda, logical block 884185, lost async page write [ 743.831665] Buffer I/O error on dev sda, logical block 884186, lost async page write [ 743.839546] Buffer I/O error on dev sda, logical block 884187, lost async page write [ 743.847580] sd 0:0:0:0: [sda] tag#0 device offline or changed [ 743.853447] blk_update_request: I/O error, dev sda, sector 7073400 op 0x1:(WRITE) flags 0x100000 phys_seg 1 prio class 0 [ 743.864602] sd 0:0:0:0: [sda] tag#0 device offline or changed [ 743.870541] blk_update_request: I/O error, dev sda, sector 7073552 op 0x1:(WRITE) flags 0x800 phys_seg 1 prio class 0 [ 743.881527] sd 0:0:0:0: [sda] tag#0 device offline or changed [ 743.887405] blk_update_request: I/O error, dev sda, sector 7073560 op 0x1:(WRITE) flags 0x100000 phys_seg 17 prio class 0 [ 743.898704] sd 0:0:0:0: [sda] tag#0 device offline or changed [ 743.904563] blk_update_request: I/O error, dev sda, sector 7073696 op 0x1:(WRITE) flags 0x800 phys_seg 1 prio class 0 [ 743.915632] sd 0:0:0:0: [sda] tag#0 device offline or changed [ 743.921560] blk_update_request: I/O error, dev sda, sector 7073704 op 0x1:(WRITE) flags 0x800 phys_seg 17 prio class 0 [ 743.932767] sd 0:0:0:0: [sda] tag#0 device offline or changed [ 743.938681] blk_update_request: I/O error, dev sda, sector 7073840 op 0x1:(WRITE) flags 0x800 phys_seg 23 prio class 0 [ 743.949776] sd 0:0:0:0: [sda] tag#0 device offline or changed [ 743.955703] blk_update_request: I/O error, dev sda, sector 7074024 op 0x1:(WRITE) flags 0x800 phys_seg 9 prio class 0 [ 743.966749] sd 0:0:0:0: [sda] tag#0 device offline or changed [ 743.972697] sd 0:0:0:0: [sda] tag#0 device offline or changed [ 743.978687] sd 0:0:0:0: [sda] tag#0 device offline or changed ** Affects: linux-raspi (Ubuntu) Importance: Undecided Status: New ** Attachment added: "Sample dmesg output" https://bugs.launchpad.net/bugs/2106646/+attachment/5870554/+files/dmesg.txt ** Changed in: linux-raspi (Ubuntu) Milestone: ubuntu-24.04.3 => None -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux-raspi in Ubuntu. https://bugs.launchpad.net/bugs/2106646 Title: MicroSD flashing fails with many I/O errors. Status in linux-raspi package in Ubuntu: New Bug description: At the end of dd-ing an image to a microSD card pretty often a lot I/O errors occur. For a brand new microSD card it started happening after 3 flashes of the UC24 image. When it started happening nothing was able to fix it. I tried different USB ports, different cables, SDWires and even commercial microSD card readers. Zapper reboot or power cycle didn’t help at all. The same microSD card and the same readers, including SDWire, worked just fine on my laptop. Flashing the image successfully on my laptop didn’t make the I/O errors on Zapper go away. It was stable and 100% reproducible. Switching Zapper image from Jammy to Noble fixed the problem. The same thing was noticed By Kevin Yeh. He reflashed all Zappers connected to RPI DUTs. It looks like it helped a lot and all the provision fails I saw after that were caused by UC image and RPI firmware mismatch. I did more experiments with flashing microSD cards and it’s not always very quick to reproduce but when it starts happening it’s 100% reproducible. I tried different microSD cards and different USB card readers. For some cards it starts to happen immediately and for other a few flashes are needed. I tried to reproduce the same I/O errors for USB sticks but I couldn’t - it looks like there is some difference between mass storage device for pure USB sticks and microSD card readers. And it looks like there's something "written" to the ailing card because I wrote earlier when it starts to happen it happens always. (venv) ubuntu@ubuntu:~/vcs/zapper/zapper-control$ uname -a Linux ubuntu 5.15.0-1075-raspi #78-Ubuntu SMP PREEMPT Wed Mar 19 05:28:45 UTC 2025 aarch64 aarch64 aarch64 GNU/Linux (venv) ubuntu@ubuntu:~/vcs/zapper/zapper-control$ cat /etc/os-release PRETTY_NAME="Ubuntu 22.04.3 LTS" NAME="Ubuntu" VERSION_ID="22.04" VERSION="22.04.3 LTS (Jammy Jellyfish)" VERSION_CODENAME=jammy ID=ubuntu ID_LIKE=debian HOME_URL="https://www.ubuntu.com/"; SUPPORT_URL="https://help.ubuntu.com/"; BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"; PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"; UBUNTU_CODENAME=jammy Sample image used to flash to the card: ubuntu-core-24-arm64+raspi.img Command: sudo dd if=ubuntu-core-24-arm64+raspi.img of=/dev/sda bs=4M status=progress Sample dmesg logs: (also attached as a dmesg.txt file) [ 743.784231] Buffer I/O error on dev sda, logical block 884180, lost async page write [ 743.792142] Buffer I/O error on dev sda, logical block 884181, lost async page write [ 743.800021] Buffer I/O error on dev sda, logical block 884182, lost async page write [ 743.807893] Buffer I/O error on dev sda, logical block 884183, lost async page write [ 743.815823] Buffer I/O error on dev sda, logical block 884184, lost async page write [ 743.823718] Buffer I/O error on dev sda, logical block 884185, lost async page write [ 743.831665] Buffer I/O error on dev sda, logical block 884186, lost async page write [ 743.839546] Buffer I/O error on dev sda, logical block 884187, lost async page write [ 743.847580] sd 0:0:0:0: [sda] tag#0 device offline or changed [ 743.853447] blk_update_request: I/O error, dev sda, sector 7073400 op 0x1:(WRITE) flags 0x100000 phys_seg 1 prio class 0 [ 743.864602] sd 0:0:0:0: [sda] tag#0 device offline or changed [ 743.870541] blk_update_request: I/O error, dev sda, sector 7073552 op 0x1:(WRITE) flags 0x800 phys_seg 1 prio class 0 [ 743.881527] sd 0:0:0:0: [sda] tag#0 device offline or changed [ 743.887405] blk_update_request: I/O error, dev sda, sector 7073560 op 0x1:(WRITE) flags 0x100000 phys_seg 17 prio class 0 [ 743.898704] sd 0:0:0:0: [sda] tag#0 device offline or changed [ 743.904563] blk_update_request: I/O error, dev sda, sector 7073696 op 0x1:(WRITE) flags 0x800 phys_seg 1 prio class 0 [ 743.915632] sd 0:0:0:0: [sda] tag#0 device offline or changed [ 743.921560] blk_update_request: I/O error, dev sda, sector 7073704 op 0x1:(WRITE) flags 0x800 phys_seg 17 prio class 0 [ 743.932767] sd 0:0:0:0: [sda] tag#0 device offline or changed [ 743.938681] blk_update_request: I/O error, dev sda, sector 7073840 op 0x1:(WRITE) flags 0x800 phys_seg 23 prio class 0 [ 743.949776] sd 0:0:0:0: [sda] tag#0 device offline or changed [ 743.955703] blk_update_request: I/O error, dev sda, sector 7074024 op 0x1:(WRITE) flags 0x800 phys_seg 9 prio class 0 [ 743.966749] sd 0:0:0:0: [sda] tag#0 device offline or changed [ 743.972697] sd 0:0:0:0: [sda] tag#0 device offline or changed [ 743.978687] sd 0:0:0:0: [sda] tag#0 device offline or changed To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux-raspi/+bug/2106646/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
