Ben Hutchings wrote:
On Tue, 2024-06-04 at 06:51 -0400, Bud Heal wrote:
I filled up the (micro)SDs, including the current one I have been scanning
into, the one that I set aside as probably defective and the pair I bought
for testing. I will append the bash script.
I appreciate that you've scripted this, but you missed step 2: remove
and reinsert the card. Without that, the script may just be reading
files out of the cache if your system has enough RAM.
Do I read this clearly enough - that you have not tried this either? I
gave bug reports mentioning three devices - a scanner saving to a
microSD, a builtin SD read/writer, and transfer to Windows 10 to
confirmation and comparison, innumerable times.
Simply removing and reinserting the microSC card can lead to
irrecoverable corruption requiring a reformat. At best it will lead to
loss of the last set of scans, prior to inserting the card into the
Debian system.
In the script
you could alternately flush the cache:
echo 1 > /proc/sys/vm/drop_caches
if you're running it as root.
You already have a bug description showing that moving the SD (FAT32) to
Windows confirm the loss of data. Should I invoke /bin/sync too?
Instead, I let the desktop inform me that it is safe to remove the device.
As I look again, I read that
you wanted this done on Windows but I did not use cygwin because putting
Windows into the mix could affect the steps that invoke the data
corruption.
I wanted you to do this on Windows so that we could see if the same
corruption could occur.
As I read, I was to copy files over and over to see if the device could
contain 32GB files, because there is a claimed horde of forged devices.
I supplied a bash script which would create a test bed without putting
thumbnails anywhere. Then, when the image files are mounted on the
Debian system, the bug is primed. I wrote the script to replicate your
request - not to fill the device completely. For that, I refer to the dd
steps I reported, months ago.
Instead, I installed Debian bookworm without a graphical
desktop (unchecked anything except system and ssh). One also activates the
network interface and DHCP. That way, the SD and its folders will not be
affected by adding thumbnails, which I have identified as a way to avoid
the problem.
So disabling automatic thumbnail generation mitigates the problem, but
we already knew that, didn't we?
After satisfying that step, I tried a SD without using the wand scanner.
This newest laptop does not have a SD slot so I used a USB device. I
enabled showing thumbnails. Bookworm's behavior is different from observed
on buster and (probably) bullseye: when one copies a folder to the
computer, and opens on the desktop during copying: (1) the folder on the SD
quickly displays the thumbnails (serially from the top) but (2) the
destination folder opens, begins to show thumbnails but is prone to
continue copying but stopping placing thumbnail. If the SD's folder is
opened, no more thumbnails are added on the computer until the copying is
finished. If the SD's folder is not opened, the thumbnails may stop or may
continue, but there is an extended pause without any indication that the
last two files have been copied, until the progress dialog is closed and
its window is refreshed. If neither folder is opened during copying, the
copying progresses without such pauses and one can then open the folders
and watch the thumbnails be created.
[...]
So one or both of:
- A change in the way this newer version of GNOME (or whichever desktop
environment you're using) does thumbnail generation
- Using USB storage instead of an SD controller
mitigates the problem, but we don't know which.
Using USB storage does not mitigate the problem. You have a bug
corrupting a file system and losing data for an unwary user. There may
be a patch ready to be backported in the latest version, but that is not
mine to know. I have given you all that is needed to replicate the
problem. Fill a device full enough to let you open a folder while it is
generating thumbnails, and test, or not.
Ben.
