Hi, I'm writing an app that will be reading potentially large files > 4 Gb in
Windows compiled using msys2/mingw64 and am seeing in issue that I believe is a
bug, hwoever, could be just Im missing something, where the function the MS CRT
function _lseek (long type parameters) Is called rather than _lseeki64 (int64
parameters)
Attempting to break it down into a simple form:
Running Ubuntu 18, autotool etc installed as well as the gcc-mingw-w64 cross gcc
----------------------------------------------------------
A main program:
#include <config.h>
#include <stdio.h>
#include <stdint.h>
int main()
{
fprintf(stderr, "test big file\n");
/* create 5G file if not there yet */
FILE *f1 = fopen("big.txt", "r");
if(!f1)
{
fprintf(stderr, "creating big file ...");
fflush(stderr);
f1 = fopen("big.txt", "w");
if(f1)
{
char buf[1024];
int64_t sz = 5LL*1024*1024*1024;
while(sz > 0)
{
fwrite(buf, sizeof(buf), 1, f1);
sz -= sizeof(buf);
}
}
fprintf(stderr, "done\n");
}
fclose(f1);
/* open file and test */
FILE *f = fopen("big.txt", "r");
fprintf(stderr, "sizeof(off_t)=%d\n",(int)sizeof(off_t));
if(f)
{
int status = fseek(f, 0, SEEK_END);
fprintf(stderr, "seek status=%d\n", status);
off_t p = ftell(f);
fprintf(stderr, "tell pos=%lld\n", (long long)p);
fclose(f);
}
return 0;
}
---------------------------------------------------------
Configure file:
AC_INIT([bigfile], [1.0.0])
AM_INIT_AUTOMAKE([-Wall -Werror foreign])
AC_CONFIG_HEADERS([config.h])
AC_CONFIG_MACRO_DIRS([m4])
# Checks for programs.
AC_PROG_CXX
AC_PROG_CC
AC_PROG_CC_STDC
gl_EARLY
# For gnulib.
gl_INIT
AC_CONFIG_FILES([lib/Makefile Makefile])
AC_OUTPUT
---------------------------------------------------------------
Makefile.am:
ACLOCAL_AMFLAGS = -I m4
SUBDIRS = lib
AM_CPPFLAGS = -I$(top_builddir)/lib -I$(top_srcdir)/lib
bin_PROGRAMS = bigtest
bigtest_SOURCES = main.cpp
bigtest_LDADD = lib/libgnu.a
EXTRA_DIST = m4/gnulib-cache.m4
-----------------------------------------------------------------
boostrap.sh
if ! [ -d gnulib ]; then \
git clone git://git.sv.gnu.org/gnulib.git gnulib
fi
./gnulib/gnulib-tool --import fseek ftell largefile stdio
touch NEWS README AUTHORS ChangeLog
autoreconf --install
------------------------------------------------------------------
Running, bootstrap to get everything ready and then shows:
Module list with included dependencies (indented):
absolute-header
errno
extensions
extern-inline
fseek
fseeko
fstat
ftell
ftello
include_next
intprops
largefile
lseek
msvc-inval
msvc-nothrow
pathmax
snippet/arg-nonnull
snippet/c++defs
snippet/warn-on-use
ssize_t
stat-time
stddef
stdio
sys_stat
sys_types
time
unistd
verify
Running a non cross configure and make works, and running the binary:
./bigtest
test big file
sizeof(off_t)=8
seek status=0
tell pos=5368709120
Running a cross configure: ./configure --host=x86_64-w64-mingw32 and then make
created bigtest.exe
Copying that to a msys2/mingw64 install from yesterday and running
./bigtest.exe fails:
./bigtest.exe
test big file
sizeof(off_t)=8
seek status=-1
tell pos=0
Running in gdb with a break at fseek:
Thread 1 hit Breakpoint 1, 0x00007ffd2f7ddfb4 in msvcrt!fseek ()
from C:\WINDOWS\System32\msvcrt.dll
>From [1] fseek in windows uses long parameters.
If I do a make dist on my sources, copying that tarball to the msys2/mingw64
system and then do a native configure/make before running it:
./bigtest.exe
test big file
creating big file ...done
sizeof(off_t)=8
seek status=-1
tell pos=0
Running in gdb shows in getting to msvcrt!fseek () as well
So it appears repeatable native msys2/mingw64 and cross at least.
----------------------------------------------------------------
Looking at configure --host=x86_64-w64-mingw32:
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for x86_64-w64-mingw32-strip... x86_64-w64-mingw32-strip
checking for a thread-safe mkdir -p... /bin/mkdir -p
checking for gawk... gawk
checking whether make sets $(MAKE)... yes
checking whether make supports nested variables... yes
checking for x86_64-w64-mingw32-g++... x86_64-w64-mingw32-g++
checking whether the C++ compiler works... yes
checking for C++ compiler default output file name... a.exe
checking for suffix of executables... .exe
checking whether we are cross compiling... yes
checking for suffix of object files... o
checking whether we are using the GNU C++ compiler... yes
checking whether x86_64-w64-mingw32-g++ accepts -g... yes
checking for style of include used by make... GNU
checking dependency style of x86_64-w64-mingw32-g++... gcc3
checking for x86_64-w64-mingw32-gcc... x86_64-w64-mingw32-gcc
checking whether we are using the GNU C compiler... yes
checking whether x86_64-w64-mingw32-gcc accepts -g... yes
checking for x86_64-w64-mingw32-gcc option to accept ISO C89... none needed
checking whether x86_64-w64-mingw32-gcc understands -c and -o together... yes
checking whether the compiler is clang... no
checking for compiler option needed when checking for declarations... none
checking dependency style of x86_64-w64-mingw32-gcc... gcc3
checking for x86_64-w64-mingw32-gcc option to accept ISO C99... none needed
checking for x86_64-w64-mingw32-gcc option to accept ISO Standard C... (cached)
none needed
checking how to run the C preprocessor... x86_64-w64-mingw32-gcc -E
checking for grep that handles long lines and -e... /bin/grep
checking for egrep... /bin/grep -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking minix/config.h usability... no
checking minix/config.h presence... no
checking for minix/config.h... no
checking whether it is safe to define __EXTENSIONS__... yes
checking whether _XOPEN_SOURCE should be defined... no
checking for Minix Amsterdam compiler... no
checking for x86_64-w64-mingw32-ar... x86_64-w64-mingw32-ar
checking for x86_64-w64-mingw32-ranlib... x86_64-w64-mingw32-ranlib
checking build system type... x86_64-pc-linux-gnu
checking host system type... x86_64-w64-mingw32
checking for _LARGEFILE_SOURCE value needed for large files... no
checking for special C compiler options needed for large files... no
checking for _FILE_OFFSET_BITS value needed for large files... 64
checking whether the preprocessor supports include_next... yes
checking whether system header files limit the line length... no
checking for complete errno.h... no
checking for EMULTIHOP value... no
checking for ENOLINK value... no
checking for EOVERFLOW value... yes
checking whether stdin defaults to large file offsets... yes
checking for pid_t... yes
checking for mode_t... yes
checking for 64-bit off_t... yes
checking for 64-bit st_size... no
checking whether fseeko is declared... yes
checking for fseeko... yes
checking whether stat file-mode macros are broken... no
checking for sys/stat.h... (cached) yes
checking for unistd.h... (cached) yes
checking for sys/param.h... yes
checking for sys/time.h... yes
checking for nlink_t... no
checking whether ftello is declared... yes
checking for ftello... yes
checking whether ftello works... guessing yes
checking for _set_invalid_parameter_handler... yes
checking for wchar_t... yes
checking for C/C++ restrict keyword... __restrict
checking for struct timespec in <time.h>... yes
checking for fseeko... (cached) yes
checking for ftello... (cached) yes
checking whether ftello works... (cached) guessing yes
checking whether lseek detects pipes... no
checking for ssize_t... yes
checking for struct stat.st_atim.tv_nsec... no
checking for struct stat.st_atimespec.tv_nsec... no
checking for struct stat.st_atimensec... no
checking for struct stat.st_atim.st__tim.tv_nsec... no
checking for struct stat.st_birthtimespec.tv_nsec... no
checking for struct stat.st_birthtimensec... no
checking for struct stat.st_birthtim.tv_nsec... no
checking for good max_align_t... yes
checking whether NULL can be used in arbitrary expressions... yes
checking which flavor of printf attribute matches inttypes macros... gnu
checking for nlink_t... (cached) no
checking that generated files are newer than configure... done
configure: creating ./config.status
config.status: creating lib/Makefile
config.status: creating Makefile
config.status: creating config.h
config.status: config.h is unchanged
config.status: executing depfiles commands
------------------------------------------------------
Looking at the cross compile:
It links the following modules are libgnu.a:
x86_64-w64-mingw32-ar cr libgnu.a stat-time.o unistd.o fstat.o lseek.o
msvc-inval.o msvc-nothrow.o stat-w32.o
And created bigtest.exe:
x86_64-w64-mingw32-g++ -DHAVE_CONFIG_H -I. -I./lib -I./lib -g -O2 -MT main.o
-MD -MP -MF .deps/main.Tpo -c -o main.o main.cpp
x86_64-w64-mingw32-g++ -g -O2 -o bigtest.exe main.o lib/libgnu.a
-------------------------------------------------------------
So a bug or not doing something correctly ?
-------------------------------------------------------------
[1]
https://docs.microsoft.com/en-us/cpp/c-runtime-library/reference/fseek-fseeki64?view=vs-2019
John D
