* lib/read-file.c (fread_file, read_file):
Use memset_explicit instead of explicit_bzero.
* modules/read-file (Depends-on): Depend on memset_explicit
instead of on explicit_bzero.
---
ChangeLog | 6 ++
lib/read-file.c | 12 ++--
modules/read-file | 2 +-
3 files changed
2021-08-07 Bruno Haible
read-file: Improve GCC 11 allocation-deallocation checking.
* lib/read-file.h: Include instead of .
(fread_file, read_file): Declare that deallocation must happen through
'free'.
diff --git a/lib/read-file.h b/lib/read-fi
Daiki Ueno wrote:
> diff --git a/lib/read-file.c b/lib/read-file.c
> index 88bc0070a..be93a7e09 100644
> --- a/lib/read-file.c
> +++ b/lib/read-file.c
> @@ -156,7 +156,6 @@ fread_file (FILE *stream, int flags, size_t *length)
>memcpy (new_buf, buf, save_alloc);
>expl
>From 69b8df2c5b011b09c8d03c13566356a906bd89dc Mon Sep 17 00:00:00 2001
From: Daiki Ueno
Date: Sat, 21 Nov 2020 17:32:58 +0100
Subject: [PATCH] read-file: remove dead assignment
* lib/read-file.c (fread_file): Remove dead assignment when
RF_SENSITIVE is set, flagged by clang-analy
-31 Paul Eggert
+ read-file-test: pacify --enable-gcc-warnings
+ * tests/test-read-file.c (test_read_file): Now static.
+
tempname: merge from glibc and coreutils
Also, merge in Gnulib’s more-recent methods of making it easier
to share between Gnulib and glibc
Bruno Haible writes:
>> On a different note, it was suggested to disable stdio buffering if
>> RF_SENSITIVE is set. I am attaching a patch for this.
>
> Reading from a regular file in an unbuffered way can be terribly slow.
> But here, fread_file reads in large chunks, therefore it's OK.
>
> Als
Hi Daiki,
> On a different note, it was suggested to disable stdio buffering if
> RF_SENSITIVE is set. I am attaching a patch for this.
Reading from a regular file in an unbuffered way can be terribly slow.
But here, fread_file reads in large chunks, therefore it's OK.
Also, in the specificatio
Bruno Haible writes:
> Let me update the uses of the module 'read-file' in Gnulib.
> I think the next weekly CI run would have caught this.
Thank you; I completely missed those uses in Gnulib.
On a different note, it was suggested to disable stdio buffering if
RF_SENSITI
> Both has been fixed and pushed. Thank you for the review!
Let me update the uses of the module 'read-file' in Gnulib.
I think the next weekly CI run would have caught this.
2020-05-28 Bruno Haible
Fix build errors due to read-file changes (regression fr
Bruno Haible writes:
> The first one is nearly perfect. Only the NEWS entry should go into section
> "User visible incompatible changes", not into section "Important general
> notes".
>
> The second one:
> - In fread_file, around line 100, you use a realloc workaround that appears
> to be
Hi Daiki,
> Here are the two separate commits for this.
The first one is nearly perfect. Only the NEWS entry should go into section
"User visible incompatible changes", not into section "Important general notes".
The second one:
- In fread_file, around line 100, you use a realloc workaround th
May 2020 08:14:44 +0200
Subject: [PATCH 1/2] read-file: add flags to modify reading behavior
* lib/read-file.h (RF_BINARY): New define.
(fread_file, read_file): Take FLAGS argument.
(read_binary_file): Remove.
* lib/read-file.c (internal_read_file): Merge into ...
(read_file): ... here.
* mo
Hi Daiki,
> > If you agree, I'd like to see two commits:
> > 1. the introduction of the flags and RF_BINARY,
> > 2. the RF_SENSITIVE flag.
> >
> > Do you want me to code the first commit, or do you want to do it?
>
> Sure, that would make things much simpler. I'm attaching a patch along
> th
ching a patch along
these lines.
Regards,
--
Daiki Ueno
>From 5a40a392b3b551cf92d4be022b4efd900f339e39 Mon Sep 17 00:00:00 2001
From: Daiki Ueno
Date: Tue, 26 May 2020 10:22:37 +0200
Subject: [PATCH] read-file: add flags to modify reading behavior
* lib/read-file.h (RF_BINARY, RF_SENSITI
Hi Daiki,
> The functions provided by the read-file module are handy, but they are
> suboptimal for reading sensitive materials, because they do not clear
> the allocated memory blocks upon failure.
> ...
> It's tempting to make this behavior enabled by default, but I worry
Hello,
The functions provided by the read-file module are handy, but they are
suboptimal for reading sensitive materials, because they do not clear
the allocated memory blocks upon failure. The attached patch adds
a set of variants that deal with that.
It's tempting to make this beh
> +2020-02-02 Pádraig Brady
> +
> + read-file: reduce max size from SIZE_MAX to PTRDIFF_MAX
Looks good as well.
Bruno
raig Brady
+
+ read-file: reduce max size from SIZE_MAX to PTRDIFF_MAX
+ On x86_64 with glibc-2.30, gcc 9.2 is giving:
+ error: argument 2 value '18446744073709551615'
+ exceeds maximum object size 9223372036854775807
+ [-Werror=alloc-size-larger-than=]
+
re we should start
the ball rolling on getting papers.
diff --git a/ChangeLog b/ChangeLog
index 4e1984c..2f847e6 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,12 @@
+2014-03-04 Kevin Cernekee
+
+ stdint, read-file: fix missing SIZE_MAX on Android (tiny change)
+ This is basically one o
On Wed, Mar 5, 2014 at 2:42 PM, Paul Eggert wrote:
> Thanks for the bug report and patch. A couple of things. First, the
> commentary is big enough so that we'd need a copyright assignment for this
> -- if you and your employer are OK with that please let me know and I'll
> start the ball rollin
Thanks for the bug report and patch. A couple of things. First, the
commentary is big enough so that we'd need a copyright assignment for
this -- if you and your employer are OK with that please let me know and
I'll start the ball rolling on the paperwork. If this is an issue we
can rederive
LCHAIN --system=linux-x86_64
rm -rf tmp
./gnulib-tool --create-testdir --dir tmp read-file
android-test.sh
Description: Bourne shell script
vin Cernekee
+
+ stdint, read-file: fix missing SIZE_MAX on Android
+ * lib/sys_types.in.h (_GL_INCLUDING_UNISTD_H): New macro, to work
+ around potential circular dependencies when stdint.h is included
+ by the system's sys/types.h.
+ * lib/stdint.in.h: Use _GL_
On 13/12/10 17:50, Paul Eggert wrote:
> On 12/13/10 02:09, Pádraig Brady wrote:
>> * Keep allocation to multiples of BUFSIZ which may be more efficient
>
> I don't see how the patch accomplishes this.
> If alloc starts off being BUFSIZ, and the assignment
> "alloc = alloc + alloc / 2" does not kee
On 12/13/10 02:09, Pádraig Brady wrote:
> * Keep allocation to multiples of BUFSIZ which may be more efficient
I don't see how the patch accomplishes this.
If alloc starts off being BUFSIZ, and the assignment
"alloc = alloc + alloc / 2" does not keep the allocation
to a multiple of BUFSIZ. Also,
= 1
read(0, "", 4096) = 0
cheers,
Pádraig.
>From d45e034e8ee0a456adeb7e9382fb3a059f0c7fcf Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?P=C3=A1draig=20Brady?=
Date: Mon, 13 Dec 2010 08:08:23 +
Subject: [PATCH] read-file: reorganize to avoid various issues
* lib/read-file.c (fr
On 10/24/2010 05:26 PM, Pádraig Brady wrote:
> On a related note I was wondering if we should fall back
> to increasing the buffer by +=BUFSIZ rather than *=2
> when we get ENOMEM?
I wouldn't bother, as it would lead to O(N**2) behavior,
which could well be worse than the problem that it'd cure.
I noticed large realloc()s still being done by fread_file() for the
common case of a regular file. As well as being inefficient,
it could needlessly fail for large files.
$ truncate -s1M t.f
$ ltrace ./a.out < t.f
__libc_start_main(0x8048624, 1, 0xbfd86c34, 0x8048940, 0x8048930
fileno(0x7b7420)
On 05/10/10 02:51, Bruno Haible wrote:
> Joachim Schmitz writes:
>> Doesn't work here, same problem:
>> Read 68 from /etc/resolv.conf...
>> FAIL: test-read-file
>>
>> And to clarify: the problem is not the symlink, it is the different
>> person
x is needed at all?
You did add ACL support for HP NonStop, and in a very short time, and I'd like
to thank you for this a lot!
Bye, Jojo
-Original Message-
From: Bruno Haible [mailto:br...@clisp.org]
Sent: Tuesday, October 05, 2010 3:52 AM
To: Schmitz, Joachim
Cc: bug-gnulib
Joachim Schmitz writes:
> Doesn't work here, same problem:
> Read 68 from /etc/resolv.conf...
> FAIL: test-read-file
>
> And to clarify: the problem is not the symlink, it is the different
> personalities in NonStop. A symlink inside the OSS personality wouldn't
>
2009
> /G/system/ztcpip/resconf
Thanks for the analysis. The bug is in the test; I'm fixing it through the
appended patch.
Paolo suggested:
> Creating the file within the test?
That would be too easy; it would always be a regular file. With /etc/resolv.conf
we can test whether S_ISREG
Bonzini
Sent: Monday, October 04, 2010 11:35 AM
To: Schmitz, Joachim
Cc: Bruno Haible; 'bug-gnulib'
Subject: Re: read-file on HP NonStop
On 10/04/2010 07:54 AM, Schmitz, Joachim wrote:
> So we'd need to make sure that in this test we use a file (or symlink
> to a file) in the OS
On 10/04/2010 07:54 AM, Schmitz, Joachim wrote:
So we'd need to make sure that in this test we use a file (or symlink
to a file) in the OSS namespace. One file that always exists here
would be e.g. /etc/profile.sample, so I'd use:
#ifdef __TANDEM
#define FILE1 "/etc/profile.sample"
#else
#define
Hi Bruno
Hmm,
$ cat /etc/resolv.conf|wc -c
68
So it reads all 68 chars.
OK, stepping thru...
$ run -debug -inspect=on ./test-read-file
TNS/E Native Inspect gdb Debugger [T1237 - 04-Jan-2010 10:26]
Copyright 1998 Free Software Foundation, Inc.
Copyright 2003-2008 Hewlett-Packard
Joachim Schmitz wrote:
> Read 68 from /etc/resolv.conf...
> FAIL: test-read-file
>
> Now hunting that failed check... ah, /etc/resolv.conf is a symlink here,
> apparently the test doesn't like this.
That should work too. When on Linux I change FILE1 to "/tmp/symlink
When read_file returns a buffer that contains a 201KB file, you don't want
that buffer to be and to stay 300KB large - that's just a waste of memory.
This fixes it.
2010-08-28 Bruno Haible
read-file: Don't occupy too much unused memory.
* lib/read-file.c (frea
save_errno = ENOMEM;
> + break;
> +}
I see not a single comment here. Please put a comment every 10 lines on
average.
I'm committing this combined patch.
2010-08-28 Giuseppe Scrivano
Eric Blake
Bruno Haible
read-file: Avoid memory
On 08/18/2010 05:43 AM, Giuseppe Scrivano wrote:
PING^2
are there more problems with this patch?
Subject: [PATCH] read-file: Avoid memory reallocations with regular files.
* modules/read-file (Depends-on): Add ftello and malloc-posix.
* lib/read-file.c: Include,,
,,.
(fread_file): With
PING^2
are there more problems with this patch?
Thanks,
Giuseppe
> From da6e41d8ca204903cc088444b882d904db5e649e Mon Sep 17 00:00:00 2001
> From: Giuseppe Scrivano
> Date: Tue, 3 Aug 2010 15:40:19 +0200
> Subject: [PATCH] read-file: Avoid memory reallocations with r
Ping.
I have rebased the patch.
Cheers,
Giuseppe
>From da6e41d8ca204903cc088444b882d904db5e649e Mon Sep 17 00:00:00 2001
From: Giuseppe Scrivano
Date: Tue, 3 Aug 2010 15:40:19 +0200
Subject: [PATCH] read-file: Avoid memory reallocations with regular files.
* modules/read-file (Depends
n files that don't fit
into memory.
Cheers,
Giuseppe
>From 985a52e1b654fa904e964977b24306fb97c586e7 Mon Sep 17 00:00:00 2001
From: Giuseppe Scrivano
Date: Tue, 3 Aug 2010 15:40:19 +0200
Subject: [PATCH] read-file: Avoid memory reallocations with regular files.
* modules/read-file (Depe
On 08/03/10 17:00, Giuseppe Scrivano wrote:
> + if (alloc_off - (off_t) (size_t) alloc_off
> + || (size_t) alloc_off + 1 < (size_t) alloc_off)
> +break;
It would be clearer without the casts. (Casts are often
overkill in C; they disable too much checking.) Also, I'm still
rom c0f45017bded0d958ec430a54b1fb1b29098f8ec Mon Sep 17 00:00:00 2001
From: Giuseppe Scrivano
Date: Tue, 3 Aug 2010 15:40:19 +0200
Subject: [PATCH] read-file: Avoid memory reallocations with regular files.
* modules/read-file (Depends-on): Add ftello and malloc-posix.
* lib/read-file.c: Include , ,
, .
(fread_file): With regular fi
On 08/03/10 15:17, Bruno Haible wrote:
>> > and errno should be set to ENOMEM if overflow occurs.
> I disagree. In this case there's likely something fishy with the fstat
> results, and it's better to start reading from the stream, like for
> non-regular files.
Hmm, well, it could happen when read
Hi Giuseppe,
> + long pos = ftell (stream);
> + if (pos < 0)
> +return NULL;
Not every regular file is seekable (think of files under /proc
on Linux). Therefore, of 'ftello' fails, it's better to not pre-allocate
a buffer but start reading from the stream, like for non-regular f
On 08/03/10 13:05, Giuseppe Scrivano wrote:
> + long pos = ftell (stream);
This isn't reliable on modern 32-bit hosts. Please use
ftello rather than ftell, as ftell is obsolete. You'll
need to alter the module to depend on the ftello module.
> + alloc = st.st_size - pos + 1;
The assi
types where st_size is applicable).
thanks for the review, I have adjusted the patch accordingly.
Cheers,
Giuseppe
>From e1276fa005fee48e036132ebbd9b69bdef297249 Mon Sep 17 00:00:00 2001
From: Giuseppe Scrivano
Date: Tue, 3 Aug 2010 15:40:19 +0200
Subject: [PATCH] read-file: Avoid memory
On 08/03/10 06:50, Giuseppe Scrivano wrote:
> if ((pos = fseek (stream, 0, SEEK_CUR)) == 0)
Surely this would be much, much slower if the file seeks slowly,
for example, if it is a tape drive.
It might be helpful to use fstat to find the file's type and size,
and to subtract the current file offs
Hi,
while using the read-file module, I found it can be optimized on
seekable files using the the file length as the initial buffer size; it
avoids some unnecessary reallocations.
Cheers,
Giuseppe
>From 41bc1ff07e437f524cf6cf235b9017aca1ed8a6a Mon Sep 17 00:00:00 2001
From: Giuseppe Scriv
Bruno Haible <[EMAIL PROTECTED]> writes:
> The result of these considerations is this. OK to commit?
Looks fine, please do!
/Simon
Hi Simon,
A "gnulib-tool --create-testdir --with-tests" directory is giving me this
output during "make check" on NetBSD 3.0:
Could not read file: No such file or directory
BAD: out[len] not zero: No such file or directory
Read -1077940928 from /etc/resolv.conf...
R
Paul Eggert <[EMAIL PROTECTED]> writes:
> Simon Josefsson <[EMAIL PROTECTED]> writes:
>
>> --- read-file.c 16 Jun 2006 21:26:36 +0200 1.1
>> +++ read-file.c 17 Jun 2006 18:22:29 +0200
>> @@ -43,6 +43,9 @@
>>if (!buf)
>> return NULL;
>>
>> + if (ferror (stream))
>>
Simon Josefsson <[EMAIL PROTECTED]> writes:
> --- read-file.c 16 Jun 2006 21:26:36 +0200 1.1
> +++ read-file.c 17 Jun 2006 18:22:29 +0200
> @@ -43,6 +43,9 @@
>if (!buf)
> return NULL;
>
> + if (ferror (stream))
> +return NULL;
> +
>while (!feof (stream))
t the same errno that malloc
reported. If malloc conforms to POSIX, then the errno will be useful.
If not, then you haven't lost much.
Other modules (e.g., getcwd) that are taken from glibc do
something similar, so read-file would be in good company here.
[EMAIL PROTECTED] (Larry Jones) writes:
> Simon Josefsson writes:
>>
>> OTOH, if the EOF marker is already set, should we really continue to
>> read from the stream? The function may take an already open stream,
>> that have been read from before.
>>
>> Perhaps the function should also test fer
Simon Josefsson writes:
>
> I believe the one in glibc does. Also, 1003.1-2004 says malloc should
> set errno:
But the C Standard doesn't. All the world isn't POSIX.
-Larry Jones
I don't want to learn this! It's completely irrelevant to my life! -- Calvin
Simon Josefsson writes:
>
> OTOH, if the EOF marker is already set, should we really continue to
> read from the stream? The function may take an already open stream,
> that have been read from before.
>
> Perhaps the function should also test ferror outside of the loop...
If either the end-of-
[EMAIL PROTECTED] (Larry Jones) writes:
> Simon Josefsson writes:
>>
>> Yup. That's the intention. (Of course, other functions may set errno
>> too, if, e.g., malloc or fclose fails.)
>
> malloc() doesn't set errno, either.
I believe the one in glibc does. Also, 1003.1-2004 says malloc should
[EMAIL PROTECTED] (Larry Jones) writes:
> Simon Josefsson writes:
>>
>> Looking at the code, it seems it depends on whether feof() return true
>> immediately after fopen on a zero-size file. Does the standards says
>> anything about this?
>
> Not explicitly, but there's nothing to suggest that f
Simon Josefsson writes:
>
> Yup. That's the intention. (Of course, other functions may set errno
> too, if, e.g., malloc or fclose fails.)
malloc() doesn't set errno, either.
-Larry Jones
I sure like summer vacation. -- Calvin
Simon Josefsson writes:
>
> Looking at the code, it seems it depends on whether feof() return true
> immediately after fopen on a zero-size file. Does the standards says
> anything about this?
Not explicitly, but there's nothing to suggest that fopen() can set the
end-of-file indicator for the s
[EMAIL PROTECTED] (Larry Jones) writes:
> Aaron Stone writes:
>>
>> If I were using a function like this, I'd expect NULL on error (and
>> errno set according to fopen), and an empty string if the file is empty.
>
> Note that the C Standard doesn't require fopen to set errno and
> traditional imp
Bruno Haible <[EMAIL PROTECTED]> writes:
> Simon Josefsson wrote:
>> Is this ready to be installed?
>
> Yes, it looks good.
I installed, with some of Paul's changes. If there are more review or
suggestions, we can install that later.
Aaron Stone <[EMAIL PROTECTED]> writes:
> On Fri, 2006-06-16 at 11:18 -0700, Paul Eggert wrote:
>> Simon Josefsson <[EMAIL PROTECTED]> writes:
>>
>> > + if (buf)
>> > +buf[size] = '\0';
>>
>> The file reading code returns NULL if the file was empty? But the
>> comments makes it sound like
Aaron Stone writes:
>
> If I were using a function like this, I'd expect NULL on error (and
> errno set according to fopen), and an empty string if the file is empty.
Note that the C Standard doesn't require fopen to set errno and
traditional implementations didn't, at least in some cases.
-Larr
xstrdup (""), and that
> would be more consistent. You might also add a test case, to read
> from /dev/null.
The intention was to return strdup (""). However, it do seem to
behave like that, from this test case:
{
size_t len;
char *out = read_file (FILE2, &
On Fri, 2006-06-16 at 11:18 -0700, Paul Eggert wrote:
> Simon Josefsson <[EMAIL PROTECTED]> writes:
>
> > + if (buf)
> > +buf[size] = '\0';
>
> The file reading code returns NULL if the file was empty? But the
> comments makes it sound like it will return xstrdup (""), and that
> would be m
Simon Josefsson <[EMAIL PROTECTED]> writes:
> + if (buf)
> +buf[size] = '\0';
The file reading code returns NULL if the file was empty? But the
comments makes it sound like it will return xstrdup (""), and that
would be more consistent. You might also add a test case, to read
from /dev/nul
Simon Josefsson wrote:
> Is this ready to be installed?
Yes, it looks good.
Bruno
The discussion regarding the read-file module (which started out as a
'strfile' module) died down, but I've made some changes. I've tried
this on i686-debian and i686-mingw. Is this ready to be installed?
Thanks!
In
71 matches
Mail list logo
