Re: [PATCH v2 1/2] libstdc++: also use sendfile for big files

[Jonathan Wakely via Gcc-patches]

On Wed, 22 Mar 2023 at 12:18, Jonathan Wakely wrote:

> On Wed, 22 Mar 2023 at 12:14, Jonathan Wakely wrote:
>
>>
>>
>> On Mon, 20 Mar 2023 at 22:30, Jonathan Wakely via Libstdc++ <
>> libstd...@gcc.gnu.org> wrote:
>>
>>> On 20/03/23 22:27 +0000, Jonathan Wakely wrote:
>>> >On 06/03/23 20:52 +0100, Jannik Glückert wrote:
>>> >>we were previously only using sendfile for files smaller than 2GB, as
>>> >>sendfile needs to be called repeatedly for files bigger than that.
>>> >>
>>> >>some quick numbers, copying a 16GB file, average of 10 repetitions:
>>> >>   old:
>>> >>       real: 13.4s
>>> >>       user: 0.14s
>>> >>       sys : 7.43s
>>> >>   new:
>>> >>       real: 8.90s
>>> >>       user: 0.00s
>>> >>       sys : 3.68s
>>> >>
>>> >>Additionally, this fixes
>>> >>https://gcc.gnu.org/bugzilla/show_bug.cgi?id=108178
>>> >>
>>> >>libstdc++-v3/ChangeLog:
>>> >>
>>> >>       * acinclude.m4 (_GLIBCXX_HAVE_LSEEK): define
>>> >>       * config.h.in: Regenerate.
>>> >>       * configure: Regenerate.
>>> >>       * src/filesystem/ops-common.h: enable sendfile for files
>>> >>         >2GB in std::filesystem::copy_file, skip zero-length files
>>>
>>> Also, the ChangeLog entry needs to be indented with tabs, name the
>>> changed functions, and should be complete sentences, e.g.
>>>
>>>         * acinclude.m4 (_GLIBCXX_HAVE_LSEEK): Define.
>>>         * config.h.in: Regenerate.
>>>         * configure: Regenerate.
>>>         * src/filesystem/ops-common.h (copy_file_sendfile): Define new
>>>         function for sendfile logic. Loop to support large files. Skip
>>>         zero-length files.
>>>         (do_copy_file): Use it.
>>>
>>>
>> Here's what I plan to commit in a few weeks when GCC 14 Stage 1 opens.
>>
>>
>>
> And similarly for the copy_file_range change.
>

And finally, here's the fix for PR libstdc++/108178, replacing the
zero-size check with checking for EOF in the source file

commit 3d994f1998c8f2efc2c8f5744615e92661bde46f
Author: Jonathan Wakely <jwak...@redhat.com>
Date:   Tue Mar 21 12:29:08 2023

    libstdc++: Make std::filesystem::copy_file work for procfs [PR108178]
    
    The size reported by stat is always zero for some special files such as
    those under /proc, which means the current copy_file implementation
    thinks there is nothing to copy. Instead of trusting the stat value, try
    to read a character from a streambuf and check for EOF.
    
    libstdc++-v3/ChangeLog:
    
            PR libstdc++/108178
            * src/filesystem/ops-common.h (do_copy_file): Check for empty
            files by trying to read a character.
            * testsuite/27_io/filesystem/operations/copy_file_108178.cc:
            New test.

diff --git a/libstdc++-v3/src/filesystem/ops-common.h 
b/libstdc++-v3/src/filesystem/ops-common.h
index 906436b484e..a28cbeb10b5 100644
--- a/libstdc++-v3/src/filesystem/ops-common.h
+++ b/libstdc++-v3/src/filesystem/ops-common.h
@@ -618,11 +618,16 @@ _GLIBCXX_BEGIN_NAMESPACE_FILESYSTEM
     if (sbout.is_open())
       out.fd = -1;
 
-    if (from_st->st_size && !(std::ostream(&sbout) << &sbin))
-      {
-       ec = std::make_error_code(std::errc::io_error);
-       return false;
-      }
+    // ostream::operator<<(streambuf*) fails if it extracts no characters,
+    // so don't try to use it for empty files. But from_st->st_size == 0 for
+    // some special files (e.g. procfs, see PR libstdc++/108178) so just try
+    // to read a character to decide whether there is anything to copy or not.
+    if (sbin.sgetc() != char_traits<char>::eof())
+      if (!(std::ostream(&sbout) << &sbin))
+       {
+         ec = std::make_error_code(std::errc::io_error);
+         return false;
+       }
 
     if (!sbout.close() || !sbin.close())
       {
diff --git 
a/libstdc++-v3/testsuite/27_io/filesystem/operations/copy_file_108178.cc 
b/libstdc++-v3/testsuite/27_io/filesystem/operations/copy_file_108178.cc
new file mode 100644
index 00000000000..25135834e21
--- /dev/null
+++ b/libstdc++-v3/testsuite/27_io/filesystem/operations/copy_file_108178.cc
@@ -0,0 +1,33 @@
+// { dg-do run { target c++17 } }
+// { dg-require-filesystem-ts "" }
+
+// C++17 30.10.15.4 Copy [fs.op.copy_file]
+
+#include <filesystem>
+#include <fstream>
+#include <unistd.h> // getpid
+#include <testsuite_fs.h>
+#include <testsuite_hooks.h>
+
+namespace fs = std::filesystem;
+
+void
+test_procfs() // PR libstdc++/108178
+{
+  auto pid = ::getpid();
+  std::string from = "/proc/" + std::to_string(pid) + "/status";
+  if (fs::exists(from))
+  {
+    auto to = __gnu_test::nonexistent_path();
+    fs::copy_file(from, to);
+    std::ifstream f(to);
+    VERIFY(f.is_open());
+    VERIFY(f.peek() != std::char_traits<char>::eof());
+    fs::remove(to);
+  }
+}
+
+int main()
+{
+  test_procfs();
+}