Hi Axel! Sure:
main.cpp:
#include <QDebug>
#include <QProcess>
#include <QFile>
#include <QThread>
int main(int argc, char *argv[])
{
QApplication app(argc, argv);
QString if_path = argv[1];
QString of_path = argv[2];
qInfo() << "test: reading from" << if_path << "writing to" << of_path;
QFile file(if_path);
if (!file.open(QIODevice::ReadOnly | QIODevice::Unbuffered))
{
qInfo() << "failed to open file";
return 1;
}
QProcess proc;
proc.setProgram("dd");
QStringList dd_args;
dd_args << "bs=1M" << QString("of=") + of_path; //careful; just a test
proc.setArguments(dd_args);
qInfo() << "starting dd...";
proc.start();
proc.waitForStarted();
QThread::sleep(5);
qInfo() << "now starting read/write loop..." << proc.processId();
int bs = 1024 * 1024;
while (true)
{
QByteArray block = file.read(bs);
if (block.isEmpty())
{
qInfo() << "empty read";
break;
}
qInfo() << "read:" << block.size();
qint64 written = proc.write(block);
if (written != block.size())
{
qInfo().noquote() << QString("write failed: %1 written (out of
%2)").arg(written).arg(block.size());
return 2;
}
qInfo() << "written:" << written;
//This should wait/block until all data are written/synced
//...but it doesn't...!?
qInfo() << "waiting for output process...";
bool ok = proc.waitForBytesWritten(-1); //this should block until all
written bytes have been sent to the process
qInfo() << "output process confirmed bytes written" << ok;
}
qInfo() << "done!";
QThread::sleep(15);
return app.exec();
}
qprocess_test.pro:
TARGET = qprocess_test
SOURCES = *.cpp
QT += widgets
$ qmake && make
$ dd bs=1M if=/dev/urandom of=/tmp/hugefile count=4096
$ ./qprocess_test /tmp/hugefile /mnt/tmp/targetfile
Note:
In my test case, /mnt/tmp/targetfile is on a slow NFS share but it could also
be a USB 2.0 storage; don't use something fast like a tmpfs.
The program waits for five seconds so you have time to open: top -p $(pidof
qprocess_test) (maybe hit e)
Then its memory usage grows to 4G which should never happen.
Philip
On 2/18/23 19:02, Axel Spoerl wrote:
Hi Philip,
Could you provide that minimal reproduction example?
Thanks
Axel
On 18 Feb 2023, at 01:17, Philip Seeger<phi...@c0xc.net> wrote:
Hi there!
I'm writing 1M chunks of data to a QProcess in a loop until all input data is
consumed. How do I wait after calling QProcess::write() until all data has been
sent to the process before calling write() again?
As documented, I have tried to call waitForBytesWritten(-1) after write() but
it only helps in the first iteration; then it does not seem to block until all
bytes are sent to the process. The result is that my program uses almost 4G of
RES memory if a 4G input file is being read and the loop finishes in a second.
Closing QProcess actually sends the data to the process (takes minutes) and I
don't see any progress.
Internally, Qt seems to use a QByteArray as writeBuffer and I could not find
where it checks if it exceeds a maximum size.
I believe QProcess storing 4 GB of data in an internal buffer is wrong under
any circumstances. Have I overlooked something in the documentation? How do I
turn this off?
I can provide a minimal example.
Philip
_______________________________________________
Interest mailing list
Interest@qt-project.org
https://lists.qt-project.org/listinfo/interest
_______________________________________________
Interest mailing list
Interest@qt-project.org
https://lists.qt-project.org/listinfo/interest
