Ok,
i implemented a simple testcase (as simple as possible).
And it crashes at exactly the same place.
The testcode is attached to the mail!
I ran 7 instances of autobench against it (httperf needs to be
installed), but maybe it will crash sooner or later if only one
is running:
http://www.xenoclast.org/autobench/
autobench --single_host --host1 192.168.2.101 --port1 8080 --uri1
/index.html --quiet \
--low_rate 1500 --high_rate 2000 --rate_step 20 --num_call 10 \
--num_conn 5000 --timeout 5 --file results.tsv
On 10.09.2013 18:03, Thiago Macieira wrote:
On terça-feira, 10 de setembro de 2013 17:29:19, Benjamin Zeller wrote:
I did some more debugging today, something changed but its still
crashing at the same place:
- The QAbstractSocket now has a QAdoptedThread affinity
- The QAbstractSocket has a 0x0 pointer to the engine
- The QAbstractSocketEngine has a WorkerThread affinity
- The QAbstractSocketEngine has no parent (0x0)
- The QReadNotifier has a WorkerThread affinity
The first question I'd ask if I hadn't been debugging this with you yesterday
would be how you had got access to a socket engine if the socket engine is
null.
I know the answer: you can see the socket engine in the stack of the crashing
thread. If the pointer leading to it is null, we must conclude that the
pointer was set to null after the call was placed. There are two
possibilities:
1) it was done by another thread
2) it was done by this thread, via recursion back to the socket
I can tell you #2 doesn't happen. So it has to be #1.
Also, since the socket engine and the notifier have been deleted, any memory
you read from them must be taken with a grain of salt.
Isn't it weird that i still can read valid data from the this
pointer when i'm in a QAbstractSocketEngine:: member func,
event if its a 0 pointer in the QAbstractSocket?
It would be, if we didn't know what was happening. See above.
i tried to put a thread test in
void QAbstractSocketPrivate::resetSocketLayer()
with no luck, seems the correct thread is the first one here.
What's "correct" here?
We concluded that both threads realise the socket closed and both threads go
on to delete the engine.
If you have any other idea where i can put in some testcode
let me know and i will try. I have a extra qt compiled for
development and can put in testcode easily.
Here you can see what i pointed out, different thread
affinities and broken parentship (i assume QAbstractSocketEngine
should have a parent):
It did. But since it has already been deleted, you're reading from a dangling
pointer. You're looking at the engine after the destructor has run and the
memory block possibly overwritten. If you turn off the pretty printers from
gdb, you may be able to see the vptr in that object pointing elsewhere too.
backtrace:
0 QCoreApplication::instance qcoreapplication.h 115
0x7fbe903103d1
1 <function called from gdb> 0x7ffff41a925f
2 typeinfo name for QObjectPrivate
/opt/Qt/5.1.1/lib/libQt5Core.so.5
0x7fbe8ff1e050
3 QAbstractSocketPrivate::canReadNotification qabstractsocket.cpp
745
0x7fbe90338f5d
[cut]
Please note that Qt Creator copy & paste backtraces are useless to me (unless
need very little information). The most interesting information we can get
from a backtrace are the parameters to the functions, which this backtrace
does not show.
Creator includes that when you ask for a full backtrace. But then it includes
a lot of other stuff that makes the backtrace hard to read.
The backtrace I want is what gdb prints with simply "bt". There's no Creator
feature to get that. So when pasting backtraces to a mailing list, use gdb
directly, not Qt Creator, or edit the locals out of the full backtrace.
Btw, what would happen if the thread does not enter a eventloop but
instead handle the socket in a blocking mode, then pushes it back to
the mainthread? Would the move be correct?
I thought it was doing that. Your first backtrace showed it crashing in
waitForBytesWritten.
I think QSocketNotifier uses a posted event to reenable itself after
a move... which means this is not delivered properly
That is a good point. But the notifier isn't needed if you're using the
blocking waitForXXX functions, since those do select() directly on the socket,
bypassing the notifier. In any case, a socket notifier only works if the control
is returned to the event loop. If the control is returned to the event loop,
then it reenables itself.
_______________________________________________
Interest mailing list
Interest@qt-project.org
http://lists.qt-project.org/mailman/listinfo/interest
#-------------------------------------------------
#
# Project created by QtCreator 2013-09-11T11:50:43
#
#-------------------------------------------------
CONFIG += C++11
QT += core network
QT -= gui
TARGET = server
CONFIG += console
CONFIG -= app_bundle
TEMPLATE = app
SOURCES += main.cpp
#include <QCoreApplication>
#include <QThread>
#include <QPointer>
#include <QTcpServer>
#include <QTcpSocket>
#include <QMutex>
#include <QMutexLocker>
#include <QWaitCondition>
#include <QQueue>
#include <QObject>
#include <QDebug>
class WorkerThread;
class Dispatcher;
QDebug tDebug();
/**
* @brief The Request class
* Handles parsing of the message internally and emits
* ready() signal when a request can be pushed to the
* WorkerThread. Can be reused for next requests by the
* client.
*/
class Request : public QObject{
Q_OBJECT
public:
enum State{
WaitForHeader,
Ready
};
Request (QTcpSocket* socket, QObject* parent = 0);
void end(){
state = WaitForHeader;
arr.clear();
}
void writeResponse (const QByteArray &data){
sock->write(data);
}
QTcpSocket* socket (){
return sock.data();
}
protected:
void onReadyRead (){
arr.append(sock->readAll());
//testcode will receive a header only anyway
//we don't need to care about the message
//find end of header
if(state == WaitForHeader){
if(arr.contains("\r\n\r\n")){
qDebug()<<"received header"<<arr;
state = Ready;
emit ready();
}
}
}
signals:
void ready();
private:
State state;
QByteArray arr;
QSharedPointer<QTcpSocket> sock;
};
/**
* @brief The WorkerThread class
* Handles a request and sends back the answer,
* then enqueues itself back into the idleThread
* list
*/
class WorkerThread : public QThread
{
Q_OBJECT
public:
enum Events{
ThreadIdle = QEvent::User+1
};
WorkerThread(int id,Dispatcher* parent);
void handleRequest (QPointer<Request> r){
mutex.lock();
myRequest = r;
myRequest->moveToThread(this);
mutex.unlock();
//wake the thread up
m_wait.wakeAll();
}
int threadId (){
return id;
}
// QThread interface
protected:
virtual void run();
private:
int id;
QMutex mutex;
QWaitCondition m_wait; //the worker thread is sleeping on this waitcondition
QPointer<Request> myRequest; //the request that is handled by this thread
Dispatcher* dispatcher;
};
/**
* @brief The Dispatcher class
* Accepts connections and creates the requests,
* also handles the threads and pushes ready Requests
* to them
*/
class Dispatcher : public QTcpServer
{
Q_OBJECT
public:
/**
* @brief takeIdleThread
* get a Idle Thread for a new Request
*/
WorkerThread* takeIdleThread ()
{
QMutexLocker l(&threadListMutex);
if(idleThreads.size()){
WorkerThread* wt = idleThreads.takeFirst();
workingThreads.insert(wt->threadId(),wt);
qDebug()<<"Idle Threads: "<<idleThreads.size();
qDebug()<<"Working Threads: "<<workingThreads.size()<<" "<<workingThreads.keys();
return wt;
}
return 0;
}
/**
* @brief takeWorkingThread
* @param thread
* put a thread back into the
* idle list
*/
void takeWorkingThread (WorkerThread* thread)
{
QMutexLocker l(&threadListMutex);
if(!workingThreads.contains(thread->threadId())){
qDebug()<<"Thread ["<<thread->threadId()<<"] goes into idle mode but is not in working Map";
return;
}
idleThreads.append(workingThreads.take(thread->threadId()));
qDebug()<<"Thread ["<<thread->threadId()<<"] finished work";
qDebug()<<"Idle Threads: "<<idleThreads.size();
qDebug()<<"Working Threads: "<<workingThreads.size()<<" "<<workingThreads.keys();
l.unlock();
QCoreApplication::postEvent(this,new QEvent((QEvent::Type)WorkerThread::ThreadIdle));
}
void initThreads (const int count){
QMutexLocker l(&threadListMutex);
for(unsigned int i = 0; i < count; i++){
WorkerThread *w = new WorkerThread(i,this);
w->start();
idleThreads.append(w);
}
}
// QTcpServer interface
protected:
virtual void incomingConnection(qintptr handle)
{
qDebug()<<"Incoming";
QTcpSocket *sock = new QTcpSocket();
if(sock->setSocketDescriptor(handle)){
Request * req = new Request (sock);
connect(req,&Request::ready,this,&Dispatcher::onRequestReady);
}else
delete sock;
}
void onRequestReady ()
{
Request* req = qobject_cast<Request*>(sender());
if(req){
QPointer<Request> p(req);
pendingRequests.push_back(p);
dispatchRequests();
}
}
void dispatchRequests ()
{
qDebug()<<"Starting request dispatch";
while(pendingRequests.size()){
WorkerThread* w = takeIdleThread();
if(!w){
qDebug()<<"We have not threads";
break;
}
QPointer<Request> r = pendingRequests.dequeue();
//maybe the request was deleted while waiting (high serverload?)
if(r)
w->handleRequest(r);
}
qDebug()<<"Finished request dispatch";
}
private:
QMutex threadListMutex;
QList< WorkerThread* > idleThreads;
QMap<int,WorkerThread* > workingThreads;
QQueue<QPointer<Request> > pendingRequests;
// QObject interface
public:
virtual bool event(QEvent *e){
//make sure that there are no pending requests
//could happen if we ran out of threads but no new
//requests are coming in
if(e->type() == WorkerThread::ThreadIdle){
if(pendingRequests.size())
dispatchRequests();
return true;
}
return QObject::event(e);
}
};
Request::Request(QTcpSocket *socket, QObject *parent) : QObject(parent),state(WaitForHeader),sock(socket){
sock->setParent(this);
connect(socket,&QTcpSocket::disconnected,this,&QObject::deleteLater);
connect(socket,&QTcpSocket::readyRead,this,&Request::onReadyRead);
}
WorkerThread::WorkerThread(int id, Dispatcher *parent) : QThread(parent), id(id),dispatcher(parent){}
void WorkerThread::run()
{
//lock the mutex only in the first run, later the mutex will be locked
//at the end of the loop
mutex.lock();
forever{
tDebug()<<"Goes to sleep";
m_wait.wait(&mutex);
tDebug()<<"Woke up";
QPointer<Request> request = myRequest;
myRequest.clear();
mutex.unlock();
QString content("<html><body>Hello World</body></html>");
QByteArray message = QString(
"HTTP/1.1 200 OK\r\n"
"Content-Type: text/html; charset=utf-8\r\n"
"Content-Length: %1\r\n\r\n%2")
.arg(content.length()).arg(content).toUtf8();
QThread::usleep(100);
tDebug()<<"Write Respnse";
request->writeResponse(message);
request->end();
request->socket()->waitForBytesWritten();
//push request object back to the dispatcher's thread
//maybe the connection is reuesed
request->moveToThread(dispatcher->thread());
mutex.lock();
dispatcher->takeWorkingThread(this);
//mutex is unlocked after we went to sleep
}
}
QDebug tDebug()
{
WorkerThread* t = qobject_cast<WorkerThread*>(QThread::currentThread());
if(t)
return qDebug()<<"["<<t->threadId()<<"]";
return qDebug();
}
int main(int argc, char *argv[])
{
QCoreApplication a(argc, argv);
Dispatcher dispatch;
dispatch.initThreads(100);
dispatch.listen(QHostAddress::Any,8080);
return a.exec();
}
#include "main.moc"
_______________________________________________
Interest mailing list
Interest@qt-project.org
http://lists.qt-project.org/mailman/listinfo/interest
