Re: [Qemu-devel] [ceph-users] qemu-1.4.0 and onwards, linux kernel 3.2.x, ceph-RBD, heavy I/O leads to kernel_hung_tasks_timout_secs message and unresponsive qemu-process, [Bug 1207686]

2013-08-13 Thread Sage Weil
Hi Oliver, (Posted this on the bug too, but:) Your last log revealed a bug in the librados aio flush. A fix is pushed to wip-librados-aio-flush (bobtail) and wip-5919 (master); can you retest please (with caching off again)? Thanks! sage On Fri, 9 Aug 2013, Oliver Francke wrote: > Hi Josh,

Re: [Qemu-devel] [ceph-users] qemu-1.4.0 and onwards, linux kernel 3.2.x, ceph-RBD, heavy I/O leads to kernel_hung_tasks_timout_secs message and unresponsive qemu-process, [Bug 1207686]

2013-08-13 Thread Sage Weil
On Mon, 5 Aug 2013, Mike Dawson wrote: > Josh, > > Logs are uploaded to cephdrop with the file name mikedawson-rbd-qemu-deadlock. > > - At about 2013-08-05 19:46 or 47, we hit the issue, traffic went to 0 > - At about 2013-08-05 19:53:51, ran a 'virsh screenshot' > > > Environment is: > > - Ce

Re: [Qemu-devel] [PATCH v3 2/2] rbd: link and load librbd dynamically

2013-06-21 Thread Sage Weil
Hi Anthony, Stefan, Paolo, [Resurrecting an old thread, here!] On Wed, 10 Apr 2013 Anthony Liguori wrote: > Stefan Hajnoczi writes: > > NACK > > > > I think we're solving the problem at the wrong level. Writing our own > > dynamic linker and adding boilerplate to juggle function pointers > > ev

Re: [Qemu-devel] Adding a persistent writeback cache to qemu

2013-06-21 Thread Sage Weil
On Fri, 21 Jun 2013, Stefan Hajnoczi wrote: > >> but if there's really a case for it with performance profiles then I > >> guess it would be necessary. But we should definitely get feedback from > >> the Ceph folks too. > > > > > > The specific problem we are trying to solve (in case that's not >

Re: [Qemu-devel] Adding a persistent writeback cache to qemu

2013-06-21 Thread Sage Weil
On Fri, 21 Jun 2013, Alex Bligh wrote: > Sage, > > --On 20 June 2013 08:58:19 -0700 Sage Weil wrote: > > > > I'd like to hear from Ceph folks what their position on kernel rbd vs > > > librados is. Why one do they recommend for QEMU guests and what are th

Re: [Qemu-devel] Adding a persistent writeback cache to qemu

2013-06-20 Thread Sage Weil
On Thu, 20 Jun 2013, Stefan Hajnoczi wrote: > > The concrete problem here is that flashcache/dm-cache/bcache don't > > work with the rbd (librbd) driver, as flashcache/dm-cache/bcache > > cache access to block devices (in the host layer), and with rbd > > (for instance) there is no access to a bloc

[Qemu-devel] [PATCH] qemu-iotests: send 'rbd rm ...' stderr to /dev/null

2013-04-16 Thread Sage Weil
The rbd cli tool now sends progress info to stderr; send that to the bit bucket too. Signed-off-by: Sage Weil --- tests/qemu-iotests/common.rc |2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/qemu-iotests/common.rc b/tests/qemu-iotests/common.rc index a536bf7

Re: [Qemu-devel] [PATCH 2/4] rbd: allow escaping in config string

2011-09-19 Thread Sage Weil
On Mon, 19 Sep 2011, Kevin Wolf wrote: > If src ends with a backslash, you read beyond the end of the string. Right, sending an updated patch. > Wouldn't it make sense to have the unescape integrated in > qemu_rbd_next_tok? Or are there any places where you would want to call > it without doing a

[Qemu-devel] [PATCH] rbd: allow escaping in config string

2011-09-19 Thread Sage Weil
The config string is variously delimited by =, @, and /, depending on the field. Allow these characters to be escaped by preceeding them with \. Signed-off-by: Sage Weil --- block/rbd.c | 29 +++-- 1 files changed, 27 insertions(+), 2 deletions(-) diff --git a/block

[Qemu-devel] [PATCH 2/4] rbd: allow escaping in config string

2011-09-15 Thread Sage Weil
The config string is variously delimited by =, @, and /, depending on the field. Allow these characters to be escaped by preceeding them with \. Signed-off-by: Sage Weil --- block/rbd.c | 29 +++-- 1 files changed, 27 insertions(+), 2 deletions(-) diff --git a/block

[Qemu-devel] [PATCH 1/4] rbd: ignore failures when reading from default conf location

2011-09-15 Thread Sage Weil
If we are reading from the default config location, ignore any failures. It is perfectly legal for the user to specify exactly the options they need and to not rely on any config file. Signed-off-by: Sage Weil --- block/rbd.c | 14 -- 1 files changed, 4 insertions(+), 10 deletions

[Qemu-devel] [PATCH 3/4] rbd: update comment heading

2011-09-15 Thread Sage Weil
Properly document the configuration string syntax and semantics. Remove (out of date) details about the librbd implementation. Signed-off-by: Sage Weil --- block/rbd.c | 28 +--- 1 files changed, 13 insertions(+), 15 deletions(-) diff --git a/block/rbd.c b/block

[Qemu-devel] [PATCH 4/4] rbd: call flush, if available

2011-09-15 Thread Sage Weil
librbd recently added async writeback and flush support. If the new rbd_flush() call is available, call it. Signed-off-by: Sage Weil --- block/rbd.c | 12 1 files changed, 12 insertions(+), 0 deletions(-) diff --git a/block/rbd.c b/block/rbd.c index 202e7ec..af65924 100644

[Qemu-devel] [PATCH 0/4] More RBD updates

2011-09-15 Thread Sage Weil
x27;:' is used as a delimiter). The last patch implements flush when rbd_flush() is available. This lets us take advantage of write buffering in newer versions of librbd, which improves performance significantly for many workloads (including the trivial qemu-img convert). Thanks! sage

Re: [Qemu-devel] [PATCH 0/2] improve qemu-img conversion performance

2011-09-11 Thread Sage Weil
On Fri, 9 Sep 2011, Kevin Wolf wrote: > Am 08.09.2011 18:36, schrieb Sage Weil: > > On Thu, 8 Sep 2011, Kevin Wolf wrote: > >> Am 08.09.2011 01:06, schrieb Yehuda Sadeh: > >>> The following set of patches improve the qemu-img conversion process > >>>

Re: [Qemu-devel] [PATCH 0/2] improve qemu-img conversion performance

2011-09-08 Thread Sage Weil
On Thu, 8 Sep 2011, Stefan Hajnoczi wrote: > On Wed, Sep 07, 2011 at 04:06:51PM -0700, Yehuda Sadeh wrote: > > The following set of patches improve the qemu-img conversion process > > performance. When using a higher latency backend, small writes have a > > severe impact on the time it takes to do

Re: [Qemu-devel] [PATCH 0/2] improve qemu-img conversion performance

2011-09-08 Thread Sage Weil
On Thu, 8 Sep 2011, Kevin Wolf wrote: > Am 08.09.2011 01:06, schrieb Yehuda Sadeh: > > The following set of patches improve the qemu-img conversion process > > performance. When using a higher latency backend, small writes have a > > severe impact on the time it takes to do image conversion. > > W

Re: [Qemu-devel] [PATCH 1/2] qemu-img: async write to block device when converting image

2011-09-07 Thread Sage Weil
On Wed, 7 Sep 2011, Yehuda Sadeh wrote: > In order to improve image conversion process, instead of synchronously > writing the destingation image, we keep a window of async writes. > > Signed-off-by: Yehuda Sadeh Small fix below: > --- > qemu-img.c | 28 +++- > 1 file

[Qemu-devel] [PATCH 3/3] rbd: fix leak in qemu_rbd_open failure paths

2011-09-07 Thread Sage Weil
Fix leak of s->snap in failure path. Simplify error paths for the whole function. Reported-by: Stefan Hajnoczi Signed-off-by: Sage Weil --- block/rbd.c | 28 +--- 1 files changed, 13 insertions(+), 15 deletions(-) diff --git a/block/rbd.c b/block/rbd.c index 2763

[Qemu-devel] [PATCH 2/3] rbd: clean up, fix style

2011-09-07 Thread Sage Weil
No assignment in condition. Remove duplicate ret > 0 check. Signed-off-by: Sage Weil --- block/rbd.c | 17 - 1 files changed, 8 insertions(+), 9 deletions(-) diff --git a/block/rbd.c b/block/rbd.c index 6135fc1..2763092 100644 --- a/block/rbd.c +++ b/block/rbd.c @@ -391

[Qemu-devel] [PATCH 1/3] rbd: allow client id to be specified in config string

2011-09-07 Thread Sage Weil
Allow the client id to be specified in the config string via 'id=' so that users can control who they authenticate as. Currently they are stuck with the default ('admin'). This is necessary for anyone using authentication in their environment. Signed-off-by: Sage Weil ---

[Qemu-devel] [PATCH v3] rbd: fix leak in qemu_rbd_open failure paths

2011-09-04 Thread Sage Weil
Fix leak of s->snap in failure path. Simplify error paths for the whole function. Reported-by: Stefan Hajnoczi Signed-off-by: Sage Weil --- block/rbd.c | 28 +--- 1 files changed, 13 insertions(+), 15 deletions(-) diff --git a/block/rbd.c b/block/rbd.c index 2763

Re: [Qemu-devel] [PATCH v2] rbd: fix leak in qemu_rbd_open failure paths

2011-09-04 Thread Sage Weil
On Sun, 4 Sep 2011, Stefan Hajnoczi wrote: > On Sat, Sep 3, 2011 at 11:04 PM, Sage Weil wrote: > > +failed_shutdown: > >     rados_shutdown(s->cluster); > > +    qemu_free(s->snap); > > Sorry for being a pain here. This patch is against an old qemu.git >

[Qemu-devel] [PATCH v2] rbd: fix leak in qemu_rbd_open failure paths

2011-09-03 Thread Sage Weil
Fix leak of s->snap in failure path. Simplify error paths for the whole function. Reported-by: Stefan Hajnoczi Signed-off-by: Sage Weil --- v2: fixes all error paths, not just the first one. block/rbd.c | 28 +--- 1 files changed, 13 insertions(+), 15 deleti

[Qemu-devel] [PATCH 1/3] rbd: fix leak in failure path

2011-08-23 Thread Sage Weil
Fix leak of s->snap when rados_create fails. Reported-by: Stefan Hajnoczi Signed-off-by: Sage Weil --- block/rbd.c |9 + 1 files changed, 5 insertions(+), 4 deletions(-) diff --git a/block/rbd.c b/block/rbd.c index d5659cd..52b79fa 100644 --- a/block/rbd.c +++ b/block/rb

[Qemu-devel] [PATCH 3/3] rbd: clean up, fix style

2011-08-23 Thread Sage Weil
No assignment in condition. Remove duplicate ret > 0 check. Signed-off-by: Sage Weil --- block/rbd.c | 17 - 1 files changed, 8 insertions(+), 9 deletions(-) diff --git a/block/rbd.c b/block/rbd.c index e239e04..5423a2d 100644 --- a/block/rbd.c +++ b/block/rbd.c @@ -391

[Qemu-devel] [PATCH 2/3] rbd: allow client id to be specified in config string

2011-08-23 Thread Sage Weil
Allow the client id to be specified in the config string via 'id=' so that users can control who they authenticate as. Currently they are stuck with the default ('admin'). This is necessary for anyone using authentication in their environment. Signed-off-by: Sage Weil ---

[Qemu-devel] [PATCH v2] rbd: allow client id to be specified in config string

2011-08-22 Thread Sage Weil
Allow the client id to be specified in the config string via 'id=' so that users can control who they authenticate as. Currently they are stuck with the default ('admin'). This is necessary for anyone using authentication in their environment. Signed-off-by: Sage Weil -

[Qemu-devel] [PATCH] rbd: allow client id to be specified in config string

2011-08-22 Thread Sage Weil
Allow the client id to be specified in the config string via 'id=' so that users can control who they authenticate as. Currently they are stuck with the default ('admin'). This is necessary for anyone using authentication in their environment. Signed-off-by: Sage Weil Revie

Re: [Qemu-devel] Qemu + RBD = ceph::buffer::end_of_buffer

2011-05-06 Thread Sage Weil
ter branches of ceph.git. Or you can apply the patch below. Thanks! sage >From 48d94f6e34da8ace2b5cb128de1bcfb34b3c40b5 Mon Sep 17 00:00:00 2001 From: Sage Weil Date: Fri, 6 May 2011 13:42:23 -0700 Subject: [PATCH] osd: used fixed size types for fiemap/mapext/sparseread encoding The client

Re: [Qemu-devel] Qemu + RBD = ceph::buffer::end_of_buffer

2011-05-06 Thread Sage Weil
On Fri, 6 May 2011, Dyweni - Qemu-Devel wrote: > Hi Sage/Lists! > > > (gdb) f 8 > #8 0x7f170174198a in decode (v=@0x7f16fd8b190c, p=...) at > include/encoding.h:80 > 80 WRITE_INTTYPE_ENCODER(uint32_t, le32) > (gdb) p n > No symbol "n" in current context. > (gdb) p s > No symbol "s" in c

Re: [Qemu-devel] Qemu + RBD = ceph::buffer::end_of_buffer

2011-05-06 Thread Sage Weil
f 9 (or 8?) p n p s (BTW this might be faster over irc, #ceph on irc.oftc.net) Thanks! sage On Fri, 6 May 2011, Dyweni - Qemu-Devel wrote: > Hi Sage/Lists! > > > (gdb) print c->bl._len > $1 = 20 > > > And in case this is helpful: > > (gdb) print *c > $2 = {lock = {name = 0x7f1701430f8d "

Re: [Qemu-devel] Qemu + RBD = ceph::buffer::end_of_buffer

2011-05-06 Thread Sage Weil
On Fri, 6 May 2011, Dyweni - Qemu-Devel wrote: > Hi Josh/Lists! > > 463 ::decode(*data_bl, iter); > (gdb) print r > $1 = 0 > (gdb) print data_bl > $2 = (ceph::bufferlist *) 0x7f16f40d6060 > (gdb) print data_bl->_len > $3 = 0 What about c->bl._len? sage > (gdb) print iter->off > $4

Re: [Qemu-devel] [PATCH v2 1/2] rbd: use the higher level librbd instead of just librados

2011-04-12 Thread Sage Weil
On Tue, 12 Apr 2011, Stefan Hajnoczi wrote: > On Tue, Apr 12, 2011 at 1:18 AM, Josh Durgin > wrote: > > On 04/08/2011 01:43 AM, Stefan Hajnoczi wrote: > >> > >> On Mon, Mar 28, 2011 at 04:15:57PM -0700, Josh Durgin wrote: > >>> > >>> librbd stacks on top of librados to provide access > >>> to rbd

[Qemu-devel] Re: [PATCH] ceph/rbd block driver for qemu-kvm (v7)

2010-11-01 Thread Sage Weil
gt; +if (*p + 8 > end) > +return -ERANGE; > + > +*v = *(uint64_t *)(*p); > +cpu_to_le64s(v); > +*p += 8; > +return 0; > +} > + > +static int decode_str(char **p, const char *end, char **s) > +{ > +uint32_t len; > +int r; > + > +if ((r =

Re: Fwd: [Qemu-devel] bdrv_flush for qemu block drivers nbd, rbd and sheepdog

2010-10-22 Thread Sage Weil
On Fri, 22 Oct 2010, Kevin Wolf wrote: > [ Adding qemu-devel to CC again ] > > Am 21.10.2010 20:59, schrieb Sage Weil: > > On Thu, 21 Oct 2010, Christian Brunner wrote: > >> Hi, > >> > >> is there a flush operation in librados? - I guess the only way t

Re: [Qemu-devel] [PATCH] ceph/rbd block driver for qemu-kvm (v4)

2010-10-07 Thread Sage Weil
On Thu, 7 Oct 2010, Anthony Liguori wrote: > On 10/07/2010 04:49 PM, Yehuda Sadeh Weinraub wrote: > > On Thu, Oct 7, 2010 at 2:04 PM, Anthony Liguori > > wrote: > > > > > On 10/07/2010 03:47 PM, Yehuda Sadeh Weinraub wrote: > > > > > > > > How is that possible? Are the callbacks deliver

Re: [Qemu-devel] [RFC PATCH 1/1] ceph/rbd block driver for qemu-kvm

2010-05-25 Thread Sage Weil
On Tue, 25 May 2010, Avi Kivity wrote: > > What's the reason for not having these drivers upstream? Do we gain > > anything by hiding them from our users and requiring them to install the > > drivers separately from somewhere else? > > > > Six months. FWIW, we (Ceph) aren't complaining about