Re: [Qemu-devel] [PATCH v9 0/8] Optimize VMDK I/O by allocating multiple clusters

2017-10-19 Thread Ashijeet Acharya
On Fri, Oct 20, 2017 at 11:58 Fam Zheng wrote: > On Mon, 10/09 22:12, Fam Zheng wrote: > > On Mon, 10/09 18:29, Ashijeet Acharya wrote: > > > Optimization test results: > > > > > > This patch series improves 128 KB sequential write performan

[Qemu-devel] [PATCH v9 8/8] vmdk: Make vmdk_get_cluster_offset() return cluster offset only

2017-10-09 Thread Ashijeet Acharya
: Ashijeet Acharya Reviewed-by: Fam Zheng --- block/vmdk.c | 56 1 file changed, 12 insertions(+), 44 deletions(-) diff --git a/block/vmdk.c b/block/vmdk.c index cbeffb1552..497e30f6df 100644 --- a/block/vmdk.c +++ b/block/vmdk.c @@ -1511,25

[Qemu-devel] [PATCH v9 6/8] vmdk: New functions to assist allocating multiple clusters

2017-10-09 Thread Ashijeet Acharya
offset of the first of the many newly allocated clusters. Also, provide proper documentation for both. Signed-off-by: Ashijeet Acharya Reviewed-by: Fam Zheng --- block/vmdk.c | 201 --- 1 file changed, 191 insertions(+), 10 deletions(-) diff

[Qemu-devel] [PATCH v9 7/8] vmdk: Update metadata for multiple clusters

2017-10-09 Thread Ashijeet Acharya
Include a next pointer in VmdkMetaData struct to point to the previous allocated L2 table. Modify vmdk_L2update to start updating metadata for allocation of multiple clusters at once. Signed-off-by: Ashijeet Acharya Reviewed-by: Fam Zheng --- block/vmdk.c | 128

[Qemu-devel] [PATCH v9 4/8] vmdk: Factor out metadata loading code out of vmdk_get_cluster_offset()

2017-10-09 Thread Ashijeet Acharya
Move the cluster tables loading code out of the existing vmdk_get_cluster_offset() function and implement it in separate get_cluster_table() and vmdk_l2load() functions. Signed-off-by: Ashijeet Acharya Reviewed-by: Fam Zheng --- block/vmdk.c | 153

[Qemu-devel] [PATCH v9 5/8] vmdk: Set maximum bytes allocated in one cycle

2017-10-09 Thread Ashijeet Acharya
Set the maximum bytes allowed to get allocated at once to be not more than the extent size boundary to handle writes at two separate extents appropriately. Signed-off-by: Ashijeet Acharya Reviewed-by: Fam Zheng --- block/vmdk.c | 13 +++-- 1 file changed, 11 insertions(+), 2 deletions

[Qemu-devel] [PATCH v9 2/8] vmdk: Rename get_whole_cluster() to vmdk_perform_cow()

2017-10-09 Thread Ashijeet Acharya
Rename the existing function get_whole_cluster() to vmdk_perform_cow() as its sole purpose is to perform COW for the first and the last allocated clusters if needed. Signed-off-by: Ashijeet Acharya Reviewed-by: Fam Zheng --- block/vmdk.c | 23 ++- 1 file changed, 14

[Qemu-devel] [PATCH v9 3/8] vmdk: Rename get_cluster_offset() to vmdk_get_cluster_offset()

2017-10-09 Thread Ashijeet Acharya
Rename the existing get_cluster_offset() to vmdk_get_cluster_offset() and update name in all the callers accordingly. Signed-off-by: Ashijeet Acharya Reviewed-by: Fam Zheng --- block/vmdk.c | 46 +++--- 1 file changed, 23 insertions(+), 23 deletions

[Qemu-devel] [PATCH v9 1/8] vmdk: Move vmdk_find_offset_in_cluster() to the top

2017-10-09 Thread Ashijeet Acharya
Move the existing vmdk_find_offset_in_cluster() function to the top of the driver. Signed-off-by: Ashijeet Acharya Reviewed-by: Fam Zheng --- block/vmdk.c | 24 1 file changed, 12 insertions(+), 12 deletions(-) diff --git a/block/vmdk.c b/block/vmdk.c index c665bcc977

[Qemu-devel] [PATCH v9 0/8] Optimize VMDK I/O by allocating multiple clusters

2017-10-09 Thread Ashijeet Acharya
hmark test results in v2 Ashijeet Acharya (8): vmdk: Move vmdk_find_offset_in_cluster() to the top vmdk: Rename get_whole_cluster() to vmdk_perform_cow() vmdk: Rename get_cluster_offset() to vmdk_get_cluster_offset() vmdk: Factor out metadata loading code out of vmdk_get_cluster_offse

Re: [Qemu-devel] [PATCH v8 0/8] Optimize VMDK I/O by allocating multiple clusters

2017-10-04 Thread Ashijeet Acharya
On Wed, Oct 4, 2017 at 1:58 PM, Fam Zheng wrote: > On Wed, 10/04 13:47, Ashijeet Acharya wrote: > > Fam: Ping? > > Hi Ashijeet, looks like this patch doesn't apply to current master, could > you > rebase and post another version? > Hello Fam, I will try to do it

Re: [Qemu-devel] [PATCH v8 0/8] Optimize VMDK I/O by allocating multiple clusters

2017-10-04 Thread Ashijeet Acharya
On Thu, Aug 10, 2017 at 11:13 PM, Stefan Hajnoczi wrote: > On Thu, Aug 10, 2017 at 9:18 AM, Ashijeet Acharya > wrote: > > On Thu, Aug 10, 2017 at 1:41 PM, Stefan Hajnoczi > wrote: > >> > >> On Thu, Jul 27, 2017 at 3:33 PM, Ashijeet Acharya > >> wrot

Re: [Qemu-devel] [PATCH v2 0/7] Refactor DMG driver to have chunk size independence

2017-09-05 Thread Ashijeet Acharya
On Tue, Sep 5, 2017 at 4:28 PM, Stefan Hajnoczi wrote: > On Wed, Aug 30, 2017 at 06:32:52PM +0530, Ashijeet Acharya wrote: > > On Tue, Aug 29, 2017 at 8:55 PM, Stefan Hajnoczi > wrote: > > > > > On Sun, Aug 20, 2017 at 1:47 PM, Ashijeet Acharya > > > wrote:

Re: [Qemu-devel] [PATCH v2 0/7] Refactor DMG driver to have chunk size independence

2017-08-30 Thread Ashijeet Acharya
On Tue, Aug 29, 2017 at 8:55 PM, Stefan Hajnoczi wrote: > On Sun, Aug 20, 2017 at 1:47 PM, Ashijeet Acharya > wrote: > > On Fri, May 5, 2017 at 7:29 PM, Stefan Hajnoczi > wrote: > >> > >> On Thu, Apr 27, 2017 at 01:36:30PM +0530, Ashijeet Acharya wrote: &g

Re: [Qemu-devel] [PATCH v2 0/7] Refactor DMG driver to have chunk size independence

2017-08-20 Thread Ashijeet Acharya
On Fri, May 5, 2017 at 7:29 PM, Stefan Hajnoczi wrote: > On Thu, Apr 27, 2017 at 01:36:30PM +0530, Ashijeet Acharya wrote: > > This series helps to provide chunk size independence for DMG driver to > prevent > > denial-of-service in cases where untrusted files are being acce

Re: [Qemu-devel] [PATCH v8 0/8] Optimize VMDK I/O by allocating multiple clusters

2017-08-10 Thread Ashijeet Acharya
On Thu, Aug 10, 2017 at 1:41 PM, Stefan Hajnoczi wrote: > On Thu, Jul 27, 2017 at 3:33 PM, Ashijeet Acharya > wrote: > > Previously posted series patches: > > v1 - http://lists.nongnu.org/archive/html/qemu-devel/2017- > 03/msg02044.html > > v2 - http://lists.nongnu.

[Qemu-devel] [PATCH v8 5/8] vmdk: Set maximum bytes allocated in one cycle

2017-07-27 Thread Ashijeet Acharya
Set the maximum bytes allowed to get allocated at once to be not more than the extent size boundary to handle writes at two separate extents appropriately. Signed-off-by: Ashijeet Acharya Reviewed-by: Fam Zheng --- block/vmdk.c | 13 +++-- 1 file changed, 11 insertions(+), 2 deletions

[Qemu-devel] [PATCH v8 6/8] vmdk: New functions to assist allocating multiple clusters

2017-07-27 Thread Ashijeet Acharya
offset of the first of the many newly allocated clusters. Also, provide proper documentation for both. Signed-off-by: Ashijeet Acharya Reviewed-by: Fam Zheng --- block/vmdk.c | 201 --- 1 file changed, 191 insertions(+), 10 deletions(-) diff

[Qemu-devel] [PATCH v8 7/8] vmdk: Update metadata for multiple clusters

2017-07-27 Thread Ashijeet Acharya
Include a next pointer in VmdkMetaData struct to point to the previous allocated L2 table. Modify vmdk_L2update to start updating metadata for allocation of multiple clusters at once. Signed-off-by: Ashijeet Acharya Reviewed-by: Fam Zheng --- block/vmdk.c | 128

[Qemu-devel] [PATCH v8 0/8] Optimize VMDK I/O by allocating multiple clusters

2017-07-27 Thread Ashijeet Acharya
lable (fam) - call bdrv_pwrite_sync() for batches of atmost 512 clusters at once (fam) Changes in v2: - segregate the ugly Patch 1 in v1 into 6 readable and sensible patches - include benchmark test results in v2 Ashijeet Acharya (8): vmdk: Move vmdk_find_offset_in_cluster() to the top vm

[Qemu-devel] [PATCH v8 8/8] vmdk: Make vmdk_get_cluster_offset() return cluster offset only

2017-07-27 Thread Ashijeet Acharya
: Ashijeet Acharya Reviewed-by: Fam Zheng --- block/vmdk.c | 56 1 file changed, 12 insertions(+), 44 deletions(-) diff --git a/block/vmdk.c b/block/vmdk.c index 4a59ca4..a84b26c 100644 --- a/block/vmdk.c +++ b/block/vmdk.c @@ -1494,25 +1494,16

[Qemu-devel] [PATCH v8 1/8] vmdk: Move vmdk_find_offset_in_cluster() to the top

2017-07-27 Thread Ashijeet Acharya
Move the existing vmdk_find_offset_in_cluster() function to the top of the driver. Signed-off-by: Ashijeet Acharya Reviewed-by: Fam Zheng --- block/vmdk.c | 24 1 file changed, 12 insertions(+), 12 deletions(-) diff --git a/block/vmdk.c b/block/vmdk.c index a9bd22b

[Qemu-devel] [PATCH v8 4/8] vmdk: Factor out metadata loading code out of vmdk_get_cluster_offset()

2017-07-27 Thread Ashijeet Acharya
Move the cluster tables loading code out of the existing vmdk_get_cluster_offset() function and implement it in separate get_cluster_table() and vmdk_l2load() functions. Signed-off-by: Ashijeet Acharya Reviewed-by: Fam Zheng --- block/vmdk.c | 153

[Qemu-devel] [PATCH v8 3/8] vmdk: Rename get_cluster_offset() to vmdk_get_cluster_offset()

2017-07-27 Thread Ashijeet Acharya
Rename the existing get_cluster_offset() to vmdk_get_cluster_offset() and update name in all the callers accordingly. Signed-off-by: Ashijeet Acharya Reviewed-by: Fam Zheng --- block/vmdk.c | 46 +++--- 1 file changed, 23 insertions(+), 23 deletions

[Qemu-devel] [PATCH v8 2/8] vmdk: Rename get_whole_cluster() to vmdk_perform_cow()

2017-07-27 Thread Ashijeet Acharya
Rename the existing function get_whole_cluster() to vmdk_perform_cow() as its sole purpose is to perform COW for the first and the last allocated clusters if needed. Signed-off-by: Ashijeet Acharya Reviewed-by: Fam Zheng --- block/vmdk.c | 23 ++- 1 file changed, 14

[Qemu-devel] [PATCH v7 8/8] vmdk: Make vmdk_get_cluster_offset() return cluster offset only

2017-06-29 Thread Ashijeet Acharya
: Ashijeet Acharya Reviewed-by: Fam Zheng --- block/vmdk.c | 56 1 file changed, 12 insertions(+), 44 deletions(-) diff --git a/block/vmdk.c b/block/vmdk.c index 60b8adc..d41fde9 100644 --- a/block/vmdk.c +++ b/block/vmdk.c @@ -1493,25 +1493,16

[Qemu-devel] [PATCH v7 5/8] vmdk: Set maximum bytes allocated in one cycle

2017-06-29 Thread Ashijeet Acharya
Set the maximum bytes allowed to get allocated at once to be not more than the extent size boundary to handle writes at two separate extents appropriately. Signed-off-by: Ashijeet Acharya Reviewed-by: Fam Zheng --- block/vmdk.c | 13 +++-- 1 file changed, 11 insertions(+), 2 deletions

[Qemu-devel] [PATCH v7 4/8] vmdk: Factor out metadata loading code out of vmdk_get_cluster_offset()

2017-06-29 Thread Ashijeet Acharya
Move the cluster tables loading code out of the existing vmdk_get_cluster_offset() function and implement it in separate get_cluster_table() and vmdk_l2load() functions. Signed-off-by: Ashijeet Acharya Reviewed-by: Fam Zheng --- block/vmdk.c | 153

[Qemu-devel] [PATCH v7 6/8] vmdk: New functions to assist allocating multiple clusters

2017-06-29 Thread Ashijeet Acharya
offset of the first of the many newly allocated clusters. Also, provide proper documentation for both. Signed-off-by: Ashijeet Acharya --- block/vmdk.c | 200 --- 1 file changed, 190 insertions(+), 10 deletions(-) diff --git a/block/vmdk.c b

[Qemu-devel] [PATCH v7 3/8] vmdk: Rename get_cluster_offset() to vmdk_get_cluster_offset()

2017-06-29 Thread Ashijeet Acharya
Rename the existing get_cluster_offset() to vmdk_get_cluster_offset() and update name in all the callers accordingly. Signed-off-by: Ashijeet Acharya Reviewed-by: Fam Zheng --- block/vmdk.c | 46 +++--- 1 file changed, 23 insertions(+), 23 deletions

[Qemu-devel] [PATCH v7 2/8] vmdk: Rename get_whole_cluster() to vmdk_perform_cow()

2017-06-29 Thread Ashijeet Acharya
Rename the existing function get_whole_cluster() to vmdk_perform_cow() as its sole purpose is to perform COW for the first and the last allocated clusters if needed. Signed-off-by: Ashijeet Acharya Reviewed-by: Fam Zheng --- block/vmdk.c | 23 ++- 1 file changed, 14

[Qemu-devel] [PATCH v7 7/8] vmdk: Update metadata for multiple clusters

2017-06-29 Thread Ashijeet Acharya
Include a next pointer in VmdkMetaData struct to point to the previous allocated L2 table. Modify vmdk_L2update to start updating metadata for allocation of multiple clusters at once. Signed-off-by: Ashijeet Acharya --- block/vmdk.c | 128

[Qemu-devel] [PATCH v7 0/8] Optimize VMDK I/O by allocating multiple clusters

2017-06-29 Thread Ashijeet Acharya
v1 into 6 readable and sensible patches - include benchmark test results in v2 Ashijeet Acharya (8): vmdk: Move vmdk_find_offset_in_cluster() to the top vmdk: Rename get_whole_cluster() to vmdk_perform_cow() vmdk: Rename get_cluster_offset() to vmdk_get_cluster_offset() vmdk: Factor ou

[Qemu-devel] [PATCH v7 1/8] vmdk: Move vmdk_find_offset_in_cluster() to the top

2017-06-29 Thread Ashijeet Acharya
Move the existing vmdk_find_offset_in_cluster() function to the top of the driver. Signed-off-by: Ashijeet Acharya Reviewed-by: Fam Zheng --- block/vmdk.c | 24 1 file changed, 12 insertions(+), 12 deletions(-) diff --git a/block/vmdk.c b/block/vmdk.c index a9bd22b

Re: [Qemu-devel] [PATCH v6 7/8] vmdk: Update metadata for multiple clusters

2017-06-29 Thread Ashijeet Acharya
On Tue, Jun 27, 2017 at 1:34 PM, Fam Zheng wrote: > On Mon, 06/05 13:22, Ashijeet Acharya wrote: >> @@ -1876,6 +1942,13 @@ static int vmdk_pwritev(BlockDriverState *bs, >> uint64_t offset, >> offset += n_bytes; >> bytes_done += n_bytes; >>

Re: [Qemu-devel] [PATCH v6 6/8] vmdk: New functions to assist allocating multiple clusters

2017-06-29 Thread Ashijeet Acharya
On Tue, Jun 27, 2017 at 1:32 PM, Fam Zheng wrote: > On Mon, 06/05 13:22, Ashijeet Acharya wrote: >> +/* >> + * vmdk_handle_alloc >> + * >> + * Allocate new clusters for an area that either is yet unallocated or >> needs a >> + * copy on write. If *cluste

[Qemu-devel] [PATCH v6 8/8] vmdk: Make vmdk_get_cluster_offset() return cluster offset only

2017-06-05 Thread Ashijeet Acharya
: Ashijeet Acharya --- block/vmdk.c | 56 1 file changed, 12 insertions(+), 44 deletions(-) diff --git a/block/vmdk.c b/block/vmdk.c index 9fa2414..accf1c3 100644 --- a/block/vmdk.c +++ b/block/vmdk.c @@ -1485,25 +1485,16 @@ static int

[Qemu-devel] [PATCH v6 4/8] vmdk: Factor out metadata loading code out of vmdk_get_cluster_offset()

2017-06-05 Thread Ashijeet Acharya
Move the cluster tables loading code out of the existing vmdk_get_cluster_offset() function and implement it in separate get_cluster_table() and vmdk_l2load() functions. Signed-off-by: Ashijeet Acharya Reviewed-by: Fam Zheng --- block/vmdk.c | 153

[Qemu-devel] [PATCH v6 6/8] vmdk: New functions to assist allocating multiple clusters

2017-06-05 Thread Ashijeet Acharya
offset of the first of the many newly allocated clusters. Also, provide proper documentation for both. Signed-off-by: Ashijeet Acharya --- block/vmdk.c | 192 +++ 1 file changed, 182 insertions(+), 10 deletions(-) diff --git a/block/vmdk.c b

[Qemu-devel] [PATCH v6 7/8] vmdk: Update metadata for multiple clusters

2017-06-05 Thread Ashijeet Acharya
Include a next pointer in VmdkMetaData struct to point to the previous allocated L2 table. Modify vmdk_L2update to start updating metadata for allocation of multiple clusters at once. Signed-off-by: Ashijeet Acharya --- block/vmdk.c | 128

[Qemu-devel] [PATCH v6 3/8] vmdk: Rename get_cluster_offset() to vmdk_get_cluster_offset()

2017-06-05 Thread Ashijeet Acharya
Rename the existing get_cluster_offset() to vmdk_get_cluster_offset() and update name in all the callers accordingly. Signed-off-by: Ashijeet Acharya Reviewed-by: Fam Zheng --- block/vmdk.c | 46 +++--- 1 file changed, 23 insertions(+), 23 deletions

[Qemu-devel] [PATCH v6 5/8] vmdk: Set maximum bytes allocated in one cycle

2017-06-05 Thread Ashijeet Acharya
Set the maximum bytes allowed to get allocated at once to be not more than the extent size boundary to handle writes at two separate extents appropriately. Signed-off-by: Ashijeet Acharya Reviewed-by: Fam Zheng --- block/vmdk.c | 13 +++-- 1 file changed, 11 insertions(+), 2 deletions

[Qemu-devel] [PATCH v6 1/8] vmdk: Move vmdk_find_offset_in_cluster() to the top

2017-06-05 Thread Ashijeet Acharya
Move the existing vmdk_find_offset_in_cluster() function to the top of the driver. Signed-off-by: Ashijeet Acharya Reviewed-by: Fam Zheng --- block/vmdk.c | 24 1 file changed, 12 insertions(+), 12 deletions(-) diff --git a/block/vmdk.c b/block/vmdk.c index a9bd22b

[Qemu-devel] [PATCH v6 0/8] Optimize VMDK I/O by allocating multiple clusters

2017-06-05 Thread Ashijeet Acharya
1 in v1 into 6 readable and sensible patches - include benchmark test results in v2 Ashijeet Acharya (8): vmdk: Move vmdk_find_offset_in_cluster() to the top vmdk: Rename get_whole_cluster() to vmdk_perform_cow() vmdk: Rename get_cluster_offset() to vmdk_get_cluster_offset() vmdk: Factor out m

[Qemu-devel] [PATCH v6 2/8] vmdk: Rename get_whole_cluster() to vmdk_perform_cow()

2017-06-05 Thread Ashijeet Acharya
Rename the existing function get_whole_cluster() to vmdk_perform_cow() as its sole purpose is to perform COW for the first and the last allocated clusters if needed. Signed-off-by: Ashijeet Acharya Reviewed-by: Fam Zheng --- block/vmdk.c | 23 ++- 1 file changed, 14

Re: [Qemu-devel] [PATCH v5 6/8] vmdk: New functions to assist allocating multiple clusters

2017-06-04 Thread Ashijeet Acharya
On Mon, Jun 5, 2017 at 11:23 AM, Ashijeet Acharya wrote: > Introduce two new helper functions handle_alloc() and > vmdk_alloc_cluster_offset(). handle_alloc() helps to allocate multiple > clusters at once starting from a given offset on disk and performs COW > if necessary for fi

[Qemu-devel] [PATCH v5 8/8] vmdk: Make vmdk_get_cluster_offset() return cluster offset only

2017-06-04 Thread Ashijeet Acharya
: Ashijeet Acharya --- block/vmdk.c | 56 1 file changed, 12 insertions(+), 44 deletions(-) diff --git a/block/vmdk.c b/block/vmdk.c index 9fa2414..accf1c3 100644 --- a/block/vmdk.c +++ b/block/vmdk.c @@ -1485,25 +1485,16 @@ static int

[Qemu-devel] [PATCH v5 4/8] vmdk: Factor out metadata loading code out of vmdk_get_cluster_offset()

2017-06-04 Thread Ashijeet Acharya
Move the cluster tables loading code out of the existing vmdk_get_cluster_offset() function and implement it in separate get_cluster_table() and vmdk_l2load() functions. Signed-off-by: Ashijeet Acharya Reviewed-by: Fam Zheng --- block/vmdk.c | 153

[Qemu-devel] [PATCH v5 5/8] vmdk: Set maximum bytes allocated in one cycle

2017-06-04 Thread Ashijeet Acharya
Set the maximum bytes allowed to get allocated at once to be not more than the extent size boundary to handle writes at two separate extents appropriately. Signed-off-by: Ashijeet Acharya Reviewed-by: Fam Zheng --- block/vmdk.c | 13 +++-- 1 file changed, 11 insertions(+), 2 deletions

[Qemu-devel] [PATCH v5 7/8] vmdk: Update metadata for multiple clusters

2017-06-04 Thread Ashijeet Acharya
Include a next pointer in VmdkMetaData struct to point to the previous allocated L2 table. Modify vmdk_L2update to start updating metadata for allocation of multiple clusters at once. Signed-off-by: Ashijeet Acharya --- block/vmdk.c | 128

[Qemu-devel] [PATCH v5 3/8] vmdk: Rename get_cluster_offset() to vmdk_get_cluster_offset()

2017-06-04 Thread Ashijeet Acharya
Rename the existing get_cluster_offset() to vmdk_get_cluster_offset() and update name in all the callers accordingly. Signed-off-by: Ashijeet Acharya Reviewed-by: Fam Zheng --- block/vmdk.c | 46 +++--- 1 file changed, 23 insertions(+), 23 deletions

[Qemu-devel] [PATCH v5 1/8] vmdk: Move vmdk_find_offset_in_cluster() to the top

2017-06-04 Thread Ashijeet Acharya
Move the existing vmdk_find_offset_in_cluster() function to the top of the driver. Signed-off-by: Ashijeet Acharya Reviewed-by: Fam Zheng --- block/vmdk.c | 24 1 file changed, 12 insertions(+), 12 deletions(-) diff --git a/block/vmdk.c b/block/vmdk.c index a9bd22b

[Qemu-devel] [PATCH v5 6/8] vmdk: New functions to assist allocating multiple clusters

2017-06-04 Thread Ashijeet Acharya
offset of the first of the many newly allocated clusters. Also, provide proper documentation for both. Signed-off-by: Ashijeet Acharya --- block/vmdk.c | 192 +++ 1 file changed, 182 insertions(+), 10 deletions(-) diff --git a/block/vmdk.c b

[Qemu-devel] [PATCH v5 0/8] Optimize VMDK I/O by allocating multiple clusters

2017-06-04 Thread Ashijeet Acharya
ke them compilable (fam) - call bdrv_pwrite_sync() for batches of atmost 512 clusters at once (fam) Changes in v2: - segregate the ugly Patch 1 in v1 into 6 readable and sensible patches - include benchmark test results in v2 Ashijeet Acharya (8): vmdk: Move vmdk_find_offset_in_cluster() to

[Qemu-devel] [PATCH v5 2/8] vmdk: Rename get_whole_cluster() to vmdk_perform_cow()

2017-06-04 Thread Ashijeet Acharya
Rename the existing function get_whole_cluster() to vmdk_perform_cow() as its sole purpose is to perform COW for the first and the last allocated clusters if needed. Signed-off-by: Ashijeet Acharya Reviewed-by: Fam Zheng --- block/vmdk.c | 23 ++- 1 file changed, 14

Re: [Qemu-devel] [PATCH v4 6/8] vmdk: New functions to assist allocating multiple clusters

2017-06-03 Thread Ashijeet Acharya
On Thu, Jun 1, 2017 at 7:27 PM, Fam Zheng wrote: > On Sat, 04/22 10:43, Ashijeet Acharya wrote: >> Introduce two new helper functions handle_alloc() and >> vmdk_alloc_cluster_offset(). handle_alloc() helps to allocate multiple >> clusters at once starting from a given offset

Re: [Qemu-devel] [PATCH v2 0/7] Refactor DMG driver to have chunk size independence

2017-04-27 Thread Ashijeet Acharya
On Thu, Apr 27, 2017 at 1:36 PM, Ashijeet Acharya wrote: > Previously posted series patches: > v1: http://lists.nongnu.org/archive/html/qemu-devel/2017-04/msg04641.html > > This series helps to provide chunk size independence for DMG driver to prevent > denial-of-service in cases

[Qemu-devel] [PATCH v2 5/7] dmg: Handle bz2 compressed/raw/zeroed chunks

2017-04-27 Thread Ashijeet Acharya
We do not need to cache the access point for these chunks but need to update our various supporting variables like chunk, sectors_read etc. to keep maintaining our code structure. Call cache_access_point() after reading chunks of these types. Signed-off-by: Ashijeet Acharya --- block/dmg.c | 18

[Qemu-devel] [PATCH v2 3/7] dmg: Refactor and prepare dmg_read_chunk() to cache random access points

2017-04-27 Thread Ashijeet Acharya
ces inside dmg_read_chunk() Signed-off-by: Ashijeet Acharya --- block/dmg.c | 159 +++- 1 file changed, 94 insertions(+), 65 deletions(-) diff --git a/block/dmg.c b/block/dmg.c index c6fe8b0..32623e2 100644 --- a/block/dmg.c +++ b/block/dmg.c @@ -

[Qemu-devel] [PATCH v2 2/7] dmg: New function to help us cache random access point

2017-04-27 Thread Ashijeet Acharya
Introduce a new cache_access_point() function which will help us first cache a random access point inside a compressed stream and then keep updating it according to our requirement at appropriate times. Signed-off-by: Ashijeet Acharya --- block/dmg.c | 18 ++ 1 file changed, 18

[Qemu-devel] [PATCH v2 7/7] dmg: Limit the output buffer size to a max of 2MB

2017-04-27 Thread Ashijeet Acharya
is. Remove the error messages to prevent denial-of-service in cases where untrusted files are being accessed by the user. Signed-off-by: Ashijeet Acharya --- block/dmg.c | 24 +--- 1 file changed, 1 insertion(+), 23 deletions(-) diff --git a/block/dmg.c b/block/dmg.c index 8

[Qemu-devel] [PATCH v2 1/7] dmg: Introduce a new struct to cache random access points

2017-04-27 Thread Ashijeet Acharya
We need to cache the random access point while performing partial decompression so that we can resume decompression from that point onwards in our next sequential read request. Introduce a new struct DMGReadState which will help us do this. Signed-off-by: Ashijeet Acharya --- block/dmg.h | 10

[Qemu-devel] [PATCH v2 6/7] dmg: Refactor dmg_co_preadv() to start reading multiple sectors

2017-04-27 Thread Ashijeet Acharya
x27;data' variable depending on our cached access point situation to align our read requests appropriately. Signed-off-by: Ashijeet Acharya --- block/dmg.c | 23 +++ 1 file changed, 15 insertions(+), 8 deletions(-) diff --git a/block/dmg.c b/block/dmg.c index f9045f9..8b7

[Qemu-devel] [PATCH v2 4/7] dmg: Handle zlib compressed chunks

2017-04-27 Thread Ashijeet Acharya
Set the output buffer size to be equal to the size of number of sectors stored in @sectors_read. Start inflating to a max output buffer size of 2MB and cache our access point to aid random access later if required. Signed-off-by: Ashijeet Acharya --- block/dmg.c | 48

[Qemu-devel] [PATCH v2 0/7] Refactor DMG driver to have chunk size independence

2017-04-27 Thread Ashijeet Acharya
s in v2: - limit the buffer size to 2MB after fixing the buffering problems (john/fam) Ashijeet Acharya (7): dmg: Introduce a new struct to cache random access points dmg: New function to help us cache random access point dmg: Refactor and prepare dmg_read_chunk() to cache random access points

Re: [Qemu-devel] [PATCH v1 3/8] dmg: Limit the output buffer size to a max of 2MB

2017-04-27 Thread Ashijeet Acharya
On Thu, Apr 27, 2017 at 12:56 PM, Fam Zheng wrote: > On Wed, 04/26 17:30, John Snow wrote: >> Seems OK otherwise, but I would normally expect you to fix the buffering >> problems first, and then reduce the size of the buffer -- not the other >> way around. This version introduces new limitations t

Re: [Qemu-devel] [PATCH v1 3/8] dmg: Limit the output buffer size to a max of 2MB

2017-04-27 Thread Ashijeet Acharya
On Thu, Apr 27, 2017 at 3:00 AM, John Snow wrote: > > > On 04/25/2017 03:59 PM, Ashijeet Acharya wrote: >> The size of the output buffer is limited to a maximum of 2MB so that >> QEMU doesn't end up allocating huge amounts of memory while >> decompressing compresse

[Qemu-devel] [PATCH v1 7/8] dmg: Refactor dmg_co_preadv() to start reading multiple sectors

2017-04-25 Thread Ashijeet Acharya
x27;data' variable depending on our cached access point situation to align our read requests appropriately. Signed-off-by: Ashijeet Acharya --- block/dmg.c | 23 +++ 1 file changed, 15 insertions(+), 8 deletions(-) diff --git a/block/dmg.c b/block/dmg.c index f643e41..b0f

[Qemu-devel] [PATCH v1 6/8] dmg: Handle bz2 compressed/raw/zeroed chunks

2017-04-25 Thread Ashijeet Acharya
We do not need to cache the access point for these chunks but need to update our various supporting variables like chunk, sectors_read etc. to keep maintaining our code structure. Call cache_access_point() after reading chunks of these types. Signed-off-by: Ashijeet Acharya --- block/dmg.c | 18

[Qemu-devel] [PATCH v1 8/8] dmg: Remove the error messages to allow wild images

2017-04-25 Thread Ashijeet Acharya
cases where untrusted files are being accessed by the user. Signed-off-by: Ashijeet Acharya --- block/dmg.c | 22 -- 1 file changed, 22 deletions(-) diff --git a/block/dmg.c b/block/dmg.c index b0f3c84..01ec40e 100644 --- a/block/dmg.c +++ b/block/dmg.c @@ -209,7 +209,6

[Qemu-devel] [PATCH v1 5/8] dmg: Handle zlib compressed chunks

2017-04-25 Thread Ashijeet Acharya
Set the output buffer size to be equal to the size of number of sectors stored in @sectors_read. Start inflating to a max output buffer size of 2MB and cache our access point to aid random access later if required. Signed-off-by: Ashijeet Acharya --- block/dmg.c | 48

[Qemu-devel] [PATCH v1 3/8] dmg: Limit the output buffer size to a max of 2MB

2017-04-25 Thread Ashijeet Acharya
e for DMG files is to be compatible with qemu-img convert. Signed-off-by: Ashijeet Acharya --- block/dmg.c | 12 ++-- 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/block/dmg.c b/block/dmg.c index c6fe8b0..7ae30e3 100644 --- a/block/dmg.c +++ b/block/dmg.c @@ -37,

[Qemu-devel] [PATCH v1 2/8] dmg: New function to help us cache random access point

2017-04-25 Thread Ashijeet Acharya
Introduce a new cache_access_point() function which will help us first cache a random access point inside a compressed stream and then keep updating it according to our requirement at appropriate times. Signed-off-by: Ashijeet Acharya --- block/dmg.c | 18 ++ 1 file changed, 18

[Qemu-devel] [PATCH v1 1/8] dmg: Introduce a new struct to cache random access points

2017-04-25 Thread Ashijeet Acharya
We need to cache the random access point while performing partial decompression so that we can resume decompression from that point onwards in our next sequential read request. Introduce a new struct DMGReadState which will help us do this. Signed-off-by: Ashijeet Acharya --- block/dmg.h | 10

[Qemu-devel] [PATCH v1 4/8] dmg: Refactor and prepare dmg_read_chunk() to cache random access points

2017-04-25 Thread Ashijeet Acharya
Refactor dmg_read_chunk() to start making use of the new DMGReadState structure and do chunk and sector related calculations based on it. Add a new argument "DMGReadState *drs" to it. Signed-off-by: Ashijeet Acharya --- block/d

[Qemu-devel] [PATCH v1 0/8] Refactor DMG driver to have chunk size independence

2017-04-25 Thread Ashijeet Acharya
ccess-to-bzip2.html [3] http://linux.math.tifr.res.in/manuals/html/manual_3.html#SEC17 Special thanks to Peter Wu for helping me understand and tackle the bz2 compressed chunks. Ashijeet Acharya (8): dmg: Introduce a new struct to cache random access points dmg: New function to help us cache r

Re: [Qemu-devel] DMG chunk size independence

2017-04-25 Thread Ashijeet Acharya
Hi, cc'ing Peter Wu in... Currently I have completed the task for zlib, uncompressed and zeroed chunks in a DMG file using the approach we discussed earlier. Unfortunately, this approach is not appropriate for bz2 chunks since we cannot restart our decompression from the access point we cached si

Re: [Qemu-devel] DMG chunk size independence

2017-04-24 Thread Ashijeet Acharya
>> For testing I am first converting the images to raw format and then >> comparing the resulting image with the one converted using v2.9.0 DMG >> driver and after battling for 2 days with my code, it finally prints >> "Images are identical." According to John, that should be pretty >> conclusive a

Re: [Qemu-devel] DMG chunk size independence

2017-04-23 Thread Ashijeet Acharya
Hi, Great news! I have almost completed this task and the results are looking promising. I have not yet attended to the DMG files having bz2 compressed chunks but that should be easy and pretty similar to my approach for zlib compressed files. So, no worries there. For testing I am first converti

[Qemu-devel] [PATCH v4 7/8] vmdk: Update metadata for multiple clusters

2017-04-21 Thread Ashijeet Acharya
Include a next pointer in VmdkMetaData struct to point to the previous allocated L2 table. Modify vmdk_L2update to start updating metadata for allocation of multiple clusters at once. Signed-off-by: Ashijeet Acharya --- block/vmdk.c | 129

[Qemu-devel] [PATCH v4 8/8] vmdk: Make vmdk_get_cluster_offset() return cluster offset only

2017-04-21 Thread Ashijeet Acharya
: Ashijeet Acharya --- block/vmdk.c | 56 1 file changed, 12 insertions(+), 44 deletions(-) diff --git a/block/vmdk.c b/block/vmdk.c index e52c373..be08bde 100644 --- a/block/vmdk.c +++ b/block/vmdk.c @@ -1486,25 +1486,16 @@ static int

[Qemu-devel] [PATCH v4 6/8] vmdk: New functions to assist allocating multiple clusters

2017-04-21 Thread Ashijeet Acharya
offset of the first of the many newly allocated clusters. Also, provide proper documentation for both. Signed-off-by: Ashijeet Acharya --- block/vmdk.c | 192 +++ 1 file changed, 182 insertions(+), 10 deletions(-) diff --git a/block/vmdk.c b

[Qemu-devel] [PATCH v4 5/8] vmdk: Set maximum bytes allocated in one cycle

2017-04-21 Thread Ashijeet Acharya
Set the maximum bytes allowed to get allocated at once to be not more than the extent size boundary to handle writes at two separate extents appropriately. Signed-off-by: Ashijeet Acharya --- block/vmdk.c | 13 +++-- 1 file changed, 11 insertions(+), 2 deletions(-) diff --git a/block

[Qemu-devel] [PATCH v4 4/8] vmdk: Factor out metadata loading code out of vmdk_get_cluster_offset()

2017-04-21 Thread Ashijeet Acharya
Move the cluster tables loading code out of the existing vmdk_get_cluster_offset() function and implement it in separate get_cluster_table() and vmdk_L2load() functions. This patch will help us avoid code duplication in future patches of this series. Signed-off-by: Ashijeet Acharya --- block

[Qemu-devel] [PATCH v4 1/8] vmdk: Move vmdk_find_offset_in_cluster() to the top

2017-04-21 Thread Ashijeet Acharya
Move the existing vmdk_find_offset_in_cluster() function to the top of the driver. Signed-off-by: Ashijeet Acharya Reviewed-by: Fam Zheng --- block/vmdk.c | 24 1 file changed, 12 insertions(+), 12 deletions(-) diff --git a/block/vmdk.c b/block/vmdk.c index a9bd22b

[Qemu-devel] [PATCH v4 2/8] vmdk: Rename get_whole_cluster() to vmdk_perform_cow()

2017-04-21 Thread Ashijeet Acharya
Rename the existing function get_whole_cluster() to vmdk_perform_cow() as its sole purpose is to perform COW for the first and the last allocated clusters if needed. Signed-off-by: Ashijeet Acharya Reviewed-by: Fam Zheng --- block/vmdk.c | 23 ++- 1 file changed, 14

[Qemu-devel] [PATCH v4 0/8] Optimize VMDK I/O by allocating multiple clusters

2017-04-21 Thread Ashijeet Acharya
ompilable (fam) - call bdrv_pwrite_sync() for batches of atmost 512 clusters at once (fam) Changes in v2: - segregate the ugly Patch 1 in v1 into 6 readable and sensible patches - include benchmark test results in v2 Ashijeet Acharya (8): vmdk: Move vmdk_find_offset_in_cluster() to the top vmdk: Rename

[Qemu-devel] [PATCH v4 3/8] vmdk: Rename get_cluster_offset() to vmdk_get_cluster_offset()

2017-04-21 Thread Ashijeet Acharya
Rename the existing get_cluster_offset() to vmdk_get_cluster_offset() and update name in all the callers accordingly. Signed-off-by: Ashijeet Acharya --- block/vmdk.c | 46 +++--- 1 file changed, 23 insertions(+), 23 deletions(-) diff --git a/block

Re: [Qemu-devel] [PATCH v3 5/6] vmdk: Set maximum bytes allocated in one cycle

2017-04-21 Thread Ashijeet Acharya
On Fri, Apr 21, 2017 at 8:23 PM, Ashijeet Acharya wrote: > On Wed, Apr 19, 2017 at 6:30 PM, Fam Zheng wrote: >> On Sat, 04/01 20:14, Ashijeet Acharya wrote: >>> Set the maximum bytes allowed to get allocated at once to be not more >>> than the extent size bound

Re: [Qemu-devel] [PATCH v3 6/6] vmdk: Update metadata for multiple clusters

2017-04-21 Thread Ashijeet Acharya
On Fri, Apr 21, 2017 at 1:45 PM, Fam Zheng wrote: > On Sat, 04/01 20:14, Ashijeet Acharya wrote: >> Include a next pointer in VmdkMetaData struct to point to the previous >> allocated L2 table. Modify vmdk_L2update to start updating metadata for >> allocation of mult

Re: [Qemu-devel] [PATCH v3 5/6] vmdk: Set maximum bytes allocated in one cycle

2017-04-21 Thread Ashijeet Acharya
On Wed, Apr 19, 2017 at 6:30 PM, Fam Zheng wrote: > On Sat, 04/01 20:14, Ashijeet Acharya wrote: >> Set the maximum bytes allowed to get allocated at once to be not more >> than the extent size boundary to handle writes at two separate extents >> appropriately. >>

Re: [Qemu-devel] [PATCH v3 4/6] vmdk: Rename get_cluster_offset() to vmdk_get_cluster_offset()

2017-04-19 Thread Ashijeet Acharya
On Wed, Apr 19, 2017 at 18:27 Fam Zheng wrote: > On Sat, 04/01 20:14, Ashijeet Acharya wrote: > > Rename the existing get_cluster_offset() function to > > vmdk_get_cluster_offset() and have it make use of the new > > get_cluster_table() to load the cluster tables. Also, it

Re: [Qemu-devel] [PATCH v3 3/6] vmdk: New functions to assist allocating multiple clusters

2017-04-19 Thread Ashijeet Acharya
On Wed, Apr 19, 2017 at 18:26 Fam Zheng wrote: > On Sat, 04/01 20:14, Ashijeet Acharya wrote: > > Move the cluster tables loading code out of the existing > > get_cluster_offset() function to avoid code duplication and implement it > > in separate get_cluster_table() and vm

Re: [Qemu-devel] DMG chunk size independence

2017-04-18 Thread Ashijeet Acharya
On Tue, Apr 18, 2017 at 10:35 PM, John Snow wrote: > >> >> I could be wrong, but that's my hunch. >> >> Maybe you can cache the state of the INFLATE process such that once you >> fill the cache with data, we can simply resume the INFLATE procedure >> when the guest almost inevitabl

Re: [Qemu-devel] DMG chunk size independence

2017-04-18 Thread Ashijeet Acharya
On Tue, Apr 18, 2017 at 01:59 John Snow wrote: > > > On 04/15/2017 04:38 AM, Ashijeet Acharya wrote: > > Hi, > > > > Some of you are already aware but for the benefit of the open list, > > this mail is regarding the task mentioned > > Here -&g

[Qemu-devel] DMG chunk size independence

2017-04-15 Thread Ashijeet Acharya
Hi, Some of you are already aware but for the benefit of the open list, this mail is regarding the task mentioned Here -> http://wiki.qemu-project.org/ToDo/Block/DmgChunkSizeIndependence I had a chat with Fam regarding this and he suggested a solution where we fix the output buffer size to a max

Re: [Qemu-devel] [PATCH v3 1/6] vmdk: Move vmdk_find_offset_in_cluster() to the top

2017-04-10 Thread Ashijeet Acharya
On Sat, Apr 1, 2017 at 8:14 PM, Ashijeet Acharya wrote: > Move the existing vmdk_find_offset_in_cluster() function to the top of > the driver. Also, introduce a new helper function size_to_clusters() > which returns the number of clusters for a given size in bytes. Here, > we le

[Qemu-devel] [PATCH v3 6/6] vmdk: Update metadata for multiple clusters

2017-04-01 Thread Ashijeet Acharya
Include a next pointer in VmdkMetaData struct to point to the previous allocated L2 table. Modify vmdk_L2update to start updating metadata for allocation of multiple clusters at once. Signed-off-by: Ashijeet Acharya --- block/vmdk.c | 136

[Qemu-devel] [PATCH v3 5/6] vmdk: Set maximum bytes allocated in one cycle

2017-04-01 Thread Ashijeet Acharya
Set the maximum bytes allowed to get allocated at once to be not more than the extent size boundary to handle writes at two separate extents appropriately. Signed-off-by: Ashijeet Acharya --- block/vmdk.c | 13 +++-- 1 file changed, 11 insertions(+), 2 deletions(-) diff --git a/block

[Qemu-devel] [PATCH v3 1/6] vmdk: Move vmdk_find_offset_in_cluster() to the top

2017-04-01 Thread Ashijeet Acharya
: Ashijeet Acharya --- block/vmdk.c | 24 1 file changed, 12 insertions(+), 12 deletions(-) diff --git a/block/vmdk.c b/block/vmdk.c index a9bd22b..22be887 100644 --- a/block/vmdk.c +++ b/block/vmdk.c @@ -242,6 +242,18 @@ static void vmdk_free_last_extent(BlockDriverState *bs

  1   2   3   4   >