[PATCH] migration/dirtyrate: use QEMU_CLOCK_HOST to report start-time

2023-09-05 Thread Andrei Gudkov via
Currently query-dirty-rate uses QEMU_CLOCK_REALTIME as the source for start-time field. This translates to clock_gettime(CLOCK_MONOTONIC), i.e. number of seconds since host boot. This is not very useful. The only reasonable use case of start-time I can imagine is to check whether previously complet

[PATCH v3] migration/calc-dirty-rate: millisecond-granularity period

2023-09-05 Thread Andrei Gudkov via
This patch allows to measure dirty page rate for sub-second intervals of time. An optional argument is introduced -- calc-time-unit. For example: {"execute": "calc-dirty-rate", "arguments": {"calc-time": 500, "calc-time-unit": "millisecond"} } Millisecond granularity allows to make predictions w

[PATCH v2] migration/calc-dirty-rate: millisecond-granularity period

2023-08-04 Thread Andrei Gudkov via
Introduces alternative argument calc-time-ms, which is the the same as calc-time but accepts millisecond value. Millisecond granularity allows to make predictions whether migration will succeed or not. To do this, calculate dirty rate with calc-time-ms set to max allowed downtime, convert measured

[PATCH] migration/calc-dirty-rate: millisecond precision period

2023-06-29 Thread Andrei Gudkov via
Introduces alternative argument calc-time-ms, which is the the same as calc-time but accepts millisecond value. Millisecond precision allows to make predictions whether migration will succeed or not. To do this, calculate dirty rate with calc-time-ms set to max allowed downtime, convert measured ra

[PATCH v2] qmp: Report page size in query-memory-size-summary

2023-06-02 Thread Andrei Gudkov via
Some commands (query-migrate and calc-dirty-rate) report values in units of pages. However, currently the only place where we can get page size is through query-migrate and only after migration has started. query-memory-size-summary seems like an appropritate place where it should be reported inste

[PATCH] qapi: better docs for calc-dirty-rate and friends

2023-05-23 Thread Andrei Gudkov via
Rewrote calc-dirty-rate documentation. Briefly described different modes of dirty page rate measurement. Added some examples. Fixed obvious grammar errors. Signed-off-by: Andrei Gudkov --- qapi/migration.json | 107 +++- 1 file changed, 77 insertions(+), 3

[PATCH] [query-memory-size-summary] Report page size

2023-05-19 Thread Andrei Gudkov via
Some commands (query-migrate and calc-dirty-rate) report values in units of pages. However, currently the only place where we can get page size is through query-migrate and only after migration has started. query-memory-size-summary seems like an appropritate place where it should be reported inste

[PATCH v2 3/4] migration/calc-dirty-rate: added n-zero-pages metric

2023-04-27 Thread Andrei Gudkov via
In sampling mode, a new metric is collected and reported: number of pages entirely filled with zeroes. Signed-off-by: Andrei Gudkov --- migration/dirtyrate.c | 40 +++- migration/dirtyrate.h | 1 + qapi/migration.json | 4 3 files changed, 40 insertio

[PATCH v2 1/4] migration/calc-dirty-rate: replaced CRC32 with xxHash

2023-04-27 Thread Andrei Gudkov via
This significantly reduces overhead of dirty page rate calculation in sampling mode. Tested using 32GiB VM on E5-2690 CPU. With CRC32: total_pages=8388608 sampled_pages=16384 millis=71 With xxHash: total_pages=8388608 sampled_pages=16384 millis=14 Signed-off-by: Andrei Gudkov --- migration/dir

[PATCH v2 4/4] migration/calc-dirty-rate: tool to predict migration time

2023-04-27 Thread Andrei Gudkov via
Signed-off-by: Andrei Gudkov --- MAINTAINERS | 1 + scripts/predict_migration.py | 283 +++ 2 files changed, 284 insertions(+) create mode 100644 scripts/predict_migration.py diff --git a/MAINTAINERS b/MAINTAINERS index fc225e66df..0c578446cf 1

[PATCH v2 2/4] migration/calc-dirty-rate: detailed stats in sampling mode

2023-04-27 Thread Andrei Gudkov via
Collect number of dirty pages for progresseively increasing time periods starting with 125ms up to number of seconds specified with calc-dirty-rate. Report through qmp and hmp: 1) vector of dirty page measurements, 2) page size, 3) total number of VM pages, 4) number of sampled pages. Signed-off-b

[PATCH v2 0/4] Migration time prediction using calc-dirty-rate

2023-04-27 Thread Andrei Gudkov via
V1 -> V2: - Extracted CRC32->xxHash into separate commit - Extacted @n-zero-samples metric into separate commit - Added description to qapi about connection between @n-dirty-samples and @periods arrays - Added (Since ...) tag to new metrics --- The overall goal of this patch is to be

[PATCH 0/2] Migration time prediction using calc-dirty-rate

2023-02-28 Thread Andrei Gudkov via
The overall goal of this patch is to be able to predict time it would take to migrate VM in precopy mode based on max allowed downtime, network bandwidth, and metrics collected with "calc-dirty-rate". Predictor itself is a simple python script that closely follows iterations of the migration algori

[PATCH 1/2] migration/calc-dirty-rate: new metrics in sampling mode

2023-02-28 Thread Andrei Gudkov via
* Collect number of all-zero pages * Collect vector of number of dirty pages for different time periods * Report total number of pages, number of sampled pages and page size * Replaced CRC32 with xxHash for performance reasons Signed-off-by: Andrei Gudkov --- migration/dirtyrate.c | 219

[PATCH 2/2] migration/calc-dirty-rate: tool to predict migration time

2023-02-28 Thread Andrei Gudkov via
Signed-off-by: Andrei Gudkov --- MAINTAINERS | 1 + scripts/predict_migration.py | 283 +++ 2 files changed, 284 insertions(+) create mode 100644 scripts/predict_migration.py diff --git a/MAINTAINERS b/MAINTAINERS index c6e6549f06..2fb5b6298a 1