On 06.12.2017 13:06, Christian Borntraeger wrote: > > > On 12/06/2017 01:04 PM, David Hildenbrand wrote: >> On 06.12.2017 13:00, Christian Borntraeger wrote: >>> KVM does not allow memory regions > KVM_MEM_MAX_NR_PAGES, basically >>> limiting the memory per slot to 7.999TB. Lets split the base memory >>> into 4TB chunks to allow go beyond 8TB for a guest. With that (and >>> optimistic overcommitment in the kernel) I was able to start a 59TB >>> guest on a 1TB system. >>> >>> Signed-off-by: Christian Borntraeger <[email protected]> >>> --- >>> hw/s390x/s390-virtio-ccw.c | 20 +++++++++++++++++--- >>> 1 file changed, 17 insertions(+), 3 deletions(-) >>> >>> diff --git a/hw/s390x/s390-virtio-ccw.c b/hw/s390x/s390-virtio-ccw.c >>> index 8425534..6735bbb 100644 >>> --- a/hw/s390x/s390-virtio-ccw.c >>> +++ b/hw/s390x/s390-virtio-ccw.c >>> @@ -157,11 +157,25 @@ static void virtio_ccw_register_hcalls(void) >>> static void s390_memory_init(ram_addr_t mem_size) >>> { >>> MemoryRegion *sysmem = get_system_memory(); >>> - MemoryRegion *ram = g_new(MemoryRegion, 1); >>> + ram_addr_t chunk, offset; >>> + gchar *name; >>> >>> /* allocate RAM for core */ >>> - memory_region_allocate_system_memory(ram, NULL, "s390.ram", mem_size); >>> - memory_region_add_subregion(sysmem, 0, ram); >>> + offset = 0; >>> + while (mem_size) { >>> + MemoryRegion *ram = g_new(MemoryRegion, 1); >>> + chunk = mem_size; >>> + /* KVM does not allow memslots >= 8 TB. Lets split into 4TB chunks >>> */> + if (chunk > 4UL * 1024 * 1024 * 1024 * 1024) { >>> + chunk = 4UL * 1024 * 1024 * 1024 * 1024; >>> + } >>> + name = g_strdup_printf("s390.ram[0x%lx]", offset); >>> + memory_region_allocate_system_memory(ram, NULL, name, chunk); >>> + memory_region_add_subregion(sysmem, offset, ram); >>> + mem_size -= chunk; >>> + offset += chunk; >>> + g_free(name); >>> + } >>> >>> /* Initialize storage key device */ >>> s390_skeys_init(); >>> >> >> This will most certainly break migration, no? > > Probably yes. Thats why the patch has RFC. I was looking for ideas.
As other architectures might also run into this problem, wonder if a) bumping up KVM_MEM_MAX_NR_PAGES makes sense. b) as I said, transparently handle it in kvm slot handling code. >> >> 1. I remember the name being used for migration, I might be wrong. >> 2. Migration of guests > 4TB is certainly broken ;) >> >> I wonder if this should rather be handled in the kvm_slot code. >> (silently create an manage two slots) > -- Thanks, David / dhildenb
