Ran into an interesting problem with the sparc64 pmap bootstrapping code. Early on, we ask the firmware what physical memory is available. Later we use this memory to set up the kernel page tables, kernel stack and per-cpu data structures. We explicitly tell the firmware about the mappings of these data structure as the firmware is handling page faults for us at this stage. To store these mappings the firmware may need to allocate more memory. And if it happens to allocate memory that we're using for some other purpose, bad things will happen. In my case dmesg stopped working because its mappings were messed up.
The following diff attempts to fix this issue by telling the firmware which pages we're stealing. It's not perfect as it doesn't prevent us from allocating the same pages as the firmware is allocating. Tests on a wide variety of sparc64 hardware would be welcome. Index: pmap.c =================================================================== RCS file: /cvs/src/sys/arch/sparc64/sparc64/pmap.c,v retrieving revision 1.96 diff -u -p -r1.96 pmap.c --- pmap.c 27 Nov 2015 15:34:01 -0000 1.96 +++ pmap.c 17 Apr 2016 19:17:45 -0000 @@ -2869,6 +2869,7 @@ pmap_get_page(paddr_t *pa, const char *w *pa = VM_PAGE_TO_PHYS(pg); } else { uvm_page_physget(pa); + prom_claim_phys(*pa, PAGE_SIZE); pmap_zero_phys(*pa); }