On Wed, May 30, 2018 at 4:30 AM, Joel Sherrill <j...@rtems.org> wrote: > > > On Tue, May 29, 2018 at 11:26 AM, Amaan Cheval <amaan.che...@gmail.com> > wrote: >> >> On Tue, May 29, 2018 at 9:28 PM, Amaan Cheval <amaan.che...@gmail.com> >> wrote: >> > Noted, thanks a ton for the details! Unrelated to the topic at hand, >> > but out of interest, is this the only reading material for further >> > details? http://ceur-ws.org/Vol-1697/EWiLi16_12.pdf >> > >> > In brief: My tests for keeping a libfake.a (compiled without -fpic) >> > and a loader.so with a user-appXYZ.c have been successful, but I'm not >> > sure if my assumptions hold for all cases. See the details below for >> > more. >> > >> > Next actions: >> > - Read more of Linkers and Loaders since it seems to be the only >> > detailed resource I've found at this point >> > - Experiment with actually using the existing librtems*.a I've got and >> > making them boot as a PE UEFI application image >> > >> > >> > -------------------------------------------------------------------------------- >> > >> > In more detail: >> > >> > The problem: >> > That UEFI needs a relocatable PE file, i.e. one that can function >> > regardless of the physical address it's loaded at (no virtual >> > addresses that early). >> > To build an ELF of that kind, the resources I've seen all build their >> > source with -fpic, and then use objcopy to convert the ELF into a >> > relocatable PE, with an embedded runtime self-relocator (akin to >> > load-time relocation, if I'm understanding correctly). >> > >> > What Joel suggested seems to be the simplest option - see if not using >> > -fpic for _all_ of RTEMS' build system is fine. I think it might be >> > from some testing, but I'm not sure if this is conclusive since I need >> > to understand the specifics of the entire development process better. >> > >> > So here's my understanding of the situation at the moment: >> > >> > - librtems*.a is made up of object files, compiled without -fpic, and >> > that should be fine because I believe object files will use RIP >> > relative addressing code by default on x64 where it can, and leave the >> > rest for link-time relocations to handle. IF this is true, this works >> > perfectly for us because all memory accesses and jumps/calls are >> > relative. > > > Just to be clear. For Deos, I am compiling all code with -fPIC. This > includes all librtemscpu.a and librtemsbsp.a. When I accidentally > missed an adapter file, that caused an issue. >
Oh my, that's me misreading majorly, sorry about that. I'm still unclear on how this works: > Unless the loader forces something, you can use PIC with no build system > changes. I meant that to build with fpic for x86_64/amd64 and nothing else, we'd end up having to add a lot of special cases within the build system for even the "generic" parts of RTEMS, such as what's in cpukit (all the subdirs except cpukit/score/cpu), right? My concern was that in doing this, the already overcomplicated build system gets more complicated. If y'all don't think that kind of "if amd64, then use -fPIC" logic feels hacky, I'm okay with this approach 100%, and we can work to find a way to make that special-casing as explicit as possible. >> >> > >> > - We can have a loader.c which acts as the core with the efi_main >> > function - compile it with -fpic into loader.so, and then link >> > loader.so, librtems*.a, and user-appXYZ.c together to form a >> > relocatable ELF, then convert it into a PE using objcopy. Note that >> > from what I can tell, the ELF generated from this still has type EXEC, >> > not DYN, according to readelf. >> >> Correction: This was a leftover file that I'd forgotten to take out >> after renaming a target. Sorry about the confusion. Just using the >> "-shared" flag does cause the resulting ELF to be of type DYN. >> >> > >> > The concerns I have are about my assumptions; if GCC generates any >> > code that uses absolute addressing and that is resolved as a link-time >> > relocation, that could be problematic because the final relocatable PE >> > may not match up with the resolved absolute address. > > > The Deos kernel guys had me check readelf on a known good executable > with the ones I was producing. The loadable sections should match up. > For example, on one architecture I missed an alignment in the linkcmds > and on another, an argument hidden in bsp_specs made a section writable > which should have been read-only. Just check what you can with readelf > and objdump section headers. > >> >> > >> > My tests with a fake static archive library, and creating a PE have >> > been successful, but I'm unsure of how to trigger the relocation >> > behavior by the UEFI firmware (i.e. loading the UEFI image at an >> > address other than it's preferred one). One idea is to have a UEFI >> > application image that loads this test UEFI application image through >> > the "LoadImage" function UEFI provides as a service and then to use >> > QEMU's monitor / gdb inspection capabilities to see if the address the >> > image is loaded at genuinely changes. > > > That's the question. Does the executable end up at a reliably known address? > _Usually_, yes. This can't be guaranteed, though, so we definitely need a completely relocatable file if we're going with the option where we create a completely self-contained hello.efi instead of a loader.efi+hello.elf pair method. Joel, if you can spare the time today, should we perhaps chat about this before/after the GSoC IRC meeting later today? I think a quick chat will help resolve this quicker. >> >> > >> > If any of you have any resources, that'd be highly appreciated. Some >> > resources I'm using so far are: >> > - >> > https://eli.thegreenplace.net/2011/11/03/position-independent-code-pic-in-shared-libraries/ >> > - >> > https://eli.thegreenplace.net/2011/08/25/load-time-relocation-of-shared-libraries/ >> > - >> > https://eli.thegreenplace.net/2012/01/03/understanding-the-x64-code-models >> > >> > Sorry about the length of the email! >> > >> > On Fri, May 25, 2018 at 10:51 PM, Joel Sherrill <j...@rtems.org> wrote: >> >> >> >> >> >> On Fri, May 25, 2018, 12:11 PM Amaan Cheval <amaan.che...@gmail.com> >> >> wrote: >> >>> >> >>> Hey! Could you link me to some code that you used for the Deos setup >> >>> you mentioned? >> >>> My understanding is that the -shared option can link static archives >> >>> to create a "shared" library in the sense that it doesn't include the >> >>> usual crt0 runtime environment and whatnot, but the code within is >> >>> still position-dependent. Given that the PE image that EFI needs is >> >>> one that needs to be truly relocatable, this may not work - BUT, I've >> >>> only just noticed the ./gnuefi/reloc_x86_64.c file which seems to >> >>> handle some kinds of runtime relocations encoded within the converted >> >>> PE file, so maybe this will work after all. I'll continue to >> >>> investigate and let you know how it goes! >> >> >> >> >> >> Deos isn't a good example except that you can compile with -fPIC and >> >> put >> >> that code into a static library. Deos is a closed source Level A (man >> >> rated >> >> flight) ARINC 653 RTOS. It's boot process reads configuration >> >> information >> >> about each partition and associates .so's with each address space per >> >> the >> >> configuration. It can't change after that. >> >> >> >> The RTEMS exe is mostly linked as normal except to use some arguments >> >> to say >> >> some symbols are from a shared library. >> >> >> >> The base address of the exe is that of the provided virtual address >> >> space >> >> with .data and .bss in their respective spaces. >> >> >> >> And our entry point is in C so there is no asm before that. Great >> >> simplification. >> >> >> >>> >> >>> Regarding how TLS differs with PIC - could you elaborate? Is it >> >>> something we'll need to solve for if we go with the -fPIC option, or >> >>> is it something we need to keep in mind as a limitation, but isn't >> >>> really a blocker? >> >> >> >> >> >> I don't think PIC changes the TLS mechanism on arm or PowerPC but on >> >> i386, >> >> when not PIC the TLS base is in %gs and it's a subroutine call when >> >> PIC. >> >> Just as well for Deos since they assume an application won't change the >> >> segment register values. >> >> >> >> Other than this one TLS difference, it is a normal exe to me. They just >> >> magically provide their .so's before we run. >> >>> >> >>> >> >>> On Fri, May 25, 2018 at 10:13 PM, Joel Sherrill <j...@rtems.org> >> >>> wrote: >> >>> > >> >>> > >> >>> > On Fri, May 25, 2018, 11:15 AM Amaan Cheval <amaan.che...@gmail.com> >> >>> > wrote: >> >>> >> >> >>> >> Hey! >> >>> >> >> >>> >> Skippable details about how FreeBSD handles the UEFI boot process! >> >>> >> >> >>> >> >> >>> >> >> >>> >> -------------------------------------------------------------------------------- >> >>> >> >> >>> >> Having looked into it a bit more, my understanding of how FreeBSD >> >>> >> handles this process is: >> >>> >> - They build a two-stage bootloader for EFI, called boot1.efi and >> >>> >> loader.efi[1] >> >>> >> - loader.efi is an interactive prompt which may autoboot, or a >> >>> >> "boot >> >>> >> kernelImg" command can be used to load the actual kernel >> >>> >> - The kernel is loaded as an ELF through helper functions. The >> >>> >> command_boot[2] function drives this: >> >>> >> - In brief, through calls go through: >> >>> >> command_boot -> mod_loadkld -> file_load -> >> >>> >> file_formats[i]->l_load (actually the loadfile function in >> >>> >> load_elf.c[3]) >> >>> >> - The loadfile function parses the program and section headers of >> >>> >> the ELF file (through more function detours that are not really >> >>> >> important) >> >>> >> - Once the ELF has been loaded at the correct entry_addr that it >> >>> >> expects to be loaded at in memory, the l_exec[4] function is >> >>> >> called, >> >>> >> which is actually elf64_exec in elf64_freebsd.c[5], at which >> >>> >> hopefully >> >>> >> through trampolining magic, the control flow will transfer to the >> >>> >> kernel or ELF module >> >>> >> >> >>> >> >> >>> >> >> >>> >> >> >>> >> -------------------------------------------------------------------------------- >> >>> >> >> >>> >> What this means for RTEMS if we go with gnu-efi is essentially 2 >> >>> >> options, given that the objcopy method of converting from ELF->PE >> >>> >> requires the ELF to be a position-independent shared library: >> >>> >> >> >>> >> - Using -fPIC to compile all of RTEMS, including the RTEMS user's >> >>> >> application code. This way we'd have librtemsbso.so, >> >>> >> librtemscpu.so, >> >>> >> etc. which would then be linked into user_app.c through -fPIC and >> >>> >> -shared flags still, creating one singular hello.so, which can then >> >>> >> finally be converted into hello.efi and put on a FAT filesystem and >> >>> >> booted. This seems doable, but I'm fairly concerned about it >> >>> >> further >> >>> >> complicating our build system and likely being quite singular in >> >>> >> its >> >>> >> focus on EFI. >> >>> > >> >>> > >> >>> > I'm using PIC on the Deos BSP. RTEMS is still a .a and exes are >> >>> > linked >> >>> > with >> >>> > our static libraries and Deos .so. >> >>> > >> >>> > Unless the loader forces something, you can use PIC with no build >> >>> > system >> >>> > changes. >> >>> > >> >>> > note that thread local storage is different on i386 with and without >> >>> > PIC. >> >>> >> >> >>> >> >> >>> >> - The FreeBSD way of a (loader.efi) and a hello.exe (ELF64) put on >> >>> >> possibly the same partition on the FAT filesystem required for UEFI >> >>> >> application images anyway. The loader.efi can find the hello.exe >> >>> >> file >> >>> >> through perhaps a config file it can read or by having a magic-name >> >>> >> like rtems.exe or something. This effectively means we need an ELF >> >>> >> dynamic linker / loader (akin to ld.so) within RTEMS' source. I >> >>> >> think >> >>> >> using FreeBSD's code for this should be fine. One added benefit of >> >>> >> this method is that librtems* and user applications remain as >> >>> >> ELF64s, >> >>> >> which in the future could also be used with Multiboot with a >> >>> >> slightly >> >>> >> modified "loader" (i.e. one which generates the apt Multiboot magic >> >>> >> header, and boots the PC from 32-bit protected mode to 64-bit long >> >>> >> mode). >> >>> >> >> >>> >> I prefer the latter approach personally. If both of these seem too >> >>> >> complicated, we can of course go back to considering generating the >> >>> >> PE >> >>> >> header format in ASM the way Linux distros use EFISTUB and the code >> >>> >> Chris shared (as I mentioned in my original blog post) for wimboot. >> >>> >> Those approaches may be significantly simpler in a sense, but may >> >>> >> limit how we use UEFI Services - I'm not sure about the details of >> >>> >> this yet - I can investigate if y'all aren't fond of the option I >> >>> >> laid >> >>> >> down above. >> >>> >> >> >>> >> Let me know! >> >>> >> >> >>> >> [1] >> >>> >> >> >>> >> >> >>> >> https://www.freebsd.org/cgi/man.cgi?query=loader&apropos=0&sektion=8&manpath=FreeBSD+11.1-RELEASE+and+Ports&arch=default&format=html >> >>> >> [2] >> >>> >> >> >>> >> >> >>> >> https://github.com/freebsd/freebsd/blob/433bd38e3a0349f9f89f9d54594172c75b002b74/stand/common/boot.c#L53 >> >>> >> [3] >> >>> >> >> >>> >> >> >>> >> https://github.com/freebsd/freebsd/blob/d8596f6f687a64b994b065f3058155405dfc39db/stand/common/load_elf.c#L150 >> >>> >> [4] >> >>> >> >> >>> >> >> >>> >> https://github.com/freebsd/freebsd/blob/433bd38e3a0349f9f89f9d54594172c75b002b74/stand/common/boot.c#L107 >> >>> >> [5] >> >>> >> >> >>> >> >> >>> >> https://github.com/freebsd/freebsd/blob/d8596f6f687a64b994b065f3058155405dfc39db/stand/efi/loader/arch/amd64/elf64_freebsd.c#L93 >> >>> >> >> >>> >> On Sun, May 20, 2018 at 10:52 PM, Joel Sherrill <j...@rtems.org> >> >>> >> wrote: >> >>> >> > >> >>> >> > >> >>> >> > On Sun, May 20, 2018, 12:10 PM Amaan Cheval >> >>> >> > <amaan.che...@gmail.com> >> >>> >> > wrote: >> >>> >> >> >> >>> >> >> On Sat, May 19, 2018 at 6:51 PM, Gedare Bloom <ged...@rtems.org> >> >>> >> >> wrote: >> >>> >> >> > On Fri, May 18, 2018 at 5:53 PM, Joel Sherrill >> >>> >> >> > <j...@rtems.org> >> >>> >> >> > wrote: >> >>> >> >> >> >> >>> >> >> >> >> >>> >> >> >> On Fri, May 18, 2018 at 3:24 PM, Amaan Cheval >> >>> >> >> >> <amaan.che...@gmail.com> >> >>> >> >> >> wrote: >> >>> >> >> >>> >> >>> >> >> >>> Hi everyone! >> >>> >> >> >>> >> >>> >> >> >>> I've written a quick blog post summarizing the options I've >> >>> >> >> >>> considered >> >>> >> >> >>> to make the x86_64 port work with UEFI firmware - the >> >>> >> >> >>> primary >> >>> >> >> >>> winner >> >>> >> >> >>> seems to be in my eyes to use "gnu-efi" and to add support >> >>> >> >> >>> for >> >>> >> >> >>> the >> >>> >> >> >>> target "pei-x86-64" (aliased to "efi-app-x86_64") to >> >>> >> >> >>> "x86_64-rtems5-objcopy" in binutils. I've submitted a patch >> >>> >> >> >>> for >> >>> >> >> >>> this >> >>> >> >> >>> here[1]. >> >>> >> >> >> >> >>> >> >> >> >> >>> >> >> >> That patch is quite simple so shouldn't be a problem if this >> >>> >> >> >> is >> >>> >> >> >> the >> >>> >> >> >> direction >> >>> >> >> >> that gets consensus. >> >>> >> >> >>> >> >>> >> >> >>> >> >>> >> >> >>> The blog post is here: >> >>> >> >> >>> https://blog.whatthedude.com/post/uefi-app-options/ >> >>> >> >> >>> >> >>> >> >> >>> I'd appreciate all feedback (and please do let me know if I >> >>> >> >> >>> haven't >> >>> >> >> >>> provided enough context)! >> >>> >> >> >>> >> >>> >> >> >>> Specifically, some concerns I'd like to discuss are: >> >>> >> >> >>> >> >>> >> >> >>> - Does everyone agree with me on choosing gnu-efi + objcopy >> >>> >> >> >>> as >> >>> >> >> >>> our >> >>> >> >> >>> method of choice? >> >>> >> >> >> >> >>> >> >> >> >> >>> >> >> >> Does using gnu-efi add code that runs on the target? Can you >> >>> >> >> >> point >> >>> >> >> >> us to the files, if so. >> >>> >> >> >> >>> >> >> Sure. The files would run on the target, yes. These are the ones >> >>> >> >> listed here (as linked to in my blog post, perhaps without >> >>> >> >> sufficient >> >>> >> >> emphasis): >> >>> >> >> https://wiki.osdev.org/UEFI#Developing_with_GNU-EFI >> >>> >> >> >> >>> >> >> >> >> >>> >> >> >> Can you tell which approach FreeBSD takes? >> >>> >> >> >> >>> >> >> FreeBSD takes the gnu-efi approach I see as the "winner" here >> >>> >> >> (also >> >>> >> >> a >> >>> >> >> link in the post): >> >>> >> >> >> >>> >> >> >> >>> >> >> >> >>> >> >> >> >>> >> >> https://github.com/freebsd/freebsd/blob/996b0b6d81cf31cd8d58af5d8b45f0b4945d960d/stand/efi/loader/Makefile#L98-L1 >> >>> >> > >> >>> >> > >> >>> >> > This is (no surprise) appropriately licensed and IMO the winning >> >>> >> > solution. >> >>> >> > Knowing it is what FreeBSD does makes it an easy choice. >> >>> >> > >> >>> >> > A comment in the readme mentions there is a i386 version of this >> >>> >> > code >> >>> >> > so >> >>> >> > that could be used to let pc386 boot from UEFI. >> >>> >> > >> >>> >> >> >> >>> >> >> >> >>> >> >> >> >> >>> >> >> >>> >> >>> >> >> >>> - How do we integrate gnu-efi into our build process? A part >> >>> >> >> >>> of >> >>> >> >> >>> the >> >>> >> >> >>> RSB, making sure the path to the libraries are in an >> >>> >> >> >>> exported >> >>> >> >> >>> variable? Or perhaps a part of the RTEMS kernel itself if >> >>> >> >> >>> the >> >>> >> >> >>> licenses >> >>> >> >> >>> are compatible (I don't see any on the project[2], only >> >>> >> >> >>> copyright >> >>> >> >> >>> notices within the source files of the release versions). >> >>> >> >> >> >> >>> >> >> >> >> >>> >> >> >> GNU-efi would be built like qemu or the device tree compiler >> >>> >> >> >> would >> >>> >> >> >> be my guess and x86_64-rtems toolset might add that to the >> >>> >> >> >> standard >> >>> >> >> >> set of tools. License on host tools being GPL isn't an issue. >> >>> >> >> >> >> >>> >> >> > >> >>> >> >> > It appears to be a standard 2-clause BSD released by Intel as >> >>> >> >> > specified in the README file of gnu-efi. >> >>> >> >> > >> >>> >> >> >> >> >>> >> >> >>> >> >>> >> >> >>> - Regardless of how we manage UEFI, do we require Multiboot >> >>> >> >> >>> support >> >>> >> >> >>> too? Multiboot drops us in a 32-bit protected mode >> >>> >> >> >>> environment, >> >>> >> >> >>> whereas 64-bit UEFI firmware will boot us into 64-bit long >> >>> >> >> >>> mode >> >>> >> >> >>> - >> >>> >> >> >>> this >> >>> >> >> >>> would mean the kernel would need to support separate >> >>> >> >> >>> code-paths >> >>> >> >> >>> for >> >>> >> >> >>> the 2 if we want to support both methods. >> >>> >> >> >> >> >>> >> >> >> >> >>> >> >> >> That's a good question. For GSoC, I think UEFI is fine and >> >>> >> >> >> perhaps a >> >>> >> >> >> ticket >> >>> >> >> >> under the general "modern PC support" ticket for multiboot >> >>> >> >> >> support. >> >>> >> >> >> Unless >> >>> >> >> >> that eliminates a LOT of PCs. >> >>> >> >> >> >> >>> >> >> >> I don't want you to spend all summer getting an image to boot >> >>> >> >> >> both >> >>> >> >> >> ways. Personally, I want you to have a working BSP one way. >> >>> >> >> >> :) >> >>> >> >> > +1 >> >>> >> >> > >> >>> >> >> >> >>> >> >> Noted, thanks! >> >>> >> >> >> >>> >> >> >>> >> >>> >> >> >>> >> >>> >> >> >>> [1] >> >>> >> >> >>> https://www.sourceware.org/ml/binutils/2018-05/msg00197.html >> >>> >> >> >>> [2] https://sourceforge.net/projects/gnu-efi/ >> >>> >> >> >> >> >>> >> >> >> >> >>> >> >> >> --joel >> >>> >> >> >> >> >>> >> >> >> _______________________________________________ >> >>> >> >> >> devel mailing list >> >>> >> >> >> devel@rtems.org >> >>> >> >> >> http://lists.rtems.org/mailman/listinfo/devel > > _______________________________________________ devel mailing list devel@rtems.org http://lists.rtems.org/mailman/listinfo/devel
