On 23/10/2018 01:18, Joel Sherrill wrote: > On Sun, Oct 21, 2018 at 6:59 PM Chris Johns <chr...@rtems.org > <mailto:chr...@rtems.org>> wrote: > > On 22/10/2018 09:11, Joel Sherrill wrote: > > On Sun, Oct 21, 2018 at 2:16 PM Christian Mauderer <l...@c-mauderer.de > <mailto:l...@c-mauderer.de> > > <mailto:l...@c-mauderer.de <mailto:l...@c-mauderer.de>>> wrote: > > Am 21.10.18 um 19:07 schrieb Joel Sherrill: > > > Hi > > > > > > I am in the middle of reconfiguring my old (~2013 i7) laptop as > an email > > > and remote workstation for home. Between helping students and > doing two > > > Kick Starts in the past 6 weeks, VM configuration, disk space > > > expectations, and time required to build the tools seem to be a > topic > > > that needs to be addressed. Under-configured VMs don't finish > building > > > or take a LONG time. > > My only concern with VMs is promoting the idea of needing a VM for RTEMS > development. A lot of work has gone into making the tools native across a > wide > number of hosts and native tools is the best solution for a user. I hope > we > encourage and teach using native tools before a VM. > > > I am sorry if it seemed the emphasis was on VMs. My intent was to include > build times for sparc-rtems5 with the source pre-downloaded on a variety > of host environments and computer levels. The time varies a lot.
Yes it does. > Yes there would be some VM advice but it would be secondary to the idea > that if you build on a Pi or an i3 with 5200 RPM laptop drive, expect it to > take > a long time. Yeap this makes sense. > Plus even on a fast machine, all I can say is that Cygwin is slow. > I can't give an estimate. The Windows builds I am posting on 'builds' at the moment are on an i7-2600 running at 3.7GHz and 8GB of ram. I turn Windows defender off when I am around but it likes to come back on and even with everything I can see to exclude there is still a sizeable 20+% overhead when running. The arm-rtems5 took 3hr19min and the Darwin build took 1hr02min and that is a 2.8GHz i5 so at a guess I would say a slow down of x4 or x5. The critical factor is the multilib'ing in gcc because of the configure overhead and the libstdc++ related stuff that happens. A libstdc++ build symlinks lots of header files and that in cygwin is a copy. > I have used native RTEMS Windows and Mac tools for development. It can > mean > learning some different work flows but in the end it is all rather > boringly > similar. The most difficult bit on Windows is debugging and the solution > tends > to be remote tcp to a GDB server else where. Who else has used Windows or > Mac > for development? > > I did list including times for MSYS2 and Cygwin in my list. I don't have a > Mac to > add that. Are the recent builds results any help? > > > I am proposing that I gather performance and configuraton notes > for > > > building SPARC tools after downloading source on a few > configurations: > > What about a link to the builds mailing list archive with something about > the > values to look for? Could the host memory and CPU type be added to the > email > reports? > > That won't help the people with problems because anyone who posts to that > list has (1) a fast machine and (2) has tuned it. I used 8 and 12 core > machines > with SSDs to report those from. I doubt they are representative of what a > GCI student uses. You could post your slow machine results :) > > > + Laptop 2013 i7: Centos on VM > > > + Laptop 2013 i7: MSYS2 > > > + Laptop 2013 i7: Cygwin > > Are there any cygwin build results posted to builds? > > I can include that in my next build sweep. Part of my plan was just to build > across > a lot and gather the same information. Report it so people could look at a > table > with some advice and know what to expect. Sure. > I mention my 2013 i7 because even though it is old, it really is not that > different > in performance from a new low-mid end laptop. Great. > > > + Laptop 2017 i7: Same three > > > + 8 core Xeon Centos native > > > + 12 core i7 Fedora native > > I would prefer we did not gathering and publish in documentation > information > that is hard to be consistent, could be misleading and is of often out of > date > as soon as it is published. I remember removing some stuff like this when > I > moved the docs to Sphinx as the data was over ten years old. I cannot > find it > now in the old doco. > > OK. I will see if I can generalize and maybe make a blog post. How you did this would be a good blog post. > I am fine with the amount of disk space needed to build a tool set and > then a > more general comment that the more cores, memory and fast disks you use > the > faster the build will be. My Windows builds are on stripped disks. > > > My point exactly. The core developers build on machines that are not > representative > of the average user. Sure. > For Windows you can document the POSIX layer for the shell etc adds > overhead > plus virus checking slows the build down so the build directory should be > tagged > as a directory not to check. I have no idea how cygwin and Windows > Defender > interact but I suspect it will slow things down by a lot and excluding it > would > help. > > On Windows does the virus scanner scan the VM disk file in real-time or > are the > VM's smart enough to have that file excluded? > > I don't know. I have seen us have to disable virus scanners on some computers > here at OAR but it tends to be on the Windows VMs themselves. On Windows 10 there is the Task Manager and the more detailed view shows the loads and then there is a tool called 'Process Explorer' on an MS download site which provides lots of detail. > > > One the 2017 7th generation i7, differences in VM configuration > can > > > result in the build time for sparc tools almost tripling. > > > > > > Does this sound useful or confusing? I know it is potentially > somewhat > > > volatile information. But my old 3rd generation i7 CPU benchmark > results > > > are comparable to an i5 that is much newer. Plus my old i7 has an > SSD > > > which many newer i3/i5 laptops do not. > > > > > > Feedback appreciated. > > > > > > --joel > > > > > > > Hello Joel, > > > > in my experience, the biggest difference in the build time is the > number > > of cores (in a VM or on a real machine). The processor generation > didn't > > seem to have that much influence. But I never measured exact > numbers. > > > > I only mention the processor generation because we don't tend to have > i3 or i5 > > CPUs available but students and users do. My 6-year old i7 benchmarks > like a > > newer i5. But often i5's don't come with SSDs so they can suffer even > more. > > > > Number of cores and RAM are the two big factors. As is making sure you > have > > enough disk space allocated to avoid turning the entire thing in an > exercise in > > frustration. > > Agreed, the RSB has been updated recently to report usage. > > And this helps. I just want to give advice based on that before someone > creates > a VM or partitions a disk. Sizing a VM is important. > > > It might would be a good idea to add some rough numbers somewhere > (maybe > > in the RSB-manual) so that a new user knows that he has to expect > for > > Hmm, User manual instead? > > > I think it should be there. I would like the RSB manual to have "internals" or > "developer" in the title. Using it should not be in it. Please feel free to change it. It's use has evolved. > I am happy for general guide lines or instructions on improving the > experience > for new users with new systems, for example with the latest patch I > pushed over > the weekend running a set builder command with `--dry-run` will let you > know if > the Python libraries are installed before anything is built. I am not > convinced > about the cost/benefit for any specific detail such as build times and > host > processor types. > > You haven't spend a day helping a room full of people try to build the tools > and finding out that many fail due to lack of disk space or take forever due > to underconfigured VMs. I am not saying we should recommend VMs, just > that we should give admit people use them and give advice. I was looking over some of Tim Ansell's FPGA things he has in timvideos projects recently and he asks users to complete a range of basic things before turning up ... https://github.com/timvideos/litex-buildenv/wiki/HowTo-LCA2018-FPGA-Miniconf Would that help? > Sometimes I have a 50+% fail rate and end up with people resizing disks > or reloading VMs. Ouch. > Lets not forget building the tools should be once for a project and not > something you do each week. > > I agree. But it is the first thing you do and that's the first impression. > > As the old saying goes, first impressions count. > Yeah this is really important. > > > example roughly 4 to 6 hours on a single core or about 1/2 to 3/4 > of an > > hour on a 8 core Linux system. It might could be interesting to have > > some rough numbers for other commonly used systems too (like MSYS2, > > Cygwin, FreeBSD or MacOS). > > The builds mailing list has real values ... > > https://lists.rtems.org/pipermail/build/ > > For Windows I have ... > > https://lists.rtems.org/pipermail/build/2018-October/001164.html > > I stopped the build because I am fixing some more places where long paths > are > being used which is why the arm build broke ... > > https://lists.rtems.org/pipermail/build/2018-October/001177.html > > > My 7th generation i7 (Dell last fall from last fall) is ~35 minutes for > SPARC as > > I tune my VM. > > A fast i7 macbook pro (PCIe SSD, 32G RAM, APFS) and native tools is > around 5min > for a bfin. The Mac posts to builds are from a Mac Mini with less > performance ... > > https://lists.rtems.org/pipermail/build/2018-October/001168.html > > and the bfin is 17mins. I use the bfin to test the RSB cause it is fast to > build. > > > Good for smoke tests. Yet all our Getting Started is for sparc and that's a > bit > more. > Looks like an hour from the same build run: > > https://lists.rtems.org/pipermail/build/2018-October/001123.html > > That's ~2x a Centos VM on my laptop. So it varies a lot. > The sparc time has exploded with all the multilibs we now build. > We also had someone on the gci@ mailing list who seemed to take days for > the build to complete. Ouch. > > A student in a Kick Start with the same laptop turned that into > > a 90 minute build by having 1 core and less RAM. So even with a fast > machine, > > the guidance on the VM is important. > > How about "Give it everything you have!" and "Don't try and play video > game!" :) > > +1 > > > Ignoring those who pick tiny virtual HD sizes and then can't even > complete the > > build no matter how long they wait. > > We should document this. I have had real problems with VirtualBox and > sharing > host disks. Last time I tried about 12 months ago it did not work. > > > AFAIK you can't successfully build on a mounted share drive. That's why it is > critical > to allocate enough disk space to the native filesystem for the virtual OS. OK so the sharing is still broken. > > > I don't think that it is useful to compare processor generations. > That > > would be an information that would have to be updated on a regular > basis > > to have any use. I would only add some (few) examples. > > > > The CPU generation wasn't the point. Just that the older one is slower. > Newer > > generation ones are often only 20% faster than the old one. Just a > reference point. > > > If you find any big influences beneath number of cores (you > mentioned VM > > settings), it might would be worth adding a general section with > tips > > for speeding up the build process. > > > > Not failing is the biggest one. I recommend downloading all source first > since that > > sometimes fails, doublechecking host packages, and Chris and I moved gdb > before > > It is what happens when we spend a couple of days commuting from Oakland > to > Haywood in the Bay area. :) > > > gcc/newlib in the tools bset so users would fail on that early rather > than > last. > > And the latest patch has code to find Python.h and libpython<M><m>.* .. > > > https://git.rtems.org/rtems-source-builder/tree/source-builder/config/gdb-common-1.cfg#n70 > > > That's about it beyond VM tuning and expectations. If you have an i3 > with a > > 5200RPM slow laptop drive, it is going to take a while. We say 30-45 > minutes > > and we all have nice computers with tuned VMs. > > > > And Windows builds are WAY slower and I can't even give you an estimate > at > > this point. I just walk away. > > Building the tools is slower but you can get the overhead to be just the > POSIX > Cygwin/MSYS overhead and not much more. A build of libbsd with native > Windows > tools should be fast and my experience it is. To me this is more > important than > the tools build time and we should not lose sight of this. My hope is > users > spend more time building applications than tools. > > > Me too but they have to finish the tools. :) > Yeap too true. > > So this was just "here's what we know about what to expect and what you > can > > do to help". > > Seem like a good idea. > > > That's all I was trying to capture. Build times seem to vary by a factor of 10 > between > core developers and users. Especially students with lower end computers. We > want > folks to succeed. Yes we do. I am OK with having info in our documentation, we just need to have it maintained so this is either what we put or a process to update it or both. Chris _______________________________________________ devel mailing list devel@rtems.org http://lists.rtems.org/mailman/listinfo/devel
