Re: [PATCH 4/7 v2] lto: Implement ltrans cache

2024-06-21 Thread Sam James
Andi Kleen writes: > FWIW I suspect not handling lockfile errors could be a show stopper > even for an initial implementation. It's not that uncommon that people > press Ctrl-C. flock on systems that have it would be a safer > alternative. > >> There are many things to do and I think it is bette

Re: [PATCH 4/7 v2] lto: Implement ltrans cache

2024-06-21 Thread Andi Kleen
On Fri, Jun 21, 2024 at 06:59:05PM +0200, Michal Jireš wrote: > > The lockfiles scare me a bit. What happens when they get lost, e.g. > > due to a compiler crash? You may need some recovery for that. > > Perhaps it would be better to make the files self checking, so that > > partial files can be de

Re: [PATCH 4/7 v2] lto: Implement ltrans cache

2024-06-21 Thread Andi Kleen
FWIW I suspect not handling lockfile errors could be a show stopper even for an initial implementation. It's not that uncommon that people press Ctrl-C. flock on systems that have it would be a safer alternative. > There are many things to do and I think it is better to do that in trunk > rahte

Re: [PATCH 4/7 v2] lto: Implement ltrans cache

2024-06-21 Thread Michal Jireš
On 6/20/24 7:45 PM, Andi Kleen wrote: There are much faster/optimized modern hashes for good collision detection over MD5 especially when it's not needed to be cryptographically secure. Pick something from smhasher. Also perhaps the check sum should be cached in the file? I assume it's cheap to

Re: [PATCH 4/7 v2] lto: Implement ltrans cache

2024-06-21 Thread Jan Hubicka
> Michal Jires writes: > > No performance data? Michal has bachelor thesis on the topic which has some statistics https://dspace.cuni.cz/handle/20.500.11956/183051?locale-attribute=en > > > + > > +static const md5_checksum_t INVALID_CHECKSUM = { > > + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,

Re: [PATCH 4/7 v2] lto: Implement ltrans cache

2024-06-20 Thread Andi Kleen
Michal Jires writes: No performance data? > + > +static const md5_checksum_t INVALID_CHECKSUM = { > + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, > +}; There are much faster/optimized modern hashes for good collision detection over MD5 especially when it's not needed to be cryptographicall

[PATCH 4/7 v2] lto: Implement ltrans cache

2024-06-20 Thread Michal Jires
Outside of suggested changes, this version: - uses #define INCLUDE_* - is rebased onto current trunk - 'fprintf (mstream,' lines - --verbose 'recompiling++' count is moved into correct if branch ___ This patch implements Incremental LTO as ltrans cache. The cache is active when directory $GCC_LT