Re: Slow history load with some values of HISTSIZE

2024-02-09 Thread Chet Ramey

On 2/3/24 3:29 PM, Casey Johnson wrote:


Bash Version: 5.1
Patch Level: 16
Release Status: release

Description:

  The current implementation of the command history takes a long time to 
load
  a large HISTFILE when history is stifled.  This is because there is a
  memmove() for every line of HISTFILE once HISTSIZE lines have been loaded.
  If N is the number of lines in HISTFILE then the cost of the memmoves ends
  up being something like (N - HISTSIZE) * HISTSIZE, peaking when HISTSIZE 
is
  roughly N/2.


Thanks for the report and the patch. I applied it with some tweaks and
saw a dramatic performance improvement for very large history files (I
tested with one that was about 69 entries) where HISTSIZE is
significantly smaller (I tested with HISTSIZE=10): from around 11
seconds to less than 0.3 seconds.

Chet

--
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/



OpenPGP_signature.asc
Description: OpenPGP digital signature


Re: Slow history load with some values of HISTSIZE

2024-02-09 Thread alex xmb sw ratchev
On Sat, Feb 10, 2024, 03:06 Chet Ramey  wrote:

> On 2/3/24 3:29 PM, Casey Johnson wrote:
>
> > Bash Version: 5.1
> > Patch Level: 16
> > Release Status: release
> >
> > Description:
> >
> > The current implementation of the command history takes a long time to
> load
> > a large HISTFILE when history is stifled.  This is because there is a
> > memmove() for every line of HISTFILE once HISTSIZE lines have been
> loaded.
> > If N is the number of lines in HISTFILE then the cost of the memmoves
> ends
> > up being something like (N - HISTSIZE) * HISTSIZE, peaking when HISTSIZE
> is
> > roughly N/2.
>
> Thanks for the report and the patch. I applied it with some tweaks and
> saw a dramatic performance improvement for very large history files (I
> tested with one that was about 69 entries) where HISTSIZE is
> significantly smaller (I tested with HISTSIZE=10): from around 11
> seconds to less than 0.3 seconds.
>

++ speed optimization

greets ++

Chet
>
> --
> ``The lyf so short, the craft so long to lerne.'' - Chaucer
>  ``Ars longa, vita brevis'' - Hippocrates
> Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
>
>