The problem with the tagged format is that it's *not* usable by grep
Awk would have no problem with it.
limited to exactly whatever magic is built into the "history" command
That's where the magic should be. If that were the official interface
to `.bash_history`, then bash has freedom to innovate on what is stored,
and how it is stored. Then we could add even more information.
Would that be few enough files to satisfy you?
My itch has already been scratched--albeit with a rat's nest of scripts
and configurations. With a better history implementation, that could be
done away with. Migrating to multiple files for such a low-bandwidth
application seems like replacing one rat's nest with another. It would
be every bit as brittle as what bash is already doing.
Another option would be to provide a more robust hook for using your own
history provider. I put my tweaks in PROMPT_COMMAND, but if we had a
HISTAPPEND_COMMAND, we could pass it environment variables relating to
all aspects of the last command--start time, end time, cpu, working dir,
PID, TTY, etc.
There are several external history providers out there, but they seem to
need hundreds of lines of bash to do a proper integration. Making it
easier for those tools to capture everything bash knows related to a
command would easily halve the size and complexity of those integration
scripts.
I've attached an example integration script for https://atuin.sh/. I
don't use it myself. I only mention it because it seems to be the most
popular one.
You can see that only around 25% of the script is for keybinds and
retrieval; the rest is for setting up the capture.
Fields of interest, a running list:
- shell start time
- shell PID
- shell TTY
- command start time (milli/micro)
- command end time (milli/micro)
- command CPU real/user/sys
- command exit code
- command working directory
On 2024-08-20 12:35 pm, Martin D Kealey wrote:
The problem with the tagged format is that it's *not* usable by grep,
so
you're limited to exactly whatever magic is built into the "history"
command.
"Yuck" is in the eye of the beholder. I've tried numerous other ways to
segregate sessions, and IMO multiple files was the "least yuck" of many
worse options.
All that said, I would like *some* additional information recorded in
the
history file, especially $PWD (when it changes - interpreting "popd"
requires significant mental effort when reading a history file), and
$BASHPID (to track nested shells). With those I would be happy to have
~/.bash_history.d/$TTY, which would greatly reduce the number of files.
Would that be few enough files to satisfy you?
-Martin
On Wed, 21 Aug 2024, 00:38 , <supp...@eggplantsd.com> wrote:
Bash or no bash, spreading history over dozens of files in
`bash_history.d/` is yuck. We already have a comment with the
timestamp
in `.bash_history`. If I were implementing the suggestion, I would add
more information to the comment, then add two new flags to the
`history`
command that filter+output the file (i.e. not the internal history
list): `--global` to display everything in `.bash_history`, and
`--local` to restrict output to entries from the current session.
Everything else would remain as-is.
So this:
#1724136363
man bash
Becomes this:
#1724136363 [sess create time] [sess PID] [sess TTY]
man bash
I think it is important to add the local/global flags because it gives
us some leeway as to how that comment is structured. If you take the
line of "that's what grep is for", then we're committed to the v1
format
forever after.
The problem with the stackoverflow solutions is that they are
all-or-nothing: either mash the history together across all sessions,
but get strange behavior on history nav & expansion, or don't mash, be
cut-off from information in concurrent sessions, and end up with the
occasional unsaved session. Being able to filter the file directly
lets
us look things up without having to slice-and-splice into the internal
history.
On 2024-08-20 6:14 am, Martin D Kealey wrote:
> "Missing/disappearing history" is entirely down to the lack of "writing
> history as you go", and yes that would be reasonable to offer as a new
> opt-in feature.
>
> As for separation of sessions, I strongly suspect that anything between
> *total* separation and *none* will result in so many ugly compromises
> that
> in the end almost nobody will be happy with it. So if there's to be an
> additional option - which I'm not convinced of - I suggest that it
> simply
> be to set HISTFILE by default to either
> $HOME/.bash_history.d/{some-pattern-here} (if the directory exists) or
> ~/.bash_history (matching the current behaviour when that directory
> does
> not exist). I would recommend that the pattern include most or all of
> $$,
> $TTY, $LOGNAME, and $((EPOCHSECONDS-SECONDS)).
>
> Lastly, an awful lot of "default behaviour" is down to whatever
> /etc/skel/.bashrc and /etc/bash/bashrc that are shipped with Bash by
> the
> various distros. Maybe Bash should start shipping some kind of
> "standard
> library" of functions that are *expected* to be included with any
> distro,
> but are not actually built into the binary.
>
> -Martin
>
> PS: complaining about "inelegant" in relation to Bash seems a bit
> pointless.
>
> On Tue, 20 Aug 2024 at 16:48, <supp...@eggplantsd.com> wrote:
>
>> I wouldn't consider dozens of stackoverflow/askubuntu/etc complaints
>> of
>> missing/disappearing history "cherry-picked". There were far more
>> than
>> I sent.
>>
>> I understand not wanting to pull the rug out from under people, but
>> the
>> kludges Kealey posted were inelegant. An opt-in for the suggested
>> behavior would be good enough.
>>
>> JS
>>
>> On 2024-08-20 2:17 am, Lawrence Velázquez wrote:
>> > On Tue, Aug 20, 2024, at 1:42 AM, supp...@eggplantsd.com wrote:
>> >> The suggestion is that the default behavior needs some work
>> >
>> > The default behavior is unlikely to change. For every cherry-picked
>> > example of someone unsatisfied with it (bugs aside), there is likely
>> > someone else who prefers it as is (or at least would not appreciate
>> > it changing out from under them). New shopt settings may be doable.
>>
>>
__atuin_bind_ctrl_r=true
__atuin_bind_up_arrow=true
# Include guard
if [[ ${__atuin_initialized-} == true ]]; then
false
elif [[ $- != *i* ]]; then
# Enable only in interactive shells
false
elif ((BASH_VERSINFO[0] < 3 || BASH_VERSINFO[0] == 3 && BASH_VERSINFO[1] < 1));
then
# Require bash >= 3.1
[[ -t 2 ]] && printf 'atuin: requires bash >= 3.1 for the integration.\n'
>&2
false
else # (include guard) beginning of main content
#------------------------------------------------------------------------------
__atuin_initialized=true
ATUIN_SESSION=$(atuin uuid)
ATUIN_STTY=$(stty -g)
export ATUIN_SESSION
ATUIN_HISTORY_ID=""
export ATUIN_PREEXEC_BACKEND=$SHLVL:none
__atuin_update_preexec_backend() {
if [[ ${BLE_ATTACHED-} ]]; then
ATUIN_PREEXEC_BACKEND=$SHLVL:blesh-${BLE_VERSION-}
elif [[ ${bash_preexec_imported-} ]]; then
ATUIN_PREEXEC_BACKEND=$SHLVL:bash-preexec
elif [[ ${__bp_imported-} ]]; then
ATUIN_PREEXEC_BACKEND="$SHLVL:bash-preexec (old)"
else
ATUIN_PREEXEC_BACKEND=$SHLVL:unknown
fi
}
__atuin_preexec() {
# Workaround for old versions of bash-preexec
if [[ ! ${BLE_ATTACHED-} ]]; then
# In older versions of bash-preexec, the preexec hook may be called
# even for the commands run by keybindings. There is no general and
# robust way to detect the command for keybindings, but at least we
# want to exclude Atuin's keybindings. When the preexec hook is called
# for a keybinding, the preexec hook for the user command will not
# fire, so we instead set a fake ATUIN_HISTORY_ID here to notify
# __atuin_precmd of this failure.
if [[ $BASH_COMMAND == '__atuin_history'* && $BASH_COMMAND != "$1" ]];
then
ATUIN_HISTORY_ID=__bash_preexec_failure__
return 0
fi
fi
# Note: We update ATUIN_PREEXEC_BACKEND on every preexec because blesh's
# attaching state can dynamically change.
__atuin_update_preexec_backend
local id
id=$(atuin history start -- "$1")
export ATUIN_HISTORY_ID=$id
__atuin_preexec_time=${EPOCHREALTIME-}
}
__atuin_precmd() {
local EXIT=$? __atuin_precmd_time=${EPOCHREALTIME-}
[[ ! $ATUIN_HISTORY_ID ]] && return
# If the previous preexec hook failed, we manually call __atuin_preexec
if [[ $ATUIN_HISTORY_ID == __bash_preexec_failure__ ]]; then
# This is the command extraction code taken from bash-preexec
local previous_command
previous_command=$(
export LC_ALL=C HISTTIMEFORMAT=''
builtin history 1 | sed '1 s/^ *[0-9][0-9]*[* ] //'
)
__atuin_preexec "$previous_command"
fi
local duration=""
# shellcheck disable=SC2154,SC2309
if [[ ${BLE_ATTACHED-} && ${_ble_exec_time_ata-} ]]; then
# With ble.sh, we utilize the shell variable `_ble_exec_time_ata`
# recorded by ble.sh. It is more accurate than the measurements by
# Atuin, which includes the spawn cost of Atuin. ble.sh uses the
# special shell variable `EPOCHREALTIME` in bash >= 5.0 with the
# microsecond resolution, or the builtin `time` in bash < 5.0 with the
# millisecond resolution.
duration=${_ble_exec_time_ata}000
elif ((BASH_VERSINFO[0] >= 5)); then
# We calculate the high-resolution duration based on EPOCHREALTIME
# (bash >= 5.0) recorded by precmd/preexec, though it might not be as
# accurate as `_ble_exec_time_ata` provided by ble.sh because it
# includes the extra time of the precmd/preexec handling. Since Bash
# does not offer floating-point arithmetic, we remove the non-digit
# characters and perform the integral arithmetic. The fraction part of
# EPOCHREALTIME is fixed to have 6 digits in Bash. We remove all the
# non-digit characters because the decimal point is not necessarily a
# period depending on the locale.
duration=$((${__atuin_precmd_time//[!0-9]} -
${__atuin_preexec_time//[!0-9]}))
if ((duration >= 0)); then
duration=${duration}000
else
duration="" # clear the result on overflow
fi
fi
(ATUIN_LOG=error atuin history end --exit "$EXIT"
${duration:+"--duration=$duration"} -- "$ATUIN_HISTORY_ID" &) >/dev/null 2>&1
export ATUIN_HISTORY_ID=""
}
__atuin_set_ret_value() {
return ${1:+"$1"}
}
# The shell function `__atuin_evaluate_prompt` evaluates prompt sequences in
# $PS1. We switch the implementation of the shell function
# `__atuin_evaluate_prompt` based on the Bash version because the expansion
# ${PS1@P} is only available in bash >= 4.4.
if ((BASH_VERSINFO[0] >= 5 || BASH_VERSINFO[0] == 4 && BASH_VERSINFO[1] >= 4));
then
__atuin_evaluate_prompt() {
__atuin_set_ret_value "${__bp_last_ret_value-}"
"${__bp_last_argument_prev_command-}"
__atuin_prompt=${PS1@P}
# Note: Strip the control characters ^A (\001) and ^B (\002), which
# Bash internally uses to enclose the escape sequences. They are
# produced by '\[' and '\]', respectively, in $PS1 and used to tell
# Bash that the strings inbetween do not contribute to the prompt
# width. After the prompt width calculation, Bash strips those control
# characters before outputting it to the terminal. We here strip these
# characters following Bash's behavior.
__atuin_prompt=${__atuin_prompt//[$'\001\002']}
# Count the number of newlines contained in $__atuin_prompt
__atuin_prompt_offset=${__atuin_prompt//[!$'\n']}
__atuin_prompt_offset=${#__atuin_prompt_offset}
}
else
__atuin_evaluate_prompt() {
__atuin_prompt='$ '
__atuin_prompt_offset=0
}
fi
# The shell function `__atuin_clear_prompt N` outputs terminal control
# sequences to clear the contents of the current and N previous lines. After
# clearing, the cursor is placed at the beginning of the N-th previous line.
__atuin_clear_prompt_cache=()
__atuin_clear_prompt() {
local offset=$1
if [[ ! ${__atuin_clear_prompt_cache[offset]+set} ]]; then
if [[ ! ${__atuin_clear_prompt_cache[0]+set} ]]; then
__atuin_clear_prompt_cache[0]=$'\r'$(tput el 2>/dev/null || tput ce
2>/dev/null)
fi
if ((offset > 0)); then
__atuin_clear_prompt_cache[offset]=${__atuin_clear_prompt_cache[0]}$(
tput cuu "$offset" 2>/dev/null || tput UP "$offset" 2>/dev/null
tput dl "$offset" 2>/dev/null || tput DL "$offset" 2>/dev/null
tput il "$offset" 2>/dev/null || tput AL "$offset" 2>/dev/null
)
fi
fi
printf '%s' "${__atuin_clear_prompt_cache[offset]}"
}
__atuin_accept_line() {
local __atuin_command=$1
# Reprint the prompt, accounting for multiple lines
local __atuin_prompt __atuin_prompt_offset
__atuin_evaluate_prompt
__atuin_clear_prompt "$__atuin_prompt_offset"
printf '%s\n' "$__atuin_prompt$__atuin_command"
# Add it to the bash history
history -s "$__atuin_command"
# Assuming bash-preexec
# Invoke every function in the preexec array
local __atuin_preexec_function
local __atuin_preexec_function_ret_value
local __atuin_preexec_ret_value=0
for __atuin_preexec_function in "${preexec_functions[@]:-}"; do
if type -t "$__atuin_preexec_function" 1>/dev/null; then
__atuin_set_ret_value "${__bp_last_ret_value:-}"
"$__atuin_preexec_function" "$__atuin_command"
__atuin_preexec_function_ret_value=$?
if [[ $__atuin_preexec_function_ret_value != 0 ]]; then
__atuin_preexec_ret_value=$__atuin_preexec_function_ret_value
fi
fi
done
# If extdebug is turned on and any preexec function returns non-zero
# exit status, we do not run the user command.
if ! { shopt -q extdebug && ((__atuin_preexec_ret_value)); }; then
# Juggle the terminal settings so that the command can be interacted
# with
local __atuin_stty_backup
__atuin_stty_backup=$(stty -g)
stty "$ATUIN_STTY"
# Execute the command. Note: We need to record $? and $_ after the
# user command within the same call of "eval" because $_ is otherwise
# overwritten by the last argument of "eval".
__atuin_set_ret_value "${__bp_last_ret_value-}"
"${__bp_last_argument_prev_command-}"
eval -- "$__atuin_command"$'\n__bp_last_ret_value=$?
__bp_last_argument_prev_command=$_'
stty "$__atuin_stty_backup"
fi
# Execute preprompt commands
local __atuin_prompt_command
for __atuin_prompt_command in "${PROMPT_COMMAND[@]}"; do
__atuin_set_ret_value "${__bp_last_ret_value-}"
"${__bp_last_argument_prev_command-}"
eval -- "$__atuin_prompt_command"
done
# Bash will redraw only the line with the prompt after we finish,
# so to work for a multiline prompt we need to print it ourselves,
# then go to the beginning of the last line.
__atuin_evaluate_prompt
printf '%s' "$__atuin_prompt"
__atuin_clear_prompt 0
}
__atuin_history() {
# Default action of the up key: When this function is called with the first
# argument `--shell-up-key-binding`, we perform Atuin's history search only
# when the up key is supposed to cause the history movement in the original
# binding. We do this only for ble.sh because the up key always invokes
# the history movement in the plain Bash.
if [[ ${BLE_ATTACHED-} && ${1-} == --shell-up-key-binding ]]; then
# When the current cursor position is not in the first line, the up key
# should move the cursor to the previous line. While the selection is
# performed, the up key should not start the history search.
# shellcheck disable=SC2154 # Note: these variables are set by ble.sh
if [[ ${_ble_edit_str::_ble_edit_ind} == *$'\n'* ||
$_ble_edit_mark_active ]]; then
ble/widget/@nomarked backward-line
local status=$?
READLINE_LINE=$_ble_edit_str
READLINE_POINT=$_ble_edit_ind
READLINE_MARK=$_ble_edit_mark
return "$status"
fi
fi
# READLINE_LINE and READLINE_POINT are only supported by bash >= 4.0 or
# ble.sh. When it is not supported, we localize them to suppress strange
# behaviors.
[[ ${BLE_ATTACHED-} ]] || ((BASH_VERSINFO[0] >= 4)) ||
local READLINE_LINE="" READLINE_POINT=0
local __atuin_output
__atuin_output=$(ATUIN_SHELL_BASH=t ATUIN_LOG=error
ATUIN_QUERY="$READLINE_LINE" atuin search "$@" -i 3>&1 1>&2 2>&3)
# We do nothing when the search is canceled.
[[ $__atuin_output ]] || return 0
if [[ $__atuin_output == __atuin_accept__:* ]]; then
__atuin_output=${__atuin_output#__atuin_accept__:}
if [[ ${BLE_ATTACHED-} ]]; then
ble-edit/content/reset-and-check-dirty "$__atuin_output"
ble/widget/accept-line
else
__atuin_accept_line "$__atuin_output"
fi
READLINE_LINE=""
READLINE_POINT=${#READLINE_LINE}
else
READLINE_LINE=$__atuin_output
READLINE_POINT=${#READLINE_LINE}
fi
}
__atuin_initialize_blesh() {
# shellcheck disable=SC2154
[[ ${BLE_VERSION-} ]] && ((_ble_version >= 400)) || return 0
ble-import contrib/integration/bash-preexec
# Define and register an autosuggestion source for ble.sh's auto-complete.
# If you'd like to overwrite this, define the same name of shell function
# after the $(atuin init bash) line in your .bashrc. If you do not need
# the auto-complete source by atuin, please add the following code to
# remove the entry after the $(atuin init bash) line in your .bashrc:
#
# ble/util/import/eval-after-load core-complete '
# ble/array#remove _ble_complete_auto_source atuin-history'
#
function ble/complete/auto-complete/source:atuin-history {
local suggestion
suggestion=$(ATUIN_QUERY="$_ble_edit_str" atuin search --cmd-only
--limit 1 --search-mode prefix)
[[ $suggestion == "$_ble_edit_str"?* ]] || return 1
ble/complete/auto-complete/enter h 0 "${suggestion:${#_ble_edit_str}}"
'' "$suggestion"
}
ble/util/import/eval-after-load core-complete '
ble/array#unshift _ble_complete_auto_source atuin-history'
# @env BLE_SESSION_ID: `atuin doctor` references the environment variable
# BLE_SESSION_ID. We explicitly export the variable because it was not
# exported in older versions of ble.sh.
[[ ${BLE_SESSION_ID-} ]] && export BLE_SESSION_ID
}
__atuin_initialize_blesh
BLE_ONLOAD+=(__atuin_initialize_blesh)
precmd_functions+=(__atuin_precmd)
preexec_functions+=(__atuin_preexec)
# shellcheck disable=SC2154
if [[ $__atuin_bind_ctrl_r == true ]]; then
# Note: We do not overwrite [C-r] in the vi-command keymap for Bash because
# we do not want to overwrite "redo", which is already bound to [C-r] in
# the vi_nmap keymap in ble.sh.
bind -m emacs -x '"\C-r": __atuin_history --keymap-mode=emacs'
bind -m vi-insert -x '"\C-r": __atuin_history --keymap-mode=vim-insert'
bind -m vi-command -x '"/": __atuin_history --keymap-mode=emacs'
fi
# shellcheck disable=SC2154
if [[ $__atuin_bind_up_arrow == true ]]; then
if ((BASH_VERSINFO[0] > 4 || BASH_VERSINFO[0] == 4 && BASH_VERSINFO[1] >=
3)); then
bind -m emacs -x '"\e[A": __atuin_history --shell-up-key-binding
--keymap-mode=emacs'
bind -m emacs -x '"\eOA": __atuin_history --shell-up-key-binding
--keymap-mode=emacs'
bind -m vi-insert -x '"\e[A": __atuin_history --shell-up-key-binding
--keymap-mode=vim-insert'
bind -m vi-insert -x '"\eOA": __atuin_history --shell-up-key-binding
--keymap-mode=vim-insert'
bind -m vi-command -x '"\e[A": __atuin_history --shell-up-key-binding
--keymap-mode=vim-normal'
bind -m vi-command -x '"\eOA": __atuin_history --shell-up-key-binding
--keymap-mode=vim-normal'
bind -m vi-command -x '"k": __atuin_history --shell-up-key-binding
--keymap-mode=vim-normal'
else
# In bash < 4.3, "bind -x" cannot bind a shell command to a keyseq
# having more than two bytes. To work around this, we first translate
# the keyseqs to the two-byte sequence \C-x\C-p (which is not used by
# default) using string macros and run the shell command through the
# keybinding to \C-x\C-p.
bind -m emacs -x '"\C-x\C-p": __atuin_history --shell-up-key-binding
--keymap-mode=emacs'
bind -m emacs '"\e[A": "\C-x\C-p"'
bind -m emacs '"\eOA": "\C-x\C-p"'
bind -m vi-insert -x '"\C-x\C-p": __atuin_history
--shell-up-key-binding --keymap-mode=vim-insert'
bind -m vi-insert '"\e[A": "\C-x\C-p"'
bind -m vi-insert '"\eOA": "\C-x\C-p"'
bind -m vi-command -x '"\C-x\C-p": __atuin_history
--shell-up-key-binding --keymap-mode=vim-normal'
bind -m vi-command '"\e[A": "\C-x\C-p"'
bind -m vi-command '"\eOA": "\C-x\C-p"'
bind -m vi-command '"k": "\C-x\C-p"'
fi
fi
#------------------------------------------------------------------------------
fi # (include guard) end of main content
