Hello, With all due respect, please familiarize yourself with the "BiDi in Terminal Emulators" work at https://terminal-wg.pages.freedesktop.org/bidi/ , as well as BiDi algorithm itself.
Several years ago the situation was that nobody knew how to do BiDi in terminals. Some folks (including mlterm and vim) did _something_ that might have been enough for certain particular needs, without seeing and understanding the whole picture. Others did something else. Other terminals that claimed to support BiDi (e.g. Konsole, PuTTY) did it differently from mlterm. Other terminal-based apps that claimed to support BiDi (e.g. Emacs) did it differently from Vim. Some combinations more or less worked "accidentally". (By the way, mlterm already does 3 different things based the command line option --bimode. How ridiculous is that? Putting on my user hat, I don't care, I just want things to work, I don't want to restart my terminal if I encounter an app that needs a different behavior, and I don't want to remember which app needs which mode. Taking off my user hat and putting on my developer hat: Which one should I develop my app against???) The terminal world is pretty complex; simple tools like "cat" and sophisticad apps like "vim" have fundamentally different needs, I'd argue that a terminal emulator is the union of at least two utterly different words (whose BiDi needs are also totally different). In addition, BiDi is pretty complex, too. Then someone with notable experience in terminals and notable experience in BiDi spent years (yes, literally years) in the back of his mind pondering about the problem space. Once the big picture became clean, he then spent months studying and evaluating prior work (documents and software), designing what needs to be done, documenting all of it along with rationale, and coming up with an implementation (along with extensive tests) in VTE, the terminal widget used by xfce4-terminal and others. (This person is me.) VTE is the first terminal that does not just do _some_ random BiDi, but understands the whole picture and does a thoroughly designed thing where finally all the parts _can_ fit together. Notice: _can_ fit together, not _do_ fit together immediately, that would be impossible. Apps have different needs against the behavior of the terminal, they need to tell the terminal which mode to switch to. Don't worry, this work does not reinvent BiDi shuffling from scratch: it just connects up the Unicode BiDi Algorithm (UBA) to the world of terminal emulation, as you also wish it would do. > non-Hebrew characters, like punctuation, appear on the wrong end of the line This is one of the possible behaviors of UBA, a behavior you can often experience in all other contexts as well (e.g. graphical text editors, web pages, even UBA's online test suite). The key concept here is "base direction" a.k.a. "paragraph direction". You can programatically configure it in VTE. VTE is the first terminal that understands that a single mode of BiDi operation cannot satisfy the needs of all the apps, therefore offers apps to choose the mode they wish to have. In fact, VTE is the first terminal that provides mathematical proof (in the aforementioned documentation) that a full-blown application, such as vim, cannot be made to function properly if it's up for the terminal to rearrange the letters. Vim can only work properly if vim itself does the shuffling, and the terminal lays down the characters from left to right. Something that VTE can be asked to do. BiDi and terminals are two extremely complicated things on their own, their combination is exponentially more complex. What happens in the mlterm + vim combo cannot be scaled up to work across all the terminal-based apps. Mind you, I'm absolutely certain it doesn't even work correctly in vim if you begin to stress test it (e.g. a mixed-direction line's width exceeds the display width). VTE is a platform where all apps can implement proper BiDi. It does not happen magically, apps need to tell VTE which mode to switch to. Please ask Vim folks to update their software accordingly. e.
