> Long-waited NetBSD support in CI would be awesome to be finished too.
As the maintainer of the linked vis project, I can confirm that the provided SSH access to the CI environment is very convenient and saves a lot of time.
I would also like to see the NetBSD support completed. It turns out that up until last week, we missed a #define to compile out of the box[1].
> A terminal app that has terminal features doesn't violate any principles of simplicity.
It still depends how these features are implemented.
For example in dvtm scroll back history is made searchable by piping it to $PAGER. Similarly, copy mode is implemented by using $EDITOR as an interactive filter.
You can be sure that the Kakoune authors are familiar with vi(m).
Having said that, if you like both vi(m) and the structural regular expression support of sam/acme you might be interested in vis which combines the two:
A number of people expressed the need to edit large files. For the development of my own editor[0] I would be interested to know what kind of usage patterns most often occur. What are the most important operations? Do you search for some (regex) pattern? Do you go to some specific line n? Do you copy/paste large portions of the file around? Do you often edit binary files? If so what types and what kind of changes do you perform?
1) Going to a specific location (e.g. a location that shows up in some error log about processing that data file) and eyeballing it. Being able to go to a specific location is sometimes important (e.g. row 12873, character 233). Syntax highlight is important, it sometimes makes obvious something that's subtly malformed. Syntax highlight that doesn't take an eternity for large files is a hard issue.
2) regex search/replace - interactive grep/sed.
3) Very large edits - e.g., find a specific location and remove all data entries before that so that the problematic entry now would be the first one; essentially cutting away half of a very large file.
4) Do note that you might have very, very large lines - it's not that uncommon to have the whole file in a single line, e.g. non-pretty-printed json data. Some editors work well with large files but simply die if there's a line with a million characters.
Yes syntax highlighting for large files is a hard issue. I'm not really aware of an accurate an high speed solution supporting editing operations in huge files.
In principle the underlying data structure used by vis supports all modifications with linear complexity in the number of editing operations since file load. This is independent of the file structure (i.e. single line files should be well supported). However the frontend code hasn't yet been optimized so in practice there might be some problems.
Unless one specifies the blackhole register when deleting large parts of a file this will create an in memory copy (to enable later pasting at a different location). Better would be to keep a reference to the existing immutable text region.
Many years ago, one compelling feature about Lugaru's emacs-family Epsilon (recently discussed here on HN) was the fact that you could quickly load _any_ file, with maximum sizes several times the available system memory, and completely regardless of text structure. You could load a fat binary, e.g. WORD.EXE, edit character strings in it, save it, and if you carefully avoided changing sizes and offsets, have a still-working .EXE program.
This is of course a slightly off-the-wall use case for a text editor, but if you happen to need that kind of thing you'll be really grateful if you have an editor that can do it.
I'm a bit hesitant to burden you with feature suggestions - sidetracking the developer(s) can easily kill many small projects. But I'd like to mention a feature I would find compellingly useful that I have yet to see in "modern" editors. Possibly this could be a distinguishing feature that gives you a niche!
IBM's mainframe editor ISPF allowed you to select a set of lines, usually based on a search (including negative search, i.e. lines _not_ containing the search data), then to manipulate the set of lines thus selected (manually removing lines, adding lines or reversing the selection) and then performing other operations, such as global search and replace, or sorting, or indenting or whatever, on that set of lines while ignoring all other text in the file.
I occasionally run into tasks where I would love to have this functionality available.
ST can do all these things. Search by regex, the "Find All" command inserts a cursor selecting each result, then you can do all the usual text editing operations on each selection in parallel. It has sorting and the like built in, and you can do regex find and replace inside the selections, etc.
for me, regex find and find-in-files is very important for analysing large codebases. Sublime is close to ideal for this (for me).
(Multi-line regexes also very useful).
Don't tend to copy large portions of files around, but goto-line is used fairly often and some kind of programatic interface (REPL-style) would also be good, similar to the Sublime console.
Plugins are also important for the long-tail features that are very niche.
...but... speed and lightness is also very important.... so don't use JS/CSS/HTML... please.
"Panopticon is under heavy development and its feature set still very basic. It's not yet able replace programs like radare2 and IDA Pro. We are working on it, check back regularly!"
As the main developer of an editor (vis) using a similar segmented data structure (a piece chain, similar to a rope, but storing the text junks in a double linked list, thus asymptotically worse) I can attest that the performance is generally very good. Some work will be required to adapt/write a regexp library around some kind of iterator interface, but in general I found that persistent data structures are very well suited for editors.
One problem with the gap buffer is that as soon as you have multiple cursors/selections or in my case structural regular expression support, then you have to move the gap around all the time. Also undo/redo support is simply not as elegant as with a persistent data structure.
For syntax highlighting copying the relevant text region into a contiguous memory block is simple and works well. For now I'm trying a completely stateless approach to syntax highlighting i.e. the text coloring is always completely recalculated. This trades highlighting accuracy for simplicity, but in practice the results aren't too bad. Efficient syntax highlighting for huge files is a hard problem.
Thanks for the hacking file, it is a good read for people like myself who are interested in text editor implementations. If time permits I will try delve a bit into the joe code base.
As for the syntax highlighting in vis, yes it currently is stateless and only considers a fixed range of text prior to the start of the window. Having no state at all simplifies some things and allows vis to highlight even extremely large files at the expense of the occasional glitch. In practice it seems to work reasonably well.
This approach also requires that the highlighting is fast, because nothing is cached, it is completely redone after every cursor motion. I'm quite happy with LPeg so far, it is convenient to express grammars in a high level language like Lua. It also supports nested grammars nicely, but due to the completely stateless approach taken by vis this is currently not fully exploited. As an added bonus there were already ~100 lexers (of admittedly varying quality) available for use.
Some things are just more convenient/efficient to do in C. As an example the mark handling[1] used to represent cursors/selection relies on pointer arithmetic.
Other things like the syntax highlighting are implemented in Lua which is high level but still has low resource usage.
And yes part of the choice is also philosophical. I consider an editor a core system tool which should have minimal dependencies.
A mark could just be an offset inside the Text buffer instead of a direct pointer. Yes, dereferencing a mark would be slower, but do you really think that matters? I mean even if you did that 1M times/sec there wouldn't be a noticeable difference to the user.
On the other hand, look at array.c. Or buffer.c. Or map.c. Or all the manual linked list management stuff. Or all the other logic that would be so much simpler with a more functional language. I mean, sure - there's an unique feeling to being so close to the metal, and that's fine, but if you want to build something new reinventing the wheel for the millionth time is a waste of time.
edit: there are compiled higher-level languages too (i.e. Haskell). If you want minimal dependencies, it doesn't really matter how the binary got built.
The nice thing about the pointer as mark thingy is that while the offset from the start of the file might change when something is inserted before it, the pointer will remain the same.
Anyway this was just an example. I agree that higher level languages (including functional ones) have their own merit. If I would start again today I might consider Rust. But again the LLVM dependency seems kind of scary.
Again for me an ideal base system is built upon a kernel, libc (musl), C compiler (cparser/libfirm), coreutils (sbase/ubase, toybox, busybox), editor (vis) etc. Writing a C compiler is non-trivial, but doable. Creating a C++ compiler on the other hand ...
A self contained (including terminfo entries etc), statically linked vis binary weights in at around ~800K. This allows usage in resource constrained sytems, I don't think the same would be possible using e.g. Haskell.
As the maintainer of the linked vis project, I can confirm that the provided SSH access to the CI environment is very convenient and saves a lot of time.
I would also like to see the NetBSD support completed. It turns out that up until last week, we missed a #define to compile out of the box[1].
[1] https://git.sr.ht/~martanne/vis/commit/01eef9f87b72b0c147367...