Re: Migrating the landing page

[Volkan Yazıcı]

Great initiative to revamp the Logging Services landing page! Go for it!
But don't change JBake and stick to AsciiDoc, please.

*Summary:*

   1. JBake is great, reproducible, and familiar
   2. "INFRA support" is not a valid argument
   3. Markdown is a dead end
   4. The future should ideally not be individual websites

*JBake is feature-wise on par with alternatives*

There are a plethora of static site generators; Jekyll, Pelican, Hugo,
JBake, etc. Almost every language ecosystem has its own take on it. But
they all boil down to a couple of basic conveniences:

   - pick a typesetting format of your preference (AsciiDoc, Markdown, etc.)
   - generate pages with a templating engine of your preference (Velocity,
   FreeMarker, Handlebars, etc.)
   - compile it to a bunch of static HTML files

Some even include batteries:

   - built-in plugins (sitemap, tagging/labeling, search, etc.)
   - preview (so you don't even need an HTTP server of yours)
   - hot reload (so you don't need to compile it manually every time)

JBake excels in all these areas.

> docker run --rm -p 4000:4000 --mount type=bind,src=$PWD,dst=/root/build
--mount type=volume,dst=/root/build/node_modules -it
apache/privacy_apache_org serve --watch --incremental

Uh... That hurts. This is how you can achieve the same in JBake: `./mvnw
jbake:watch`.

*The unspoken killer feature: reproducibility*

I have been tinkering with static site generators for more than a decade –
I love them. I don't know of your experience, but anything beyond Java
sucks a big time when it comes to reproducibility. You cannot run a Ruby
application written 10 years ago on a modern operating system. Many Ruby
libraries depend on system libraries that either are not shipped with the
distro anymore or broke the ABI in the past. I need to use this
hand-crafted `Dockerfile`
 running on
Ubuntu `14.04` with a particular constellation of Ruby dependencies so that
I can install a version of `nanoc` that compiles my website. It is an
operational nightmare.

But let's talk about Java and JBake: `./mvnw jbake:watch`. This only
requires the host to have a decent OS, shell, and JDK. That is all. No
more, no less. Docker? Nah. Python? Nah. Some weird OS package? Nah. I can
confidently say this command will highly likely run without a single line
of change for several decades. Given its historical track record, I don't
think any other alternative can beat Java in this area and it is of
uttermost importance given how hard it is for the PMC to afford time on the
website.

*The argument of familiarity*

90% of Logging Services committers are Java developers. That is where our
expertise lies in. If you throw at them a `pom.xml` and a bunch of AsciiDoc
files in `src/site/asciidoc`, without blinking an eye, they would correctly
guess what to do. Moving away from this safe zone to uncharted territories,
in particular, without factually justifying the rationale, will simply do
more harm than benefit, if there is any at all.

*The argument of "INFRA supports Jekyll and Pelican"*

What does that support exactly entail? You don't need to compile the site
and push changes to a particular branch that INFRA monitors to serve the
actual domain? It is a dozen lines of GitHub Actions workflow YAML. I
volunteer to do this.

ASF INFRA serves projects by providing infrastructural functions that
didn't exist... decades ago? Many of its offerings are superseded by far
better alternatives in platforms like GitHub, GitLab, etc.

*Markdown-vs-AsciiDoc*

If you look close (and long?) enough to a Markdown-based document
collection, you will notice HTML tags, everywhere. Wait a second? Why did
we decide to use Markdown in the first place? To avoid manually typing
HTML! Yup. The moment you want to do formatting that is slightly
sophisticated (putting a code block under a bullet, annotating source code,
admonitions, etc.) Markdown collapses. Hence, people reach out to sprinkle
HTML into their Markdown. This makes templating impossible in the long run,
since every single hand-written HTML will have its own style, formatting,
structure, etc. which totally defeats the purpose of using a unified markup
language. Hence, please use a markup language that would suffice the
formatting needs of a technical document. Given its rich feature set,
wealthy ecosystem, and large community, I doubt if there is an alternative
here besides AsciiDoc.

*The future of Logging Services websites*

In its current state, we have several projects erecting their own websites
by tooling of their preferences: Log4cxx uses Doxygen, Log4j Scala API uses
`asciidoc-maven-plugin`, Log4j 2 and Log4j Kotlin API uses
`maven-site-plugin`, etc. Such a diverse ecosystem requires significant
maintenance investment. Maintenance of not just the tooling, but also
styling. In my opinion, ideally, projects should simply provide a set of
global pages (about, support, etc.) in their `main` branch and

Re: [Chainsaw] Thoughts on moving forward

[Robert Middleton]

I do know that there are a bunch of settings to hide things that aren't
useful, I'm just saying that even hiding things I still don't find it super
useful.  The table view that exists is theoretically useful, but I find
that it likes to cut off information because it is, well, a table.

The VfsLogFilePatternReceiver I have never gotten to work properly.  I
think it's good and needed, but it definitely needs a user-friendly way of
selecting a log file and viewing a live example of how it will parse the
log file.  Although for my purposes, we don't actually write to a log file,
so it's a bit of a moot point.

The next question of course is what do I feel would be better?  I spent
some time working on that tonight as a proof of concept, and here's what I
came up with(with chainsaw in the background for reference):
[image: Screenshot from 2023-09-25 22-12-07.png]

Note: this picture is from my VM, so it's definitely squashed since it was
running at a weird resolution so I could take the screenshot.  But this
shows a good point: in this screenshot, only about 60% of the UI is
actually used to view logs.  From the top there's the GNOME title bar,
title bar, file menu bar, toolbar, tabs, search bar, column headers, and
then we finally get to what chainsaw is supposed to do - display logs.

What I've done is intentionally a proof of concept at the moment, but it
provides a much simpler view.  Think of it like viewing logs on the
terminal, but adding in the capability to do more advanced operations(e.g.
what chainsaw does with the right-click context menu).  The GUI is at the
moment just a JTextPane instead of a table, so that fields do not get cut
off(like they currently do in the table) and it can better scroll like a
real terminal when new messages come in.  Right-clicking will open up a
pop-up menu contextually depending on what it is you have clicked on(the
time, logger name, level, or the message).

Anyway, that's what I personally feel would be more useful, but I would
love to hear some other ideas.

-Robert Middleton

On Sat, Sep 23, 2023 at 11:35 PM Scott Deboy  wrote:

> Please review the various display setting controls. Most of what you
> mention can be toggled from visible to hidden.
>
> VfsLogFilePatternReceiver does exactly what you're describing. Allowing
> live remote tailing of logs over an ssh accessible path.
>
> You control if these logs end up in separate tabs or the same tab via the
> routing expression in preferences.
>
> We should slack sometime so I can go over the main features and what was
> nuked in the log4j1 removal path and what of those are worth restoring.
>
> Scott
>
> On Sat, Sep 23, 2023, 6:08 PM Robert Middleton 
> wrote:
>
> > Since Scott has said that he would help with maintenance and Rob T has
> also
> > indicated that he would perhaps help, this is my view of the current
> status
> > of Chainsaw and what I feel its current deficiencies are.
> >
> > Current status: Master builds and mostly works.  The last thing that I
> had
> > been working on was updating the config files in order to remove xstream
> > and better standardize on using commons configuration.  I think some of
> the
> > configuration settings don't save/load correctly, but some do.  All of
> > log4j1 has been removed.  Certain features have been removed too that
> were
> > largely dependent on log4j1.
> >
> > What I feel would be useful for Chainsaw: For me, I do a lot of embedded
> > work.  Most of the log files that we have at my current job just go to
> > syslog on our device(syslog is provided by busybox).  So viewing logs is
> a
> > matter of SSHing into the system and just reading from the buffer.
> Having
> > a utility running on a separate computer that lets me see the
> > logs(especially if it can connect automatically to a device) could be
> very
> > useful.  Specific potential use-case: at my last job, I wrote a quick log
> > viewing utility that would correlate log messages between two separate
> > devices.  This was needed because one device was embedded that logged out
> > the serial port, the other was Linux and would log over the network, but
> > neither had reliable time.
> >
> > Current limitations that I find with Chainsaw: The current GUI is not
> very
> > useful.  A large portion of the screen is taken up by toolbars/context
> > information that I generally don't find useful.  I think most of the
> > features that are in the GUI are very useful(for example, being able to
> > trace messages and add matches) but is limited by the fact that I only
> see
> > a small portion of the context at a time.  In my mind, an ideal solution
> > would be to get rid of the toolbars as much as possible and to focus more
> > on the log messages like you would see in a terminal, but still having
> the
> > capability to right-click on a message/message components and investigate
> > individual messages or flag them appropriately.
> >
> > Perhaps the best way to organize this would be to have a logical split i

Re: [Chainsaw] Thoughts on moving forward

[Scott Deboy]

Your screenshot didn't come through.

My mental model is usually:

* Hide logs from packages you know aren't relevant  - the tree-based
(package-level) filtering and focus-on actions in the tree are super
helpful for doing that quickly
* Once you've filtered the view down to the set of packages that may be
relevant, hide logs that are getting in the way - via the 'always hide'
expression box, available at the bottom of the tree view
* Use filter expressions to further narrow results
* Annotate those filtered results (the 'marker' column is editable)
* Clear the filter and search the remaining results, annotating if needed
* Optionally save off the logs and share with other devs (they can import
your filtered logs)

All of this and a lot more is possible with the existing set of features.

UI-wise I agree, less is more most of the time. You don't need to parse the
logs, you can put the entire log line in the 'msg' field, but that prevents
you from using some expression support facilitated by the parsing, such
as "level > INFO" or "exception exists".

On Mon, Sep 25, 2023 at 7:28 PM Robert Middleton 
wrote:

> I do know that there are a bunch of settings to hide things that aren't
> useful, I'm just saying that even hiding things I still don't find it super
> useful.  The table view that exists is theoretically useful, but I find
> that it likes to cut off information because it is, well, a table.
>
> The VfsLogFilePatternReceiver I have never gotten to work properly.  I
> think it's good and needed, but it definitely needs a user-friendly way of
> selecting a log file and viewing a live example of how it will parse the
> log file.  Although for my purposes, we don't actually write to a log file,
> so it's a bit of a moot point.
>
> The next question of course is what do I feel would be better?  I spent
> some time working on that tonight as a proof of concept, and here's what I
> came up with(with chainsaw in the background for reference):
> [image: Screenshot from 2023-09-25 22-12-07.png]
>
> Note: this picture is from my VM, so it's definitely squashed since it was
> running at a weird resolution so I could take the screenshot.  But this
> shows a good point: in this screenshot, only about 60% of the UI is
> actually used to view logs.  From the top there's the GNOME title bar,
> title bar, file menu bar, toolbar, tabs, search bar, column headers, and
> then we finally get to what chainsaw is supposed to do - display logs.
>
> What I've done is intentionally a proof of concept at the moment, but it
> provides a much simpler view.  Think of it like viewing logs on the
> terminal, but adding in the capability to do more advanced operations(e.g.
> what chainsaw does with the right-click context menu).  The GUI is at the
> moment just a JTextPane instead of a table, so that fields do not get cut
> off(like they currently do in the table) and it can better scroll like a
> real terminal when new messages come in.  Right-clicking will open up a
> pop-up menu contextually depending on what it is you have clicked on(the
> time, logger name, level, or the message).
>
> Anyway, that's what I personally feel would be more useful, but I would
> love to hear some other ideas.
>
> -Robert Middleton
>
> On Sat, Sep 23, 2023 at 11:35 PM Scott Deboy 
> wrote:
>
>> Please review the various display setting controls. Most of what you
>> mention can be toggled from visible to hidden.
>>
>> VfsLogFilePatternReceiver does exactly what you're describing. Allowing
>> live remote tailing of logs over an ssh accessible path.
>>
>> You control if these logs end up in separate tabs or the same tab via the
>> routing expression in preferences.
>>
>> We should slack sometime so I can go over the main features and what was
>> nuked in the log4j1 removal path and what of those are worth restoring.
>>
>> Scott
>>
>> On Sat, Sep 23, 2023, 6:08 PM Robert Middleton 
>> wrote:
>>
>> > Since Scott has said that he would help with maintenance and Rob T has
>> also
>> > indicated that he would perhaps help, this is my view of the current
>> status
>> > of Chainsaw and what I feel its current deficiencies are.
>> >
>> > Current status: Master builds and mostly works.  The last thing that I
>> had
>> > been working on was updating the config files in order to remove xstream
>> > and better standardize on using commons configuration.  I think some of
>> the
>> > configuration settings don't save/load correctly, but some do.  All of
>> > log4j1 has been removed.  Certain features have been removed too that
>> were
>> > largely dependent on log4j1.
>> >
>> > What I feel would be useful for Chainsaw: For me, I do a lot of embedded
>> > work.  Most of the log files that we have at my current job just go to
>> > syslog on our device(syslog is provided by busybox).  So viewing logs
>> is a
>> > matter of SSHing into the system and just reading from the buffer.
>> Having
>> > a utility running on a separate computer that lets me see the
>> > logs(especial

Re: [Chainsaw] Thoughts on moving forward

[Scott Deboy]

Here's how it looks if you hide the tree - note the search results are at
the bottom. If you're not searching, that search area is hidden as well.

Because it's parsing the log, you can filter on logger, level, the message
- any field resolved during parsing.

The annotated lines show up as green by default.

The UI supports multiple tabs (log sources) and will live tail for you as
well.

Note the search results show up as blue in the log panel, and the gutter on
the right displays the individual line colors (black is a search match).



On Mon, Sep 25, 2023 at 8:46 PM Scott Deboy  wrote:

> Your screenshot didn't come through.
>
> My mental model is usually:
>
> * Hide logs from packages you know aren't relevant  - the tree-based
> (package-level) filtering and focus-on actions in the tree are super
> helpful for doing that quickly
> * Once you've filtered the view down to the set of packages that may be
> relevant, hide logs that are getting in the way - via the 'always hide'
> expression box, available at the bottom of the tree view
> * Use filter expressions to further narrow results
> * Annotate those filtered results (the 'marker' column is editable)
> * Clear the filter and search the remaining results, annotating if needed
> * Optionally save off the logs and share with other devs (they can import
> your filtered logs)
>
> All of this and a lot more is possible with the existing set of features.
>
> UI-wise I agree, less is more most of the time. You don't need to parse
> the logs, you can put the entire log line in the 'msg' field, but that
> prevents you from using some expression support facilitated by the parsing,
> such as "level > INFO" or "exception exists".
>
> On Mon, Sep 25, 2023 at 7:28 PM Robert Middleton 
> wrote:
>
>> I do know that there are a bunch of settings to hide things that aren't
>> useful, I'm just saying that even hiding things I still don't find it super
>> useful.  The table view that exists is theoretically useful, but I find
>> that it likes to cut off information because it is, well, a table.
>>
>> The VfsLogFilePatternReceiver I have never gotten to work properly.  I
>> think it's good and needed, but it definitely needs a user-friendly way of
>> selecting a log file and viewing a live example of how it will parse the
>> log file.  Although for my purposes, we don't actually write to a log file,
>> so it's a bit of a moot point.
>>
>> The next question of course is what do I feel would be better?  I spent
>> some time working on that tonight as a proof of concept, and here's what I
>> came up with(with chainsaw in the background for reference):
>> [image: Screenshot from 2023-09-25 22-12-07.png]
>>
>> Note: this picture is from my VM, so it's definitely squashed since it
>> was running at a weird resolution so I could take the screenshot.  But this
>> shows a good point: in this screenshot, only about 60% of the UI is
>> actually used to view logs.  From the top there's the GNOME title bar,
>> title bar, file menu bar, toolbar, tabs, search bar, column headers, and
>> then we finally get to what chainsaw is supposed to do - display logs.
>>
>> What I've done is intentionally a proof of concept at the moment, but it
>> provides a much simpler view.  Think of it like viewing logs on the
>> terminal, but adding in the capability to do more advanced operations(e.g.
>> what chainsaw does with the right-click context menu).  The GUI is at the
>> moment just a JTextPane instead of a table, so that fields do not get cut
>> off(like they currently do in the table) and it can better scroll like a
>> real terminal when new messages come in.  Right-clicking will open up a
>> pop-up menu contextually depending on what it is you have clicked on(the
>> time, logger name, level, or the message).
>>
>> Anyway, that's what I personally feel would be more useful, but I would
>> love to hear some other ideas.
>>
>> -Robert Middleton
>>
>> On Sat, Sep 23, 2023 at 11:35 PM Scott Deboy 
>> wrote:
>>
>>> Please review the various display setting controls. Most of what you
>>> mention can be toggled from visible to hidden.
>>>
>>> VfsLogFilePatternReceiver does exactly what you're describing. Allowing
>>> live remote tailing of logs over an ssh accessible path.
>>>
>>> You control if these logs end up in separate tabs or the same tab via the
>>> routing expression in preferences.
>>>
>>> We should slack sometime so I can go over the main features and what was
>>> nuked in the log4j1 removal path and what of those are worth restoring.
>>>
>>> Scott
>>>
>>> On Sat, Sep 23, 2023, 6:08 PM Robert Middleton 
>>> wrote:
>>>
>>> > Since Scott has said that he would help with maintenance and Rob T has
>>> also
>>> > indicated that he would perhaps help, this is my view of the current
>>> status
>>> > of Chainsaw and what I feel its current deficiencies are.
>>> >
>>> > Current status: Master builds and mostly works.  The last thing that I
>>> had
>>> > been working on was updating the config files in