An average guy's IT (mis)adventures

It's been a few days now that I'm experiencing downtimes at night, early mornings.

When I wake up, connect to the VPS and attach to tmux, I am welcomed by these messages in console:

        Message from syslogd@pandora at Nov 3 05:37:13 ...
        kernel:[1586232.350737] Dazed and confused, but trying to continue

        Message from syslogd@pandora at Nov 3 05:37:24 ...
        kernel:[1586235.049143] Uhhuh. NMI received for unknown reason
        30 on CPU 1.

        Message from syslogd@pandora at Nov 3 05:37:24 ...
        kernel:[1586235.049145] Dazed and confused, but trying to continue

        Message from syslogd@pandora at Nov 3 05:37:55 ...
        kernel:[1586273.642163] watchdog: BUG: soft lockup - CPU#2 stuck
        for 27s! [dockerd:526408]

        Message from syslogd@pandora at Nov 3 05:38:00 ...
        kernel:[1586278.545172] watchdog: BUG: soft lockup - CPU#1 stuck
        for 24s! [systemd-journal:257]

        Message from syslogd@pandora at Nov 3 05:38:02 ...
        kernel:[1586281.187611] watchdog: BUG: soft lockup - CPU#3 stuck
        for 35s! [lua5.4:1702]

There's no need to say that when this happens, the server is completely frozen and doesn't respond to anything.

I already contacted support, but they didn't investigate at all, I believe. They manually restarted my VPS once and did some pings and connection tests (VNC, SSH) afterwards.. "everything is working fine!"

This last Saturday I was up when it happened, so I did a mtr from my PC to the VPS's IP and logged it, then I sent another email with the output to support.. Still waiting for them to reply, I guess tomorrow (Monday).

Friends like lorenzo and shai are having difficulties too, with the same provider, so I'm not imagining things.

Well, that's all I got to say, will keep you posted if any news.

I am most productive at night, when there is finally silence - apart from background music. The last couple hours of this Wednesday brought some nice additions to the @wpn support MUC:

• chat is not XMPP only anymore, but it got bridged to Matrix, via "matterbridge".
• a bot is now sending "BOFH excuses" to the chat, once a day.

XMPP MUC can be reached from: https://invite.woodpeckersnest.space/#status@chat.woodpeckersnest.space?join

Matrix room can be reached from: https://matrix.to/#/#wpn:matrix.org

Bridged chat means that everything being written in any of those rooms, gets relayed to the other by bots: there are in fact 2 bots connected to the 2 rooms, one is an XMPP bot and the other is a Matrix one, operated by "matterbridge", as I said earlier.

About "BOFH excuses", they are some fortune cookies from the relative Linux package, which are all about "Bastard Operators From Hell" excuses.. They are some short funny sentences about IT which an imaginary Operator (sysop, sysadmin, you name it) could say as excuse to justify issues.

Example?

BOFH excuse #266:
All of the packets are empty.

I have the same script running on gemini, at gemini://woodpeckersnest.space/cgi-bin/bofh

Basically it's some python spaghetti which runs the fortune command, does some formatting to the resulting text and sends it over gemini, or in the case of @wpn chat, over webhook to one other bot, which then posts its content to XMPP. It's more complicated to explain than do it 😀

P.S. @wpn support MUC is not publicly listed, so even if I'm posting links which may be used to join it, it should be treated as limited to @wpn users only.

gemlog