An average guy's IT (mis)adventures

It's been a few days now that I'm experiencing downtimes at night, early mornings.

When I wake up, connect to the VPS and attach to tmux, I am welcomed by these messages in console:

        Message from syslogd@pandora at Nov 3 05:37:13 ...
        kernel:[1586232.350737] Dazed and confused, but trying to continue

        Message from syslogd@pandora at Nov 3 05:37:24 ...
        kernel:[1586235.049143] Uhhuh. NMI received for unknown reason
        30 on CPU 1.

        Message from syslogd@pandora at Nov 3 05:37:24 ...
        kernel:[1586235.049145] Dazed and confused, but trying to continue

        Message from syslogd@pandora at Nov 3 05:37:55 ...
        kernel:[1586273.642163] watchdog: BUG: soft lockup - CPU#2 stuck
        for 27s! [dockerd:526408]

        Message from syslogd@pandora at Nov 3 05:38:00 ...
        kernel:[1586278.545172] watchdog: BUG: soft lockup - CPU#1 stuck
        for 24s! [systemd-journal:257]

        Message from syslogd@pandora at Nov 3 05:38:02 ...
        kernel:[1586281.187611] watchdog: BUG: soft lockup - CPU#3 stuck
        for 35s! [lua5.4:1702]

There's no need to say that when this happens, the server is completely frozen and doesn't respond to anything.

I already contacted support, but they didn't investigate at all, I believe. They manually restarted my VPS once and did some pings and connection tests (VNC, SSH) afterwards.. "everything is working fine!"

This last Saturday I was up when it happened, so I did a mtr from my PC to the VPS's IP and logged it, then I sent another email with the output to support.. Still waiting for them to reply, I guess tomorrow (Monday).

Friends like lorenzo and shai are having difficulties too, with the same provider, so I'm not imagining things.

Well, that's all I got to say, will keep you posted if any news.

I am most productive at night, when there is finally silence - apart from background music. The last couple hours of this Wednesday brought some nice additions to the @wpn support MUC:

• chat is not XMPP only anymore, but it got bridged to Matrix, via "matterbridge".
• a bot is now sending "BOFH excuses" to the chat, once a day.

XMPP MUC can be reached from: https://invite.woodpeckersnest.space/#status@chat.woodpeckersnest.space?join

Matrix room can be reached from: https://matrix.to/#/#wpn:matrix.org

Bridged chat means that everything being written in any of those rooms, gets relayed to the other by bots: there are in fact 2 bots connected to the 2 rooms, one is an XMPP bot and the other is a Matrix one, operated by "matterbridge", as I said earlier.

About "BOFH excuses", they are some fortune cookies from the relative Linux package, which are all about "Bastard Operators From Hell" excuses.. They are some short funny sentences about IT which an imaginary Operator (sysop, sysadmin, you name it) could say as excuse to justify issues.

Example?

BOFH excuse #266:
All of the packets are empty.

I have the same script running on gemini, at gemini://woodpeckersnest.space/cgi-bin/bofh

Basically it's some python spaghetti which runs the fortune command, does some formatting to the resulting text and sends it over gemini, or in the case of @wpn chat, over webhook to one other bot, which then posts its content to XMPP. It's more complicated to explain than do it 😀

P.S. @wpn support MUC is not publicly listed, so even if I'm posting links which may be used to join it, it should be treated as limited to @wpn users only.

gemlog

Last evening, at around 22:00 CET, Contabo migrated my VPS from old Hub at Nuremberg to the newly built "Hub Europe".

They rebooted it and everything came back up as usual.. I wasn't at home when it happened, didn't even notice anything until I connected and found no tmux session running. Even my phone which was connected to wireguard, quietly continued working and sending notifications 😎

Meanwhile, my friends and I watched "The Creator" on Disney+

Can't really say I do recommend it, but for some casual evening/entertainment, it wasn't half bad.

gemlog

I've been rethinking about the publication license of these blog posts.

Changes I made:

• Some "public" posts were turned to "CC BY-NC-SA"
• Some "all-rights-reserved" posts were turned to "CC BY-NC-SA"

At the moment there are still other public and all-rights-reserved posts.. Most of the public ones are from me, while some are from friends who told me to publish them with such license.

The all-rights-reserved ones are mostly from content I grabbed over the net and published here, where I wasn't able to contact the original authors (so credits are to themselves) and some others are e.g. posts with pictures I took myself.

So.. From now on, all these types of licenses will co-exist and each post will be tagged with the correct/relative one (hopefully).

Disclaimer: If not specified, you can assume it's "CC BY-NC-SA" by Simone "roughnecks" Canaletti

https://creativecommons.org/licenses/by-nc-sa/4.0/

You can try this script on #fediverso at irc.libera.chat, where me, cage, ndo and other friends hang out

bot: "verne", running on @wpn

SearXNG instance: https://search.woodpeckersnest.space/

Thanks to cage for the script and ndo for creating the channel o/

# © cage released under CC0, public domain
# https://creativecommons.org/publicdomain/zero/1.0/

# Date: 16-08-2024
# Version: 0.1

# Package description: do a web search using your searxng instance
# Public ones won't probably work because of "limiter"

# Authorize your channel from the partyline with:
# .chanset +searxng #your-channel

# Do a search
# .search <query> | .search paris (this query goes to default engine)
# .search +<engine> <query> | .search +wp paris (this query goes to
# wikipedia)
# .search !images paris | this query search only paris' images

# List of engines: https://docs.searxng.org/user/configured_engines.html

# tcllib is required

############## configuration directives ############################

# url of the HTTP(S) server of the search engine

set searxconfig(website_url)    "https://example.com/searxng"

# serach command to trigger the search

set searxconfig(cmd)            ".search"

# default search engine

set searxconfig(default_engine) "ddg"

# maximum number of search results printed

set searxconfig(max_results)    3

# time tracker file
# NB: when this script runs any file with the same name within the path in the
# working  directory  (depending of  what  is  considered the  working
# directory of the script) will be erased and overwritten!

set searxconfig(file_millis)    "searx_millis.tmp"

# Minimum search frequency in milliseconds.
# This is  the minimum  time that must  pass between  two consecutive
# search

set searxconfig(max_freq)       30000

############## configuration ends here #############

# tcllib is required
package require csv

setudef flag searxng

if { !([info exists searxconfig(lastmillis)]) } {
    set searxconfig(lastmillis) 0
}

bind pub - $searxconfig(cmd) search:searxng

proc send_message {message} {
    putserv "PRIVMSG $message"
}

proc slurp_file {path} {
    set fp [open $path r]
    set file_data [read $fp]
    close $fp
    return $file_data
}

proc process_csv {csv channel} {
    global searxconfig
    set rows [split $csv "\n"]
    set count 0
    #remove the header
    set rows [lrange $rows 1 [llength $rows]]
    if {[llength $rows] < 1} {
        send_message "$channel :Something gone wrong."
    } else {
        foreach row $rows {
            if {$count < $searxconfig(max_results)} {
                set row_splitted [csv::split $row]
                set title [lindex $row_splitted 0]
                set url   [lindex $row_splitted 1]
                send_message "$channel :$title $url"
                incr count
            } else {
                break
            }
        }
    }
}

proc encode {query} {
    set query [regsub -all { } $query "%20"]
    set query [regsub -all {&} $query "%26"]
    set query [regsub -all {=} $query "%3D"]
    set query [regsub -all {!} $query "%21"]
}

proc get_query_results {engine query} {
    global searxconfig
    set query [encode $query]
    set engine [encode $engine]
    set url "$searxconfig(website_url)search?q=$query&format=csv&engines=$engine"
    ## decomment the line below for debugging purposes
    # putlog $url
    return [exec curl -sS $url]
}

proc get_last_millis { } {
    global searxconfig
    if {[file exists $searxconfig(file_millis)]} {
        set searxconfig(lastmillis) [slurp_file $searxconfig(file_millis)]
    } else {
        set fp [open $searxconfig(file_millis) w]
        puts $fp 0
        close $fp
        get_last_millis
    }
}

proc set_last_millis { } {
    global searxconfig
    set fp [open $searxconfig(file_millis) w]
    puts $fp [clock milliseconds]
    close $fp
}

proc search:searxng {nick host hand chan text} {
    global searxconfig

    if {!([channel get $chan searxng])} {
        send_message "$chan :This script has not been authorized to run in this channel."
        return 0
    }

    set millis [clock milliseconds]

    get_last_millis

    if { [expr $millis - $searxconfig(lastmillis)] > $searxconfig(max_freq) } {
        ## test antiflood superato

        set_last_millis

        set text_splitted [split $text " {}"]
        set engine [lindex $text_splitted 0]
        set text_length [llength $text_splitted]
        set query [lrange $text_splitted 1 $text_length]


        if {![regexp {^\+} $engine]} {
            set engine $searxconfig(default_engine)
            set query $text_splitted
        } else {
            set engine [string range $engine 1 [string length $engine]]
        }

        if {$query == {}} {
            send_message "$chan :Missing search criteria."
        } else {
            set csv [get_query_results $engine $query]
            process_csv $csv $chan
        }

        return 1;
    }
        send_message "$chan :Try again later."
    return 0
}

putlog "SearXNG Loaded"`

gemlog

Yet another small update about gemini.

You can now browse gemini://woodpeckersnest.space even from regular HTTP, here: https://gemini.woodpeckersnest.space/

I've applied some fixes (like) to HTML and CSS (the latter is pretty much the same used by the @wpn onboarding page, but obviously customized). As for accessibility, I think it should work well for desktop and also mobile browsers; CGIs work as well.

The proxy I used is Loxy. I also already opened an issue on their repo for a problem with query strings, still waiting for someone to reply. Apart from that, everything checks out.

gemlog

Hello,

just a brief update on gemini here at @wpn.

I switched TLD from ".eu" to ".space": seemed more appropriate for gemini.

gemini://woodpeckersnest.space/

gemlog

I was running 2 separate apps for shell/XMPP account registration at @wpn, so far..

This night I made some changes to the original code (provided by Schimon) and I got just one app with account choice - meaning that you must choose what account type you want in the form. Shell accounts are for friends only, as it's always been.

As a consequence of that, I shut down the old address for XMPP account onboarding and left only the main one, which is:

https://hello.woodpeckersnest.space/

gemlog

I'm always a bit busy when it comes to pandora (the VPS running WPN: woodpeckersnest.space/eu). I like experimenting new things, fixing/improving existing things.. I cannot stay still 😀

After migrating the homepage to homarr - which took really no time for the initial setup, but a lot of work afterward to fix layouts for mobile devices and non-full-hd screens for desktop PCs - I started messing up with a brand new toy: gemini!!

Not even a week since I installed molly-brown, the actual gemini server, to today, I can count lots of improvements..

• Installed terminal gemini browser client amfora for wpn's shell users and also gtl, a tinylog reader, always for the shell.
• Configured a local tinylog which groups together all wpn's capsuleer tinylogs, so it's easy to follow all of the local server users in one single place; the log is generated by gtl itself, refreshed and published every 5 minutes: can't miss a thing!
• Initially configured gemlog mentions starting from a script by @bacardi55, who is the author of many gemini-related things, like the before mentioned gtl software. When I realized it lacked multi-capsules support, I started modifying it and came up with some spaghetti code, which is working surprisingly well and it was deployed earlier today.

gemini@wpn

Onboarding on WPN didn't go as well as I thought, but at least the first user (hey, Mario, I'm looking at you! :) registered and, I believe, everything is working fine for them! On this topic, the onboarding page was migrated from PHP and Email to Python and XMPP, thanks to my friend Schimon! He also kept the UI pretty much intact, so I think most people who looked at it before and after, wouldn't even notice the changes under the hood.

https://hello.woodpeckersnest.space/

Something else I've been doing was setting up: https://invite.woodpeckersnest.space/

which is a landing page to allow people to join an XMPP MUC or add an XMPP contact from a web interface, which will also guide them in choosing a client for their platform. It's rather simple but very useful at the same time.

The chatmail server was upgraded (more or less) at the beginning of August and running smoothly so far; it got some cool new improvements like automatic account deletion after #amount of days from last login and lots of fixes. Total number of registered accounts, so far, is 117.

https://chatmail.woodpeckersnest.space/

Services which I dismissed include:

• Jitsi Meet (wasn't really using it and it was wasting quite a lot of resources just to be running)
• Isso comments service, which powered the old homepage contact section and also a shaarli instance, which is still running but it's more a private thing, rather than a public one.

One more proposition: from now on, I will be publishing these (B)log posts in both protocols, HTTP here as you're reading and gemini on roughnecks' gemlog. I will be probably publishing less often than usual though, at least in this format, and send more status updates through the tinylog on WPN, the microlog at Station and my fediverse account.

In the next days I will be monitoring how everything goes and relax a bit, if I manage.. Today I didn't feel so good after a few stressful days, too much computing and too less sleep hours - it's 01:40 AM right now, so yeah, tomorrow will be another of “those” days, I guess.

gemlog