Kirby and httpd(8) on OpenBSD—also static site generation

I've briefly mentioned the tech stack that I'm using for this website. In this post I'll go into more detail on the actual configuration that I'm using.

Here's a list of the desired functionality (a.k.a "dreamcode"):

• use Let's Encrypt for SSL Certificate (httpS)
• use solely httpd(8)/ httpd.conf(5) as web server on OpenBSD
• use Kirby with single-domain license
• publishing by means of static site generation (SSG) for html, image/ media assets and feeds (both, RSS/ XML and JSON)
• ability to preview new and changed (=draft) contents from Kirby panel, prior to publish
• clean, minimalistic, mobile-friendly design —› Kirby 3 theme
• only expose generated static site to the public
• do not expose Kirby's PHP handler (incl. API) to the public, i.e. require authentication

Package Installation

# Let's Encrypt Certbot
pkg_add py3-setuptools
pkg_add certbot

# php-8.1.13
pkg_add php
# /etc/php-8.1.ini
cat /usr/local/share/doc/pkg-readmes/php-8.1
pkg_add php-curl
ln -sf /etc/php-8.1.sample/curl.ini /etc/php-8.1/
pkg_add php-gd
ln -s /etc/php-8.1.sample/gd.ini /etc/php-8.1/
# /etc/php-fpm.conf
rcctl enable php81_fpm
rcctl start php81_fpm

# composer
pkg_add composer

# Let's Encrypt Certbot
certbot certonly \
  --agree-tos \
  --webroot \
  -w /var/www/htdocs/openwebcraft.com/static \
  -m <email> \
  -d openwebcraft.com,www.openwebcraft.com

#certbot renew --force-renewal

httpd(8)

The directory and file structure for the website is as follows:

# /var/www/htdocs/openwebcraft.com
.
|-- composer.json
|-- composer.lock
|-- content
|-- errdocs
|   `-- 404.html -> ../static/404/index.html
|-- kirby
|   `-- […]
|-- public
|   `-- index.php
|-- site
|   |-- blueprints
|   |-- collections
|   |-- config
|   |   |-- .license
|   |   `-- config.php
|   |-- controllers
|   |-- plugins
|   |   |-- static-site-composer
        `-- […]
|   |-- snippets
|   |-- templates
|   `-- […]
`-- static
    |-- error
    |   `-- index.html
    |-- favicon.ico
    |-- feed.json
    |-- feed.rss
    |-- index.html
    |-- kassets
    |   `-- css
    |-- kmedia
    `-- […]
`-- vendor

httpd.conf(5)

Let's get the commodity stuff out of the way: properly redirecting http(s)://(www.)openwebcraft.com to /.

# /etc/httpd.conf

types {
  include "/usr/share/misc/mime.types"
}
ext_addr = egress
server www.openwebcraft.com {
  listen on $ext_addr tls port 443
  tls {
    certificate "/etc/letsencrypt/live/ \
            openwebcraft.com/fullchain.pem"
    key         "/etc/letsencrypt/live/\
            openwebcraft.com/privkey.pem"
  }
  block return 301 "/\
    $REQUEST_URI"
}
server www.openwebcraft.com {
  listen on $ext_addr port 80
  block return 301 "/\
    $REQUEST_URI"
}
server openwebcraft.com {
  listen on $ext_addr port 80
  block return 301 "/\
    $REQUEST_URI"
}

The following server configuration holds rules for specific locations.

server openwebcraft.com {
  listen on $ext_addr tls port 443
  connection { max request body 8388608 }
  root "/htdocs/openwebcraft.com/static"
  directory index "index.html"
  tls {
    certificate "/etc/letsencrypt/live/\
            openwebcraft.com/fullchain.pem"
    key         "/etc/letsencrypt/live/\
            openwebcraft.com/privkey.pem"
  }

  # have httpd serve existing media files
  location found "/kmedia*" { pass }
  # …or rewrite to Kirby's PHP handler 
  location not found "/kmedia*" {
   root "/htdocs/openwebcraft.com/public"
   directory index "index.php"
   authenticate "Kirby" with \
    "/auth/openwebcraft.com/.htpasswd"
   fastcgi {
     socket "/run/php-fpm.sock"
     param SCRIPT_FILENAME \
        "/htdocs/openwebcraft.com/public/index.php"
     param SCRIPT_NAME "/index.php"
    }
  }

  # rewrite path segment to Kirby's PHP handler
  location "/kirby/*" {
   root "/htdocs/openwebcraft.com/public"
   directory index "index.php"
   authenticate "Kirby" with \
    "/auth/openwebcraft.com/.htpasswd"
   fastcgi {
     socket "/run/php-fpm.sock"
     param SCRIPT_FILENAME \
        "/htdocs/openwebcraft.com/public/index.php"
     param SCRIPT_NAME "index.php"
    }
  }

  location "/kapi/*" {
   root "/htdocs/openwebcraft.com/public"
   directory index "index.php"
   authenticate "Kirby" with \
    "/auth/openwebcraft.com/.htpasswd"
   fastcgi {
     socket "/run/php-fpm.sock"
     param SCRIPT_FILENAME \
        "/htdocs/openwebcraft.com/public/index.php"
     param SCRIPT_NAME "index.php"
    }
  }

  errdocs from "/htdocs/openwebcraft.com/errdocs"
}

Kirby

The following utlines the core configuration for Kirby, file path relative to /var/www/htdocs/openwebcraft.com.

Plugins

I felt the need to write a small plugin:

Kirby 3 Static Site Composer. A Kirby 3 plugin for composing a static site. Essentially a wrapper to integrate and trigger suitable community plugins:

• Static Site Generator
• Feed

The plugin can be installed via composer like so—for now dev-main:

composer require matthiasjg/kirby3-static-site-composer:dev-main`

index.php

# public/index.php

<?php
include __DIR__ . '/../kirby/bootstrap.php';
$kirby = new Kirby([
  'roots' => [
    'index'   => __DIR__,
    'site'    => __DIR__ . '/../site',
    'content' => __DIR__ . '/../content',
    'media'   => __DIR__ . '/../static/kmedia'
  ],
  'urls' => [
    'index'  => '/',
    'media'  => '/kmedia',
    'assets' => '/kassets'
  ]
]);
echo $kirby->render();

Site Configuration

# site/config/config.php

<?php
return [
  'servers' => ['httpd'],   # OpenBSD httpd
  'panel' => [
    'slug' => 'kirby' # kirby/* route
  ],
  'api' => [
     'slug' => 'kapi'  # kirby/* route
  ],
  'debug' => false,
  'home'  => 'home',
  # […]
  'matthiasjg' => [
    'static_site_composer' => [
    'endpoint' => 'compose-static-site',
    'output_folder' => '../static',
    'preserve' => ['notes','kassets','kmedia','favicon.ico'],
    'base_url' => '/',
    'skip_media' => true,
    'skip_templates' => [],
    'pages_parent_home_root' => true,
    'preview_url_slug' => 'kirby/preview',
    'feed_formats' => ['rss','json'],
    'feed_description' => 'Latest writing',
    'feedollection' => 'posts',
    'feed_collection_limit' => 10,
    'feed_collection_datefield' => 'published',
    'feed_collection_textfield' => 'text'
     ]
  ]
];

Blueprints

# site/blueprints/site.yml

title: Site
options:
  preview: /kirby/preview/home
# […]
columns:
  - width: 1/1
    fields:
      staticSiteComposer:
        label: Compose Site
# […]

# site/blueprints/pages/default.yml

title: Page
options:
  preview: "/kirby/preview{{page.parent.url}}/{{page.slug}}"
# […]
columns:
  - width: 1/1
    fields:
      staticSiteComposer:
        label: Compose Site
# […]

# site/blueprints/pages/post.yml

title: Post
options:
  preview: "/kirby/preview/{{page.parent.slug}}/{{page.slug}}"
# […]
columns:
  - width: 1/1
    fields:
      staticSiteComposer:
        label: Compose Site
# […]

Web log analysis

Option 1: GoAccess

First—for approach and basic setup—follow along chevybeef's blog post.

For compatibility w/ httpd(8)'s log format — either common or combined — simply uncomment the respective section in /etc/goaccess/goaccess.conf—if not specified, the default is common):

# httpd(8) common log format
date-format %d/%b/%Y
time-format %T %z
log-format %v %h %^ %^ [%d:%t] "%r" %s %b

Here's some background reading on httpd(8) log styles from Stefan Kreutz.

One can test the setup w/ this command:

zcat /var/www/logs/access.log.*.gz | cat /var/www/logs/access.log - | grep -v syslog | goaccess

In order to setup/ automate the scheduled creation/ update of the stats report and to secure the stats/ folder with htpasswd simply follow again chevybeef's blog post.

Note: in case the cron job is not properly executed (i.e. you get an error like "Error occurred at: src/goaccess.c - initializer […] No input data was provided nor there's data to restore.") you'll find the solution in this stackoverflow.

---

Option 2: Webalizer

Some useful hints and config sample to be found on calomel.org.

Adjust /etc/webalizer.conf to your needs and test w/ following command (also goes into crontab):

/usr/local/bin/webalizer >> /dev/null 2>&1

Option 3: How to analyze OpenBSD's httpd access.log with a shell script

A #SimpleElegant shell script to analyze the access logs from httpd. Love this ❤️!

#!/bin/sh

LOGFILE="/var/www/logs/access.log"
RESPONSE_CODE="200"

filters() {
grep $RESPONSE_CODE \
| grep -v "<UNKNOWN>" \
| grep -v "favicon.ico" \
| grep -v "/kapi/" \
| grep -v "/kassets/" \
| grep -v "/kmedia/" \
| grep -v "logfile turned over"
}

filter_response_codes()
{
grep -v "<UNKNOWN>" \
| grep -v "logfile turned over" \
| awk '{print $10}'
}

filter_404_response() {
grep "404"
}

ips() {
awk '{print $2}'
}

pages() {
awk '{print $8}'
}

domain() {
awk '{print $1}'
}

methods() {
awk '{print $7}' | cut -d'"' -f2
}

sort_count() {
sort | uniq -c
}

sort_desc() {
sort -rn
}

top_ten() {
head -10
}

sep() {
echo "=================================================="
}

##
# Actions
##
action_request_ips() {
    echo ""
    echo "Top requests from IPs"
    sep
    cat $LOGFILE \
        | filters \
        | ips \
        | sort_count \
        | sort_desc \
        | top_ten
    echo ""
}

action_request_methods() {
    echo ""
    echo "Count requests methods"
    sep
    cat $LOGFILE \
        | filters \
        | methods \
        | sort_count
    echo ""
}

action_pages() {
    echo ""
    echo "Top requested pages"
    sep
    cat $LOGFILE \
        | filters \
        | pages \
        | sort_count \
        | sort_desc \
        | top_ten
    echo ""
}

action_404() {
    echo ""
    echo "Top requests 404"
    sep
    cat $LOGFILE \
        | filter_404_response \
        | pages \
        | sort_count \
        | sort_desc \
        | top_ten
    echo ""
}

action_response_codes() {
    echo ""
    echo "Response code"
    sep
    cat $LOGFILE \
        | filter_response_codes \
        | sort_count \
        | sort_desc
    echo ""
}

action_request_ips
action_request_methods
action_response_codes
action_pages
action_404

Known Issues

Because there's always room to improve…

• Basic auth prompting occasionaly while saving contents in Kirby Panel.
• For newly created (or updated) contents with images the generated static page needs to be accessed once w/ valid basic auth—to allow for Kirby to generate meda asset.