Helios64 reloaded: Armbian Trixie w/ stock Samba

Back in 2022 I wrote up my Openmediavault & Nextcloud on Armbian Server Helios64 setup. It served me well. But it aged badly: Debian 10 Buster, Armbian kernel 4.4, OMV 5, NextcloudPi w/ PHP 7.4 from the Sury repo. When the PHP2Shell vulnerabilities (CVE-2026-109618/109619) came along, there simply was no way to patch: the Buster repos are gone (404), and so is any PHP 7.x update. 💀 (Sure, I could have compiled a patched PHP 7.4 myself—on a Buster box w/o security updates for anything else. No thanks.)
Time for a clean slate. My goals:
- Armbian on the eMMC, as close to stock as possible—every deviation has to earn its place
- the RAID and all SMB shares stay exactly as they are (same names, paths, users, even the same SMB passwords)
- Nextcloud goes—w/o a replacement on this box. Honestly, I only ever used it for calendars and contacts, and those are going to Radicale on another ARM box, a Raspberry Pi. Separate post on that one. 🤓
- no OMV either: plain Debian packages, w/ Cockpit as dashboard
Kobol is long gone (RIP 🪦), but Armbian still builds images for the Helios64—community supported, but they boot. Mostly. More on that below. 🙈
Like with the MacBook, I did this pair-debugging w/ Claude Code—over SSH, and over the serial console when SSH wasn’t an option.
Before touching anything
I did NOT do a full backup of the 2.3 TB on the RAID (RAID10 out of 5× 4 TB IronWolf, mdadm → LVM → ext4). The plan was to never write to it, only re-mount it. What I did instead:
- SMART on all five disks: PASSED, zero reallocated/pending/uncorrectable sectors
- a full RAID check (
echo check > /sys/block/md0/md/sync_action): ~13 h,mismatch_cnt0 - calendars (.ics) and contacts (.vcf) exported straight from the Nextcloud database, plus a
mysqldump—and a final export right before shutting Nextcloud down, b/c the calendar had of course changed in the meantime /etc, OMV’sconfig.xml,mdstat/fstab/lsblk, package list- later on: a full image of the old eMMC (see below)
SD card first
Armbian 26.8.1 Trixie minimal (kernel 6.18.43) on an SD card, checksum + GPG verified. The Helios64 boots SPI → eMMC → SD, and jumper P10 skips the eMMC. TIL: w/ P10 set, the eMMC is invisible to Linux, too (mmc1: Failed to initialize a non-removable card)—not just to the boot ROM.
Booted right away. The RAID showed up on its own as md127 (auto-read-only), LVM stayed inactive w/o lvm2 installed. Good: nothing touched it.
Getting Armbian onto the eMMC
The Kobol wiki assumes an empty eMMC. Mine had a bootloader on it, so the Helios64 always booted from the eMMC, SD card or not. What worked:
- From the still-running old system: back up the first 16 MiB of the eMMC, then wipe only the bootloader area (sectors 64–32767; the first partition starts at 32768). With no bootloader on the eMMC, the boot ROM falls back to SD. No jumper needed.
- SD card w/ Kobol’s
helios64_sdcard_u-boot-only.img.xz. That’s a U-Boot that runsums mmc 0and hands the eMMC over USB-C as a mass storage device. - Full image of the old eMMC to the Mac first (15.6 GB, 3.9 GB gzipped). UMS is slow: ~6–7 MB/s, ~40 min. ☕
- Then flash Armbian w/ the Armbian Imager straight onto the eMMC.
Two gotchas:
- My first USB-C cable was charge-only. Nothing showed up on the Mac, not even the serial chip. 🤦♂️
- The serial console (FTDI FT230X, 1500000 baud) lives on the same USB-C port. It vanishes from the Mac as soon as Linux takes over the port. Which is actually a good sign: the kernel is running.
The boss level: random kernel panics at boot
Installed, booted… and then roughly every other boot died ~3 s in. Red LED blinking, or the blue LED steady on w/o heartbeat and the fan going full blast. On the serial console:
SError Interrupt on CPU…
… rockchip_pcie_rd_conf …
Kernel panic - not syncing: Asynchronous SError Interrupt
That’s the PCIe bus scan—where the JMB585 SATA controller sits. Score w/ 6.18.43: 2 good boots out of 7.
What I tried, and what didn’t help:
- PCIe PERST overlay from the Armbian forum (
ep-gpioson&pcie0, present in the mainline DT, missing in Armbian’s). Loaded fine, still panics. - Downgrade to 6.18.10, which another forum user reported as stable. The very first warm reboot failed at the exact same spot, this time as
synchronous external abort→ a silent freeze instead of a panic. pcie_rockchip_host.bus_scan_delay=1000. Armbian’s kernel carries the 2023 patch “configurable delay for Rockchip PCIe bus scan”. It kicks in (wait 1000 ms (from command-line) before bus scan)—and the panic simply comes a second later.
So it’s not the kernel version. It’s a known RK3399 hardware quirk (RockPro64 and Rock Pi 4 are affected as well): a config read on a PCIe device that isn’t ready gets reported to the CPU as an abort. Timing-dependent, hence random.
The recipe: don’t prevent it, recover from it
The crash only happens during the PCIe scan at boot, before the SATA driver loads. The disks are never touched. So instead of fighting the hardware, I let the box heal itself and otherwise stay on stock Armbian: current kernel, no overlay, no apt-mark hold.
Exactly two deviations from stock:
1. Panic → reboot in /boot/armbianEnv.txt:
extraargs=panic=10 oops=panic
oops=panic matters: it turns the silent freeze (“synchronous external abort”) into a panic as well, and panic=10 reboots 10 s later. Armbian’s stock watchdog daemon catches whatever is left—no need for your own watchdog config. (I added one at first. It stole /dev/watchdog from the stock daemon. Removed. 🙃)
2. Reboot if the SATA controller is missing. Rarely the scan doesn’t crash but fails differently (PCIe link training gen1 timeout! … probe failed with error -110). Linux comes up fine—w/o disks. A tiny oneshot service catches that, /usr/local/sbin/helios64-pcie-check:
#!/bin/sh
# Helios64: reboot if the SATA controller (JMB585 on PCIe) didn't come up at boot.
# At most 3 attempts in a row.
C=/var/lib/helios64-pcie-check.count
if [ "$(cat /sys/bus/pci/devices/0000:01:00.0/vendor 2>/dev/null)" = "0x197b" ]; then
rm -f "$C"
exit 0
fi
n=$(cat "$C" 2>/dev/null || echo 0)
if [ "$n" -ge 3 ]; then
logger -t helios64-pcie-check "SATA controller still missing, giving up after $n reboots"
exit 1
fi
echo $((n + 1)) > "$C"
logger -t helios64-pcie-check "SATA controller missing, reboot $((n + 1))/3"
systemctl reboot
/etc/systemd/system/helios64-pcie-check.service:
[Unit]
Description=Helios64: reboot if the SATA controller is missing
After=local-fs.target
Before=mdmonitor.service smbd.service
[Service]
Type=oneshot
ExecStart=/usr/local/sbin/helios64-pcie-check
[Install]
WantedBy=multi-user.target
systemctl enable helios64-pcie-check, done.
Result: two rounds of 10 reboots each, 20/20 back on their own, all five disks every time. In between, 4–7 caught panics—some boots took one or two extra laps. 💪
Tip: journalctl --list-boots (the journal is persistent on Armbian) shows the panicked boots as gaps between boots. Way more reliable than the serial console on warm reboots, b/c the FTDI chip doesn’t always re-enumerate in time.
From OMV to plain Debian
With OMV gone, the NAS part is a handful of config files:
mdadm+lvm2. In/etc/mdadm/mdadm.confthe array line,MAILADDR root, thenupdate-initramfs -u→ back tomd0instead ofmd127:ARRAY /dev/md0 metadata=1.2 name=helios64:store UUID=…fsck.ext4 -nfirst (clean), then/etc/fstabw/ the same mount point OMV used, so all paths stay the same. OMV’s quota options and bind mounts are gone:LABEL=media /srv/dev-disk-by-label-media ext4 defaults,nofail,user_xattr,acl 0 2the users w/ the same UIDs/GIDs as before (my user 1000, primary group
usersagain; a second user w/nologin) → owners and permissions on the RAID simply match
Samba. The share sections came over 1:1 from OMV’s generated smb.conf (create mask 0664, directory mask 0775, inherit acls, hide dot files, valid users…). [global] is slim: workgroup, standalone, fruit:aapl = yes, multicast dns register = yes. Plus wsdd2 (Windows) and avahi-daemon (Finder).
The nicest trick of the whole reinstall: take the SMB passwords along. I unpacked the old eMMC image onto the RAID, mounted its root partition read-only (losetup -r -o 16777216, mount -o ro,noload) and copied /var/lib/samba/private/passdb.tdb and secrets.tdb over. The clients reconnected w/ their stored credentials, no password prompt. 🤓
Tailscale serve: one HTTPS name for everything
Tailscale from the official repo. Before tailscale up I deleted the old node in the admin console and afterwards set its old IP again → name and IP 1:1.
Then tailscale serve does what the Apache vhosts + cert script did in 2022:
tailscale serve --bg --https=443 --set-path=/cockpit http://127.0.0.1:9090/cockpit
→ https://helios64.<tailnet>.ts.net/cockpit/ is the dashboard, w/ a valid Let’s Encrypt cert. In the LAN it’s reachable directly as well.
TIL: an nftables rule on tailscale0 does nothing for tailscale serve. tailscaled accepts those connections itself, in userspace, before netfilter ever sees them. Access control belongs in the tailnet policy (ACLs). You can check what actually applies on the node w/ tailscale debug netmap (→ PacketFilter).
Cockpit
Stock Debian, w/o the recommends: cockpit-ws cockpit-system cockpit-bridge cockpit-storaged. To run under a path behind tailscale serve, /etc/cockpit/cockpit.conf:
[WebService]
UrlRoot = /cockpit
Origins = https://helios64.<tailnet>.ts.net https://helios64:9090 https://helios64.local:9090
ProtocolHeader = X-Forwarded-Proto
AllowUnencrypted = true
Two things Cockpit complained about afterwards:
- “PCP missing for metrics history”: there’s no separate
cockpit-pcpanymore (Cockpit 337), the PCP channel is part ofcockpit-bridge.apt install pcp python3-pcp+systemctl enable --now pmcd pmlogger. History starts from then on,pmloggerkeeps 14 days. Error initializing module 'lvm2'in the storage log:apt install udisks2-lvm2, otherwise Cockpit only half-understands the LVM on the RAID. Thebtrfs/iscsilines stay—harmless (no btrfs here, and there’s noudisks2-iscsiin Debian anyway).
Careful: the Storage page can also delete RAIDs and volumes. Read only. 😬
Mail, SMART & Docker
- Mail:
msmtp-mta+bsd-mailx,/etc/msmtprcw/ my provider’s SMTP (587, STARTTLS), password in a root-only file viapasswordeval,/etc/aliasessends root to me.mdadm --monitor --scan --test --oneshotsends a test mail per array. - smartd: short self-test every Sunday, long one on the first Saturday of the month, temperature warnings at 45/55 °C, mails to root.
- Docker:
docker.iofrom Debian—and on Trixiedocker-cliis a separate package. Without it there’s nodockercommand at all. 🤷♂️hello-worldruns, no workload yet. - Plus
etckeeper(/etcin git) andunattended-upgradesfor the Debian origins. Armbian kernel updates I do by hand.
Log noise you can ignore
On a healthy box, journalctl -p err still shows a few things, all from stock packages:
- alsa-restore udev rule:
GOTO="alsa_restore_std" has no matching label - watchdog:
cannot set scheduler systemd-networkd-wait-onlinetimeouts when it runs after boot (e.g. during package installs): the 2.5 GbE port has no cable, and Armbian’s netplan config does DHCP on all interfaces. At boot it’s done after ~5 s, so I left it alone.
Bottom line
Boot in ~19 s (when it doesn’t take an extra lap 😉), SoC at ~41 °C, RAID [UUUUU]. Gone are OMV, Nextcloud, MariaDB, PHP and Apache. Instead it’s Samba, Cockpit and a few config files I actually understand. And the only Helios64-specific bits are one kernel parameter line and a tiny shell script.
The Helios64 lives on. 🤞