Documentation Overview

  1. Operation documentation
    1. Installation
      1. Run UBOS on a PC (64bit)
      2. Run UBOS from a boot stick on a PC (64bit)
      3. Run UBOS in a VirtualBox virtual machine (64bit)
      4. Run UBOS on an Amazon Web Services EC2 virtual server
      5. Run UBOS on Raspberry Pi 5
      6. Run UBOS on ESPRESSObin
      7. Run UBOS in a Linux container on a PC (64bit)
      8. Run UBOS in an aarch64 Linux container
    2. Setting up your first Site and App
    3. Setting up networking and DNS
    4. How to create a website secured by SSL/TLS
    5. How to set up a website as a Tor hidden service
    6. Managing Sites and Apps
    7. Backup and restore
    8. Upgrading and keeping UBOS current
    9. The UBOS Staff
    10. App-specific notes
      1. Reliably send e-mail via Amazon Web Services’ Simple E-mail Service: amazonses
      2. Static website hosting with rsync-based upload: docroot
      3. Notes on Mastodon
      4. Notes on Nextcloud
      5. Notes on Redirect
      6. Notes on Wordpress
    11. Device-specific Notes
      1. ESPRESSObin
      2. Raspberry Pi
    12. Advanced management
      1. Enabling non-standard package repositories
      2. Migrating from one App to another
      3. Pinning resources
    13. Command reference
    14. FAQ, HOWTOs and Troubleshooting
      1. “Package not found error” when installing a new App or Accessory
      2. A UBOS container comes up degraded
      3. Booting UBOS on a PC starts out fine, but then the screen goes blank
      4. Can I run UBOS in a Docker container?
      5. Can I use UBOS without purchasing a domain name?
      6. Cannot access MySQL database. File missing: /etc/mysql/root-defaults-ubos.cnf
      7. Cannot boot UBOS from boot stick on a PC
      8. Cannot connect to the public internet from a UBOS container
      9. Cannot create a temporary backup; the backup directory is not empty
      10. Failed to create file /sys/devices/system/cpu/microcode/reload
      11. How are the various UBOS images different from each other?
      12. How can I install more than one web App on the same Device?
      13. How do I set up WiFi?
      14. How to enable non-standard Package Repositories
      15. How to get help
      16. How to log into your UBOS Device
      17. How to modify the configuration of your Site
      18. How to report a bug or issue
      19. How to use Pagekite to reach your UBOS Device behind a firewall
      20. How to use SSH
      21. I need a Package that isn’t in UBOS
      22. I need root
      23. I own a domain name, and I’d like to use it for my UBOS Device. How do I do that?
      24. I want to move from one device to another, or from/to the cloud to/from a device
      25. I want to run ssh on a non-standard port
      26. I’m running out of disk space, what now?
      27. Installing a new Package or updating fails with a message about “invalid or corrupted package” or “key is disabled”
      28. Installing a new Package or upgrading fails with a message about “unknown trust”
      29. Is it safe to have my Site accessible from the public web?
      30. My non-English keyboard layout is all screwed up
      31. My SD card is much larger than the UBOS image. How do I use the rest of the space?
      32. Nothing happens when UBOS is supposed to be booting
      33. Problems with “IPv6 Packet Filtering Framework”
      34. UBOS is in a “degraded” state
      35. ubos-admin status reports “Systemd unit … has failed”
      36. Verify your downloaded UBOS image
      37. What is the default “root” password?
      38. What text editor can I use on UBOS?
      39. Why did you derive UBOS Linux from Arch Linux, and what is the relationship between UBOS Linux and Arch?
      40. Why is it called UBOS?
      41. Writing a disk image to a USB stick or SD card
        1. Writing an image to a USB stick or SD card on Linux
        2. Writing an image to a USB stick or SD card on macOS
        3. Writing an image to a USB stick or SD card on Windows
  2. Developer documentation
    1. Developer setup
      1. Developing using Arch Linux on VirtualBox x86_64 with a systemd-nspawn container
      2. Developing using a UTM Arch Linux VM on Apple Silicon computers with UBOS in a systemd-nspawn container
      3. Alternate developer setups
        1. Developing using a systemd-nspawn container (Linux host only)
        2. Developing using Arch Linux using Parallels on Apple Silicon with a systemd-nspawn container
        3. Developing using a UTM Arch Linux VM on Apple x86_64 computers with UBOS in a systemd-nspawn container
    2. Developer tutorials for standalone UBOS apps (not UBOS Mesh)
      1. Build and run your first UBOS standalone App
      2. How to package UBOS standalone Apps built with a variety of languages
        1. Hello World
        2. Glad-I-Was-Here (PHP, Mariadb)
        3. An Accessory for Glad-I-Was-Here (PHP, Mariadb)
        4. Glad-I-Was-Here (PHP, Postgresql)
        5. Glad-I-Was-Here (Python, Mariadb)
    3. UBOS Gears Reference
      1. UBOS Manifest
        1. Structure of the UBOS Manifest
        2. Info section
        3. Roles section
        4. Customization points section
        5. Appinfo section
        6. Accessoryinfo section
        7. Variables available at deploy or undeploy
        8. Functions that may be applied to variables
        9. Creating random values
        10. Scripts in UBOS Manifests
      2. Site JSON
      3. A complex deployment example
      4. UBOS Networking
      5. Allocating and opening up non-default ports
      6. Logging
      7. UBOS state
      8. UBOS Backup format
      9. Format of the App Status JSON
      10. Testing standalone Apps with “webapptest”
      11. Understanding ubos-admin
        1. Command: ubos-admin backup
        2. Command: ubos-admin backupinfo
        3. Command: ubos-admin createsite
        4. Command: ubos-admin deploy
        5. Command: ubos-admin hostid
        6. Command: ubos-admin init-staff
        7. Command: ubos-admin list-data-transfer-protocols
        8. Command: ubos-admin listnetconfigs
        9. Command: ubos-admin listsites
        10. Command: ubos-admin read-configuration-from-staff
        11. Command: ubos-admin restore
        12. Command: ubos-admin setnetconfig
        13. Command: ubos-admin setup-shepherd
        14. Command: ubos-admin showappconfig
        15. Command: ubos-admin shownetconfig
        16. Command: ubos-admin showsite
        17. Command: ubos-admin status
        18. Command: ubos-admin undeploy
        19. Command: ubos-admin update
        20. Command: ubos-admin write-configuration-to-staff
    4. Release channels and UBOS release process
    5. Miscellaneous
      1. Potentially useful infrastructure for standalone Apps
        1. The UBOS rsync server
      2. Middleware-specific notes
        1. Node.js notes
        2. SMTP notes
      3. Setting up an Arch Linux system as a UBOS development system
        1. How to create a UBOS development VM for VirtualBox on x86_64
        2. How to create a UBOS development VM for UTM on x86_64 Apple computers
        3. How to create a UBOS development VM for UTM on Apple Silicon computers
        4. How to create a UBOS development VM for Parallels Desktop on Apple Silicon computers
      4. Creating cloud images
        1. Amazon Web Services EC2
    6. Developer FAQ
      1. Doesn’t apt / dpkg / yum / pacman etc. does what UBOS Gears does already?
      2. Doesn’t puppet / chef / ansible etc. does what UBOS Gears does already?
      3. Doesn’t Docker do what UBOS Gears does already?
      4. Is it possible to run UBOS Gears or Mesh on an operating system other than UBOS Linux?
      5. Can I manage apps packaged as Docker containers with UBOS?
  3. Architecture
  4. Glossary
    1. Accessory
    2. App
    3. AppConfigId
    4. AppConfigItem
    5. AppConfiguration
    6. Arch
    7. Arch Linux
    8. Attribute
    9. blessing
    10. Bot
    11. Context Path
    12. Customization Point
    13. Data Transfer Protocol
    14. Deployment
    15. Depot
    16. Device
    17. Device Class
    18. diet4j module framework
    19. EntityType
    20. Flock
    21. Gradle
    22. Handlebars
    23. History
    24. Home Server
    25. HostId
    26. Hostname
    27. IDE
    28. Installation
    29. LetsEncrypt
    30. mDNS
    31. MeshBase
    32. MeshObject
    33. MeshObjectIdentifier
    34. MeshType
    35. MeshTypeIdentifier
    36. Middleware
    37. Model
    38. Network Configuration
    39. Package
    40. Pagekite
    41. Parallels Desktop
    42. Personal Server
    43. PKGBUILD
    44. Property
    45. PropertyType
    46. Relationship
    47. RelationshipType
    48. Release Channel
    49. Repository
    50. Retention Bucket
    51. Role
    52. RoleAttribute
    53. RoleProperty
    54. RoleType
    55. Rolling Release
    56. Shepherd
    57. Site
    58. Site JSON
    59. Site JSON Template
    60. SiteId
    61. Transaction
    62. Transaction Log
    63. UBOS Gears
    64. UBOS Linux
    65. UBOS Manifest
    66. UBOS Mesh
    67. UBOS Mesh code generator
    68. UBOS Project
    69. UBOS Staff
    70. unblessing
    71. UTM
    72. VirtualBox
    73. VMWare
    74. Wildcard hostname

Setting up networking and DNS

/docs/operation/networking/

Introduction

UBOS can be run on many different physical and virtual hardware configurations. For example, a Device may or may not have an Ethernet port; or it may have several. It may or may not have WiFi, which may be on board or be a separate USB dongle. It may have a virtual Ethernet port if run in a container or in the cloud. Networks that a Device connects to might also be very different, from typical home networks to virtual networks on host machines running UBOS in Linux containers.

Because of all these differences, networking for Linux server devices is usually complex to set up and requires substantial expertise.

With UBOS, as usual, we try to simplify things. We do this by pre-defining some common Network Configurations, and automatically install the one most likely to work “out of the box” for a new Device of a given class without further configuration. But if you wish to use the Device in a different setting, we provide a single command to switch to a different Network Configuration.

Unlike other network management tools, UBOS Network Configurations manage all necessary layers of the networking stack at the same time including:

  • individual network interfaces such as Ethernet interfaces

  • IP addresses

  • DHCP

  • DNS on the local network

  • zeroconf/mDNS

  • Firewall

  • Masquerading

  • open ports for applications.

This works for wired (Ethernet) and wireless (WiFi) connections.

Currently available Network Configurations

The following Network Configurations are currently available:

Network configuration: client

For most Device Classes, this is the default Network Configuration after UBOS installation.

If your Device has an Ethernet port, and you connect it to your Ethernet network, the Device will automatically connect.

If your Device has more than one network interface, it should not matter which you use.

In this mode:

  • network interfaces: All network interfaces are active, and looking to obtain an IP address, default gateway and DNS server information via DHCP.

  • mDNS: The Device will advertise itself on all network interfaces.

  • ports: Application-level ports (such as HTTP and HTTPS ports 80 and 443) will be accessible from all network interfaces.

  • ssh: ssh connections are accepted on all network interfaces.

  • firewall: All other ports are firewalled.

If you’d like to have a predictable IP address for your Device in this Network Configuration, we recommend that you configure your DHCP server to always hand out the same IP address to your Device. This tends to be an easier, and typically more maintainable process than maintaining static IP addresses on all the Devices on your network. For example, depending on the model of your home router, you may be able to do this:

  • Log into the administrative interface of your home router, and look for a page that lists all currently connected devices on your network.

  • Look for your UBOS device, and note its MAC address.

  • In some part of the administrative interface of your home router, you may be able to assign a consistent IP address for the device with this MAC address.

Network configuration: standalone

This Network Configuration is useful to set up a network that is disconnected from the public internet. In this configuration, the Device assumes the role of network manager and manages the network by running a DHCP server, and a DNS server.

Other computers connected to the Device will be able to receive an IP address and use network services as if they were connected to a typical home network, but without upstream connection to the public internet.

If your Device has more than one network interface, UBOS will create separate subnets for each network interface.

In this mode:

  • network interfaces: All network interfaces are active and have a static IP address assigned. Other connected computers can obtain IP address, default gateway and DNS server information from a DHCP server and a DNS server running on the Device.

  • mDNS: The Device will advertise itself on all interfaces.

  • ports: application-level ports (such as HTTP and HTTPS ports 80 and 443) will be accessible from all interfaces.

  • ssh: ssh connections are accepted on all interfaces.

  • firewall: All other ports are firewalled.

Network configuration: gateway

This Network Configuration was created to make it easy to use UBOS for home routers. In this configuration, UBOS will connect to the public internet via broadband through one network interface, and obtain an IP address from the ISP via DHCP. All other network interfaces will have statically allocated, private IP addresses that connect to the public internet via Network Address Translation (NAT, also known as masquerading).

This configuration requires the Device to have at least two network interfaces: one for the upstream connection to the public internet via the ISP, and one for the local network. Devices connecting to the local network can obtain local IP addresses from the UBOS device via DHCP. The Device also provides DNS for the local network, with delegation to the ISP’s DNS server.

If the Device has more than two network interfaces, additional interfaces will be created as separate local networks on different subnets.

Sites running on the Device are accessible from the local network (“downstream”) but not from the public internet (“upstream”).

In this mode:

  • network interfaces: All network interfaces are active.

    • The first (“upstream”) interface (which one that is depends on the hardware; trial and error helps) connects to the upstream broadband connection. It obtains IP address, default gateway and upstream DNS server information via DHCP.

    • All other (“downstream”) interfaces have static IP addresses assigned. Computers connecting to those interfaces can obtain a local area network IP address, default gateway and DNS server information from a DHCP server and a DNS server running on the Device.

  • masquerade: Computers connected to a “downstream” interface are masqueraded behind the IP address of the “upstream” interface.

  • mDNS: The Device will advertise itself on the “downstream” interfaces only.

  • ports: Application-level ports (such as HTTP and HTTPS ports 80 and 443) will be accessible from computers connected to the “downstream” interfaces only.

  • ssh: ssh connections are accepted on all interfaces.

  • firewall: All parts other than ssh are firewalled on the “upstream” interface. Application ports are accessible from the “downstream” interfaces.

Network configuration: public-gateway

This Network Configuration is identical to gateway, except that Sites running on the Device are accessible both from the local network (“downstream”) and from the public internet (“upstream”).

Network configuration: container

This Network Configuration is used by UBOS when run in a Linux container started by systemd-nspawn or the like. It is very similar to client but there are no mDNS advertisements.

Network configuration: off

In this network configuration, UBOS has turned off all networking. This is useful as an emergency setting.

Network configuration: espressobin

This ESPRESSObin-specific Network Configuration is like gateway. It configures the Ethernet port closest to the USB 3 port (the blue USB port) on the ESPRESSObin to be the upstream network interface, and the other network interfaces to be downstream.

mDNS hostnames

By default, Devices announce themselves on the local-area network with a name derived from the type of Device, such as ubos-pc.local.

The advantage of using these mDNS hostnames is that no DNS setup is required, and you do not need to assign a static IP address to your Device.

The disadvantage of using these hostnames is that they only work on the local network, and that you cannot run more than one Site on the same Device. There may also be collisions if you run more than one Device of the same type on the same network.

If you wish to change your Device’s mDNS hostname, change its Linux hostname, and restart the Avahi daemon. Assuming you would like the new name to be mydevice, you can do this by executing the following commands as root:

% hostname mydevice
% hostname > /etc/hostname
% sudo systemctl restart avahi-daemon

Non-mDNS (regular) hostnames

If you would like to use more than one Site on the same Device, or you would like to use a hostname of your choosing (say, family.example.com) you need to set up DNS yourself. This can sometimes be performed in the administration interface of your home router.

For example, depending on the model of your home router, you may be able to do this:

  • Log into the administrative interface of your home router, and look for a page that lists all currently connected devices on your network.

  • Look for your Device, and note its MAC address.

  • In some part of the administrative interface of your home router, you may be able to assign a consistent hostname for the device with this MAC address.

Unfortunately, this entirely depends on the features of your home router, and is outside of the control of UBOS.

Persistence of network configuration settings

Once a Network Configuration is set with:

% sudo ubos-admin setnetconfig <name>

it will survive a reboot. Furthermore, when a Network Configuration is restored – for example because temporarily another Network Configuration was activated – the previous settings will be restored as much as possible. Consider this sequence:

% sudo ubos-admin setnetconfig standalone
% sudo ubos-admin setnetconfig off
% sudo ubos-admin setnetconfig standalone

In the standalone Network Configuration, UBOS assigns static IP addresses to all network interfaces found. Which IP address is assigned to which network interface is basically random. However, it would be desirable if the same IP address was assigned to the same interface when the standalone network configuration was restored after temporarily being off. UBOS accomplishes this by saving the actual assignments in file /etc/ubos/netconfig-standalone.json (replace standalone with the name of the network configuration). If such a file exists, UBOS will restore its settings as much as possible.

This enables a user not scared of editing a JSON file to override the standard settings of a particular Network Configuration. For example, if a Device has two network interfaces and is used in the client network configuration, by editing /etc/ubos/netconfig-client.json and executing ubos-admin setnetconfig client again, the user could, for example, keep one of those interfaces off, or have different ports open.

To display information about the most recently set Network Configuration:

% sudo ubos-admin shownetconfig

Extra DHCP and DNS configuration settings

UBOS uses dnsmasq (home page) to issue DHCP leases and manage DNS on the local network. UBOS makes it straightforward for you to add your own settings to the ones managed by UBOS:

  • Settings generated and managed by UBOS are in directory /etc/dnsmasq.ubos.d.

  • Add your settings to directory /etc/dnsmasq.d. You can use any file name, as long as it ends in .conf.

For details, please refer to the dnsmasq documentation.

For example: you could use this setup to implement a DHCP “white list” so only devices that are know are allowed to obtain an IP address, which in turn will be the same every time the same device connects to the network.

External Dynamic DNS

If your Device can be reached over the public internet, such as if it acts as a broadband router, it may be advantageous to run Dynamic DNS: this provides a consistent DNS hostname for the Device, even if the internet service provider changes the Device’s IP address from time to time.

For that purpose, UBOS includes the ddclient package, which supports a broad range of paid and free dynamic DNS providers. To set it up, first install the package:

% sudo pacman -S ddclient

Then, as root, edit /etc/ddclient/ddclient.conf with the settings for your dynamic DNS provider. Finally, run ddclient as a daemon:

% sudo systemctl enable --now ddclient.service

This is described in more detail on the Arch Linux Wiki.