Documentation Overview

  1. Operation documentation
    1. Installation
      1. Run UBOS on a PC (64bit)
      2. Run UBOS from a boot stick on a PC (64bit)
      3. Run UBOS in a VirtualBox virtual machine (64bit)
      4. Run UBOS on an Amazon Web Services EC2 virtual server
      5. Run UBOS on Raspberry Pi 5
      6. Run UBOS on ESPRESSObin
      7. Run UBOS in a Linux container on a PC (64bit)
      8. Run UBOS in an aarch64 Linux container
    2. Setting up your first Site and App
    3. Setting up networking and DNS
    4. How to create a website secured by SSL/TLS
    5. How to set up a website as a Tor hidden service
    6. Managing Sites and Apps
    7. Backup and restore
    8. Upgrading and keeping UBOS current
    9. The UBOS Staff
    10. App-specific notes
      1. Reliably send e-mail via Amazon Web Services’ Simple E-mail Service: amazonses
      2. Static website hosting with rsync-based upload: docroot
      3. Notes on Mastodon
      4. Notes on Nextcloud
      5. Notes on Redirect
      6. Notes on Wordpress
    11. Device-specific Notes
      1. ESPRESSObin
      2. Raspberry Pi
    12. Advanced management
      1. Enabling non-standard package repositories
      2. Migrating from one App to another
      3. Pinning resources
    13. Command reference
    14. FAQ, HOWTOs and Troubleshooting
      1. “Package not found error” when installing a new App or Accessory
      2. A UBOS container comes up degraded
      3. Booting UBOS on a PC starts out fine, but then the screen goes blank
      4. Can I run UBOS in a Docker container?
      5. Can I use UBOS without purchasing a domain name?
      6. Cannot access MySQL database. File missing: /etc/mysql/root-defaults-ubos.cnf
      7. Cannot boot UBOS from boot stick on a PC
      8. Cannot connect to the public internet from a UBOS container
      9. Cannot create a temporary backup; the backup directory is not empty
      10. Failed to create file /sys/devices/system/cpu/microcode/reload
      11. How are the various UBOS images different from each other?
      12. How can I install more than one web App on the same Device?
      13. How do I set up WiFi?
      14. How to enable non-standard Package Repositories
      15. How to get help
      16. How to log into your UBOS Device
      17. How to modify the configuration of your Site
      18. How to report a bug or issue
      19. How to use Pagekite to reach your UBOS Device behind a firewall
      20. How to use SSH
      21. I need a Package that isn’t in UBOS
      22. I need root
      23. I own a domain name, and I’d like to use it for my UBOS Device. How do I do that?
      24. I want to move from one device to another, or from/to the cloud to/from a device
      25. I want to run ssh on a non-standard port
      26. I’m running out of disk space, what now?
      27. Installing a new Package or updating fails with a message about “invalid or corrupted package” or “key is disabled”
      28. Installing a new Package or upgrading fails with a message about “unknown trust”
      29. Is it safe to have my Site accessible from the public web?
      30. My non-English keyboard layout is all screwed up
      31. My SD card is much larger than the UBOS image. How do I use the rest of the space?
      32. Nothing happens when UBOS is supposed to be booting
      33. Problems with “IPv6 Packet Filtering Framework”
      34. UBOS is in a “degraded” state
      35. ubos-admin status reports “Systemd unit … has failed”
      36. Verify your downloaded UBOS image
      37. What is the default “root” password?
      38. What text editor can I use on UBOS?
      39. Why did you derive UBOS Linux from Arch Linux, and what is the relationship between UBOS Linux and Arch?
      40. Why is it called UBOS?
      41. Writing a disk image to a USB stick or SD card
        1. Writing an image to a USB stick or SD card on Linux
        2. Writing an image to a USB stick or SD card on macOS
        3. Writing an image to a USB stick or SD card on Windows
  2. Developer documentation
    1. Developer setup
      1. Developing using Arch Linux on VirtualBox x86_64 with a systemd-nspawn container
      2. Developing using a UTM Arch Linux VM on Apple Silicon computers with UBOS in a systemd-nspawn container
      3. Alternate developer setups
        1. Developing using a systemd-nspawn container (Linux host only)
        2. Developing using Arch Linux using Parallels on Apple Silicon with a systemd-nspawn container
        3. Developing using a UTM Arch Linux VM on Apple x86_64 computers with UBOS in a systemd-nspawn container
    2. Developer tutorials for standalone UBOS apps (not UBOS Mesh)
      1. Build and run your first UBOS standalone App
      2. How to package UBOS standalone Apps built with a variety of languages
        1. Hello World
        2. Glad-I-Was-Here (PHP, Mariadb)
        3. An Accessory for Glad-I-Was-Here (PHP, Mariadb)
        4. Glad-I-Was-Here (PHP, Postgresql)
        5. Glad-I-Was-Here (Python, Mariadb)
    3. UBOS Gears Reference
      1. UBOS Manifest
        1. Structure of the UBOS Manifest
        2. Info section
        3. Roles section
        4. Customization points section
        5. Appinfo section
        6. Accessoryinfo section
        7. Variables available at deploy or undeploy
        8. Functions that may be applied to variables
        9. Creating random values
        10. Scripts in UBOS Manifests
      2. Site JSON
      3. A complex deployment example
      4. UBOS Networking
      5. Allocating and opening up non-default ports
      6. Logging
      7. UBOS state
      8. UBOS Backup format
      9. Format of the App Status JSON
      10. Testing standalone Apps with “webapptest”
      11. Understanding ubos-admin
        1. Command: ubos-admin backup
        2. Command: ubos-admin backupinfo
        3. Command: ubos-admin createsite
        4. Command: ubos-admin deploy
        5. Command: ubos-admin hostid
        6. Command: ubos-admin init-staff
        7. Command: ubos-admin list-data-transfer-protocols
        8. Command: ubos-admin listnetconfigs
        9. Command: ubos-admin listsites
        10. Command: ubos-admin read-configuration-from-staff
        11. Command: ubos-admin restore
        12. Command: ubos-admin setnetconfig
        13. Command: ubos-admin setup-shepherd
        14. Command: ubos-admin showappconfig
        15. Command: ubos-admin shownetconfig
        16. Command: ubos-admin showsite
        17. Command: ubos-admin status
        18. Command: ubos-admin undeploy
        19. Command: ubos-admin update
        20. Command: ubos-admin write-configuration-to-staff
    4. Release channels and UBOS release process
    5. Miscellaneous
      1. Potentially useful infrastructure for standalone Apps
        1. The UBOS rsync server
      2. Middleware-specific notes
        1. Node.js notes
        2. SMTP notes
      3. Setting up an Arch Linux system as a UBOS development system
        1. How to create a UBOS development VM for VirtualBox on x86_64
        2. How to create a UBOS development VM for UTM on x86_64 Apple computers
        3. How to create a UBOS development VM for UTM on Apple Silicon computers
        4. How to create a UBOS development VM for Parallels Desktop on Apple Silicon computers
      4. Creating cloud images
        1. Amazon Web Services EC2
    6. Developer FAQ
      1. Doesn’t apt / dpkg / yum / pacman etc. does what UBOS Gears does already?
      2. Doesn’t puppet / chef / ansible etc. does what UBOS Gears does already?
      3. Doesn’t Docker do what UBOS Gears does already?
      4. Is it possible to run UBOS Gears or Mesh on an operating system other than UBOS Linux?
      5. Can I manage apps packaged as Docker containers with UBOS?
  3. Architecture
  4. Glossary
    1. Accessory
    2. App
    3. AppConfigId
    4. AppConfigItem
    5. AppConfiguration
    6. Arch
    7. Arch Linux
    8. Attribute
    9. blessing
    10. Bot
    11. Context Path
    12. Customization Point
    13. Data Transfer Protocol
    14. Deployment
    15. Depot
    16. Device
    17. Device Class
    18. diet4j module framework
    19. EntityType
    20. Flock
    21. Gradle
    22. Handlebars
    23. History
    24. Home Server
    25. HostId
    26. Hostname
    27. IDE
    28. Installation
    29. LetsEncrypt
    30. mDNS
    31. MeshBase
    32. MeshObject
    33. MeshObjectIdentifier
    34. MeshType
    35. MeshTypeIdentifier
    36. Middleware
    37. Model
    38. Network Configuration
    39. Package
    40. Pagekite
    41. Parallels Desktop
    42. Personal Server
    43. PKGBUILD
    44. Property
    45. PropertyType
    46. Relationship
    47. RelationshipType
    48. Release Channel
    49. Repository
    50. Retention Bucket
    51. Role
    52. RoleAttribute
    53. RoleProperty
    54. RoleType
    55. Rolling Release
    56. Shepherd
    57. Site
    58. Site JSON
    59. Site JSON Template
    60. SiteId
    61. Transaction
    62. Transaction Log
    63. UBOS Gears
    64. UBOS Linux
    65. UBOS Manifest
    66. UBOS Mesh
    67. UBOS Mesh code generator
    68. UBOS Project
    69. UBOS Staff
    70. unblessing
    71. UTM
    72. VirtualBox
    73. VMWare
    74. Wildcard hostname

Hello World

/docs/development/tutorials-gears/toyapps/helloworld/

Introduction

Our version of Hello World is an extremely simple Web applications that that just displays “Hello World” in a user’s web browser. We use it to give you a taste for what is involved to package a real web application for UBOS. Here is a complete :-) screen shot:

Screenshot

To obtain the source code:

% git clone https://github.com/uboslinux/ubos-toyapps

Go to subdirectory helloworld.

Package lifecycle and App deployment

The first thing to understand about how UBOS Apps are packaged and deployed is the lifecycle of a package from the developer’s perspective, and then how a user deploys and undeploys the App contained in the package to their website:

  1. The developer creates the files for the package. If you have cloned the git repository above, you find the files for helloworld in directory helloworld; or you can browse them on github. They are:

    • index.php: minimalistic PHP file printing “Hello World”;

    • htaccess: Apache configuration file fragment that makes Apache default to index.php when the App is installed;

    • PKGBUILD: script used to create the package (see below);

    • ubos-manifest.json: UBOS meta-data file (see below);

    • appicons: icon files (optional) that will be used for the icon the user can click on to visit the App.

  2. The developer creates the package by executing, in the package’s root directory:

    % makepkg -f
    

    This will create the package file named helloworld-*-any.pkg.tar.xz (where * is a particular version number defined in the PKGBUILD file). makepkg is the packaging command for pacman packages, the types of packages we use in UBOS. (You can read more about pacman on the Arch Linux Wiki.)

  3. Once the package has been created, the developer makes the package available to the user. In the general case, this involves uploading the package to the Depot, UBOS quality assurance etc etc, but for our purposes here, ignore all that and simply assume that the package file created by the developer has somehow arrived on the user’s Device, such as by file transfer.

  4. The user installs the package on the target device:

    % sudo pacman -U helloworld-*-any.pkg.tar.xz
    

    This command will install a locally built package locally, but it is equivalent to what happens when a user obtains the same App via the UBOS Depot.

    Note that this unpacks the package on the hard drive and runs whatever installation scripts that the package specifies (the latter is rare, e.g. Hello World does not have such scripts). However, installing the package does not deploy the App at any Site or virtual host. To do that, see the next step:

  5. The user deploys the web App defined by the package at a particular Site (aka virtual host) with a command such as:

    % sudo ubos-admin createsite
    

    This commend asks some questions; answer the questions as you like, and use the App name helloworld (see also Setting up your first Site and App).

    This will put all the right files in the right web server directories, activate needed Apache modules, restart servers, and the like. When this command completes, the App is ready for use.

  6. The user can now visit the fully deployed App at the respective URL at which it was installed.

  7. Now assume that a new version of the package is available. If the new package is available locally, the user can perform a software upgrade of the helloworld package (only):

    % sudo ubos-admin update --pkgfile <pkgfile>
    

    where <pkgfile> is a new version of the package file created as shown above. If distributed through the UBOS Depot, the argument --pkgfile will be omitted, and UBOS will upgrade all software on the host to the most recent version.

  8. Undeploy the App by undeploying the entire Site:

    % sudo ubos-admin undeploy --siteid <siteid>
    

    where <siteid> is the identifier of the installed Site. Note that this will keep the App’s package on the Device so it can be redeployed without re-installing the package.

  9. If the user wishes to remove the package entirely:

    % sudo pacman -R helloworld
    

Anatomy of the package

The PKGBUILD script’s package function puts the package together on behalf of the developer:

package() {
# Manifest
    install -D -m0644 ${startdir}/ubos-manifest.json ${pkgdir}/ubos/lib/ubos/manifests/${pkgname}.json

# Icons
    install -D -m0644 ${startdir}/appicons/{72x72,144x144}.png -t ${pkgdir}/ubos/http/_appicons/${pkgname}/
    install -D -m0644 ${startdir}/appicons/license.txt         -t ${pkgdir}/ubos/http/_appicons/${pkgname}/

# Code
    install -D -m0644 ${startdir}/index.php -t ${pkgdir}/ubos/share/${pkgname}/
    install -D -m0644 ${startdir}/htaccess  -t ${pkgdir}/ubos/share/${pkgname}/
}

You can see that this script creates installs a few files in subdirectories of ${pkgdir}, which is a staging directory for creating the package tar file. For more information about PKGBUILD, consider the Arch Linux wiki PKGBUILD page; there is nothing UBOS-specific about this.

This corresponds to what the package file contains after makepkg has completed:

% tar tfJ helloworld-*-any.pkg.tar.xz
.PKGINFO
.BUILDINFO
.MTREE
ubos/
ubos/lib/
ubos/http/
ubos/share/
ubos/share/helloworld/
ubos/share/helloworld/index.php
ubos/share/helloworld/htaccess
ubos/http/_appicons/
ubos/http/_appicons/helloworld/
ubos/http/_appicons/helloworld/72x72.png
ubos/http/_appicons/helloworld/144x144.png
ubos/http/_appicons/helloworld/license.txt
ubos/lib/ubos/
ubos/lib/ubos/manifests/
ubos/lib/ubos/manifests/helloworld.json

The first three files, .PKGINFO, .BUILDINFO and .MTREE contain metadata that is automatically generated by makepkg.

Directory ubos/share/helloworld contains the files that constitute the application. For this extremely simple App, there are only two: the PHP code that emits the “Hello World” HTML, and an Apache htaccess file so this HTML is emitted even if the path ends with a slash instead of index.php. More complex web Apps would put the bulk of their code and auxiliary files there.

In a typical Linux distro, these files would be located at /usr/share/helloworld or perhaps at /srv/http/helloworld. In UBOS, these files are located below /ubos, which is the place where users mount a large data disk if they have a separate disk.

Hello world icon

The files below ubos/http/_appicons/ are simply graphics files that can be used by UBOS to show to the user a logo for the application. This image is shown to the right. They are optional and are added in the package() section of PGKBUILD.

Finally, ubos/lib/ubos/manifests/ contains the UBOS Manifest file for this application, which describes what needs to happen upon ubos-admin deploy and when other ubos-admin commands are executed. For details, read on:

UBOS Manifest

For this App, the UBOS Manifest file looks as follows:

{
  "type" : "app",

  "roles" : {
    "apache2" : {
      "defaultcontext" : "/hello",
      "depends" : [
        "php",
        "php-apache"
      ],
      "apache2modules" : [
        "php7"
      ],
      "appconfigitems" : [
        {
          "type"         : "file",
          "name"         : "index.php",
          "source"       : "index.php",
        },
        {
          "type"         : "file",
          "name"         : ".htaccess",
          "source"       : "htaccess",
        }
      ]
    }
  }
}

Let’s discuss these items in sequence:

  • "type" : "app" declares this to be an App, not an Accessory. (For a discussion of Accessories, see An Accessory for Glad-I-Was-Here (PHP, Mariadb)).

  • This App only uses a single Role: apache2. Apps could also specify other Roles, such as mysql, if they make use of Mariadb in addition to Apache.

  • By default, this App wants to be deployed at the relative path /hello of a Site. This can be overridden by the user in the Site JSON file or when entering a different path during execution of ubos-admin createsite.

  • For the apache2 role, this App requires packages php and php-apache, as it is a PHP App. It requires that the Apache module php7 has been enabled before it can be run.

  • Finally, each installation of this App requires two files to be installed in the web server’s document directory tree: a file called index.php, which is simply copied, and a file called .htaccess which is copied from a slightly different name. By convention, the “source” path is relative to the package installation directory /ubos/share/helloworld; and the destination path is relative to the correct directory from which Apache serves files, given the Site and context at which the App runs. Here, this may be /ubos/http/sites/sa6e789f5d919c464d2422f6620eaf9cba789c4a5/hello/ (auto-provisioned by UBOS).

When the user invokes ubos-admin deploy, UBOS processes the UBOS Manifest and “makes it so”. We recommend you package and then helloworld with the example commands above, and then examine how UBOS made the App appear by, for example examining the generated Apache configuration files below /etc/httpd.

When the user invokes ubos-admin undeploy, UBOS processes the UBOS Manifest in reverse sequence, and restores the system to its previous state.