Installation🔗

This document explains how the PlanktoScope OS's installation process works, as a companion to our non-standard installation guide which carries out the process explained below.

The installation process is initiated by booting into an appropriate installation of the Raspberry Pi OS and then downloading and running the installation bootstrap script, which in turn downloads and runs the appropriate distro setup scripts according to the installation parameters provided to the installation bootstrap script.

Installation bootstrap script🔗

The installation bootstrap script is provided so that a one-line command can be executed to automatically perform the entire process of installing the PlanktoScope OS on top of the Raspberry Pi OS. The GitHub repository which contains that script always publishes the latest version on its stable branch to install.planktoscope.community/distro.sh via GitHub Pages; other versions can be downloaded from GitHub via the corresponding permalinks for those versions of the file (e.g. https://github.com/PlanktoScope/install.planktoscope.community/raw/v2023.9.0/distro.sh for the version from the v2023.9.0 tag in the repository). The installation bootstrap script performs the following steps:

The script loads some parameters (set by environment variables and/or corresponding command-line arguments) which set the behavior of the script.
The script installs git, if it was not already installed (as is the case on the "Lite" image of the Raspberry Pi OS); if the yes parameter was not set and git was not already installed, the script will first ask the user to confirm that they wish to install git before the script continues. git is required to resolve version query parameters provided to the script, so that the script can determine how to download the requested version of the PlanktoScope OS's distro setup scripts.
The script clones a minimal "mirror" copy of the specified repository (set by the repo parameter) of distro setup scripts to a temporary directory (i.e. a directory created in /tmp). This "mirror" copy is used to:
- Resolve the version query parameters (version-query, query-type, and - when query-type is tag - tag-prefix) into a specific commit hash for the repository.
- Determine a (pseudo-)version string for the resolved commit based on the last release tag (whose name is prefixed with the tag-prefix parameter) ancestral to that commit.
The script clones a copy of the specified repository (set by the repo parameter) to a temporary directory and checks out the specific commit which was resolved by the previous step; if the yes parameter was not set, the script will first ask the user to confirm that they wish to download the resolved commit of the distro setup scripts before the script continues. Because the repository containing the distro setup scripts may have many large files (e.g. image files for documentation) which are unrelated to the distro setup scripts, this step only downloads files in the specific commit needed for the distro setup scripts.
The script records versioning information for the downloaded installation scripts, saving that information in two YAML files in a particular directory; if that directory already exists and the yes parameter was not set, the script will first ask the user to confirm that they wish to delete and re-create that directory before the script continues. Installed programs can read these files in order to determine the installed version of the PlanktoScope OS.
The script runs the specified script (set by the setup-entrypoint parameter) within the downloaded copy of the specified repository; if the yes parameter was not set, the script will first ask the user to confirm that they wish to run the downloaded distro scripts before the script continues.

Script parameters🔗

Command-Line Flags	Environment Variable Name	Description
`-r` `--repo`	`REPO`	URL of the Git repository used for downloading the distro setup scripts. If a protocol is not specified, the URL will be assumed to use HTTPS. Default value: `github.com/PlanktoScope/PlanktoScope`
`-v` `--version-query`	`VERSION_QUERY`	The version of the repository to download. The version query is interpreted differently depending on the query type set by the `query-type` parameter: Query type `branch`: query should be a branch name (e.g. `software/beta`). Query type `tag`: query should be a Git tag name, excluding the tag prefix specified by the `tag-prefix` parameter (e.g. `v2024.0.0` if `tag-prefix` is `software` and the Git tag is `software/v2024.0.0`). Query type `hash`: query should be a commit hash and may be abbreviated (e.g. `2d6928e`). Default value: `software/stable`
`-t` `--query-type`	`QUERY_TYPE`	The type of version query set by the `version-query` parameter. Allowed values: `branch`, `tag`, `hash`. Default value: `branch`
`-H` `--hardware`	`HARDWARE`	The hardware configuration, passed as the first argument to the entrypoint of the distro setup scripts. The distro setup scripts in the `github.com/PlanktoScope/PlanktoScope` repo currently expect either `adafruithat` or `planktoscopehat`. Default value: `planktoscopehat`
`--tag-prefix`	`TAG_PREFIX`	The prefix for Git version tags when resolving the version query (with query type `tag`) and when resolving tags (for pseudoversion strings). Default value: `software/`
`--setup-entrypoint`	`SETUP_ENTRYPOINT`	The entrypoint of the distro setup scripts, which will be executed in order to run the downloaded distro setup scripts. This should be a subdirectory path within the repository (set by the `repo` parameter) which will be downloaded at the specified commit (set by the `version-query` and `query-type` parameters and - when `query-type` is `tag` - by the `tag-prefix` parameter) to obtain the distro setup scripts. Default value: `software/distro/setup/setup.sh`
`-y` `-f` `--yes` `--force`	`FORCE`	Whether to automatically confirm all user confirmation prompts: `FORCE=1` (or including the command-line flag) automatically confirms all prompts. Not setting `FORCE` (or not including the command-line flag) requires user input for confirmation on all prompts. Default: User input required
`-V` `--verbose`	`VERBOSE`	Whether to display additional (verbose) output: `VERBOSE=1` (or including the command-line flag) enables verbose output. Not setting `VERBOSE` (or not including the command-line flag) disables verbose output. Default: Don't display verbose output
`-h` `--help`		Whether to display a help message (which includes this parameter reference table and a list of example commands using this script) and quit without doing any work.

Example combinations of command-line arguments using the above parameters:

-v software/stable -H planktoscopehat or -H planktoscopehat: install the latest stable release of the standard PlanktoScope OS (i.e. from the software/stable branch) for a PlanktoScope with the PlanktoScope HAT.
-v software/beta -H adafruithat: install the latest beta pre-release or stable release of the standard PlanktoScope OS (i.e. from the software/beta branch) for a PlanktoScope with the Adafruit HAT.
-v master -H planktoscopehat: install the latest development version of the standard PlanktoScope OS (i.e. from the master branch) for a PlanktoScope with the PlanktoScope HAT.
-t tag -v v2024.0.0-alpha.1 -H adafruithat: install the v2024.0.0-alpha.1 pre-release of the standard PlanktoScope OS (i.e. from the software/v2024.0.0-alpha.1 tag) for a PlanktoScope with the Adafruit HAT.
-t hash -v 2d6928e -H planktoscopehat: install 2d6928e commit of the standard PlanktoScope OS for a PlanktoScope with the PlanktoScope HAT.
-v master -r github.com/LaurentPV/PlanktoScope -H adafruithat: install the latest development commit of master branch of the github.com/LaurentPV/PlanktoScope fork of the PlanktoScope OS for a PlanktoScope with the Adafruit HAT.

Recorded versioning information🔗

Currently the installer script creates two YAML files, both in the /home/pi/.local/etc/pkscope-distro directory. (Note: this directory may change in the future.)

/home/pi/.local/etc/pkscope-distro/installer-config.yml records the values of the parameters with which the installer script was invoked:

YAML Field Name	Corresponding Script Parameter	Example Value
`repo`	`repo`	`"github.com/PlanktoScope/PlanktoScope"`
`version-query`	`version-query`	`"v2024.0.0-alpha.1"`
`query-type`	`query-type`	`"tag"`
`hardware`	`hardware`	`"adafruithat"`
`tag-prefix`	`tag-prefix`	`"software/"`
`setup-entrypoint`	`setup-entrypoint`	`"software/distro/setup/setup.sh"`

/home/pi/.local/etc/pkscope-distro/installer-versioning.yml records information about the version of the PlanktoScope OS's distro setup scripts which was used to install the PlanktoScope OS:

Field Name	Description
`repo`	The path of the Git repository which provided the distro setup scripts, with any leading protocol string (e.g. `https://`) removed. Example Values: `"github.com/PlanktoScope/PlanktoScope"` `"github.com/PlanktoScope/PlanktoScope"`
`commit`	The full hash of the exact commit which provided the distro setup scripts. Example Values: `"2d6928e596b28f0c4c268fecb588c85215b1027e"` (for commit `2d6928e`) `"4d8b882616a8374918730bc1c6d300edfd7a523a"` (for commit `4d8b882`)
`tag`	The full name of the Git tag (whose name starts with the prefix set by the `tag-prefix` script parameter) at the exact commit which provided the distro setup scripts, if such a tag exists; otherwise, the empty string (`""`). Example Values: `"software/v2024.0.0-alpha.1"` (so commit `2d6928e` has Git tag `software/v2024.0.0-alpha.1`) `""` (so commit `4d8b882` has no Git tag)
`version`	A version string or pseudo-version string describing the exact commit which provided the distro setup scripts. If a Git tag (with a name starting with the prefix set by the `tag-prefix` script parameter) exists at that commit, this is a version string which just the name of that tag, but with the `tag-prefix` stripped from the name. Otherwise, this is a pseudo-version string with format `"{tag}-{distance}-g{commit}"`, where: `{tag}` is the name of the most recent ancestral tag (with a name starting with the prefix set by the `tag-prefix` script parameter) reachable from the commit, but with the `tag-prefix` stripped from the name. `{distance}` distance (in number of commits) between the commit and the most recent ancestral tag. `{commit}` consists of the first seven characters of the hash of the commit. Example Values: `"v2024.0.0-alpha.1"` (so commit `2d6928e` has Git tag `software/v2024.0.0-alpha.1`) `"v2024.0.0-alpha.1-4-g4d8b882"` (so commit `4d8b882` is 4 commits above the commit with Git tag `software/v2024.0.0-alpha.1`, which was the last tag in the ancestry of the commit)

Distro setup scripts🔗

Currently, the entrypoint of the distro setup scripts, at software/distro/setup/setup.sh, takes exactly one command-line argument which must be one of the following values:

adafruithat: this will cause the distro setup scripts to install a version of the PlanktoScope hardware controller and the PlanktoScope Node-RED dashboard specific to PlanktoScopes using the Adafruit HAT, and to set the default hardware configuration files accordingly.
planktoscopehat: this will cause the distro setup scripts to install a version of the PlanktoScope hardware controller and the PlanktoScope Node-RED dashboard specific to PlanktoScopes using the PlanktoScope HAT, and to set the default hardware configuration files accordingly.

Currently, the distro setup scripts must be run by a user named pi. The scripts should not be run with sudo!

Currently, the distro setup scripts are split into two phases: setup of the base operating system, and setup of the PlanktoScope application environment. In the future, as we remove various setup steps from these scripts (and instead manage those steps using forklift), we may consolidate these two phases into a single phase.

Base system setup🔗

This phase performs steps which might (in theory) be useful for other projects which don't use the PlanktoScope hardware but would still benefit from many of the functionalities provided by the PlanktoScope OS. This phase consists of the following steps:

Installation of base tools: Docker, Cockpit, and various command-line tools are installed.
Installation of forklift and a Forklift pallet: a hard-coded version of forklift is downloaded to /usr/bin/forklift , a hard-coded version of a hard-coded pallet (namely, github.com/PlanktoScope/pallet-standard) is downloaded and prepared for deployment, and the forklift-apply.service systemd service is created and enabled. (Note: in the future, it will be possible to specify the pallet to be installed as a command-line argument.)
Partial configuration of Raspberry Pi-specific hardware: the SPI and I2C hardware interfaces are enabled, and the serial port and serial port console are enabled (note: the serial port console will be disabled by the PlanktoScope application environment setup phase so that the serial port can be used for the PlanktoScope's GPS receiver instead), and legacy camera support is disabled.
Configuration of the system locale: the system's language is changed to en_US.UTF-8, but the time and measurement formats are changed to en_DK.UTF-8 so that the date format is yyyy-mm-dd and units are metric. The system timezone is set to UTC.
Partial configuration of networking: various system services are installed and configured, namely dhcpcd, dnsmasq, hostapd, and firewalld. The enable-interface-forwarding.service and autohotspot.service systemd services are created and enabled. The Raspberry Pi's Wi-Fi country is set to the US.
Cleanup: SSH keys are reset to be regenerated on the next boot, unnecessary APT files are removed, and the OS machine ID is reset to be regenerated on the next boot.

PlanktoScope application environment setup🔗

This phase performs steps specific to the PlanktoScope's hardware:

Remaining configuration of networking: a hard-coded version of machine-name is downloaded to /usr/bin/machine-name, avahi-utils is installed using APT, and various systemd services are created and enabled to update the PlanktoScope OS's networking configurations based on a machine name which will be determined by machine-name from the Raspberry Pi's serial number at every boot. Additional systemd services are created and enabled so that the PlanktoScope will be accessible over some additional mDNS names (namely, pkscope.local and planktoscope.local).
Setup of the PlanktoScope hardware controller: various Python tools (pip, venv, and poetry) are installed using APT, a hard-coded version of a hard-coded Git repository (namely github.com/PlanktoScope/device-backend) is cloned, and various dependencies (both system libraries and Python packages) of the hardware controller are installed. The planktoscope-org.device-backend.controller-adafruithat.service and planktoscope-org.device-backend.controller-planktoscopehat.service systemd services are created, and the appropriate one is enabled depending on which HAT the PlanktoScope OS is being installed for. The appropriate hardware configuration file will also be copied into the location expected by the hardware controller. (Note: once the PlanktoScope hardware controller is containerized and managed in Forklift, this step will be eliminated.)
Setup of GPIO stepper initialization at boot: a systemd service is created to release the stepper motors at startup. (Note: this service currently doesn't work and will eventually be deleted or replaced.)
Setup of the PlanktoScope Node-RED dashboard: Node-RED is installed, as well as a Python package required by the adafruithat version of the PlanktoScope Node-RED dashboard (Note: the dependency on that package will eventually be removed.). The appropriate version of the PlanktoScope Node-RED dashboard and will be copied to the location expected by Node-RED depending on which HAT the PlanktoScope OS is being installed for, along with the appropriate configuration file. Finally, npm packages required by the Node-RED dashboard are installed. (Note: once the Node-RED dashboard is containerized and managed in Forklift, this step will be eliminated.)
Setup of hardware support for the PlanktoScope's real-time clock module: A kernel devicetree overlay is enabled. (Note: this currently enables support for the wrong hardware real-time clock chip, so it doesn't work yet.)
Setup of hardware support for the PlanktoScope's GPS receiver: gpsd and chrony are installed and configured. (Note: currently the configurations may be incorrect.)
Configuration of CPU overclocking: the CPU is overclocked so that the PlanktoScope segmenter will run more quickly. (Note: in the future, this will be removed.)
Cleanup: unnecessary APT and pip files are removed.