Architecture

No manual scripting

This post will explore a robust solution now conveniently packaged into the no-subscription tool - that is meant to do one thing and do it well. It is also a loose follow-up to the critical look at some popular “post-install” scripts utilised by users of no-subscription setups, where we identified pain points of those methods, in particular patching and strategy of re-applying the patches after an upgrade.

The better way

Debian already provides a neat mechanism for handling a situation that was to be tackled above - by the packaging system itself. A Debian package can bring in its executables - which can be scripts and configuration, but also - importantly - explicitly declare its interest to be notified when other packages are altering specific files on the system. It is the system that decides when it will trigger actions implemented by the interested package and under no other than declared rules. When the package is removed, so are its components responsible for handling these triggers.

Myths around Debian packages

First, we will dispel some myths that might surround the seemingly complicated matters:

• No dubious APT repository. A package can be installed manually - from a single downloaded file - without having to trust an unknown repository. This one-off approach will NOT keep it updated, but this is the safer way to run code from strangers. Installing a package simply means files will be copied into well-defined locations. The system will take care of removing those same files during uninstall.

• Transparency. Debian package is essentially an archive, the lack of transparency might come if it contains compiled binaries, but packages that contain only cleartext payload are as transparent to the user as a compressed script.

• Root access. It is true that a e.g. a postinst hook (not to be confused with APT hooks mentioned above) runs under root privileges when package is installed, but this is the case for any script run from through CLI in Proxmox stock system. Furthermore, these scripts have well-known place in the package archive and are easy to find - in fact, they should be the first ones of interest to audit.

Further, maintainers of Debian packages typically follow certain standards when creating them - if they want it to pass a check.

free-pmx-no-subscription

Checking thoroughly any scripts is vital. Debian packages that contain scripts are no different. For the sake of convenience, GitHub repository of free-pmx-no-subscription is available, but users should always be ready to dissect a package downloaded as already packaged .deb file first-hand - or package it themselves.

The locations to examine:

• DEBIAN/postinst - contains the script that runs with the package installation when it directly launches the two user commands (below);
• DEBIAN/triggers - list of files that when altered, e.g. by Proxmox upgrades, will launch the postinst script automatically;
• bin/ - user commands - actual scripts of no-subscription and no-nag;
• usr/lib/free-pmx/ - shared partial scripts that are sourced (utilities) or run (sub-commands) from the user command scripts (more below);
• usr/share/free-pmx/no-subscription-aptsrcs/ - templates for APT sources’ files that will replace the stock originals;
• usr/share/free-pmx/no-nag-nag-patchdefs/ - patch definitions containing templates for search&replace-like substitutions;
• etc/free-pmx/no-subscription.conf - default configuration file.

Conceptually, these are all BASH scripts relying on limited use of external tools, namely:

• perl with one-liners taking advantage of its capabilities when it comes to regular expressions;
• shasum to calculate checksums;
• gnupg to display APT repository key details;
• wget to download Proxmox release key from their site.

User commands

No direct filesystem actions are performed in the user command scripts per se, i.e. they process options and arguments, display output and make use of the sub-tools as necessary.

The actual declarations of all essential tokens can be found in the shared:

• /usr/lib/free-pmx/no-subscription-common

This is used equally by both user commands as well as the postinst script, which needs to recognise whether a file getting upgraded on the system is of its interest. This common component also provides for shared initialisation (checking configuration file, reading command options) of the user command runs.

Second (and last) much more streamlined shared component contains utility snippets, e.g. the Perl one-liners:

• /usr/lib/free-pmx/no-subscription-util

The internal sub-tools (named out below) are purposely small and take all the inputs from their arguments, not environment. The exception is output formatting when file descriptors are explicitly defined for normal output, user-visible error messaging, original standard error output (suppressed) and potential debugging output.

no-subscription

User-level documentation can be found in the manual page.

There are two internal sub-tools that are called:

• aptsrc-list-replace to perform the actual repository sources file replacement; and
• aptsrc-key-check to ensure that Proxmox release key is present on the system.

no-nag

User-level documentation can be found in the manual page.

This script relies only on patchdef-apply sub-tool, which implements the individual patching of front-end related (JavaScript) file. The user command script determines which files are to be patched, the sub-tool runs the well-known Perl one-liner with single regular expression - see standalone post on patching the “No valid subscription” notice. All patches are applied in the same manner.

The difference within this implementation lies in the fact that there are more target files to patch and potentially more patches (per target) to be applied. Patches are defined in a series of modular .patchdef files held in usr/share/free-pmx/no-nag-nag-patchdefs/. These definitions are pure variable based and refer to actual raw block files, which are further versioned. The versions correspond to the component (Proxmox package) from which they became applicable.

Patching

The reason for the “more elaborate” patching method lies in its robustness - either whole blocks of code are identified and replaced with both syntactically and semantically correct substitutes or the whole patch fails gracefully, i.e. the target file remains untouched. The worst case scenario allows for safe failing of e.g. some of the patches - a good reason to e.g. consciously look for updated definitions if need be, but by no means an operational necessity. This is why this tool aims to be reliable in the long run - even if left outdated.

The format of patch definitions also aims to provide educational background on Proxmox implementation details and encourage users to tinker with it some more - if they desire. Hopefully it will become playground for JavaScript aficionados down the road - as no more is required.

And before breaking off - a little example: a single patch definition targeting the annoying highlight of the ’no-subscription’ repository line - reveals that the original code is completely oblivious to the fact that Ceph is not highlighted.

Before pwt-32-repos-list-line:

		    if (components[0].match(/\w+(-no-subscription|test)\s*$/i)) {
			metaData.tdCls = 'proxmox-warning-row';
			err = '<i class="fa fa-fw warning fa-exclamation-circle"></i> ';

			let qtip = components[0].match(/no-subscription/)
			    ? gettext('The no-subscription repository is NOT production-ready')
			    : gettext('The test repository may contain unstable updates')
			    ;

After pwt-32-repos-list-line:

		    if (components[0].match(/\w+(test)\s*$/i)) {
			metaData.tdCls = 'proxmox-warning-row';
			err = '<i class="fa fa-fw warning fa-exclamation-circle"></i> ';

			let qtip = gettext('The test repository may contain unstable updates');

There’s basically 2 target files only (for each product):

• Proxmox Widget Toolkit (hence ‘pwt’ in the patch names above), which is common; and
• a specific one for each PVE / PBS / PMG - targeting dashboards.

No other than JavaScript files are getting patched and therefore only GUI is targeted - of a single host that the tool is being used on. Essentially, it influences the view of the otherwise unchanged situation - the other parts of stack, especially API are completely original. In other words, it is completely possible to run just one node patched like this with no influence on the rest.

Also, if the user decides to activate a license, this is still possible in the GUI with absolutely no ill-effects and will behave as per original implementation.

Updates

Expect this post to be further updated with more details depending on your feedback.