Project: Auto-updater for LibreWolf AppImages

LibreWolf hasn’t achieved widescale repo support yet for availability though there’s still plenty of options including AppImage and Flatpak format.

Personally I adore AppImages but i’ve never made the most of them so this is a rolling project for an elegant way to use LibreWolf as an AppImage. Please chime in if you have ideas/code/criticism/thoughts, i’m just going to be snow plowing through this thing and would love any guidance.

The biggest problem is making updates easy. The best place to check for updates appears to be their GitLab releases so i’ll need to pull that information, test it against last-known and download/deploy as necessary on a recurring job.

Progress 1: Getting project data from GitLab’s REST API
https://docs.gitlab.com/ee/api/#project-resources

I need the project ID which is shown on the Project overview page. Be advised Dark Reader makes it invisible.

librew

The LibreWolf the project ID for Linux is: 12829184

That makes the root API call: https://gitlab.com/api/v4/projects/12829184 (which returns basic project information).

https://docs.gitlab.com/ee/api/releases/index.html#create-a-release

Checking the release API, the release information can be obtained by appending /releases so that’s: https://gitlab.com/api/v4/projects/12829184/releases

Everything’s there! The first instance of the array contains the latest build.

I could use the “name” value to assume what the download link will be though they may change the file naming scheme so it’d be better to get it from [0]assets.links[2]

I can’t depend on that being in the same order though so I should iterating through all of them for a pattern match.

I also have to parse the JSON and preferably without an extra program like jq :confused:

grab the file somewhat seperated
curl --stderr - https://gitlab.com/api/v4/projects/12829184/releases | awk 'BEGIN {FS=",";OFS="\n\n"} {$1=$1} 1' > ~/.cache/librewolf.tmp

grab the version
grep name ~/.cache/librewolf.tmp -m1 | cut -d \" -f 4 | sed 's/v//'

grab the urls for your arch for appimage
awk '/direct_asset_url/&&/AppImage/&&/87.0.1/&&/x86_64/' ~/.cache/librewolf.tmp

I’m sure there’s an easier way to do this …

1 Like

Wow… never considered replacing commas with \n, I probably would have done something ridiculous with RegEx but that makes it really easy to search.

Using -s with curl will get rid of the curl progress output in the tmp file.

I’ll probably poke at the search terms but that’s a really good solution.

Experimenting with a few more options with some quick scratch code…

BASH + Python

JSON=$(curl -s 'https://gitlab.com/api/v4/projects/12829184/releases')
printf '%s' $JSON | python3 -c "import sys, json; print(json.load(sys.stdin)[0]['name'])"
for i in {1..11}; do
	printf '%s' $JSON | python3 -c "import sys, json; print(json.load(sys.stdin)[0]['assets']['links'][$i]['direct_asset_url'])"
done

Output:

v87.0-1
https://gitlab.com/librewolf-community/browser/linux/uploads/1fc8e9847abdef4281e70310f63381e3/LibreWolf-87.0-1.aarch64.flatpak.sig
https://gitlab.com/librewolf-community/browser/linux/uploads/4c8bb4ed933bf5fbae3d6645d1152340/LibreWolf-87.0-1.x86_64.AppImage
...ect

BASH + jq

JSON=$(curl -s 'https://gitlab.com/api/v4/projects/12829184/releases')
printf '%s' $JSON | jq '.[0].name'
for i in {0..11}; do
	printf '%s' $JSON | jq ".[0].assets.links[$i].direct_asset_url"
done

Output:

v87.0-1
"https://gitlab.com/librewolf-community/browser/linux/uploads/1fc8e9847abdef4281e70310f63381e3/LibreWolf-87.0-1.aarch64.flatpak.sig"
"https://gitlab.com/librewolf-community/browser/linux/uploads/4c8bb4ed933bf5fbae3d6645d1152340/LibreWolf-87.0-1.x86_64.AppImage"
...ect

BASH + NodeJS

JSON=$(curl -s 'https://gitlab.com/api/v4/projects/12829184/releases')
node <<EOF
	const obj = ${JSON}
	console.log(obj[0].name);

	const links = obj[0].assets.links
	for (let i=0; i < links.length; i++){
		console.log(links[i].direct_asset_url)
	}
	# Values need to be passed back to BASH, just showing how they can be pulled.
EOF

Output:

v87.0-1
https://gitlab.com/librewolf-community/browser/linux/uploads/1fc8e9847abdef4281e70310f63381e3/LibreWolf-87.0-1.aarch64.flatpak.sig
https://gitlab.com/librewolf-community/browser/linux/uploads/4c8bb4ed933bf5fbae3d6645d1152340/LibreWolf-87.0-1.x86_64.AppImage
...ect

Weighing the options

From what i’ve seen of Shell JSON parsing, there’s code that matches all property names as if there was no structure but nothing that can climb heirchachly. This’d be another option for a native solution though i’m still weighing the options. It’s a pitty there isn’t a Shell script that fully does JSON.

1 Like

Version checking

A better way of doing this might be comparing filenames instead of the last known version held in storage. If someone syncs their dotfiles across machines for example the updater may think it’s already updated when it hasn’t.

The issue with comparing filenames though is if LibreWolf adopts generic filenames without version numbers which’d make the updater think there’s never a new version.

That outcome could be solved by affixing another value into the filename like the id prior to checking and using that name for the AppImage when downloading.

Before: LibreWolf-87.0-1.x86_64.AppImage
After: LibreWolf-87.0-1.x86_64_187136.AppImage
Updater asks: Is this “LibreWolf-87.0-1.x86_64_187136.AppImage” present?

Deploying a new AppImage

As the filename will always be changing this’ll break shortcuts, the updater should creating/overwrite a softlink with the generic name librewolf that points to new file versions.

Proposed updater rundown

Every 12hrs… check the GitLab release JSON for a file change (including id affix). If there’s a change:

  1. Alert the user of the version difference using notify-send if available.
  2. Download AppImage to ~/.cache with id affixed filename.
  3. Move the file to a user defined location for their LibreWolf AppImages (default?).
  4. Create/overwrite the librewolf softlink in the user defined location pointing to the new file (default? /usr/bin?).
  5. Delete the older version unless the user defines a folder for keeping older versions.
  6. Alert the user that the new version is ready.

For 3 and 4 i’m not sure if there should be defaults and what they should be?

As always (and for any post I ever make on the forum), please criticize if anything else could be better, even if just slightly.

Ermagurd! It took me a while to find these but they’re Shell/BASH/Awk scripts that only need basic packages like GREP and Awk to read JSON.

POSIX Shell: (Reader) low footprint:

Awk: (Reader) JSON.sh ported to Awk

BASH: (Reader and writer) emulates JS command syntax and utilizes BASH arrays

BASH: (Converter) Converts JSON to CSV opening the door to CSV readers

Borrowing a bit of @grumpey’s Awk code, a one-line solution could be:

curl -s 'https://gitlab.com/api/v4/projects/12829184/releases' | \
grep -Po '"'"direct_asset_url"'"\s*:\s*"\K([^"]*)' | \
awk '/AppImage/&&/x86_64.AppImage$/ {print $1;exit;}'

Output:

https://gitlab.com/librewolf-community/browser/linux/uploads/4c8bb4ed933bf5fbae3d6645d1152340/LibreWolf-87.0-1.x86_64.AppImage
  1. curl downloads the JSON
  2. grep returns every value of every property that begins with “direct_asset_url”
  3. awk outputs the first match that ends in x86_64.AppImage which’ll be the latest release as new releases are located at the beginning of the array so older releases will come later in the string.

Improved the RegEx and included the Awk filter for a 1-search 1-line solution:

curl -s 'https://gitlab.com/api/v4/projects/12829184/releases' | \
grep -Po '"direct_asset_url"\s*:\s*"\K([^"\\]|\\.)*x86_64\.AppImage(?=")' | \
head -n1

RegEx breakdown:

  1. "direct_asset_url"\s*:\s*"\K - Capture "direct_asset_url" proceeded by a : with any number of word, digit or whitespaces on either side followed by ". Then remove the captured string using \K and proceed with capture from that point.
  2. ([^"\\]|\\.)* - Capture everything that isn’t non-escaped " or a newline (captures \" ends at ")
  3. x86_64.AppImage(?=") - Require that the capture ends with x86_64\.AppImage and that a " follows right after.

See on RegExr: RegExr: Learn, Build, & Test RegEx

Not sure if it’s possible to do a “non-greedy” search with grep -P but for now i’m using head -n1 to just return the first match.

1 Like

LibreWolf AppImage auto-updater, working prototype:

#!/usr/bin/env bash

# User adjustable variables
NotifySendTitle="=== LibreWolf Updater ==="
notify-send "$(echo -e "$NotifySendTitle\n\nVERBOSE TESTING:\n\nScript started successfully")" -t 0
StorageDirPath="$HOME/.appimages/librewolf_bin" # Where to store the binary (must be an otherwise empty folder strictly reserved for this script!)
LinkDirPath="$HOME/.appimages" # Where to put the generic symbolic link that the system should use to run the latest binary (named: librewolf)

# Create directories if they don't exist
if [ ! -d "$LinkDirPath" ]; then mkdir -p "$LinkDirPath"; fi
if [ ! -d "$StorageDirPath" ]; then mkdir -p "$StorageDirPath"; fi

# Download JSON from LibreWolf's GitLab releases and parse out the latest AppImage release
DirectAssetUrl=$(curl -s 'https://gitlab.com/api/v4/projects/12829184/releases' | grep -Po '"direct_asset_url"\s*:\s*"\K([^"\\]|\\.)*x86_64\.AppImage(?=")' | head -n1)

# Get the new and current binary filenames
NewFileName="${DirectAssetUrl##*/}"
CurrentFileName="$(ls "$StorageDirPath")"

notify-send "$(echo -e "$NotifySendTitle\n\nVERBOSE TESTING:\n\nCurrent: $CurrentFileName\nLatest: $NewFileName")" -t 0

if [ "$NewFileName" != "$CurrentFileName" ]; then
	notify-send "$(echo -e "$NotifySendTitle\n\nNew LibreWolf update available, downloading:\n\nCurrent: $CurrentFileName\nLatest: $NewFileName")" -t 0

	# Download the lastest binary to cache
	cd "$HOME/.cache"
	wget "$DirectAssetUrl"

	# Move the downloaded binary to the storage location and make it executable to the current user
	mv "./$NewFileName" "$StorageDirPath"
	chmod u+x "$StorageDirPath/$NewFileName"

	# Delete the old symlink and create a new one pointing to the new binary
	if [ -f "$LinkDirPath/librewolf" ]; then rm "$LinkDirPath/librewolf"; fi
	ln "$StorageDirPath/$NewFileName" "$LinkDirPath/librewolf"

	# Delete the older version of LibreWolf
	if [ -f "$StorageDirPath/$CurrentFileName" ]; then rm "$StorageDirPath/$CurrentFileName"; fi

	notify-send "$(echo -e "$NotifySendTitle\n\nDeployed: $NewFileName\n\nClose all active LibreWolf sessions to use the new binary.")" -t 0
fi

notify-send "$(echo -e "$NotifySendTitle\n\nVERBOSE TESTING:\n\nScript ended successfully")" -t 0

The script will obtain the URL to download the latest AppImage from GitLabs API, it’ll then extract the filename from the URL and compare it to the one in the user defined binary directory.

Aside: It does this using ls because the script has no prior knowledge of the current version so this directory needs to be reserved exclusively for the script. I could do pattern matching for the binary like LibreWolf* but the simple ls method is far more forgiving of upstream file naming changes.

If the current binary filename doesn’t match the new one, it’ll download the AppImage into the user’s .cache, then move it to the user defined binary directory, delete the old symlink in the user defined symlink directory, create a new symlink and delete the old AppImage binary.

Summary:

This script enables auto-updating and seamless switching between LibreWolf versions on the fly. If an update occurs while LIbreWolf is running, a user just needs to close their LibreWolf sessions and re-open them to be using the new version.