Terminal Takeaway 🥡

Tmux version would be sweeeeeet!

newsboatthere can only be one

newsboat is the RSS feeder of choice for many terminal connoisseurs but there’s an unseemly error that occurs if one attempts to run more than one instance.

$ newsboat
Starting newsboat 2.20.1...
Error: an instance of newsboat is already running (PID: 55701)

Proposing two potential shims for placement in one’s ~/.bashrc to handle newsboat startup.

Either: kill-ing the existing instance prior to running newsboat.

function newsboat (){
	PidOfNewsboat=$( pidof newsboat )
	[ -n "$PidOfNewsboat" ] && kill $PidOfNewsboat
	/usr/bin/newsboat "$@"
}

Or: Using xdotool to shift to the running instance if one exists.

function newsboat (){
	if [ -n "$( pidof newsboat )" ]; then

		# Switch to the workspace `newsboat` is on and
		# unminimize, raise and focus the window.
		xdotool search --name "newsboat" windowactivate
	else

		# `xdotool` needs a unique name applied to the
		# window so it can find it later.
		xprop -id "$WINDOWID" -f WM_NAME 8u -set WM_NAME "newsboat"

		/usr/bin/newsboat "$@"
	fi
}

Notes:

  • When using a shim, always remember to use the full path for the program you’re intending to run with the shim or it’ll create an infinite loop re-running the shim.

  • While xdotool does have the option to search by pid ( ex: xdotool search --pid $Pid ), i’ve never gotten it to work and having dug into getting a window id from a pid it gets very convoluted very quickly. Tools like xdotool, wmctrl and xwit abstract away a lot of weirdness.

Bonus

# Change the titlebar name of the current window
xprop -id "$WINDOWID" -f _NET_WM_NAME 8u -set _NET_WM_NAME "New Title"
3 Likes

Cool. This problem always bothers me with newsboat.

ShellCheck

A terminal analysis tool (also available in many plugins) for getting feedback on ways to improve your Shell scripts.

Live Demo

Click “Load random example” top-left for a nice showcase.

The goals of ShellCheck

  • "To point out and clarify typical beginner’s syntax issues that cause a shell to give cryptic error messages.
  • To point out and clarify typical intermediate level semantic problems that cause a shell to behave strangely and counter-intuitively.
  • To point out subtle caveats, corner cases and pitfalls that may cause an advanced user’s otherwise working script to fail under future circumstances."

https://github.com/koalaman/shellcheck

Available in most popular repos, i’d HIGHLY recommend checking out the project’s git repo for more information.

In action

Example of usage

sudo apt install shellcheck # Debian, Ubuntu
sudo dnf install ShellCheck # Fedora

shellcheck /usr/bin/firefox
In /usr/bin/firefox line 16:
MOZ_APP_LAUNCHER=`which $0`
             ^--------^ SC2006: Use $(...) notation instead of legacy backticked `...`.
                    ^-- SC2086: Double quote to prevent globbing and word splitting.

Did you mean: 
MOZ_APP_LAUNCHER=$(which "$0")
# Wait... firefox is a shell script??

ls -l `which firefox`
lrwxrwxrwx 1 root root 25 Oct 13  2020 /usr/bin/firefox -> ../lib/firefox/firefox.sh
less /usr/lib/firefox/firefox.sh
# Read about how 4 dudes made a debugger helper for alpha/beta phases

# Run the firefox binary directly as God intended
/usr/lib/firefox/firefox
2 Likes

Nice. I learned something about my own scripts. Namely:

  echo -e "###   "$(date '+%Y/%m/%d %H:%M:%S') >> ${podcast_directory}/${podcast_name}.log
                   ^-- SC2046: Quote this to prevent word splitting.

could be written as:

echo -e "###   $(date '+%Y/%m/%d %H:%M:%S')" >> ${podcast_directory}/${podcast_name}.log
  

I forgot that, like perl and php, variable interpolation can occur in double quotes in bash.

Mouse support in terminal with pure BASH

Adding an action to mouse input can be as easy as:

function MouseClick1() {
	echo 'Mouse button 1 was clicked!'
}

This has been a pet Shell project of mine for some time and while it’s definitely not finished this is the first easy to use version. Please excuse the lack of code comments and explanations, this project went a mile deep and I honestly wouldn’t know where to start. If there’s something you’d like to know just ask.

The GIF fps is pretty low so it catches some blank frames but it’s smoother in practice (especially in Alacritty). I also don’t know any tricks yet for making Shell scripts visually performative.

Captures

  • L/M/R mouse click
  • L/M/R mouse drag
  • L/M/R mouse button state (up/down)
  • Scroll wheel up/down
  • Row/Column Coordinates of mouse actions

mouse-in-terminal demo

mouse-in-terminal

mouse-in-terminal

#!/usr/bin/env bash

# To do: <FOSS License here>

# Must be run in the active session to catch inputs
# Examples: `source mouse-in-terminal` or `. mouse-in-terminal`

LogVerbose=
Log(){
	[ -n "$LogVerbose" ] && echo "$1"
}

FunctionExists() {
	declare -f -F "$1" > /dev/null
	return $?
}

# Enable capture
printf '\033[?1000;1002;1006;1015h'

# Declare binds
declare -A Bindings=(
	['<0;']='ReadInput MouseClick1 \<0;'
	['<1;']='ReadInput MouseClick2 \<1;'
	['<2;']='ReadInput MouseClick3 \<2;'
	['<32;']='ReadInput MouseDrag1 \<32;'
	['<33;']='ReadInput MouseDrag2 \<33;' # Supported in Alacritty
	['<34;']='ReadInput MouseDrag3 \<34;'
	['<64;']='ReadInput MouseScrollUp \<64;'
	['<65;']='ReadInput MouseScrollDown \<65;'
)

# Apply binds
for KeySeq in "${!Bindings[@]}"; do
	bind -x "\"\033[$KeySeq\":${Bindings[$KeySeq]}"
done

ReadInput() {
	[ -n "$LogVerbose" ] && Log '=== Read Input ==='
	declare -A Input=()
	Type=$1
	Input['Type']=$Type
	Axis='X'
	Buffer=''

	while read -r -n 1 -s Key; do
		[ -n "$LogVerbose" ] && Log "Reading:$Key"
		Buffer="$Buffer$Key"

		if [[ $Key == ';' ]]; then
			Axis='Y'
		elif [[ $Key =~ [0-9] ]]; then
			Input[$Axis]="${Input[$Axis]}$Key"
		else
			Input['State']=$Key
			break
		fi
	done

	if [ -n "$LogVerbose" ]; then
		Log "EscSeq=\033[$2$Buffer"
		Log "Buffer=$Buffer"
		for AKey in "${!Input[@]}"; do
		    Log "${AKey}=${Input[$AKey]}"
		done
	fi

	# If a function for the type of mouse input exists, run it.
	FunctionExists "${Type}" && ${Type}
}

What I used with it to produce the GIF demo

PS1=''
tput civis
tput setaf 2 # Set color to green
clear

function MouseClick1() {
	tput setaf 2
	DrawBox "${Input['X']}" "${Input['Y']}" "██████ Click"
}

function MouseDrag1() {
	tput setaf 1
	DrawBox "${Input['X']}" "${Input['Y']}" "██████ Drag"
}

function MouseClick2() {
	tput setaf 5
	DrawBox "${Input['X']}" "${Input['Y']}" "██████ M-Click"
}

function MouseDrag2() {
	tput setaf 5
	DrawBox "${Input['X']}" "${Input['Y']}" "██████ M-Drag"
}

function MouseClick3() {
	tput setaf 4
	DrawBox "${Input['X']}" "${Input['Y']}" "██████ R-Click"
}

function MouseDrag3() {
	tput setaf 3
	DrawBox "${Input['X']}" "${Input['Y']}" "██████ R-Drag"
}

ScrollY=10
function MouseScrollUp() {
	tput setaf 6
	(( ScrollY-- ))
	DrawBox 3 "$ScrollY" "██ Scroll Up"
}
function MouseScrollDown() {
	tput setaf 6
	(( ScrollY++ ))
	DrawBox 3 "$ScrollY" "██ Scroll Down"
}

function DrawBox() {
	X=$(( $1 - 3 ))
	Y=$(( $2 - 1 ))

	# Only draw if coords are different
	if [ "$X" != "$LastX" -o "$X" != "$LastY" ]; then

		clear
		tput home
		echo -e "\n  x=$X y=$Y state="${Input['State']}""

		LastX=$X
		LastY=$Y

		# Draw new box
		for N in 0 1 2; do
			DrawAtPos "$(( Y + N ))" "$X" "$3"
		done
	fi
}

function DrawAtPos(){
	printf '\033[s' # Save cursor position
	printf "\33[%d;%dH%s" "$@"
	printf '\033[u' # Restore cursor position
}

Showcase for attaching a function to a mouse event

# The script will call functions with the following names when
# a mouse event occurs matching what the name implies. If a
# function for the action doesn't exist, it'll do nothing.

# MouseClick1
# MouseClick2
# MouseClick3
# MouseDrag1
# MouseDrag2
# MouseDrag3
# MouseScrollUp
# MouseScrollDown

# To run a function when a user clicks mouse button 1 for
# instance, just name it "MouseClick1", example:

function MouseClick1() {
	X="${Input['X']}" # Column action occured on
	Y="${Input['Y']}" # Row action occured on
	Type="${Input['Type']}" # Source of the action (ex: MouseClick1)
	State="${Input['State']}" # M is down, m is up (as read)

	if [ "$State" = "m" ]; then HumanReadableState='up'
	elif [ "$State" = "M" ]; then HumanReadableState='down'
	fi

	echo "You did $Type at coordinate $X,$Y with the button $HumanReadableState"
}
2 Likes

A tremendous thank you to the Terminalforlife for covering this thread and helping with feedback!

If you’d like to watch the thread review live here’s the link:

If you’re not subscribed you’re missing out, TFL’s a wonderful channel.

4 Likes

Get rec -d

Recording microphone input from the terminal.

There’s an app named sox with general repo availability. It’s “the Swiss Army knife of audio manipulation” and they have great examples on their man page.

sudo apt install sox

# Record input from the default input device to file test.ogg
rec --default-device ./test.ogg
# Ctrl+c to quit

# Play recorded audio
play ./test.ogg

The format of the output above is read from the file extension. For example sox will error if given one it doesn’t support.

rec -d ./test.omg

rec FAIL formats: no handler for file extension `omg’

Examples of recording popular formats:

rec -d ./test.ogg
rec -d ./test.mp3
rec -d ./test.wav
rec -d ./test.flac

Sox can record, play and manipulate a tremendous amount of audio formats from .la to .fap (PARIS Audio File), see the full list here.

3 Likes

Used to use sox back about 10 years ago with my recording work. I remember that at that time there was a curious gotcha with the silence on the end of a file. Sox couldn’t remove silence at the end of a file but you could “reverse” the file, trim the silence at the beginning, and then “re-reverse” the file to the normal order.

I’m not sure why the development on sox stopped. Personally, I moved on to ffmpeg.

I must say that this record from the default device looks very easy.

Deleting a file named --help

Credit to Kris Occhipinti’s channel: Deleting files with Double Dashes in your SHELL - YouTube

The following will fail to delete a file named --help and instead use the file name as if it was an option.

rm --help
rm '--help'
rm "--help"
rm ~/test/*
rm \-\-help
File='--help'
rm "$File"

This applies to anything starting with a -, so -help and -------help are both interpreted as options instead of values

This is also not limited to rm, it’s a quirk of how variable expansion works in the Shell so programs can’t tell how/if things are encapsulated when they read arguments.

This may explain why some projects use syntax like myoption= for interpretation.

Looking at man bash:

--        A  --  signals the end of options and disables further option
          processing.  Any arguments after the -- are treated as  file‐
          names and arguments.  An argument of - is equivalent to --.

By using a -- before --help it removes the problem.

# We can now remove file: --help
rm -- --help

This behavior also occures in ZSH (see: Kris’s video) and probably other shells.

It’s something worth noting for situations where someone else may be in control of file names which are having commands run against them. It could be worth using -- as a best practice in those situations.

3 Likes

I thought I could do it with

rm -i *.

… but no joy.

Nice. So esoteric – but may be handy some day.

1 Like

Decide the next post

  • Self-host a high frequency VPN leak tester
  • Auto-type the contents of your clipboard
  • Self-host your own pure BASH website
  • Visualize ping over time optimized for peripheral awareness
  • Connect and select bluetooth headphones by mac address (Pipewire guide)
  • Use optical recognition to convert text in a selected area of the screen into clipboard text
  • Password creator that uses atmospheric microphone noise for absolute randomness

0 voters

Is it just a thing for TT to always have votes end in perfect draws? lol Next DLN video by any creator posted to the forum decides the pick according to the 1st letter of the first word used at/after the 5 minute mark.

b to g = Auto-type clipboard
h to m = Mic noise password gen
n to s = Self-host BASH website
t to y = Optical recognition to clipboard
a or z = Move to next letter

edit
Just posted, word at 5-min was “called” so auto-type clipboard is next though i’ll try to get to the other things that were voted on in the near future.

The thing is the vote count is just too low I guess.

True and some people may have not noticed you can do multiple votes.

It’s good information though, i’d prefer to do posts people would like most.

Auto-type the contents of your clipboard

cliptokeys

Why auto-type the clipboard?

There’s infrequent occasions I can’t paste something but really need to. Usually it’s when i’m using Spice to perform a new OS install or i’m looking at a TTY. There’s also times I have to use specific remote viewers for work that don’t have clipboard support.

Why isn’t this an xdotool one-liner script?

Auto-typing the contents of the clipboard is deceptively difficult. It can be done in a one-liner but the outcome is best summarized as spray and pray.

  • Most auto-typers including xdotool will turn accidental presses of ctrl, shift and alt into shortcut commands (see: fireworks). This is especially problematic if you’re launching the one-liner with a shortcut combo as it’ll react to these keys the moment it starts.
  • It’s a freight train that won’t stop till it’s done, hope you pasted to the right place.
  • Unless that one-liner is very long, there’s no removal of artifacts or trailing newlines. If text is selected from a document to the last letter of a line, it’ll always include a newline which will auto-run commands if pasted to a terminal and produce un-intuitive auto-type results.

All that’s fixed, introducing…

cliptokeys

( pronounced like an ancient Roman name )

  • Auto-types the contents of the keyboard at a user defined rate.
  • Stops typing the moment the current window loses focus so it can be ended with a mouse click.
  • Freezes keyboard input until typing finishes which maintains typing integrity.
  • Removes the trailing newline associated with selecting text to the end of the line.
  • When encountering a newline, the Enter key is pressed.
#!/bin/env sh

# To do: <FOSS License here>

# Examples of usage:
#     cliptokeys
#     cliptokeys 0.5   # 0.5 seconds between keystrokes

# Dependencies:
# xdotool, xinput and (xsel or xclip)

# Installation:
# mkdir -p ~/.local/bin/
# mv ./cliptokeys ~/.local/bin/
# chmod u+x ~/.local/bin/cliptokeys

TypeWaitSec=$1;
[ -z "$TypeWaitSec" ] && TypeWaitSec='0.05'

# Check for missing package(s)
if ! which xinput 2> /dev/null; then
	echo "Missing package: xinput"
	exit
elif ! which xdotool 2> /dev/null; then
	echo "Missing package: xdotool"
	exit
fi

FocusedWindowId=$(xdotool getwindowfocus)

# Get clipboard contents
if which xsel 2> /dev/null; then
	ClipOut=$(xsel -ob)
elif which xclip 2> /dev/null; then
	ClipOut=$(xclip -o -sel c)
else
	echo "Missing package: xsel OR xclip"
	exit
fi

# Handle keyboard locking
KeyboardIds=$(xinput list | grep keyboard | cut -d '=' -f 2 | cut -f 1)
KeyboardStates() {
	for Id in $KeyboardIds; do xinput --$1 $Id; done
}
trap 'KeyboardStates enable' EXIT
KeyboardStates disable

# Read the output of the clipboard one character at a time into xdotool to type
CharCount=0
ClipOutLen=$(wc -c <<< $ClipOut) # Note: Using `expr` doesn't count last newline

while read -rN1 Char; do
	(( CharCount++ ))

	# If the current window loses focus, exit
	[ "$FocusedWindowId" != "$(xdotool getwindowfocus)" ] && exit

	if [ "$Char" = $'\n' ]; then
		# If the newline isn't the last character type the Enter key. Otherwise ignore.
		[ "$CharCount" -ne "$ClipOutLen" ] && xdotool key Return
	else
		xdotool type "$Char"
	fi
	sleep $TypeWaitSec

done <<< $ClipOut

Working with remote viewers

Remote viewers often capture keystrokes so local keyboard shorcuts don’t work. You can get around this by clicking on the titlebar of the viewer, hit your chosen shortcut to start the script and it should “type” the text into the focused window inside the viewer. In my case it this works with spice viewer launched from virt-manager.

The Digital Ocean browser based console hates pasting long strings, like SSH keys, and drops tons of characters throughout the string. This probably would have saved me the grief.

Screenshot > text recognition > clipboard

Similar to edge cases you may need cliptokeys for, you may also want to get text that’s stuck in an image or an interface that doesn’t allow select to copy.

Scripting screenshot selection

# Try it
sudo [apt/dnf] install maim
maim --select --hidecursor --format=png --quality=10 "~/MyScreenshot.png"
xdg-open "~/MyScreenshot.png"

There’s A LOT of ways to take a screenshot but I wanted something as script friendly as possible. maim fit the bill well though a good improvement would be supporting what the user already has in a way that’s minimally invasive to the script.

OCR (Optical Character Recognition)

# Try it (bad result)
sudo [apt/dnf] install tesseract
tesseract "~/MyScreenshot.png" "~/MyText.txt"
cat ~/MyText.txt

tesseract is the go-to Linux solution for OCR (it’s been in the TT rules since day 1). From my personal guess it was developed to handle scanned documents way back. The images from which would be very large so if you zoomed to 100% you’d see huge text.

Solving problems: This may be why tesseract is unusable for reading small text from screenshots. Luckily the problem is fixed by up-sizing images by 400% from a great answer here.

Upsizing images for good OCR

A natural choice for scripting image editting is ImageMagik though I have a feeling there’s a lot of ways to do this and i’d love to make the script use Krita, GIMP, ect if the user already has them.

# Try it (good result)
sudo apt install imagemagik # Debian, Ubuntu
sudo dnf install ImageMagik # Fedora

mogrify -modulate 100,0 -resize 400% "~/MyScreenshot.png"
tesseract "~/MyScreenshot.png" "~/MyText.txt"
cat ~/MyText.txt

“All of your namespace is belong to us.”
~ 2050/1/1, ImageMagik upon conquering all of Linux namespace.

Combining the above, I present…

screentoocr (beta)

A project with a terrible name that runs your area selected screenshot through OCR and places the interpreted text in your clipboard.

This project is somewhat accurate but would benefit a lot from better image preparation for OCR such as sharpening edges and contrast.

Problem:

# sed -i 's/ForumCantShowThisCharacter//g' "$TmpFilePathPrefix.txt"

This command is intended to remove instances of an artifact character that keeps showing up for reasons I don’t know but the forum won’t show it. Just run the program, copy the character into the sed command and uncomment it.

#!/usr/bin/env sh

# Dependencies:
# ImageMagick, sed, xsel/xclip, tesseract, maim

TmpFilePathPrefix="$HOME/.cache/shottoclip$$"

function CleanUp(){
	[ -f "$TmpFilePathPrefix.txt" ] && rm "$TmpFilePathPrefix.txt"
	[ -f "$TmpFilePathPrefix.png" ] && rm "$TmpFilePathPrefix.png"
}
trap CleanUp EXIT

# User drags cursor over what they want OCR'ed
maim --select --hidecursor --format=png --quality=10 "$TmpFilePathPrefix.png"

# Resize to 400%
mogrify -modulate 100,0 -resize 400% "$TmpFilePathPrefix.png"

# OCR the image
tesseract "$TmpFilePathPrefix.png" "$TmpFilePathPrefix" &> /dev/null

# Remove an artifact that keeps showing up for some reason
# sed -i 's///g' "$TmpFilePathPrefix.txt"

# Place text in clipboard
if [ $(which xsel 2> /dev/null) ]; then
	cat "$TmpFilePathPrefix.txt" | xsel --clipboard
elif [ $(which xclip 2> /dev/null) ]; then
	xclip -selection clipboard "$TmpFilePathPrefix.txt"
else
	echo "Missing packages: xsel OR xclip"
	exit
fi

BONUS: Screenshot > clipboard

Putting an area selected screenshot in your clipboard as an image:

#!/usr/bin/env sh

# Dependencies: maim, xclip

TmpPath="$HOME/.cache/ss-to-clip-tmp$.png"
trap '[ -f "$TmpPath" ] && rm "$TmpPath"' EXIT
maim --select --hidecursor --format=png --quality=10 "$TmpPath"
xclip -selection clipboard -t image/png "$TmpPath"
1 Like

Commenting multi-line commands

Multi-line commands combined with full length flags go a long way for decipherability at a glance. Example:

# Try it:
df \
--local \
--all \
--human-readable

…but they can be taken one step further with comments.

It took a while to figure this out and some asking around but the solution is a comment within a subshell prior to the \ next line indicator.

Below is an rsync explicitly defining what it’s doing (favoring no alias bundles like --archive) and the comments come from the man page.

Performance Note: As creating multiple subshells prior to the command running isn’t performant, this is for situations that aren’t millisecond sensitive (although technically commenting anywhere isn’t performative as shell scripts are an interpreted language).

#!/usr/bin/env sh
rsync \
	--update                `# skip files that are newer on the receiver `\
	--links                 `# copy symlinks as symlinks `\
	--atime-preserve=system	`# preserve access times `\
	--hard-links            `# preserve hard links `\
	--executability         `# preserve executability `\
	--acls                  `# preserve ACLs (implies --perms) `\
	--xattrs                `# preserve extended attributes `\
	--owner                 `# preserve owner (super-user only) `\
	--group                 `# preserve group `\
	--devices               `# preserve device files (super-user only) `\
	--specials              `# preserve special files `\
	--times                 `# preserve modification times `\
	--atimes                `# preserve access (use) times `\
	--devices               `# preserve device files (super-user only) `\
	--itemize-changes       `# output a change-summary for all updates `\
	--delete-during         `# receiver deletes during the transfer `\
	--force                 `# force deletion of dirs even if not empty `\
	--recursive             `# recurse into directories `\
	MySourceDir \
	MyDestinationDir
1 Like

Put the current date in your clipboard with a keyboard shortcut using XFCE4.

XFCE4 (maybe others) won’t run scripts as a shortcut command but can pass them as an argument to something that will. Ex: /usr/bin/sh -c '<command>'

App Menu > Keyboard > Application Shortcuts (tab) > + Add

# Format: YYYY-MM-DD

# Using xsel
/usr/bin/sh -c 'date +%Y-%m-%d | xsel --clipboard'

# Using xclip
/usr/bin/sh -c 'date +%Y-%m-%d | xclip -selection clipboard'
2 Likes