How I Over-Engineered My Dotfiles

linux dotfiles

You want to customize your Linux dotfiles, whether you already know it or not. After investing way too much time into this, I’ve decided to share some results and tricks in this blog post.

General Structure

The first thing to do is to create a general structure for your dotfiles and all associated files and scripts. I came up with this structure:

dotfiles
├── aliases
├── bashrc
├── bindings: Additional key bindings
├── exports: Shell-wide exports
├── i3: Config and bar for i3 window manager
│   ├── config
│   ├── polybar
│   └── scripts
├── Makefile: Generate readme and call shellcheck on push
[...]
├── nanorc
├── README.md
├── secrets-*
│   ├── stuff
├── termite: Terminal emulator config
├── tmux.conf
├── vimrc
└── zshrc

This whole structure can then be managed in a git repository. Using a dotfile manager like yadm makes it easy to deploy the dotfiles on a system. My own approach involves using Ansible as you will see at the end of this post. Using this, it becomes possible to deploy the same public dotfiles repository on various systems while maintaining a separate private repository for system-specific configuration values. These “secret” values can be absolute file system paths that you don’t want to expose on GitHub or in a public repository at all. All of these values are stored in the secrets-* folders. Of course you shouldn’t just go and store your keys in there, there are better ways to do this – for example using Ansible Vault.

The exports file holds configuration values that will be sourced upon spawning a shell. This file may hold configuration options for a shell, the whole system itself or applications that are going to be spawned using the shell. Its general purpose is to gather all customized options in one file.

You may have noticed the Makefile in the directory tree above. It has various purposes:

update-submodules: ## Update all submodules
	git submodule update --init --recursive && \
	git submodule foreach git pull --recurse-submodules origin master

gen-vscode-info: ## Generate a list of VS Code plugins
	code --list-extensions | xargs -d "\n" -rI % python -c "print('- [%](https://marketplace.visualstudio.com/items?itemName=' + '%' + ')')"

The Makefile is self-documented. This means that upon executing make, a help message will be displayed like this:

$ make
help                           This help
push                           Push all changes
update-submodules              Update all submodules
gen-sublime-info               Generate a list of installed sublime plugins
gen-vim-info                   Generate a list of installed vim plugins
gen-vscode-info                Generate a list of VS Code plugins
test                           Runs all the tests on the files in the repository.
shellcheck                     Runs the shellcheck tests on the scripts.

This works by generating a default action for the Makefile that parses all commands and prints the comment specified with ## after it in a list:

help: ## This help
	@awk 'BEGIN {FS = ":.*?## "} /^[a-zA-Z_-]+:.*?## / {printf "\033[36m%-30s\033[0m %s\n", $$1, $$2}' $(MAKEFILE_LIST)

Shell

After setting up the structure, the most obvious thing to do is to customize the shell.

ZSH

You’ve probably heard about zsh and the OhMyZSH framework, which also acts as a plugin manager. However, only few people seem to use antibody as a plugin manager instead. Compared to OhMyZSH it comes with less features but with a higher performance instead, since it uses concurrency when starting up a shell. The best thing is that it’s still possible to use individual OhMyZSH features in antibody. This is my plugin list:

robbyrussell/oh-my-zsh path:plugins/colored-man-pages
robbyrussell/oh-my-zsh path:plugins/extract
robbyrussell/oh-my-zsh path:plugins/z
robbyrussell/oh-my-zsh path:lib/history.zsh
robbyrussell/oh-my-zsh path:lib/key-bindings.zsh
zsh-users/zsh-syntax-highlighting
zsh-users/zsh-completions
zsh-users/zsh-autosuggestions

geometry-zsh/geometry
zdharma/fast-syntax-highlighting

Using this, you can get tabbed auto suggestions and all other neat OhMyZSH features back without being slowed down by unnecessary bloat.

Two plugins are of particular interest:

Aliases

I’m just going to drop some cool aliases in here:

Perform A Web Search From Terminal

function _web_search() {
    emulate -L zsh

    # define search engine URLS
    typeset -A urls
    urls[google]="https://www.google.com/search?q="
    urls[duckduckgo]="https://www.duckduckgo.com/?q="
    urls[startpage]="https://www.startpage.com/do/search?q="
    urls[github]="https://github.com/search?q="

    # check whether the search engine is supported
    if [[ -z "${urls[$1]}" ]]; then
        echo "Search engine $1 not supported."
        return 1
    fi

    # search or go to main page depending on number of arguments passed
    if [[ $# -gt 1 ]]; then
        # build search url:
        # join arguments passed with '+', then append to search engine URL
        # shellcheck disable=SC2154
        url="${urls[$1]}${(j:+:)@[2,-1]}"
    else
        # build main page url:
        # split by '/', then rejoin protocol (1) and domain (2) parts with '//'
        # shellcheck disable=SC2154
        url="${(j://:)${(s:/:)urls[$1]}[1,2]}"
    fi

    open_command "$url"
    return 0
}

function web_search() {
    _web_search "$@" && ~/.i3/scripts/launchBrowser.sh
}
alias google='web_search google'
alias ddg='web_search duckduckgo'
alias sp='web_search startpage'
alias github='web_search github'

# bangs
alias wiki='web_search duckduckgo \!w'
alias news='web_search duckduckgo \!n'
alias youtube='web_search duckduckgo \!yt'
alias map='web_search duckduckgo \!m'
alias image='web_search duckduckgo \!i'
alias ducky='web_search duckduckgo \!'

greping In All PDFs

function pdfgrep() {
    # shellcheck disable=SC2156
    find . -name '*.pdf' -exec sh -c '/usr/bin/pdftotext "{}" - | grep --with-filename --label="{}" --color '"$1" \;
}

cd And ls In One Command

I always execute ls after a cd, so let’s just make it one command:

alias cd="cl"
function cl {
    if [ "$#" -eq 0 ]; then
        "cd" || return
    else
        "cd" "$1" || return
    fi
    ls -lah --color=auto
}

grep For Processes

alias pg="pg"
function pg {
    pgrep -fa "$1" | grep -E --color "$1"
}

Create Encrypted Archives Using 7zip

mkzip() {
    7z -mhe=on -p a "$@"
}

Invoking mkzip encrypted.7z <directory> will then create an encrypted archive from a directory.

Search For Files And Page The Results

function s() { find . -iname "*$**" | less; }

Receive Notifications After Command Finished

alias alert='notify-send --urgency=low -i "$([ $? = 0 ] && echo terminal || echo error)" "$(history|tail -n1|sed -e '\''s/^\s*[0-9]\+\s*//;s/[;&|]\s*alert$//'\'')"'

You can use this with sleep 5 && alert "YOLO"

rm To Trash

Never remove files again by accident :D

alias rm='safedelete'
function safedelete {
    if command -v gio > /dev/null; then
        for f in "$@"
        do
            gio trash -f "$f"
        done

    elif command -v gvfs-trash > /dev/null; then
        for f in "$@"
        do
            gvfs-trash "$f"
        done

    elif [ -d "$HOME/.local/share/Trash/files" ]; then
        for f in "$@"
        do
            mv "$f" "$HOME/.local/share/Trash/files"
        done

    else
        for f in "$@"
        do
            # shellcheck disable=SC1012
            \rm "$f"
        done
    fi
}

Use Clipboard From Terminal

# pipe into ccopy --> clipboard
alias ccopy='xclip -selection clipboard'
# paste with cpaste
alias cpaste='xclip -selection clipboard -o'

Getting Internal And Public IPs

alias pubip="dig +short myip.opendns.com @resolver1.opendns.com"
alias localip="ifconfig | grep -Eo 'inet (addr:)?([0-9]*\\.){3}[0-9]*' | grep -Eo '([0-9]*\\.){3}[0-9]*' | grep -v '127.0.0.1'"

Key Bindings

Go backward and forward word with Shift + Arrow keys

# key codes determined using cat
bindkey "^[[1;2C" forward-word
bindkey "^[[1;2D" backward-word

Delete current word with CTRL+W:

bindkey "^W" backward-kill-word

FZF

If you’ve used this once, you don’t want to use a shell without it ever again. Or, to quote the developer of it:

It’s an interactive Unix filter for command-line that can be used with any list; files, command history, processes, host names, bookmarks, git commits, etc.

Yes, this is better and faster than the default history search you may know from bash:

fzf shell

It also finds files, e.g. starting from the home folder:

fzf files

By default this only works from the current folder downwards. In order do make this start from $HOME or even / you can paste this into your bindings file:

# fzf: override ctrl+t --> ctrl+a using home folder
fzf-homefolder() {
  # shellcheck disable=SC1001,SC2164
  \cd ~
  LBUFFER="${LBUFFER}~/$(__fsel)"
  local ret=$?
  zle redisplay
  typeset -f zle-line-init >/dev/null && zle zle-line-init
  # shellcheck disable=SC1001,SC2164
  \cd -
  return $ret
}
zle     -N   fzf-homefolder
bindkey '^A' fzf-homefolder

Fuzzy Search In Chrome History

And you can do even moar. You can search your entire Chrome web browsing history using fzf and its fuzzy-finding features:

# Browse chrome history
# https://junegunn.kr/2015/04/browsing-chrome-history-with-fzf/
chistory() {
    local cols sep
    cols=$(( COLUMNS / 3 ))
    sep='{::}'

    cp -f ~/.config/chromium/Default/History /tmp/h

    sqlite3 -separator $sep /tmp/h \
        "select substr(title, 1, $cols), url
    from urls order by last_visit_time desc" |
        awk -F $sep '{printf "%-'$cols's  \x1b[36m%s\x1b[m\n", $1, $2}' |
        fzf --ansi --multi | sed 's#.*\(https*://\)#\1#'
}

Tuning FZF

You can speed up fzf even more by making it use ag, a faster version of grep if you have it installed. Just add this to the exports file:

export FZF_DEFAULT_COMMAND='ag --nocolor -g ""'

It has many more features that are worth reading on the official GitHub page. Among others, you can also integrate it into Vim.

tmux

Organizing various shells, grouping them and splitting windows becomes possible with tmux. You can find various articles on tmux all over the interwebz, so I will just list my most important configurations here:

  1. Create new tmux window and prompt for a name
bind-key c command-prompt -p "window name:" "new-window; rename-window '%%'"
  1. Turn on mouse support
setw -g mouse on
  1. Use the mouse drag to re-order windows
bind-key -n MouseDrag1Status swap-window -t=
  1. Middle click to paste from the clipboard
unbind-key MouseDown2Pane
bind-key -n MouseDown2Pane run "tmux set-buffer \"$(xclip -o -sel clipboard)\"; tmux paste-buffer"
  1. Scroll History
set -g history-limit 30000
  1. Use | and - for window splitting
unbind '"'
unbind %
bind | split-window -h
bind - split-window -v

tmux plugins

Most of these and even more can be found here.

Powerline

Organize tmux windows with Powerline. This is not a tmux plugin per se but I only use it in combination with tmux. Here is the tmux.conf snippet to include it:

source ~/.powerline.conf
run-shell "powerline-daemon -q"

I display the current network load in tmux via Powerline:

{
  "segments": {
    "right": [{
        "function": "powerline.segments.common.net.network_load"
    },]
  },
}

Even Moar tmux Plugins

The tmux plugin manager tpm can be used to manage all other plugins. Here’s a list of the plugins I’m using:

set -g @plugin 'tmux-plugins/tpm'
set -g @plugin 'tmux-plugins/tmux-copycat'
set -g @plugin 'tmux-plugins/tmux-open'
set -g @plugin 'tmux-plugins/tmux-sessionist'

set -g @plugin 'laktak/extrakto'
# Enable fzf support
set -g @extrakto_fzf_tool "$HOME/.dotfiles/fzf/bin/fzf"

# Copy with mouse
set -g @plugin 'tmux-plugins/tmux-yank'
set -g @yank_selection 'clipboard'
set -g @yank_selection_mouse 'clipboard'

Let’s highlight two of them:

Automatically Launching tmux

As you will see in a following chapter, I’m using custom scripts to launch a terminal from i3. In order to launch tmux automatically, I’ve added this to my zshrc:

# auto start tmux
if [ "$TMUX" = "" ]; then
    # check for old session
    if [ "$(tmux ls | grep -v attached | wc -l)" -gt "0" ]; then
        # attach to old session
        tmux a -t "$(tmux ls | grep -v attached | cut -d ":" -f1 | head -n 1)"
    else
        # start new session - dont use exec so it's possible to run without tmux
        tmux
    fi
fi

Terminal Emulator

Termite is an efficient and configurable terminal emulator for Linux. Among other things, I’ve configured the terminal font to be Hack Font for better visuals when working with code:

[options]
font = Hack 10
clickable_url = true
scrollback_lines = 10000

Also, the color theme can be customized. I’m using a solarized color scheme.

Vim

I’m using Vundle as a plugin manager. The important ones for me are:

These two things are quite interesting too:

  1. Save files using sudo after opening it with user privileges:
" Allow saving of files as sudo when I forgot to start vim using sudo.
cmap w!! w !sudo tee > /dev/null %
  1. Persistent file history:
" Keep undo history across sessions by storing it in a file
if has('persistent_undo')
    let TheUndoDir = expand(vimDir . '/undo')
    " Create dirs
    call system('mkdir ' . vimDir)
    call system('mkdir ' . TheUndoDir)
    let &undodir = TheUndoDir
    set undofile
endif

This way you can even undo previous file changes after closing a file or rebooting the system.

i3 Window Manager

Yes, it’s true: No more dragging windows like it’s 1887.

i3 is a tiling window manager that enables an efficient workflow. You can organize applications in a configurable manner and it’s scriptable. I recommend this video series for a detailed introduction to i3. It may look complicated at first, especially with custom menus or bars like polybar, but the reality is that it’s even more complicated :D

Configuration

You need to create a config file first. The most important aspects are:

# Set the workspace icon/glyph
set $ws_music "7:<icon>"

# Assign all windows with window class `Spotify` to the `music` workspace
set $music "Spotify"
assign [class=$music] $ws_music
[...]
assign [class=$terminal] $ws_terminal
assign [class=$sublime] $ws_code

My workspace overview and polybar look like this:

Twerkspaces

bindsym $mod+y exec --no-startup-id ~/.i3/scripts/launchTerminal.sh

I’m using this approach to launch terminals, file managers and browser using a single key combination, for example:

#!/bin/bash

# shellcheck source=/dev/null
source ~/.exports

count=$(pgrep -fc "$TERMINAL")

# If theres a terminal already
if [ "$count" -gt 0 ]; then
    # if terminal already focused: always open a new instance
    if [[ $(xprop -id "$(xdotool getactivewindow)" | grep WM_CLASS | grep -v grep | cut -d ' ' -f 3 | tr -d '",' ) == "$TERMINAL" ]]; then
		# shellcheck disable=SC2091
        $(env "$TERMINAL")

    # else just focus
    else
        i3-msg "[class=$TERMINALWINDOWCLASS] focus"
    fi

# None exists, start a new one
else
	# shellcheck disable=SC2091
    $(env "$TERMINAL")
fi

The neat thing is that this re-uses existing terminal windows instead of always creating a fresh one. This way you won’t end up with 234562345 abandonned terminal windows at the end of the day. If really a new terminal is required, the same key combination can be used again in order to spawn a new terminal.

rofi

The most efficient way for me to switch and launch applications is rofi. It can be invoked from i3 with a single key combination and accepts input right after. This way, you can switch and launch applications using the fuzzy finding feature of rofi as fast as possible.

I’m using this to launch it:

bindsym $mod+q exec --no-startup-id "rofi -combi-modi window,drun -show combi -modi combi"

Alt-Tab

Yes, i3 doesn’t handle Alt-Tab directly. Boo. Just use this instead.

exec_always --no-startup-id alttab -d 1 -sc 1

Media Keys

Out of the box, i3 also doesn’t handle media keys. Boo.

Time to do copy-pasta again:

# Volume
bindsym XF86AudioMute        exec --no-startup-id "amixer -q set Master toggle"
bindsym XF86AudioRaiseVolume exec --no-startup-id "amixer -q set Master 5%+ unmute"
bindsym XF86AudioLowerVolume exec --no-startup-id "amixer -q set Master 5%- unmute"

# Media player controls
bindsym XF86AudioNext exec --no-startup-id playerctl next
bindsym XF86AudioPlay exec --no-startup-id playerctl play-pause
bindsym XF86AudioPause exec --no-startup-id playerctl play-pause
bindsym XF86AudioPrev exec --no-startup-id playerctl previous

# Screen brightness controls
bindsym XF86MonBrightnessUp exec --no-startup-id ~/.i3/scripts/setbrightness.sh 1
bindsym XF86MonBrightnessDown exec --no-startup-id ~/.i3/scripts/setbrightness.sh 0

Lockscreen

i3 doesn’t come with an integrated lockscreen. I’m using xss-lock.

Autorandr

If you’re using a docking station or other setups that involve various monitor setups, this is for you. Autorandr automatically detects the attached display setup and reconfigures the system according to a pre-defined profile. I recommend installing this from the included Makefile since this will also install hooks in the system that cause autorandr to be invoked as soon as a new monitor setup is present. That way, you can simply undock a laptop and autorandr will do the rest. It can even execute custom scripts with whatever action that may be required on a monitor change.

Automatic Linting

To ensure that all my dotfile shell scripts don’t contain obvious errors, I’m using shellcheck. This is a great utility that checks shell scripts for scripting errors and bad practices. Also, it has a great wiki that explains how to correct the errors.

To make this whole thing work, you have to configure Travis CI to run after every push to the dotfiles git repository. The job will:

  1. Clone the dotfiles repository
  2. Invoke make test from it:
test: shellcheck ## Runs all the tests on the files in the repository.

# if this session isn't interactive, then we don't want to allocate a
# TTY, which would fail, but if it is interactive, we do want to attach
# so that the user can send e.g. ^C through.
INTERACTIVE := $(shell [ -t 0 ] && echo 1 || echo 0)
ifeq ($(INTERACTIVE), 1)
	DOCKER_FLAGS += -t
endif

.PHONY: shellcheck
shellcheck: ## Runs the shellcheck tests on the scripts.
	docker run --rm -i $(DOCKER_FLAGS) \
		--name df-shellcheck \
		-v $(CURDIR):/usr/src:ro \
		--workdir /usr/src \
		r.j3ss.co/shellcheck ./.test.sh

This spins up a shellcheck docker container, which internally calls the .test.sh script that in turn performs the linting:

#!/bin/bash
# Stolen from: https://github.com/jessfraz/dotfiles/blob/master/test.sh

set -e
set -o pipefail

ERRORS=()

# find all executables and run `shellcheck` on them
# define excludes here:
for f in $(find . -type f \
        -not -iwholename '*.git*' \
        -not -iwholename './tpm*' \
        -not -iwholename "./oh-my-zsh-custom*" \
        -not -iwholename "./nanorc-folder*" \
        -not -iwholename "./powerline*" \
        -not -iwholename "./xinitrc*" \
        -not -iwholename "./fzf*" | sort -u); do


	if file "$f" | grep --quiet shell; then
		{
			shellcheck "$f" && echo "[OK]: sucessfully linted $f"
		} || {
			# add to errors
		ERRORS+=("$f")
	}
	fi
done

if [ ${#ERRORS[@]} -eq 0 ]; then
	echo "No errors, hooray"
else
	echo "These files failed shellcheck: ${ERRORS[*]}"
	exit 1
fi

To get this really over-engineered you can then send out a Telegram notification indicating success or failure to your mobile phone using curl and a secret Telegram token that’s configured in the Travis CI job.

Deployment

Creating dotfiles is cool, but testing whether they can be deployed on various Linux distributions at the same time is something else. For this, I’ve created an Ansible project. For more information regarding this, check out one of my previous blog posts.

Related

ShhPlunk: Muting the Splunk Forwarder

reverse-engineering c++ linux

Using Shellcheck and Docker to Automatically Lint Dotfiles

shell dotfiles docker travis

Automated and Tested Dotfile Deployment Using Ansible and Docker

shell dotfiles ansible travis docker