Sam Gleske

Create a Nexus proxy for downloading GitHub releases

Fri, 06 Oct 2023 00:00:00 -0400

What to proxy from GitHub
Routing rules
Proxying GitHub releases
How to download

In large environments, it makes sense to cache artifacts locally in-network where downloads occur frequently. An example of this is a proxy for CICD environments performing regular builds.

This guide covers how to configure Sonatype Nexus for downloading releases from GitHub. In general, you should only cache items you expect to never change or wouldn’t want to change.

What to proxy from GitHub

There’s three types of GitHub downloads which I think fit this description:

GitHub releases
Source code archives downloaded by Git SHA1
Source code archives downloaded by Git tag

The least reliable of the three types are GitHub releases and Git tag because open source projects can change these. The intent of a proxy is to provide stability in an environment so even if the upstream changes the artifact the local cache would not change (nor would I want it to).

Routing rules

Routing rules in Nexus restrict what content is allowed to be resolved from a repository within Nexus. Because GitHub has standard URLs for downloading for all projects, you can use a regular experession in a routing rule.

Routing rule settings:

Name: GitHubReleases
Description: Download GitHub Releases. Download source code archives via Git SHA1 or tag.
Mode: Allow (meaning any requests which do not match regex are not allowed)

Finally, Matchers are regular expressions which restrict downloads. You only need two matchers.

/[^/]+/[^/]+/releases/download/[^/]+/.+

/[^/]+/[^/]+/archive/([0-9a-f]{40}|refs/tags/.+)\.(zip|tar\.gz)

The first matcher covers downloading from GitHub releases. The second matcher covers downloading archives from Git SHA1 or Git tag; in both, zip or tar.gz formats.

Proxying GitHub releases

Nexus has a raw repository type which can be configured to proxy URLs. Here are its settings.

Name: github
Content Disposition: Attachment
Remote Storage: https://github.com
Auto blocking enabled: enable (my opinion; if there’s GitHub outages)
Maximum component age: -1 (for release artifacts)
Maximum metadata age: 30 (minutes; my opinion)
Strict Content Type Validation: Unchecked i.e. not strict
Set the Routing Rule to the rule created in the previous section.
Not found cache enabled: enable it
Not found cache TTL: 5 (minutes; my opinion)

How to download

If you did all of this on a local host Nexus then you would prefix the repository URL with the GitHub URL replacing github.com with your Nexus repository.

Instead of

https://github.com/samrocketman/yml-install-files/releases/download/v2.14/universal.tgz

You would download

http://localhost:8081/repository/github/samrocketman/yml-install-files/releases/download/v2.14/universal.tgz

Securing tmp space

Sat, 29 Apr 2023 00:00:00 -0400

Securing tmp
Preparing on-disk tmp
Adding fstab entries for boot
Mount options explained

I use Linux as a Desktop and this is one thing (of many) I do to better secure it for general internet browsing.

Securing tmp

There’s three main temporary files paths in Linux which is standard.

/tmp on disk temp space
/var/tmp on disk temp space
/dev/shm in-memory temp space

These should be secured so that programs cannot be executed from them. This prevents a wide array of attacks which assume tmp is capable of executing. Also, since /dev/shm is in-memory it should be limited. I like to limit it to at least 1GB or smaller since not a lot of programs use it. 1GB is a safe limit and it will only take up memory if files are written to it.

Preparing on-disk tmp

I like to share /tmp and /var/tmp with the same file system. This limits the combination of both spaces. The following prepares an on-disk image meant to be used as temporary file storage. The following commands are executed as root.

mkdir /mnt/tmp /root/images

# create a 2GB file-based filesystem
dd of=/root/images/tmp2g if=/dev/zero bs=1024M count=2
mkfs.ext4 /root/images/tmp2g

# change permissions to match /tmp with sticky bit
mount -o loop /root/images/tmp2g /mnt/tmp
chmod 1777 /mnt/tmp
umount /mnt/tmp

# clean up
rmdir /mnt/tmp

Adding fstab entries for boot

With the new file system stored under /root we’ll be able to mount /tmp with a file system limited to 2GB. To finish securing temporary files you’ll want to add the following /etc/fstab entries.

/root/images/tmp2g /tmp ext4 loop,strictatime,noexec,nodev,nosuid 0 0
/tmp /var/tmp none bind 0 0
tmpfs /dev/shm tmpfs defaults,noexec,nodev,nosuid,seclabel,size=1G 0 0

A bind mount was created between /tmp and /var/tmp so they share the same space-limited filesystem. Once you reboot, the temporary filesystems will all be updated (you don’t need to reboot but this is the lazy approach).

Mount options explained

loop will set up a loopback interface. This treats a file like a device (e.g. USB stick).
strictatime It updates the access time each time a file or its cache is accessed. This increases the disk writes.
noexec does not allow executables to run (even if their execute bit is set).
nodev character or block devices are not allowed on the file system. Examples include /dev/null, /dev/zero, etc. so devices with similar behavior are not allowed in /tmp.
nosuid will not honor set-user-ID and set-group-ID bits or file capabilities when executing programs from this filesystem. This may be redundant with noexec but in general it is a good practice to have this set with noexec.
defaults will use the default options: rw, suid, dev, exec, auto, nouser, and async.
seclabel indicates that the filesystem is using xattrs for labels and that it supports label changes by setting the xattrs. If you’re not using SELinux, then this is not necessary.
size=1G will limit the size of the in-memory tmpfs to 1GB.

Guide to production docker images

Tue, 25 Oct 2022 00:00:00 -0400

What makes a good production image
An example application
Diving into “from scratch”
Creating our most minimal Docker container
ARM support
Full Docker example with TLS CA and tzinfo
Working with shared libraries
Summary

What makes a good production image

I would like to cover initial basics agnostic to Linux distrobution or recommended base images. The intent here is to start by defining what makes a good Docker image for an application.

The Docker image should only have dependencies necessary for running the application. This provides a few benefits.
- Fewer extra software means improved security.
- A smaller Docker image which means your app or API starts faster in cloud provisioning services because it is fast to download.
The app should log to stdout and stderr. Not write logs to disk. This is important because Docker can handle log management but not if it is written to disk.
The app should launch in the foreground and manage its own child processes from there. Docker does a good job at process management. Let Docker manage passing signals to your application.
Even if launched in the foreground, you’ll still want the docker image to have a process dedicated to PID 1 functions. If you’re not familiar with this issue I recommend the original phusion blog post calling out the problem. My preference is to bake in Yelp dumb-init for a couple of reasons. dumb-init is small and I don’t have to worry about the orchestration service supporting --init flags for Docker.
- If not addressing this issue while in production, you’ll find that occasionally your service will auto-scale. The auto-scaling could be a symptom due to being constrained with resources. It won’t be immediately clear without good monitoring and easy to mistake for traffic balancing.
When your application exits, provide a meaningful exit code. Zero for success and non-zero for failure. Depending on the complexity of your application it might assist troubleshooting if you provide documentation around failure exit codes. The documentation could even include tips on what went wrong and how to resolve it such as a run book for a support team.
Your application should run as a normal user and not root user inside of the container. This improves security of both your app and the host in which it resides.
Your application should start the container with default Linux capabilities provided by Docker. If possible, try not to add any extra capabilities which would reduce the security if your application runtime.
If your app requires outbound network connectivity involving TLS, then it should include standard TLS certificate authorities for whatever cloud or platform you’re working from.
If your app requires timezone information then you should include Linux timezone files.
Even if you’re using a minimal image, you’ll want to follow the Linux filesystem hierarchy standard and provide all of the above recommendations even if building from scratch. I will provide an example at the end of this article.
If you add SHELL ["/bin/sh", "-exc" ] at the beginning of your Dockerfile you get better debug output from /bin/sh while Docker is building the image. This helps narrow down issues to the exact command that fails when a Docker build fails. It also forces a multi-statement RUN command to exit with an error; without the need to use && between commands.
- Alternately, if you prefer bash to be you Docker RUN shell you can set SHELL ["/bin/bash", "-exo", "pipefail", "-c" ]; see bash manual for the set builtin.
- If you only want to debug a single RUN command you can prefix commands with set instead of changing all RUN shells. For example, instead of RUN \ within examples, you could remove SHELL and add a single RUN set -ex; \.

There are other good practices in general for applications such as integrating application performance monitoring (APM), unit testing with code coverage, documentation, and data flow diagrams. However, this article is mostly highlighting Docker so I don’t plan to dive into these other topics here. They are still good to build in as observability will add to service quality.

An example application

Flask is a framework based on the current/old standard for Python web frameworks: WSGI.
FastAPI is based on Starlette, which uses the newer standard for asynchronous web frameworks: ASGI.

For the purposes of this write-up I developed a simple flask application (a REST API with only one endpoint from the flask REST API tutorial). If you’re starting an API from scratch you might want to consider FastAPI, instead. However, for the purposes of this writeup the application or framework doesn’t matter. The real star is showing examples of production ready Docker images.

You can go view the example at docker-production-ready-flask. It follows recommendations from Flask documentation on how to deploy flask to production paired with Docker best practices. The project README goes into more detail.

Diving into “from scratch”

Ivan Velichko has a great writeup on, why Google distroless instead of “from scratch”? The short version of the article is the following.

FROM scratch images are bad (e.g. minimal go apps from scratch) because:

They can’t run as non-root.
Timezone information is missing so you will have Go bugs encountered at runtime.
Standard directories (e.g. /var/tmp and /tmp) are missing which cause issues with native Go calls that rely on temporary files.

Let’s inspect and verify. For the purposes of inspection I will pull in a statically compiled version of bash.

Here’s our minimal Dockerfile for inspection.

FROM alpine
ADD https://github.com/robxu9/bash-static/releases/download/5.1.016-1.2.3/bash-linux-x86_64 /bash
RUN chmod 755 /bash
FROM scratch
COPY --from=0 /bash /

Now to build and look around.

docker build -t minimal .

Launch the container.

docker run -it --rm minimal /bash

Look around. There’s no standard GNU utilities so we’ll be limited to shell built-in functions.

bash-5.1# echo *
bash dev etc proc sys

bash-5.1# echo dev/* 
dev/console dev/core dev/fd dev/full dev/mqueue dev/null dev/ptmx dev/pts
dev/random dev/shm dev/stderr dev/stdin dev/stdout dev/tty dev/urandom dev/zero

bash-5.1# echo etc/*
etc/hostname etc/hosts etc/mtab etc/resolv.conf

bash-5.1# echo $$
1

Some observations:

It is a minimal container.
bash is our shell and proc/sys are kernel filesystems so no need to inspect them since they are kernel API mounts.
Because the bash prompt has # it means we are running as root user.
We don’t need to worry about standard devices or filesystems like /dev/shm, /dev/zero, etc.
We are missing temporary directories such as /tmp and /var/tmp.
We’re running as PID 1 (verified with echo $$). Bash can’t handle PID 1 signals. Any app you put into a from scratch image will also unlikely handle signals unless the app is explicitly designed for it.
/etc has standard networking mounts provided by Docker but we don’t have standard user files like /etc/passwd and /etc/group.

Creating our most minimal Docker container

Accounting for best practices (except for TLS certificates and tzinfo) let’s create an example Dockerfile.

ARG base=alpine
FROM ${base}

SHELL ["/bin/sh", "-exc"]
RUN \
  # Prerequisites
  apk add --no-cache build-base; \
  # Directory structure and permissions
  mkdir -p base/bin base/tmp base/var/tmp base/etc base/home/nonroot base/sbin base/root; \
  chmod 700 /root; \
  chown root:root /root; \
  chmod 1777 base/tmp base/var/tmp; \
  chown 65532:65532 base/home/nonroot; \
  chmod 750 base/home/nonroot; \
  # UID and GID
  echo 'root:x:0:' > /base/etc/group; \
  echo 'nonroot:x:65532:' >> /base/etc/group; \
  echo 'root:x:0:0:root:/root:/sbin/nologin' > /base/etc/passwd; \
  echo 'nonroot:x:65532:65532:nonroot:/home/nonroot:/sbin/nologin' >> /base/etc/passwd; \
  # init binary for PID 1
  wget -O base/bin/dumb-init https://github.com/Yelp/dumb-init/releases/download/v1.2.5/dumb-init_1.2.5_"`uname -m`"; \
  chmod 755 base/bin/dumb-init; \
  # nologin binary
  echo 'int main() { return 1; }' > nologin.c; \
  gcc -Os -no-pie -static -std=gnu99 -s -Wall -Werror -o base/sbin/nologin nologin.c; \
  echo "Minimal Container version $VERSION" > /etc/issue


# Add our example program (bash)
# Note: you don't need this for your own application.  In this case static bash
#   is the example application running in user context within a minimal image
# Comment out these lines and update CMD for your own app.
RUN \
  wget -O base/bin/bash https://github.com/robxu9/bash-static/releases/download/5.1.016-1.2.3/bash-linux-"`uname -m`"; \
  chmod 755 base/bin/bash

FROM scratch
COPY --from=0 /base/ /
ENTRYPOINT ["/bin/dumb-init", "--"]
USER nonroot
ENV HOME=/home/nonroot USER=nonroot
WORKDIR /home/nonroot
CMD ["/bin/bash"]

Build it.

docker build -t minimal .

Let’s look around!

docker run -it --rm minimal
bash-5.1$

bash-5.1$ echo $$
7

bash-5.1$ /proc/1/exe --version
dumb-init v1.2.5

bash-5.1$ pwd
/home/nonroot

bash-5.1$ while IFS=$'\0' read -r -d $'\0' line; do echo "$line"; done < /proc/self/environ
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
HOSTNAME=7cdf06c28dd1
TERM=xterm
HOME=/home/nonroot
USER=nonroot

bash-5.1$ echo /sbin/* /bin/* /var/* /t*
/sbin/nologin /bin/bash /bin/dumb-init /var/tmp /tmp

bash-5.1$ cd /; echo *
bin dev etc home proc root sbin sys tmp var

bash-5.1$ echo hello > /tmp/file

bash-5.1$ echo /tmp/*
/tmp/file

Observations

We’re running as a normal user because the bash prompt has a dollar sign $ (not root).
Our shell is no longer PID 1.
We used the /proc kernel file system to verify that PID 1 is the dumb-init process. If you’re curious about /proc you can read about it within the kernel doc proc.txt.
We are in the nonroot home directory and our environment reflects the user environment.
We have our binaries in a standard paths. Following the Linux filesystem hierarchy standard.
/sbin/nologin will return non-zero if a user login is attempted.
As a user, we can write to the /tmp filesystem verifying its permissions are set correctly along with the sticky bit.

ARM support

This minimal Docker image is meant to be cross platform for both AMD64 and ARM64.

docker buildx build --platform linux/arm64 --build-arg base=arm64v8/alpine -t minimal-arm .

Or if you’re on a computer which is already native ARM you can run the original docker command and it should build just fine.

docker build -t minimal .

Full Docker example with TLS CA and tzinfo

I work a lot with Amazon web services. In my case, it makes sense to copy certificates and timezone information from amazonlinux:2. However, if you’re in another cloud provider or data center; then use whatever base image of your choice. Copying these paths are pretty standard for nearly any Linux distribution.

ARG base=alpine
FROM ${base}

SHELL ["/bin/sh", "-exc"]
RUN \
  # Prerequisites
  apk add --no-cache build-base; \
  # Directory structure and permissions
  mkdir -p base/bin base/tmp base/var/tmp base/etc base/home/nonroot base/sbin base/root; \
  chmod 700 /root; \
  chown root:root /root; \
  chmod 1777 base/tmp base/var/tmp; \
  chown 65532:65532 base/home/nonroot; \
  chmod 750 base/home/nonroot; \
  # UID and GID
  echo 'root:x:0:' > /base/etc/group; \
  echo 'nonroot:x:65532:' >> /base/etc/group; \
  echo 'root:x:0:0:root:/root:/sbin/nologin' > /base/etc/passwd; \
  echo 'nonroot:x:65532:65532:nonroot:/home/nonroot:/sbin/nologin' >> /base/etc/passwd; \
  # init binary
  wget -O base/bin/dumb-init https://github.com/Yelp/dumb-init/releases/download/v1.2.5/dumb-init_1.2.5_"`uname -m`"; \
  chmod 755 base/bin/dumb-init; \
  # nologin binary
  echo 'int main() { return 1; }' > nologin.c; \
  gcc -Os -no-pie -static -std=gnu99 -s -Wall -Werror -o base/sbin/nologin nologin.c; \
  echo "Minimal Container version $VERSION" > /etc/issue


# Add our example program (bash)
# Note: you don't need this for your own application.  In this case static bash
#   is the example application running in user context within a minimal image
# Comment out these lines and update CMD for your own app.
RUN \
  wget -O base/bin/bash https://github.com/robxu9/bash-static/releases/download/5.1.016-1.2.3/bash-linux-"`uname -m`"; \
  chmod 755 base/bin/bash

# Pull TLS certifactes and timezone info from amazon
FROM amazonlinux:2
RUN \
  mkdir -p base/etc base/usr/share; \
  cp -r /etc/ssl /etc/pki base/etc/; \
  cp -r /usr/share/zoneinfo base/usr/share/


FROM scratch
COPY --from=0 /base/ /
COPY --from=1 /base/ /
ENTRYPOINT ["/bin/dumb-init", "--"]
USER nonroot
ENV HOME=/home/nonroot USER=nonroot
WORKDIR /home/nonroot
CMD ["/bin/bash"]

Working with shared libraries

I wrote a script named copy-bin.sh which makes it easy to copy binaries which were compiled with shared object dependencies.

For example, if you wanted to copy busybox and all of its utility references into an image created from scratch to inspect it; the following docker image would apply.

FROM alpine as busybox
SHELL ["/bin/sh", "-exc"]
RUN \
  wget -qO /usr/local/bin/copy-bin.sh https://raw.githubusercontent.com/samrocketman/home/main/bin/copy-bin.sh; \
  chmod 755 /usr/local/bin/copy-bin.sh
RUN \
  copy-bin.sh --prefix /base --ldd /bin/busybox --links /bin:/sbin:/usr/bin:/usr/sbin

FROM scratch
COPY --from=busybox /base/ /
CMD ["/bin/sh"]

With the above content in a Dockerfile; create the image.

docker build -t scratch-busybox -f Dockerfile.busybox .
docker run -it --rm scratch-busybox ls

Summary

If you’re building statically compiled binaries then you can rely on all of the recommended best practices for Linux and Docker to have both a safer and smaller Docker image than one provided to you by a Linux distribution.

Here’s a small summary of the image sizes. All sized exclude bash assuming you would remove it to replace it with a statically compiled application.

76.4kB minimal example. If your app doesn’t need CA certificates or timezone information then you get all of the security and stability goodies at a very small storage price.
3.53MB when including TLS CA certificates and timezone information.

Windows on Linux: SENA and USB Device Support

Wed, 25 Aug 2021 00:00:00 -0400

Introduction and Audience
Install VirtualBox and Vagrant
- Download Windows
- Add Windows to Vagrant
Provisioning Windows
Adding USB 3 support
- Attaching USB Devices to VirtualBox

Introduction and Audience

This article is loosely based on a GitHub gist I wrote for updating my SENA, but this includes more convenience automation via Vagrant; details to follow.

This post describes how to provision Windows on Linux for free. This process is legal and does not involve illegally downloading software. Microsoft provides free Windows virtual machines through its Microsoft Edge Developer program at no cost to you, the user.

Minimum specifications to follow along with this article:

4 core CPU which has hardware virtualization extensions (check /proc/cpuinfo for vme, vmx, or both). You may have to enable virtualization in your BIOS or UEFI which is not covered in this article.
16GB RAM.
30GB harddrive space free. Windows 10 is pretty large and you’ll require three times the size of Windows. You’ll need three times the size because you’ll download a Windows virtual machine as a zip, extract the zip, and then install it within Vagrant. You can clean up the download files when you’re done which will free up 14GB of space.

Audience

I’ll assume you use Linux day to day and have some familiarization with computer hardware. While these instructions attempt to take into account beginners to Linux it is not geared towards a beginner. If you’re new to Linux and this article is confusing, then I recommend the following tutorials. Once you’ve gone through them you can come back to this page and try again.

Linux Journey web tutorials:

Grasshopper: Command Line
Grasshopper: Advanced Text-Fu although I personally like vim. If you install vim on your machine, then you can run the vimtutor command to learn how to use it.
Journeyman: Devices

Why Windows on Linux?

It is a reality that I’ll have some piece of hardware such as a Garmin GPS or a SENA Motorcycle headset which needs firmware updates but only works from Windows or Mac. This guide is intended to provide instruction on how to connect such hardware and update it from Linux using a Windows virtual machine at no cost of ownership.

About Software

This article will use a mix of free and proprietary software; all of which costs no money to use. The following is a summarized list. Refer to the website of each software for their given licensing and usage.

Microsoft Windows. No explanation required, I think.
Vagrant is software used to automate provisioning virtual machines. It provides convenience around installation and booting of virtual machines for VirtualBox.
VirtualBox is a hypervisor and virtual machine technology. It is widely availabot for Linux, Mac, and Windows.

My machine

The following are the specifications for the machine I used for testing commands and writing this blog post.

Software specifications:

$ uname -rms
Linux 5.4.0-81-generic x86_64

$ lsb_release -a
LSB Version:	core-9.20170808ubuntu1-noarch:printing-9.20170808ubuntu1-noarch:security-9.20170808ubuntu1-noarch
Distributor ID:	Ubuntu
Description:	Pop!_OS 18.04 LTS
Release:	18.04
Codename:	bionic

$ bash --version | head -n1
GNU bash, version 4.4.20(1)-release (x86_64-pc-linux-gnu)

Hardware specifications:

Make/model: system76 Darter Pro model darp5
Processor: Intel(R) Core(TM) i7-8565U CPU @ 1.80GHz 4 core/8 threads with vme/vmx extensions. See /proc/cpuinfo to check if your processor has these virtualization extensions.
Memory: 32GB RAM

Install VirtualBox and Vagrant

You can find instructions for how to install VirtualBox and Vagrant from the respective project website.

Install VirtualBox on Pop OS 18.04 and Ubuntu 18.04.

sudo apt-get install virtualbox virtualbox-dkms virtualbox-ext-pack virtualbox-guest-additions-iso virtualbox-qt
sudo usermod -a -G vboxusers "$USER"

Note: Changes made by usermod command will not take effect until after you start a new login session. I recommend restarting your computer for simplicity sake. If you’re on another Linux OS you can check the /etc/group file for the VirtualBox group you need to add to your user.

Installing Vagrant on Pop OS 18.04 and Ubuntu 18.04.

curl -fsSL https://apt.releases.hashicorp.com/gpg | sudo apt-key add -
sudo apt-add-repository "deb [arch=amd64] https://apt.releases.hashicorp.com $(lsb_release -cs) main"
sudo apt-get update
sudo apt-get install vagrant

You’ll also want to install a plugin which automates installing VirtualBox Guest Additions.

vagrant plugin install vagrant-vbguest

Download Windows

Visit Microsoft Edge Developer Virtual Machines page.
Under Virtual Machines choose MSEdge on Win10 x64.
Choose VM Platform: Vagrant.
Check your Downloads folder for MSEdge.Win10.Vagrant.zip.

Add Windows to Vagrant

Go to your downloads directory and unzip the Zip file.

cd ~/Downloads/
unzip MSEdge.Win10.Vagrant.zip

Add the resulting *.box file to Vagrant with a name of your choice. Since Microsoft offers multiple versions of Windows I recommend naming the box similar to the version of Windows you’ve downloaded. This will give you additional flexibility for importing additional versions of Windows.

vagrant box add ./MSEdge\ -\ Win10.box --name windows/10edge

Provisioning Windows

Windows will be automatically provisioned by using Vagrant. Vagrant automates booting and installing VirtualBox virtual machines. Vagrant uses a file named Vagrantfile to describe a virtual machine and its operating system.

Vagrantfile

Create a Windows directory which will simplify managing your Vagrant virtual machine. All following commands will assume this working directory.

mkdir ~/Windows
cd ~/Windows/

Create a file named ~/Windows/Vagrantfile. Ensure it has the following contents.

Vagrant.configure("2") do |config|
  config.vm.box = "windows/10edge"
  config.vm.box_check_update = false
  # Windows remote management settings
  config.vm.guest = :windows
  config.vm.communicator = "winrm"
  config.winrm.username = "IEUser"
  config.winrm.password = "Passw0rd!"
  if Vagrant.has_plugin?("vagrant-vbguest") then
    config.vbguest.auto_update = false
  end

  config.vm.provider "virtualbox" do |vb|
    # Hardware settings
    vb.gui = true
    vb.cpus = "4"
    vb.memory = "8192"

    # Operating system
    vb.customize ["modifyvm", :id, "--ostype", 'Windows10_64']

    # Video Settings with remote desktop disabled
    vb.customize ["modifyvm", :id, "--vram", "256", "--accelerate3d", "on", "--vrde", "off"]

    # DVD Drive for VBox Guest Additions
    vb.customize ["storageattach", :id, "--storagectl", "IDE Controller", "--port", "0", "--device", "1", "--type", "dvddrive", "--medium", "emptydrive"]

    # USB 3 support; it should only run initially
    unless File.exists? "usb-setup-complete"
      vb.customize ["storagectl", :id, "--name", "USB", "--add", "usb", "--controller", "USB", "--hostiocache", "on"]
      vb.customize ["modifyvm", :id, "--usb", "on", "--usbxhci", "on"]
    end
  end

  # Switch logic for USB 3 support
  config.trigger.after :up do |trigger|
    trigger.info = "Checking usb-setup-complete..."
    trigger.run = {inline: "bash -c '[ -f usb-setup-complete ] || touch usb-setup-complete'"}
  end
  config.trigger.after :destroy do |trigger|
    trigger.info = "Removing usb-setup-complete..."
    trigger.run = {inline: "bash -c '[ ! -f usb-setup-complete ] || rm -f usb-setup-complete'"}
  end
end

Note: if you use another version of Windows be sure to change the operating system --ostype. You can view a list of all supported OS types by running the following command in your terminal.
VBoxManage list ostypes

First-time Windows setup

First-time setup only needs to be performed on initial provisioning and does not need to be repeated for the life of the virtual machine.

Start Windows via Vagrant.
Log into Windows and enable Windows Remote Management.
Ensure VirtualBox Guest Additions is installed. This is done automatically, but you want to make sure it was successful.

Provision and start Windows with the following command.

vagrant up

As soon as the Windows login screen is visible log into the IEUser using the password: Passw0rd!. From the start menu search for cmd. Right click on Command Prompt and select Run as administrator. From the administrator command prompt run the following command:

WinRM quickconfig
sc config WinRM start=auto

Choose Y to enable Windows Remote Management. The second command will force Windows Remote Management to always autostart without delay. From this point, Vagrant should automatically continue with setup and installation of VirtualBox Guest Additions.

This log shows the full output when you’ve completed all of the steps successfully.

$ vagrant up
Bringing machine 'default' up with 'virtualbox' provider...
==> default: Importing base box 'windows/10edge'...
==> default: Matching MAC address for NAT networking...
==> default: Setting the name of the VM: Windows_default_1629924069660_49626
==> default: Clearing any previously set network interfaces...
==> default: Preparing network interfaces based on configuration...
    default: Adapter 1: nat
==> default: Forwarding ports...
    default: 5985 (guest) => 55985 (host) (adapter 1)
    default: 5986 (guest) => 55986 (host) (adapter 1)
    default: 22 (guest) => 2222 (host) (adapter 1)
==> default: Running 'pre-boot' VM customizations...
==> default: Booting VM...
==> default: Waiting for machine to boot. This may take a few minutes...
    default: WinRM address: 127.0.0.1:55985
    default: WinRM username: IEUser
    default: WinRM execution_time_limit: PT2H
    default: WinRM transport: negotiate
==> default: Machine booted and ready!
==> default: Checking for guest additions in VM...
    default: The guest additions on this VM do not match the installed version of
    default: VirtualBox! In most cases this is fine, but in rare cases it can
    default: prevent things such as shared folders from working properly. If you see
    default: shared folder errors, please make sure the guest additions within the
    default: virtual machine match the version of VirtualBox you have installed on
    default: your host and reload your VM.
    default: 
    default: Guest Additions Version: 6.0.4
    default: VirtualBox Version: 5.2
==> default: Mounting shared folders...
    default: /vagrant => /home/sam/Windows

You should see VirtualBox Guest Additions running from the Windows system tray in the bottom right corner of the OS.

Starting and stopping Windows

Always start Windows using Vagrant and shut down Windows from within the VM. To start Windows run the following command. You must be in the same directory as the Vagrantfile for all vagrant commands to succeed.

cd ~/Windows/
vagrant up

At the Windows login screen, enter the IEUser password: Passw0rd!. This password is also mentioned on the Microsoft Edge Developer Virtual Machines webpage.

To shut down Windows gracefully, run the following command.

vagrant halt

Note: you can also shut Windows down from its start menu.

If you would like to completely delete the Windows virtual machine, then you must destroy it. The following command will permanently delete Windows but it will still be available for provisioning through Vagrant.

vagrant destroy

Adding USB 3 support

Windows 10 was provisioned with USB 3 support enabled. If you’d like to connect your USB hardware to Windows, then you’ll need to attach the device through VirtualBox.

Attaching USB Devices to VirtualBox

List all USB devices.

lsusb

There are two pieces of information to take note when looking at the list of USB devices. The Vendor ID and Product ID. You can use these to automatically directly attach the USB device to Windows.

For example, let’s take a look at my SENA hardware. The following is lsusb output.

Bus 003 Device 005: ID 092b:5530 Sena Technologies, Inc.

You can get additional information about this USB device.

lsusb -d 092b:5530 -v

Go to VirtualBox Manager (while the Windows computer is powered off). View the virtual machine settings.
Click on USB and off to the right there are tiny icons for adding and removing USB filters.
Add a new USB filter. In my case, I filled out the following settings.
- Name: Sena
- Vendor ID: 092b (taken from idVendor in lsusb output)
- Product ID: 5530 (taken from idProduct in lsusb output)

Note: The rest of the USB filter settings I left alone. The fewer details you add to the filter the more broadly devices will match (e.g. you can just specify only the Vendor ID and Manufacturer to match all SENA devices).

Power off and disconnect the USB device. Reconnect and power on the device. Because of the USB filter, the device will automatically connect directly to the virtual machine for Windows to manage next time it is powered on.

Fixing Ubuntu Linux hang on boot after upgrade

Mon, 17 May 2021 00:00:00 -0400

Scenario
Attempted fix
Permanent Fix
Conclusion

Scenario

I have mostly been using my Laptop the past several months.
After a few months I attempted to boot my Desktop which has Ubuntu 18.04.

It would hang on boot with the following message.

[Done] Started hold until boot process finishes up

Attempted fix

I booted into Ubuntu recovery mode successfully via the boot option: advanced options. I enabled networking and then connected to a root terminal.

First, upgraded packages.

apt-get update
apt-get upgrade
apt-get install <held back packages>

Second, I removed some snaps which appeared to be giving me trouble.

snap list
snap remove <package>

I also refreshed the core snaps which basically upgraded them to latest.

snap refresh

None of it seemed to fix the issue.

Permanent Fix

After web searching, I realized Wayland may be loaded by GDM. I found a good stack overflow post which addressed fixing the issue. Edit the file /etc/gdm3/custom.conf and uncomment the following line.

[daemon]
WaylandEnable=false

This forced GNOME Desktop Manager 3 to use Xorg instead of Wayland.

Conclusion

This isn’t the first time Wayland has caused me issues. My opinion is not very high of it. I like to play steam games through Proton and other games through WINE. However, I seem to always have issues with Wayland even when using XWayland.

I’ve been avoiding Ubuntu 20.04 because of Wayland… When Ubuntu 18.04 loses upgrade support may be the day I stop using Ubuntu unless I can verify gaming on Linux is stable when I’m ready for the next edition. I’ll probably hold out for Ubuntu 22.04 and evaluate it for gaming when the time comes.