Install docker & plex on NAS4Free via bhyve

Intro

In order to get the premium music feature (and to escape from the « signal 11 » issue) , and because docker seems to be fun, I did try to install the plex docker container on my NAS (NAS4Free) system.
To achieve this, I installed a Virtual Machine through bhyve, and installed coreOS and the plex docker container in the VM.

Pre-requisites :

  • System running NAS4Free or freebsd11
  • 1 GB RAM & 8 GB Hard disk (file or zvol) for the Virtual Machine
  • CPU capable to run bhyve see §21.7
  • Virtualbox service must be disabled

NAS4Free setup

Datasets preparation (through NAS4Free GUI) :

Create three datasets (and configure NFS exports for each of them).
On my setup I have :

  • /mnt/Pool1/plexconfig : space to host DB and config file
  • /mnt/Pool1/plextranscode : space for transcoding the media
  • /mnt/Pool1data : for the media

Create space to host coreOS :
I choose to create a ZVOL (select a « dev » type ZVOL). 8GB should be enough
My ZVOL is located under /dev/zvol/Pool2/vol2

Load kernel modules

As explained here
$ sudo kldload if_tap
$ sudo kldload if_bridge
$ sudo kldload vmm
$ sudo kldload nmdm
Configure network interfaces :
$ sudo sysctl net.link.tap.up_on_open=1
$ sudo sysctl net.inet.ip.forwarding=1
$ sudo ifconfig bridge0 create
$ sudo ifconfig tap0 create
$ sudo ifconfig bridge0 addm bge0 addm tap0 (bge0 is my NAS interface)
$ sudo ifconfig bridge0 up

coreOS Installation (through NAS4Free CLI)

coreOS installation reference here
Choose a folder on your NAS and cd into it (ou can create a dataset for it)
$ cd /mnt/Pool1/bhyve
Download coreOS iso from the official website and put it in your working bhyve folder (/mnt/Pool1/bhyve)

create a coreos.map file with the content as below :
$ cat coreos.map

(hd0) /dev/zvol/Pool2/vol2
(cd0) coreos_production_iso_image.iso

create a coreos.grub_cd0.cfg with content as below :
$ cat coreos.grub_cd0.cfg

linux (cd0)/coreos/vmlinuz coreos.autologin
initrd (cd0)/coreos/cpio.gz
boot

Launch grub-bhyve to start from the iso file :
$ sudo grub-bhyve -m coreos.map -r cd0 -M 1024M coreos < coreos.grub_cd0.cfg
Now, start the VM with the following command :
$ sudo bhyve -A -H -P -s 0:0,hostbridge -s 1:0,lpc -s 2:0,virtio-net,tap0 -s3:0,virtio-blk,/dev/zvol/Pool2/vol2 -l com1,stdio -c 2 -m 1024M coreos
The machine should boot up, pompt will appear, it is now time to install coreOS from the iso to the ZVOL (the ZVOL should appear in /dev/disk/by-id/virtio-BHYVE-xxxxxxxx)
Lauch the installer :
$ sudo coreos-install -d /dev/disk/by-id/virtio-BHYVE-xxxxxxxx -C stable
When it is finished, stop coreOS, and kill the VM :
$ sudo haltFrom NAS4Free shell :
$ sudo bhyvectl --destroy --vm=coreos

Launch coreOS from the ZVOL

create a coreos.grub_hd0.cfg with content as below :
$ cat coreos.grub_hd0.cfg

linux (hd0,gpt1)/coreos/vmlinuz-a console=ttyS0 ro root=LABEL=ROOT usr=LABEL=USR-A coreos.autologin
boot

launch grub-bhyve and bhyve:
$ sudo grub-bhyve -m device.map -r hd0 -M 1024M coreos < coreos.grub_hd0.cfg
$ sudo bhyve -A -H -P -s 0:0,hostbridge -s 1:0,lpc -s 2:0,virtio-net,tap0 -s3:0,virtio-blk,/dev/zvol/Pool2/vol2 -l com1,stdio -c 2 -m 1024M coreos

Next steps in coreOS

coreOS will boot and you should see a prompt :
core@localhost / $

As my NAS4Free setup has the IP : 192.168.0.30, let’s try to contact the nfs server :
$ showmount -e 192.168.0.30

Export list for 192.168.0.30:
/mnt/Pool1/data          192.168.0.0
/mnt/Pool1/plexconfig    192.168.0.0
/mnt/Pool1/plextranscode 192.168.0.0

so far, so good ! Let »s configure systemd to autmount the shares to be used with plex :
I have decided to mount the shares in /mnt/. The names of the configuration files must respect a naming convention : to mount on /mnt/data, the configuration files must be named mnt-data.mount & mnt-data.automount, and must be stored in /etc/systemd/system
$ cat mnt-data.mount

[Unit]
Description = nfs mount for data
[Mount]
What = 192.168.0.30:/mnt/Pool1/data
Where = /mnt/data
Type = nfs
Options = vers=3,udp
[Install]
WantedBy=default.target

$ cat mnt-data.automount

[Unit]
Description = nfs automount for data
[Automount]
Where = /mnt/data
[Install]
WantedBy=default.target

Create also mnt-plextranscode.mount & mnt-plextranscode.automount, mnt-plexconfig.mount & mnt-plexconfig.automount

Let’s mount these shares :
$ sudo systemctl start mnt-data.mount
$ sudo systemctl start mnt-plexconfig.mount
$ sudo systemctl start mnt-plextranscode.mount

It’s time to create the plex docker container (ref)
$ docker run -d --name plex --network=host -e TZ="Europe/nameofcity" -v /mnt/plextranscode:/transcode -v /mnt/plexconfig:/config -v /mnt/data:/data -e PLEX_UID=1000 -e PLEX_GID=2000 plexinc/pms-docker
PLEX_UID & PLEX_GID are matching the UID & GID of the user owning the files in the « data » shares.

docker will download plex, and run it.

In order to start automatically plex when coreOS is launched, let’s create a systemd config file in /etc/systemd/system
$ cat docker-plex.service

[Unit]
Description=plex container
Requires=docker.service
Requires=mnt-data.automount
Requires=mnt-plexconfig.automount
Requires=mnt-plextranscode.automount
After=docker.service
[Service]
Restart=always
ExecStart=/usr/bin/docker start -a plex
ExecStop=/usr/bin/docker stop -t 2 plex
[Install]
WantedBy=default.target

stop plex and lauch it through the systemd config file :
$ docker stop plex
$ sudo systemctl start docker-plex
Enable the service so that plex will autostart at next boot :
$ sudo systemctl enable docker-plex

Created symlink /etc/systemd/system/default.target.wants/docker-plex.service  /etc/systemd/system/docker-plex.service.
$

When the NAS will reboot….

… you will need to start the VM
I am working on a script, but for the moment, I need to issue the commands :
To load the kernel modules :
$ sudo kldload if_tap
$ sudo kldload if_bridge
$ sudo kldload vmm
$ sudo kldload nmdm
Configure network interfaces :
$ sudo sysctl net.link.tap.up_on_open=1
$ sudo sysctl net.inet.ip.forwarding=1
$ sudo ifconfig bridge0 create
$ sudo ifconfig tap0 create
$ sudo ifconfig bridge0 addm bge0 addm tap0
$ sudo ifconfig bridge0 up
To launch grub-bhyve :
$ cd /mnt/Pool1/bhyve
$ sudo grub-bhyve -m coreos.map -r hd0 -M 1024M coreos < coreos.grub_hd0.cfg
To launch bhyve, but with a slight modification to run it in the background, and to use a nullmodem device to access the console
$ sudo bhyve -A -H -P -s 0:0,hostbridge -s 1:0,lpc -s 2:0,virtio-net,tap0 -s3:0,virtio-blk,/dev/zvol/Pool2/vol2 -l com1,/dev/nmdm0A -c 2 -m 1024M coreos &
to open the console :
$ sudo cu -l /dev/nmdm0B
Nota : to exit the console, type : ~.

Next steps :

  • create a script for NAS4Free to autostart the coreOS VM
  • modify the bhyve command to use fbuf (VNC) to access the console. Something like :
    $ sudo bhyve -A -H -P -s 0:0,hostbridge -s 1:0,lpc -s 2:0,virtio-net,tap0 -s3:0,virtio-blk,/dev/zvol/Pool2/vol2 -s 11,fbuf,tcp=0.0.0.0:5900 -c 2 -m 1024M coreos is only giving a dark screen (but respond to CTRL-ALT-DEL !)
  • correct the wording (you already guessed that I am not a native english speaker :-)) )

References :

https://users.monkeybrains.net/support/bhyve
https://imil.net/blog/2016/06/21/CoreOS_on_bhyve/
https://docs.docker.com/engine/admin/host_integration/
https://freebsd.org/doc/handbook/virtualization-host-bhyve.html