How to set up a Red Hen capture station

Configure the capture device

• See Capture Station Manual ; if you use a satellite signal
• Configure the HDHR4 by clicking on the HDHomeRun icon on the desktop and find your channels:

• • If you want automated scheduling, open FreeGuide and configure it for your location (optional; we now automate without Freeguide).
• Identify the news programs to record and determine the teletext pages for subtitle captions.
• Click on the globe icon next to the menu (a browser) and type in “what is my ip?”

Once you send us your IP address, we can access the system; if you’re behind a firewall, we’ll let you know how to set up a tunnel. We will complete the configuration with your help, ensuring that the Raspberry Pi can talk to the HDHR3 and start recording. Here are some of the steps:

• • Run find-tuners.sh and edit /tvspare/tuners/signal
  • Get the PID on the interesting channels, as described here, and add them to /tvspare/tuners/lineup
  • Add the local tv program grabber definition to the script /usr/local/bin/xmltv-download
  • Run the scheduler script /usr/local/bin/schedule on the chosen programs and update crontab
  • Add the new machine to the list of Red Hen capture stations at our HPC processing pipelines at UCLA and CWRU

Transferring data

Where the internet connection is fast enough, Red Hen transfers data directly and no further action is needed. If the connection is too slow to handle the transfer of uncompressed files, the best solution is to add a local server to handle the compression task; compressed files are typically around 5% of the size of uncompressed files, and therefore transfer much more easily.

If neither a fast internet connection nor a server to handle local compression is available, the fallback is to fill up a drive and swap it out for manual transport.

Add the new drive by plugging it into power and into a spare USB port on the RPi. Check dmesg for the logical device it attaches to — typically /dev/sdb, but it could switch places with the existing drive. Use gdisk to partition it into one linux partition. First press x and then z to remove previous partitions, then start gdisk again and add a new partition by pressing n and then w to write. Next, add the file system and label it:

• • mkfs.xfs /dev/sdb1
  • xfs_admin -L HD2 /dev/sdb1
  • add it to /etc/fstab, copying the HD1 line and changing to HD2
  • mount /mnt/HD2
  • chown csa:csa /mnt/HD2 (as root)

We can either copy the recorded files to the new drive or switch recording to the new drive. It’s generally a bad idea to move video files from one drive to another unless you really have to; the files are so large that they place extra strain on the drives that can break them. However, sometimes this is still the best option, and we can use a command like this (can be given as a one-liner):

day 2017-02-02

for d in {478..0} ; do D=/mnt/HD2`pwd` ; mkdir -p $D

 if [ ! -d $D ] ; then break ; fi

 for f in *mpg ; do rsync ${f%.*}.{mpg,txt,len} $D -av ; sleep 10 ; echo . ; done

 sleep 60 ; echo .. ; day + 1

done

— where 2017-02-02 is the first day to copy, moving forward a day at a time to the present, 478 is the number of consecutive days to copy, and HD2 is the name of the drive we’re transferring the data to. The job pauses ten seconds between video files and a minute between days to lower the strain on the drive. When the copy job is done, unmount and unplug the transfer drive.

If instead we replace the old drive with the new, we first need to create the directories we need, as user csa:

• • mkdir -p /mnt/HD2/tv /mnt/HD2/ts /mnt/HD2/xmltv /mnt/HD2/software /mnt/HD2/system/log /mnt/HD2/system/logs

We then need to change these symlinks — first done by user root, but then chown to csa:csa:

• • ln -sf /mnt/HD2/tv /tv (root directory)
  • ln -sf /mnt/HD2/spool /mnt/spool
  • ln -sf /mnt/HD2/software /root/software

These links should be created by user csa:

• • ln -sf /mnt/HD2/tv /home/csa/tv
  • ln -sf /mnt/HD2/ts /home/csa/ts
  • ln -sf /mnt/HD2/system /home/csa/system
  • ln -sf /mnt/HD2/xmltv /home/csa/xmltv
  • ln -sf /mnt/HD2/xmltv /nest/xmltv
  • ln -sf /mnt/HD2/system/log /nest/log
  • ln -sf /mnt/HD2/system/logs /nest/tuners/logs

Once everything is switched over, the Red Hen software will record to the new drive. Unmount /mnt/HD1 and unplug it, and transport it to a location with a faster internet connection so that the data can be transferred.

If we need to read the XFS-formatted transfer drive on OS X, use fuse-xfs — it’s read-only and slow, but it works.

To check that the capture drive remains available, we use this spell on root’s crontab:

# Attempt to remount the hard drive if it’s failed

59 * * * * if [ ! -d /mnt/HD1/spool ] ; then shutdown -r now ‘Hard drive detached’ ; fi

If the capture drive has been moved to /mnt/HD2, make sure to change this cronjob too.

If the drive fails to mount, here are some tools:

• • smartctl -d usbsunplus -t short /dev/sda
  • smartctl -d usbsunplus -a /dev/sda
  • xfs_repair /dev/sda1
  • xfs_repair -L /dev/sda1

Setting up an Intel NUC

• • Intel NUC6CAYB and Intel NUC7CJYH specs — Celeron amd64
  • Installing Debian on an Intel NUC7PJYH (the only difference is the CPU)
  • In tasksel, select SSH server; do not install a graphical user interface
  • To activate the NIC, install openresolv firmware-realtek
    • Create the file /etc/systemd/network/dhcp.network with this content:
      • [Match] Name=en* [Network] DHCP=yes
    • As user root, issue “systemctl restart systemd-networkd” — the NIC is called enp2s0
  • To activate the wifi (Intel 8086:31dc), install wireless-tools firmware-iwlwifi
    • modprobe -r iwlwifi ; modprobe iwlmvm
    • dmesg should have something like this:
    • iwlwifi 0000:00:0c.0: Detected Intel(R) Dual Band Wireless AC 9462, REV=0x318
    • iwlwifi 0000:00:0c.0: enabling device (0000 -> 0002)
    • iwlwifi 0000:00:0c.0: loaded firmware version 34.0.0 op_mode iwlmvm
    • However, the firmware is not being loaded; we need to figure out why

Installing Red Hen

• • Copy /nest from ca:/home/svarog/system/nest/svarog/nest/ rather than from dola
  • Edit /nest/cfg/redhen.config — include wifi credentials if available
  • Modify /nest/cfg/sources.list to use local Debian repositories
  • Use /nest/cfg/install-NUC.sh

The NUC can function as a capture machine (as in Croatia and Forschheim) or as a compression machine (as in France and India).

• If it is used as a capture machine, see the section “Configure the capture device” above
• if it is used as a text extraction and compression device
  • • on the paired RPi capture machine, install NFS and export the HD1 drive to the NUC in /etc/exports
    • on the NUC, install NFS and mount the RPi capture drive
    • install a recent version of CCExtractor that handles the text extraction needed
    • install recent versions of ffmpeg and HandBrake-cli for file compression
    • script the transfer of compressed files to cartago
    • alert hoffman2 to the location of the files for on-screen text OCR

For compression NUCs, see kalpana’s vishnu and corbac’s corbeau (DVB text extraction). For capture NUCs, see sif and svarog.