nicholas/web-log
1
0
Fork 0
Code Pull Requests Actions Packages Activity

Clean history, git LFS

Browse Source
This commit is contained in:
nicholas
2025-07-05 15:36:34 +00:00
commit 70423ce723
141 changed files with 8484 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
container/site/content/posts/wildlife-stream/index.md
+342
View File
@@ -0,0 +1,342 @@
+++
categories = ["software","build"]
tags = ["docker"]
date = 2024-08-21T18:13:24Z
description = ""
draft = false
slug = "wildlife-stream"
title = "📷 Wildlife Cam Livestream with Frigate + Nikon Z50"
author = "nicholas"
+++
I want to capture images and video of animals that visit my deck (mostly squirrels, woodpeckers, and finches). Previously, I have used an IP camera to accomplish this, and while satisfying in a technical sense, the images were of too low quality to be of much artistic interest to me. The image that IP cameras produce are typically of very large field of view, which makes the subjects I am interested in appear very small. It has the distinct look of a security camera which is not associated with beauty or visual interest. I will need to use a better camera.
{{< image
src="images/apartment-inside.jpg"
caption="Apartment view" >}}
## Equipment
* **Camera**
* I replaced my IP camera with an interchangeable-lens type camera – the Nikon z50, with a Nikkor Z DX 50-250mm lens. The longer focal length range 50-250mm will allow me to place the camera far away from the subject and nearer to the computer that will capture the video and process it. This setup gives me plenty of flexibility, which is useful because the project is more artistic than utilitarian / well-defined – and subject to the whims of the aesthetic preferences of the day.
* **Capture Device**
* I need a way to get the video feed to the server so I can process it, so I bought a USB HDMI capture device – Elgato Camlink 4k.
* **Dummy Battery**
* I am using a dummy battery that supplies the camera with the same voltage and battery identifier signals as its proprietary battery, basically a 120v power adapter. This allows me to power the camera indefinitely.
## The Scene
Here is the scene I am attempting to capture. I have placed a small container on the ledge of the deck railing that I fill each day with a handful of peanuts. This is where the animals come to eat. I have a whole cart of houseplants to contend with that may occlude the view, but I can make it work.
{{< image
src="images/hinoki-bonsai.jpg"
caption="Squirrel scene" >}}
I have placed the camera something like 15 ft. away from the subject – right next to my server. Here is what this looks like:
{{< image
src="images/camera-pov-1.jpg"
caption="Camera POV" >}}
{{< image
src="images/camera-pov-2.jpg"
caption="Camera POV 2" >}}
## Software
Here I run `lsusb` to be sure the system recognizes the HDMI capture card. This is successful
```shell
lsusb
Bus 002 Device 108: ID 0fd9:007b Elgato Systems GmbH Cam Link 4K
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 002: ID 8087:0aa7 Intel Corp. Wireless-AC 3168 Bluetooth
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
```
Now I see where the video is mounted.
```shell
ls /dev/video*
/dev/video0
/dev/video1
```
There are two video devices – `video0` and `video1`. I will need to determine which is the video feed I am interested in. I will run a Video4Linux `v4l2-ctlcommand` to discover the capabilities of each device.
### device video0
```shell
/dev/video0
v4l2-ctl --device=/dev/video0 --all
Driver Info:
Driver name : uvcvideo
Card type : Cam Link 4K: Cam Link 4K
Bus info : usb-0000:00:14.0-2
Driver version : 6.1.55
Capabilities : 0x84a00001
Video Capture
Metadata Capture
Streaming
Extended Pix Format
Device Capabilities
Device Caps : 0x04200001
Video Capture
Streaming
Extended Pix Format
Media Driver Info:
Driver name : uvcvideo
Model : Cam Link 4K: Cam Link 4K
Serial : FX02M2A10475
Bus info : usb-0000:00:14.0-2
Media version : 6.1.55
Hardware revision: 0x00000100 (256)
Driver version : 6.1.55
Interface Info:
ID : 0x03000002
Type : V4L Video
Entity Info:
ID : 0x00000001 (1)
Name : Cam Link 4K: Cam Link 4K
Function : V4L2 I/O
Flags : default
Pad 0x01000007 : 0: Sink
Link 0x02000010: from remote pad 0x100000a of entity 'Extension 3' (Video Pixel Formatter): Data, Enabled, Immutable
Priority: 2
Video input : 0 (Input 1: ok)
Format Video Capture:
Width/Height : 1920/1080
Pixel Format : 'NV12' (Y/UV 4:2:0)
Field : None
Bytes per Line : 1920
Size Image : 3110400
Colorspace : sRGB
Transfer Function : Rec. 709
YCbCr/HSV Encoding: Rec. 709
Quantization : Default (maps to Limited Range)
Flags :
Crop Capability Video Capture:
Bounds : Left 0, Top 0, Width 1920, Height 1080
Default : Left 0, Top 0, Width 1920, Height 1080
Pixel Aspect: 1/1
Selection Video Capture: crop_default, Left 0, Top 0, Width 1920, Height 1080, Flags:
Selection Video Capture: crop_bounds, Left 0, Top 0, Width 1920, Height 1080, Flags:
Streaming Parameters Video Capture:
Capabilities : timeperframe
Frames per second: 23.976 (27021/1127)
Read buffers : 0
User Controls
brightness 0x00980900 (int) : min=0 max=255 step=1 default=128 value=128
contrast 0x00980901 (int) : min=0 max=255 step=1 default=128 value=128
saturation 0x00980902 (int) : min=0 max=255 step=1 default=128 value=128
hue 0x00980903 (int) : min=0 max=255 step=1 default=128 value=128
```
### device video1
```shell
/dev/video1
v4l2-ctl --device=/dev/video1 --all
Driver Info:
Driver name : uvcvideo
Card type : Cam Link 4K: Cam Link 4K
Bus info : usb-0000:00:14.0-2
Driver version : 6.1.55
Capabilities : 0x84a00001
Video Capture
Metadata Capture
Streaming
Extended Pix Format
Device Capabilities
Device Caps : 0x04a00000
Metadata Capture
Streaming
Extended Pix Format
Media Driver Info:
Driver name : uvcvideo
Model : Cam Link 4K: Cam Link 4K
Serial : FX02M2A10475
Bus info : usb-0000:00:14.0-2
Media version : 6.1.55
Hardware revision: 0x00000100 (256)
Driver version : 6.1.55
Interface Info:
ID : 0x03000005
Type : V4L Video
Entity Info:
ID : 0x00000004 (4)
Name : Cam Link 4K: Cam Link 4K
Function : V4L2 I/O
Priority: 2
Format Metadata Capture:
Sample Format : 'UVCH' (UVC Payload Header Metadata)
Buffer Size : 10240
```
Both of these outputs indicate they refer to the same device – Cam Link 4k, but `/dev/video0` looks like the video device node that I can use to access the device's video stream.
`/dev/video0` lists "Video Capture" under both `Capabilities` and `Device Caps`, confirming it can capture video.
`/dev/video1` lists only "Metadata Capture", which means it is only capturing metadata and not the video itself.
## Docker
I will be using docker to run the NVR software (frigate). Most of the configuration file is defaults, but one thing I will need to do is pass the video capture device to the docker container.
```yaml
services:
frigate:
container_name: frigate
privileged: true
restart: unless-stopped
image: ghcr.io/blakeblackshear/frigate:stable
shm_size: "512mb"
devices:
- /dev/video0:/dev/video0
- /dev/dri/renderD128 # for intel hwaccel
volumes:
- /etc/localtime:/etc/localtime:ro
- ./config:/config
- type: tmpfs # Optional: 1GB of memory, reduces SSD/SD Card wear
target: /tmp/cache
tmpfs:
size: 1000000000
ports:
- "8971:8971" # web interface authenticated
- "5000:5000" # web interface
- "8554:8554" # RTSP feeds
- "8555:8555/tcp" # WebRTC over tcp
- "8555:8555/udp" # WebRTC over udp
- "1984:1984" # go2rtc
```
## Frigate Configuration
Frigate requires its own configuration file: /config/config.yml. Using the built-in go2rtc tool I will publish a new video stream of the USB capture device /dev/video0 that I will ingest as a camera in frigate. First I will need some information about the video stream offered by the capture device.
I will use a v4l2-ctl command with the long option --list-formats-ext to display information about the video formats supported by my capture device. The output shows supported resolutions, pixel formats, and framerate. I will need this information to define my camera in frigate configuration.
```shell
v4l2-ctl --device=/dev/video1 --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
Type: Video Capture
[0]: 'YUYV' (YUYV 4:2:2)
Size: Discrete 1920x1080
Interval: Discrete 0.042s (23.976 fps)
[1]: 'NV12' (Y/UV 4:2:0)
Size: Discrete 1920x1080
Interval: Discrete 0.042s (23.976 fps)
```
This output pertains to a line I must add to the configuration file
`ffmpeg:device?...`
| Variable | Value | Description |
| -------------- | ------------ | ----------------------- |
| `video` | `dev/video0` | USB capture device path |
| `input_format` | `nv12` | Color format |
| `video_size` | `1920x1080` | Input resolution |
| `video` | `h264` | Video codec |
| `hardware` | `vaapi` | Hardware acceleration |
I have named my stream *nikonz50* and defined it as a ffmpeg device using the table of variables above. I configured frigate to detect and record only animals.
```yaml
go2rtc:
streams:
nikonz50:
ffmpeg:device?video=/dev/video0&input_format=nv12&video_size=1920x1080#video=h264#hardware=vaapi
cameras:
nikonz50:
objects:
track:
- animal
ffmpeg:
inputs:
- path: rtsp://127.0.0.1:8554/nikonz50
roles:
- record
- detect
hwaccel_args: preset-vaapi
input_args: -tag:v hvc1
record:
enabled: true
events:
pre_capture: 2
post_capture: 2
objects:
- animal
snapshots:
enabled: true
detect:
enabled: true
width: 1920
height: 1080
detectors:
ov:
type: openvino
device: AUTO
model:
path: /openvino-model/ssdlite_mobilenet_v2.xml
model:
width: 300
height: 300
input_tensor: nhwc
input_pixel_format: bgr
labelmap_path: /openvino-model/coco_91cl_bkgr.txt
labelmap:
15: animal
16: animal
17: animal
version: 0.14
```
Now that I have a `docker-compose.yml` file and a frigate `config.yml` file I can start up Frigate and see how it looks. I run the compose command with short option `-d` (detached mode) to run the container in the background.
```shell
sudo docker compose up -d
```
**It works!**
{{< image
src="images/frigate-ui.PNG"
caption="Frigate UI" >}}
## Object Detection
Frigate is capable of running object detection on the camera streams. Since my machine is running on an intel CPU, I can configure its object detection to use OpenVino, (Open Visual Inference and Neural Network Optimization). This uses Intel's integrated GPU to offload the work of inference from the CPU to the GPU. This improves performance on my machine a great deal. If I used CPU detector my CPU Intel Pentium Gold G5400 would be pegged at 100% utilization and the detection latency would be abysmal.
{{< image
src="images/detect-squirrel.jpg"
caption="Detect squirrel" >}}
## Recording
I have configured frigate to capture a video each time an animal is detected. Here is one such video.
{{< video
src="videos/squirrel.mp4"
width="100%"
>}}
## Improvements
My CPU is my biggest bottleneck. It is possible to increase stream resolution to `2160p` but this results in unacceptable CPU utilization. One potential solution is acquiring a secondhand Optiplex Micro or some other small computer dedicated to video streaming. I would not mind if the CPU usage was excessive. Facebook Marketplace has many that contain CPUs that exceed the specs of my current server CPU, all for ~$65.
Another option is to upgrade my current CPU, but I do not want it to get any hotter as it hosts all my HDDs. I do not know. I may make an edit to this if I upgrade. The project is just for fun and not serious. 4k stream can wait.
{{< image
src="images/usage.PNG"
caption="Performance" >}}
Done.
**UPDATE 2025-05-01: I have upgraded the streaming computer to a miniature computer**