NVIDIA GPU monitoring on Linux and Docker

SkyView reads NVIDIA GPU utilization through nvidia-smi. When SkyView runs in Docker, the NVIDIA driver, GPU devices, matching driver libraries, and nvidia-smi must be exposed by the Docker host. They cannot be provided reliably by installing packages only inside the SkyView container.

If the System Monitor Server tab shows SkyView GPU – N/A, complete the host and Compose setup below.

Requirements

  • A supported NVIDIA GPU and NVIDIA driver on the Linux host
  • Docker Engine and Docker Compose
  • NVIDIA Container Toolkit installed and configured on the host
  • docker-compose.yml and docker-compose.nvidia.yml from the same SkyView release

The normal SkyView image already requests the NVIDIA utility capability used by nvidia-smi. The NVIDIA overlay requests the GPU from Docker. It does not install the host driver or NVIDIA Container Toolkit.

1. Verify the host driver

Run this on the Docker host:

nvidia-smi

Do not continue until this command displays the GPU and driver information. If it fails, install or repair the NVIDIA driver for the host operating system first.

2. Install NVIDIA Container Toolkit on the host

Follow NVIDIA's Container Toolkit installation guide to add the repository and install the package for your Linux distribution.

On Debian or Ubuntu, after adding NVIDIA's package repository, install and configure the toolkit with:

sudo apt-get update
sudo apt-get install -y nvidia-container-toolkit
sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker

The nvidia-ctk command updates Docker's host configuration. Restarting Docker activates the NVIDIA runtime.

3. Verify GPU access from Docker

Run NVIDIA's sample container test:

docker run --rm --runtime=nvidia --gpus all ubuntu nvidia-smi

This command must display the GPU. If Docker reports that it cannot select the NVIDIA device driver, verify the toolkit installation, rerun nvidia-ctk runtime configure, and restart Docker.

4. Start SkyView with the NVIDIA overlay

Keep both Compose files in the installation directory, then run:

docker compose -f docker-compose.yml -f docker-compose.nvidia.yml up -d

The second file is a Compose overlay. It adds the NVIDIA GPU device request and driver capabilities to the SkyView service. Do not replace docker-compose.yml with it; both files are required.

The current overlay selects Flywheel's headless CUDA compositor with STVIEW_EXPERIMENTAL_CUDA=1; the SDK retains that variable name for compatibility. NVIDIA EGL is also selected for CUDA/GL graphics interoperability. The image includes the EGL vendor registration and CUDA runtime dependencies; the host toolkit supplies the matching NVIDIA driver libraries. At container startup, SkyView rebuilds its GStreamer registry and verifies the selected CUDA runtime and hardware Flywheel worker probe. See Flywheel's Linux container runtime for the CUDA, EGL/GL, and software paths, and desktop versus shared VM GPUs for deployment considerations.

To make the overlay selection persistent, add this line to the installation .env file:

COMPOSE_FILE=docker-compose.yml:docker-compose.nvidia.yml

After that, the normal command uses both files:

docker compose up -d

For a SkyView .deb installation, edit /etc/sky-view/sky-view.env instead:

COMPOSE_FILE=docker-compose.yml:docker-compose.nvidia.yml

Then restart the service:

sudo systemctl restart sky-view

5. Verify SkyView

Confirm that nvidia-smi is available inside the running container:

docker exec skyview nvidia-smi

Then open System Monitor → Server in SkyView. Total GPU utilization and SkyView GPU availability are tracked separately. Docker or a virtual GPU driver may expose total counters but omit per-process readings; in that case SkyView GPU shows N/A even when hardware acceleration is working. A numeric 0% indicates a measured idle value. Missing process counters are not treated as zero or replaced with total GPU usage.

Troubleshooting

Codecs are listed but Flywheel is unavailable

Flywheel checks the complete compositor-to-encoder path separately from codec detection. Use the NVIDIA overlay from the same release as the image. If the capabilities dialog reports an incompatible worker ABI, rebuild or replace the image with a matching server and native worker; installing additional GPU drivers will not fix a protocol mismatch. The image build checks the worker's advertised probe ABI.

The hardware probe processes frames and requires encoded transport output before reporting success. Linux GL texture input uses a GStreamer-owned context sharing the compositor's textures; a wrapped application context alone cannot service GStreamer streaming-thread allocations. For GPU release verification, run scripts/test-flywheel-worker-linux.sh with SKYVIEW_FLYWHEEL_REQUIRE_HARDWARE=1 in the GPU-enabled runtime environment (including its DISPLAY). This makes a failed hardware frame/transport probe fail the test suite instead of accepting software-only availability.

For NVIDIA source and orientation regression testing, run python3 scripts/test-flywheel-gpu-frames.py --output-dir /tmp/flywheel-frame-test in the same runtime environment. It requires FFmpeg with libx264, GStreamer tools, and the native worker. The test compares software and hardware output using asymmetric source images, checks source replacement and recovery after timeout, and compares the offline card's orientation. Repeat with --codec h265 for H.265 output. Logs and captures remain in the output directory. The test uses its own loopback ports and does not change configured streams.

nvidia-smi fails on the host

The host driver is missing, incompatible, or not loaded. Fix the host driver before troubleshooting Docker.

Docker cannot select the NVIDIA driver

NVIDIA Container Toolkit is missing or is not configured for Docker. Run:

sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker

Then repeat the sample container test.

nvidia-smi works on the host but not in SkyView

Recreate SkyView with the overlay so Docker adds the GPU request:

docker compose -f docker-compose.yml -f docker-compose.nvidia.yml up -d --force-recreate

Also confirm that both Compose files came from the same SkyView release.

Installing NVIDIA packages inside SkyView does not help

The NVIDIA kernel driver and GPU device access belong to the host. NVIDIA Container Toolkit performs the required device and driver-library injection when Docker creates the container. Installing the toolkit or a standalone nvidia-smi package inside the application container does not grant GPU access.

Intel or AMD host

Do not enable docker-compose.nvidia.yml. Intel and AMD GPU monitoring uses /dev/dri, the monitoring utilities in the SkyView image, and the permissions in the base docker-compose.yml.