CUDA Toolkit / cuDNN Installation Guide

Overview

CUDA Toolkit is NVIDIA's platform for general-purpose GPU computing (GPGPU). cuDNN is NVIDIA's library for deep neural networks. By installing these, ailia SDK can perform faster inference using NVIDIA GPUs.

Version Compatibility

The cuDNN version requires a specific minimum version of CUDA Toolkit and ailia SDK.

cuDNN Version	Required CUDA Toolkit	Required ailia SDK
cuDNN 8.0.2 - 8.2.4	CUDA Toolkit 11.0 or later	ailia SDK 1.2.4 and later
cuDNN 8.3.0 - 8.7.0	CUDA Toolkit 11.5 or later	ailia SDK 1.2.11 and later
cuDNN 8.8.0 - 8.9.7	CUDA Toolkit 12.0 or later	ailia SDK 1.2.15 and later
cuDNN 9.0.0 - 9.x	CUDA Toolkit 12.0 or later	ailia SDK 1.4.0 and later
cuDNN 10.0.0 -	CUDA Toolkit 12.0 or later	ailia SDK 1.7.0 and later (planned)

Windows

1. Install CUDA Toolkit

Go to the CUDA Toolkit download page and click "Download Now".

Select your operating system, architecture, version, and installer type:

Operating System: Windows
Architecture: x86_64
Version: Your Windows version (e.g., 10, 11)
Installer Type: exe (local)

Click the "Download" button under "Base Installer" to download.

CUDA Toolkit download page - selecting OS, architecture, version, and installer type

Run the downloaded installer. The default extraction directory is fine.

CUDA setup package - extraction path dialog

Review the NVIDIA Software License Agreement and click "Agree and Continue".

NVIDIA installer - license agreement screen

Select "Custom (Advanced)".

Note: Some versions of CUDA Toolkit may fail to install with the "Express" option. In that case, use the Custom install and select only the components listed below. See the NVIDIA forum post for details.

In the custom installation options, enable the following components:

CUDA > Runtime
CUDA > Documentation
CUDA > Development
Driver components

Custom install options - selecting CUDA components

Custom install options - selecting Driver components

When installation finishes, you will see a list of installed components.

NVIDIA installer - installation complete screen

2. Install cuDNN

Go to the cuDNN page and click "Download cuDNN Library".

NVIDIA cuDNN page - click Download cuDNN Library

An NVIDIA Developer account is required to download cuDNN. Sign up if you don't have one.

After logging in, select "Local Installer for Windows (Zip)" on the download page.

cuDNN download page - select Local Installer for Windows (Zip)

Extract the downloaded zip file. Move the extracted folder (e.g., cudnn-windows-x86_64-8.9.6.50_cuda12-archive) to a location of your choice (e.g., C:\nvidia\).

Extracted cuDNN folder contents - bin, include, lib, LICENSE

3. Set Environment Variables

Add the cuDNN bin folder to your system PATH so that the DLLs can be found at runtime.

Open Windows "Settings" → "System" → "About", then click "Advanced system settings" to open "System Properties". Click the "Environment Variables" button.

Windows System Properties - Environment Variables button

Select the "Path" variable under "User variables" and click "Edit". Click "New" and add the path to the cuDNN bin folder.

C:\nvidia\cudnn-windows-x86_64-8.9.6.50_cuda12-archive\bin

Edit environment variable - adding cuDNN bin folder to PATH

4. Install zlib

cuDNN 8.3 and later on Windows requires zlibwapi.dll.

Download zlib123dllx64.zip from the ZLIB DLL home page and extract it.

Copy zlibwapi.dll from the extracted files to a directory on your PATH. The simplest option is to copy it into the cuDNN bin folder.

copy dll_x64\zlibwapi.dll C:\nvidia\cudnn-windows-x86_64-8.9.6.50_cuda12-archive\bin\

Linux (Ubuntu)

1. Add the NVIDIA CUDA Repository

First, add the NVIDIA CUDA apt repository. Go to the CUDA Toolkit download page, select Linux / x86_64 / Ubuntu / your version / deb (network), and follow the displayed commands. The typical steps are:

wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-keyring_1.1-1_all.deb
sudo dpkg -i cuda-keyring_1.1-1_all.deb
sudo apt update

Note: The URL for the keyring package changes depending on your Ubuntu version. Always use the exact commands shown on the NVIDIA download page for your environment.

2. Install CUDA Toolkit and cuDNN

Once the repository is added, install both CUDA Toolkit and cuDNN with apt:

sudo apt install cuda-toolkit libcudnn9-cuda-12

Note: The cuDNN package name depends on the version. For cuDNN v8, use libcudnn8. For cuDNN v9 with CUDA 12.x, use libcudnn9-cuda-12. Check available packages with apt search libcudnn.

3. Set Environment Variables

Add the CUDA paths to your shell profile (~/.bashrc or ~/.zshrc):

export PATH=/usr/local/cuda/bin:$PATH
export LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH

Reload your shell and verify the installation:

source ~/.bashrc
nvcc --version

NVIDIA Jetson

On NVIDIA Jetson devices (Jetson Orin Nano, Jetson AGX Orin, etc.), CUDA Toolkit and cuDNN are pre-installed as part of NVIDIA JetPack SDK. No additional installation is required.

You can verify the pre-installed versions with:

nvcc --version
dpkg -l | grep cudnn

Note: Make sure to use a version of ailia SDK that is compatible with the cuDNN version included in your JetPack release. See the version compatibility table above.

Verify the Installation

Python API

Use an ailia-models sample to verify that CUDA is detected.

cd ailia-models/image_classification/resnet50
python3 resnet50.py --env_list

If CUDA appears in the environment list, the setup is successful.

env[2]=Environment(id=2, type='GPU',
name='cuDNN-NVIDIA GeForce RTX 3080 (8.6, FP32)',
backend='CUDA', props=[])

Unity Plugin

If "CUDA" appears in the GPU name, the setup is successful.

Tip: By default, ailia SDK automatically selects the fastest available environment on your platform. To use a specific environment, pass the env_id explicitly.