# cuda-repo-management **Repository Path**: mirrors_NVIDIA/cuda-repo-management ## Basic Information - **Project Name**: cuda-repo-management - **Description**: Scripts for managing Debian and RPM package repositories - **Primary Language**: Unknown - **License**: MIT - **Default Branch**: main - **Homepage**: None - **GVP Project**: No ## Statistics - **Stars**: 0 - **Forks**: 0 - **Created**: 2021-11-08 - **Last Updated**: 2026-04-18 ## Categories & Tags **Categories**: Uncategorized **Tags**: None ## README # cuda repo management [![License](https://img.shields.io/badge/license-MIT-green.svg)](https://opensource.org/licenses/MIT-license) [![Contributing](https://img.shields.io/badge/Contributing-Developer%20Certificate%20of%20Origin-violet)](https://developercertificate.org) ## Overview Scripts for managing Debian and RPM package repositories containing many files. See [concept](#concept), [metadata](#metadata), and [presentations](#presentations) sections below. ## GPG keys Active pubkeys for [https://developer.download.nvidia.com/compute/cuda/repos/](https://developer.download.nvidia.com/compute/cuda/repos/) | Distro | Shortname | Entrypoint | | ----------- | ----------- | ---------- | | amzn2023 | `D42D0685` | [cuda-amzn2023.repo](https://developer.download.nvidia.com/compute/cuda/repos/amzn2023/x86_64/cuda-amzn2023.repo) | | azl3 | `D42D0685` | [cuda-azl3.repo](https://developer.download.nvidia.com/compute/cuda/repos/azl3/x86_64/cuda-azl3.repo) | | debian12 | `3bf863cc` | [cuda-keyring_1.1-1_all.deb](https://developer.download.nvidia.com/compute/cuda/repos/debian12/x86_64/cuda-keyring_1.1-1_all.deb) | | debian13 | `8793F200` | [cuda-keyring_1.1-1_all.deb](https://developer.download.nvidia.com/compute/cuda/repos/debian13/x86_64/cuda-keyring_1.1-1_all.deb) | | fedora42 | `D42D0685` | [cuda-fedora42.repo](https://developer.download.nvidia.com/compute/cuda/repos/fedora42/x86_64/cuda-fedora42.repo) | | fedora43 | `1940C73E` | - | | kylin10 | `D42D0685` | [cuda-kylin10](https://developer.download.nvidia.com/compute/cuda/repos/kylin10/x86_64/cuda-kylin10.repo) | | el8 | `D42D0685` | [cuda-rhel8](https://developer.download.nvidia.com/compute/cuda/repos/rhel8/x86_64/cuda-rhel8.repo) | | el9 | `D42D0685` | [cuda-rhel9](https://developer.download.nvidia.com/compute/cuda/repos/rhel9/x86_64/cuda-rhel9.repo) | | el10 | `CDF6BA43` | [cuda-rhel10](https://developer.download.nvidia.com/compute/cuda/repos/rhel10/x86_64/cuda-rhel10.repo) | | opensuse15 | `D42D0685` | [cuda-opensuse15](https://developer.download.nvidia.com/compute/cuda/repos/opensuse15/x86_64/cuda-opensuse15.repo) | | sles15 | `D42D0685` | [cuda-sles15](https://developer.download.nvidia.com/compute/cuda/repos/sles15/x86_64/cuda-sles15.repo) | | ubuntu2204 | `3bf863cc` | [cuda-keyring_1.1-1_all.deb](https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-keyring_1.1-1_all.deb) | | ubuntu2404 | `3bf863cc` | [cuda-keyring_1.1-1_all.deb](https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2404/x86_64/cuda-keyring_1.1-1_all.deb) | | ubuntu2604 | `60DF8A40` | - | ## Table of Contents - [Overview](#overview) - [Deliverables](#deliverables) - [Demo](#demo) - [Usage](#usage) * [repo-overlay](#repo-overlay) * [repo-debian](#repo-debian) * [repo-rpm](#repo-rpm) * [repo-mirror](#repo-mirror) * [repo-validate](#repo-validate) - [Prerequisites](#prerequisites) * [Clone this git repository](#clone-this-git-repository) * [(Optional) fetch genmodules](#optional-fetch-genmodules) * [Install build dependencies](#install-build-dependencies) - [Concept](#concept) * [OverlayFS](#overlayfs) * [Caching strategy](#caching-strategy) - [Metadata](#metadata) * [Debian](#debian) * [RPM](#rpm) - [Presentations](#presentations) * [PackagingCon](#packagingcon) - [Related](#related) * [Precompiled kmod](https://github.com/NVIDIA/yum-packaging-precompiled-kmod) * [Tarball and Zip Deliverables](https://github.com/NVIDIA/build-system-archive-import-examples) - [See also](#see-also) * [RHEL driver](https://github.com/NVIDIA/yum-packaging-nvidia-driver) * [Ubuntu driver](https://github.com/NVIDIA/ubuntu-packaging-nvidia-driver) * [SUSE driver](https://github.com/NVIDIA/zypper-packaging-nvidia-driver) - [Contributing](#contributing) ## Deliverables This repo contains scripts to generate repository metadata for **Debian** and **RPM** packages: * `apt`-based distros (**Debian** or **Ubuntu**) ```shell - Packages - Packages.gz - Release - Release.gpg ``` * `dnf`-based distros (**RHEL** >= 8 or **Fedora** >= 29) > _note_: modules.yaml requires `genmodules.py` ```shell - repodata/repomd.xml - repodata/repomd.xml.asc - repodata/repomd.xml.key - repodata/${sha256}-modules.yaml.gz - repodata/${sha256}-primary.xml.gz - repodata/${sha256}-primary.sqlite.bz2 - repodata/${sha256}-filelists.xml.gz - repodata/${sha256}-filelists.sqlite.bz2 - repodata/${sha256}-other.xml.gz - repodata/${sha256}-other.sqlite.bz2 ``` * `yum`-based distros (**RHEL** == 7) and `zypper`-based distros (**openSUSE** or **SLES**) ```shell - repodata/repomd.xml - repodata/repomd.xml.asc - repodata/repomd.xml.key - repodata/${sha256}-primary.xml.gz - repodata/${sha256}-primary.sqlite.bz2 - repodata/${sha256}-filelists.xml.gz - repodata/${sha256}-filelists.sqlite.bz2 - repodata/${sha256}-other.xml.gz - repodata/${sha256}-other.sqlite.bz2 ``` ## Demo ![Demo](https://developer.download.nvidia.com/compute/github-demos/cuda-repo-management/demo.gif) [![asciinema](https://img.shields.io/badge/Play%20Video-asciinema-red)](https://developer.download.nvidia.com/compute/github-demos/cuda-repo-management/demo-ascii/) ## Prerequisites ### Clone this git repository ```shell git clone https://github.com/NVIDIA/cuda-repo-management cd cuda-repo-management ``` ### (Optional) fetch genmodules > _note_: [genmodules.py](https://github.com/NVIDIA/yum-packaging-precompiled-kmod/blob/main/genmodules.py ) is needed for generating modularity streams for NVIDIA driver packages ```shell wget https://raw.githubusercontent.com/NVIDIA/yum-packaging-precompiled-kmod/main/genmodules.py ``` ### Install build dependencies ```shell # RPM repos yum install createrepo_c python3 # Debian repos yum install dpkg-dev # OverlayFS yum install e2fsprogs rsync # Misc that should already be installed yum install bash coreutils util-linux gawk sed findutils file gzip ``` ## Usage ### repo-overlay Update multiples repos at once using OverlayFS to layer directories ```shell ./repo-overlay.sh --mirror=path/to/snapshot (--output=path/to/save) (--tempdir=path/to/workdir) path/to/repos > ex: time ./repo-overlay.sh --mirror=/data/snapshot ``` ### repo-debian Generate Debian package repository metadata using `bash` and `dpkg` ```shell ./repo-debian.sh --mirror=path/to/snapshot --input=path/to/repos --repo=$distro/$arch > ex: time ./repo-debian.sh --mirror=/data/snapshot --input=$HOME/repos --repo=ubuntu1804/x86_64 ``` ### repo-rpm Generate RPM package repository metadata using `createrepo_c` ```shell ./repo-rpm.sh --mirror=path/to/snapshot --input=path/to/repos --repo=$distro/$arch > ex: time ./repo-rpm.sh --mirror=/data/snapshot --input=$HOME/repos --repos=rhel8/sbsa ``` ### repo-mirror Download Debian or RPM packages from an existing repository ```shell ./repo-mirror.sh --distro=$distro --arch=$arch (--version=$version) (--url=$repository) (--dryrun) > ex: ./repo-mirror.sh --output=/data/snapshot --distro=sles15 --arch=x86_64 ``` ### repo-validate Verify that metadata matches RPM and Debian packages (filesize and SHA256 checksum) ```shell ./repo-validate.sh --mirror=path/to/snapshot (--distro=$distro) (--arch=$arch) (--dryrun) > ex: ./repo-validate.sh --mirror=/data/snapshot --distro=rhel8 --arch=x86_64 ``` ## Concept Package managers do not scan repository directories (often served via HTTP/HTTPS), so from `apt`/`dnf`/`yum`/`zypper`'s perspective whether a package exists or not, is defined by pre-generated [metadata](#metadata) manifests. These metadata files provide Debian or RPM package availability information, which is used to resolve dependencies and for the selected files, determine the URLs to download. Adding new packages to a repository requires re-generating this metadata. Using tools such as `apt-ftparchive` or `createrepo_c`, the full set of packages must be present in a single directory. This presents logistical challenges that involve copying thousands of large files around. Moreover, it lacks an elegant "undo" mechanism for iterative software development processes such as CI/CD pipelines. ### OverlayFS Union file-systems such as OverlayFS, allow temporary overlapping of directory hierarchies using mount points. These mounts can cross file-system boundaries and utilize copy-on-write (COW). In other words, non-destructive repository merges that can be rolled back with `umount` without the need for redundant file copying. The mount syntax is layered right-to-left, with read-only (RO) lower layers, and one read-write (RW) upper layer. ```shell sudo mount -t overlay -o \ lowerdir=${layer3}:${layer2}:${layer1},\ upperdir=${layer4}/upper,\ workdir=${layer4}/workdir \ none \ /mnt/overlayfs ``` ### Caching strategy Generating repository metadata for thousands of packages is CPU intensive and thus can take a very long time. Rather than building from scratch each time, old metadata can be re-used to reduce the workload. #### Append-only The `repo-debian.sh` script scans each Debian package in the directory, recording the key-value pair of filename and size (in bytes), to a variable. Then it parses `Packages.gz` for filename and size (in bytes) key-value pair and saves it to another variable. The two variables are compared using the `comm` command, which eliminates the duplicate entries. The remaining packages are processed using a combination of `dpkg -I`, `du`, and `md5sum`/`sha1sum`/`sha256sum`/`sha512sum`. #### repodata The `createrepo_c` command has an `--update` parameter to utilize existing RPM repository metadata in the `repodata/` directory. This prints "CACHE HIT" when cache is successfully applies and "metadata are obsolete" when it detects a package mismatch requiring regenerating this data. A wrinkle when this is used with OverlayFS is the `repodata/` directory will only be available in the lower layer, as indicated by the error message "invalid cross-device link". The solution is to create a "write" operation, by moving the directory and then passing `--update-md-path` to specify the new location to scan for `repodata/`. ## Metadata A brief explanation of the format and contents of these repository metadata files. ### Debian The `apt`/`apt-get` package manager looks for `${repo}/InRelease` and `${repo}/Release` files as its entry-point. This contains a timestamp, repository flag options, and a list of package manifest files. For the latter filename, size in bytes, MD5sum, SHA1sum, and SHA256sum hashes are provided. #### Release The package manager compares the information found in `Release` with `Packages` to determine if there is corruption or tampering. Additionally the `Release` file is signed with a public-private GPG key pair and signature is detached as `Release.gpg`. This effectively signs the entire Debian package repository and requires clients to import the public GPG key to validate authenticity. ```shell Origin: Label: Architecture: Date: $(date -R --utc) MD5Sum: $md5 $bytes Packages $md5 $bytes Packages.gz SHA1: $sha1 $bytes Packages $sha1 $bytes Packages.gz SHA256: $sha256 $bytes Packages $sha256 $bytes Packages.gz Acquire-By-Hash: ``` The `Release` file references `Packages` and a gzipped copy `Packages.gz` #### Packages.gz Gzipped text file, containing blocks representing each package in the Debian repository. Blocks contain key-value pairs, such as `Package:`, `Version:`, and `Depends:`. For multi-line values (common for `Depends` and `Description`) start line with a space. Blocks are separated with an empty line. ```shell Package: $name Version: ${version}-${revision} Architecture: Priority: optional Section: <$repotype>/<$category> Maintainer: $user <$email> Installed-Size: $extract_bytes Depends: $packageA (>= $versionA), $packageB (>= $versionB), $packageC (>= $versionC) Filename: ./${name}_${version}-${revision}_${arch}.deb Size: $download_bytes MD5sum: $md5 SHA1: $sha1 SHA256: $sha256 SHA512: $sha512 Description: What this package is used for. Prefix each additional line with a space. Longer description for search with `apt-cache`. Separate each package block with an empty line. Package: ${packageA} Version: ${versionA}-1 ... ``` ##### Beware, "here be dragons" Some gotchas apply, for example the order of the package blocks affects dependency resolution. Chronological order (oldest → newest) release is recommended. Therefore if `repo-debian.sh` locates an existing `Packages.gz` manifest, it appends new blocks to the end. Additionally, there is a convention to the order of the key-values, as generated by `dpkg`. Some unofficial package tools such as `CMake` do not follow this convention, which in some cases has lead to undocumented behavior. Again `repo-debian.sh` tries to follow this convention as closely as possible. _ _ _ ### RPM The `yum`/`dnf` and `zypper` package managers look for a `${repo}/repodata/repomd.xml` file as its entry-point. This contains a list of package manifest files (SHA256-prefixed). Each `` XML tag contains the unique filename, size in bytes, timestamp, and SHA256sum hash (both compressed and extracted). The package manager uses this XML file to determine the locations of `primary.{xml.gz,sqlite.bz2}`, `filelists.{xml.gz,sqlite.bz2}`, `other.{xml.gz,sqlite.bz2}`, and optionally `modules.yaml.gz`. To be more CDN friendly, these RPM metadata files are prefixed with their `SHA256` hash, such that the filename is unique, each time its contents have been modified. #### repomd.xml The `repomd.xml` file is signed with a public-private GPG key pair and signature is detached as `repomd.xml.asc`. Individual RPM packages must be signed but this is an additional layer to sign the RPM package repository metadata and requires clients to import the public GPG key (`repomd.xml.key`) to validate authenticity. ```shell $epoch $SHA256 $extract_SHA256 $epoch $bytes $extract_bytes ... ... ... ... $SHA256 $extract_SHA256 $epoch $bytes $extract_bytes 10 $SHA256 $extract_SHA256 $epoch $bytes $extract_bytes ``` #### primary.xml.gz Gzipped XML file, containing `` tags representing each package in the RPM repository. Contains package name, version, description, dependencies, SHA256 checksum, timestamp, size in bytes, etc. It is recommended to use `createrepo_c` to generate these files. ```shell $name $arch $SHA256 Short description Longer multi-line description $packageA ... ``` #### modules.yaml Multi-document YAML text file, with one document per modularity stream. Each stream contains timestamp, unique `context` identifier, list of package names available for that stream, and one or more modularity profiles. Streams are designated by branch (choose one) and profiles represent bundle use cases (install one or more sets). The `genmodules.py` script generates this metadata file if any packages containing the string `nvidia-driver` are present in the repository. It is then injected into `repomd.xml` using `modifyrepo_c` command. ```shell document: modulemd version: 2 data: name: $module stream: $stream version: $(date +%Y%m%d%H%M%S) context: $(echo $name $stream $version $distro | md5sum | cut -c -10) arch: $arch summary: Short description description: >- Long multi-line description. artifacts: rpms: - $packageA-$epoch:$version-$release.$arch - $packageB-$epoch:$version-$release.$arch - $packageC-$epoch:$version-$release.$arch profiles: $profile: description: Profile description rpms: - $packageA - $packageB - $packageC ... --- document: modulemd version: 2 data: name: $module stream: $streamB ``` ## Presentations #### PackagingCon > Hi, [PackagingCon](https://packaging-con.org/) attendees! Thank you very much, grateful for the opportunity to participate, please keep in touch :smile: - [Repository Management](https://pretalx.com/packagingcon-2021/talk/88VTJZ/) - 2021 (1st year) - [video](https://developer.download.nvidia.com/presentations/2021/packagingcon/Repository_Management_PackagingCon.mp4) (MP4) - [slides](https://developer.download.nvidia.com/presentations/2021/packagingcon/Repository_Management_PackagingCon.pdf) (PDF) ## Related ### Precompiled kmod * [https://github.com/NVIDIA/yum-packaging-precompiled-kmod](https://github.com/NVIDIA/yum-packaging-precompiled-kmod) ### Tarball and Zip Deliverables * [https://github.com/NVIDIA/build-system-archive-import-examples](https://github.com/NVIDIA/build-system-archive-import-examples) ## See also ### RHEL driver * [https://github.com/NVIDIA/yum-packaging-nvidia-driver](https://github.com/NVIDIA/yum-packaging-nvidia-driver) ### Ubuntu driver * [https://github.com/NVIDIA/ubuntu-packaging-nvidia-driver](https://github.com/NVIDIA/ubuntu-packaging-nvidia-driver) ### SUSE driver * [https://github.com/NVIDIA/zypper-packaging-nvidia-driver](https://github.com/NVIDIA/zypper-packaging-nvidia-driver) ## Contributing See [CONTRIBUTING.md](CONTRIBUTING.md)