# puppet-compiler

**Repository Path**: mirrors_wikimedia/puppet-compiler

## Basic Information

- **Project Name**: puppet-compiler
- **Description**: Mirror of https://gerrit.wikimedia.org/g/operations/software/puppet-compiler See https://www.mediawiki.org/wiki/Developer_access for contributing
- **Primary Language**: Unknown
- **License**: Not specified
- **Default Branch**: master
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 0
- **Forks**: 0
- **Created**: 2022-04-05
- **Last Updated**: 2026-01-24

## Categories & Tags

**Categories**: Uncategorized

**Tags**: None

## README

# Testing

Use the following command to test locally
```
$ CHANGE=731113 NODES=cloudservices1003.wikimedia.org BUILD_NUMBER=1 python3 -m puppet_compiler.cli --debug --force
```
Or to debug a specific host failing
```
$ python3 -m puppet_compiler.debug_host -c 650494 sretest1001.eqiad.wmnet
```
## Steps to create testing environment (Linux)
The following instructions are pretty rough but should enable on to set up an
environment that allows some local hacking.  however please keep in mind that
these instructions don't set up or configure the puppetdb server.  As such you
want be able to test code which makes uses of exported resources or puppetdb
queries

* Create a `puppet-compiler.conf` file and place it in a valid configuration location. In order of precedence, this is:
    * The path specified in the `PC_CONFIG` environment variable
    * The current working directory
    * `XDG_CONFIG_HOME`
    * /etc

A sample configuration file, changing `PATH_TO_REPO` for the full path to the root of the git repository:

```yaml
base: "<PATH_TO_REPO>/.workspace/jenkins-workspace"
puppet_src: "<PATH_TO_REPO>/.workspace/catalog-differ/production"
puppet_private: "<PATH_TO_REPO>/.workspace/catalog-differ/private"
puppet_netbox: "<PATH_TO_REPO>/.workspace/catalog-differ/netbox-hiera"
http_url: "http://localhost"
puppet_var: "<PATH_TO_REPO>/.workspace/catalog-differ/puppet"
store_configs: False
```

* sudo apt-get install ruby-httpclient ruby-ldap ruby-rgen ruby-multi-json puppet
* ./utils/setup_workspace.sh
* ./utils/fetch_facts.sh

## Installing Python dependencies
The puppet compile have a few Python dependencies, which can be installed using:

```
$ python3 -mvenv .venv
$ pip install -r requirements.txt
```

This creates a new virtualenv and installes the required dependencies. Alternatively these can be installed using apt.
You can also use tools like pyenv or Conda rather than virtualenv if that is your preferred way of managing Python dependencies. 

## Running the CI tests locally
You can easily run the same tests that CI is running using the script:
* ./utils/run_ci_locally.sh

## Working with cloud hosts

If you would also like to test cloud hosts then you will need access to the openstack api for both the enc and custom hiera backend.  For this you will need a copy of
    * /usr/local/bin/puppet-enc
    * /etc/puppet-enc.yaml
and also set up a tunnle to the cloud puppetmaster (notice the different port number used to bypass nginx)
    * ssh -N -L8100:localhost:8101 cloud-puppetmaster-03.cloudinfra.eqiad.wmflabs
Finaly add the following to /etc/hosts
    * `127.0.0.1 puppetmaster.cloudinfra.wmflabs.org`

## Testing HTML rendering

The compilation output adds a presentation layer suitable for humans
consumption. You can trigger a dummy rendering to a "tmpdir" directory (which
will be deleted) using:
```
$ python3 -m puppet_compiler.debug_presentation -o tmpdir --force
...
Rendered files:
tmpdir/output/1911/42/srv001.example.org/index.html
tmpdir/output/1911/42/srv001.example.org/fulldiff.html
tmpdir/output/1911/42/srv001.example.org/corediff.html
```

The rendering comes from Jinja2 templates in `./puppet_compiler/templates`.