A simple server/agent architecture with all services running on a single machine.
Intended Audience
This architecture is intended for small to medium static infrastructures and is easier to set up and maintain than a load balanced compiler farm. Generally speaking, redundancy and HA are most useful for dynamic environments. A stopped Puppet server doesn’t prevent the existing configuration from being enforced but it will halt any new deployments or configuration updates. If those constraints fit your needs, then this architecture is suggested.
The simple service view (the more complex firewall view, with monitoring connections, is at the bottom):
Setup and Usage
We recommend to store your code in a git repository. For public code (e.g. Puppet modules), many people rely on GitHub. There are different solutions available to host your own git repositories, one of them is GitLab. GitLab has a open source version with MIT license and Vox Pupuli offers a Puppet module for it. We recommend to use any git hosting solution that supports calling webhooks for merges/pushes.
Git Repository
We recommend organizing your code as a Control Repository with branches for environments. See the reference repository for an example.
Foreman
Foreman is a complete lifecycle management tool for physical and virtual servers. It will provide you with a graphical classifier, a Hiera data source, and report monitoring. It also includes the power to easily automate repetitive tasks, quickly deploy applications, and proactively manage servers, on-premise or in the cloud.
Puppet Webhook
Vox Pupuli offers webhook-go, an open source (Apache-2.0 licensed) webhook that can listen for events from:
- GitLab on-premise
- GitLab Cloud
- GitHub
- GitHub Enterprise
- Azure DevOps
- Bitbucket
- Bitbucket server
- Gitea
webhook-go triggers r10k to deploy modules or environments. webhook-go calls r10k always locally, so it has to run on your OpenVoxServer. The r10k puppet module can configure r10k itself and the webhook.
Code Deployment
r10k is considered the default Puppet code deployment tool. Install it on your server in your infrastructure and use it to deploy your control repository as needed.
If you’re a Golang shop, you might consider g10k as well.
OpenVox Stack
We recommend managing each of these components with the supported module.
- OpenVoxDB
- puppetlabs/puppetdb
- The default PostgreSQL database is recommended.
- OpenVoxServer and agents
- Hiera Data Manager (HDM)
- r10k & webhook-go
- nftables firewalling
- InfluxDB, Telegraf & Grafana for monitoring
- puppetlabs/puppet_operational_dashboards (ships ready to use dashboards)
Scaleout options
- OpenVoxDB, OpenVoxServer, Foreman, PostgreSQL can all run on different systems.
- OpenVoxDB and Foreman use individual databases. They can run on the same PostgreSQL cluster, they don’t have to.
- It’s common to run OpenVoxServer and OpenVoxDB on the same system.
Containerisation
- Vox Pupuli offers containers for OpenVoxServer and OpenVoxDB
- Vox Pupuli also has containers for Puppetboard and HDM
- There’s also a container for r10k and for webhook-go
- crafty (Containerized Resources And Funky Tools in YAML) pulls all the images together
Firewalling and network access
Below is a flowchart for the network traffic between all those services. It’s currently work in progress.
Notes:
- webhook-go has no network connection to the puppetserver, it just deploys code that puppetserver needs
- Telegraf can run on each OpenVoxServer system, pull metrics locally and push them into InfluxDB
- If you prefer, you can run a central Telegraf on the monitoring node and let it pull metrics from the remote APIs (default behaviour for puppetlabs/puppet_operational_dashboards)
- Running Telegraf on each VM allows it to also collect local system metrics (disk/network/memory/CPU util etc)