Metadata-Version: 2.4
Name: infra-mgr
Version: 0.3.1
Summary: CLI for querying, checking, and mutating an ansible YAML inventory
Requires-Python: >=3.10
Description-Content-Type: text/markdown
Requires-Dist: PyYAML>=6.0
Provides-Extra: dev
Requires-Dist: pytest>=7.0; extra == "dev"

# infra-mgr

CLI for querying, checking, and mutating an ansible YAML inventory, and
for checking icinga2 host-registration conformity against it.

## Prerequisites

> **⚠️ Install Ansible before using `infra-mgr`.** It isn't pulled in by
> `pip`/`pipx`, and most commands shell out to it:
>
> | Commands | Needs |
> |---|---|
> | `ls`, `cat`, `ssh`, `check`, `add`, `rm` (anything that reads the inventory) | `ansible-inventory` |
> | `run`, `sync` | `ansible-playbook` |
> | `init`, `update` | `ansible-galaxy` (installs the third-party roles in `roles/requirements.yml`) |
>
> On Debian/Ubuntu:
>
> ```sh
> sudo apt install ansible
> ```
>
> Use the full `ansible` package, not just `ansible-core`: the roles use
> modules from bundled collections (e.g. `ansible.posix.authorized_key`).
> `init`/`update` check for `ansible-galaxy` up front and refuse to start
> without it, rather than leaving a `roles/` that can't run the playbooks.

## Install

```sh
pip install --index-url https://git.interhacker.space/api/packages/interhack/pypi/simple/ \
  --extra-index-url https://pypi.org/simple/ infra-mgr
```

`--extra-index-url` is needed because the Forgejo registry only hosts
`infra-mgr` itself, not its dependencies (PyYAML).

### Debian / Ubuntu

Recent Debian (12+) and Ubuntu (23.04+) refuse `pip install` / `pip
install --user` into the system Python (`error:
externally-managed-environment`, PEP 668). Use `pipx`, which gives
`infra-mgr` its own venv and puts the command on your PATH:

```sh
sudo apt install pipx
pipx ensurepath   # once, then open a new shell
pipx install infra-mgr \
  --index-url https://git.interhacker.space/api/packages/interhack/pypi/simple/ \
  --pip-args="--extra-index-url https://pypi.org/simple/"
# later, to upgrade -- pass the indexes again:
pipx upgrade infra-mgr \
  --index-url https://git.interhacker.space/api/packages/interhack/pypi/simple/ \
  --pip-args="--extra-index-url https://pypi.org/simple/"
```

Both indexes are treated as equal by pip, so on machines that matter,
pin the version (`infra-mgr==X.Y.Z`) to rule out a same-named package
on PyPI being picked instead.

Check what's installed with `infra-mgr --version` (also shown at the
top of every `--help`).

This is the normal way to install `infra-mgr` -- a real published
package, versioned and released independently of any one inventory
repo. (Installing from a local source checkout instead is for
developing `infra-mgr` itself, not for day-to-day use -- see
"Developing infra-mgr itself" near the bottom of this file.)

`infra-mgr` on its own only knows how to read/write a YAML inventory
and talk to `ansible-playbook` -- it has no org-specific logic baked
in. To actually manage hosts, you also need a real ansible repo (an
inventory, `site.yml`, and a `roles/` directory built from the
`infra_ansible_roles` package) to point it at. If you're working on
behalf of `tmplab` or `interhack`, ask for that org's own repo address
-- that gives you a real, already-working starting point rather than
building one from scratch. See `init`/`update` below for how `roles/`
itself gets populated into a repo like that.

## Usage

```sh
infra-mgr ls hosts | ls groups | ls services
infra-mgr cat host <name> | cat group <name> | cat service <name>
infra-mgr ssh [--group <name>] [--lan] [--pager N]
infra-mgr add host <hostname> [--ipv4 ...] [--groups g1,g2]
infra-mgr add group <hostname> <group>
infra-mgr rm host <hostname> [--apply]            # remove host + all its group memberships
infra-mgr rm group <hostname> <group> [--apply]   # remove one membership only
infra-mgr check <hostname>                  # icinga2 registration conformity
infra-mgr sync <hostname> [--apply]         # check + fix (dry-run by default)
infra-mgr run <hostname> [playbook] [--apply]     # run any playbook (dry-run by default)
infra-mgr lint <playbook> [<playbook> ...]  # cross-check services.py against roles actually used
infra-mgr init [--dev] [--version X]        # populate a fresh roles/ checkout
infra-mgr update [--version X] [--apply]    # refresh an existing roles/ checkout (dry-run by default)
```

Run any subcommand with no arguments (or `--help`) to see worked
examples, not just a bare usage line.

`run` is the general-purpose sibling to `sync` (which only ever knows
how to fix icinga2 registration specifically): it builds `ansible-playbook
-i <inventory> -l <host> <playbook>` and, same dry-run-by-default/
`--apply` pattern as `sync`, only prints the command unless you pass
`--apply`. If `playbook` is omitted, it looks for `site.yml` next to
the inventory file's own directory (e.g. `<org>/site.yml` when run with
the default `<org>/inventory/hosts.yml` layout) -- anything without
such a sibling (e.g. a flat `/etc/ansible/hosts`) has no derivable
default and `run` says so rather than guess. Needs a real
`--inventory-yaml` path -- `--inventory-json` is fixture/offline-only
and isn't a valid `ansible-playbook -i` argument, so `run` refuses it
outright.

`lint` is a pure static check, no ansible/live target/inventory needed:
it parses the `roles:` lists out of the given playbook file(s) and
cross-checks them against `services.py`'s SERVICES table two ways --
every service's declared role should be used by some play (else the
table is stale), and every composite role used in a play should have a
service entry (else a new composite was added without registering it).
Pass one org's playbooks together (e.g. `infra-mgr lint site.yml
backup.yml`) -- run per-org, not mixed across orgs, since a service
genuinely unused by one org is expected, not a bug, and running
org-by-org keeps the tool from reporting expected absences as findings.
Exits non-zero (with the specific role/service names) if anything's out
of sync.

`check`/`sync`'s `--icinga-conf-dir` defaults to the standard Debian
icinga2 path (`/etc/icinga2/conf.d/hosts.d`) -- override it for tests
or an unusual setup.

`rm` is the mirror of `add`, same dry-run-by-default/`--apply` pattern
as `sync`/`run`: `rm host` removes the host from the inventory AND
every group it currently belongs to (an `add host` call can add to
several groups at once; the reverse removes all of them too, so no
group-membership entry is left pointing at a host that no longer
exists). `rm group` only removes one membership, leaving the host and
its other groups untouched. Neither writes anything without `--apply`
-- the dry-run output shows exactly what would be removed.

**`init`/`update` have a hard dependency: they only make sense inside
an ansible repo that was actually built against the
`infra_ansible_roles` package** -- a `site.yml`/inventory that
references those roles by name, same as `tmplab`'s and `interhack`'s
own repos do. Running `init` somewhere else just gets you a `roles/`
folder full of files nothing refers to. If you're working on behalf of
one of those orgs, ask for their repo's address rather than starting
from an empty directory -- that's the compatible starting point, not
something you build by hand.

Given that, `init`/`update` populate and refresh that repo's `roles/`
directory from the published `infra_ansible_roles` package registry --
kept in a separate repo/package from the inventory/playbooks
specifically so the two can be versioned, released, and updated
independently.

- `infra-mgr init` populates a fresh, empty `roles/` directory. Run it
  once, from the directory that should contain `roles/` as a
  subdirectory (normally the same place your `site.yml`/inventory
  lives). Defaults to the latest published version; `--version X`
  pins to a specific one instead (e.g. to match what your playbooks
  were last tested against).
- `infra-mgr update` refreshes an existing `roles/` that `init` already
  created. Dry-run by default -- it prints the current version, the
  version it would move to, and the exact download URL, without
  changing anything; pass `--apply` to actually do it. Also defaults
  to latest, with the same `--version` override.
- Both finish by running `ansible-galaxy install -r
  roles/requirements.yml -p roles/`, for the third-party roles
  (`jloh.nagios_nrpe_server`, `albancrommer.borgbackup`) that the
  package deliberately doesn't ship. This needs `ansible-galaxy` (see
  "Prerequisites") and outbound access to github.com. `update`
  reinstalls them too, since it clears `roles/` before unpacking.
- Both take `--roles-dir PATH` if `roles/` isn't where you're
  currently standing (default: `roles`, relative to your current
  directory).

**Two ways to populate `roles/`, picked with `--dev`:**

- **Without `--dev` (the default, for normal use):** `init` downloads
  the published package for the chosen version and unpacks it as
  plain files -- no `.git` inside, nothing to commit, nothing to push.
  This is what `update` always refreshes. Use this unless you're
  specifically about to edit a role's own code.
- **With `--dev` (for developing a role):** `init --dev` does a real
  `git clone` of the roles repo into `roles/` instead -- a live,
  ordinary git checkout you can edit, commit, and push from, on its
  own, independently of whatever inventory/playbook repo it happens to
  sit inside. With no `--version`, it's left on its default branch
  (ready to start new work); `--dev --version X` checks out that tag
  instead (detached -- useful for reproducing or fixing something
  against a specific released version).

**`update` refuses to touch a `roles/` that was created with `--dev`.**
If it finds a live `.git` inside `roles/`, it stops immediately --
before changing anything, even on a dry-run -- and tells you to either
pull the latest changes yourself (`cd roles && git pull`) or remove the
directory and run `init` again if you want to go back to a normal,
package-managed install. `update` will never silently overwrite a git
checkout you're actively working in.

```sh
# normal use: get the latest roles, and keep them current
infra-mgr init
infra-mgr update              # later: dry-run, shows what would change
infra-mgr update --apply      # actually refresh to the latest

# pin to a known-good version instead of always-latest
infra-mgr init --version 0.1.2

# developing a role: get a real, editable git checkout
infra-mgr init --dev
cd roles && git checkout -b my-fix
# ... edit, test, commit, push from inside roles/ as you would any repo ...

# `update` protects that checkout rather than overwriting it:
cd .. && infra-mgr update --apply
# roles/ contains a live .git -- this looks like a developer's own checkout
# (from `infra-mgr init --dev`), not a package-managed install. Refusing
# to overwrite it.
#   - to pick up new upstream commits: cd roles/ && git pull
#   - to switch back to a package-managed install: remove roles/ yourself,
#     then run `infra-mgr init` again
```

All output (except `ssh`, which must stay plain space-separated for
tools like `cssh` to consume) is real YAML via `yaml.safe_dump`.

By default, reads/writes `inventory/hosts.yml` **relative to your
current working directory** (override with `--inventory-yaml`). A
missing or invalid inventory path (including a real file that just
isn't a valid inventory source) raises a clear error rather than
silently returning empty output. Pass `--inventory-json <export.json>`
to point at a pre-captured `ansible-inventory --list --export` dump
instead (used by the test suite's fixture, also handy for offline
inspection).

**`check`/`sync` may need elevated privileges** on a real target: they
read `/etc/icinga2/conf.d/hosts.d/<host>.conf` directly, which an
unprivileged user typically can't. Run these two with `sudo infra-mgr
...` if you hit a permissions error; `ls`/`cat`/`ssh`/`add`/`rm` don't
need it.

## Developing infra-mgr itself

The rest of this file is for using `infra-mgr`; this section is for
changing its own code. Install editable, from a source checkout,
instead of the published package:

```sh
python3 -m venv .venv && .venv/bin/pip install -e '.[dev]'
```

(Use a venv: `pip install --user -e .` is refused on recent
Debian/Ubuntu, see "Debian / Ubuntu" above. On Debian, `python3 -m
venv` needs `sudo apt install python3-venv` first.)

This points the `infra-mgr` command at whatever you're currently
editing in this checkout, rather than a fixed released version --
useful for iterating on the CLI's own code, not for day-to-day
inventory management (use the published package for that, per
"Install" above).

## Design

- `models.py` -- typed `Host`/`Group`/`Service`, the internal
  representation everything else operates on.
- `inventory.py` -- read path: parses `ansible-inventory --list --export`
  JSON.
- `store.py` -- write path: reads/mutates/writes the structured YAML
  inventory source directly. Every mutation persists immediately.
- `services.py` -- static service -> group metadata (which role, which
  icinga check).
- `conformity/icinga2.py` -- derives expected icinga2 `vars.*` from a
  host's actual group membership and diffs against the real `.conf`
  file.
- `commands/*.py` -- one function per verb, no CLI concerns inside
  them. `run.py` deliberately does NOT know what any given playbook
  does -- same "decide whether/what to run, hand off to
  ansible-playbook for how" split as `sync.py`.
- `output.py` -- the one shared `dump()` used by every command's
  formatter except `ssh`.
- `cli.py` -- argparse wiring only.

## Tests

```sh
python3 -m pytest
```

All tests run against `tests/fixtures/inventory_export.json`, a fixture
with fabricated hostnames/secrets -- no live inventory, ansible
install, or SSH access needed to run the suite.

### Four tiers, one runner: `infra-test`

The command above (`python3 -m pytest`) is the fast **unit** tier only
-- it mocks the `Runner` callable for `sync`/`run` (see `test_sync.py`)
and never actually invokes real `ansible-playbook`. That gap is exactly
what the e2e tier below exists to close: a command can build a
plausible-looking `ansible-playbook` invocation that's subtly wrong
(missing `-i`, a `--tags` that doesn't exist) and the unit suite alone
would never catch it, since it never runs the real subprocess.

`./infra-test` runs the unit/molecule/e2e tiers by default and reports
one YAML document, readable by a human or an LLM agent equally:

```sh
./infra-test                              # default: unit, molecule, e2e
./infra-test -u                           # just the fast unit suite (no Docker needed)
./infra-test role nrpe_check              # molecule, scoped to one role
./infra-test command sync                 # unit, scoped to sync-ish tests (pytest -k, approximate)
./infra-test -e -r /path/to/ansible/checkout --groups web,mysql
./infra-test -l                           # lifecycle: init/update against the real registry
./infra-test -a --out results.yml         # unit+molecule+e2e, written to a file instead of stdout
```

- **unit** -- this directory's own suite (above), always runnable.
- **molecule** -- `<role>/molecule/default/` for every role with a real
  scenario. Needs Docker.
- **e2e** -- `tests/e2e/`, drives the real `infra-mgr` CLI against a
  real ansible tree (`site.yml` + `roles/`) inside a throwaway
  container, nothing mocked. Needs Docker and `--ansible-root`/`-r`
  pointed at a real checkout -- see `tests/e2e/README.md` and
  `infra-test --help` for the one-time setup. Runs fully
  network-isolated (`--network=none` on every container).
- **lifecycle** -- `tests/lifecycle/`, drives the real `init`/`update`
  commands against the real published package registry, inside a
  (much smaller) throwaway container. Needs Docker and real outbound
  network access -- see `tests/lifecycle/README.md`. **Not** included
  in `-a`/`--all` or the no-flags default, and **not** network-isolated
  like every other Docker-based tier here -- it's the one tier whose
  entire point is proving a real network round-trip works, so it only
  runs when explicitly asked for with `-l`/`--lifecycle`.

Output shape (per tier): a `header` with an HTTP-flavored
`return_code` (200 all passed, 400 the tier ran and something failed,
500 the tier could not run at all -- e.g. Docker missing) plus
counts/timing, and a `body` listing `{check_name, message}` for
whatever failed (empty when `return_code` is 200). The process exits 0
only if every tier that ran was 200, 1 otherwise -- never fails fast in
multi-tier runs, so one invocation surfaces the whole backlog to fix,
not one failure at a time across repeated pushes.

Molecule, e2e, and lifecycle all need Docker -- `infra-test` reports a
clean 500 rather than failing opaquely when Docker isn't available.

## Deploying against a real inventory

This CLI has no org-specific logic -- point `--inventory-yaml` at
whatever real inventory you're managing:

```sh
infra-mgr --inventory-yaml /path/to/inventory/hosts.yml ls hosts
```

`add`/`rm --apply`/`sync --apply` mutate whatever file
`--inventory-yaml` points at -- double-check you're pointed at the file
you actually mean to change before running with `--apply`.

### Example session

```sh
$ infra-mgr --inventory-yaml inventory/hosts.yml cat host host1.example.org
host: host1.example.org
vars:
  vpn_address: 10.254.0.254
  vpn_interface: tinc0
  vpn_software: tinc
  web_software: apache2
groups:
- mysql
- vpn
- web

$ sudo infra-mgr --inventory-yaml inventory/hosts.yml check host1.example.org
host: host1.example.org
ok: false
missing_in_icinga2_config:
- mysql
- vpn

$ infra-mgr --inventory-yaml inventory/hosts.yml sync host1.example.org
# dry-run: prints the ansible-playbook command that would fix it

$ infra-mgr --inventory-yaml inventory/hosts.yml sync host1.example.org --apply
# applies it for real
```
