HashiCorp Nomad

Nomad's secret block lets a job name a credential instead of holding one. seekrit ships a secret provider plugin for it, so a job file looks like this:

job "billing-api" {
  namespace = "billing"

  secret "seekrit" {
    provider = "seekrit"
    path     = "billing-prod"
  }

  group "api" {
    task "server" {
      driver = "docker"

      env {
        DATABASE_URL      = "${secret.seekrit.DATABASE_URL}"
        STRIPE_SECRET_KEY = "${secret.seekrit.STRIPE_SECRET_KEY}"
      }
    }
  }
}

Nothing in that file is a credential, and nothing in Nomad's state is a plaintext.

How it works

Nomad execs the plugin on the client node that will run the task, and reads one JSON object from its stdout. The plugin holds the service token, calls GET /v1/resolve, and decrypts in a process that lives for one fetch.

job `secret` block ─▶ Nomad client ──exec──▶ seekrit plugin ──/v1/resolve──▶ seekrit API
  (names a profile)    (your host)            (decrypts here)                (ciphertext only)

Prerequisites

Nomad 1.11 or later — the secret block and the secret provider plugin framework arrived in that release.

A seekrit service token bound to the app environment the jobs need (see Service tokens):

seekrit token create --name nomad-billing --app billing --env production
# prints:  skt_XXXXXXXX_…    (save it now)

1. Install the plugin on every client

curl -fsSL https://nomad.seekrit.dev/install.sh | sh

or, with the CLI already installed:

seekrit nomad install

The plugin has to land in the secrets/ subdirectory of the agent's plugin directory, and the agent has to be told where that is:

# /etc/nomad.d/client.hcl
client {
  enabled           = true
  common_plugin_dir = "/opt/nomad/plugins"
}

→ /opt/nomad/plugins/secrets/seekrit

2. Give the client a credential

The plugin reads its configuration from /etc/seekrit/nomad.json on the client. seekrit nomad init prints the whole set of files for the machine you run it on:

seekrit nomad init --profile billing-prod --namespace billing

Profiles are named credentials the node operator declares. A job picks one by name; it cannot introduce one, widen one, or supply its own token.

{
  "profiles": {
    "billing-prod": {
      "token_file": "/etc/seekrit/tokens/billing",
      "namespaces": ["billing"],
      "jobs": ["billing-api", "billing-worker"]
    }
  }
}

# The token itself lives in its own file, not in the config.
sudo install -m 600 /dev/null /etc/seekrit/tokens/billing
echo "skt_…" | sudo tee /etc/seekrit/tokens/billing > /dev/null
sudo chmod 600 /etc/seekrit/nomad.json

Unknown keys are a hard error, deliberately: a misspelled namespaces would otherwise parse as "no restriction" and silently open the fence.

3. Restart the agent

sudo systemctl restart nomad

4. Check your work

seekrit nomad check --path billing-prod --namespace billing --job billing-api

nomad plugin check
installed    ok — /opt/nomad/plugins/secrets/seekrit
fingerprint  ok — 0.1.0
fetch        ok — 8 secrets

check runs the plugin exactly the way Nomad does, including the namespace and job id it would pass, so a profile your job is fenced away from fails here rather than at 3am. It prints a count, never names or values.

Paths

The first segment of path is always a profile name, never a secret name.

A whole-environment fetch is the usual choice, because Nomad interpolates ${secret.<block>.<KEY>} per key. Naming a single secret narrows what the job's templates can reach:

secret "stripe" {
  provider = "seekrit"
  path     = "billing-prod/STRIPE_SECRET_KEY"
}

Trying it on one client

For a single node, you can skip the config file entirely: put a token in the Nomad agent's own environment and use the default profile.

# /etc/systemd/system/nomad.service.d/seekrit.conf
[Service]
Environment=SEEKRIT_TOKEN=skt_…

secret "seekrit" {
  provider = "seekrit"
  # no `path` — the default profile, whole environment
}

Every job on that client can then read that one environment. Good for trying the plugin; not the shape for a shared cluster.

Availability

Nomad resolves secrets once per task placement, so a transient API blip during a placement storm would otherwise fail every task on the node at once. Turn on the last-known-good cache to survive that:

{
  "cache": { "dir": "/var/lib/seekrit/nomad-cache", "max_age": "15m" }
}

Only the still-encrypted resolve body is written to disk — the same bytes the API served, no more sensitive than the token next to it. Only a transport failure falls back: a 401 does not, because serving a cached payload after the API refused the token would quietly extend a credential that was revoked. max_age bounds how long that can last.

Telemetry

Nomad shows a plugin failure as one string on a task event and discards the plugin's stderr, so the plugin exports OpenTelemetry to your collector if you configure one:

Environment=OTEL_EXPORTER_OTLP_ENDPOINT=http://otel-collector:4318

It records the profile name, the calling namespace and job id, a secret count, whether the cache was used, and an error kind — never a value, a token, or ciphertext. See Telemetry.

Troubleshooting

Every way to get a Nomad plugin wrong fails silently, so work down this list rather than looking for an error message:

seekrit nomad check distinguishes the first three from the last two.

Reference

• CLI commands: seekrit nomad init, seekrit nomad install, seekrit nomad check.
• Minting and scoping tokens: Service tokens.
• Nomad's own docs: secret block and secret provider plugins.
• Runtime injection instead of a plugin: seekrit run.