This is an old revision of the document!
Table of Contents
Proxmox ACME DNS automation
I think we should have valid HTTPS also for internal services. I think it makes life easier for everyone. My one usecase here is Proxmox and the klynge001, but there is no reason for this to only be usable for those hosts.
Problem
The Proxmox web interfaces on host006 and host007 were observed serving expired Let's Encrypt certificates.
Both nodes have Proxmox ACME configuration, but renewal attempts failed with invalid ACME challenge status.
Current DNS for the node names resolves to internal management addresses.
This means HTTP-01 validation is not a good fit. Let's Encrypt cannot validate an internal/VPN-only Proxmox management endpoint unless that endpoint is deliberately exposed or routed from the public internet. We do not want to expose Proxmox management publicly just to renew certificates.
Expired browser-facing TLS on Proxmox can also interfere with browser-based Hacker-ID/OpenID login flows, I think. Not confirmed thats the issue, but anyways
Current DNS setup
Hackeriet hosts authoritative DNS for hackeriet.no on blade using NSD.
The hackeriet.no zone is also transferred to external secondary, ns.hyp.net.
Safe manual DNS changes should:
- edit the zone file on blade
- bump the SOA serial
- run nsd-checkzone
- reload NSD
- verify that ns0.hackeriet.no and ns.hyp.net agree
Options considered
Keep HTTP-01
Rejected. The Proxmox node names resolve to internal 10.10.50.x addresses. Making HTTP-01 work would require exposing or routing validation traffic to Proxmox management endpoints, which is the wrong security tradeoff.
Manual DNS-01
Works for immediate recovery. An operator can add temporary TXT records for _acme-challenge.host006.hackeriet.no and _acme-challenge.host007.hackeriet.no, complete issuance, then remove them.
This does not solve automatic renewal and will fail again unless someone repeats the process before expiry.
Git-pull automation on blade
Possible. Other systems could submit signed or reviewed DNS-change requests to a git repository, and blade could periodically pull, validate, update the NSD zone file, bump serial, check the zone, reload NSD, and commit the result.
This has good security properties because blade pulls rather than accepting inbound writes. It is also custom automation and therefore needs careful locking, validation, and operational ownership.
RFC2136 dynamic DNS updates
Standard DNS UPDATE with TSIG is a known way to automate DNS-01. It requires an authoritative DNS setup that supports dynamic updates and a carefully scoped key policy.
This may be a good fit for some DNS servers, but the current authoritative setup is NSD with git-managed zone files. Introducing dynamic updates would be a larger design change.
ACME challenge delegation
Preferred. This is a common ACME pattern. Keep stable records in the main hackeriet.no zone, and delegate only the _acme-challenge names to an automation-friendly validation target.
Example shape:
- _acme-challenge.host006.hackeriet.no CNAME host006.some-validation-zone.example.
- _acme-challenge.host007.hackeriet.no CNAME host007.some-validation-zone.example.
The ACME client updates TXT records at the delegated target. The main hackeriet.no zone stays mostly static.
Proposed solution
Use DNS-01 challenge delegation for Proxmox node certificates.
Add stable CNAME records in hackeriet.no for each Proxmox node that needs automated certificates. The CNAME targets should live in an automation-friendly validation zone or service such as acme-dns - https://github.com/acme-dns/acme-dns .
Security properties:
- Proxmox hosts do not get write access to hackeriet.no.
- blade keeps authoritative DNS control for the main zone.
- The main hackeriet.no zone only needs stable delegation records.
- Per-host ACME credentials can be scoped to one challenge name.
- More internal hosts can be added later using the same pattern.
This should be preferred over exposing Proxmox management publicly or giving host006/host007 broad DNS write access.
Suggested rollout
- Pick or deploy the validation backend. acme-dns is a common small service for this purpose.
- Create per-host validation names and credentials.
- Add CNAME records in hackeriet.no for host006 and host007 _acme-challenge names.
- Bump SOA serial, check the zone, reload NSD, and verify secondaries.
- Configure Proxmox ACME DNS validation for host006 first.
- Issue or renew the host006 certificate.
- Verify pveproxy serves the new certificate.
- Test Hacker-ID login against host006 after TLS is fixed.
- Repeat for host007.
Risks and constraints
- Do not expose Proxmox port 8006 publicly just to make HTTP-01 work.
- Do not put broad hackeriet.no DNS credentials on Proxmox hosts.