workflows: add check for docbook/md manual equality

we want to make sure that rendering the manual from markdown without
going through docbook produces (semantically) the same output as with
going through docbook. to ensure this we'll build the manual twice, run
each manual through html-tidy to generate a normalized form and diff
the normalized forms. we don't want to compare raw output because that
exposes us to a lot of whitespace we'd have to reproduce exactly in the
MD render.

this check may be relaxed even further in the future, but hopefully not
by much.
This commit is contained in:
pennae 2022-09-03 16:35:28 +02:00 committed by pennae
parent ec75c8efff
commit c45deeb2aa
2 changed files with 84 additions and 0 deletions

21
.github/workflows/compare-manuals.sh vendored Executable file
View file

@ -0,0 +1,21 @@
#!/usr/bin/env nix-shell
#! nix-shell -i bash -p html-tidy
set -euo pipefail
shopt -s inherit_errexit
normalize() {
tidy \
--anchor-as-name no \
--coerce-endtags no \
--escape-scripts no \
--fix-backslash no \
--fix-style-tags no \
--fix-uri no \
--indent yes \
--wrap 0 \
< "$1" \
2> /dev/null
}
diff -U3 <(normalize "$1") <(normalize "$2")

63
.github/workflows/manual-rendering.yml vendored Normal file
View file

@ -0,0 +1,63 @@
name: "Check NixOS Manual DocBook rendering against MD rendering"
on:
schedule:
# * is a special character in YAML so you have to quote this string
# Check every 24 hours
- cron: '0 0 * * *'
permissions:
contents: read
jobs:
check-rendering-equivalence:
permissions:
issues: write # for peter-evans/create-or-update-comment to create or update comment
if: github.repository_owner == 'NixOS'
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: cachix/install-nix-action@v17
with:
# explicitly enable sandbox
extra_nix_config: sandbox = true
- uses: cachix/cachix-action@v10
with:
# This cache is for the nixpkgs repo checks and should not be trusted or used elsewhere.
name: nixpkgs-ci
signingKey: '${{ secrets.CACHIX_SIGNING_KEY }}'
- name: Build DocBook and MD manuals
run: |
export NIX_PATH=nixpkgs=$(pwd)
nix-build \
--option restrict-eval true \
-o docbook nixos/release.nix \
-A manual.x86_64-linux
nix-build \
--option restrict-eval true \
--arg configuration '{ documentation.nixos.options.allowDocBook = false; }' \
-o md nixos/release.nix \
-A manual.x86_64-linux
- name: Compare DocBook and MD manuals
id: check
run: |
.github/workflows/compare-manuals.sh \
docbook/share/doc/nixos/options.html \
md/share/doc/nixos/options.html
# if the manual can't be built we don't want to notify anyone.
# while this may temporarily hide rendering failures it will be a lot
# less noisy until all nixpkgs pull requests have stopped using
# docbook for option docs.
- name: Comment on failure
uses: peter-evans/create-or-update-comment@v2
if: ${{ failure() && steps.check.conclusion == 'failure' }}
with:
issue-number: 189318
body: |
Markdown and DocBook manuals do not agree.
Check https://github.com/NixOS/nixpkgs/actions/runs/${{ github.run_id }} for details.