diff --git a/.gitattributes b/.gitattributes index 7c081f94b1..445793c205 100644 --- a/.gitattributes +++ b/.gitattributes @@ -9,4 +9,5 @@ CHANGELOG.md merge=union /.github export-ignore CHANGELOG.md export-ignore +.changes export-ignore .styleci.yml export-ignore \ No newline at end of file diff --git a/AGENTS.md b/AGENTS.md index faef475560..bf8bb0f009 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -296,17 +296,53 @@ The persistent-storage implementation is the reference pattern: `Project\Service - Merge `next`, `release/v*` branches, and patch branches (`v4.x`, `v4.4.x`) with a merge commit. Never squash merge them, because their commits then come back as conflicts in each later merge. - Create `release/v*` and patch branches only when the user asks. +## Change Files + +Each change that users of Coolify can see gets a file in `.changes/`, in the same commit as the change. `scripts/release.sh` writes the files into `CHANGELOG.md`, and the release notes come from there. + +Write a file with `scripts/change.sh` and its flags: + +```bash +scripts/change.sh --type fixed --summary 'Ignored concurrent build limit of build servers' --detail 'Each build counts on the server that runs it' +scripts/change.sh --type changed --breaking --summary 'Required the `delete` ability for every delete through the API' +scripts/change.sh --notes # the entries of the next release +``` + +- Write one file for each change. When a commit changes something that an unreleased change file describes, edit that file instead of adding a second one. +- Pass `--commit ` only to describe a commit that exists already. +- Tests, documentation, refactors, and CI, scripts, or release tooling that users do not see need no file. Use `maintenance` only for a change that matters to people who build or release Coolify. + +| Type | For | Next release | +| --- | --- | --- | +| `added` | Something new for users | minor | +| `changed` | Existing behavior that works differently | minor | +| `deprecated` | A feature that is going away | minor | +| `removed` | A feature that is gone | minor | +| `fixed` | A bug that users can see | patch | +| `security` | A closed vulnerability or hardening | patch | +| `maintenance` | A notable change that only maintainers see | patch | + +Add `--breaking` when users must act after the upgrade. The entry moves to `Breaking Changes`, and the next release is a major. + +The summary reads as one sentence with the heading of its section, never in the imperative of a commit subject: + +| Section | The summary | Example | +| --- | --- | --- | +| `Added`, `Changed`, `Deprecated`, `Removed` | What the heading acts on | Added `Automatic cancelling of the older queued webhook deployments` | +| `Fixed` | The bug, not the fix | Fixed `Ignored concurrent build limit of build servers`, not `Apply the concurrent build limit` | +| `Breaking Changes`, `Security`, `Maintenance` | Starts with a verb in the past tense | `Compressed the layers with zstd instead of gzip` | +| A `--detail` under any section | How it works now, in the present tense | `The default is none` | + +- Start with a capital letter or a code span, and end without a period. A reason follows after a comma. +- Write for the person who runs Coolify. Name the page, setting, or API field that changes, not the class or the file. +- Do not use bold. Put each UI name, such as a page, tab, button, setting, or label, in its own code span, for example `Advanced` > `Deployment`. +- Do not put exploit details, payloads, or unreleased advisory IDs in a change file. + ## Releases [`RELEASE.md`](RELEASE.md) is the source of truth for releases. Read it before you do a release, a backport, or a major version, and before you change `upstream.json`, `scripts/release.sh`, `scripts/plan-version.sh`, `scripts/sync-cdn.sh`, or the release, nightly, pull request, or merge forward workflows. -- Do not edit `CHANGELOG.md` unless the user asks. When the user asks you to update it: - - Collect the changes since the last release tag (`git log ..HEAD`, and the PR bodies of squash merges such as `next` into `main`). Check the code when an existing entry or a PR body conflicts with it. - - Write under `## [Unreleased]`, with the sections in this order: Breaking Changes, Security, Added, Changed, Fixed, Removed. Leave out empty sections. - - Write one entry for each big change: `- **Short title**: one or two short sentences`. Group related fixes in one entry by area, for example **Deployments**, **Backups**, **Cloudflare DNS**, or **Service templates**. - - Leave out small changes: UI tweaks, help texts, internal refactors, tests, CI, scripts, and Coolify Cloud internals. - - Put the issue and PR links at the end of the entry, for example `([#1234](https://github.com/coollabsio/coolify/issues/1234))`. Do not invent numbers. - - Use Simplified Technical English. Security entries describe the protection only, never exploit details or advisory IDs. +- Do not edit `CHANGELOG.md`. Describe each change in a change file (see [Change Files](#change-files)). - A push that changes `release.json` publishes a release. - NEVER edit `release.json` by hand. `scripts/release.sh` changes it. - `scripts/plan-version.sh` shows what the current branch releases. `scripts/plan-version.sh --nightly` shows the next nightly. diff --git a/RELEASE.md b/RELEASE.md index acd1d75f21..e6594fa846 100644 --- a/RELEASE.md +++ b/RELEASE.md @@ -36,11 +36,12 @@ Every release is tagged `v`, such as `v4.6.0-beta.1`, and publishes the > [!NOTE] > `next` publishes prereleases only. For its stable version, merge `next` into `main` with a merge commit and create the release on `main`. -1. Describe the changes under `## [Unreleased]` in `CHANGELOG.md`, they become the release notes -2. Check what the branch releases: +1. Describe the changes in change files with `scripts/change.sh`, they become the release notes, see [Change files](#change-files) +2. Check what the branch releases and the entries it lists: ```sh scripts/plan-version.sh + scripts/change.sh --notes ``` 3. Select the stage and the version, the script commits them and asks to push: @@ -59,6 +60,27 @@ Every release is tagged `v`, such as `v4.6.0-beta.1`, and publishes the > scripts/undo-release.sh > ``` +## Change files + +Every change has a file in `.changes/`, `AGENTS.md` describes the types and the wording: + +```markdown +--- +type: fixed +commit: e0b94662e9 +--- +Ignored concurrent build limit of build servers + +- Each build counts on the server that runs it +- A freed build server slot starts the waiting deployments of other servers +``` + +- `scripts/change.sh` writes a file, it asks for what its flags leave out, and `--notes` prints the entries of the next release +- `scripts/release.sh` preselects the version the files need and writes them into `CHANGELOG.md`, the release workflow reads its notes from there +- A prerelease writes the files under `## [Unreleased]`, keeps them and adds its version to the files that have none +- A stable release writes the files under `## [v4.6.1] - ` and removes them +- `scripts/undo-release.sh` brings the files back as they were + ## Backport a fix `main` is on 4.5, and 4.3 and 4.4 still get fixes. A fix reaches each of them through a patch branch and a pull request of its own: diff --git a/scripts/change.sh b/scripts/change.sh new file mode 100755 index 0000000000..5f20d6fd66 --- /dev/null +++ b/scripts/change.sh @@ -0,0 +1,167 @@ +#!/usr/bin/env bash +set -euo pipefail + +CHANGE_DIRECTORY=.changes +TYPES=(added changed deprecated removed fixed security maintenance) +SECTIONS=('Breaking Changes' Security Added Changed Deprecated Removed Fixed Maintenance) + +BOLD=$'\033[1m' GREEN=$'\033[32m' RED=$'\033[31m' DIM=$'\033[2m' RESET=$'\033[0m' +[ -t 1 ] || BOLD='' GREEN='' RED='' DIM='' RESET='' + +heading() { printf '\n %s%s%s\n' "$BOLD" "$*" "$RESET"; } +ok() { printf ' %s✔%s %s\n' "$GREEN" "$RESET" "$*"; } +die() { + printf ' %s✖%s %s\n' "$RED" "$RESET" "$*" >&2 + exit 1 +} +confirm() { + local answer + printf '\n %s [y/N] ' "$*" + read -r answer + [[ "$answer" =~ ^[Yy]$ ]] +} + +cd "$(dirname "$0")/.." +trap 'tput cnorm 2> /dev/null || true' EXIT + +main() { + read_arguments "$@" + if [ -n "$notes" ]; then + print_notes + return + fi + [ -n "$kind" ] && [ -n "$summary" ] || ask_questions + write_change +} + +read_arguments() { + kind='' summary='' commit='' breaking='' notes='' details=() + while [ $# -gt 0 ]; do + case "$1" in + --notes) notes=true ;; + --breaking) breaking=true ;; + --type | --summary | --detail | --commit) + [ $# -ge 2 ] || die "$1 has no value, expected one" + case "$1" in + --type) kind="$2" ;; + --summary) summary="$2" ;; + --detail) details+=("$2") ;; + --commit) commit="$2" ;; + esac + shift + ;; + *) die "'$1' is not an option, expected --type, --summary, --detail, --breaking, --commit or --notes" ;; + esac + shift + done +} + +ask_questions() { + local selected detail + [ -t 0 ] || die "--type and --summary are required without a terminal" + if [ -z "$kind" ]; then + heading "Select the type" + selected="$(menu 0 "${TYPES[@]}")" + kind="${TYPES[$selected]}" + fi + if [ -z "$summary" ]; then + heading "Summary, one sentence with the heading" + printf ' ' + read -r summary + fi + heading "Details, an empty line ends them" + while printf ' - ' && read -r detail && [ -n "$detail" ]; do + details+=("$detail") + done + if confirm "Breaking change?"; then + breaking=true + fi + printf '\n' +} + +write_change() { + local path detail + printf '%s\n' "${TYPES[@]}" | grep -qx -- "$kind" || die "the type is '$kind', expected ${TYPES[*]}" + [ -n "$summary" ] || die "the summary is empty, expected one sentence" + if [ -n "$commit" ]; then + git rev-parse --verify --quiet "$commit^{commit}" > /dev/null || die "$commit is not a commit of this repository" + commit="$(git rev-parse --short "$commit")" + fi + mkdir -p "$CHANGE_DIRECTORY" + path="$CHANGE_DIRECTORY/$(file_name).md" + { + printf -- '---\ntype: %s\n' "$kind" + [ -z "$breaking" ] || printf 'breaking: true\n' + [ -z "$commit" ] || printf 'commit: %s\n' "$commit" + printf -- '---\n%s\n' "$summary" + [ "${#details[@]}" = 0 ] || printf '\n' + for detail in ${details[@]+"${details[@]}"}; do + printf -- '- %s\n' "$detail" + done + } > "$path" + ok "$path describes the $BOLD$kind$RESET change" +} + +print_notes() { + local section entries + compgen -G "$CHANGE_DIRECTORY/*.md" > /dev/null || die "$CHANGE_DIRECTORY has no change files" + for section in "${SECTIONS[@]}"; do + entries="$(awk -v section="$section" ' + FNR == 1 { fences = 0; type = ""; breaking = 0 } + fences < 2 { + if ($0 == "---") fences++ + else if ($1 == "type:") type = $2 + else if ($0 == "breaking: true") breaking = 1 + next + } + NF && tolower(section) == (breaking ? "breaking changes" : type) { + prefix = /^- / ? " " : "- " + print prefix $0 + } + ' "$CHANGE_DIRECTORY"/*.md)" + [ -z "$entries" ] || printf '### %s\n\n%s\n\n' "$section" "$entries" + done +} + +file_name() { + local stem name number=2 + stem="$(printf '%s' "$summary" | LC_ALL=C tr -cs '[:alnum:]' '\n' | LC_ALL=C tr '[:upper:]' '[:lower:]' | awk 'NF && ++count <= 6' | paste -sd - -)" + name="$stem" + while [ -e "$CHANGE_DIRECTORY/$name.md" ]; do + name="$stem-$number" + number=$((number + 1)) + done + echo "$name" +} + +menu() { + local selected="$1" options key i + shift + options=("$@") + tput civis >&2 2> /dev/null || true + while true; do + for i in "${!options[@]}"; do + if [ "$i" = "$selected" ]; then + printf ' %s❯ %s%s\n' "$GREEN" "${options[$i]}" "$RESET" >&2 + else + printf ' %s %s%s\n' "$DIM" "${options[$i]}" "$RESET" >&2 + fi + done + IFS= read -rsn1 key + if [ "$key" = $'\x1b' ]; then + IFS= read -rsn2 key + if [ "$key" = '[A' ] && [ "$selected" -gt 0 ]; then + selected=$((selected - 1)) + elif [ "$key" = '[B' ] && [ "$selected" -lt $((${#options[@]} - 1)) ]; then + selected=$((selected + 1)) + fi + elif [ -z "$key" ]; then + break + fi + printf '\033[%dA\033[J' "${#options[@]}" >&2 + done + tput cnorm >&2 2> /dev/null || true + echo "$selected" +} + +main "$@" diff --git a/scripts/release.sh b/scripts/release.sh index 1f8f3f243c..9568414c5b 100755 --- a/scripts/release.sh +++ b/scripts/release.sh @@ -3,9 +3,10 @@ set -euo pipefail OPTION="$*" VERSION_FILE=release.json +CHANGE_DIRECTORY=.changes +SECTIONS=('Breaking Changes' Security Added Changed Deprecated Removed Fixed Maintenance) ARTIFACT=coolify UPSTREAM= -BREAKING_CHANGE='^([a-z]+(\([^)]*\))?!|BREAKING[ -]CHANGE): ' WORK_DIR="$(mktemp -d)" BOLD=$'\033[1m' BLUE=$'\033[34m' GREEN=$'\033[32m' YELLOW=$'\033[33m' RED=$'\033[31m' DIM=$'\033[2m' RESET=$'\033[0m' @@ -38,6 +39,7 @@ main() { select_version heading "Releasing v$new_version" update_changelog + update_change_files update_version commit push @@ -64,9 +66,9 @@ check_repository() { read -r behind ahead <<< "$counts" [ "$behind$ahead" = 00 ] || die "$branch is $ahead commit(s) ahead of and $behind behind origin/$branch, expected it in sync" ok "$branch is clean and in sync with origin/$branch" - entries="$(awk '/^## \[Unreleased\]$/ { inside = 1; next } /^## / { inside = 0 } inside && /^- / { count++ } END { print count + 0 }' CHANGELOG.md)" - [ "$entries" != 0 ] || die "CHANGELOG.md has no entries under ## [Unreleased], describe the changes first" - ok "CHANGELOG.md lists $BOLD$entries$RESET change(s) under [Unreleased], the notes of this release" + entries="$(git ls-files "$CHANGE_DIRECTORY" | wc -l | tr -d ' ')" + [ "$entries" != 0 ] || die "$CHANGE_DIRECTORY has no change files, describe the changes first with scripts/change.sh" + ok "$CHANGE_DIRECTORY lists $BOLD$entries$RESET change(s), the notes of this release" } read_versions() { @@ -81,19 +83,17 @@ read_versions() { } read_changes() { - local tag range hash subject - tag="$(git describe --tags --abbrev=0 --match "v${UPSTREAM:+*-v}$stable" 2> /dev/null)" || true - range="${tag:+$tag..}HEAD" + local file required="patch" - if [ -n "$(git log -E --grep="$BREAKING_CHANGE" --format=%h "$range")" ]; then + if grep -qx 'breaking: true' "$CHANGE_DIRECTORY"/*.md; then required=major [ "${stable%%.*}" != 0 ] || required=minor - elif git log --format=%s "$range" | awk '/^feat(\([^)]*\))?!?: / && !/^feat\(release\): bump / { found = 1 } END { exit !found }'; then + elif grep -qxE 'type: (added|changed|deprecated|removed)' "$CHANGE_DIRECTORY"/*.md; then required=minor fi - info "the commits since ${tag:-the first commit} need a $BOLD$required$RESET release" - git log -E --grep="$BREAKING_CHANGE" --format='%h %s' "$range" | while read -r hash subject; do - warn "$DIM$hash$RESET $subject is a breaking change" + info "the change files need a $BOLD$required$RESET release" + for file in $(grep -lx 'breaking: true' "$CHANGE_DIRECTORY"/*.md || true); do + warn "$file is a breaking change" done } @@ -152,19 +152,58 @@ select_version() { } update_changelog() { - local today - if [ "$channel" = preview ]; then - info "CHANGELOG.md keeps the entries under [Unreleased] until the stable release dates them" - return - fi - today="$(date +%Y-%m-%d)" - awk -v heading="## [v$new_version] - $today" ' - /^#/ && NR > 1 && previous != "" { print "" } - { print; previous = $0 } - /^## \[Unreleased\]$/ { print ""; print heading; previous = heading } + local heading='' + [ "$channel" = preview ] || heading="## [v$new_version] - $(date +%Y-%m-%d)" + print_notes > "$WORK_DIR/notes.md" + awk -v heading="$heading" -v notes="$WORK_DIR/notes.md" ' + /^## / { inside = 0 } + /^## \[Unreleased\]$/ { + inside = 1 + print + print "" + if (heading != "") { print heading; print "" } + while ((getline line < notes) > 0) print line + next + } + !inside ' CHANGELOG.md > "$WORK_DIR/CHANGELOG.md" cp "$WORK_DIR/CHANGELOG.md" CHANGELOG.md - ok "CHANGELOG.md dates the unreleased entries as ${BOLD}v$new_version$RESET, $today" + ok "CHANGELOG.md lists the changes under $BOLD${heading:-## [Unreleased]}$RESET" +} + +print_notes() { + local section entries + for section in "${SECTIONS[@]}"; do + entries="$(awk -v section="$section" ' + FNR == 1 { fences = 0; type = ""; breaking = 0 } + fences < 2 { + if ($0 == "---") fences++ + else if ($1 == "type:") type = $2 + else if ($0 == "breaking: true") breaking = 1 + next + } + NF && tolower(section) == (breaking ? "breaking changes" : type) { + prefix = /^- / ? " " : "- " + print prefix $0 + } + ' "$CHANGE_DIRECTORY"/*.md)" + [ -z "$entries" ] || printf '### %s\n\n%s\n\n' "$section" "$entries" + done +} + +update_change_files() { + local file + if [ "$channel" = stable ]; then + rm "$CHANGE_DIRECTORY"/*.md + ok "$CHANGE_DIRECTORY is empty, CHANGELOG.md lists the changes now" + return + fi + for file in "$CHANGE_DIRECTORY"/*.md; do + ! grep -q '^version: ' "$file" || continue + awk -v version="$new_version" '/^---$/ && ++fences == 2 { print "version: " version } { print }' "$file" > "$WORK_DIR/change.md" + cp "$WORK_DIR/change.md" "$file" + done + ok "$CHANGE_DIRECTORY names ${BOLD}v$new_version$RESET in the change files that had no version" } update_version() { @@ -174,7 +213,7 @@ update_version() { } commit() { - git add "$VERSION_FILE" CHANGELOG.md + git add --all -- "$VERSION_FILE" CHANGELOG.md "$CHANGE_DIRECTORY" git commit --quiet -S --message "feat(release): bump $ARTIFACT version to v$new_version" ok "signed commit $DIM$(git rev-parse --short HEAD)$RESET $(git log -1 --format=%s)" } diff --git a/scripts/undo-release.sh b/scripts/undo-release.sh index 843e107c9e..8444a4e632 100755 --- a/scripts/undo-release.sh +++ b/scripts/undo-release.sh @@ -3,6 +3,7 @@ set -euo pipefail OPTION="$*" VERSION_FILE=release.json +CHANGE_DIRECTORY=.changes BOLD=$'\033[1m' BLUE=$'\033[34m' GREEN=$'\033[32m' RED=$'\033[31m' DIM=$'\033[2m' RESET=$'\033[0m' @@ -26,7 +27,7 @@ main() { [ -z "$OPTION" ] || die "'$OPTION' is not an option, expected none" check_repository heading "Undoing $(git log -1 --format='%h %s')" - info "resets the commit and restores $VERSION_FILE and CHANGELOG.md" + info "resets the commit and restores $VERSION_FILE, CHANGELOG.md and $CHANGE_DIRECTORY" confirm "Continue?" || die "aborted, nothing changed" undo_commit } @@ -35,18 +36,18 @@ check_repository() { local branch git rev-parse --is-inside-work-tree > /dev/null 2>&1 || die "$PWD is not a git repository" branch="$(git rev-parse --abbrev-ref HEAD)" - git diff --quiet HEAD~1 HEAD -- . ":(exclude)$VERSION_FILE" ":(exclude)CHANGELOG.md" || die "the last commit changes more than $VERSION_FILE and CHANGELOG.md, expected a release commit" + git diff --quiet HEAD~1 HEAD -- . ":(exclude)$VERSION_FILE" ":(exclude)CHANGELOG.md" ":(exclude)$CHANGE_DIRECTORY" || die "the last commit changes more than $VERSION_FILE, CHANGELOG.md and $CHANGE_DIRECTORY, expected a release commit" [ "$(versions_at HEAD~1)" != "$(versions_at HEAD)" ] || die "the last commit keeps the versions in $VERSION_FILE at $(versions_at HEAD), expected a release commit" - [ -z "$(git status --porcelain -- "$VERSION_FILE" CHANGELOG.md)" ] || die "$VERSION_FILE or CHANGELOG.md has uncommitted changes, commit or stash them first" + [ -z "$(git status --porcelain -- "$VERSION_FILE" CHANGELOG.md "$CHANGE_DIRECTORY")" ] || die "$VERSION_FILE, CHANGELOG.md or $CHANGE_DIRECTORY has uncommitted changes, commit or stash them first" git fetch --quiet origin || die "could not fetch origin" [ "$(git rev-list --count "origin/$branch..HEAD")" -ge 1 ] || die "the release commit is on origin/$branch already, nothing to undo locally" } undo_commit() { git reset --quiet --soft HEAD~1 - git restore --staged --worktree "$VERSION_FILE" CHANGELOG.md + git restore --staged --worktree "$VERSION_FILE" CHANGELOG.md "$CHANGE_DIRECTORY" ok "reset to $DIM$(git rev-parse --short HEAD)$RESET $(git log -1 --format=%s)" - ok "$VERSION_FILE is back to $BOLD$(versions_at HEAD)$RESET, the entries are back under [Unreleased]" + ok "$VERSION_FILE is back to $BOLD$(versions_at HEAD)$RESET, the change files are back" } versions_at() {