Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Revert to non-incremental docbuild #36469

Closed
wants to merge 7 commits into from
Closed
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
81 changes: 13 additions & 68 deletions .github/workflows/doc-build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -40,46 +40,15 @@ jobs:

build-docs:
runs-on: ubuntu-latest
container: ghcr.io/sagemath/sage/sage-ubuntu-focal-standard-with-targets:dev
container: ghcr.io/sagemath/sage/sage-ubuntu-focal-standard-with-targets:10.2.beta1
needs: [get_ci_fixes]
steps:
- name: Checkout
uses: actions/checkout@v4

- name: Update system packages
run: |
apt-get update && apt-get install -y git zip

- name: Add prebuilt tree as a worktree
id: worktree
run: |
set -ex
git config --global user.email "ci-sage@example.com"
git config --global user.name "Build & Test workflow"
git config --global --add safe.directory $(pwd)
# If actions/checkout downloaded our source tree using the GitHub REST API
# instead of with git (because do not have git installed in our image),
# we first make the source tree a repo.
if [ ! -d .git ]; then git init && git add -A && git commit --quiet -m "new"; fi
# Tag this state of the source tree "new". This is what we want to build and test.
git tag -f new
# Our container image contains a source tree in /sage with a full build of Sage.
# But /sage is not a git repository.
# We make /sage a worktree whose index is at tag "new".
# We then commit the current sources and set the tag "old". (This keeps all mtimes unchanged.)
# Then we update worktree and index with "git reset --hard new".
# (This keeps mtimes of unchanged files unchanged and mtimes of changed files newer than unchanged files.)
# Finally we reset the index to "old". (This keeps all mtimes unchanged.)
# The changed files now show up as uncommitted changes.
# The final "git add -N" makes sure that files that were added in "new" do not show
# as untracked files, which would be removed by "git clean -fx".
git worktree add --detach worktree-image
rm -rf /sage/.git && mv worktree-image/.git /sage/
rm -rf worktree-image && ln -s /sage worktree-image
if [ ! -f worktree-image/.gitignore ]; then cp .gitignore worktree-image/; fi
(cd worktree-image && git add -A && git commit --quiet --allow-empty -m "old" -a && git tag -f old && git reset --hard new && git reset --quiet old && git add -N . && git status)
# Keep track of changes to built HTML
new_version=$(cat src/VERSION.txt); (cd /sage/local/share/doc/sage/html/en && find . -name "*.html" | xargs sed -i '/class="sidebar-brand-text"/s/Sage [0-9a-z.]* /Sage '$new_version' /'; git init && (echo "*.svg binary"; echo "*.pdf binary") >> .gitattributes && (echo ".buildinfo"; echo '*.inv'; echo '.git*'; echo '*.svg'; echo '*.pdf'; echo '*.png'; echo 'searchindex.js') > .gitignore; git add -A && git commit --quiet -m "old")
- name: Checkout
uses: actions/checkout@v4

- name: Download upstream artifact
uses: actions/download-artifact@v3
Expand All @@ -91,50 +60,27 @@ jobs:
# After applying the fixes, make sure all changes are marked as uncommitted changes.
run: |
if [ -r upstream/ci_fixes.patch ]; then
(cd worktree-image && git commit -q -m "current changes" --allow-empty -a && git am; git reset --quiet old; git add -N .) < upstream/ci_fixes.patch
git config --global --add safe.directory "$GITHUB_WORKSPACE"
git config --global user.name "Sagemath bot"
git config --global user.email "ci-sage@example.com"
(git tag -f old && git am; git reset --quiet old; git add -N .) < upstream/ci_fixes.patch
fi

- name: Incremental build
id: incremental
- name: Prepare
run: |
# Now re-bootstrap and build. The build is incremental because we were careful with the timestamps.
./bootstrap && make build
working-directory: ./worktree-image
env:
MAKE: make -j2 --output-sync=recurse
SAGE_NUM_THREADS: 2

- name: Build (fallback to non-incremental)
id: build
if: always() && steps.worktree.outcome == 'success' && steps.incremental.outcome != 'success'
run: |
set -ex
make doc-clean doc-uninstall sagelib-clean && git clean -fx src/sage && ./config.status && make build
working-directory: ./worktree-image
env:
MAKE: make -j2 --output-sync=recurse
SAGE_NUM_THREADS: 2

# Reuse built SAGE_LOCAL contained in the Docker image
./bootstrap
./configure --enable-build-as-root --prefix=/sage/local --with-sage-venv --enable-download-from-upstream-url

- name: Build docs
id: docbuild
if: always() && (steps.incremental.outcome == 'success' || steps.build.outcome == 'success')
# Always non-incremental because of the concern that
# incremental docbuild may introduce broken links (inter-file references) though build succeeds
run: |
set -ex
export SAGE_USE_CDNS=yes
mv /sage/local/share/doc/sage/html/en/.git /sage/.git-doc
make doc-clean doc-uninstall sagelib-clean && git clean -fx src/sage
mkdir -p /sage/local/share/doc/sage/html/en/ && mv /sage/.git-doc /sage/local/share/doc/sage/html/en/.git
./config.status && make doc-html
working-directory: ./worktree-image
run: make doc-html
env:
MAKE: make -j2 --output-sync=recurse
SAGE_NUM_THREADS: 2

- name: Copy docs
id: copy
if: always() && steps.docbuild.outcome == 'success'
run: |
set -ex
mkdir -p ./docs
Expand All @@ -152,7 +98,6 @@ jobs:
zip -r docs.zip docs

- name: Upload docs
if: always() && steps.copy.outcome == 'success'
uses: actions/upload-artifact@v3
with:
name: docs
Expand Down
Loading