Release procedure for HPX

Below is a step-wise procedure for making an HPX release. We aim to produce two releases per year: one in March-April, and one in September-October.

This is a living document and may not be totally current or accurate. It is an attempt to capture current practice in making an HPX release. Please update it as appropriate.

One way to use this procedure is to print a copy and check off the lines as they are completed to avoid confusion.

1. Notify developers that a release is imminent.

2. Make a list of examples and benchmarks that should not go into the release. Build all examples and benchmarks that will go in the release and make sure they build and run as expected.

   • Make sure all examples and benchmarks have example input files, and usage documentation, either in the form of comments or a readme.

3. Send the list of examples and benchmarks that will be included in the release to hpx-users@stellar.cct.lsu.edu and stellar@cct.lsu.edu, and ask for feedback. Update the list as necessary.

4. Write release notes in docs/sphinx/releases/whats_new_$VERSION.rst. Keep adding merged PRs and closed issues to this until just before the release is made. Add the new release notes to the table of contents in docs/sphinx/releases.rst.

5. Build the docs, and proof-read them. Update any documentation that may have changed, and correct any typos. Pay special attention to:

   • $HPX_SOURCE/README.rst
     • Update grant information
   • docs/sphinx/releases/whats_new_$VERSION.rst
   • docs/sphinx/about_hpx/people.rst
     • Update collaborators
     • Update grant information

6. This step does not apply to patch releases. For both APEX and hpxMP:

   • Change the release branch to be the most current release tag available in the APEX/hpxMP git_external section in the main CMakeLists.txt. Please contact the maintainers of the respective packages to generate a new release to synchronize with the HPX release (APEX, hpxMP).

7. If there have been any commits to the release branch since the last release create a tag from the old release branch before deleting the old release branch in the next step.

8. Unprotect the release branch in the github repository settings so that it can be deleted and recreated.

9. Delete the old release branch, and create a new one by branching a stable point from master. If you are creating a patch release, branch from the release tag for which you want to create a patch release.

   • git push origin --delete release
   • git branch -D release
   • git checkout [stable point in master]
   • git branch release
   • git push origin release
   • git branch --set-upstream-to=origin/release release

10. Protect the release branch again to disable deleting and force pushes.

11. Check out the release branch.

12. Make sure HPX_VERSION_MAJOR/MINOR/SUBMINOR in CMakeLists.txt contain the correct values. Change them if needed.

13. Remove the examples and benchmarks that will not go into the release from the release branch.

14. This step does not apply to patch releases. Remove features which have been deprecated for at least 2 releases. This involves removing build options which enable those features from the main CMakeLists.txt and also deleting all related code and tests from the main source tree.

    The general deprecation policy involves a three-step process we have to go through in order to introduce a breaking change

    1. First release cycle: add a build option which allows to explicitly disable any old (now deprecated) code.
    2. Second release cycle: turn this build option OFF by default.
    3. Third release cycle: completely remove the old code.

    The main CMakeLists.txt contains a comment indicating for which version the breaking change was introduced first.

15. Switch Buildbot over to test the release branch

    • https://github.com/STEllAR-GROUP/hermione-buildbot/blob/rostam/master/master.cfg
    • branch field in c['change_source'] =  GitPoller

16. Repeat the following steps until satisfied with the release.

    1. Change HPX_VERSION_TAG in CMakeLists.txt to -rcN, where N is the current iteration of this step. Start with -rc1.
    2. Tag a release candidate from the release branch, where tag name is the version to be released with a -rcN suffix and description is “HPX V$VERSION: The C++ Standards Library for Parallelism and Concurrency”.
       • git tag -a [tag name] -m '[description]'
       • git push origin [tag name]
       • Create a pre-release on GitHub
    3. This step is not necessary for patch releases. Notify hpx-users@stellar.cct.lsu.edu and stellar@cct.lsu.edu of the availability of the release candidate. Ask users to test the candidate by checking out the release candidate tag.
    4. Allow at least a week for testing of the release candidate.
       • Use git merge when possible, and fall back to git cherry-pick when needed. For patch releases git cherry-pick is most likely your only choice if there have been significant unrelated changes on master since the previous release.
       • Go back to the first step when enough patches have been added.
       • If there are no more patches continue to make the final release.

17. Update any occurrences of the latest stable release to refer to the version about to be released. For example, quickstart.rst contains instructions to check out the latest stable tag. Make sure that refers to the new version.

18. Add a new entry to the RPM changelog (cmake/packaging/rpm/Changelog.txt) with the new version number and a link to the corresponding changelog.

19. Change HPX_VERSION_TAG in CMakeLists.txt to an empty string.

20. Add the release date to the caption of the current “What’s New” section in the docs, and change the value of HPX_VERSION_DATE in CMakeLists.txt.

21. Tag the release from the release branch, where tag name is the version to be released and description is “HPX V$VERSION: The C++ Standards Library for Parallelism and Concurrency”. Sign the release tag with the contact@stellar-group.org key by adding the -s flag to git tag. Make sure you change git to sign with the contact@stellar-group.org key, rather than your own key if you use one. You also need to change the name and email used for commits. Change them to STE||AR Group and contact@stellar-group.org, respectively. Finally, the contact@stellar-group.org email address needs to be added to your GitHub account for the tag to show up as verified.

    • git tag -s -a [tag name] -m '[description]'
    • git push origin [tag name]

22. Create a release on GitHub

    • Refer to the ‘What’s New’ section in the documentation you uploaded in the notes for the Github release (see previous releases for a hint).
    • A DOI number using Zenodo is automatically assigned once the release is created as such on github.
    • Verify on Zenodo (https://zenodo.org/) that release was uploaded. Logging into zenodo using the github credentials might be necessary to see the new release as it usually takes a while for it to propagate to the search engine used on zenodo.

23. Roll a release candidate using tools/roll_release.sh (from root directory), and add the hashsums generated by the script to the “downloads” page of the website.

24. Upload the packages the website. Use the following formats:

    http://stellar.cct.lsu.edu/files/hpx_#.#.#.zip
    http://stellar.cct.lsu.edu/files/hpx_#.#.#.tar.gz
    http://stellar.cct.lsu.edu/files/hpx_#.#.#.tar.bz2
    http://stellar.cct.lsu.edu/files/hpx_#.#.#.7z
    

25. Update the websites (stellar-group.org and stellar.cct.lsu.edu) with the following:

    • Download links on the download page
    • Documentation links on the docs page (link to generated documentation on GitHub Pages)
    • A new blog post announcing the release, which links to downloads and the “What’s New” section in the documentation (see previous releases for examples)

26. Merge release branch into master.

27. This step does not apply to patch releases. Create a new branch from master, and check that branch out (name it for example by the next version number). Bump the HPX version to the next release target. The following files contain version info:

    • CMakeLists.txt
    • Grep for old version number
    1. Create a new “What’s New” section for the docs of the next anticipated release. Set the date to “unreleased”.
    2. Update $HPX_SOURCE/README.rst
       • Update version (to the about-to-be-released version)
       • Update links to documentation
       • Fix zenodo reference number
    3. Merge new branch containing next version numbers to master, resolve conflicts if necessary.

28. Switch Buildbot back to test the main branch

    • https://github.com/STEllAR-GROUP/hermione-buildbot/blob/rostam/master/master.cfg
    • branch field in c['change_source'] =  GitPoller

29. Update Vcpkg (https://github.com/Microsoft/vcpkg) to pull from latest release.

    • Update version number in CONTROL
    • Update tag and SHA512 to that of the new release

30. Announce the release on hpx-users@stellar.cct.lsu.edu, stellar@cct.lsu.edu, allcct@cct.lsu.edu, faculty@csc.lsu.edu, faculty@ece.lsu.edu, xpress@crest.iu.edu, the HPX Slack channel, the IRC channel, Sonia Sachs, our list of external collaborators, isocpp.org, reddit.com, HPC Wire, Inside HPC, Heise Online, and a CCT press release.

31. Beer and pizza.