Release Procedures¶
Annual¶
Pre-Release (e.g., v1.3rc1
)¶
Release (e.g., v1.3
)¶
- Do final pass before release tag
- Tag (pre)release
- Initialize release branch
- Build Conda Psi4 stack at specific commit
- Publish to main conda label
- Build Psi4conda set
- Generate download page for psicode.org
- Publish GitHub release
- Publish psicode release
- Finalize release
- Reset psi4meta for nightly operation
Post-Release (e.g., v1.3.1
)¶
- Assemble postrelease changes
- Tweak Conda for postrelease
- Do final pass before release tag
- Tag postrelease
- Build Conda Psi4 stack at specific commit
- Publish to main conda label
- Build Psi4conda set
- Generate download page for psicode.org
- Publish GitHub postrelease
- Publish psicode release
- Finalize release
- Reset psi4meta for nightly operation
Update copyright year¶
cd ~/path/to/psi4
- Primary target is licenses
grep -rl "(c) 2007-2017" * | xargs sed -i '' "s/(c) 2007-2017/(c) 2007-2018/g"
- On Linux, drop the
''
in above command - Need to do
psi4/
anddocs/
dirs
- Also, license in these files
tests/runtest.py
README.md
tests/psitest.pl
- Also, in content of https://github.com/psi4/psi4/blob/master/doc/sphinxman/source/conf.py.in#L118
Update samples¶
- Run
make sphinxman
at least once by hand - Check in resulting
psifiles.py
and all the updated and newsamples/
files and dirs - Make a lone PR and warn reviewers not to read it, since autogenerated
Collect new authors¶
- Survey contributions to current Milestone. Add new contribs and PR lists to release notes GitHub issue
- Figure out any new “Additional Contributors” authors since last release
- Edit
psi4/header.py
accordingly, make PR - Get permission of new authors and their particulars for
codemeta.json
- Invite any contributors with at least 3 PRs to join GitHub Organization
Anticipate next release¶
- Bump version in
codemeta.json
, https://github.com/psi4/psi4/blob/master/codemeta.json#L9 - Add to branch list in
.travis.yml
, https://github.com/psi4/psi4/blob/master/.travis.yml#L179 - Add to branch list in
azure-pipelines.yml
, https://github.com/psi4/psi4/blob/master/azure-pipelines.yml#L4
Build Conda ecosystem stack¶
By “ecosystem stack”, mean packages that are upstream, downstream, required, and optional for a fully featured Psi4 build and which we can’t get from “defaults” or “conda-forge” channels.
- Main directions are in [cbcy](https://github.com/psi4/psi4meta/blob/master/conda-recipes/conda_build_config.yaml#L90-L103) and [poodle](https://github.com/psi4/psi4meta/blob/master/psinet-nightly/kitandkapoodle.py#L357-L411)
- A couple weeks before the first “rc” is planned, start going through L/LT in poodle, checking with upstream to see if new versions have been released. If good changes present, rebuild the packages, changing the version numbers in the respective recipes
- When L/LT all built and passed, edit the individual package version numbers in cbcy and increment to a new
ltrtver
with updated version numbers and/or build numbers (only if code changes) - Build L/PSI4. If any trouble, edit psi4 code. Iterate until builds and passes. This stage is the only full ctest & pytest on Psi4+upstream
- Build L/RT-MP. If any trouble, edit code in L/RT and rebuild those package(s). Iterate until builds and passes. This stage is the only full ctest & pytest of Psi4+downstream
- Build L/DEV. If any trouble, edit psi4 build system, plugin system, or OpenMP setup. Iterate until builds and passes
- Build L/DOCS. If any trouble, edit the docs or the tests. Iterate until builds and passes
- Results of last should upload to psicode.org (docs) and codecov.io (coverage)
- Changes to targets’ “source” and “version” in individual recipes should be edited in psi4
external/*/*/CMakeLists.txt
files - Once everything’s working on Linux, repeat on Mac
- At this point, ready to fine-tune builds of “Psi4 stack”
Assemble postrelease changes¶
- Collect PRs with “backport” label, and request other backport suggestions on upcoming RN issue
- Cherry-pick backport PRs and commits, apply other changes manually, not forgetting CI files
- Do the draft parts of Publish GitHub postrelease
- Test thoroughly locally, including psi4-rt (may have to step back for dependency versions or builds)
- Note that while can’t change maintenance branch’s history, can push to it directly on upstream
- If want to do trial conda builds, requires
source/git_tag: 1.3.x
and fakepackage/version: v1.3.1rc1
Tweak Conda for postrelease¶
- In psi4-* recipes, find the best combination of master meta.yaml/build.sh and the v1N(x-1)-labeled ones
- comment out any dependencies in master but not yet in postrelease
- add
{{ dep_bld }}
entries if needed to step back build. make sure normal operation is''
- In cbcy,
- create a postrelease new line and record anything not in continuous order with <> or blanking
- step back any dependencies versions or build numbers to ones compatible with the postrelease
Do final pass before release tag¶
- Check that
external/
repos and commits have been updated to match conda recipes sources. Also check versions withconda_build_config.yaml
- Check
docs/sphinxman/source/introduction.rst
for any compiler and Python minimum requirements to edit.
Tag (pre)release¶
Thorough version bump directions at master http://psicode.org/psi4manual/master/manage_git.html#how-to-bump-a-version
Below is tl;dr
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42
# be on clean master up-to-date with upstream in both commits and tags # * mind which version strings get "v" and which don't # * if not fork, replace "upstream" with "origin" >>> vi psi4/metadata.py >>> git diff diff --git a/psi4/metadata.py b/psi4/metadata.py ... -__version__ = '1.3rc1' -__version_long = '1.3rc1+5a7522a' -__version_upcoming_annotated_v_tag = '1.3rc2' +__version__ = '1.3rc2' +__version_long = '1.3rc2+zzzzzzz' +__version_upcoming_annotated_v_tag = '1.3rc3' >>> git add psi4/metadata.py >>> git commit -m "v1.3rc2" [master bc8d7f5] v1.3rc2 >>> git log --oneline | head -1 bc8d7f5 v1.3rc2 >>> git tag -a v1.3rc2 bc8d7f5 -m "v1.3rc2" >>> vi psi4/metadata.py >>> git diff diff --git a/psi4/metadata.py b/psi4/metadata.py ... -__version_long = '1.3rc2+zzzzzzz' +__version_long = '1.3rc2+bc8d7f5' >>> git add psi4/metadata.py >>> git commit -m "Records tag for v1.3rc2" [master 16dbd3e] Records tag for v1.3rc2 # goto GH:psi4/psi4 > Settings > Branches > master > Edit # https://github.com/psi4/psi4/settings/branch_protection_rules/424295 # uncheck "Include administrators" and Save changes >>> git push upstream master >>> git push upstream v1.3rc2 # re-engage "Include administrators" protections
Tag postrelease¶
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 # be on clean maintenance branch up-to-date with upstream in both commits and tags # * mind which version strings get "v" and which don't # * if not fork, replace "upstream" with "origin" >>> git checkout 1.3.x Switched to branch '1.3.x' >>> vi psi4/metadata.py >>> git diff diff --git a/psi4/metadata.py b/psi4/metadata.py ... -__version__ = '1.3' -__version_long = '1.3+zzzzzzz' +__version__ = '1.3.1' +__version_long = '1.3.1+zzzzzzz' >>> git add psi4/metadata.py >>> git commit -m "v1.3.1" [1.3.x 2ce1c29] v1.3.1 >>> git log --oneline | head -1 786fb2b v1.3.1 >>> git tag -a v1.3.1 2ce1c29 -m "v1.3.1" # skipping the hash recording and "upcoming" step b/c only tags matter on maintenance branch # free pushing to maintenance branches at present so GitHub interface steps not needed # goto GH:psi4/psi4 > Settings > Branches > 1.3.x > Edit # https://github.com/psi4/psi4/settings/branch_protection_rules/4385008 # !Varies! # uncheck "Include administrators" and Save changes >>> git push upstream 1.3.x >>> git push upstream v1.3.1 # re-engage "Include administrators" protections
Initialize release branch¶
follow tagging procedure
before re-engaing the “include admin” button, push a branch at the tag commit (not the records commit)
1 2 3 4 5 6 7 8
>>> git log --online | head -2 45315cb Records tag for v1.3 20e5c7e v1.3 >>> git checkout 20e5c7e >>> git checkout -b 1.3.x Switched to a new branch '1.3.x' >>> git push upstream 1.3.x
set up new branch as protected branch through GitHub psi4 org Settings
Build Conda Psi4 stack at specific commit¶
By “Psi4 stack”, mean packages psi4
, psi4-rt
, psi4-dev
, psi4-docs
.
Other packages, the “ecosystem stack” (e.g., libint
, v2rdm_casscf
) should be already built.
- Check poodle for stray channels that may have crept in for dependencies (like c-f for ACS season). Copy over new dependencies if needed to psi4 channel
- Particularly before release (not prerelease), consider max pinnings on dependencies, particularly any fast-moving dependencies (e.g., qcel) and whether they need version space to grow compatibly and grow incompatibly.
- Nightly conda-builds work from
master
psi4. (Postrelease “practice” works from1.N.x
psi4.)- Instead, switch
source/git_tag
frommaster
to tag (e.g.,v1.3rc1
) in:- psi4-multiout on Linux & Mac, https://github.com/psi4/psi4meta/blob/master/conda-recipes/psi4-multiout/meta.yaml#L10
- psi4-docs on Linux, https://github.com/psi4/psi4meta/blob/master/conda-recipes/psi4-docs/meta.yaml#L10 on L
- In cbcy, edit
psi4ver
to “v”-less tag
- Instead, switch
- For releases and postreleases (not prereleases), in
conda_build_config.yaml
, editltrtver
to a new non-dev label (probably a ditto) matching the release (e.g., “1.3”) - Set
kitandkapoodle.py
to the normal***
stack. Should be (psi4
,psi4-rt
,psi4-dev
) * python_versions for Linux & Mac. Also singlepsi4-docs
from Linux - Run
kitandkapoodle.py
and allow stack to upload to anaconda.org topsi4/label/dev
. Poodle emits with--label dev
so will go to the subchannel. May need to delete packages to clear out space on anaconda.org - Copy
meta.yaml
andbuild.sh
of at leastpsi4-multiout
andpsi4-rt
to e.g., v121-build.sh files for easy reference or rebuilding as dependencies in master change.
Publish to main conda label¶
- Go through each active conda package off https://anaconda.org/psi4/repo
- Find the most recent build set (Linux/Mac, active py versions) that psi/psi-rt/psi-dev is using
- _add_ (not replace) the
main
label.
- This makes a
conda install psi4 -c psi4
get everything psi4 needs. For the momentconda install psi4 -c psi4/label/dev
will get the same set, until package psi4-1.4a1.dev1 gets uploaded. May help to check versions and build versions against ltrtver inconda_build_config.yaml
. - This step is manual, so takes a while. (It gets checked when Psi4conda installers are built b/c that pulls from “main”, not “dev”)
Build Psi4conda set¶
Installers are build using the project constructor
and build binary bash scripts, one per OS per Python version. In analogy to Miniconda, they’re called Psi4Conda. They can be built anywhere (Mac can be built on Linux) and get served from vergil (cdsgroup webserver).
Need a conda env with
constructor
andcookiecutter
. This env presently accessed throughconda activate cookie
.Enter “constructor-cutter-unified” in the psi4meta repo. There’s a good README there, https://github.com/psi4/psi4meta/blob/master/conda-recipes/constructor-cutter-unified/README.md
Edit
cookiecutter/cookiecutter.json
for control- Edit which python versions, if necessary
- Edit
release
field - Edit
hash
field. This is the 7-char hash that’s on every psi4 conda pkg as part of version - Edit
ltrtver
field. This matches the current setting inconda_build_config.yaml
- For prereleases,
"channel_tag": "/label/dev"
, while for (post)releases, it should be the empty string - Leave this file set to a “rc” with Git, as that has more details
For (post)releases (not prereleases), copy cookiecutter.json to cookiecutter.json-vXXX
Edit
cookiecutter/{{.../construct.yaml
for templating. This is rarely neededIf it’s been a while or you need the space, clear out
~/.conda/constructor
, where the downloaded packages are cachedNote that installers get written to
build/
and this gets regenerated each time. Clear out between runs.python run.py
[Outdated] Watch out for
py_
in buildstring as this means a noarch package has been pulled. It must be eliminated. Constructors can’t handle “noarch” packages and will fail at runtime. If see a “noarch” package, must find the recipe and rebuild for all OS & Python combinations. Then run constructor again.[Replacement] After adding a post_install to reposition the noarch packages, they are safe to include.
If fetching times out, may have to run run.py several times. Clear out build/ in between. It’s the fetching that takes a long time, not constucting
In the end, should have several installers
1 2 3 4 5
>>> ls -l build/psi4conda-1.3-py3.*/*64.sh -rwxr-xr-x. 516M Feb 28 20:30 build/psi4conda-1.3-py3.6-linux-64/psi4conda-1.3-py36-Linux-x86_64.sh -rwxr-xr-x. 299M Feb 28 20:31 build/psi4conda-1.3-py3.6-osx-64/psi4conda-1.3-py36-MacOSX-x86_64.sh -rwxr-xr-x. 518M Feb 28 20:30 build/psi4conda-1.3-py3.7-linux-64/psi4conda-1.3-py37-Linux-x86_64.sh -rwxr-xr-x. 299M Feb 28 20:31 build/psi4conda-1.3-py3.7-osx-64/psi4conda-1.3-py37-MacOSX-x86_64.sh
Upload installer files to vergil,
scp -r build/Psi4*/Psi4*sh root@vergil.chemistry.gatech.edu:/var/www/html/psicode-download/
Log in to vergil root and make WindowsWSL symlinks
Generate download page for psicode.org¶
- Be in repo psicode-hugo-website
- Copy and edit new file akin to https://github.com/psi4/psicode-hugo-website/blob/master/content/installs/v13rc2.md. Note the edition string
v13rc2
in frontmatter for this and future filenames - Copy and edit new file akin to https://github.com/psi4/psicode-hugo-website/blob/master/data/installs/v13rc2.yaml for menu and notes content
- Enter
scripts/
dir and edit primarily https://github.com/psi4/psicode-hugo-website/blob/master/scripts/install-generator.py#L9 but also any other arrays or messages that should change. - Run the
install-generator.py
in place. It will dump new files intodata/installs/
_subdirs_. Be sure togit add
them. - Installer page is now ready.
- Shift “latest” alias in frontmatter from whichever page is currently active to the new page. This makes sure “Downloads” on the navigation bar points to new page.
- Conscientiously, one should test
- installer downloads in Mac and Linux. And actually installing them and
psi4 --test
them. - that download button and
curl
downloading register on the download counters on vergil
- installer downloads in Mac and Linux. And actually installing them and
- Commit the new files, PR, and deploy psicode site
- Petition on Slack for testers
Publish GitHub release¶
- On GitHub site “Draft a New Release” with newly minted tag
- Fill in frontmatter style and links from previous GitHub release
- Fill in RN from hopefully existing RN issue
- Fill in RN by going through the top posts from all PRs from this milestone
- “publish” release. This establishes release date for GitHub API
- Close the RN issue.
- Close the milestone (should be 100% complete).
Publish GitHub postrelease¶
- On GitHub site “Draft a New Release” for anticipated or newly minted tag
- Fill in frontmatter style and links from previous GitHub release
- Fill in RN as cherry-pick to or edit on branch
- “publish” release. This establishes release date for GitHub API
Publish psicode release¶
- Copy a recent release page like https://github.com/psi4/psicode-hugo-website/blob/master/content/posts/v1p2.md
- Edit its filename, title, date, image, and links
- Execute https://api.github.com/repos/psi4/psi4/releases/latest and note the
id
field value - Use the
id
value in the shortcode call at the bottom
Finalize release¶
- Make new PR with
* edits to main
README.md
badges, python versions, etc. * edits toCMakeLists.txt
find_package(PythonLibsNew 3.6 REQUIRED)
- On godaddy, grab the exact tag 1.3 manual before it gets overwritten and preserve it:
cp -pR master 1.3
- Tweet about release
Reset psi4meta for nightly operation¶
On both Linux and Mac:
- After (post)release (not prerelease), in
conda_build_config.yaml
, editltrtver
to a new “release.dev” label - After postrelease, unpin any dependencies in cbcy that needed older either ver or bld (e.g., v2rdm_casscf_bld)
- Edit
psi4ver
back to''
in cbcy - Edit
source/git_tag
back tomaster
for psi4-multiout, psi4-docs - Edit build string back to
0
if psi4-multiout needed multiple passes - Edit kitandkapoodle.py back to
***
stack - Check in all release, construct, recipe changes on Linux and Mac. Synchronize both to GitHub psi4meta
- Copy meta.yaml and build.sh files to vMmp-prefixed files for the record.
- Edit crontab back to 2am “norm”. Comment out “anom”