Compiling and Installing from Source

Planning: how to configure Psi4 and invoke CMake

PSI4 is built through CMake. Prior to 1.1, PSI4 had a Python script setup as a frontend to CMake, but this is no more, and cmake is now invoked directly. An abbreviated build guide can be found within the source itself.

CMake does a good job scanning your computer to locate libraries, header files, and executables needed for compilation. So it’s very possible that from top-level-psi4-dir source directory, you can call cmake -H. -Bobjdir without any further arguments, and it will invoke cmake, detect some appropriate defaults, configure the build, create a build directory objdir, and complete, leaving you to only invoke make in the new build directory.

Should this happy scenario not come to pass, or if the default build options are not to your taste, use the links within core dependencies and add-on dependencies to plan a set of arguments to cmake tailored to your computer. Proceed to quick build or detailed build.

How to build and install Psi4, the compact version

This section outlines the main steps of configuring, compiling, and installing PSI4. More detail is given here.

1
2
3
4
5
>>> cd {top-level-psi4-dir}
>>> cmake -H. -Bobjdir [your configuration options]
>>> cd objdir
>>> make -j`getconf _NPROCESSORS_ONLN`
>>> make install

How to build, test, and install Psi4, in detail

1. Plan Directories

Get ahold of the PSI4 codebase, and navigate to the top level source directory, hereafter top-level-psi4-dir.

1
>>> cd {top-level-psi4-dir}

Choose a compilation directory, hereafter objdir

Choose an installation directory, hereafter prefix

2. Plan Configuration

Examine the strict and optional software requirements to make sure the target computer has all the necessary dependencies installed.

Prepare any necessary or desired configuration options for cmake, hereafter [your configuration options]

3. Configure

Run CMake with planned options and directories, as below. It reports on software found or unfound as it scans the computer, then (upon success) creates objdir ready for compilation.

1
>>> cmake -H. -B{objdir} -DCMAKE_INSTALL_PREFIX={prefix} [your configuration options]

4. Compile

Compile the code (optional -j triggers parallel compilation).

1
2
>>> cd {objdir}
>>> make -j`getconf _NPROCESSORS_ONLN`

5. Test

Optionally, use CTest (thorough) or pytest (cursory) to test the build.

1
>>> ctest -j`getconf _NPROCESSORS_ONLN`

1
>>> make pytest

6. Install

If tests pass, install the code.

1
>>> make install

7. Configure Runtime

To run Psi4 after installation, you need to configure a few variables:

What are the tools and dependencies strictly required for building Psi4

The core PSI4 build requires the software below. Note that Python, CMake, NumPy, and Libint (and even C++ compilers on Linux) can be satisfied through conda. The links below give examples of how to configure that software for PSI4 and any notes and warnings pertaining to it.

The following are also required for PSI4, but if not detected, the build system will automatically download and build.

What are the add-on capabilities for Psi4 and what are their dependencies

Each of the items below is an independent additional capability that can be built with PSI4. Sub-items below are the respective additional dependencies of the add-on. Select which, if any, you want, and examine the links for appropriate enabling arguments to cmake. Note that many are available pre-built from conda.

Additionally, there are runtime-only capabilities:

How to configure code to use high angular momentum basis sets

The Libint integral code handles arbitrary order angular momentum, but compiling that is prohibitive. The default of 5 is generally good. 6 has met all of a research group’s needs for years. 4 is handy for quickly testing other parts of the build.

  • Build with Higher Angular Momentum

    >>> cmake -DMAX_AM_ERI=6

  • Relevant CMake Options:

    MAX_AM_ERI=N        # The maximum angular momentum level (1=p, 2=d, 3=f,
                    # etc.) for the libint integrals and derivative
                    # integrals. A value of N implies a maximum first
                    # derivative of N-1, and maximum second derivative of
                    # N-2, so for an atom such as Neon, the default 5 gets
                    # you conventional cc-pV5Z for energies, cc-pVQZ for
                    # gradients, cc-pVTZ for frequencies and density-fitted
                    # cc-pVQZ for energies, cc-pVTZ for gradients, cc-pVDZ
                    # for frequencies. [default: 5]

Note that since PSI4 1.1, it is possible to build Libint independently (or install just the libint conda package), then have any/all PSI4 builds detect that installation at compile-time.

How to get high angular momentum integrals from conda

To switch from the default libint package to the really large high AM package, do the below. The channel/subchannel(s) containing the am8 metapackage and the high-AM libint package must be supplied (or in .condarc).

1
conda install am8 -c psi4

To go back to the default libint package, do the below. The channel/subchannel containing the default libint package must be supplied (or in .condarc); otherwise, it’ll just remove libint and every package depending on libint.

1
conda remove --features am8 -c psi4

The default package is AM6 because of its manageable file size (on Linux, 10MB for libint and 40MB for libderiv). The AM7 are 20/100, respectively, and the AM8 is 50/210.

How to see what build configuration options are available

CMake doesn’t provide a summary for this (unless you want to try the CMake GUI, which the developers have never looked at). However, the top half of the main CMakeLists.txt is a passable summary:


################  Options: Overview and Not Otherwise Mentioned  ###############

#  <<<  CMake build overview  >>>
#
#    >>> ls
#    external/  LICENSE  psi4/  tests/  ...
#    >>> cmake -H. -Bobjdir -DCMAKE_INSTALL_PREFIX=/path/to/install-psi4 ...
#    ...
#    -- Generating done
#    -- Build files have been written to: /current/dir/objdir
#    >>> cd objdir && make -j`getconf _NPROCESSORS_ONLN`
#    >>> make install

#  These three "###  Options  ###" sections contain useful CMake variables for build configuration.

#  <<<  Compilers and flags  >>>
#
#    - CMAKE_C_COMPILER "C compiler"
#    - CMAKE_C_FLAGS "Additional C flags"
#    - CMAKE_CXX_COMPILER "C++ compiler"
#    - CMAKE_CXX_FLAGS "Additional C++ flags"
#    - CMAKE_Fortran_COMPILER "Fortran compiler (required for some add-ons)"
#    - CMAKE_Fortran_FLAGS "Additional Fortran flags"

#  <<<  Detecting dependencies and add-ons  >>>
#
#    - PYTHON_EXECUTABLE "Python interpreter to use (e.g., /path/to/bin/python2.7)"
#    - PYTHON_LIBRARY "Python library that goes with the interpreter (e.g., /path/to/lib/python2.7.so)"
#    - PYTHON_INCLUDE_DIR "Path to the python include files (e.g., /path/to/include/python2.7)"
#    - SPHINX_ROOT "Root directory for Sphinx: 'bin/sphinx-build' (or similar) should be in this dir."
#
#    For any ${AddOn} of: ambit, CheMPS2, dkh, libefp, erd, gdma, Libint, PCMSolver, pybind11, simint
#    - CMAKE_PREFIX_PATH "Set to list of root directories to look for externally built add-ons and dependencies
#      (e.g., /path/to/install-libint;/path/to/install-gdma where exists /path/to/install-libint/lib/libderiv.a)"
#    - ${AddOn}_DIR "Set to directory containing ${AddOn}Config.cmake file to facilitate detection of external build"
#    - CMAKE_DISABLE_FIND_PACKAGE_${AddON} "Set to OFF to force internal build"

#  <<<  Detecting BLAS/LAPACK  >>>
#
#    - ENV(MATH_ROOT) "Root directory where BLAS/LAPACK libraries should be detected (e.g., ${MATH_ROOT}/lib/libblas.so)"
#    - BLAS_TYPE "Target BLAS distribution for math detection
#                 (default: search order MKL>OPENBLAS>ESSL>ATLAS>ACML>SYSTEM_NATIVE on Linux; MKL>Accelerate>... on Mac)"
#    - LAPACK_TYPE "Target LAPACK distribution for math detection
#                 (default: search order MKL>OPENBLAS>ESSL>ATLAS>ACML>SYSTEM_NATIVE on Linux; MKL>Accelerate>... on Mac)"
#    - LAPACK_LIBRARIES "Location of BLAS/LAPACK libraries as ";"-separated list of full paths, bypassing math detection"
#    - LAPACK_INCLUDE_DIRS "Location of BLAS/LAPACK headers (only needed for MKL), bypassing math detection"

#  <<<  Install  >>>
#
#    - CMAKE_INSTALL_PREFIX "Location to which Psi4 and internally built add-ons are installed (default: /usr/local/psi4)"
#    - CMAKE_INSTALL_BINDIR "Location within CMAKE_INSTALL_PREFIX to which executables are installed (default: bin)"
#    - CMAKE_INSTALL_LIBDIR "Location within CMAKE_INSTALL_PREFIX to which libraries are installed (default: lib)"
#    - CMAKE_INSTALL_DATADIR "Location within CMAKE_INSTALL_PREFIX to which resources are installed (default: share)"
#    - CMAKE_INSTALL_INCLUDEDIR "Location within CMAKE_INSTALL_PREFIX to which headers are installed (default: include)"
#    - PYMOD_INSTALL_LIBDIR "Location within CMAKE_INSTALL_LIBDIR to which python modules are installed (default: /)
#                            Must start with /"

############################  Options: Build What?  ############################
option(ENABLE_ambit "Enables the ambit tensor library" OFF)
option(ENABLE_CheMPS2 "Enables CheMPS2 for DMRG (requires HDF5)" OFF)
option(ENABLE_dkh "Enables DKH integrals (requires Fortran)" OFF)
option(ENABLE_libefp "Enables LIBEFP for fragments" OFF)
option(ENABLE_erd "Enables use of ERD instead of Libint (requires Fortran)" OFF)
option(ENABLE_simint "Enables use of SIMINT two-electron integral library" OFF)
option(ENABLE_gdma "Enables Stone's GDMA multipole code (requires Fortran)" OFF)
option(ENABLE_PCMSolver "Enables PCMSolver library (requires Fortran)" OFF)
# These options are relevant to pasture
option(ENABLE_ccsort "Enables ccsort plugin installed from psi4pasture" OFF)
option(ENABLE_transqt2 "Enables transqt2 plugin installed from psi4pasture" OFF)

# Append modules added to pasture as needed
if(ENABLE_ccsort OR ENABLE_transqt2)
  set(ENABLE_pasture ON)
  message(STATUS "Enabling pasture plugins")
endif()

if(ENABLE_gdma OR ENABLE_dkh OR ENABLE_erd OR ENABLE_PCMSolver)
    enable_language(Fortran)
    set(Fortran_ENABLED ON)  # communicate required languages with psi4-core
    message(STATUS "Enabling Fortran")
endif()

if(ENABLE_erd)
    message(WARNING "ERD will build, link, and run in Psi4 just fine. However, it has not been hooked into Psi4 in all roles, notably gradients, LRC DFT energies, and ESP. So upon activating through ``set integral_package erd``, known failures will be caught and halted, but perhaps other types not tested and identified will give *wrong* answers. Consider this your warning.")
endif()

############################  Options: Build How?  #############################
include(psi4OptionsTools)
option_with_print(BUILD_SHARED_LIBS "Build internally built Psi4 add-on libraries as shared, not static" OFF)
option_with_print(ENABLE_OPENMP "Enables OpenMP parallelization" ON)
option_with_print(ENABLE_AUTO_BLAS "Enables CMake to auto-detect BLAS" ON)
option_with_print(ENABLE_AUTO_LAPACK "Enables CMake to auto-detect LAPACK" ON)
option_with_print(ENABLE_PLUGIN_TESTING "Test the plugin templates build and run" OFF)
option_with_flags(ENABLE_XHOST "Enables processor-specific optimization" ON
                  "-xHost" "-march=native")
option_with_flags(ENABLE_CODE_COVERAGE "Enables details on code coverage" OFF
                  "-ftest-coverage")
option_with_flags(ENABLE_BOUNDS_CHECK "Enables bounds check in Fortran" OFF
                  "-ftrapuv -check all -fpstkchk" "-fcheck=all" "-fbounds-check -fcheck-array-temporaries")
option_with_flags(ENABLE_ASAN "Enables address sanitizer" OFF
                  "-fsanitize=address" "-fno-omit-frame-pointer")
option_with_flags(ENABLE_TSAN "Enables thread sanitizer" OFF
                  "-fsanitize=thread" "-fno-omit-frame-pointer -pie")
option_with_flags(ENABLE_UBSAN "Enables undefined behavior sanitizer" OFF
                  "-fsanitize=undefined" "-fno-omit-frame-pointer")
option_with_default(MAX_AM_ERI "Maximum angular momentum for integrals" 5)
option_with_default(CMAKE_BUILD_TYPE "Build type (Release or Debug)" Release)
option_with_default(FC_SYMBOL "The type of Fortran name mangling" 2)
option_with_default(BUILD_FPIC "Compile static libraries with position independent code" ON)

Note that external projects will have their own sets of build configuration options. Only the most-common user knobs of those are mentioned above.

How to install elsewhere than /usr/local/psi4

The installation directory is the filesystem location for the executable script, the Python module, basis set data, and other administrative files. Unless using the conda package, which is relocatable, the installation directory must be specified with CMake variable CMAKE_INSTALL_PREFIX before compiling.

  • Build with Specific Install Directory

    cmake -DCMAKE_INSTALL_PREFIX=/nfs/common/software/psi4

  • Relevant CMake Options:

    CMAKE_INSTALL_PREFIX=PATH  # Location to which Psi4 and internally built
                           # add-ons are installed (default: /usr/local/psi4)

How to compile for debugging

Flags to turn optimizations off and debugging on can be set across the project and plugins with CMake variable CMAKE_BUILD_TYPE before compiling. Note that these flags will not propagate to any add-ons that are detected pre-built rather than built.

  • Build without optimization

    cmake -DCMAKE_BUILD_TYPE=debug

                              set the CMake build type [default: release]

  • Relevant CMake Options:

    CMAKE_BUILD_TYPE=[debug|release]  # Build type (Release or Debug)" [default: release]

How to fix error “RuntimeError: value for ERI“

You will need to rebuild Libint. Reissue cmake or edit CMakeCache.txt with larger MAX_AM_ERI and rebuilt.

How to choose the compilation directory, {objdir}

  • there is no default
  • common choices are objdir or build under top-level-psi4-dir
    • cd {top-level-psi4-dir} && cmake -H. -Bobjdir
    • cd {top-level-psi4-dir} && cmake -H. -Bbuild
  • in-source builds (*.cc and *.o in same directory) are disallowed
  • builds outside top-level-psi4-dir are permitted

How to save configuration settings for a future compilation

Create a file like do-configure with the cmake command and options on one line.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
>>> cd {top-level-psi4-dir}
>>> cat do-configure
    cmake -H. -B{objdir} \
        -DCMAKE_INSTALL_PATH="/Users/me/psi4" \
        -DCMAKE_PREFIX_PATH="/Users/me/externals/install-libint" \
        -DMAX_AM_ERI=6 \
        -DENABLE_gdma=ON \
        -DBUILD_SHARED_LIBS=ON
>>> chmod u+x do-configure
>>> ./do-configure

What is the directory layout of the installed or staged Psi4

After compilation (cd objdir && make), a directory structure like the below will exist at objdir/stage/prefix. This may be tested and used just like a full installation.

After installation (cd objdir && make && make install), a directory structure like the below will exist at /prefix. This is a full installation.

/
bin/                                                   (executables for psi4 + any external proj)
bin/psi4                                             (psi4 executable, actually just a py script)
include/                                         (installed headers for psi4 + any external proj)
include/psi4/                                                     (header files for #include-ing)
include/psi4/psi4-dec.h                                                     (primary psi4 header)
include/psi4/masses.h                                                (a project-wide psi4 header)
include/psi4/libmints/                                                     (psi4 library headers)
include/psi4/libfock/                                                                     (ditto)
share/                                  (read-only arch-indep files for psi4 + any external proj)
share/cmake/psi4/                                         (files for detecting installed targets)
share/cmake/psi4/psi4Config.cmake                                       (psi4 build/install info)
share/cmake/psi4/psi4ConfigVersion.cmake                                      (psi4 version info)
share/doc/psi4/html/                                                  (sphinx html documentation)
share/psi4/                                                           (text files needed by psi4)
share/psi4/basis                                                                     (basis sets)
share/psi4/plugins                                                        (plugin template files)
share/psi4/fsapt                                                                  (fsapt scripts)
share/psi4/samples/                                                          (sample input files)
lib/                               (shared libraries and py modules for psi4 + any external proj)
# ordinary
lib/psi4/                                                                          (object files)
lib/psi4/driver/                                                            (py-side, uncompiled)
lib/psi4/header.py                                                           (prints file header)
lib/psi4/__init__.py                                         (module marker/loader for psi4.core)
lib/psi4/core.so                                         (c-side, compiled and bound by pybind11)
# conda
lib/pythonX.X/site-packages/psi4/

The following environment variables point to certain places in the above directory structure. None to few need to be set; see for details: running compiled executable, running compiled Python module, running conda binary.

  • PATH pointing to bin
  • PYTHONPATH pointing to lib (ordinary) or lib/pythonX.X/site-packages (conda)
  • PSIDATADIR pointing to share/psi4

How to run Psi4 as executable after compilation

Substituting the full installation directory prefix and a suitable scratch directory, issue the following commands directly in your terminal or place them into your “rc” file and open a new terminal. (To use a staged installation directory, substitute objdir/stage/prefix for prefix.)

# csh, tcsh: add to shell or ~/.tcshrc file
setenv PATH {prefix}/bin:$PATH
setenv PSI_SCRATCH /path/to/existing/writable/local-not-network/directory/for/scratch/files

# sh, bash: add to shell or ~/.bashrc (Linux/Windows) or ~/.bash_profile (Mac) file
export PATH={prefix}/bin:$PATH
export PSI_SCRATCH=/path/to/existing/writable/local-not-network/directory/for/scratch/files

Run PSI4.

1
2
3
4
5
6
7
8
>>> cat sample.in
molecule {
He
}
energy('hf/cc-pvdz')
compare_values(-2.85518839, get_variable('current energy'), 5, 'SCF E')
>>> psi4 sample.in
SCF E.............................................................PASSED

todo how to check if current py is compatible with compilation

How to run Psi4 as Python module after compilation

Substituting the full installation directory prefix and a suitable scratch directory, issue the following commands directly in your terminal or place them into your “rc” file and open a new terminal. (To use a staged installation directory, substitute objdir/stage/prefix for prefix.)

# csh, tcsh: add to shell or ~/.tcshrc file
setenv PYTHONPATH {prefix}/lib:$PYTHONPATH
setenv PSI_SCRATCH /path/to/existing/writable/local-not-network/directory/for/scratch/files

# sh, bash: add to shell or ~/.bashrc (Linux/Windows) or ~/.bash_profile (Mac) file
export PYTHONPATH={prefix}/lib:$PYTHONPATH
export PSI_SCRATCH=/path/to/existing/writable/local-not-network/directory/for/scratch/files

Note

If you know the location of the PSI4 executable (bin/psi4) and want to know the corresponding location to add to PYTHONPATH, execute psi4 --psiapi-path. It will return bash commands to set PATH (for correct python interpreter) and PYTHONPATH (to find psi4 module) correctly, after which python -c "import psi4" will work.

Run PSI4.

1
2
3
4
5
6
7
8
9
>>> cat sample.py
import psi4
mol = psi4.geometry("""
He
""")
psi4.energy('hf/cc-pvdz')
psi4.compare_values(-2.85518839, psi4.core.get_variable('current energy'), 5, 'SCF E')
>>> python sample.py
SCF E.............................................................PASSED

How to run Psi4 as executable or Python module from conda installation

The configuration commands below are generic versions of the ones printed to your screen as advice upon installing PSI4 into a Anaconda, Miniconda, or Psi4conda distribution, condadist = ana|mini|psi4conda. If which conda python psi4 points to your condadist and echo $PSI_SCRATCH is set, skip ahead to the “Run PSI4” commands below. Otherwise, issue the following commands directly in your terminal or place them into your “rc” file and open a new terminal.

If you installed the Psi4conda distribution or installed the PSI4 conda package into the main environment of an Anaconda or Miniconda distribution and added that to your PATH, as prompted, then which psi4 likely yields condadist/bin/psi4 and the PATH setting lines below are redundant.

If you installed into a conda environment p4env and performed source activate p4env, then which psi4 likely yields condadist/envs/p4env/bin/psi4 and the PATH setting lines below are redundant.

# csh, tcsh: add to shell or ~/.tcshrc file
unsetenv PSIDATADIR
setenv PATH {prefix}/bin:$PATH
setenv PSI_SCRATCH /path/to/existing/writable/local-not-network/directory/for/scratch/files

# sh, bash: add to shell or ~/.bashrc (Linux/Windows) or ~/.bash_profile (Mac) file
unset PSIDATADIR
export PATH={prefix}/bin:$PATH
export PSI_SCRATCH=/path/to/existing/writable/local-not-network/directory/for/scratch/files

Run PSI4 as executable.

1
2
3
4
5
6
7
8
>>> cat sample.in
molecule {
He
}
energy('hf/cc-pvdz')
compare_values(-2.85518839, get_variable('current energy'), 5, 'SCF E')
>>> psi4 sample.in
SCF E.............................................................PASSED

or Run PSI4 as Python module.

1
2
3
4
5
6
7
8
9
>>> cat sample.py
import psi4
mol = psi4.geometry("""
He
""")
psi4.energy('hf/cc-pvdz')
psi4.compare_values(-2.85518839, psi4.core.get_variable('current energy'), 5, 'SCF E')
>>> python sample.py
SCF E.............................................................PASSED

How to run Psi4 as executable after compilation using driver from source

When developing python driver code, it can be annoying to keep makeing to test the code. PSI4 can be run “inplace” through the following procedure. To be clear, this is running compiled C++ from the build directory and python from the source directory. This is an expert option for development, and not all functionality will be available.

1
2
3
>>> cd {objdir}
>>> ln -s {top-level-psi4-dir}/{objdir}/stage/{prefix}/lib/psi4/core.so ../psi4/core.so
>>> python ../psi4/run_psi4.py --inplace input.dat

How to set PSIDATADIR and why

PSIDATADIR is an environment variable containing the location of the text resource parts of the PSI4 codebase (e.g., basis sets, databases, EFP fragments). It is for developer use only. In PSI4 1.1 and beyond, the program always knows where its resources are, and the only reason to set this variable is to point to another location. Previously in PSI4 1.0 and previous, only installed executables knew the location, so it always needed to be explicitly set when run from the compilation directory.

At runtime

>>> psi4 -p {top-level-psi4-dir}/psi4/share/psi4

Or in the shell

# csh, tcsh: add to shell or ~/.tcshrc file
setenv PSIDATADIR {top-level-psi4-dir}/psi4/share/psi4

# sh, bash: add to shell or ~/.bashrc (Linux/Windows) or ~/.bash_profile (Mac) file
export PSIDATADIR={top-level-psi4-dir}/psi4/share/psi4

How to configure C++ and C compilers for building Psi4

Role and Dependencies

  • Role — In PSI4, a C++ compiler is vital for building the code.
  • Downstream Dependencies — PSI4 \(\Leftarrow\) C++ Compiler

CMake Variables

  • CMAKE_CXX_COMPILER — CMake variable to specify name or full path to C++ compiler.
  • CMAKE_C_COMPILER — CMake variable to specify name or full path to C compiler.
  • CMAKE_CXX_FLAGS — CMake variable to specify any additional custom compiler flags for C++ source.
  • CMAKE_C_FLAGS — CMake variable to specify any additional custom compiler flags for C source.

Examples

  1. Build with detected compilers from PATH
>>> cmake
  1. Build with specific (Intel) compilers from PATH
>>> cmake -DCMAKE_CXX_COMPILER=icpc -DCMAKE_C_COMPILER=icc
  1. Build with specific (GNU) compilers not in PATH
>>> cmake -DCMAKE_CXX_COMPILER=/path/to/gcc6.2/bin/g++ -DCMAKE_C_COMPILER=/path/to/gcc6.2/bin/gcc
  1. Build with specific (Intel) compilers from PATH based on GCC not in PATH
>>> cmake -DCMAKE_C_COMPILER=icc \
          -DCMAKE_CXX_COMPILER=icpc \
          -DCMAKE_C_FLAGS="-gcc-name=${GCC5}/bin/gcc" \
          -DCMAKE_CXX_FLAGS="-gcc-name=${GCC5}/bin/gcc -gxx-name=${GCC5}/bin/g++"
  1. Build with specific (Intel) compilers from PATH based on GCC not in PATH and also building Fortran Add-Ons
>>> cmake -DCMAKE_C_COMPILER=icc \
          -DCMAKE_CXX_COMPILER=icpc \
          -DCMAKE_Fortran_COMPILER=ifort \
          -DCMAKE_C_FLAGS="-gcc-name=${GCC5}/bin/gcc" \
          -DCMAKE_CXX_FLAGS="-gcc-name=${GCC5}/bin/gcc -gxx-name=${GCC5}/bin/g++" \
          -DCMAKE_Fortran_FLAGS="-gcc-name=${GCC5}/bin/gcc -gxx-name=${GCC5}/bin/g++"

What C and C++ compilers and versions are approved

On Linux, the following work nicely.

  • GNU: gcc, g++
  • Intel: icc, icpc
  • Clang: clang, clang++

On Mac, the following work nicely.

  • Apple Clang: clang, clang++
  • Intel: icc, icpc

PSI4 requires full C++11 compliance, meaning, most importantly, GCC >= 4.9. This compliance is checked for at build-time with file psi4/cmake/custom_cxxstandard.cmake, so either consult that file or try a test build to ensure your compiler is approved. Note that Intel compilers also rely on GCC, so both icpc and gcc versions are checked.

How to obtain C and C++ compilers for Mac without Fink, MacPorts, or Homebrew

The easiest compiler to obtain is clang which is a drop-in replacement for gcc and g++. Just install XCode. Some old versions of XCode can’t handle some of the advanced C++ language features, but this is a software not hardware limitation. Checks for version compliance performed at build-time.

How to satisfy the GCC >= 4.9 requirement on Linux without updating the OS

# See if GCC too old (in this case, yes)
>>> gcc --version
gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-4)

Even if you’re not using GCC as a compiler, your Intel compiler seeks gcc to generate code compatible with your current GCC version. If your GCC is too old (like above), you can update your system GCC through your package manager or install an acceptable version elsewhere on your system. The latter route, tested on Linux with Intel compilers, is below.

# Install GCC 5.2 into a non-primary conda environment
>>> conda create -n gcc52 -c psi4 gcc-5-mp

# To Build, either:

#    (A) activate environment (prepends PATH)
>>> source activate gcc52

#    (B) tell CMake to tell the compiler which GCC to use
>>> vi do-configure
   GCC5=/path/to/miniconda/envs/gcc52
   cmake -H. -Bbuild \
     -DCMAKE_C_COMPILER=icc \
     -DCMAKE_CXX_COMPILER=icpc \
     -DCMAKE_C_FLAGS="-gcc-name=${GCC5}/bin/gcc" \
     -DCMAKE_CXX_FLAGS="-gcc-name=${GCC5}/bin/gcc -gxx-name=${GCC5}/bin/g++" \
     ...
     # if Fortran active ...
     -DCMAKE_Fortran_COMPILER=icpc \
     -DCMAKE_Fortran_FLAGS="-gcc-name=${GCC5}/bin/gcc -gxx-name=${GCC5}/bin/g++" \

# Configure and build

# To Run:
>>> export LD_LIBRARY_PATH=${GCC5}/lib:$LD_LIBRARY_PATH

How to configure a Psi4 build on Cray

Cray systems strongly prefer to build static libraries, but PSI4 needs to be dynamic to function as a Python module. Courtesy of @misha at the forum and various supercomputer guides, building PSI4 on Cray requires setting environment variables CRAYPE_LINK_TYPE and CRAY_ADD_RPATH before running cmake.

1
CRAYPE_LINK_TYPE=dynamic CRAY_ADD_RPATH=yes cmake ...

How to configure Fortran compilers for building Psi4

Role and Dependencies

  • Role — In PSI4, a Fortran compiler in unneeded for core features but may be required for add-ons.
  • Downstream Dependencies
    • PSI4 (\(\Leftarrow\) optional) Fortran Compiler
    • erd, dkh, gdma, PCMSolver \(\Leftarrow\) Fortran Compiler

CMake Variables

  • CMAKE_Fortran_COMPILER — CMake variable to specify name or full path to Fortran compiler.
  • CMAKE_Fortran_FLAGS — CMake variable to specify any additional custom compiler flags for Fortran source.

Examples

  1. Build with detected compiler from PATH
>>> cmake
  1. Build with specific (Intel) compiler from PATH
>>> cmake -DCMAKE_Fortran_COMPILER=ifort

What Fortran compilers are approved

On Linux and Mac, the following work nicely.

  • GNU: gfortran
  • Intel: ifort
  • Packages to install for specific OS or package managers:
    • Ubuntu gfortran
    • conda gcc or gcc-5 to get gfortran

How to obtain a Fortran compiler for Mac without Fink, MacPorts, or Homebrew

Xcode does not provide a Fortran compiler. A way to get one is to download the gcc conda package. This provides gcc, g++, and gfortran compilers for Mac. The two former are 4.8.5 and so are too old to compile PSI4, but the Fortran compiler will work.

How to configure BLAS/LAPACK for building Psi4

Role and Dependencies

  • Role — In PSI4, BLAS and LAPACK control much of the speed and efficiency of the code since computational chemistry is essentially linear algebra on molecular systems.
  • Downstream Dependencies — PSI4 \(\Leftarrow\) LAPACK Libraries

CMake Variables

  • BLAS_TYPE — CMake variable to specify which BLAS libraries to look for among MKL|OPENBLAS|ESSL|ATLAS|ACML|SYSTEM_NATIVE.
  • LAPACK_TYPE — CMake variable to specify which LAPACK libraries to look for among MKL|OPENBLAS|ESSL|ATLAS|ACML|SYSTEM_NATIVE.
  • MKL_ROOT — Environment variable set by Intel compilervars scripts. Sufficient to trigger math detection of MKL at this location.
  • MATH_ROOT — Environment variable to specify root directory in which BLAS/LAPACK libraries should be detected (e.g., ${MATH_ROOT}/lib64/libblas.so and ${MATH_ROOT}/lib64/liblapack.so).
  • LAPACK_LIBRARIES — CMake variable to specify BLAS/LAPACK libraries explicitly, bypassing math detection. Should be semicolon-separated list of full paths.
  • LAPACK_INCLUDE_DIRS — CMake variable to specify BLAS/LAPACK header location explicitly, bypassing math detection. Only needed for MKL.

Examples

  1. Build with any LAPACK in standard location
>>> cmake
  1. Build with native Accelerate LAPACK on Mac (MKL not also present)
>>> cmake
  1. Build with native Accelerate LAPACK on Mac (MKL also present)
>>> cmake -DBLAS_TYPE=SYSTEM_NATIVE -DLAPACK_TYPE=SYSTEM_NATIVE
  1. Build with Intel MKL
>>> source /path/to/intel/vers/linux/mkl/bin/mklvars.sh intel64  # adjust sh/csh and arch as needed
>>> cmake

>>> MATH_ROOT=/path/to/intel/vers/linux/mkl/ cmake
  1. Build with Intel MKL from conda
# won't work, as mkl.h header also needed
  1. OpenBLAS
>>> MATH_ROOT=/path/to/openblas/0.2.13_seq/x86_64/gcc_5.2.0/lib cmake
  1. Build with explicit MKL LAPACK
>>> cmake -DLAPACK_LIBRARIES="/path/to/lib/intel64/libmkl_lapack95_lp64.a;/path/to/lib/intel64/libmkl_rt.so" -DLAPACK_INCLUDE_DIRS="/path/to/mkl-h-include/"
  1. Build with explicit non-MKL LAPACK
>>> cmake -DLAPACK_LIBRARIES="/path/to/lib/liblapack.so;/path/to/lib/libblas.a"

Notes

  • Much of PSI4‘s speed and efficiency depends on the corresponding speed and efficiency of the linked BLAS and LAPACK libraries (especially the former). Consider the following recommendations:
    • It is NOT wise to use the stock BLAS library provided with many Linux distributions like RedHat, as it is usually just the completely unoptimized netlib distribution. The choice of LAPACK is less critical, and so the unoptimized netlib distribution is acceptable.
    • Perhaps the best choice, if available, is Intel’s MKL library, which includes efficient threaded BLAS and LAPACK (as of PSI4 v1.1, earliest known working version is MKL 2013). On Mac, the native Accelerate libraries are also recommended.
    • For open-source LAPACK distributions, OpenBLAS (formerly GotoBLAS) is known to work, while ATLAS is known (https://github.com/psi4/psi4/issues/391) to have stability issues with the DFOCC module.
    • ACML libraries are known to work with PSI4 v1.1 at ACML 6.
  • The BLAS/LAPACK detected for PSI4 are also linked into any Add-Ons (e.g., libefp) that require them, rather than relying on those packages’ native math detection.
  • The separation between BLAS and LAPACK seen in detection printing and CMake variables is purely formal. In practice, they get run together and linked as ${LAPACK_LIBRARIES} ${BLAS_LIBRARIES}.
  • Sometimes the CMake’s library search capabilites falter at SONAMEs (e.g., libblas.so.3 vs. libblas.so), extensions (static vs. dynamic), or suffixes (e.g., libacml_mp.so vs. libacml.so). The developers would be interested in hearing of such problems to expand the math detection capabilities. The immediate solution, however, is to form symlinks between the library names that exist and the names expected. Consult file psi4/cmake/math/MathLibs.cmake for the library patterns being sought.
  • The BLAS/LAPACK interface is standardized, so only libraries, not headers, need to be detected. The exception is MKL, where the mkl.h header defines additional functionality; it must be located to use BLAS threading.

How to configure Python for building Psi4

Role and Dependencies

  • Role — In PSI4, Python allows the core compiled C++ code to be flexibly accessed for manipulation and extension in an interpreted language.
  • Downstream Dependencies — PSI4 \(\Leftarrow\) Python Interpreter

CMake Variables

  • PYTHON_EXECUTABLE — specify name or full path to Python interpreter.
  • PYTHON_LIBRARY — specify path to Python library.
  • PYTHON_INCLUDE_DIR — specify directory of Python headers. Contains Python.h.

Examples

  1. Build with detected Python from PATH
>>> cmake
  1. Build with specific Python
>>> cmake -DPYTHON_EXECUTABLE=/path/to/interp/python2.7
  1. Build with full Python specification to root directory ${PFXC}
>>> cmake -DPYTHON_EXECUTABLE="${PFXC}/bin/python" \
          -DPYTHON_LIBRARY="${PFXC}/lib/libpython3.5m.so" \
          -DPYTHON_INCLUDE_DIR="${PFXC}/include/python3.5m"

What Python is Psi4 running

The Python detected at build-time is embedded into the PSI4 executable. That is, the top line of bin/psi4 is something like #!/path/to/miniconda/envs/p4deps/bin/python3.5, and that’s the Python through which PSI4 is running, rather than the Python of which python. To use a different Python with PSI4 in the short term, just path/to/desired/python psi4 on the command line to override the shebang line. To use a different Python with PSI4 in the long term, edit the shebang line.

If you’re using PSI4 as a Python module, then PSI4 is running the Python of which python.

How to fix “undefined symbol: _Py_FalseStruct“

You’re probably loading a Py3-compiled Psi4 in Py2. Switch interpreters and re-run. A python of proper Py2 or Py3-ness is baked into the PSI4 “executable”, so you’ll see this error only for Psi4 as Python module.

How to use gdb and lldb with Psi4

Debugging PSI4 has gotten a little confusing now that it’s running through Python. Here’s the syntax

1
2
3
>>> cd {objdir}
>>> lldb -- python stage/{prefix}/bin/psi4 ../tests/tu1-h2o-energy/input.dat
>>> (lldb) run

1
2
3
>>> cd {objdir}
>>> gdb --args python stage/{prefix}/bin/psi4 ../tests/tu1-h2o-energy/input.dat
>>> (gdb) run

How to see the actual compiling commands (or errors) with cmake

CMake by default hides a lot of useful debugging information to make the compilation cleaner. Issue make VERBOSE=1 to display the full compilation commands and errors.

How to highlight git merge conflicts in vi

Edit your ~/.vimrc file to include the lines below. Hitting the F7 key will toggle highlighting of git’s conflict markers.

>>> cat ~/.vimrc
set hlsearch
map <F7> :/\(<<<<<<<\\|=======\\|>>>>>>>\)<CR>

How to handle “runtime library may be hidden” when building with Anaconda Python

When building against Ana/Miniconda python (e.g., cmake -DPYTHON_EXECUTABLE=/path/to/conda/bin/python), the warning below often appears. It is harmless, proceed.

CMake Warning at src/bin/psi4/CMakeLists.txt:58 (add_executable):
  Cannot generate a safe runtime search path for target psi4 because files in
  some directories may conflict with libraries in implicit directories:

    runtime library [libm.so.6] in /usr/lib64 may be hidden by files in:
      /theoryfs2/common/software/anaconda/lib

Some of these libraries may not be found correctly.

How to set up the scratch directory

The scratch directory is where Psi4 stores potentially large files during computation. It should thus be on a local, fast disk to minimize any computational inefficiencies caused by I/O. The scratch directory is commonly set up through the PSI_SCRATCH environment variable:

# csh, tcsh: add to shell or ~/.tcshrc file
setenv PSI_SCRATCH /path/to/existing/writable/local-not-network/directory/for/scratch/files

# sh, bash: add to shell or ~/.bashrc (Linux/Windows) or ~/.bash_profile (Mac) file
export PSI_SCRATCH=/path/to/existing/writable/local-not-network/directory/for/scratch/files

See also the more general scratch documentation.

How do I retain specific Psi4 scratch files

You can set up a specific path for PSI4 scratch file and keep them for later use through the psi4_io handler.

How to use Psi4 within a PBS queue

You will usually need to set up a PBS job script that is setting all necessary environment variables, making sure the scratch directories are set up, and invokes the executable. An example PBS script is provided in the manual, but make sure to also consult your own PBS documentation for appropriate setup.

How to update and rebuild Psi4

Obtain code updates as appropriate from Binary Installer, Clone from GitHub Repository, or Fork from GitHub Repository. Move into objdir and reissue make, whereupon CMake may reconfigure but will only rebuild objects and libraries depending on changed files. It is scarcely ever necessary for the user to reinvoke cmake to update objdir.

How to run a minute’s worth of tests

When you want to do a very minimal test of the build and have CTest installed, the following command can be useful.

1
>>> ctest -L smoke -j`getconf _NPROCESSORS_ONLN`

If you have pytest installed, very similar coverage is obtained through:

1
>>> make pytest

How to run a subset of tests

CTest allows flexibly partitioned running of the test suite. In the examples below, testname are regex of test names, and testlabel are regex of labels (e.g., cc, mints, libefp).

  • Run tests in parallel with -j flag. For maximum parallelism: ctest -j`getconf _NPROCESSORS_ONLN`
  • Run full test suite: ctest
  • Run about a third of the tests in 10–20 minutes, the so-called quicktests: ctest -L quick
  • Run the minimal number of tests to ensure Psi4 and any add-ons in working order: ctest -L smoke
  • Run tests matching by name: ctest -R testname
  • Run tests excluding those by name: ctest -E testname
  • Run tests matching by label: ctest -L testlabel
  • Run tests excluding those by label: ctest -LE testlabel

How to see CTest testing errors

1
2
3
4
5
6
7
>>> ctest
Test project /your/path/2/psi4/build/directory/tests
    Start 248: tu1-h2o-energy
1/2 Test #248: tu1-h2o-energy ...................   Passed    1.73 sec
     Start  6: cc1
2/2  Test  #6: cc1 ..............................***Failed    0.07 sec
...

When ctest reports that some (or all) tests have failed, look in your build directory for file objdir/tests/Testing/Temporary/LastTest.log. It may have a .tmp extension, depending on whether the last test was interrupted and a few other factors. Either way, this file should contain CMake’s testing output, as well as everything that was printed to the screen.

How to test a Psi4 installation

ctest requires a connection to source files and cmake machinery and so can only be performed from objdir (staged installation). To test an installed PSI4 (full or staged installation), a limited number of “smoke” tests are available to be run via pytest.

How to refer to Psi4

Ways to refer to PSI4 in text, in order of decreasing goodness:

  • as Psi4 in Optima regular font with “si” in custom small caps according to psi4/media/README.md.
    • html: <span style="font-family: Optima, sans-serif; color: #273896;">P<span style="font-size: 82%;">SI</span>4</span>
  • as Psi4 with “si” in generated small caps
    • html: <span style="font-variant: small-caps;">Psi4</span>
  • as Psi4 with “si” in lowercase
  • as psi4 in code
  • NOT PSI4 or PSI