In the process of mastering the use of Density Functional Theory (DFT) on GAMESS, I’ve also been playing with a tool called Multi-configuration self-consistent field (MCSCF). One of the big problems with Hartree-Fock (HF) is that it determines molecular wave equations without taking account of a particular quantum mechanical interaction that has come to be called Correlation Energy. DFT and MCSCF attempt to do this.
The idea of correlation energy is that electrons present in a molecular system can enjoy reduced energy by experiencing correlations with electrons of opposing spins. This interaction is roughly the equivalent of exchange energy experienced between electrons of the same spin, where electrons swap identities with each other. DFT has become popular because it performs an HF-like calculation at relatively little computational expense that includes correlation energy directly. DFT, of course, suffers from the problem of being insensitive to soft interactions, like dispersion, which can be important in the diffuse part of the wave function and play a role in soft materials and distant molecular interactions. A neighboring technique which suffers less with this problem is an ab initio post-HF technique called configuration interaction (CI), which uses variation with full determinant configurations to help recover some of the correlation energy. The full determinant configurations are full wave functions composed of molecular orbitals, which are then populated with electrons in various combinations, and then shifted by variation of a coefficient to minimize the energy of the combination. MCSCF takes CI and goes a step further; it optimizes the CI coefficient, then updates the molecular orbitals, then optimizes the CI coefficient with the updated orbitals. As such, MCSCF has greater variational flexibility than CI and can recover the correlation energy more effectively if performed correctly.
On the flip side, MCSCF and full CI are profoundly costly to execute. Given a basis set of N members, if HF goes as N^4 in computation time, CI and MCSCF can go as N^5 or more. MCSCF tries to strike a balance by reducing its variational load to only include orbitals that are important to a chemical question of interest: you target a so-called “active space,” by assuming that only a limited number of electrons and molecular orbitals participate in whatever you happen to be observing. Unfortunately, this can be initially a difficult consideration because the so-called “symmetry adapted” molecular orbitals which emerge from HF tend to be spread out across the molecule in ways that make them totally uninterpretable. It’s really hard to pick “useful” orbitals since you can’t tell whether or not they should be useful.
This comes back to my little argument about the Organic Chemistry Lie. Molecular orbitals reflecting actual energy eigenstates of a molecule do not have logical forms: there are no sigma- or pi- electron bond structures that actually exist unless the symmetry of the bond fits with the symmetry of the molecule. When you look at the molecular orbitals of water, for instance, all your freshman chemistry intuition goes totally out the window because there are no actual lone pairs. The orbitals simply do not look like that. As it turns out, Quantum Chemists have had fits since the 1960s about how molecular orbital theory wrecked the nice neat picture provided to us by Pauling and Slater and others in the 1930s. One way to get around this is proposed somewhat presciently by Vladimir Fock in the 1930s. As it turns out, the molecular wave equation is invariant to unitary transformations within the molecular orbital space… which is just to say that any particular molecular orbital basis is non-unique. Another molecular orbital basis is just as good if it covers the same overall electron density distribution since the actual eigenstates of energy can be rewritten as linear combinations of this hypothetical other basis. The unitary transform is then simply a way to map the actual energy eigenstates onto another basis set of our choice.
To talk about this, I will use the hydronium-water interaction as an example. You will recall the following image from this post.
The only thing that you can really gather from this is that the proton is being shared between the two waters in an electron density flux tube.
Of the 15 orbitals produced in the calculation, 10 are doubly occupied. 3 turn out to be important to the chemistry problem and they would be difficult to pick out without some thought; seen here are orbitals 5, 8 and 15. The first is a bonding orbital and the other two are either antibonding or some weird mixture; two electrons are present in the first, two in the second and none in the last.
A tool used to figure out important orbital combinations to run MCSCF is a technique called “Localization.” Localization was invented in the 1960s in order to create molecular orbital basis sets that make sense. This is simply creating linear combinations of molecular orbitals in order to fit some criterion of choice. If that criterion happens to be “make the orbitals seem localized and sensible,” so be it! The basis can work.
Here are the sequence of localized orbitals using what is called the “Boys” method, after S.F. Boys (the same guy who suggested using Gaussian functions as a computationally efficient basis).
What!? Look at what’s emerged! Here are two lone pairs and an antibonding orbital! The hydronium-water interaction has suddenly gained a very interesting and explicable twist. Keep in mind, though, that these orbitals are purely construct at this point, they do not necessarily reflect any eigenstates.
The next step is to use this basis as an active space in the MCSCF calculation to produce another series of molecular orbitals. The MCSCF energy decreases the HF energy somewhat and produces two pairs of orbitals, the so-called MCSCF Natural Orbitals, which are merely a basis set:
These are basis functions that have been further optimized by the MCSCF calculation and they are subtly different from the localized functions above. The actual energy eigenstates are then constructed from linear combinations of these. Further, the final determinants in the MCSCF wave function have fractional occupations of these orbitals, with 1.99 and 1.98 electrons in the two bonding orbitals and 0.013 electrons in the antibonding orbital. This can no longer be considered a closed-shell case, but the importance is actually the configuration, which would suggest that the wave function has most shells closed with two electrons, but sometimes promotes an electron from one or the other on the right into the antibonding orbital on the left.
MCSCF then gives back optimized HOMO and LUMO orbitals:
From the MCSCF calculation, the hydronium story now becomes very much more nuanced. In the version now, what you could say is that both waters are sticking to the proton, each using one set of their lone electron pairs. They then constitute a sigma-like bonding orbital in HOMO -1 and two forms of sigma antibonding orbitals between HOMO and LUMO. Electrons are now smeared across all three, mostly in HOMO and HOMO -1, but also sometimes in LUMO. (All if I’ve interpreted the MCSCF correctly, which I’m cautious to say I haven’t quite mastered.)