Molecular dynamics: I have a bad feeling about this.

Computer models of chemical and biological systems are not reality; rather they are what I call “invitations to reality”. They provide guidance to experimentalists to try out certain experiments, test certain techniques. They are suggestive, not factual. However as any good modeler and chagrined experimentalist knows, it’s not hard to mistake models for reality, especially when they look seductive and are replete with bells and whistles.

This was one of the many excellent points that Anthony Nicholls made in his lunch critique of molecular dynamics yesterday at the offices of OpenEye scientific software in Cambridge, MA. In his talks and papers Anthony has offered not just sound technical criticism but also a rare philosophical and historical perspective. He has also emerged as one of the sharpest critics of molecular dynamics in the last few years, so we were all eager to hear what it exactly is about the method that rubs him the wrong way. Many of his friends and colleagues call him ‘Ant’, so that’s what I will do here.

Here’s some background for a general audience: Molecular dynamics (MD) is a computational technique that is used to simulate the motion of atoms and molecules. It is used extensively in all kinds of fields, from biochemistry to materials science. Most MD employed in research is classical MD, based on Newton’s laws of motion. We know that the atomic world is inherently quantum mechanical in nature, but it turns out we can get away to a remarkable extent using classical mechanics as an approximation. Over the last few years user-friendly software and advances in computing hardware have brought MD to the masses, so that even non-specialists can now run MD calculations using brightly colored and accessible graphical user interfaces and desktop computers. A leader in this development is David E. Shaw, creator of the famed D E Shaw hedge fund who has made the admirable decision to spend all his time (and a good deal of his money) developing MD software and hardware for biochemistry and drug discovery.

Ant’s 2-hour talk was very comprehensive and enjoyable, covering several diverse topics including a few crucial ones from the philosophy of science.

It would be too much to describe everything that Ant said and I do hope OpenEye puts the video up on their website. I think it would be most convenient to summarize his main points here.

MD is not a useless technique but it’s not held up to the same standards as other techniques, and therefore its true utility is at best unknown: Over the last few years the modeling community has done a lot of brainstorming about the use of appropriate statistical and benchmarking methods to evaluate computational techniques. Statistical tests have thus emerged for many methods, including docking, shape-based screening, protein-based virtual screening and quantum chemical calculations. Such tests are however manifestly lacking for molecular dynamics. As Ant pointed out, almost all statements in support of MD are anecdotal and uncontrolled. There are almost no follow-up studies.

MD can accomplish in days what other techniques can achieve in seconds or hours: No matter how many computational resources you throw at it, the fact remains (and will likely always remain) that MD is a relatively slow technique. Ant pointed out cases where simpler techniques gave the same results as MD but in much lesser time. I think this reveals a more general caveat; that before looking for complicated explanations for any phenomenon in drug discovery or biology (potency, selectivity, differences in assay behavior etc.), one must look for simple ones. For instance is there a simple physicochemical property like molecular weight, logP, number of rotatable bonds or charge that correlates with the observed effect? If there is one, why run a simulation lasting hours or days to get the same result?

A case in point is the recent Nature paper by D. E. Shaw’s group described by Derek on his blog. Ant brought our attention to the Supporting Information which says that they got the same result for the ligand pose using docking which they got using MD, a difference translating to a simulation time of days vs seconds. In addition they saw a protein pocket expansion in the dynamics simulation whose validity was tested by synthesizing one compound. That they prospectively tested the simulation is a good thing, but one compound? Does that prove that MD is predictive for their system?

MD can look and feel “real” and seductive: This objection really applies to all models which by definition are not real. Sure, they incorporate some elements of reality but they also leave many others out. They simplify, use fudge factors and parameters and often neglect outliers. This is a not a strike against models since they are trying to model some complex reality and they cannot do this without simplification, but it does indicate reasons for being careful when interpreting their results. However I agree that MD is in a special category since it can generate very impressive movies that emerge from simulations run on special purpose machines, supercomputers or GPUs for days or months at a time. Here’s one that looks particularly impressive and denotes a drug molecule successfully “finding” its binding site on a protein.

This apparently awesome power of computing power and graphical software brought to bear on an important problem often makes MD sound way more important than what it is. The really damning thing though may be that shimmering protein on your screen. It’s very easy for non-computational chemists to believe that that is how the proteins in our body actually move. It’s easy to believe that you are actually seeing the physics of protein motion being simulated, down to the level of individual atoms.

But none of this is really true. Like many other molecular models what you are seeing in front of you is a model, replete with approximations and error bars. As Ant pointed out, it’s almost impossible to get real variables like statistical mechanical partition functions, let alone numbers from experiment, out of such simulations. Another thing that’s perpetually forgotten is that in the real world, proteins are not isolated but are tightly clustered together with other proteins, ions, small molecules and a dense blanket of water. Except perhaps for the water (and poorly understood water at that), we are ignoring all of this when we are running the simulation. There are other problems in real systems, like thermal averaging and non-ergodicity which physicists would appreciate. And of course, let’s not even get started on the force fields, the engines at the heart of almost every simulation technique that are consistently shown to be imperfect. No, the picture that you see in a molecular dynamics movie is a shadow of its “real” counterpart, even if there is some agreement with experiment. At the very least this means you should keep your jaw from dropping every time you see such a movie.

Using jargon, movies and the illusion of reality, MD oversells itself to the public and to journals: Ultimately it’s not possible to discuss the science behind MD without alluding to the sociological factors responsible for its perception. The fact is that top journals like Nature or Science are very impressed when they see a simulation shepherded by a team led by Big Name Scientist being run for days using enough computing power to fly a jetfighter. They are even more impressed when they see movies that apparently mirror the actual motion of proteins. Journals are only human, and they cannot be entirely faulted for buying into seductive images. But the unfortunate consequence of this is that MD gets oversold. Because it seems so real, because simulations that are run for days must undoubtedly be serious stuff because they have been run for days, because their results are published in prestigious journals like Nature, therefore it all must be important stuff. This belief is however misplaced.

What’s the take home message here? What was strange in one sense was that although I agreed with almost everything that Ant said, it would not really affect the way I personally use MD in my day-to-day to work, and I suspect this is going to be the case for most sane modelers. For me MD is a tool, just like any other. When it works I use its results, when it doesn’t I move on and use another tool. In addition there are really no other ways to capture protein and ligand motion. I think Ant’s talk is best directed at the high priests of MD and their followers, people who either hype MD or think that it is somehow orders of magnitude better than other modeling techniques. I agree that we should all band together against the exhortations of MD zealots.

I am however in the camp of modelers who have always used MD as an idea generator, a qualitative tool that goads me into constructing hypothesis and making suggestions to experimentalists. After all the goal of the trade I am involved in is not just ideas but products. I do care about scientific rigor and completeness as much as the other person, but the truth is that you won’t get too far in the business I am involved in if you constantly keep worrying about scientific rigor rather than the utility – even if it’s occasional – of the tools we are using. And this applies to theoretical as well as experimental tools; when was the last time my synthetic chemistry friends used a time-tested reaction on a complex natural product and got the answer they expected? If we think MD is anecdotal, we should also admit that most other drug design strategies are anecdotal too. In fact we shouldn’t expect it to be otherwise. In a field where the validity of ideas is always being tested against a notoriously complex biological system whose workings we don’t understand and where the real goal is to get a useful product, even occasional successes are treasured and imperfect methods are constantly embraced.

Nonetheless, in good conscience my heart is in Ant’s camp even if my head protests a bit. The sound practice of science demands that every method be duplicated, extensively validated, compared with other methods, benchmarked and quantified to the best of our abilities if we want to make it part of our standard tool kit. This has manifestly not happened with MD. It’s the only way that we can make such methods predictive. In fact it’s part of a paradigm which as Ant pointed out goes back to the time of Galileo. If a method is not consistently predictive it does not mean it is useless, but it does mean that there is much in it that needs to be refined. Just because it can work even when it’s not quantitative does not mean trying to make it quantitative won’t help. As Ant concluded, this can happen when the community comes together to compare and duplicate results from their simulations, when it devotes resources to performing the kind of simple benchmarking experiments that would help make sense of complicated results, when theorists and experimentalists both work together to achieve the kinds of basic goals that have made science such a successful enterprise for five hundred years.

Can you at least get the solvation energy right?

Basic physical property measurement and prediction is not supported at the granting level and is considered too far from the issues directly affecting drug development to have been pursued by industry. This has left a critical gap in the basic scientific method that drives theoretical methods forward, that is, the observation, hypothesis, and testing methodology that Bacon, al-Haytham, and others championed and that Galileo applied to great effect in the formulative years of modern science...if basic physical science is supported in this area there is great potential for improvement and eventual achievement of long-desired goals of molecular modeling in the pharmaceutical industry- Prescient Soothsayers of Solvation

Sometimes it's a wonder computational predictions of protein and ligand activity work at all. Consider the number of factors we still don't have a good handle on; among other things, calculating protein conformational entropy is virtually beyond reach, calculation of hydrogen bond strengths that depend intimately on the surrounding environment is still quite tricky and calculation of favourable hydrophobic entropy gain because of expulsion of water molecules from the the active site is still a murky area.

But there are things even simpler than these which we have not learnt to calculate well. Foremost among these is a crucial factor influencing every instance of protein ligand binding, the interaction of both assemblies with bulk water. If we can't even get the aqueous solvation energy right, can we make a statement about progress in modeling protein-ligand interactions at all? Water has been probably the most studied solvent for decades and dozens of water models have sprung up, none of which is significantly superior in calculating the properties of this stunningly deceptively simple liquid.

The two foremost implicit methods (as opposed to explicit solvent methods like MD) currently used for calculating solvation energy are the Born solvation method and ones based on the Poisson-Boltzmann equation. Calculating solvation energy ultimately will involve getting the basic science right. With this view in mind, a group from OpenEye and Astra Zeneca narrate their successes and failures in a blind test for calculating solvation energies of 56 druglike organic molecules called SAMPL1. They do a fine job in investigating individual cases and talking about the effect of two crucial variables on the solvation energies; atomic radii (which inversely relate to the solvation) and even more importantly, charges. The group essentially fiddle around with these two variables, modifying the charges and the atomic radii until they get the solvation energy about right. It's a classic case of both the virtues and pitfalls of parametrization and indicates that real parameterization should not involve blindly adding terms to get experimental agreement but instead focus on the two or three scientifically most interesting and important variables.

Believe it or not, but there are a dozen different methods for calculating atomic charges in computational chemistry. Fixed charge models don't capture a very important phenomenon- polarization- that can profoundly affect bond strengths and especially hydrogen bond strengths. In real life charges on atoms don't stay constant in a changing environment. At the same time there is no one "correct" charge model, and as in the case of models in general, what matters ultimately is a model that works. In an earlier blind test, the group had used a particular quantum chemical method called AM1-BCC to calculate charges, and this gave them a mean error of about 2 kcal/mol in the solvation energy. The AM1-BCC method is a well-established semiempirical method that actually calculates slightly overpolarized charges, thus fortuitously and conveniently mimicking the change in charge distribution for a molecule as it transfers from the gas to the aqueous medium. In this paper the group calculate charges at the DFT level and find that this makes a significant difference for a large subset of the previous molecules.

Another interesting phenomenon investigated in the study is the effect of conformations on the calculation of solvation energy. The first axiomatic truth to realize is that molecules exist as several different conformations in both gas and aqueous phases. But low energy conformations for a typical organic molecule in the gas phase will be very different from aqueous conformations. Conformations calculated in the gas phase are typically 'collapsed' and have oppositely charged polar groups too close for comfort because of the lack of intervening solvent that would usually break them up. If you want to use only one conformation for a solvation energy calclation, you would use a collapsed gas phase conformation and a relatively extended aqueous phase conformation. Ideally though you should be more realistic and should use multiple conformations. In the study, the effect of multiple conformations for calculating the vacuum and aqueous phase partition functions and solvation free energy was studied. Interestingly the results obtained with multiple conformations are generally worse than the results obtained with single conformations! There must probably be some added noise that is introduced from unrealistic calculated conformations. The authors also find out, not surprisingly, that using different charges for different conformations of the same molecule can make a difference, although not much. At the same time charges for certain atoms don't change much if the atoms are buried; a failure to realize this leads to two screaming outliers, which however only provides a good opportunity to learn what's wrong.

There are several interesting paragraphs on how the authors played with the atomic radii and the charges and how they explained and were puzzled by outliers. In the end, a particular combination of DFT charges along with a particular combination of radii (termed ZAP10 radii) provided the smallest error in calculation of solvation energies. Interestingly some radii had to be maintained at their default Bondi radii values (which are derived from crystal data) in order to work well.

What I like about this study is that it is told from the real-time viewpoint and illustrates the calculation as it actually evolved. The pitfalls and the possibilities are cogently explored. Certain functional groups and atom types seem to perform better than others. It is clear that much care is devoted to understanding the basic science.

The basic science is also going to involve the accurate experimental determination of solvation energies. Such measurements are typically considered too mundane and basic to be funded. And yet, as the authors make clear in the paragraph quoted at the beginning, it's only such measurements that are going to aid the calculation of aqueous solvation energies. And these calculations are going to be ultimately key to calculating drug-protein interactions. After all, if you cannot even get the solvation energy right...

Nicholls, A., Wlodek, S., & Grant, J. (2009). The SAMP1 Solvation Challenge: Further Lessons Regarding the Pitfalls of Parametrization The Journal of Physical Chemistry B, 113 (14), 4521-4532 DOI: 10.1021/jp806855q

A first-class mental workout

I am back from the CUP X OpenEye conference in Santa Fe. Of all the conferences I go to, none is more intellectually stimulating, and very few have the same quality of food for thought. I have also not been to another conference where people grapple with such fundamental scientific problems; shape, electrostatics, statistics, dipole moments, force fields, tautomers. It's a treat for the brain, and it's just the shot of intellectual energy that I need to re-energize myself for doing and discussing science. Plus, I get to visit Santa Fe whose mountains can inspire even the most muddle-headed scientific thinker to come up with at least a few inspired ideas. As Linus Pauling said, first you need to have lots of ideas, then throw the bad ones away.

The cast of characters this year was delicious since they had invited keynote speakers from all nine previous CUPs back. You therefore got to hear about a smorgasbord of topics from folks like Barry Honig, Vijay Pande, Tack Kuntz, Ajay Jain, Paul Labute, Anthony Nicholls, Chris Bayly, Yvonne Martin and many more. The topics ranged all over the place, the humor flowed abundantly, but the focus was always on the basic science. In computational chemistry (or finance or physics or biology for that matter...) we build so many models, yet how many of them reflect true understanding of the underlying physical basis? Simply adding parameters can make a model fit the data, yet how many of us would be nonchalant about using it for new prediction? As von Neumann said, with enough parameters we can indeed fit elephants on a curve. Yet who knows if we would be able to fit all those wondrous creatures that currently exist only in our imagination?

Scientists at CUP X grappled with these issues with infinite concern and zeal. They asked questions like; Can we say we can predict if we can't even get the dipole moment right? Would force fields ever reach the golden standard? Can we predict which tautomer of a molecule will bind to a protein? Can we make quantitative calculations of thermodynamic quantities that we can compare to accurate quantities obtained from ITC data? How can we predict solvation energies? What biases do we have in modeling? Can we get rid of them? And then, how can we ensure only the most rigorous standards for the experimental data itself? As someone indicated, a PDB or CSD structure of a molecule that you see on a screen is not the data, it is only a model of the data. And finally, an eternal question; can quantum mechanics get us to heaven?

If you are any kind of chemist concerned about and connected with building models of chemical and biochemical reality, I would strongly urge you to attend the OpenEye conference, held every year in March in the Land of Enchantment. Registration is free, a few meals are provided, alcohol splashes around with abandon during the poster sessions, and the conference is usually in a nice downtown Santa Fe hotel (the elegant and spacious El Dorado in this case), deals for which are usually cheap if done early. I am going to be here, if possible, every single year that I can. Being here reminds me of a reviewer's assessment of Douglas Hofstadter's magnificent "Gödel, Escher, Bach": It is like having a first-class mental workout in one of the finest intellectual gyms around

Southwestern interlude

"Prediction is very difficult...especially about the future" - Niels Bohr

Finally back from The Land of Enchantment. The guys at OpenEye are awesome and immensely smart. I learnt more in this mentally exhausting conference than in any until now. There was much modeling, crystallography, screening, mathematical similarity, statistics and whatnot. In the past few weeks, I have had to deal with previously arcane (to me) statistical concepts like ROC curves which are fascinating. The conference indicated that there is much success we still have by chance, and one of the best ways to tackle molecular complexity in all its forms is by starting simple, for example in trying to predict solvation energies for simple organic molecules, still a highly challenging endeavor. Prediction is difficult indeed, especially about the simple things.

On the whole, New Mexico is indeed very special (and those who say that only Santa Fe is nice haven't really been around Albuquerque). I lived in a splendid studio apartment in a lively and charming old-age home, ate enough green chile to last a lifetime, visited the National Atomic Museum, admired turkey quilts in the Pueblo Indian museum, kicked around dust in the Santa Fe Plaza, bought a Sidney Harris cartoon t-shirt, and got a free one from OpenEye with the quote above, a favourite.

The air is thin, dry and crystal clear. One of the consequences of this is that...let me just say that a little alcohol goes a long way, especially for someone who hardly ever drinks. The sun is brighter and lights up your thoughts. The miles upon miles of pinon-covered hillocks and the snow-capped Sandia and Sangre de Cristo mountains in the distance are marvelous. The ski slopes of Santa Fe make you happily giddy. The landscape is like nothing I had seen before, and it seems to be wholesome for arty, existentialists types. Robert Oppenheimer once said that his two great loves were physics and desert country. He thought it was a pity they could not be combined, until he discovered Los Alamos. New Mexico seems to provide one of the few avenues for such a heady combination.

SAMPLing out in the desert

I am in the absolutely charming Santa Fe, NM for the CUP IX OpenEye meeting. OpenEye's ROCS and related programs have become quite popular with the modeling and drug discovery community in the last couple of years. This year, as I had written about before, they have a challenge named SAMPL for all groups across the country involved in drug discovery modeling- provided with a few ligands and targets, do virtual screening, pose prediction and binding affinity prediction. It was a great opportunity for our small group of four grad students and postdocs to take part in the challenge.

The OpenEye meeting is always a spirited, extremely informal and funny meeting. The titles of talks makes it clear what kind of atmosphere enlivens the event. There are some really smart and interesting people here, heavily involved with all aspects of modeling and drug discovery. The Hotel La Posada, with its quaint spread-out low-lying adobe hut-like rooms adds to the charm.

The talks have been enlightening, and especially today's talks on crystallography and structure were very good. The most useful were those which focused on errors in crystal structures and crystallographic models- and there are a lot of them out there. The most terrific talk was by Gerhard Kleywegt from Uppsala. He talked about the myriad number of errors existing in crystal structures in the PDB, including misplaced ligands in little or no electron density (protein crystallographers can be especially negligent about ligand fitting, which unfortunately is of greatest interest to medicinal chemists), neglect of water molecules, incorrect conformations, and of course the rash of strained structures, poor resolutions and bad B-factors and R-factors. There are programs such as Afitt which can refine these structures. But a lot of people use crystal structures as they are and don't take account of such inadequacies. Some crystal structures are completely wrong, and yet exist in the PDB. Kleywegt's talk was very funny and informative, and gave a very good reason for why anyone who uses PDB crystal structures should be more than cautious in using them as they are.

Tomorrow there are some great talks on lead optimization and statistical techniques in methods evaluation. And Wednesday is going to be devoted to a brainstorming session on the SAMPL challenge. It would be very interesting to compare our own work with that of others. All in all, it is getting to be a refreshing scientific experience.

Otherwise the city of Santa Fe is very scenic and historical. Buildings have to be built to certain historical Spanish-Indian construction guidelines. And just a short walk from our hotel is 109 East Palace, which was the Manhattan Project's front office, where new arrivals were "inducted" and passes were issued to them so that they could travel to the secret city of Los Alamos, a place that did not exist on the map. Today the room is just a shop with no inkling of what went on there. Little do people strolling around know the great men that passed through those doors and walked that street- Oppenheimer, Fermi, Bethe, Teller, Frisch, Bohr...the list goes on.

The OpenEye SAMPL challenge

Finally something exciting. Me and some colleagues are taking part in the SAMPL (Statistical Assessment of the Modeling of Proteins and Ligands) challenge issued by OpenEye Software for their upcoming annual March meeting in Santa Fe, NM. OpenEye is well known for their ligand-based similarity searching tools that have proven to be superior to many others for virtual screening. I am looking forward both to visiting the state- a dream I have had since I was a kid- and working on the challenge.

The challenge basically is to perform the kinds of procedures to find and rank actives that are now a standard part of modeling in the pharmaceutical industry and elsewhere. The company will hand out three sets of data with small differences between them. Every set will have a couple of thousand ligands, with actives and lots of decoys mixed in with them. Sometimes a protein structure for the ligands might be thrown in. The goals are well-established and standard:

1. Virtual screening: find the actives, identify the decoys.
2. Crystallographic pose determination: find the correct crystallographic conformation for a few ligands in the active site
3. Estimating binding affinity: the hardest task, probably the holy grail of the industry. What more could we want if we could correctly rank order compounds beforehand in a project and estimate their binding affinity?

Literature searching is discouraged. The honor system is in effect. You can use whatever tools you can access. Participants in the challenge include many well-known academic groups as well as people from both Big Pharma and "Small" Pharma. Depending on the data set, we can choose all three or a subset of the above protocols as a challenge. Once we finish one set, we submit the results before a deadline and the next set will be released to us. The goal is not to win: in fact it's a win-win situation because we will always end up learning something interesting. Valuable lessons inevitably learned will include ligand preparation, docking, solvation energy estimation, and other aspects of both ligand-based and structure-based design. In this case, the goal is to see and analyze how people throughout the country can tackle some standard issues in early-stage drug discovery.

This should be fruitful and fun.