Read about the talk that stole the show at a recent ME/CFS conference in Simon McGrath’s two-part blog. Here’s Part 1 …
Last November, science star Professor George Davey Smith gave a talk at the UK CFS/ME Research Collaborative (CMRC) Annual Science Conference that focused on bigger, better, smarter approaches to research.
Since then, Davey Smith has said he’s keen to play a role in the largest set of studies ever proposed for ME/CFS: Professor Stephen Holgate’s Grand Challenge, which is now moving forward.
The plans are for a ‘big data’ study using a huge cohort that could be 10,000 patients strong. The total budget for this project is likely to be well over £5 million.
While not all of the points he made in his November talk are currently applicable to ME/CFS research, Davey Smith outlined the power and pitfalls of the types of genomics and other big-data approaches that the Grand Challenge is now planning to apply to ME/CFS research.
He showed how, surprisingly, large-scale genetic data can also be used to study non-genetic potential causes of illness, such as the link between vitamins and heart disease. Throughout, he emphasised how ingenious and rigorous approaches can tease out the true causes of disease from misleading, empty clues. His remarks also showed the mind of a top-notch scientist at work, one who has been playing in the biggest of scientific sandboxes.
Davey Smith doesn’t have a high public profile, but he’s published over one thousand studies, and has a huge reputation among fellow scientists. Few researchers are as widely cited by their peers: he has a citation h-index of 150, which ranks him alongside the top life scientists in the world. Much of his work has focused on better ways of doing science, and he’s also played a major role in many large population studies.
Davey Smith’s pioneering approaches and studies have done a great deal to sort out what’s good advice to help you live longer and in better health (sadly even low-level alcohol drinking is harmful) and what’s not (certain vitamins can reduce the risk of heart problems).
Off-beat scientist
Davey Smith gives the firm impression of being entertained by life. The close-cropped hair, jeans and T-shirt mark him out as a little different, not to mention the liberal use of cartoons and jokes in his presentation — and few others would include slides of their favourite take-away restaurant in a conference talk, as he did.
When Davey Smith was a medical student in Cambridge, he bunked off for a cycling holiday with his girlfriend while he was supposed to be learning about epidemiology.
He got back just in time for a tutorial on epidemiology from his mates in the pub, before excelling in the exam the next day. Perhaps that’s why he was captured by the subject, which studies the pattern of disease: who gets ill, when, where, and most importantly, why.
Don’t be fooled by the relaxed attitude, though. Davey Smith is one of the most highly-rated life scientists in the world.
He doesn’t know the disease – but he does know good research
He began his CMRC talk by showing a photo of a boy in a dunce’s hat and declaring, ‘I know nothing about this illness’. But he does know about genomics research, and on that basis, he started off in a typical style for him: breaking a few eggs in pursuit of the right answer.
He showed four recent ME/CFS studies identifying links between differences in some genes and the illness. Davey Smith was not impressed.
‘The statistical power [to detect a real effect] is literally zero’, he said. These studies were simply too small to show anything, and any apparent findings are effectively guaranteed to be false positives — that is, associations simply happening by chance.
‘ME/CFS gene studies today’, said Davey Smith, ‘appear much like other gene association studies a decade ago — hopelessly unreliable.’
Davey Smith and his colleagues helped to dramatically reduce such problems more than a decade ago. They wrote a paper for The Lancet ‘that didn’t make us enormously popular’, pointing out that almost all published association studies up until 2002, including those published in The Lancet, were proving unreliable.
They argued that the false associations were showing up primarily because of publication bias (only studies that found an association got published, while those that did not were ignored), way-too-small sample sizes, and poor statistical techniques.
As a result of that 2003 paper, things changed. Funders such as the Wellcome Trust took the lead and refused to finance further studies unless researchers collaborated to create studies that were big enough to give reliable results.
The upshot was that huge numbers of genetic variants discovered since 2005 have stood the test of time, while almost all of the associations people found before ‘have now just gone’.
Researchers can now search through more than 10 thousand robustly established genetic associations with disease, including obesity, diabetes and heart disease. But currently there are none for ME/CFS.
The Grand Challenge is poised to change that.
Davey Smith then revealed how smart genomics-based approaches can even identify how non-genetic factors, such as diet, do (or don’t) cause disease — overcoming problems faced by more traditional techniques.
The biggest problem: Correlation is not causation!
If you judge by the nutritional advice you read in the newspapers, you might conclude that scientists don’t know much. For example, Davy Smith pointed to these two headlines about the impact of eggs on diabetes:
Eating eggs raises the risk of diabetes — Daily Mail
Eat eggs to slash risk of diabetes — Daily Express
The problem here isn’t just confused journalists — it’s confused research. And Davey Smith explained that often it’s because scientists have forgotten one of the great mantras of science: Correlation is Not Causation.
Correlation simply means that when A happens, B tends to happen as well. That’s a first step to show that A causes B — but sometimes, it’s a step on the path to a dead end.
Challenge 1: Getting Confused by Confounding
Davey Smith showed one way that scientists can mistake correlation for causation, by focusing the example of the supposed link between vitamin E and coronary heart disease.
Numerous studies had shown that people who took more vitamin E supplements, as well as those who actually had more vitamin E in their blood, were less likely to develop coronary heart disease. And the effect was potentially enormous — 40% less in one study! Vitamin E appeared to be a fantastic, inexpensive way to help control humanity’s biggest killer.
Researchers took one additional step to confirm their finding: they ran science’s ‘gold standard’ of randomised controlled trials. Previous studies had been ‘observational’: researchers had simply observed that people who chose to take vitamin E supplements were less likely to develop coronary heart disease.
In randomized controlled trials, on the other hand, researchers randomly put people in two groups, and then they gave one group a vitamin E supplement and the other a placebo. But they found that the two groups fared just the same. Vitamin E had no effect on the rate of coronary heart disease.
How can that be?
Simple confounding example
Coffee drinkers tend to have higher rates of pancreatic cancer than non-coffee drinkers, but the coffee itself has nothing to do with it.
It turns out that coffee drinkers are more likely to smoke — and smoking indeed causes cancer.
So smoking is called a ‘confounder’ for the relationship between coffee and pancreatic cancer.
(From Mann & Wood:
Davey Smith and others argued that the problem was that people who chose to take vitamin E supplements were also more likely to take exercise, smoke less and have a low-fat diet, for instance. And while vitamin E itself has no impact on heart disease — exercise, smoking and diet do. Vitamin E was just a passenger along for the ride, a buddy of the real drivers of health changes.
So Vitamin E is correlated with increased exercise, decreased smoking, and a better diet — and it’s also correlated with reduced heart disease. But vitamin E doesn’t cause reduced heart disease. The hidden, true causes that confuse matters — in this case, exercise, smoking and diet (amongst others) — are called ‘confounding factors’.
Challenge 2: The Curious Problem of ‘Reverse Causation’
A second problem in observational studies is ‘reverse causation’. Consider the bizarre finding that ex-smokers are more likely to die of the lung disease emphysema than smokers. Could it really be that quitting smoking increases your risk of emphysema?
No. This finding only shows a correlation between quitting smoking and emphysema, not causation. What’s going on instead is that smokers become ex-smokers when they’ve been diagnosed with emphysema. Thus the causation is running in the opposite direction. The emphysema is causing the quitting, rather than the quitting causing the emphysema.
In this case, the problem isn’t so hard to spot. But it has led to lots of false findings.
Solution: Nature’s own randomized trials
Randomized controlled trials are the very best way to solve these problems, but they come with their own difficulties. They are terribly slow and expensive.
Suppose, for example, you want to test the strong observational finding that vitamin C lowers the risk of heart disease. To do so with a randomized controlled trial, you’d need to recruit many thousands of people, give half of them vitamin C and half of them a placebo, and then track them over many years to see how many in each group developed heart disease.
A clever new method called ‘Mendelian randomization’ offers many of the benefits of a randomized control trial, and it generally does so far faster and more cheaply. The key is that it lets nature do the ‘randomization.’
Davey Smith, who has championed, developed and used Mendelian randomization to great effect, explained how it works using vitamin C, which, like vitamin E, seemed to reduce the risk of coronary heart disease.
Mendelian randomization is possible because gene differences help predict vitamin C levels. Even if two people consume the same amount of the vitamin, their blood levels of the vitamin may be significantly different — because differences in their genes may make them more or less effective at absorbing vitamin C from their gut.
Crucially, which genes you are born with are, of course, unrelated to potential confounding factors such as exercise, smoking, diet, and income level later in life.
Just as important, if you sort people by the genes for vitamin C absorption, a whole bunch of other genes won’t tag along for the ride (which would introduce new confounding problems). That is how the method gets its odd name — from Gregor Mendel, the father of genetics, who showed that, in almost all situations, genes are inherited independently.
This leads to a new way to study the impact of vitamin C on heart disease.
Test the genes of thousands of people and use gene differences to sort them into lower vitamin C and higher vitamin C groups. (Testing shows that groups do indeed, on average, have higher and lower levels of vitamin C, as predicted). Then you can simply check to see how many people in each group have developed heart disease.
It’s a dream setup, a better way for testing the effect of any factor where there are natural variations due to the genes we inherit, and one that avoids the pitfalls of reverse causation and confounding.
Such genetic testing has become remarkably inexpensive. And even better, large databases of such genetic information already exist, ready for scientists to analyse with no additional cost to collect the data.
Researchers used a Mendelian randomization with 100,000 people — and found that vitamin C had no effect on heart disease. Heart disease is such a big killer (and vitamin C such a cheap and promising treatment) that expensive randomized controlled trials were done as well, and they came to the same conclusion.
So, as for vitamin E — another antioxidant — good evidence suggests that vitamin C has fewer health benefits than previously thought:
Mendelian randomization champions vitamin D for multiple sclerosis
For a long time researchers suspected that vitamin D played a role in multiple sclerosis, not least because of the ‘sunlight effect.’ We need sunlight on our skin to make vitamin D, so people who live closer to the equator, and hence get more sun, tend to have higher vitamin D levels.
Studies had also shown that the closer people lived to the equator as children, the lower their risk of multiple sclerosis.
But such studies only show correlation, not causation. And for Davey Smith, alarm bells went off: ‘I was very sceptical about a causal association’, he said. His guess was that the finding was confounded by many other differences between those who grow up near the equator and those who live nearer the poles.
But Davey Smith was part of a group who ran a large Mendelian randomization for vitamin D in MS. They used genetic variations linked to the production and degradation of vitamin D to sort patients into those with higher and lower levels of the vitamin. And, in fact, the group found that higher vitamin D levels do significantly and substantially reduce the risk of multiple sclerosis.
The multiple sclerosis study highlights another big benefit of Mendelian randomization: researchers can often use existing data from previous genomic studies, and as in this case, don’t need to recruit new patients.
The market votes for Mendelian randomization
A few weeks before Davey Smith gave his talk, shares of the pharmaceutical giant Eli Lilly plunged 8 percent as it pulled the plug on a drug that aimed to reduce heart attacks by boosting ‘good cholesterol’ (high density lipoprotein).
The failure is expected to cost Eli Lilly $90 million. Two other pharmaceutical companies have expensive failures of similar drugs, and another paid $300 million just for the rights to another similar drug.
All these firms were pursuing bets based on observational data that more ‘good cholesterol’ was linked to fewer heart attacks. Yet Mendelian randomization studies, free of confounders and risk of reverse causation, had shown no benefit for good cholesterol.
Similar expensive bets on drugs targeting C-reactive protein failed, too. Mendelian randomization was right in this case too, while the observational findings were wrong.
Not surprisingly, Mendelian randomization is now of great interest to Big Pharma, and I suspect Davey Smith is getting a lot of requests to do consultancy in this field.
What large-scale genetic studies can do for ME/CFS
ME/CFS is unlikely to have the types of environmental causes that Mendelian randomization can easily detect. But gene association studies could identify differences in genes that increase the chance of developing ME/CFS, giving clues about what causes the illness.
In addition, gene association studies can identify genetic variations that influence getting better or worse in people who already have a disease. This calls for a second type of study. Instead of comparing healthy people to those with the disease, it would instead track a large cohort of patients and look at the genes (or other factors such as diet) associated with relapse or remission over time.
Omics is nothing without huge samples
But Davey Smith stressed that we learn ‘nothing at all if the sample size is too small’. Many thousands are needed as a minimum. The Grand Challenge, aiming to collect data from 10,000 patients, is the first time such a thing has been attempted for this disease.
The first big meeting to thrash out the details of how the Grand Challenge will work, in preparation for the all-important grant application, comes in April. The Wellcome Trust, which has grants tailored to this kind of project, has made it clear that they are very open to an application on ME/CFS.
With its huge cohort and rigorous approach, the Grand Challenge aims to bring in top talent from other fields to work on ME/CFS for the first time. Davey Smith signing up would be the perfect way to start.
The Grand Challenge will consider a whole range of omics techniques, including large-scale gene expression and epigenetic studies, which GDS covers in part 2.
Disclaimer: This two-part blog is my take on Professor Davey Smith’s excellent talk, based on the YouTube video. The official summary of his talk, written by Emily Beardall with Action for ME, and approved by Davey Smith, is available to download here.
Simon McGrath tweets on ME/CFS research:
[donate_message]