Plain Language Summary
By reading this article, you will:
- Understand what happens behind the scenes in keto science.
- Get greater insight into how studies get funded.
- Learn to critically question research without relying on black/white thinking.
What we did, and why
In part 1 of our ketogenic diet series, we collected and analyzed a large amount of ketogenic diet studies. On closer inspection, we discovered that many of the central keto researchers had direct connections to the keto industry. We also discovered that their conflicts of interest are not always disclosed.
Since transparency is very important for scientific integrity, we decided to take a closer look at the keto scientists, their books, companies, products, and funding sources.
This is the first analysis of its kind.
Funding sources and conflicts of interest
A conflict of interest (COI) means that a scientist may have something to gain by making a claim or by conducting a study. A common conflict of interest is who the scientist gets paid by (funding source), but it can be other things too.
Many people question research that has been funded by large organizations or industries (i.e. “Big pharma”, “Big dairy”, etc.). You’ve probably heard people say that industry funded research is biased (funding bias) and can’t be trusted. This claim has some scientific backing, but it varies from industry to industry.
Still, a funding source or conflict of interest does not automatically imply that the researchers are biased. This means that we can’t discount a ketogenic diet study just because the scientists are affiliated with keto companies. If we want to figure out if a ketogenic diet study is biased, we have to critically engage with:
- The study methodology
- How the study connects to the rest of the literature (is it an outlier?)
- The author backgrounds (business interests, personal opinions, values)
- Funding sources
- How the data was analyzed and interpreted
Only after analyzing these points can we make a decision about bias.
Central keto companies
There are two companies that connect to several of the biggest keto researchers. They are Virta Health and Prüvit. They sell products, services, employ keto scientists, and fund research.
Keto scientists and their COIs
- Researchers from all sides of the keto debate have COIs such as books, products they sell, company affiliations, industry funding, and more. This includes researchers who are not pro-keto.
- On the pro-keto side, many studies are conducted by central groups of keto scientists. Many of these researchers have industry connections and business interests such as selling keto books, keto supplements, and consulting services). In many cases, these conflicts of interest are not disclosed. The connections are illustrated below:
There are many conflicts of interest in keto research, but we cannot and should not discredit keto research on this observation alone. Having a conflict of interest does not invalidate research, it only warrants increased skepticism and critical analysis of the studies themselves.
There isn’t a simple answer to the issues presented in our article, but we encourage you to be aware of the conflicts of interest, study limitations, and funding sources.
You have now read through our short summary. But, you still don’t know the details about:
- Keto companies such as Pruvit and Virta Health
- The 15 keto scientists we’ve looked into
- Why it is so hard for scientists to get funding
- What increases or decreases the risk of bias
Keep on reading, because the full investigation continues below.
The Ketogenic Diet Series
Do you want to learn more about fitness and nutrition?
Sign up for the newsletter.
Central Keto Companies
Below you can see the connections between keto scientists and companies. We’ve only included companies with direct links to keto scientists.
Central keto companies
Owned by: Phinney, Volek, and Inkinen.
Associated researchers: Phinney, Volek, Wood
Virta Health launched in early 2017 with a mission to reverse type 2 diabetes. Virta Health was founded by Sami Inkinen, Dr. Stephen Phinney, and Dr. Jeff Volek using ~$37 million possibly invested by Inkinen himself after his experimentation with the ketogenic diet to treat his self-diagnosed prediabetes. The goal of Virta is to use personalized nutrition, coaching, and remote monitoring to reverse type 2 diabetes without medication.
Quote from CEO Sami Inkinen:
“Virta draws on the latest medical and nutrition science and advances in technology to address the cause of Type 2 diabetes, and deliver on-demand, personalized treatment and expert support on a near real-time basis.”
Virta Health has already collaborated with Indiana University to run one clinical trial using a ketogenic diet (McKenzie et al., 2017). They are also paired with Purdue University to offer their services to employees with type 2 diabetes. To see an update on their recent activities click here.
Sells: Ketone supplements. The flagship product is Keto//OS (Ketone Operating System).
According to the Ketogenic supplement reviews, Pruvit launched in 2012. They are located in Melissa, Texas where they filed as a domestic for-profit corporation in 2014. They are not accredited by the Better Business Bureau and have an F rating, although there are only a few reviews.
The Prüvit statement of policies and procedures can be found here.
Their compensation plan can be found is here.
Prüvit claim that they are doing keto research.
They recruit people as promoters of their products, similar to multilevel marketing:
“The Pruvit Compensation Plan is an exciting opportunity that rewards you for selling products and services and for sponsoring other participants who do the same. Although the opportunity is unlimited, individual results will vary depending on commitment levels and sales skills of each participant.” - Pruvit income disclaimer
Connection between Pruvit and keto researchers:
Noteworthy keto websites
Owned by: Not stated (identity hidden during website registration)
Associated researchers: Jacob Wilson, Ryan Lowery, and Rachel Gregory
Business affiliations: Markets The Ketogenic Bible
My Keto Recipes
Owned by: Austin Miller (Independent Pruver)
Business affiliations: Affiliated with Pruvit
Keto Scientists and their COIs
How we chose which scientists to investigate
Choosing scientists is a sensitive topic, because it is easy to single out specific scientists. On one hand, we can choose to only look at pro-keto researchers, in which case the analysis becomes biased because we are only critical of one side of the debate. On the other hand, if we include all scientists who have ever done keto research, we will have over a hundred people to investigate. The analysis would thus become muddied, unfocused, and filled with noise.
We’ve chosen a different approach - we’ve looked at the most prominent and/or outspoken authors from both sides of the keto debate. By “keto debate”, we are referring to the discussions on how ketogenic diets compare to other diets for losing body fat, building muscle, improving health, etc. We’ve also chosen researchers that have direct financial links to the industry, such as selling books about keto or being affiliated with keto companies.
There isn’t a sizable camp of “anti-keto researchers”, but there is a camp that favours the calories-in-calories-out model rather than the proposed metabolic advantage of keto.
However, most CICO scientists have not done keto research. This has lead to us looking into more “pro-keto” scientists than “pro-CICO” scientists. Why? Because there is a natural imbalance in the literature - there are many more outspoken keto promoting scientists, who have also done keto research. We may do a COI analysis for the CICO literature in the future.
If you feel like we’ve missed central pro-keto or pro-CICO researchers in our analysis, please send us an e-mail. It is required that the researcher in question has done keto related research and has made statements about keto or CICO.
Overview of keto scientists
Note that the descriptions below, such as “promotes keto”, “pro-keto” “pro-CICO” are not meant to be judgements, but descriptors. Pro-keto implies that an author is for the diet, but does not necessarily actively promote it.
|Conflicts of interest (COIs)||COI reporting|
|Volek||Promotes keto||Has multiple COIs (keto books, consulting, business interests, etc.)||Reports some COIs in some studies.|
|Phinney||Promotes keto||Has multiple COIs (books, business interests ...)||Reports some COIs in some studies.|
|Noakes||Promotes keto||Has multiple COIs (books, business interests ...)||No COIs reported|
|Wilson||Promotes keto||Has multiple COIs (books, business interests ...)||No COIs reported|
|Lowery||Promotes keto||Has multiple COIs (books, business interests ...)||No COIs reported|
|D’Agostino||Promotes keto||Has multiple COIs||Reports some COIs in some studies.|
|Westman||Promotes keto||Has multiple COIs (keto books, products, website ...)||No COIs reported|
|Gregory||Promotes keto||Has COIs (article, website)||No COI reported, but she had no COIs in Gregory, 2016|
|Klement||Pro-keto||Has no COIs||No COI reported|
|Paoli||Pro-keto||Has a COI||No COIs reported, sometimes reports funding.|
|Wood||Pro-keto||Has a COI||No COI reported|
|Urbain||Neutral||Has a COI||No COIs reported|
|Burke||Neutral||Has some COIs (books, funding)||Reports some COIs|
|Kevin Hall||Pro-CICO||Has multiple COIs||Reports some COIs|
|Has multiple COIs||Reports some COIs|
- Works with: Phinney, Westman, Wood, D’Agostino, Wilson, Lowery, Paoli, Feinman
- Sells: carbohydrate drinks (Superstarch): “SuperStarch is a complex carbohydrate (derived from non-GMO corn) that uniquely stabilizes blood sugar and causes virtually no reaction from the fat-storage hormone insulin. It's backed by proven science. Finally there's a healthier, more efficient energy source than sugars, caffeine, or high-carb meals.”
Volek founded "KetoThrive" (aka KetoThrive Consulting). The trademark was filed on May 14th, 2014 for use with a medical consulting firm. Any diet/keto/T2D/etc... papers that he has authored since that date in 2014 should have listed the fact that he founded and owns a keto consulting firm as a financial conflict. His most relevant degrees are an R.D. and a B.S. in dietetics. His advanced degrees are both related to exercise and kinesiology.
KetoThrive eventually became VirtaHealth for which Dr.Volek is listed as both a founder and CSO. VirtaHealth is “an online specialty medical clinic that reverses type 2 diabetes without medications or surgery". The clinic has funded some of Volek’s research, and in this 2017 study, the COIs of his co-authors and himself were listed.
More on VirtaHealth:
- https://www.forbes.com/sites/davidshaywitz/2017/03/08/can-silicon-valley-cure-diabetes-with-low- carbs-and-high-tech
- https://www.nytimes.com/2017/04/11/well/live/tackling-weight-loss-and-diabetes-with-video- chats.html
Dr. Volek has authored five books (with the earliest LC/Keto-specific title showing a publication date of March 2010); at least three of his books specifically deal with low carb or ketogenic dieting, one involves carbohydrate timing, and all at least deal with his theories on carbs. Westman and Phinney (discussed below) have co-authored several of the books. Volek has promoted some of his books through his scientific papers:
Source: The twisted tale of saturated fat (Volek et al., 2012)
In Feinman et al., 2015 one of Volek’s books was referenced:
Furthermore, most scientists we looked into on the pro-keto side are linked to Volek. He is the center of several clusters. They are typically connected as co-authors on Volek’s studies.
Volek's lab at Ohio State University lists him as having procured $7MM in research grants. He is or has been a consultant for Atkins Nutritionals, Metagenics,and UCAN (hence the Superstarch patent below). He has also received grants from the National Dairy Council and the Malaysian Palm Oil Board. He has had studies funded by or been an author on studies funded by the Dairy Research Institute, the Dr. Robert C. Atkins Foundation (since at least 2004), the California Grape Commission, and the American Egg Board.
- Physiogenomic method for predicting response to diet (US20070196841A1)
- Heat moisture treated carbohydrates and uses thereof (Application US20110287131A1)
Interview with Volek: https://www.youtube.com/watch?v=lobEiSgjb9Q
Dr. Klement is a medical physicist and researcher in Germany. He graduated from University of Heidelberg with a degree in Physics. He then completed a PhD in astronomy. Afterwards he trained as a medical physicist at the University Hospital of Wurzburg where he pursued research on the ketogenic diet.
Klement has given talks and done some keto studies but holds no patents and endorses no dietary supplements. Klement’s personal website has a list of publications and a research review.
Neutral and pro-CICO scientists
Burke has authored several books on nutrition and the majority of her funding seems to come from government or academic and professional sources. She is, however, a member of the Australian Sports Institute, an organization which has received funding from Gatorade (as has every major American organization that researches sports performance and/or sports nutrition). Additionally, her work can be found published by the Gatorade Sports Science Exchange. However, it's been reported that she has no personal financial ties to the company.
She has authored several studies attempting to develop protocols that make low carb/keto diets enhance performance, and has stated that she has tried to make these approaches work.
“We seem to be in a funny war with people who say that endurance athletes should eat high fat diets and think that we are saying they should eat high carbs all the time. I don’t think any blunt approach like this is sensible” - Burke, 2016
Clustering of pro-keto authors
Clustering happens when several researchers form groups. These groups will tend to work on a specific topic (in this case, the ketogenic diet and ketosis). It’s possible that clustering could increase bias (Décombaz et al., 2003), because the researchers typically have a shared set of opinions.
In the context of this article, the researchers form clusters around what they believe in. You’d probably not see a CICO scientist such as Kevin Hall form a cluster with pro-keto scientists such as Wilson and Volek.
Birds of the same feather flock together.
One reason for why clustering could matter, is allegiance bias. This is when a scientist believes very strongly in the treatment he is researching (Munder et al, 2013; Dragioti et al., 2015). Several researchers are looking into the best way of fixing allegiance bias. Some say that we should “balance” allegiances (Wilson et al., 2011). In our context, this means including both pro-keto and pro-CICO scientists as authors on the same study.
Top 5 conflicts of interest
Prominent keto researchers typically have one or more of the following conflicts of interest:
- Keto business interests
- Selling keto books
- Working for keto companies
- Providing consulting services for a low-carb lifestyle
- Having studies funded by keto companies
In addition, scientists may:
- Not disclose conflicts of interest
- Mostly work with other pro-keto authors (clustering)
All of the above may suggest bias, but does not guarantee it.
Now, let's say there are two scientists.
Scientist 1 has done a keto study with lab materials sponsored by a keto company. He discloses the source of the materials.
Scientist 2 has several conflicts of interest, he mostly works in clusters, and only discloses some conflicts of interest.
Both scientists may be “sponsored by the industry”, but the two cases are clearly different. This is why nuance is so important.
Our conclusion is that we have more reason to be critical of the latter scientist, than the former. However, we would still need to analyze their studies to make a well-informed judgement. Jump down to Do Conflicts of Interest Imply Bias? for a deeper analysis.
Do Funding Sources Matter?
When industry-funded studies are published, people may say that they are biased. By this reasoning, only government or university funded studies are unbiased. Yet, is this true?
As we all know, scientific research is not free. Conducting an 8 week study with 30 participants is very costly, because you require not only a lab, but also equipment, supplies, tools, subjects, and scientists. The study must be planned, performed, and the data collected must then be analyzed using statistics software and visualized using graphics software. Finally, the study must be written, reviewed by all authors, peer-reviewed and edited before it can even be accepted for publication in a scientific journal. Most scientific journals also require large fees to publish.
These things take time, effort, and are not cheap. Scientists must find ways to pay for their expenses.
How scientists get studies funded
Researchers can write applications to get funded by the government. These are called grants. Federally funded organizations such as the National Institute of Health (NIH) and National Science Foundation (NSF) have grant application cycles throughout the year.
The chance of getting a grant funded by the NIH/NSF is very low. The typical budget for a large research grant (R01) is $250,000 a year for up to 5 years. After accounting for salaries and cost of supplies or reagents, this leaves little money for studies outside of those in the submitted grant application. This often leads to a need for industry funding to supplement studies that the NIH or NSF do not fund. Furthermore, it is very difficult to get a grant funded to study exercise or nutrition in healthy people.
Companies can provide grants, tools, equipment, a lab, or materials (i.e. products) for a study.
Typically, industry funding has less peer-review 1This means a panel of independent scientists reviews the funding application requirements than funding from the National Institute of Health (NIH) or National Science Foundation (NSF). However, this is not always the case.
Industry funding can result in very large grants or funding opportunities that propel research forward in cases where NIH funding is not enough. This is especially true for small groups of researchers.
Internal University funding
Universities can fund their own research. They usually fund small amounts (<200k) where the NIH/NSF give out larger grants. University funding sources can come from start up packages for faculty, “institute” grants, or other internal sources.
In some cases where expensive equipment is not needed, scientists can fund their own studies. For example, they may be doing a case study and they already have free access to the equipment they need to use, such as a DXA machine.
Difficulties in getting funding
Getting studies funded without the industry providing some type of compensation can be difficult. For example, it is not uncommon for supplement companies to provide protein for studies (Antonio et al., 2016). Even some coaching services fund research. There are also large organizations who partner with companies to provide funds. These are all well-established and do not invalidate a study.
Funding bias or “funding effect” is the tendency of research outcomes to support
the best interests of the study sponsor (Krimsky, 2005. Krimsky, 2012; Lexchin, 2012). There have been multiple analysis to determine if specific fields have funding bias. More recently, this has become an issue in the nutrition field (Lucas, 2015).
Before we start our dive into the conflicts of interest, we have to answer some questions:
- Does funding from a company imply that the results of a study will be biased, or that a researcher is biased?
- Should a study be discarded on the basis of funding alone?
- Is a COI, such as a book or product, enough to discard the claims of a person or a study?
Here are arguments for both sides of the issue:
Why industry funding may not affect the results of a study
- The methods section of a study is more important: How rigorous was the study?
- Most registered clinical trials require a statistical analysis plan to be submitted before the study begins. This can prevent data torturing [footnote explanation]. You can find most clinical trials at Clinicaltrials.gov which is a database of private and publicly funded clinical trials.
- A researcher risks his or her entire career by forging data/lying so that a study conforms to the expectations of a company or funder. If it is later discovered that the researchers forged data, he or she might lose their job, PhD, and reputation. There are many cases where this has in fact happened (see Retraction Watch, The Scientist 2016, The Scientist 2015)
- Many studies have multiple funding sources, including state grants and self-funding by large universities. Multiple funders could reduce the chance that a single funder influences the outcome.
- Several of the most important and influential nutrition researchers have performed research funded by the industry. Would you dismiss for example, Stuart Phillips, one of the most important protein researchers, because of prior funding?
- Following up on the point above, if you claim that a researcher is biased due to industry funding, the onus is on you to provide evidence that this is the case. Merely dismissing a person’s hard work due to a potential funding bias, is not intellectually or methodologically rigorous.
- Several researchers and journals report that funders have no hand in the design of the study or in the presentation of the results. Lying about this puts a scientists career at risk.
- Many journals require Conflict of Interest (COI) to be disclosed.
“Transparency and objectivity are essential in scientific research and the peer review process” - Elsevier
Why industry funding may affect the results of a study
- Conflicts of interest and funding sources are often not disclosed in scientific studies and books (Jagsi et al., 2009; Roseman et al., 2011; Piper et al., 2015; Rasmussen et al., 2015; Shnier et al., 2016; Resnik et al., 2017). What is the motivation for not disclosing COIs, especially when journals require it?
- Several reviews suggest that funding bias might influence study results (Bekelman et al., 2003; Lexchin et al., 2003; Huss et al., 2007; Jagsi et al., 2009; Lundh et al., 2012; Lexchin, 2012; Flacco et al., 2015; Schillinger et al., 2016; Lundh et al., 2017). Though some disagree (Buchkowsky and Jewesson, 2004; Mugambi et al., 2013; Naci et al., 2014; Chartres et al., 2016). It might depend on which industry and field we are discussing (some fields might be at greater risk of funding bias). It seems like pharmacology might be such a field (Buchkowsky and Jewesson, 2004; Lexchin, 2012; Lundh et al., 2017).
- There are studies undertaken on behalf of companies that produce, market, and sell the products being tested. A dead giveaway is that the study has 0 journal citations even if it’s several years old. Perhaps it is only cited by the same authors. Examples: Oben et al. (2008) followed by Anderson (2013). In this case, Oben's work has been cited primarily in class action lawsuits against companies selling the tested product (1, 2)."
- If the primary paper (the first paper reporting the findings of the study) has a statistical analysis plan that is different from the initially submitted one it is likely that there may have been “p-hunting”..
“Pharmaceutical companies fund the vast majority of the clinical research that is undertaken on medications but face a conflict of interest between producing good science and results that will enhance the sales of their products.” - Lexchin, 2012
Do Conflicts of Interests Imply Bias?
According to Elsevier, a conflict of interest is “When an investigator, author, editor, or reviewer has a financial/personal interest or belief that could affect his/her objectivity, or inappropriately influence his/her actions”. This means that there are both financial and non-financial conflicts of interest (Marcovitch, et al., 2010)
As we have discussed in the section above, we can make arguments for why industry funding matters, and why not. If we want to find out whether a conflict of interest matters, we have to be rigorous and not only analyze the study in question, but also the background of the researchers.
Factors that increase risk of bias
There are several situations where scientists are more likely to be biased. Here are some things to look out for.
Undisclosed conflicts of interest
“Everyone has competing interests; financial or private, or both. The main problem with competing interests is nondisclosure . As with all competing interests, it is not possible to reliably judge our own biases. Instead, declaring them allows others to make informed judgments about whether the competing interests are relevant or not.” - PLoS Medicine, 2008
“Undeclared financial conflicts may seriously undermine the credibility of the journal, the authors, and the science itself.” - Elsevier
There isn't anything wrong in having competing interests. They may be near universal in medicine. Most doctors have had their lunch bought for them by a pharmaceutical company and write their prescriptions with pens provided by the industry. Many have been flown to very agreeable locations by the companies to listen to speakers, staying—at the companies' expense—at the best hotels. The problem with conflicts of interest is not declaring them. - Smith, 2002
Many conflicts of interest
This is when researcher(s) have multiple COIs. For example, they can be working for a company, sell books, promote products, provide consulting services, and more.
Selective data presentation and statistical analysis
Scientific data is typically collected and presented in numbers. Numbers may seem objective, but they can be manipulated to tell any story you would like. This is called data torturing (Mills, 1993).
“If you torture the data long enough, it will confess to anything” - Ronald Coase
Scientists can selectively choose, interpret, and present data (Mills, 1993). This is typically done in the conclusion, discussion, or the abstract of a study. This involves hiding or ignoring data that is “unwanted” and focusing on the data that supports the researcher’s conclusion.
“This sort of ‘within-study publication bias’ is usually known as outcome reporting bias or selective reporting bias, and may be one of the most substantial biases affecting results from individual studies (Chan 2005).” - Cochrane
This is why good data reporting and transparency is crucial. It’s also important that scientists follow the original statistical analysis plan submitted with the registration of the trial.
Poor study design
Researchers can design a study with a methodology that favors a desired outcome.
A typical example in the keto literature is to have the keto group eat more protein than control.
Protein improves satiety, helps build muscle, and slightly increases the Thermic Effect of Food (energy expenditure). Thus, it is easier to improve body composition on a high protein diet.
Ignoring alternative hypotheses
This occurs when researchers do not critically reflect on their claims, opinions, or products.
I.e. only presenting data supporting his or her conclusion.
Clustering is when scientists work together over multiple projects. (Example: Décombaz et al., 2003).
Direct links to companies
Researchers directly working for a company with their own lab, only doing research on their product. Especially if they are not affiliated with a university.
Factors that decrease risk of bias
“Trial registration is the practice of publishing the methodology of clinical trials before they begin. This reduces problems from publication bias and selective reporting (such as choosing the methodology after the results are known)” - Wikipedia
Studies can be registered at Clinicaltrials.gov
Disclosure of conflicts of interest
All relevant COIs and funding sources are disclosed. This shows honesty and transparency, which are important values.
Solid data reporting and interpretation
Solid data reporting (i.e. reporting objective values, all relevant data shown). Being fair and balanced when writing the abstract and conclusions.
It is crucial to note that data can be gathered, interpreted, analyzed, and presented in many different ways. Minute changes in data analysis can completely change how it is interpreted.
I.e. doing significance tests for percentage changes rather than absolute values. Reporting performance per kg body weight, so that a decrease in bw means an increase in relative performance.
Solid study methodology that does not favor one group over the other. I.e. proper randomization, representativeness, sufficient study power, and more.
Independence of trial
Those that fund a study should not play a role in designing, analyzing or publishing the research. Some companies have their own labs where they actively design their own trials and study methodologies.
Should scientists always disclose their conflicts of interest?
A recent viewpoint was published discussing disclosure in nutrition research (Ioannidis & Trepanowski, 2017). The main idea was that authors should disclose all types of opinions and conflicts of interest (financial and nonfinancial). It is important for the health and fitness field to disclose conflicts of interests in books and other content since it can greatly influence a large number of people who are unaware of how to interpret the research themselves.
All of the resources below are free full texts, and are not hidden behind a paywall.
Our Conflicts of Interest
Our Conflicts of Interest
This article was written and published without sponsorship from any company, individual, or organization. The Science of Fitness (SCI-FIT) receives no compensation or funding from any source, and is not affiliated with any other companies or organizations.
- Adam Tzur is the head of SCI-FIT. No COIs to report.
- Anthony Roberts is a journalist who writes articles on medium.com
- Brandon Roberts is a scientist and physique/strength coach. He works for The Strength Guys.