Shitty Research

I have a Bachelor’s of Science in Kinesiology with an emphasis in Exercise Science. It’s okay I guess. I knew that I didn’t want to pursue graduate work in this area of study because I wanted to coach, but I wasn’t impressed with the field itself. It’s just so BAD. The exercise prescriptions are ineffective, nobody understands mechanics, and the research is a joke. A lot of people have mentioned how they would like to hear my thoughts on the problems in Exercise Science, and I think it starts with the research.

I’ve had to look at a lot of shitty studies in school, and I can dismantle them all. The articles in various journals, including the NSCA, are usually not very good. Rippetoe and I used to look at several articles and point out how badly they were done. However most studies shouldn’t even be done in the first place because the researchers are asking either an irrelevant question or the wrong question. The majority of studies that are published are irrelevant or easily dismissed because of crappy methodology.

Why am I not doing anything to change this? Because I don’t care. Why would I aim to change the governing boards of organizations that allow bad research on things that don’t matter? I’m busy getting people to perform better and teaching them how to do it. I mean no disrespect to researchers — especially the good ones — but most of them…don’t quite get it. Let’s look at this example.

The title of this news article is, “Squat lifts likely cause of stress fractures in young athletes, study finds”. We’ll soon find why this news article is just as bad as the study itself since this conclusion can’t be drawn. Here is the “press release study abstract” that the shitty journalist used to write the article. I will be directly referencing this piece of information in the rest of the post since the actual article is not made available (this, by the way, is another flaw in the system: let’s charge money to see the shitty research we’re doing so that nobody can debunk it unless they want to spend $30 to see our boring journals).

Abstract TL;DR — The researchers had patients (who were kids) that said that they hurt their backs while squats, so they performed a study to see if squats caused the “lumbar fractures of the pars interarticularis” that they found. The study itself observed the “pelvic incidence and tilt, sacral
slope, and lumbar lordosis while in normal standing posture and then during a front and back squat” in 20 volunteers. Here is a picture of the author, John McClellan.




Now let’s tear this apart. I’m upset to say that it was too easy. I made notes as I read the abstract itself. The first point I want to make: in my 13 years of training with barbells, playing football in high school and briefly in college, and training and coaching for the majority of it, especially the last five years, I have never seen or heard of this injury (pars interarticularis fracture). There is some merit to this statement as I’ve directly coached at least 50,000 squats in the last couple years and have seen many more over these years (along with performing at least 20,000 in training).

1. The number of incidence or rate or injury are not included.

2. Fracture healing is 3% for bilateral and 25% for unilateral pars fracturees. Sounds like the authors aren’t very good at healing this.

3. How old were the males? Were they untrained? What experience did they have other than willing to volunteer? Why was it not relevant to include their mean age? Why can the results of using these males (who are undoubtedly over the age of 18) be generalized to adolescents?

4. Sacral slope merely indicates a back angle, and we can quite clearly assume it to change during a squat. A back squat will naturally have more of an incline than a front squat, so this “sacral slope” is completely irrelevant and tells us what we already know.

5. Since the squat itself was not quantified whatsoever, any readings taken during the squat are completely irrelevant. There’s no way to know whether or not these researchers or subjects have any clue on how to do the squat. This is the biggest problems in most, if not all, squat studies: it’s never quantified. If you don’t quantify the movement, you cannot reproduce it in further studies or even know what standards the subjects were held to in the first place.

6. Furthermore, the sample size is too low to imply anything even if the researchers/subjects actually knew what they were doing to begin with.

7. This study showed that the lordosis (or natural curvature of the lower back) decreased during the squats. This would appear to imply that the lordosis lessened, or in other words the subject slightly rounded his back. Everyone reading this site knows that lumbar flexion is not supposed to occur during a squat, and if that’s what this data means then the subjects are implementing it incorrectly. This, of course, would mean the whole study is erroneous and irrelevant.

8. The changes “would be expected” to increase “lumbar extension forces” and “play a role in pars fractures”. Of course the author of the news article (John Fauber) would turn this vernacular into “squats likely cause” fractures. If he had made any effort to analyze this from a scientific perspective, he’d see that this statement would absolutely not apply given the information presented. Furthermore, “expecting” something to happen isn’t science, it’s wishful thinking, so the fault lies with the researchers as well.

9. If the researchers disagreed with the arguments thus far, then they should make their article freely available to the public instead of requiring payment and membership to read it. Doing so could validate their data, yet it would probably show how bad of a study this is. Regardless, this press release abstract shows how painfully erroneous all of these claims are. It not only makes them look like fools, but it makes John Fauber look like a complete dolt.

10. This study ONLY displays information on changing of sacral slope during two movements that aren’t even quantified. No one else can replicate this study because of this lack of quantification. Additionally, it is pointing out what is already quite clear to anyone who has a mediocre grasp of musculoskeletal anatomy and strength training (meaning that we already know there is a change in back/sacral angle during squatting — this means that wasting time performing this study was a waste of time).

11. The study makes a grand leap of an assumption that squats are to blame for the adolescent injuries (a data point that isn’t quantified either). It assumes that squats cause the fracture instead of 100+ pound kids running into each other at full speed while in lumbar flexion or extension. It ignores the data that exists on incidence of injury in sport compared to weightlifting (weightlifting rate of injury is incredibly low when compared to very high rates of injury in other sports like soccer and football, see Starting Strength).

12. Most of all, it ignores the fact that an adolescent, especially when un-trained in barbell lifting, will not have good muscular or skeletal density around their spine and hips. Most kids don’t even know how to contract these muscles, much less hold them in place properly when applying or receiving force. These kids are then placed in inefficient positions — due to not having developed the musculature and skill to do so properly — and then making full contact with other kids or the ground.

13. Bones and muscles will not increase their density or ability to withstand force without the application of stress. In order to make either stronger, they must be subjected to forces. There is no better way to strengthen these structures — both postural and force applyers — than a squat through a full range of motion.

14. High school football coaches are notorious for not teaching very good technique in lifting. For that matter, NSCA certified “professionals” aren’t that good either, and there are many examples of NCAA collegiate strength coaches who do a poor job of coaching correct lifting mechanics. This doesn’t mean that all high school or collegiate coaches are bad, but an overwhelming amount allow inefficient mechanics. This article assumes that there isn’t any fault in the coaches. Despite this lack of skill in coaching, the rate of injury is still low when compared to the rates of injury in sport.

15. In my 13 years of training with barbells, playing football in high school and briefly in college, and training and coaching for the majority of it, especially the last five years, I have never seen or heard of this injury.

39 thoughts on “Shitty Research

  1. In addition to your 15 points, the comments on the news story site itself are fantastic. The “nutrition science” industry has the same problem–basic violations of the scientific method followed by an echo chamber. I’d be interested to know who/what funded the study since generally someone stands to benefit every time something is labeled dangerous. Regarding point 14, I agree with this 100%. All of the “trainers” at my gym have some sort of bullshit certification credentials, yet none of them know anything and only half of them could be considered atheletes by any measure. Also you’ve got to wonder why they picked 95 lbs–probably because that’s the mid size plate. I’d bet a bunch of the kids in the “study” didn’t weigh much more than that. I like these soap box articles. Sometimes the bullshit detector goes off so loud that you just have to say something.

    Kids weren’t used in the study. You have to be 18 to be a participant in something like this.

    There wasn’t a lot of funding needed for the study as they merely radiographed different positions. It’s just a poor study looking into a question that is irrelevant.

    –Justin

  2. the thing with this injury, is that from looking at pictures of what it supposedly is, it would seem that it is caused by excessive lordosis, or arching the back too much.

    (assuming I am interpreting the anatomy pics correctly, as I have no background in said field)

    supposedly this is the injury or condition caused: http://en.wikipedia.org/wiki/Spondylolysis

    I was going to talk about the injury itself, but the post was already long enough. I’d have to look and think about it more (and probably ask Dr. Hartman, I’ll do so now) before I could say for sure what type(s) of movement would cause it.

    –Justin

  3. “These changes would be expected to increase the lumbar extension forces on the posterior elements of the lumbar spine and may play a role in adolescent pars fractures.”

    Not exactly groundbreaking, and opens avenues for future research. This is fluff you have to put in everything you present of publish to answer the “so what” question. What crappy science reporters and the doctors marketing dept do with this information is something else completely.

    BTW, this isn’t a paper, it’s a study presentation for the North American Spine Society conference that happened last week. They haven’t published anything yet. They’re not leaping to assumptions, they’re trying to figure out what sort of movements cause an injury they are interested in, and get brownie points in NASS.

    Furthermore, until PloS adds an exercise science journal, there aren’t that many peer-reviewed open-access spine science journals to choose from. Just posting a pdf with results will make the science worse, since you could make grand claims without worrying about a reviewer going “hey wait a minute”.

    Good points. I don’t think that the question of back angle changing is even worth asking, yet if they are going to further their research on this topic (even if the line of thought is stupid to begin with), they have to do this preliminary stuff.

    The author of the news article still blows as this ‘study’ says nothing. I guess I’ll have to go over the shitty squat articles from the Journal of Strength and Conditoining Research to make you happy.

    –Justin

  4. Pardon my ignorance, but in an exercise science program how much discussion is there of barbell training? Any? My cousin is majoring in kinesiology and I doubt that she’s ever (or rarely ever) squatted. Next time I see her I’m going to ask her about the program.

    Um, not much. I had a lab where we actually did some squatting on one day. Squatting was not discussed really in any class, including the various biomechanics classes. The movement of squatting (unweighted) was analyzed, of course, but not with implements.

    –Justin

  5. Been a big fan of the site for a long time and I wholeheartedly agree with your assessment of the shitty research. I live and am a sports performance coach near Milwaukee and luckily haven’t had to deal with any backlash yet. Squatting used in a progressive manner from body weight to externally loaded have been and always will be the cornerstone of any program worth a shit.

  6. @ Justin
    Actually it would make me happy, since I try to read those journals to learn things and end up confused as to why they suck.
    I don’t have the kinesthesiology background know what BS to ignore and what BS might be relevant.

    I wasn’t trying to be a dick. If you’re a spine scientist or MD that doesn’t often deal with novice lifters, it’s useful to have one of your own break it down for you. We’re not the target audience of what I imagine is a 5 minute presentation or poster at a very specialized conference.

  7. totally unrelated.

    Do you guys have any affordable (less 30… aiming for like 20 dollars) shorts to train in that are shorter than basketball shorts. But not the scrotum showcase shorts that lascek wears (soffes)? Like I am 6 foot and just want a short that doesnt go past my knee but might sit 1 to 2 inches above my knee. I tried on like 4 different kinds at dicks and they were all basketball styled…

    I posted a link here last time this was brought up. I found the good kind of shorts I have by Russel, but the site that carries them doesn’t have them in size. I hope they get re stocked.

    –Justin

  8. Just to clarify, there is a little bit of utility in exercise science. It lies with anatomy and physiology.

    The basic A&P courses are typicall pre-requisites to get into a program, and they are good at providing an overview about the body. The labs are anatomy driven and there are either cadavers or models of the various systems (the latter is more likely).

    The major course work itself may have some classes that aren’t great, but they can provide a foundation for programming and training. Mechanics are not specifically covered (at least not in my specific program), yet the Exercise Physiology course is necessary and one of my favorites.

    My professor for biomechanics and biophys had an engineer background, and I appreciate that even more now than I did when taking the classes. Even though we didn’t look at barbell training, I can apply everything I learned into this realm.

    We covered many things in these classes, including the properties of muscle, bone, tendons, ligaments, and nerves, but also how the function within the confines of movement. For example, when squatting down, the hip and knee extensors are applying force, albeit eccentrically. Our movement analysis problems required us to know every muscle involved in a given action and explain how it was involved in said movement.

    The biomechanics class itself provided the math behind a lot of instances in human movement like projectile motion when running. Even though we weren’t looking at barbell training or Olympic lifting (I’m not sure any program even has a class on this, much less a quality one), this information and experience can be applied to strength training. We learned various topics in this class like using still photographs or video to calculate the center of mass and how it effects the mechanics of the body. It was the application of physics to movement.

    My university (Georgia Southern University) also got cadavers towards the end of my time there. Cadavers are an immensely beneficial way to learn. Aside from the amazement and wonder in holding a human heart in my hands, I have a much clearer picture of how the body works, whether that is systemically or locally, whether talking about mechanics, movement, and lifting or the digestive system. There is no better way to understand the body than studying cadavers, and if you have that chance, I suggest that you take it as it will make dealing with the less worthwhile stuff worth it.

    So you see, I didn’t just plod through a stupid program that has stupid prescriptions. It isn’t all bosu balls and machines. I learned hard science in anatomy and physiology. I expanded on that in how everything works in exercise physiology. I applied that information in biomechanics and biophys. I reinforced all of these lessons when studying cadavers and models of bones and muscles. And I was given a foundation that I have been able to apply and exponentially build on in my time since being in school.

    There is a lot of stupid bullshit in the realm of exercise science, but there is still a lot to gain from it that you can’t get in other majors.

  9. Justin,

    I have spondylolysis and spondylolisthesis (the vertebrae actually moves forward into further extension) and I am a chiropractor. I have extensive knowledge and experience with this condition. Mine, and most cases are not caused by squatting! Current research offers many theories, but no concrete answers. I can say from my practice experience, they are much more common among male athletes. Personally, I still squat and DL with no problems. As a matter of fact, once I perfected my squat and DL my pain became very minimal and most of the time non-existent.

    Thanks for the info. This is what I have seen/heard in the past regarding the spondylolisthesis. I’ve been told that doing exercises that take someone with this injury (condition? it can be congenital) shouldn’t go into full lumbar flexion (as in a back extension). Is this a problematic position to be in?

    –Justin

  10. Is the shit quality of sports science limited mostly to the US of A or are other countries producing the same kind of dribble?

    I liked Jim Wendler’s logic for being an English Major: he wanted to “keep the weight room sacred” and not fill his training up with too much geeky shit.

    It’s the same internationally as well.

    That’s funny that Wendler could determine that while in college.

    –Justin

  11. @ stroup- try out soccer shorts- they work well for me, cheap adidas ones for 10 to 15 bucks. I am in the market for straps to snatch with. My bar that I got off craigslist has a lack of knurling on the snatch areas because it was used inside of a power rack for a while so it makes snatching hard. I have been using pieces of an old webbed belt that I had, but it has started to lose its tightness and grip ability. Suggestions?
    Also, anyone here wear tights or leggings whilst lifting? Got any suggestions for compresion wear or singlets?

  12. @PatrickStroup
    http://www.target.com/p/C9-by-Champion-Men-s-Gym-Short/-/A-11967465
    I bought a couple pairs of these from Target and am extremely satisfied with them. They come down either to the knee or just above, and have pockets to put an mp3 player in if that’s what you’re into (or a wallet and keys for a trip to the store at 11 o’clock at night). They’re real light, not clingy, and stretch plenty in all the basic barbell movements.

  13. Justin,

    If you’re doing another Q&A Friday, could you talk about high bar vs low bar squats?

    I’ve been reading posts by Pendlay and also comments by Hatfield about how HB may not be inferior and may in fact be superior at least based on anthropometry.

    Jake

    I’ve been meaning to get into this for a while, but there is lots of utility in high bar squatting. I don’t know if it will come up in this week’s Q&A because I want to give it a lot of attention.

    –Justin

  14. @harveymushman – those look money as fuck. But they are cotton and I usually squat in mesh / polyester materials. I am wondering how bad my ass sweat would go through cotton shorts…

    Fuck it they are cheap and at target so I will snag a pair and try em out. Thanks man!

    They’re polyester. Special Forces are not in your future.

    –Justin

  15. I’m in an exercise science program right now. Funny thing is that they just added a “concentration” in strength and conditioning. The only difference is 3 added classes: Strength And Conditioning Lab (ok), “Weight Training,” and “Body Shaping.” The latter two being 1 credit courses open to ANY major, and being taught by a 60 year old woman. I can tell the major is going to be mostly BS but with the “concentration” i at least come out of college with a CSCS cert, which if i HAVE to be labeled I’d rather that than CFS (Certified Fitness Specialist). I fully intend on telling the head of the kinesiology department (in the nicest way possible) that theres a ton of BS in the major.

    That being said, about the Justin’s clarification, I fully agree that A&P is a great class and if taken seriously, can be applied to any kind of fitness and is CRUCIAL to understanding how the body works. Unfortunately my school does not have access to cadavers but I will be attending PT school and most likely will be working with cadavers in my Gross Anatomy class. I’m looking forward to it.

  16. My issue is the statement, “The exercises were done under the guidance of a physical therapist.” as if that qualifies it as a proper squat. I have several family members who are pta’s and pt’s. I’m sure they could help teach an elderly woman how to stand up from the toilet but none of them have ever squatted. I have had too many conversations with them about the “dangers” of squatting deep, of wearing a belt, and of holding your breath. My strongest argument always comes with “show me the pile of bodies.” ala Starting Strength…

  17. You’d figure that any discipline which could obviously (to me, at least) benefit the large professional sports industries in the US and abroad would sort of naturally have money for studies pouring in, and a plethora of peer-reviewed journals and other avenues of research, where the weak science was rapidly culled. I’m sad to see that’s not the case. I wonder why that is.

  18. “Under guidance from a physical therapist”

    This being another piece of information that tells you absolutely nothing. I’ve talked to a lot of friends who are students in one of the top PT programs in the country about what they learn about squatting… it’s disturbing. I recently went to a PT after breaking my ankle and starting rehab. Besides looking like he had never picked up a weight in his life, he had no idea how to squat either.

    Before you think I have it out for all PTs, a training partner is a PT who just squatted 182.5kg x 7 on sunday. He knows his shit. My point is, saying a PT directed the athletes means absolutely nothing.

    Reminds me of a study about the power clean we were reading that was done on untrained athletes… wtf? I mean 95% of crossfitters, who seem to like to do the lift all the time, are very poor at it. If you’re not performing that study on highly trained, skilled lifters, what is the point?

  19. Justin, something that sticks out to me about that study you posted is that not one of those researchers has a PhD (they didn’t even collaborate with one). Medical school does not teach you how to perform research, if you want to do good research get a PhD or MD/PhD.

    If exercise science was good enough to be published in PLoS it would be. Any scientific research can be published in PLoS ONE (as long as it is scientifically sound they publish it). Just doing a quick search on PLoS ONE for “exercise science” didn’t turn up much. Most of it looked to be studies showing that physical activity is better for your health than bed rest.

    One of the difficulties in medical science is that certain hypotheses are inherently unethical to test. As an example, to show causality between squats and fractures of the pars interarticularis. You would actually have to have a pars interarticularis fracture occur during the study, one way or another (positive controls, 1000s of poor form squats, etc.). It is unethical to purposefully harm a human (animals can suck it) in a research study. This means that you can not conduct a study, with humans, that shows that something causes (not is correlated with) harm to human health. This is probably why most of exercise and nutrition research journals are laughed at by researchers outside of the field.

    Note: I also did a search through Nature and Science (two of the best basic science research journals in the world) for “exercise science”. Again they only publish research showing that something is better than nothing.

  20. @PatrickStroup Don’t know where you saw cotton but the ones I have and linked to are 100% polyester. I can second you on the sweating (I got it bad) and assure you they pose no problem in that area. My shirts usually get pretty dank but the shorts wick just fine.

  21. You guys are dumb. I stopped squatting years ago. And now I do 3 sets of 12 front foot elevated Bulgarian Weight Vest Split Squats super setted with some single LEG KB RDLs then I finish up with some KB pistol squats onto a high box for my ME work.

    My squat have never been better. Why just yesterday after not squatting in 3 months I 22″ box squated 315 FOR A DOUBLE. At 295 lbs bodyweight and 22% fat.

    Exercise Science works you uneducated needle dicks. You mormon pussy butt puppets just dont know how to apply it.

  22. @harveymushman – oh thank the seven. I did a quick google on the model of those shorts and it said cotton. Going after work to snag a pair I will be sure to give you the internetz high five if I am sucessful in finding them.

  23. I wonder if part of the problem is the type of person that typically (not always) goes into the sports sciences stuff. In my experience it wasn’t usually the academically-oriented person, it was the one who just wanted to get by.

  24. You don’t need a Ph.D. to do good research.
    Sure, it helps to have some training, but there’s tons of meta studies of published studies done by Ph.D.s that show how poorly they use statistics, abuse of p-value, sample size, all that jazz.

    Can you propose a better study design?
    I go first: It would have to be a longitudinal study of a large number of people dedicated to learning and performing correct, USAPL-approved squats weekly for a long period of time, in front of a camera or with attached sensors so accurate data can be measured.

    It would be cool to do this with the entire freshman class of an exercise science program, track how they move as they learn how to squat, adapt, and record any injuries or whatever that emerge through their four years of college, and new study participants would arrive every year. You’d have some serious compliance issues though. It might be good to try this in a nunnery or something, since those people are used to ritual, don’t go out drinking, and stay in the same location for years/decades.

    Now that would be a fun study.

  25. @Maslow

    Yes. I think that is part of the problem. When I was first out of high school, I was into playing guitar, so I went to Music College as a way to feel like I was going to college yet still got to be around music, and not have to pick another major that i thought was boring.

    I notice a similar thing now that I’m back at college pursuing an Exercise Science major. I get the impression its a lot of athletes who dont want to major in something “boring” so they think Exercise Science is a good way around that and they still get to feel like they are around sports.

    This seems to be why the majority of the students in the major can’t pass Chemistry class….and are actually angry they have to take it (i guess they weren’t expecting college to be hard if their major had the word “exercise” in the title and they are athletes.)

    Obviously I’m doing a lot of generalizing and this is not the case for all students or all schools for that matter, but I do think that a decent chunk of exercise science majors are not in it because they want to change the way we research the squat, but because they thought it’d be an easy major.

  26. I’m a master’s student in exercise science and the situation isn’t really different here in Germany – in my opinion, the overwhelming majority of studies dealing with strength training is irrelevant and useless. My program at the university includes some interesting stuff in the fields of robotics, biomechanics, motor learning and sports psychology, but if I didn’t read so much training related stuff online for myself, I wouldn’t know anything about barbell training at all.

  27. Pingback: South Baltimore CrossFit » Wednesday, Nov 9th: Squats Cause Cancer

  28. Actually full lumbsr flexion feels pretty awesome for people with a spondy. The bottom part while on the back extension or any other kind of forward bending can be quite relieving. Hyperextension = bad. If I over-extend at the top of my deadlift I always know it the next day

  29. Pingback: 11/09/2011 » CrossFit Mount Laurel -

  30. @I Can Make You a Man

    I will admit the medical sciences sometimes have poor statistical abilities as well.

    My idea would be to write a grant to examine the “long term systemic benefits and consequences of the sport of weightlifting”. The point would be to recruit “untrained college-age males” and “trained college-age males” (athletes) into a weightlifting club and track biomarkers and general health over their college careers.

    To sum up the feelings of most principle investigators. Everyone’s research is shit except my own (and whoever gets published in Nature, Science, and the Lancet). Just like all your stuff is shit and all my shit is stuff.

    http://www.youtube.com/watch?v=MvgN5gCuLac

    Justin, about having an engineer as a biomech/phys teacher. My BS/MS was in bioengineering. Now that I’m working in the life sciences I realize my degree is a lot more useful than a biology or chemistry degree. Solely because I think like an engineer.

  31. @wreeder
    haha, medical scientists aren’t exactly famous for their statistical abilities.

    Your biomarker idea would be cool, I guess you’d expect a jump in “good biomarker” in the naive population and slow upward trend in the athletic population. My only problem with it is that some college students binge drink regularly, others don’t sleep, they eat like shit and…so many variables when you’re tracking biomarkers.

    With my pretend kinesiology study, you get to see how naive and athletic populations learn to access their muscles, quantify what mistakes beginners make that might cause the injuries the uninitiated associate with squatting, et cetera. Plus you’d not need a grant if you’re willing to work for free and have access to camera+software, since there’s less regulation about that than biomarker collection.

  32. Pingback: TJ’s Gym WOD for Fri. Nov. 11th « TJ's Blog

  33. Hey. Just wanted to comment on #9. When you submit to a journal, you have to sign the copyright over to the publishers. They’re the ones who decide whether the article is free or not (and generally, all are not). It’s not behind a paywall because the authors are cowards trying to hide their data, but because that’s the unfortunate status quo of academic publishing.

    The rest of your points I agree with.

    Good point, and good distinction to make. Unfortunately prestigious journals are guarded.

    –Justin

This site uses Akismet to reduce spam. Learn how your comment data is processed.