I have to say, I am completely amused by all the uproar about a recently-published study in Pediatrics that was reported to have found that “Spongebob ruins attention spans.” And not just because a scientist got published (and probably paid) for writing an article that actually includes the line describing “a very popular fantastical cartoon about an animated sponge that lives under the sea.” No, I’m amused because the study is a giant pile of shit and nobody in the media - or the official goddamn journal of the American Academy of Pediatrics - has apparently noticed.
In short, the the researchers (and I am using this term very, very lightly) got a bunch of four year-old kids. A third of them got shown some Spongebob Squarepants, a third watched an educational cartoon and a third were given some materials and told to draw whatever they wanted. After a little bit, the TV shows and art supplies got taken away and the kids had to do some pretty straightforward measures of executive function, including tests of memory, attention span and the always adorable Marshmallow Test. Unsurprisingly, the kids who watched Spongebob did worse on the tests.
Why is it unsurprising? Because it’s ridiculous to expect a kid to pay attention to you seconds after you have taken away a fun, exciting cartoon. Hell, try something: go and turn off the TV a grown-ass adult is watching and give them some homework. After they are done screaming at you for being an asshole, think about how silly it is to expect a small child without the years of development that a grown-ass adult has to pay attention. A kid quietly drawing might see the tests as a similar exercise. A kid watching a terrible educational cartoon probably isn’t as jacked up as a kid who just watched a fun cartoon. We don’t know for sure. However, it was the scientists’ job to think of these things and either account for them or investigate them, and they did neither.
Consider this: how did they estimate the kids’ “normal” functioning? A good scientist might give them the executive function tests before the TV/drawing as well, perhaps even on an entirely different day. Instead, the researchers here just asked the parents afterward. Besides the fact that comparing a parent questionnaire to an actual test of a child’s function is like comparing apples and ballpoint pens, imagine you are a parent who has just watched their kid perform poorly on a test because it had the bad luck of interrupting their favourite cartoon.
Oh, I don’t know what happened! Mitchell is normally so quiet and well-behaved! He always listens!
Does that sound familiar? It should, because it is a totally normal reaction to watching your kid flip the fuck out or not meet expectations. Parents generally want to portray their children in a positive light, especially after embarrassment, and they’re probably glossing over the fact that the kid does, from time to time, act like a bit of a dick when the TV gets turned off. Whatever. They’re kids. I get pissed off and refuse to listen when somebody stands in front of the TV to get my attention. Is a kid supposed to be better?
There is a section of an academic journal article specifically devoted to talking about the things you missed or could have done better. It’s the scientific community’s built-in humility check. The researchers here basically say, “We don’t know why Spongebob fucks kids up, but we probably should have used kids of various ages because our methods are pretty rad anyway.” They show a startling lack of insight. Or maybe I’m just used to hating myself and everything I’ve done after I’ve stared at twenty different versions of a manuscript for four months. That could be possible. Either way, their inability to admit they were much less than perfect is kind of galling to me, personally, as a scientist who mostly sees the purpose of science to show what we still don’t know.
Pediatrics is a peer-reviewed journal with a good impact factor, the de facto measurement for how “important” a journal is. It’s the biggest journal in its field. And somewhere during the review process, one of the 4+ people that were instructed to read it and pick out the things the study did poorly, who were ostensibly picked because of their ability to know when another researcher in the field of pediatrics has fucked up, should have sent it back to the authors with the nicely-worded equivalent of “Come back when you’re a scientist and not an unlicensed day care with delusions of grandeur.” The article should have never seen the light of day.
The reason journal articles need to be good and not sensational is because when they gets reported in the media, the reporters and readers generally aren’t scientists who understand methodology. They don’t understand that the researchers only say the immediate executive function, not the long-term one, is harmed by a “fast-paced television cartoon,” which is to say that Spongebob was not shown to fuck up kids’ attention span for longer than a few minutes. All they see is “Spongebob” and “attention span” and WON’T SOMEBODY PLEASE THINK OF THE CHILDREN? They don’t know this is a shitty study. Even if they do, they often don’t recognize that there are other good, valuable studies that don’t generate needlessly sensational headlines, and frequently use bad ones like this to argue against research funding because they generally don’t know that money that went to a study like this probably was never, ever going to go to cancer research because those are different groups in different buildings reporting to different people. They just don’t know because they’re not scientists, and that’s fine, but along the way someone is supposed to keep shit like this from their eyes for exactly that reason.
Bad studies like this hurt psychologists, scientists and higher learning. At a time when intelligence and science are increasingly attacked by groups for whom “elite” doesn’t mean “the best” but “snobs”, who think scientists are all the same group of godless liberal commies (exact words I’ve heard), we can’t give them any support. We need to be better, and part of that needs to be making sure studies like this don’t get published.
