463 posts tagged with statistics.
Displaying 1 through 50 of 463. Subscribe:
Slow burn or cold turkey?
Some TV shows take a while to "get good." Modern classics like Breaking Bad, The Wire, Community, and Bojack Horseman are notorious for "starting slow" and are often recommended with a disclaimer like "Give it a few episodes; I promise it gets good!" At the same time, some shows never get good. Recently, I started a spy series called The Agency, which could best be characterized as premium mediocre (at least so far). There are big-name actors (Michael Fassbender, Jeffrey Wright, Richard Gere), expensive sets, and glossy camerawork—but after a few installments, I'm trapped in a liminal space between engaged and listless. At the end of each episode, I'm left with the same thought: "Maybe the next one will get good."How Many Episodes Should You Watch Before Quitting a TV Show? A Statistical Analysis [more inside]
Committing to a mediocre program or continuing with a floundering series elicits a state of (mildly) torturous ambiguity. Should you cut your losses, or is this show some late-blooming classic like Breaking Bad? What is the optimal number of episodes one should watch before cleansing a subpar series from their life? Surely, a universal number must exist! Like 42, but for television. So today, we'll explore how long it takes a new show to reach its full potential and how many lackluster episodes you should grant an established series before cutting ties.
AO3, data, and changes in fandom
Article by Allegra Rosenberg about AO3 statistics and what they show about changes on the site: "The data collected by statistically minded fans is hugely helpful in visualizing the shifts brought about by successive fan generations". Via Smart Bitches, Trashy Books.
Spooky in ways that are at once real and imagined, novel and banal
This reaches into other outlets that serve the terror-consumption cycle. In the pre-internet landscape all pain (while no less felt) was local. With the internet, what gives all forms of destruction their double-edged blade is their everpresence, their relatability, and their inescapability. The things that go on in someone’s backyard are no longer confined to their backyard. People don’t have to be caught in an immigration raid to be spooked by it. Listeners needn’t be in earshot of a firearm to hear it crack. from A United State of Fear [The Ringer] [CW: grisly details]
Finding groups by their agreement
Analyzing UK people by their opinions doesn’t reconstruct generational cohorts. Preliminary research? Better-than-average marketing research? Common fallacies rediscovered? [more inside]
Some AI restrictions in Europe, but..
The European Commission has clarified eight AI use cases banned within the EU. Although good, the rules mostly limit the damages from spurious correlation, but even there broad contain terrorism exemptions. At the same time over the pond.. [more inside]
One of the few objects left that can summon a virtuous aura of salvation
The idea of men who need new stories but refuse to read them is also exaggerated and hyperbolic. It has become its own kind of story. It’s a legend, one that’s been repeated for years, haunted by zombie statistics and dubious facts. Its continued flourishing says a lot about what our culture worries about and all the things we hope will heal us. from Are men’s reading habits truly a national crisis? [Vox; ungated]
Is that enough to account for all human bias?
This blog post is a bit ... different. Normally, a blog post is a static document, and the direction of communication is from the screen to the user. But this piece requires you to interact. You're not just reading the content; you'll actually change the story of the blog post as you interact with it. This makes the piece a bit more experimental, but hopefully, also much more interesting! from An Inverse Turing Test
The world is my country, all mankind are my brethren
"MyLifeElsewhere is a collaborative community that allows you to compare the country you live in with other countries around the world. We'll show you various statistics that differentiate your country from others, including cost of living, geographic size, and more." [more inside]
Statistically, Arnold Schwarzenegger is better than Sylvester Stallone
Maybe move to Lichtenstein and take up skiing?
Olympic medals per capita or GDP. Choose to view the data for one games or all of them, weighted or not, per capita or by GDP. [more inside]
The 101st most successful music act of all time
Since then, the myth of Nickelback's awfulness has only grown through gifs, worst-of polls, clickbait articles, comedian punchlines, and YouTube mashups. But why is Nickelback the internet's punching bag of choice, and what seeded this collective animosity? Is there a quantifiable explanation for all of this Nickelback hatred? from Why Do People Hate Nickelback So Much? A Statistical Analysis [Stat Significant]
Justice League
Major League Baseball has incorporated the statistics of former Negro Leagues players into its historical records on its website, meaning legendary leaders in some categories like Babe Ruth and Ty Cobb have now been replaced in the record books by players who were not allowed to play on the same fields as them during segregation. Josh Gibson, one of the greatest sluggers in the history of the Negro Leagues, is now listed as MLB’s new all-time career leader in batting average at .372, moving ahead of Ty Cobb at .367. The MLB website shows Gibson also overtaking Babe Ruth in career slugging percentage. [more inside]
I guess I have no choice but to love this song forever
Ultimately, cultural preferences are subject to generational relativism, heavily rooted in the media of our adolescence. It's strange how much your 13-year-old self defines your lifelong artistic tastes. At this age, we're unable to drive, vote, drink alcohol, or pay taxes, yet we're old enough to cultivate enduring musical preferences. The pervasive nature of music paralysis across generations suggests that the phenomenon's roots go beyond technology, likely stemming from developmental factors. So what changes as we age, and when does open-eardness decline? from When Do We Stop Finding New Music? A Statistical Analysis
Which animals cause the most deaths in Australia? Horses
Which animals cause the most deaths in Australia?
Horses: 172 total deaths between 2001 and 2017, many of them from falling off a horse. [more inside]
"try to analogise these great matters of state to your daily life"
Daniel Davies is a finance expert, journalist, and former investment banker whose writing I've been reading for over 20 years on Crooked Timber and on his own blog as well as elsewhere. Sometimes he writes analogies, games, or flights of fancy to help readers think about complex issues more clearly. [more inside]
The rate of catastrophes appears roughly constant over time
One argument against using historical base rates is that the present is so different from the past (e.g. due to technology) that base rates are meaningless. While today’s world is indeed different from the past, base rates can help sharpen rather than neglect these differences, by clarifying what’s actually new. For instance, the mere presence of technology cannot move us far above the base rate, because many technologies have been developed throughout history and none has caused a catastrophe in the sense defined above. Instead, we should look for technology that shares properties with the historical drivers of catastrophe: epidemics, famines, wars, political turmoil, climate changes, natural disasters, invasive species, and humans. from Analyzing the Historical Rate of Catastrophes
Separating hyperplanes with Shoggoth Shalmaneser
A jargon-free explanation of how AI large language models work - "Want to really understand large language models? Here's a gentle primer." [link-heavy FPP!] [more inside]
The journey of your life
"how it will be allowed to be interpreted"
Fred Clark of Slacktivist (previously) quotes Biblical scriptures on honest weights and measures while critiquing corporate survey metrics and their dishonest usage by bosses to punish individual workers. "Your job is simply to give all 5s. To everyone, everywhere, every time. This is your task because it is the only honest answer available to an honest person. Because 4≠0. Because differing weights are an abomination and false scales are not good. Because your wealthy are full of violence with tongues of deceit in their mouths and bags full of dishonest weights." From June 2019.
Future History
What is ignored or neglected by the media -- but will be studied by historians? a thread by George Mack
Honest as the data is wrong
Professor Francesca Gino, "who focuses on why people make the decisions they do at work" is on administrative leave from Harvard Business School and had some papers retracted because it looks like data were faked. Guardian. Or with more detail: Vox. Irony alert: the seemingly dishonest research was about how honest are regular people. [more inside]
He Could Have Gotten Richer Faster Working in Finance
Veteran gamblers know you can’t beat the horses. There are too many variables and too many possible outcomes. Front-runners break a leg. Jockeys fall. Champion thoroughbreds decide, for no apparent reason, that they’re simply not in the mood ... Play for long enough, and failure isn’t just likely but inevitable—so the wisdom goes. “If you bet on horses, you will lose,” says Warwick Bartlett, who runs Global Betting & Gaming Consultants and has spent years studying the industry ... What if that wasn’t true? from The Gambler Who Cracked the Horse-Racing Code [Bloomberg; ungated]
Happy 40m!
Statistics Canada has a page where they extrapolate from census data, births, deaths, immigration, etc. and smooth it out to estimate the exact population of the country at any moment.
The country will, by this metric, hit 40 million in the next half hour or so.
Merging the Negro Leagues into MLB stats has not been seamless
MLB’s plan to integrate Negro League numbers and statistical legacies with its own remains years from completion. More than two years after its announcement, MLB is still in the initial phase of the project: data acquisition. (archive.today link)
The end of the road for FiveThirtyEight?
After Disney laid off more than half of FiveThirtyEight's 35 staff members earlier this week, editor-in-chief Nate Silver said on Twitter that he was unlikely to renew his contract when it expires this summer. [more inside]
14% FALL TO DEATH
Elden Ring Stats Reveal What Bosses Killed Players The Most [Bandai Namco] “Over 20 million people played Elden Ring. Now, a year after FromSoftware’s open-world Soulsborne released, we finally know which part of the game destroyed players the most. It was Malenia of course. Players attempted to fight her 329 million times, [...] Publisher Bandai Namco released that stat along with a bunch of other new ones to highlight the 2022 GOTY’s first anniversary. While the infographics don’t say exactly how many times Malenia killed players, it’s a safe bet that her being the most-attempted boss means she was also responsible for the most failed attempts. [...] The number one cause of death, meanwhile, was enemies and NPCs. That’s how 69 percent of players bit the dust (nice). Player-vs-player kills were only a measly 2 percent. Most summons were for co-op as well.” [via: Kotaku]
Canada at a Glance 2022 | Coup d'œil sur le Canada 2022
Prepared by Statistics Canada, Canada at a Glance presents current statistics on Canadian society. Subjects covered in Canada at a Glance include population, education, health, and others to provide a statistical portrait of Canada. [more inside]
How many apples?
“race scientists” and neo-Nazis
Choose Wisely
It's a simple concept: Given a choice between two random movies, which one do you like best? That's the driving force behind Flickchart, an addictive review site for movie lovers. Faced with two posters, click the one for the title you prefer (weeding out the ones you haven't seen). Good! Now do it again. And again. And again. With each new face-off, Flickchart perfects a growing list of your favorite films -- and there can be no ties. This leads to some difficult dilemmas: Star Wars or Raiders of the Lost Ark? Citizen Kane or The Godfather? WALL-E or Spirited Away? But you needn't struggle alone -- Flickchart is also social. By drawing on the data of tens of thousands of fellow users, you can create remarkably specific lists: Martin Scorsese's Best Period Films. The Best Road Movies of the 1980s. The Worst Movies of All Time. If you rank enough films, you can generate interesting personalized charts, like "Your Favorite Musicals" or "The Best Movies You Haven't Seen." These filters carry over to the ranking system, letting you judge nothing but Horror movies or 1960s movies or unranked movies or movies from your top 100. You can also comment on popular match-ups, lending your voice to contentious debates like Ghostbusters vs. Back to the Future or Jaws vs. Predator. Not a movie fan? Don't worry. Flickchart will be expanding into books, games, and music soon at some point. Until then, you can give your own data sets the Flickchart treatment using this tool from Gwern Branwen. [more inside]
convenient, flexible and yet entirely impirical
Statistician David Cox, known among other things for the proportional hazards model, died last week [paywalled WSJ].
"a kind of tangible curiosity that statistics encourages"
"one of my co-workers had a tortoise called Pietro who could supposedly predict the weather ... I pondered how one might go about rigorously evaluating this claim". Conner Jackson collects and analyses data on the accuracy of weather prediction by Pietro the tortoise. The Royal Statistical Society explains why the article won their early-career writing award. Pietro's Instagram account. Other weather-forecasting tortoises include Herman in New Zealand.
Dance of the p values
"You could play this at parties, develop a dance for the dance of the p values and reflect on how ridiculous it is that p values are right at the centre of our thinking about drawing conclusions from research."
These data are not just excessively similar. They are impossibly similar
Evidence of Fraud in an Influential Field Experiment About Dishonesty is a blog post at Data Colada 🍹 where researchers uncovered dishonest data meddling in a PNAS-published paper about... dishonesty. [more inside]
Simpson's Paradox
If you look at Covid data from Israel across all ages, vaccine efficacy against severe disease is 67.5%. But if you break it down by age it turns out to be significantly higher: for those under 50 it's 91.8%, and those over 50 it's 85.2%. What's going on? "Simpson’s paradox arises when there are 'lurking variables' that split data into multiple separate distributions." [more inside]
The Abstract Representation of Things
Combinators and the Story of Computation - "The idea of representing things in a formal, symbolic way has a long history... But was there perhaps some more general and fundamental infrastructure: some kind of abstract system that could ultimately model or represent anything? Today we understand that's what computation is. And it's becoming clear that the modern conception of computation is one of the single most powerful ideas in all of intellectual history—whose implications are only just beginning to unfold. But how did we finally get to it? Combinators had an important role to play, woven into a complex tapestry of ideas stretching across more than a century." (also btw The Nature of Computation previously) [more inside]
But in the [surrogate] end[point], it doesn't even matter
FDA approves Alzheimer's drug Aduhelm (sl arstechnica). The approval is controversial because the FDA admits the drug has not been shown to be effective against Alzheimer's and carries a 40% risk of brain swelling. The approval is based instead on efficacy in reducing plaque and the reasoning that this is likely to have some positive effect. [more inside]
"Scented candles: An unexpected victim of the COVID-19 pandemic"
On Twitter, Terri Nelson noted the proliferation of complaints on Yankee Candle's website about the lack of scent in their scented candles. Kate Petrova responded (Threadreader version) with a tweet thread analyzing Amazon reviews for scented candles before and during the pandemic, and the results are interesting (and very easy to understand). [more inside]
A full moon American fever dream
CNN reports the coronavirus pandemic does not appear to be slowing down anytime soon in the United States, according Michael Osterholm, head of the Center for Infectious Disease Research and Policy at the University of Minnesota in Minneapolis, and 23 states are reporting a rise in new cases compared to the previous week, according to data from Johns Hopkins University. NBC News reports more than 120,000 people have now died from coronavirus in the US, and over 2.2 million people have been infected across the country, while CNBC reports hospitalizations due to Covid-19 were growing in 14 states as of Sunday, according to a CNBC analysis of data compiled by the Covid Tracking Project. Meanwhile, The Guardian reports that Trump, who hosted a racist campaign-style rally in Oklahoma over the weekend, refuses to wear a mask in public, regularly contradicts health officials, and mischaracterizes the scientific landscape. [more inside]
The coronavirus invades Trump country
Axios reports that according to a recent poll, Republicans are less worried about the coronavirus than Democrats or independents, even as it spreads out from primarily urban areas into suburban and rural Republican-leaning areas. [more inside]
What does the replication crisis mean for psychotherapy?
The evidence for evidence-based therapy is not as clear as we thought (Aeon): "Is the credibility of the evidence for ESTs [empirically supported treatments] as strong as that designation suggests? Or does the evidence-base for ESTs suffer from the same problems as published research in other areas of science?" Of the 70 ESTs listed by the Society of Clinical Psychology, researchers found that 20% performed well, 30% had mixed results, and 50% had subpar outcomes. [more inside]
Milestones: journeying through adulthood
A blog post from the Office of National Statistics about the ages at which life events happen for adults: women having their second child, highest earning period, divorce, providing unpaid care, becoming a grandparent, stopping working, and greatest happiness period.
Monetizing Bowser
Although everyone knows the true value of a dog's life is incalculable, a recent study in the Journal of Benefit-Cost Analysis attempts to quantify what the authors call the “value of statistical dog life,” or VSDL, following a widely accepted method for putting a price tag on a typical human life. (Herald Net | WaPo)
The Food Flow Model, a web of connections across the continental U.S.
All Americans, from urban to rural are connected through the food system. Consumers all rely on distant producers, agricultural processing plants, food storage like grain silos and grocery stores, and food transportation systems. [...] Now, residents in each county can see how they are connected to all other counties in the country via food transfers. Overall, there are 9.5 million links between counties on our map. The first map of America’s food supply chain is mind-boggling -- Most of our food is moved across great distances—and through many different forms of transit—before it reaches our plates. (Fast Company write up on Food flows between counties in the United States, an open access scholarly article) [more inside]
H is for Hypersphere
Joseph Nebus is (once again) blogging about mathematics from A-Z: So far this fall, Abacus and Buffon's Needle. You can follow along with the Fall 2019 series as it appears (at about two topics per week), and you can browse through past years' mathematical A-Z's (there's a partial index of old topics for your convenience).
Great. Now I'm even uncertain about how uncertain I should be.
An uptick in the inability to reproduce recent scientific conclusions, aka the Replication Crisis, is calling into question the practice of using significance testing to make inferences. Some think we should use Bayes' Theorem instead to make inferences and quantify uncertainty.
The slow-moving oppression of a sprawling government bureaucracy
How USDA distorted data to conceal decades of discrimination against black farmers: "Here, we present Nathan Rosenberg and Bryce Stucki’s two-year investigation into the ways agricultural census data were distorted to depict a fictional renaissance in black farming. This false narrative inflated USDA’s record on civil rights and further hurt black farmers—the very people the department claimed it had made historic efforts to help." [CW: racist language.]
Did well playing an unfair game
Elbertie Foudray and the Adventure of Life: I never set out to write a book about the dream in 1940 of devising the most scientific census ever. Instead, I went looking for an obscure census mathematician named Elbertie Foudray and, to my surprise, discovered the census. This is a census story all about chance and gender (as well as power) and it began with an effort to recreate a very odd paper machine. Dan Bouk, a historian at Colgate University, shares a story of a woman who played a critical role in the development of the U.S. Census in the early 20th century, but was never properly recognized for it in her lifetime.
census, eugenics, computers
"A racial category for Chinese was added after railroad companies began importing cheap, exploitable laborers from China. Categories for “mulatto” came after the abolition of slavery caused a panic about the dangers of racial mixing. Questions about mental health and race were first added at the behest of a Southern senator right before the outbreak of Civil War. The results seemed to show that free blacks living in Northern states were on average 11 times more likely to be insane than Southern blacks living in slavery. Such questionable statistics were taken up by Southern politicians to bolster racist theories and argue against abolition." [A longread on Hollerith, eugenics in the US, Nazi Germany and more]
Statistical significance is bad for science, p<.05
Scientists rise up against statistical significance. In a comment piece published in the March 20 issue of the journal Nature, zoologist Valentin Amrhein, epidemiologist Sander Greenland, statistician Blake McShane, and over 800 co-signatories argue that the time has come to abandon the use of statistical significance in science. [more inside]