Full Width [alt+shift+f] Shortcuts [alt+shift+k]
Sign Up [alt+shift+s] Log In [alt+shift+l]
80
Here’s another installment in Data Q&A: Answering the real questions with Python. Previous installments are available from the Data Q&A landing page. sample_size Sample Size Selection¶ Here’s a question from the Reddit statistics forum. Hi Redditors, I am a civil engineer trying to solve a statistical problem for a current project I have. I have a pavement parking lot 125,000 sf in size. I performed nondestructive testing to render an opinion about the areas experiencing internal delimitation not observable from... Read More Read More
9 months ago

More from Probably Overthinking It

Algorithmic Fairness

This is the last in a series of excerpts from Elements of Data Science, now available from Lulu.com and online booksellers. This article is based on the Recidivism Case Study, which is about algorithmic fairness. The goal of the case study is to explain the statistical arguments presented in two articles from 2016: Both are about COMPAS, a statistical tool used in the justice system to assign defendants a “risk score” that is intended to reflect the risk that they... Read More Read More

a week ago 13 votes
Confidence In the Press

This is the fifth in a series of excerpts from Elements of Data Science, now available from Lulu.com and online booksellers. It’s based on Chapter 16, which is part of the political alignment case study. You can read the complete example here, or run the Jupyter notebook on Colab. Because this is a teaching example, it builds incrementally. If you just want to see the results, scroll to the end! Chapter 16 is a template for exploring relationships between political... Read More Read More

4 weeks ago 39 votes
Political Alignment and Outlook

This is the fourth in a series of excerpts from Elements of Data Science, now available from Lulu.com and online booksellers. It’s from Chapter 15, which is part of the political alignment case study. You can read the complete chapter here, or run the Jupyter notebook on Colab. In the previous chapter, we used data from the General Social Survey (GSS) to plot changes in political alignment over time. In this notebook, we’ll explore the relationship between political alignment and... Read More Read More

a month ago 50 votes
Reject Math Supremacy

The premise of Think Stats, and the other books in the Think series, is that programming is a tool for teaching and learning — and many ideas that are commonly presented in math notation can be more clearly presented in code. In the draft third edition of Think Stats there is almost no math — not because I made a special effort to avoid it, but because I found that I didn’t need it. For example, here’s how I present... Read More Read More

a month ago 47 votes
Young Americans are Marrying Later or Never

I’ve written before about changes in marriage patterns in the U.S., and it’s one of the examples in Chapter 13 of the new third edition of Think Stats. My analysis uses data from the National Survey of Family Growth (NSFG). Today they released the most recent data, from surveys conducted in 2022 and 2023. So here are the results, updated with the newest data: The patterns are consistent with what we’ve see in previous iterations — each successive cohort marries... Read More Read More

a month ago 51 votes

More in science

An update, + a paper as a fun distraction

My post last week clearly stimulated some discussion.  I know people don't come here for political news, but as a professional scientist it's hard to ignore the chaotic present situation, so here are some things to read, before I talk about a fun paper: Science reports on what is happening with NSF.  The short version: As of Friday afternoon, panels are delayed and funds (salary) are still not accessible for NSF postdoctoral fellows.  Here is NPR's take. As of Friday afternoon, there is a new court order that specifically names the agency heads (including the NSF director), saying to disburse already approved funds according to statute.   Looks like on this and a variety of other issues, we will see whether court orders actually compel actions anymore. Now to distract ourselves with dreams of the future, this paper was published in Nature Photonics, measuring radiation pressure exerted by a laser on a 50 nm thick silicon nitride membrane.  The motivation is a grand one:  using laser-powered light sails to propel interstellar probes up to a decent fraction (say 10% or more) of the velocity of light.  It's easy to sketch out the basic idea on a napkin, and it has been considered seriously for decades (see this 1984 paper).  Imagine a reflective sail say 10 m\(^{2}\) and 100 nm thick.  When photons at normal incidence bounce from a reflective surface, they transfer momentum \(2\hbar \omega/c) normal to the surface.  If the reflective surface is very thin and low mass, and you can bounce enough photons off it, you can get decent accelerations.  Part of the appeal is, this is a spacecraft where you effectively keep the engine (the whopping laser) here at home and don't have to carry it with you.  There are braking schemes so that you could try to slow the craft down when it reaches your favorite target system. A laser-powered lightsail (image from CalTech) Of course, actually doing this on a scale where it would be useful faces enormous engineering challenges (beyond building whopping lasers and operating them for years at a time with outstanding collimation and positioning).  Reflection won't be perfect, so there will be heating.  Ideally, you'd want a light sail that passively stabilizes itself in the center of the beam.  In this paper, the investigators implement a clever scheme to measure radiation forces, and they test ideas involving dielectric gratings etched into the sail to generate self-stabilization.   Definitely more fun to think about such futuristic ideas than to read the news. (An old favorite science fiction story of mine is "The Fourth Profession", by Larry Niven.  The imminent arrival of an alien ship at earth is heralded by the appearance of a bright point in the sky, whose emission turns out to be the highly blue-shifted, reflected spectrum of the sun, bouncing off an incoming alien light sail.  The aliens really need humanity to build them a launching laser to get to their next destination.)

5 hours ago 3 votes
Chatbot Software Begins to Face Fundamental Limitations

Recent results show that large language models struggle with compositional tasks, suggesting a hard limit to their abilities. The post Chatbot Software Begins to Face Fundamental Limitations first appeared on Quanta Magazine

yesterday 3 votes
Links in Progress: We can still build beautifully

A tour of interesting developments built in the last two decades

yesterday 3 votes
The Value of Foreign Diplomas

Is that immigrant high-skilled or do they just have a fancy degree?

yesterday 9 votes
Incorruptible Skepticism

Everything, apparently, has a second life on TikTok. At least this keeps us skeptics busy – we have to redebunk everything we have debunked over the last century because it is popping up again on social media, confusing and misinforming another generation. This video is a great example – a short video discussing the “incorruptibility’ […] The post Incorruptible Skepticism first appeared on NeuroLogica Blog.

2 days ago 2 votes