Simple Models

So, if you float around astrobiology or science fiction spaces (or the very round, entirely circle-ish overlap between the two) you’ve seen this:

N = R~star~ × f~p~ × n~e~ × f~l~ × f~i~ × f~c~ × L

or heard of the Drake Equation by name. Astrobiologist Frank Drake proposed it in 1961 as a simple model to predict N, the number of extraterrestrial civilizations from which humans could expect to observe signals. Here are the terms, by the way:

TermExplanation
Nthe number of potential communicating partner civilizations
R~star~the average rate of star formation
f~p~the fraction of stars with planets
n~e~the average number of life-capable planets per planet-having star
f~l~the fraction of life-capable planets where life emerges
f~i~the fraction of life-having planets that develop civilizations

Easy, right? Multiply everything on the right together and you get a number. Boom, easy math.

You may be aware of some of the famous issues already. One is that the value of N is famously, observably 0. Now that’s not necessarily an issue with the model, such as it is. It’s entirely possible that one of the numbers or fractions is so small that 0 is the entirely accurate value for N, and in fact that was one of the main motivators beyond crystallizing thought patterns into this equation in the first place. Why 0? Where are all the aliens?

Now, this generates serious scientific discussion, philosophical what-if sessions, and the premises of hundreds of science fictions. All well and good, but the other thing is that the simplicity of slapping a multiplicative term in an equation masks how knowable a number actually is as well as how known it is (not the same thing). Think about f~i~, the fraction of life-having planets that develop civilizations. What can you do with that other than shrug? We know it’s not 0 because we’re here and the insert joke about cultural product exists, but, beyond that?

Estimating each term is also not of equal difficulty. While the civilizationability of life is a question mark, we (humanity) have some pretty solid brackets around R~star~. The Milky Way has about 100 million stars, or up to 400 million. That might sound bad, but it’s not nothing to dial in orders of magnitude and say it’s not 1 million and not 1 billion.

Worse is quibbles with the structure of the Drake equation: are these terms the only important ones, is the structure misleading, is it overly simplistic? The answer to the latter is most assuredly “yes.” It is too simple, which Frank Drake would admit. It frames a discussion though, gives some concrete points to discuss, and throws up some evaluable metrics.

Incomplete Criteria

I have been charmed by the MAGIC criteria for years and have to give credit to my sister for introducing them to me while she was studying as a nurse midwife. The MAGIC criteria were introduced by psychologist Robert Abelson in 1995 in Statistics as Principled Argument. They were intended to frame discussions on research results framed in statistical terms. Here they are:

TermExplanation
MagnitudeHow large an effect was measured?
ArticulationHow precisely stated is the effect?
GeneralityHow broadly is the effect applicable?
InterestingnessHow much can the effect prompt change?
CredibilityHow believable is the effect?

The criteria are meant to help researchers and communicators speak intelligently about results and qualitatively compare results to one another.

“Interestingness,” in particular, stands out as both important and hard to pin down. From Abelson:

For a statistical story to be theoretically interesting, it must have the potential, through empirical analysis, to change what people believe about an important issue.

I like that. It yanks the consideration out of the potentially sterile world of p-values into what you do with knowledge. It could admittedly be a few different things, though. Is it about being surprising? Is it about focusing on topics people care about? Is it about focusing on “vegetables” topics that are really important to people? If that last one is true, it takes me back to a past doing some news reporting where you know people really need to pay attention to school board elections but they really want to pay attention to celebrity gossip.

We could step through each of the criteria and talk about vagueness, or at least multiple metrics for defining them.

MAGIC doesn’t have the same amount of reach or discourse surrounding it as the Drake equation (less interestingness, maybe), but it shares some strengths and weaknesses. Simple, memorable, and definitely includes topics of importance, regardless of interpretation.

Accessible Tools

In 2004, California State University Librarian Sarah Blakeslee introduced the world to CRAAP, another backronym like MAGIC, this one for evaluating source quality. Here’s another table:

TermExplanation
CurrencyHow recent is the source?
RelevanceHow directly does the source meet needs?
AuthorityHow knowledgeable and trustworthy is the source?
AccuracyHow true is the source’s information?
PurposeWhat motivated the creation of the resource?

Wonderful! I’m sure students over the last 20 years have had fun labeling sources CRAAP or its opposite. We’re moving away from the concrete again here toward something eminently empirical. There’s a pragmatism here I could see appealing to librarians I have known.

And yet…and yet. One could quibble about each and every one of these, maybe not currency so much, but the other four to be sure. Even if interlocutors meet on an even playing field, in good heart and temper, each of these could become a battle.

What value do they have, then? Simply stating that “relevance” is under consideration can open up the topic. Are there degrees of relevance? Are there missing criteria?

Like MAGIC, like the Drake equation, CRAAP is a flag in the sand saying, heroically “here’s some important stuff, here’s a starting point for discussion.” None of these were meant to be the end of their respective topics and, as tools, they each have a very chunky resolution.

What they do, though, is crystallize some important factors, yanking them down out of the aether into conversation to be handled, maybe roughly. They can handle the rough use, though, and can change as needed.

Take Ladder

The Internet is absolutely full to the brim with takesmiths, folks whose business is generating opinions. Not predictions, not positions, but opinions, where the factor being optimized for is immediate interest (engagment or rage bait, pick your side of the coin).

That’s fine for what it is, though I’d aspire to do a bit better whenever possible. I have my own gut checks and takes, but it would be good to call them out as such, and not pretend they are based on more than they truly are. Provisionally, here’s a backronym: PARS.

TermExplanation
PerspectiveWhere do I come from when I talk about this? What are my biases? What are my baked in prejudices for good or ill?
Amount of EvidenceIs there much evidence in the world for this? Have I included the evidence that matters?
Relevance of EvidenceHow directly does included evidence relate to the topic?
Strength of EvidenceIs the evidence high quality by whichever criteria applies most naturally?

I played around with different versions of this: SPAR or RAPS. I thought about “Expertise” rather than “Perspective,” so EARS or ARSE. I would have like that last one to work. Versions with both E and P: PEARS, SPEAR, SPARE, or PARSE.

Playing with these really made me think about the purpose of these simple models. PARSE is in line with what I’d like to do here, promote understanding. For what it’s worth, in my mind I’m pronouncing PARS identically to “parse.” I didn’t think it was worth the extra length to include a marginally distinct E, though and I think we’ve all heard enough backronyms that are eye-rollingly forced in their execution.

SPAR was right there, but there are those aggressive undertones. I have little faith in the power of debate. I’m trying to think and write in an open-handed, open-hearted style, not a pugilistic one. SPAR also splits up the order in which I’d like to think through these topics, where I think it makes the most sense to consider my own qualifications and starting point first.

I’d aspire to rate myself and my idea presentation whenever it makes sense to do so and be honest about how high up that ladder from raw take to absolute truth I can climb. If you read, you may see something like this:

PARSScore
P◼◼◻
A◼◼◻
R◼◼◼
S◼◻◻

So eight out of a potential twelve, by my own assessment, where 0-3 filled boxes articulates something like the range from none to a little to some to a lot.

Is this set in stone? Of course not. It’s a good practice, but I reserve the right to modify and change my mind in the future, as we all should, with receipts shown. Is it an ideal model? Of course not, but it is a model, a simple one, a flag in the sand to say we aim to be correct and to grow correct-er over time. That we want to know more, and know better.