Forecasting, Eucatastrophe and Wicked Problems

Sci Fi author Karl Schroeder has been guest blogging on Charlie Stross’ blog about forecasting the future, which of course is total must-read material for me. He had me just by saying that he has a Master’s degree in Strategic Forecasting and Innovation. This caused a spate of actual academic envy in me, a very rare phenomenon in this universe, I can assure you.

For those who think the future is a completely unknowable thing, or who have the inkling of a hope that maybe we can somehow prepare for or anticipate the future, this is also must-read. Money quote:

We seem to do everything about our future except try to design it. It’s a funny thing: nobody ever questions your credentials if you predict doom and destruction. But provide a rosy picture of the future, and people demand that you justify yourself. Increasingly, though, I believe that while warning people of dire possibilities is responsible, providing them with something to aspire to is even more important. The foresight programme has given me a lot of tools to do that in a justifiable way, so I might as well use them.

So say we all. Unless some evil people also use it, right? Well, then…

He also has a great post on Eucatastrophe, which is kind of the opposite of a catastrophe, when everything goes right. Wouldn’t it be fun to pull that off by forecasting the future? And if you really want your brain to hurt in the other direction, he discusses ‘wicked‘ problems that we as a human race seem to be utterly failing to solve for pretty mundane reasons.

I should say something intelligent about all of this, but it’s too late at night (for me) to put coherent, intelligent thoughts together. Suffice it to say that these ideas play a central role in my sci-fi thriller novel about saving the world through shaping the future by studying projections of it.

The science behind “The School Counselor”

The science fiction idea at the heart of my short story “The School Counselor” was predicting students’ futures in a quantitative manner. This is more feasible in the near future than you may think, if we adapt two tools from other fields: market research and microsimulation models, and invent a third: quantitative career planning.

Market Research

School counselors build their own market segmentation naturally by seeing hundreds of students over a number of years. However, the market research industry has turned segmentation into a science (granted, it’s proprietary and lucrative, so is not much in public view). Our data-rich society already segments students into marketing niches because they are two-legged spending machines – why not sell/supply that data to counselors so students could better plan their futures? (If someone had showed me the PRIZM segments back in high school, it would have made my planning a lot clearer.)

Actually, the Department of Education surveys high school students beyond graduation, but those quantitative findings are mainly used for academic studies and not to inform student choices. (Disclosure: I used DoE’s longitudinal NELS88 for a grad school project.)

Microsimulation Models

Microsimulation model methodology suggests treating the future consequences of a student’s decision as missing data and imputing it from the experiences of former students in the same segment. This could be done by building an equation based on the experiences of everyone in the segment, or by finding only the ‘nearest-neighbor’ and assume that their exact experience would occur to the student if they chose that path. To do the predictions in the story, especially predicting a student’s future social life, either approach would require a lot more research on personality types, social networks, and socioeconomic indicators. So someone ought to get on that.

Quantitative Career Planning

School counselors offer career planning now, of course, but they can only dream of saying that a student’s chance of career success would be X% with a given approach, Y% with that approach and better grades, and Z percentage points higher if the student had better grades and did a related internship before college. To do this, one would need in-depth industry profiles that tied success to various explanatory factors the student has control over. This is doable, it seems, and could be a profit center for the market research folks or the college prep folks if they could convince wealthy parents that this would be a leg up for their kids. (Hint, hint, big corporations.)

An example

Consider a rebellious student who dies her hair blue to piss off her middle-class parents: the market research data show that 68% of girls generally similar to her eventually become suburban middle-class moms, 25% are childless artists in a big city, and 7% are in jail (they like bad boys). But the microsimulation projections based on her segment show that 95% of girls just like her tend to follow in the footsteps of older relatives with careers in art, film, design, or fashion. Maybe she’ll see that projection and tack towards the suburban mom track if she wants children, or maybe she’ll want to know how to nudge that 95% to 99%, or maybe try to bridge the two outcomes.

If I have any of the science wrong, or am missing a key research finding, please comment and let me know. Microsimulation modeling of my segment suggests I’m only perfectly correct 22.3% of the time.