Hybrid Intelligence

Experiments done in the context of the IARPA HFC tournament

How do we design systems that use human and machine intelligence?

Variational Autoencoders

A simple idea. Use VAEs to represent in low dimensions the correlation existing between people's judgment. Introduce bots to decorrelate such judgments.

DiversityXGroup Size

We experimentally manipulated group diversity and team size. We found that diversity is good for large teams but not for small ones.

Collective Payoffs

Individual incentives make us individually better but collectively worse. Collective incentives make us individually worse but collectively better. Who would have guessed?

...

...

Every Brutal Choice Has Elegance

The HFC program seeks to develop and test hybrid geopolitical forecasting systems. These systems will integrate human and machine forecasting components to create maximally accurate, flexible, and scalable forecasting capabilities. Human-generated forecasts may be subject to cognitive biases and/or scalability limits. Machine-generated (i.e., statistical, computational) forecasting approaches may be more scalable and data-driven, but are often ill-suited to render forecasts for idiosyncratic or newly emerging geopolitical issues. Hybrid approaches hold promise for combining the strengths of these two approaches while mitigating their individual weaknesses. Performers will develop systems that will integrate human and machine forecasting contributions in novel ways. These systems will compete in a multi-year competition to identify approaches that may enable the Intelligence Community (IC) to radically improve the accuracy and timeliness of geopolitical forecasts.


Economy, politics, what Erdogan has eaten for breakfast.

Forecasting is an interesting problem. The issue with forecasting is that it is hard. And geopolitical forecasting is even harder. It's not like, hey, what do you think the weight of *YOUR FAVOURITE BOVINE* is? It's more like, hey, Will Fidesz and KDNP win 133 or more seats in Hungary’s upcoming parliamentary election? You are trying to predict the probability of events of global relevance. You are trying to compress into a single number between 0 and 1, a lot of information. Economy, politics, what Erdogan has eaten for breakfast.


Develop a system that naturally blends humans and machines to augment. Humans are good at human stuff, like jumping to conclusions from very little data. Machines are good at doing machine stuff, like using godzillions of observations before making any sense. How would you even design your system? Do you have machines presenting summarized data to the humans? Do you have humans organizing data for the machine? Do you have machines and humans making decisions together? Or in parallel? The possibilities are limitless.