NAS Workshop Proceedings

by | 5 October 2022

On December 17 and 21 of 2021, Executive Director Seth Baum delivered a remote talk called The challenges of addressing rare events and how to overcome them at the workshop Anticipating Rare Events of Major Significance, hosted by the US National Academies of Sciences, Engineering, and Medicine. The proceedings from the workshop can now be found online, and Baum’s remarks from the workshop can be found in Chapter 8, Active Prevention and Deterrence. A summary of his presentation is as follows:

Classic expected utility theory suggests that rare events can be important to address if their consequences are large enough. This presentation discusses several challenges faced by individuals and institutions seeking to address rare events. First, the high degree of uncertainty about rare events can make it difficult to identify which decision options have net positive benefits. Methodological rigor and humility are needed to evaluate the merits of decision options. Second, psychological and institutional biases can cause individuals and institutions to respond inappropriately to rare events, often by letting rare events go unadressed or under-addressed. In some cases, it can be possible to motivate individuals and institutions to hold better views on rare events and act accordingly. However, other approaches can be helpful, such as by leveraging synergies between actions to address rare and frequent events. These points are illustrated through discussion of pandemics and nuclear war.

Slides from the workshop can be found online.

Author

Recent Publications

Climate Change, Uncertainty, and Global Catastrophic Risk

Climate Change, Uncertainty, and Global Catastrophic Risk

Is climate change a global catastrophic risk? This paper, published in the journal Futures, addresses the question by examining the definition of global catastrophic risk and by comparing climate change to another severe global risk, nuclear winter. The paper concludes that yes, climate change is a global catastrophic risk, and potentially a significant one.

Assessing the Risk of Takeover Catastrophe from Large Language Models

Assessing the Risk of Takeover Catastrophe from Large Language Models

For over 50 years, experts have worried about the risk of AI taking over the world and killing everyone. The concern had always been about hypothetical future AI systems—until recent LLMs emerged. This paper, published in the journal Risk Analysis, assesses how close LLMs are to having the capabilities needed to cause takeover catastrophe.

On the Intrinsic Value of Diversity

On the Intrinsic Value of Diversity

Diversity is a major ethics concept, but it is remarkably understudied. This paper, published in the journal Inquiry, presents a foundational study of the ethics of diversity. It adapts ideas about biodiversity and sociodiversity to the overall category of diversity. It also presents three new thought experiments, with implications for AI ethics.

Climate Change, Uncertainty, and Global Catastrophic Risk

Climate Change, Uncertainty, and Global Catastrophic Risk

Is climate change a global catastrophic risk? This paper, published in the journal Futures, addresses the question by examining the definition of global catastrophic risk and by comparing climate change to another severe global risk, nuclear winter. The paper concludes that yes, climate change is a global catastrophic risk, and potentially a significant one.

Assessing the Risk of Takeover Catastrophe from Large Language Models

Assessing the Risk of Takeover Catastrophe from Large Language Models

For over 50 years, experts have worried about the risk of AI taking over the world and killing everyone. The concern had always been about hypothetical future AI systems—until recent LLMs emerged. This paper, published in the journal Risk Analysis, assesses how close LLMs are to having the capabilities needed to cause takeover catastrophe.

On the Intrinsic Value of Diversity

On the Intrinsic Value of Diversity

Diversity is a major ethics concept, but it is remarkably understudied. This paper, published in the journal Inquiry, presents a foundational study of the ethics of diversity. It adapts ideas about biodiversity and sociodiversity to the overall category of diversity. It also presents three new thought experiments, with implications for AI ethics.