Aligning AI
with what really matters
We are experts in meaning and human values.
As a research organization, we apply this expertise to two urgent problems.
AI Alignment
We believe intelligence is not enough to ensure a good future. In order to solve alignment, we need Wise AI – systems that are not just intelligent, but morally astute.
-
We define wisdom as knowing what the right thing to do is in a particular situation. Knowing which goals to pursue in the first place.
Our understanding of values, ethics and wisdom is building on the work of philosophers like Charles Taylor and Ruth Chang.
-
Wisdom is dispersed amongst us, rather than concentrated in a small elite. People who have lived through unique life experiences, forcing them to grapple with unique moral situations, will have the wisest values for those contexts.
To surface and elicit wise values from a population democratically, we built Democratic Fine-Tuning, a process that fine-tune models on a democratically sourced notion of wisdom that we call a moral graph.
-
We believe it is possible to build ASW (Artifical Superwisdom) – systems that are wiser than any one person alive, that can find win-win solutions we didn’t know existed. Perhaps it is even necessary, to face the challenges of the 21st century.
Our work on moral graphs could theoretically be scaled to superwisdom – systems that develops new values, like we do, through “moral self-play”.How to do this in practice is an open research question.
-
Our first step towards Wise AI is democratically fine-tuning models with a moral graph. We have confirmed that our values elicitation process works, in collaboration with OpenAI, and have seen convergence in values across the political spectrum.
Our next step is verifying that the resulting model beats alternatives, like preference-based RLHF and Constitutional AI.
In parallel, we are researching systems that build and extrapolate upon moral graphs independent of human input.
Helping Society Adapt to AI
AI will change everything, even if alignment is solved. To help society navigate all the changes that this will bring to our jobs and our lives, we’re researching new narratives, LLM/market mechanisms, and community structures that put human flourishing at the center.
-
We believe real progress should be about more than economic growth or technological advance, it should be about helping people live more meaningful lives.
For this, we envision a future where LLMs have a granular understanding of people’s sources of meaning, and community-owned models help allocate resources and make recommendations that help people live meaningfully
-
Currently, markets and technology are very misaligned with meaning, leading to economies built on atomization, overconsumption and digital addiction.
We want a future where AI helps us reverse this trend, rather than amplify it.
-
We’ve built an AI community tool that helps people in a group or community articulate their values, and then gives people suggestions for activities to do together.
We host events where people go through this process together, creating a beautiful experience that serves as a teaser for our larger vision.
Research
