Aligning AI
with what really matters

We’ve researched meaning and human values for a decade.

As an organization, we apply this expertise to AI alignment and post-AGI futures.

AI Alignment

We believe intelligence is not enough to ensure a good future. In order to ensure AI systems are aligned towards what’s good for humanity, we need AI that’s not just intelligent, but wise.

  • We think of practical wisdom as knowing what the right thing to do is in a particular situation.

    Our understanding of values, ethics and wisdom is building on the work of philosophers like Charles Taylor and Ruth Chang.

  • Wisdom is dispersed amongst us, rather than concentrated in a small elite. People who have lived through unique experiences, forcing them to grapple with unique moral situations, will have the wisest values for those contexts. Not armchair philosophers.

    To surface and elicit wise values from a population, we designed Moral Graph Elicitation (MGE), a process for fine-tuning models on a crowd-sourced notion of wisdom that we call a moral graph.

  • Our first step towards Wise AI is fine-tuning models on moral graphs. We have confirmed that our values elicitation process works, in collaboration with OpenAI, and have seen convergence in values across the political spectrum.

    We are currently working on improvements to existing alignment techniques based on MGE.

Post-AGI futures

AGI will change everything, even if alignment is solved. To help society navigate these changes, we’re researching new LLM/market mechanisms, institutions and narratives, centered around human flourishing.

  • We envision a future where AI systems deeply understand our sources of meaning, and allocate resources and suggest trades in order to make our lives as meaningful as possible.

  • Currently, markets, democracies and technology all understand human wants in terms of surface-level preferences rather than deeper values.

    As a consequence, these systems optimize for clicks and likes rather than what would make our lives truly meaningful.

    AI can either reverse or amplify this trend.

  • We’re researching LLM-based market alternatives, powered by a rich understanding of people’s values.

    We’re also hosting events where people articulate their values and are introduced to our vision, giving them a teaser of what the future could feel like, if we succeed.

Research