Should we automate discussion about utopia?
The air of Inkhaven is increasingly impregnated with thoughts of utopia, the illegible “ought” that comes after every last lump of alignment-problem is chucked into the shredder. What comes after? If we survive the century? There's a lot of stuff out there. “Who’s” stuff is it?
Answers come in shards: we should not go too far in the way of generating masochistic non-sentient house elves; we should not split the sexes; we should not tile what-is with haunted amusement parks.
We're not even certain of these shoulds: every one of these posts was written in reaction to another human's conception of utopia. If humans cannot legibilize the latent shape of “human values” themselves, then the default plan—seemingly getting the AI to do our homework for us—seems increasingly to be our only real bet.
What are the alternatives? How effective is human conversation at pinning down ideal futures? Have either of Bostrom, Alexander, or Hanson, (all diligent priests of Aumann!) changed their minds? Have either of Bjartur or Sacha? If they do change their mind, so what? Harmoniously joining forces to tear open the Overton window has only ever served to feed the maw of Moloch.
Why suggest anything, when every suggestion you make gets hurled in one of two directions, battered bloody until scissor-shaped, and then half the country hates you?
But, but, getting the AI to do our homework is the default plan for everything! Surely to merely gesture at this trick every time, and pin the philosophical weight of the future on the solution to a provincial materialistic problem is going to have some adverse consequences? How long are we expected to keep getting away with this? (Perhaps 2027.)
Waymos are cute, but do we really want a Waymo solving our utopia problem? It feels… cowardly.
If keeping the “indomitable human spirit” intact is our goal—is undercutting it now, before we even move on to the issue of providing a constant stream of it, a good idea? If “courage” is a dimension we want to maintain about the future, is not rolling up our sleeves and solving our own shit the only way we'll find a solution which doesn't silently knock that trait out of us?
What does it even mean to be a species which doesn't shrink away from challenge, wants to maintain that quality about itself forever, and thinks the best way to do so is to offload all work on another intelligence because the problem is “too hard”?
Perhaps this is an overdramatization and actually we're allowed to take an outside-view to our values, allowed to grasp the safer shortcut to achieving our goals, no matter some website's opinion on “what's dignified". To automate discussion about utopia might appear eerie, but if it works, it works!
It's not like anyone will care about the olden day's notion of “dignity”, when the only threat the human courage muscle is flexed against is bio-engineered dragons and absurdly elaborate great-looming-threat ancestor simulations.
“Our ancestors way back in dreamtime were insane to hesitate even one second before automating utopia discussions,” they will say of this post, pinned to the whiteboard as an example of how foolish the 21st century was, “and how simple saving the world ended up being when they realized how silly they were being!”
So, um. Is anyone explicitly working on automating discussion about utopia?