Using Technology to Deepen Democracy, Using Democracy to Ensure Technology Benefits Us All

Friday, October 15, 2010

"In Heaven, everything is fine. In the future, not so much."

I take back all the modestly annoyed things I said about Annalee Newitz's conversation with James Hughes here. Now she's written a lovely and sensible anti-singulartiarian piece over at io9, and, as the song says, "I'm in love again, and I can't rise above it, I'm in love again, and I love, love, love it," or, at any rate, I'm pleased to be pleased. Read the whole thing, of course, and this taste tells you why:
If you do live forever, it will only be by turning into a creature so unlike you that "living forever" won't mean the same thing anymore. And so what if you can control atoms when your giant spaceships keep getting subatomic particle infections that you can't contain? Unless you plug your brain into a bliss program, you're going to have to deal with all that evidence from history that culture-changing inventions don't ever behave the way you expect them to. And, by the way, neither does culture. All I'm saying is that if you're looking for a narrative that explains the future, consider this: Does the narrative promise you things that sound like religion? A world where today's problems are fixed, but no new problems have arisen? A world where human history is irrelevant? If yes, then you're in the fog of Singularity thinking.




1 comment:

jimf said...

Annalee Newitz writes (in her io9 article):

> Once I met a Singularity zealot who claimed that eating
> potato chips after the Singularity would induce sublime
> ecstasy. Our senses would be so heightened that we could
> completely focus our whole attention on the ultimate chippiness
> of the chip.

So the Singularity, according to this zealot, would be
sort of like smoking pot, only all the time.

More seriously, Newitz writes:

> If you do live forever, it will only be by turning into
> a creature so unlike you that "living forever" won't mean
> the same thing anymore.

Back when I participated in some of the on-line >Hist
venues (10 years and more ago now), at a time when I was
inclined to take some of their techo-hype, especially that
surrounding artificial intelligence, more seriously than
I do now, I tried to make similar points. I mentioned,
for example, that even if "superintelligences" were
grown from human progenitors rather than being created
ex nihilo, if they were indeed to be (as Hugo de Garis claims
they will be) "trillions of times more intelligent"
(whatever exactly that means) than a human being, then
in what sense could the "seed" human's personal identity be
preserved in the course of such a developmental trajectory?
Analogously, in what sense can a full-grown human being be said to
have the same "personal identity" (in the usual sense of
the phrase) as the original zygote from which it grew?
In what sense is a complete human being the "same person"
as one of a human body's skin cells?

I found it very telling indeed that I was shouted down
very strongly for making such observations. The cryonics
enthusiasts who want to be resurrected and then uploaded
so that they can personally **become** "superintelligences"
place such an overweening value on their own personal
continuity (as well all do to some degree, because we can't help it,
not because we think it is, or should be, a sign of
"rationality" or a law of the universe) that it screamed
**narcissism** to me. It also screamed something akin
to desperate religious belief. It's a big red flag for
**cult**.