Using Technology to Deepen Democracy, Using Democracy to Ensure Technology Benefits Us All

Thursday, April 09, 2009

Roko Oh Noes

Meanwhile, back at Michael's place, a last few tired turns on the dusty dancefloor:

Dale: I don’t think there is any way of talking about this sort of thing that Roko would regard as “fact-based” “rigorous” or not "name calling."

Roko: How do you know? You haven’t even tried yet!

Dale: You have literally just proved my point.

Roko: Ok, so I’ll ask again. Dale, please present me with an argument whose conclusion is “smarter than human AI is impossible” or “smarter than human AI is extremely unlikely to be developed within the next 100 years”.

I’ll give you an example of the sort of thing I am looking for:
1. The human brain is extremely complex
2. Software is really hard to write
3. Any smarter than human AI must be as complex (or more) than the human brain
4. Therefore, no-one will write software that clever in the next 100 years
5. Therefore, smarter than human AI is extremely unlikely to be developed within the next 100 years.

Dale: I have no doubt that this is what you are looking for, inasmuch as there is no version of this formulation that does not confirm the very prejudices at issue. The primary weight in your formulation is borne by the metaphorical usage of the word “smarter” in both 3 and 5 to describe what you take to be on offer when computer programs grow more complex. That’s not a "fact," that’s a figure. The argument relies on framing, not evidence. I disagree that there is anything in the facts to justify the analogy and I believe all the other business about timelines and so on actually functions to distract True Believers from noticing what thin ice you are on when you try to extrapolate from this sort of development to a grand narrative eventuating in a post-biological superintelligence or the Robot God.

If I may anticipate your next objection, this concern of mine is not at all equivalent to declaring consciousness supernatural, but just my insisting that the actual materialization of human consciousness is non-negligible in ways superlative futurologists seem ill-disposed to take into account in ways that empower most of their glib and deranging talk on this subject. A pile of gravel may be as complex as a skyscraper, but one hardly is indulging in mysticism to note that they are different nor to express skepticism that even a growing gravel pile is sure to crystallize into a skyscraper in the fullness of time.

Roko: As a matter of fact, there is a way of talking that I would regard as both fact-based rather than ad-hominem… It’s very simple: write a comment whose last line is “therefore smarter than human AI is not possible or is extremely unlikely in the next 100 years”, and write a justification of that statement above it. Finally, delete from your comment any personal attacks on people who believe the negation of your conclusion.

So, I ask you for the fifth(?) time: will you present a logical argument supporting the conclusion that smarter than human AI is not possible or is extremely unlikely in the next 100 years.

Dale: Endlessly failing, and loudly, to get my point that we disagree on premises and at the level of definitions isn’t exactly the stirring demonstration of your superior rationality and scientificity that you seem to think it is.

Roko: What are the prejudices at issue? The use of the word 'intelligence' to describe certain computer programs?

Dale: In a word, yes. Given what computers are and given what intelligence is, it's actually problematic to glibly associate them. When in the 50s people referred to room-sized computers as "electronic brains," it was, you know, a metaphor. I think the metaphor was not an illuminating one.

Roko: I am not claiming that all complex computer programs are smart.

Dale: "Are"? Well, that's a mercy. After all, such a claim would be obviously crazy.

Roko: I am claiming that there is a significant chance that some human team of software engineers and AI researchers might create a particular computer program that has a greater intellectual ability than any human.

Dale: Despite the fact that the locution "there is a significant chance that" sounds superficially sciency (the stock in trade of Robot Cultism), there are, of course, no actual empirical instances from which you can possibly be determining these "odds." Your utterance is essentially an expression of faith, unless you mean it, you know, figuratively, as a kind of bad poetry or something.

Roko: You claim that smarter than human intelligence is a nonsensical concept, right? But reality doesn’t care about your confusion!

Dale: Moments like these are my favorites!

First of all, as an atheist I don't attribute consciousness to "reality" any more than I do to software, and so I don't think reality "cares" about what either of us are saying. But more to the point, it cracks me up that you are so cocksure that you have "reality" on your side in the first place.

Given that literally no actually-existing computer exhibits intelligence and given that literally every exhibition of intelligence in the world has always been organismically embodied, you'll forgive me if I don't concede your faithful utterances to the contrary of that reality the force of "reality."

Indeed, the only reality I can associate with your performance is the long line of AI-ideologues speaking with exactly your certainty about the imminent arrival of AI, every year on the year for literally decades, and never once being anything but endlessly absolutely wrong about everything.

Roko: I am (and so are the rest of the singularitarian community) erring on the side of caution: we are like insurance against Superintelligent AI actually making sense and posing a real threat. It might not… but it just might.

Dale: There is no "it" in the terms on which your discourse depends. Your frames are confused and hence confusing. The singularitarian "community" (note the glancing admission there that singularitarianism isn't an argument but a sub(cult)ure) is erring, but not in anything like a useful way. There are many problems, among them security threats, associated with complex software and dynamic coding, and there are many intelligent people working on these problems. Nobody needs a fandom of boys-with-toys who cannot distinguish science from science fiction to arrive on the scene and save us from ourselves. To my mind, the larger more proximate threat by far is not so much the imminent arrival of the unfriendly Robot God, but the deranging discourse of singularitarianism itself that sensationalizes and hyperbolizes technical questions in ways that make sensible deliberation less easy by far, investing code with the utterly irrelevant cadences of hysterical apocalypse and wish-fulfillment fantasies of personal transcendence.

In a surprise move, the one and only Giulio Prisco, made a brief appearance on the scene at this point to contribute the following comment:
Yes Dale, facts have this unpleasant habit of getting in the way of the serious business of hair-splitting and mental masturbation. I am sure when you will meet a superintelligent AI at the pub, assuming you ever visit such mundane places, you will argue his non-existence with him based on the same nebulous empty pseudo-arguments that you use now. Perhaps he will disappear in a puff after your “proof” of his own non-existence, but somehow I doubt it.

Dale: I cannot distinguish this from the rantings of a lunatic. Yes, "facts," yes, I'll pull up a barstool and have a beer with the Robot God, and that'll show me. Mm-kay...

Prisco had more to say (follow the link if you like) but I will leave aside here the inevitable turn to the Wright Brothers that ensued, since this is a move I have lampooned incessantly here already. The long and the short of it: Giulio Prisco, you are not the Wright Brothers. Giulio Prisco, you are not Einstein. Giulio Prisco, you are somebody's crazy uncle building a perpetual motion machine in the garage out of milk bottles and popsickle sticks.

But Roko was apparently inspired by Giulio's comment to demand of me "an argument as to why superintelligent AI can’t exist! That’s what I’ve been asking him for for the last 50 comments!"

Dale: I wonder if there are any logicians in the house who can name the fallacy "Roko" is indulging in through this gambit? Anybody who has had the misfortune of trying to have a conversation with a frothing True Believer in God or UFOs or the Hollow Earth or fairies or Nessie will know what I am talking about.

Roko: If you think I am a member of a “Cult”, then you are acting in an extremely unethical way if you don’t respond to my requests for a clear explanation of what exactly is wrong with the Cult’s doctrine.

Dale: Bwa-ha-ha-ha-ha! If ridiculing the ridiculous is wrong I don't wanna be right! What critique of a cult will count as "clear" to the cultist that doesn't concede the organizing assumptions of the cult? C'mon, "Roko," surely you can do better than this! We've arrived at diminishing returns, I fear (Giulio Prisco's arrival on the scene is a sure-fire signal of that, if nothing else). I've been as clear as your provocation warrants, and I'm satisfied my point is made. I leave it to the peanut gallery to make their own assessments from here on out. Best to all.

12 comments:

jimf said...

Dale wrote:

> If I may anticipate your next objection, this concern of mine is
> not at all equivalent to declaring consciousness supernatural, but
> just my insisting that the actual materialization of human consciousness
> is non-negligible in ways superlative futurologists seem ill-disposed
> to take into account in ways that empower most of their glib and
> deranging talk on this subject. A pile of gravel may be as complex as
> a skyscraper, but one hardly is indulging in mysticism to note that
> they are different nor to express skepticism that even a growing gravel
> pile is sure to crystallize into a skyscraper in the fullness of time.

As, for example, Gerald M. Edelman has pointed out.

"[Are] artifacts designed to have primary consciousness...
**necessarily** confined to carbon chemistry and, more specifically,
to biochemistry (the organic chemical or chauvinist position)[?]
The provisional answer is that, while we cannot completely
dismiss a particular material basis for consciousness in the
liberal fashion of functionalism, it is probable that there will
be severe (but not unique) constraints on the design of any
artifact that is supposed to acquire conscious behavior. Such
constraints are likely to exist because there is every indication
that an intricate, stochastically variant anatomy and synaptic
chemistry underlie brain function and because consciousness is
definitely a process based on an immensely intricate and unusual
morphology" (_The Remembered Present_, pp. 32-33)

Giulio Prisco said...

Why, I like this image of a crazy uncle building a perpetual motion machine in the garage out of milk bottles and popsickle sticks. It has a refreshing retro flavor, like the best SF of the 50s. And often crazy uncles do build useful things in garages -- probably more often than useless, pompous academicians with self-inflated egos (another cliché of SF in the 50s).

You, my friend, are like that Iraqi Information minister who claimed on CNN that US forces were still hundreds of miles from Baghdad and would never be allowed to get closer... while we all could see the US tanks right behind him on the TV screen.

Dale Carrico said...

So... you actually think you can see intelligent software and immortal uploads and superlongevity pills and the Robot God and nanobot treasure-caves and so on, indeed, you think everybody can see these things like you do, and it's just me who isn't seeing them?

Giulio Prisco, ladies and gentlemen.

PS: I'm not your friend.

Giulio Prisco said...

Yes my unfriend, I actually think I can see many of these things on the horizon. I am not alone -- many persons much smarter and more knowledgeable than I see them.

Of course you are not the only one who does not (want to) see them. We ae still a minority, but a growing one:

http://transumanar.com/index.php/site/the_transhumanist_edge/

These people may be other mad uncles, in which case I would be VERY proud to be one.

Dale Carrico said...

Do you see a pony on the horizon?

Dale Carrico said...

We a[r]e still a minority, but a growing one

What, has your world-historical planetary movement nudged the membership needle from one thousand North Atlantic white boys with their toys to two thousand North Atlantic white boys with their toys after two decades of incessant sensationalizing hype?

You may sa-a-a-y he's a dreamer, but he's not the only one...

Next year on the L-5 torus, dood!

Nato Welch said...

I seem to have stopped reading Michael's blog just in time: now I can read it here!

jimf said...

Dale wrote:

> But Roko was apparently inspired by Giulio's comment to demand of me
> "an argument as to why superintelligent AI can’t exist! That’s what I’ve
> been asking him for for the last 50 comments!"
>
> Dale: I wonder if there are any logicians in the house who can name
> the fallacy "Roko" is indulging in through this gambit? Anybody who
> has had the misfortune of trying to have a conversation with a frothing
> True Believer in God or UFOs or the Hollow Earth or fairies or Nessie
> will know what I am talking about.

Hey, who says Objectivists can never be useful?

http://rous.redbarn.org/objectivism/writing/DavidKing/GuideToObjectivism/FALLACYS.HTM#237

* PROVING A NEGATIVE
(The Objectivist Newsletter, April 1963) "Proving the non-existence of
that for which no evidence of any kind exists. Proof, logic, reason,
thinking, knowledge pertain to and deal only with that which exists. They
cannot be applied to that which does not exist. Nothing can be relevant or
applicable to the non-existent. The non-existent is nothing. A positive
statement, based on facts that have been erroneously interpreted, can be
refuted - by means of exposing the errors in the interpretation of the
facts. Such refutation is the disproving of a positive, not the proving of a
negative.... Rational demonstration is necessary to support even the claim
that a thing is possible. It is a breach of logic to assert that that which
has not been proven to be impossible is, therefore, possible. An absence
does not constitute proof of anything. Nothing can be derived from nothing."


I suppose there are extenuating circumstances for Roko's mistake.
In mathematics, it is possible to prove a negative.


http://www.graveyardofthegods.net/ar...enegative.html

[T]he statement that “you cannot prove a negative” is simply false.
On the surface, it seems to be true: if Person A says “I think God exists”
and Person B says “I don’t think God exists,” it’s pretty clear that
Person B is going to have a hard time proving that there isn’t a God.
However, if you look a little closer, it actually depends on the nature
of the negative statement being made. Here are some negative statements
that can be proven very easily:

Five is not equal to four
The ancient Egyptians did not watch Seinfeld
The tsetse fly is not native to North America

Clearly, it’s possible to prove a negative statement. The real problem
here is clearly the nature of the positive statement being refuted. When a
person asserts that God exists, he does not specify the nature of God – that is,
is God small, large, blue, red? And where is he? Of course it is not possible
to prove that God does not exist, if “God” is a thing that has no definition,
no characteristics, and no location. In fact, you can prove just about any
kind of negative you can think of – except for (surprise!) the non-existence
of mystical beings. When you get right down to it, the statement
“you cannot prove a negative” is really just a different way of saying
“You can’t prove me wrong because I don’t even know what I’m talking about.”


http://forums.randi.org/showthread.php?t=12045

The whole "can't prove a negative" thing refers only to questions of form:

There EXISTS some entity X with properties Y.

To prove such a claim false you have to search the whole domain of the
problem. If such a complete search can be made, then you can prove negative.
For example, in mathematics you can prove that there doesn't exists a
rational number q such that q is the square root of 2.

Unfortunately, you can't do that in Real Life. Here also comes the additional
problem that the word "proof" has two meanings. The strict mathematical and
the common sense one, where a thing is considered proven if it is extremely
unlikely to be false.

Of the claims present in the first post:

> I am not a turnip.
> The ancient Egyptians did not watch Seinfeld
> The tsetse fly is not native to North America

can be considered to be provable in the common sense way, but even they cannot be
proven in mathematical way.

The ancient Egyptians might have had access to time machines and use them to watch
Seinfield. The tsetse fly might be originally native to North America but all
of them spontaneously decided to migrate to Africa, and you might be a super-intelligent
turnip in disguise about to conquer the world.

Of course, the probabilities of the three alternative explanations are [negligible]
and the original propositions can be said to be proven in the everyday sense. But
not in the mathematical sense.

jimf said...

Nato Welch on Unfalsifiable Fictitious Objects:

http://n8o.r30.net/dokuwiki/doku.php/blog:unfalsifiablefictitiousobject

jimf said...

> You may sa-a-a-y he's a dreamer, but he's not the only one...

Imagine. . .

If Giulio Prisco were a big enough celebrity to appear on
Kathy Griffin's radar.

http://www.youtube.com/watch?v=lYNwsfq9HrA

And now it's time for Sharon Stone to get up there and do her
opening statements, right. So she goes up and looks, she holds
the podium and looks out in the audience and she says, totally
seriously, she goes like this, "You know, ummm, I've spoken about
this topic so many times, I don't know if there's anything new
I can say about it. So, I'm going to read you something I once
read a long. time. ago." And everyone's looking at each other,
"What is she...?" I don't know. So...right, it's kinda weird, right?
So, then she takes out a piece of paper and goes like this,
totally seriously, she goes like this, "Imagine there's...no heaven/
It's easy...if you try/No hell below us/Above us only sky"...At this point,
I get what I like to call the "church giggles." That's when you're
laughing, inappropriately at an inappropriate time and you can't stop,
no matter what. I mean I got the shoulders going like this. *starts shaking*
I'm crying a little bit. Here's the thing: Sharon Stone is reciting the
lyrics to "Imagine" and CRYING AT HER OWN PERFORMANCE....that's funny! I'm
only human! Alright, so then she stops and goes, "Wow, ummm, that reminds me.
I was walking down the street here in New York many years ago and I passed
a man and he passed me. And I went, 'Whoa, that's John Lennon.' And I
smiled at him and he smiled at me." In the mean time, I'm watching this
thinking, "Oh, I thought John Lennon was so into Yoko. No, he's cruising
the park, hitting on Sharon Stone." And then she goes, "And then we smiled
at each other and then walked away...You may say, I'm a dreamer." Alright,
she finishes and I look around and there are people literally moved to
tears. I'm fucked! 'Cause if they love that, they're not going to like
me and my vagina jokes! . . .

So, I'm sitting at my little front table, and I know they just loved
that Sharon Stone moment, and I'm sittin' there and then Rosie O'Donnell
comes over to my table, sits down, and she goes like this: "Is she
outta her fuckin' mind?"

Antonin said...

If the coming AI is as intellectually inadaptable and intractably narrow-minded as its advocates, that is, incapable of addressing contrary arguments without framing them as EITHER probabilistic nitpicking OR irrational naysaying, than I don’t see what the fuss is all about.

Seriously, a moderately well-read atheist schoolkid will be thinking circles around the thing.

NVeshecco said...

So, I'm how many years away from being able to effectively enter into this discussion? Two? Five?