Hawking warns of the dangers of AI

[Fist and Faith]

We can't explain what it is, or how it came about. We have not the foggiest idea. Which means we can't claim to know that it cannot exist in another seeing.

And yet people are confident we can build this thing that we can't explain what is it or how it came about?

Not knowing what it is or how it works, I do not see how you can know in which mediums it can and cannot exist.

[Zarathustra]

Fist, you don't think the point about Godel's theorem casts serious doubts on the idea? Software is a formal system. I don't think consciousness can exist in a formal system. Consciousness is more than symbols, it has content. I don't have to know how consciousness is produced as a practical matter to rule it out in principle. My chair isn't conscious. My shirt isn't conscious. Math and logic aren't conscious. Computer software is math and logic.

[Avatar]

It's a formal system right now. I think we're heading toward more dynamic ones rapidly.

--A

[Fist and Faith]

I was going to say maybe we can make an informal system. Dynamic sounds better. Heh.

Consciousness already arose in a system in which, by their own nature, its constituents must interact in absolutely specific ways. A piece of iron cannot not be attracted to a magnet. A photon cannot not be emitted when an electron drops back down to its stable orbit. Sand cannot not form glass when heated just so, and cannot not shatter when hit by a rock. There's nothing informal about any of it. Yet, here we are.

I don't see how it is possible to say consciousness can only exist within the medium of the components that make up life on this planet. I don't see how we can say it is impossible for it to exist within this other medium. Maybe even others.

[Zarathustra]

There's nothing informal about any of it. Yet, here we are.

I have no idea what you mean. "Informal" isn't the opposite of "formal" in this case. A formal system is an abstract, axiomatic system consisting entirely of symbols and rules for manipulating them. Pure form, no content.

I don't see how it is possible to say consciousness can only exist within the medium of the components that make up life on this planet. I don't see how we can say it is impossible for it to exist within this other medium. Maybe even others.

I'm not saying it can only exist within brains. However, brains are uniquely sufficient and necessary for the production of consciousness, therefore whatever brains do must also be done by another system which one hopes will produce consciousness. Computers can't do what brains do, not even in principle, therefore I doubt they can be conscious.

Brains do much more than process algorithms. In fact, brains only process algorithms in extremely limited cases (e.g. when doing logic, math, ect.). Most of the time, we're not running algorithms in our heads ... and yet we're conscious during this time. Thus, it seems highly misguided to expect that we can build consciousness entirely from algorithms, when this is not the quintessential feature of consciousness. It would be like saying that since conscious beings play chess, you can make a computer conscious merely from the rules of chess. (That's not much of an exaggeration.)

Are you willing to say that it's impossible for a chess program to be conscious? If so, how can you say this, given your argument that we can't eliminate the possibility as long as we don't know how brains are conscious?

[Fist and Faith]

brains are uniquely sufficient and necessary for the production of consciousness

We have no way of knowing that the particular type of matter/energy that brains are composed of is the only type of matter/energy within which consciousness can reside.

[Zarathustra]

I agree, Fist. But we should start there, since it's the most obvious place to look for consciousness. And AI engineers are doing this, to some extent, when they build neural nets. And programmers are modeling some aspect of what brains do, when they program algorithms. However, as I said, brains do much more than this.

I think the fundamental reason that calculators and laptops aren't conscious is much more than complexity. The problem won't be solved with more processors or faster processing. Computers already do certain tasks much faster than we do. There is a qualitative difference, not merely a quantitative one.

Also, I don't think you can discount the fact that conscious beings are also living beings. There is a reason that evolution can bring about consciousness, why natural selection can "get a grip" on it and turn organisms this way. There is something about living matter that makes this possible. Nature never forms consciousness out of dead matter. I think the two are inextricably tied.

Instead of building "smarter" machines, we should try to build living machines. It should be much easier to build simple consciousness than to build complex, intelligent consciousness. But we're starting at the wrong end.

[Fist and Faith]

Also, I don't think you can discount the fact that conscious beings are also living beings. There is a reason that evolution can bring about consciousness, why natural selection can "get a grip" on it and turn organisms this way. There is something about living matter that makes this possible. Nature never forms consciousness out of dead matter. I think the two are inextricably tied.

Well, fact is, all consciousness, and all life, was formed from dead matter. But I know what you mean. Assuming abiogenesis occurred generally along the lines of the way we think it did, there were characteristics of life before there was any sort of consciousness. But I wonder if it must be. The only examples of life and consciousness we have are from this tiny speck of dust in an impossibly vast cosmos. I wonder if anything else is out there. Maybe somewhere out there there are counterparts of nucleotides the went to consciousness without anything else that we would consider characteristics of life. And I wonder if we would then say consciousness alone is sufficient to classify something as living.

(On a tangent, I'm not a fan of Nagel's phrase "dead matter". If something is dead, it means it was once alive. I think "Non-living" is a better way of referring to what we're talking about.)

Instead of building "smarter" machines, we should try to build living machines. It should be much easier to build simple consciousness than to build complex, intelligent consciousness. But we're starting at the wrong end.

I assume this approach is being taken by some. People must be trying to make life; some along the biological lines we're familiar with, and I imagine others are using other materials.
Whatever method, give it a way to reproduce at a very early age, like minutes or hours, and I'll bet there will be mutations, just as there are with life. Unless we manage to make a perfect replication process, which I doubt we can. Unless we oversee ever detail every instant, I suspect we'd be surprised when we check in on things.

I also assume people are writing programs that do nothing but reproduce themselves.

I guess we need to put "death" into the system. Otherwise, whatever medium is involved would be overrun.

[Vraith]

Instead of building "smarter" machines, we should try to build living machines. It should be much easier to build simple consciousness than to build complex, intelligent consciousness. But we're starting at the wrong end.

Related to that, I've seen headlines but haven't gotten around to reading much on it, so may be misstating...

At least a couple different groups have used DNA as memory. Didn't really look at it, so I don't know if they meant memory like RAM or as storage, like hard drive.

At least a few people are growing functioning, bodiless mini-brains.

Both those would seem to be useful in the direction I want this stuff to go [which I'm sure folk are bored with hearing me repeat]--integration of the tech with our flesh brains, so they're one thing with each part doing what it is best at.

[peter]

(On a tangent, I'm not a fan of Nagel's phrase "dead matter". If something is dead, it means it was once alive. I think "Non-living" is a better way of referring to what we're talking about.)

Surely "Organic" and "Inorganic" are the least ambiguous terms (though granted they do not cover the distinction between living and dead organic matter).

But on the question of consciousness, is it an absolute given that it will always emerge beyond a (yet to be defined?) level of connectivity/complexity. Perhaps so, but perhaps not. Perhaps we are really just a one in a billion fluke, even amongst widespread massively complex networks of 'stuff' found in lots of places across the universe. After all life on earth got along perfectly well for three and a half billion years without it and the as yet unknown reasons for its emergence suggest that it's appearance is by no means a done deal.

[Fist and Faith]

(On a tangent, I'm not a fan of Nagel's phrase "dead matter". If something is dead, it means it was once alive. I think "Non-living" is a better way of referring to what we're talking about.)

Surely "Organic" and "Inorganic" are the least ambiguous terms (though granted they do not cover the distinction between living and dead organic matter).

Not least ambiguous, since it might be referring to the presence/absence of carbon; or it might be referring to the standards of food production, which are not at all standardized. But in no case does it have to do with living/dead or living/non-living. Diamonds are made of carbon, and all life on earth contains carbon.

But on the question of consciousness, is it an absolute given that it will always emerge beyond a (yet to be defined?) level of connectivity/complexity. Perhaps so, but perhaps not. Perhaps we are really just a one in a billion fluke, even amongst widespread massively complex networks of 'stuff' found in lots of places across the universe. After all life on earth got along perfectly well for three and a half billion years without it and the as yet unknown reasons for its emergence suggest that it's appearance is by no means a done deal.

The carbon in diamonds is said to be highly organized. We could say a cliff's rock strata is an example of organization. Stars, solar systems, and galaxies are organized in various ways. So we have organization in the universe without life. We also have life without consciousness. So neither life nor consciousness seems inevitable. But I haven't read "The Secret Life of Plants" that Z and V mentioned, and have not finished Nagel's book (it's not easy to keep the things he's saying straight, so I have to go back a few pages every few pages ), so there are other thoughts out there.

[Avatar]

For anybody who is interested, here's the annual report from AI Index, an open, non-profit project to track activity and progress in AI.

aiindex.org/2017-report.pdf

--A

[peter]

Very interested Av - but time/understanding constraints prevent me from digesting the whole thing. The 'overview' said what each section did - but did not summarise the key findings; is there anything striking of note that stood out for you?

On the same subject. I was reading a quote the other day from President Putin who said effectively that whoever led the world in AI would effectively lead the world point blank. China it appears are taking this really seriously - their expenditure on AI research has sky-rocketed to the point where the article suggested that by 2020 they would lead the world in the field.

[wayfriend]

Of course. Aside from the obvious military uses, AI lends itself to political uses, hacking uses, social engineering uses (think troll farm), legal uses (imagine an AI lawyer), civil engineering uses, and, of course, technology uses (using AI to design better AI).

[Vraith]

Of course. Aside from the obvious military uses, AI lends itself to political uses, hacking uses, social engineering uses (think troll farm), legal uses (imagine an AI lawyer), civil engineering uses, and, of course, technology uses (using AI to design better AI).

Heh...pretty much by definition of "intelligence," [at least our definition] AI will lend itself to all current and future intelligence-based uses. The only question is will WE control the uses, will IT/THEY control the uses, or will it be a cooperative enterprise? In all real ways, the second two of those three are actually purposes, not uses...but that's a different issue. Maybe.

[[the middle and last book of the series I'm currently enmeshed in pretending I'm writing addresses those...and even has answerness-es that I THINK [or maybe HOPE if people-like things are to survive] make sense...the first book is mostly involved with forming my particular universe in which those questions come to matter.

[wayfriend]

Oh ... we will control the uses until such time as we can no longer comprehend the choices of the AIs, and start blindly doing whatever they say because it always works to our advantage. Then they will be in control, even though we have the off switch. Until we are tricked into disabling the off switch.

Let's face it, has humanity ever adopted a technology slowly and carefully where money and power are to be had?

[Avatar]

Very interested Av - but time/understanding constraints prevent me from digesting the whole thing. The 'overview' said what each section did - but did not summarise the key findings; is there anything striking of note that stood out for you?

Dunno...haven't had time to read it yet.

--A

[Vraith]

until such time as we can no longer comprehend the choices of the AIs,

From what I've seen lately, we ALREADY don't comprehend the choices the useful AI's are making...and they aren't even very "I" yet.

[Avatar]

https://futurism.com/new-black-metal-al ... ade-human/

--A

[Zarathustra]

I think that people fear AI because they imagine it will do tbings we don't want it to do, and take over. But as smart as these programs will be, they won't actually want things. They won't have their own goals. They won't be able to decide that they're better off without these pesky humans.

We're already using AI with absolutely no negative consequences. Your texting app uses AI to anticipate what words you're trying to type. Anyone frightened of their texting app suddenly deciding to text people without your permission? Much less take over the world? Of course not. But if AI is so unpredictable and uncontrollable, why do we all use it daily without worrying about it?

It's like the people who worry about the government putting chips in us and tracking our every move ( I know people like this personally) but don't worry about the chip they carry around with them everywhere they go. Cell phones already present all the dangers they allegedly fear, but they don't worry about them. AI will be like that. It already is.