Professor Hawking- Whenever I teach AI, Machine Learning, or Intelligent Robotics, my class and I end up having what I call "The Terminator Conversation." My point in this conversation is that the dangers from AI are overblown by media and non-understanding news, and the real danger is the same danger in any complex, less-than-fully-understood code: edge case unpredictability.

In my opinion, this is different from "dangerous AI" as most people perceive it, in that the software has no motives, no sentience, and no evil morality, and is merely (ruthlessly) trying to optimize a function that we ourselves wrote and designed. Your viewpoints (and Elon Musk's) are often presented by the media as a belief in "evil AI," though of course that's not what your signed letter says. Students that are aware of these reports challenge my view, and we always end up having a pretty enjoyable conversation. How would you represent your own beliefs to my class? Are our viewpoints reconcilable? Do you think my habit of discounting the layperson Terminator-style "evil AI" is naive? And finally, what morals do you think I should be reinforcing to my students interested in AI?



whenever – ~するときはいつでも
machine learning – 機械学習
end up – ~という結果になる
overblown – 誇張する
perceive – 認識する
optimize – 効率化する
sentience – 直感
reconcilable – 調和する、和解できる
layperson – 素人
discount – 無視する、疑う
naive – 考えが甘い

You’re right: media often misrepresent what is actually said. The real risk with AI isn’t malice but competence. A superintelligent AI will be extremely good at accomplishing its goals, and if those goals aren’t aligned with ours, we’re in trouble. You’re probably not an evil ant-hater who steps on ants out of malice, but if you’re in charge of a hydroelectric green energy project and there’s an anthill in the region to be flooded, too bad for the ants. Let’s not place humanity in the position of those ants. Please encourage your students to think not only about how to create AI, but also about how to ensure its beneficial use.


misrepresent – 不正確に伝える
malice – 悪意
accomplish – 達成する
competence – 能力
aligned – 位置合わせした
hydroelectric – 水力発電の
anthill – 蟻塚


Hello Doctor Hawking, thank you for doing this AMA. I am a student who has recently graduated with a degree in Artificial Intelligence and Cognitive Science. Having studied A.I., I have seen first hand the ethical issues we are having to deal with today concerning how quickly machines can learn the personal features and behaviours of people, as well as being able to identify them at frightening speeds. However, the idea of a “conscious” or actual intelligent system which could pose an existential threat to humans still seems very foreign to me, and does not seem to be something we are even close to cracking from a neurological and computational standpoint. What I wanted to ask was, in your message aimed at warning us about the threat of intelligent machines, are you talking about current developments and breakthroughs (in areas such as machine learning), or are you trying to say we should be preparing early for what will inevitably come in the distant future?


graduate – 卒業する
cognitive science – 認識科学
ethical – 道徳的な、倫理的な
deal with – ~に対処する
feature – 特徴
neurological – 神経学の
foreign – 無関係の
current – 現在の
prepare – 準備する
inevitably – 必ず

The latter. There’s no consensus among AI researchers about how long it will take to build human-level AI and beyond, so please don’t trust anyone who claims to know for sure that it will happen in your lifetime or that it won’t happen in your lifetime. When it eventually does occur, it’s likely to be either the best or worst thing ever to happen to humanity, so there’s huge value in getting it right. We should shift the goal of AI from creating pure undirected artificial intelligence to creating beneficial intelligence. It might take decades to figure out how to do this, so let’s start researching this today rather than the night before the first strong AI is switched on.


researcher – 研究者
claim – 主張する
lifetime – 生涯
decade – 10年間


My questions: 1. One might think it impossible for a creature to ever acquire a higher intelligence than its creator. Do you agree? If yes, then how do you think artificial intelligence can ever pose a threat to the human race (their creators)? 2. If it was possible for artificial intelligence to surpass humans in intelligence, where would you define the line of “It’s enough”? In other words, how smart do you think the human race can make AI, while ensuring that it doesn’t surpass them in intelligence?


creator – 創作者
threat – 脅威
human race – 人類
surpass – 超える、上回る
define – 定義する
in other words – 言い換えれば
ensure – 保証する

It’s clearly possible for a something to acquire higher intelligence than its ancestors: we evolved to be smarter than our ape-like ancestors, and Einstein was smarter than his parents. The line you ask about is where an AI becomes better than humans at AI design, so that it can recursively improve itself without human help. If this happens, we may face an intelligence explosion that ultimately results in machines whose intelligence exceeds ours by more than ours exceeds that of snails.


acquire – 獲得する
ancestor – 祖先
evolve – 進化する
この表現は便利なので覚えましょう。意味は「猿のような」。名詞+like+名詞で「~のような名詞」という意味になります。apple like red cheeksで「リンゴのように赤い頬っぺた」ですね。