Yoshua Bengio

I knew there was trouble as soon as I saw "Nobel Prize [winner]" and "Adjunct professor" in the same sentence. What's it take to get tenure track these days? But she mentions the rate of publications. Kps noted that Peter Higgs said something similar as well. There's many others too! Turing prize winner Hinton had this to say about ML and I couldn't agree more > One big challenge the community faces is that if you want to get a paper published in machine learning now it's got to have a table in it, with all these different data sets across the top, and all these different methods along the side, and your method has to look like the best one. If it doesn’t look like that, it’s hard to get published. I don't think that's encouraging people to think about radically new ideas. > Now if you send in a paper that has a radically new idea, there's no chance in hell it will get accepted, because it's going to get some junior reviewer who doesn't understand it. Or it’s going to get a senior reviewer who's trying to review too many papers and doesn't understand it first time round and assumes it must be nonsense. Anything that makes the brain hurt is not going to get accepted. And I think that's really bad. Or from Bengio > In the rush preceding a conference deadline, many papers are produced, but there is not enough time to check things properly and the race to put out more papers (especially as first or equal-first author) is humanly crushing. On the other…

or (also) Yoshua Bengio? https://yoshuabengio.org/2023/05/22/how-rogue-ais-may-arise/ https://yoshuabengio.org/2023/06/24/faq-on-catastrophic-ai-risks/ unfortunately for parent comment, categorizing something as sci-fi doesn't make it so

It is very obvious you have never read the book Superintelligence or any literature on the subject because you try to post here shower thoughts. But here is the thought of the most highly rated h-index computer scientist in the world to help out: https://yoshuabengio.org/2023/06/24/faq-on-catastrophic-ai-risks/

There is. AI progress in the last 10 years was massive, and there is no end in sight. The risks are well established in countless essays. See e.g. https://yoshuabengio.org/2023/06/24/faq-on-catastrophic-ai-risks/

Why not both: a clear argument for concern written by an expert? https://yoshuabengio.org/2023/05/22/how-rogue-ais-may-arise/

That's what is usually confusing about mathematicians. They don't think using a depth-first tree going straight to the goal like someone from IT would, but rather go breadth-first, painting plenty of interesting concepts along the way, and chaining them seemingly aimlessly together until they crystalize right toward the right solution seemingly magically. In his Physics Project Wolfram when talking about "Branchial space" is toying around the idea of recombining trees. Where one configuration of the state-space can be reached by various ways. This general concept of recombining trees has for example been used in generative flow networks https://yoshuabengio.org/2022/03/05/generative-flow-networks/ where they use it to directly generate molecule graphs that have useful properties among the sea of useless ones.

The concern is that near future systems will be much more capable than current systems, and by the time they arrive, it may be too late to react. Many people from the large frontier AI companies believe that world-changing AGI is 5 years or less away; see Situational Awareness by Aschbrenner, for example. There's also a parallel concern that AIs could make terrorism easier[1]. Yoshua Bengio has written in detail about his views on AI safety recently[2][3][4]. He seems to put less weight on human level AI being very soon, but says superhuman intelligence is plausible in 5-20 years and says: > Faced with that uncertainty, the magnitude of the risk of catastrophes or worse, extinction, and the fact that we did not anticipate the rapid progress in AI capabilities of recent years, agnostic prudence seems to me to be a much wiser path. Hinton also has a detailed lecture he's been giving recently about the loss of control risk. In general, proponents see this as narrowly tailored bill to somewhat address the worst case worries about loss of control and misuse. [1] https://www.theregister.com/2023/07/28/ai_senate_bioweapon/ [2] https://yoshuabengio.org/2023/05/22/how-rogue-ais-may-arise/ [3] https://yoshuabengio.org/2023/06/24/faq-on-catastrophic-ai-risks/ [4] https://yoshuabengio.org/2024/07/09/reasoning-through-arguments-agains…

Take your pick? https://yoshuabengio.org/2023/05/22/how-rogue-ais-may-arise/ https://intelligence.org/files/AlignmentHardStart.pdf https://www.youtube.com/watch?v=pYXy-A4siMw https://europepmc.org/article/med/26185241 https://en.wikipedia.org/wiki/Superintelligence:_Paths,_Dangers,_Strat… I could keep going but the reading on this spans for tens of thousands of pages of detailed reasoning and analysis, including mathematical proofs and lab-scale experiments.