Luke Muehlhauser

Instead of telling you what they recommended, I'm going to give you my modern list instead of assurance activities that could be considered for the next evaluation scheme. Orange Book had a subset of these. I pulled from the whole field to get a larger list. https://lobste.rs/s/mhqf7p/static_typing_will_not_save_us_from_broken/… The two summaries of the history of successes and problems in TCSEC are here: http://lukemuehlhauser.com/wp-content/uploads/Bell-Looking-Back-Addend… https://www.acsac.org/2004/papers/ClassicPaperSchafer.pdf

Here is a page with a lot of good examples of more accessible modern classical music: http://lukemuehlhauser.com/how-to-fall-in-love-with-modern-classical-m…

Tim Urban is a great writer, but read something like this - http://lukemuehlhauser.com/a-reply-to-wait-but-why-on-machine-superint… - if you want more nuance. (Caveat: It's been some time since I read either article, or about superintelligence.)

What I've heard is that MIRI has an explicit philosophy of concentrating on the more abstract & theoretical aspects of AI safety. The idea being that if AI safety is something that you can just tack on to a working design at the end, they don't have a comparative advantage there: it's difficult to predict which design will win and the design's implementor is best positioned to tack on the safety bit themselves. >...imagine a hypothetical computer security expert named Bruce. You tell Bruce that he and his team have just 3 years to modify the latest version of Microsoft Windows so that it can’t be hacked in any way, even by the smartest hackers on Earth. If he fails, Earth will be destroyed because reasons. >Bruce just stares at you and says, “Well, that’s impossible, so I guess we’re all fucked.” >The problem, Bruce explains, is that Microsoft Windows was never designed to be anything remotely like “unhackable.” It was designed to be easily useable, and compatible with lots of software, and flexible, and affordable, and just barely secure enough to be marketable, and you can’t just slap on a special Unhackability Module at the last minute. >To get a system that even has a chance at being robustly unhackable, Bruce explains, you’ve got to design an entirely different hardware + software system that was designed from the ground up to be unhackable. And that system must be designed in an entirely different way than Microsoft Windows is, and no team in the world could do…

I think Billionaire alignment is a much larger problem than AI alignment. To use Bostrom's language, it's not full-on owl domestication, but sparrows with owl-like powers that we need to worry about. https://lukemuehlhauser.com/bostroms-unfinished-fable-of-the-sparrows/

You can chart outcomes instead of trying to score inventions. See for example: http://lukemuehlhauser.com/wp-content/uploads/all-curves-with-events.p… I'd caution that these graphs are completely useless for talking about a technological singularity though, and - even worse - they make it seem like "something must happen" because that line is so obviously "almost vertical". This simply isn't the case, and - importantly - is almost the opposite to the argument for a singularity.

Musk replied to Ng's comment: http://lukemuehlhauser.com/musk-and-gates-on-superintelligence-and-fas… Or as someone else quipped, the moment a human steps foot on Mars, Mars has become overpopulated.

Be sure to de-lurk if that's the case! I guess. A lot of people already did. http://lukemuehlhauser.com/if-youre-an-ai-safety-lurker-now-would-be-a…

This reminds me a little of "Being or Nothingness" by Joe K (hehe) http://lukemuehlhauser.com/the-riddle-of-being-or-nothingness