Synced Review

Two recent related threads: https://news.ycombinator.com/item?id=21840924 https://news.ycombinator.com/item?id=21840101 Also https://medium.com/syncedreview/cmu-senior-develops-worlds-first-class… via https://news.ycombinator.com/item?id=21864706 (but no comments there).

https://medium.com/syncedreview/simulations-pave-the-road-for-self-dri… If you are not convinced, you can look for "waymo simulation" in your favorite search engines.

Wave Field recordings are probably the most complete known descriptions of the brain and its nonlinear fields? How such fields relate to one or more Quantum Wave functions might entail near-necessity of QFT: Quantum Fourier Transform. When you replace the Self-attention Network part of a Transformer algorithm with classical FFT Fast Fourier Transform: ... From https://medium.com/syncedreview/google-replaces-bert-self-attention-wi… : > > New research from a Google team proposes replacing the self-attention sublayers with simple linear transformations that “mix” input tokens to significantly speed up the transformer encoder with limited accuracy cost. Even more surprisingly, the team discovers that replacing the self-attention sublayer with a standard, unparameterized Fourier Transform achieves 92 percent of the accuracy of BERT on the GLUE benchmark, with training times that are seven times faster on GPUs and twice as fast on TPUs." > > Would Transformers (with self-attention) make what things better? Maybe QFT? There are quantum chemical interactions in the brain. Are they necessary or relevant for what fidelity of emulation of a non-discrete brain? > Quantum Fourier Transform: https://en.wikipedia.org/wiki/Quantum_Fourier_transform

- "Some of the models are so big that even in MILA we can’t run them because we don’t have the infrastructure for that. Only a few companies can run these very big models they’re talking about" [1]. NOTE: MILA is a very good AI research center and, while I don't know too much about him, that person being quoted has great credentials so I would generally trust them. - "the current version of OpenAI Five has consumed 800 petaflop/s-days" [2]. - Check out the Green AI paper. They have good number on the amount of compute to train a model and you can translate that into numbers. - https://medium.com/syncedreview/the-staggering-cost-of-training-sota-a… NOTE: That XLNet number has to be wrong - it should be 5-figures, not 6. I'm not an expert in on-prem ML costs, but I know many of the world's best on-prem ML users use the cloud to handle the variability of their workloads so I don't think on-prem is a magic bullet cost wise. $1M annually per project (vs per lab) isn't bad at all. It's also way out of whack with what I saw when I was doing AI research in academia, but that was pre deep learning revolution, so what do I know. Re: the moving goalposts - the distinction is between the cost of a training run and the cost of a paper-worth research result. Due to inherent variability, architecture search, hyperparameter search and possibly data cleaning work, the total cost is a couple orders of magnitude more than the cost of a training run (multiple will vary…

Just some days ago researchers from Peking U and Microsoft published a paper[0] saying they can access "knowledge neurons" in pretrained embeddings that will enable "fact editing"[1]. [0]: https://arxiv.org/pdf/2104.08696.pdf [1]: https://medium.com/syncedreview/microsoft-peking-u-researchers-identif…

> It would be a shame if ML also became a field people avoided because they didn't want to contribute to evil, in their own view. Both ethics and privacy considerations have recently become pretty regular at Computer Vision and Multimedia Processing conferences. One very popular object detection model (called YOLO) had the main author recently leaving the field because he got concerned about military applications using his research results. https://medium.com/syncedreview/yolo-creator-says-he-stopped-cv-resear…

There are: https://medium.com/syncedreview/mit-google-propose-automl-for-model-co… I went to a more recent talk by Google and results they handpicked to show were impressive.

Medium article https://medium.com/syncedreview/unsupervised-image-to-image-translatio… Arxiv: https://arxiv.org/pdf/1907.10830.pdf GitHub: https://github.com/taki0112/UGATIT h/t to Reddit, thought HN would be interested.

This is second large framework making the switch to Pytorch. https://medium.com/syncedreview/japanese-unicorn-preferred-networks-mi…

I suppose this is intended to be a link to https://medium.com/syncedreview/say-woof-ai-in-animal-language-transla… ?

there is lot of training happening on the edge (mobile devices) as well, look up 'federated learning' https://medium.com/syncedreview/federated-learning-the-future-of-distr…

You can just train a conditional GAN on a variety of datasets, such as BigGAN. https://medium.com/syncedreview/biggan-a-new-state-of-the-art-in-image…

(I work at OpenAI.) It comes down to whether you believe AGI is achievable. We've talked about why we think it might be: https://medium.com/syncedreview/openai-founder-short-term-agi-is-a-ser… https://www.youtube.com/watch?v=YHCSNsLKHfM And we certainly have more of a plan for building it than warp drives :). EDIT: I personally think the case for near-term AGI is strong enough that it'd be hard for me work on any other problem — and find it important to put in place guardrails like https://openai.com/blog/openai-lp/ and https://openai.com/charter/. Even if AGI turns out to be out of reach, we'll still be creating increasingly powerful AI technologies — which I think pretty clearly have the potential to alter society and require special care and thought.

Federated Learning[1] you can train across multiple customer data by sending the in-training model to every device, then exfiltrating only the backprop adjustments you get back. In-theory, user data never leaves the device. In-practice, the backprop data has the information encoded within it to an extent, it's very difficult to secure. [1] https://medium.com/syncedreview/federated-learning-the-future-of-distr…

FB post links to https://medium.com/syncedreview/oxford-university-ai-policy-researcher…

> that's all ML seems to be. You have to be very selective about what you consider "ML" to come to that conclusion. There has been a constant parade of incredible, mind-blowing results out of ML over the past decade, advancing the state of the art by leaps and bounds both in research and in real applications. Do you not remember how terrible speech recognition and speech synthesis were just a few short years ago? Did you not see DeepMind finally crack Go? Check out BigGAN [1]. Try search in Google Photos. See translation getting better every year. Yes, there are quacks and charlatans and people who are just plain wrong. But ML is real, and it solves real problems that people failed to solve any other way despite decades of concerted effort. [1] https://medium.com/syncedreview/biggan-a-new-state-of-the-art-in-image…

Background: https://seekingalpha.com/article/4206602-first-look-liulishuos-72-mill… > The company has 83.8 million registered users, of which 1 million are paid users. > The company's rapid growth (~6x in the first half of this year) Per the F-1, the average Liulishuo user spends over 60 minutes per day learning English in the app. How their product works: https://medium.com/syncedreview/liulishuos-ai-app-is-teaching-english-…

Or perhaps you don't even need a human at all, according to this work: https://medium.com/syncedreview/busting-moves-with-dancenet-ai-6efa5c5…