Salesforce AI Research Blog

DALL-E so far has no way to create consistent characters across multiple pictures. However good news if this discriminator approach https://blog.salesforceairesearch.com/gedi/ caries over to images such constraints could be imposed. This is just a handful of follow up publications away.

Interesting. Just a few weeks back, I was reading about their previous work https://atscaleconference.com/the-evolution-of-aiops-at-meta-beyond-th… -- didn't realise there's more work! Also, some more researches in the similar space by other enterprises: Microsoft: https://yinfangchen.github.io/assets/pdf/rcacopilot_paper.pdf Salesforce: https://blog.salesforceairesearch.com/pyrca/ Personal plug: I'm building a self-service AIOps platform for engineering teams (somewhat similar to this work by Meta). If you're looking to read more about it, visit -- https://docs.drdroid.io/docs/doctor-droid-aiops-platform

More info on the Salesforce blog: https://blog.salesforceairesearch.com/mint-1t/

Those look all programming related tokens, and I think they should relate to their focus of improving this model's code generation capabilities, and adding quite a bit of data related to that (BigCode Starcoder in the Second stage in pre-training https://blog.salesforceairesearch.com/xgen/#pre-training-data )

> Yes, in a very very very very very limited environment like a board game. They work in math and coding, and these two are quite general. > CodeRL is a new framework for program synthesis through holistic integration of pretrained language models and deep reinforcement learning. By utilizing unit test feedback as part of model training and inference, and integrating with an improved CodeT5 model, CodeRL achieves state-of-the-art results on competition-level programming tasks. https://blog.salesforceairesearch.com/coderl/

Anyone could just push the repos to GitHub, including GitHub employees. Here are links to Amazon and Salesforce https://aws.amazon.com/codewhisperer/ https://blog.salesforceairesearch.com/codegen/

My whole point is that the way in which AI progress can intersect itself can lead to compounding effects in ways technology has not. One way to improve upon GPT-3 is almost certainly going to be adding discriminators into the mix: "GeDi: A Powerful New Method for Controlling Language Models" https://blog.salesforceairesearch.com/gedi/ And it may be that rather than it being a CNN discriminator such as in that case, it will be a second transformer: "[2102.07074] TransGAN: Two Pure Transformers Can Make One Strong GAN, and That Can Scale Up" https://arxiv.org/abs/2102.07074 My point is we may not need to reinvent some new model separate from transformers/GANs and the work so far as long as novel ways of intersecting trained models produces results that can in turn feed intersect with other models. NVIDIA in particular has had some interesting focus on using AI to generate training data for AI. "NVIDIA Omniverse Replicator Generates Synthetic Training Data for Robots | NVIDIA Technical Blog" https://developer.nvidia.com/blog/generating-synthetic-datasets-isaac-… I think it's very unlikely we need to wait 25 years for a new paradigm. In fact, I'm willing to wager a bet that whatever new paradigm will arrive will itself be the output of AI simulating different models.