AI Weirdness

ChatGPT is excellent at comedy. https://www.aiweirdness.com/ For anything other than comedy, which is not equally well done by typing your query into your favorite search engine, it seems that this kind of "AI" has no "I".

Janelle Shane, who does this sort of thing for real on the excellent blog https://aiweirdness.com gave a good breakdown on Twitter: https://twitter.com/janellecshane/status/1007061610005794817

So we have a single-page site that quotes and links to a few paragraphs of someone else's work, advertises the "creator"'s social media and the SSG used, and then a row of icons to reflect various social/political campaigns (presumably ones this person sides with). This doesn't seem worth registering a domain, and it would have been more useful to submit the original article (https://www.aiweirdness.com/ignore-all-previous-instructions/) directly.

The evolution of the AI weirdness blog (https://www.aiweirdness.com/) over the past few years somewhat supports this idea, except that the author got plenty of funny results with LLMs - but mostly the early ones, up to around GPT3 (and smaller variants of GPT3). For example, cereal names generated by the Ada version of GPT were much funnier than the Da Vinci ones: https://www.aiweirdness.com/new-breakfast-cereals-from-ai/

Sure, but I, at least, am here largely for entertainment purposes. Which is one place where LLMs are mildly useful, actually; while ChatGPT isn't very good at it (I think deliberately so) some LLMs can produce very amusing output (see https://www.aiweirdness.com - while occasionally, as now, there's a serious post, it's mostly AI stuff doing silly things).

Reminds me of a post [0] where someone shared potential novel opening lines that had been generated by GPT-2. So many of them are interesting and/or hilarious. A few of my favorites: I am forced to write to my neighbors about the beast. I was playing with my dog, Mark the brown labrador, and I had forgotten that I was also playing with a dead man. I wish someone would make a novel out of these! [0]: https://aiweirdness.com/post/189170306297/how-to-begin-a-novel

"Shane’s peculiar genius is to feed her algorithms cultural oddments such as paint colours, recipes and ice-cream flavours, rather than (say) crystallographic data. The results – a warm shade of pink the neural network dubs “Stanky Bean”; a recipe for “clam frosting”; and (my personal favourite) an ice-cream flavour called “Necrostar with Chocolate Person” – are eccentric and often endearing. The book’s title is another example: apparently, “you look like a thing and I love you” is an algorithm’s idea of a great pick-up line. Frankly, I’ve heard worse." [1] https://aiweirdness.com/rss

#EBABEC is now "Pink Hork" in honor of Janelle Shane[0]. 0: https://aiweirdness.com

The linked TED talk is by Janelle Shane, an optics research scientist and an AI researcher. She maintains a blog, which is quite funny and has also written a book drawn from her experiences with NN's, particularly GPT-2. https://aiweirdness.com/

This reminds me of Janelle C. Shane's work on AI[1]. Computer vision's been learning based on the photos that people take, not the visions that people see, and this is giving AI a preference for photogenics instead of reality. People generally don't take pictures of rolling green hillsides. But they very often take pictures of rolling green hillsides with sheep on them. So if you ask the robot to draw a picture of rolling green hillsides, it will include sheep. Or, if you ask it to draw a picture of the savanna, it will want to include giraffes. Now you're being asked to find a bus in a photo without a bus because it's a street scene, and every street scene has a bus in it. I haven't read her book, yet, but her Twitter[2] is often full of amusing anecdotes like this. [1] https://aiweirdness.com/ [2] https://twitter.com/JanelleCShane

Just FYI, if you follow the links, it tracks back to the same blog. As do many/most internet posts about "weird AI names for things", at least in my experience. https://aiweirdness.com/post/174885047997/rock-climbing-routes-generat…

Janelle Shane: https://wandering.shop/@janellecshane She runs the http://aiweirdness.com blog, which is delightful

> Sometimes after a night’s sleep, we wake up with an insight on a topic or a solution to a problem we encountered the day before. The current crop of models do not "sleep" in any way. The associated limitations on long term task adaptation are obvious barriers to their general utility. > When conversing with LLMs, I never get the feeling that they have a solid grasp on the conversation. When you dig into topics, there is always a little too much vagueness, a slight but clear lack of coherence, continuity and awareness, a prevalence of cookie-cutter verbiage. It feels like a mind that isn’t fully “there” — and maybe not at all. One of the key functions of REM sleep seems to be the ability to generalize concepts and make connections between "distant" ideas in latent space [1]. I would argue that the current crop of LLMs are overfit on recall ability, particularly on their training corpus. The inherent trade-off is that they are underfit on "conceptual" intelligence. The ability to make connections between these ideas. As a result, you often get "thinking shaped objects", to paraphrase Janelle Shane [2]. It does feel like the primordial ooze of intelligence, but it is clear we still have several transformer-shaped breakthroughs before actual (human comparable) intelligence. 1. https://en.wikipedia.org/wiki/Why_We_Sleep 2. https://www.aiweirdness.com/

Oh man, good question. I'm always up for swapping these stories. A lot of these came from a paper on weird AI tricks, and resulting best-of list on a blog collecting these stories.[1][2] Suffice to say, the people who think the orthogonality thesis is a weird hypothetical aren't keeping up with the state of things. - The aforementioned Tetris story: an undirected learner was set to maximize score at Tetris learned normal play techniques, but also learned to pause the game immediately before losing so that the score wouldn't "decline" at game over. - In the same vein as interns quitting, proxy detection of all sorts. Identify "field with sheep" by finding green fields with grey skies, or letting heuristics like "humans pick up dogs and cats" override correct identifications. (It's a goat until you pick it up, then it's a dog!) - An agent playing Q*bert found a known bug for infinite lives, then escalated to an unknown bug which disabled the game while overflowing the score counter. - Agents in a physics sim tasked with jumping as high as possible instead learned to 'fly' by abusing collision detection bugs, hitting themselves in ways that created upward momentum. - Another "maximize jump height" task demonstrated that "highest" is an extremely fuzzy term. Initially measured by highest point, they became incredible tall. Measured by lowest point, they stayed tall and grew topheavy to 'kick' their base upwards. - Number-handling bugs of all kinds. In one case, small…

One of the things I'm rather worried about here is AI's ability to learn on the most unlikely (to human cognition) features. Take the image recognizer that finds "sheep" by seeing pastures where it thinks sheep might graze http://aiweirdness.com/post/172894792687/when-algorithms-surprise-us , or take any of the adversarial perturbing examples like https://blog.openai.com/robust-adversarial-inputs/ . You're not allowed to discriminate on race in housing. But is an AI that determines your creditworthiness for mortgages allowed to discriminate on what you eat, where you go to church, what your favorite music is, etc.? Maybe it doesn't have that data, but it will have one level higher - what store credit cards you have and how much you use them and where you opened them. If you train an AI on a segregated city with a history of actively discriminatory citizens, where the few people of race X who moved into a not-race-X neighborhood got harassed out and sold before they paid off their mortgages, how easily will the AI conclude that people born in certain neighborhoods are more likely to pay off their mortgage if they avoid certain other neighborhoods? Is that illegal? (My guess is there's no way to prove to a court, to the court's usual standards, that the AI happened to learn the city's racial tensions.) Should it be illegal, if outright racial discrimination is illegal?

This is a couple of years old now, but at one point Janelle Shane found that the only reliable way to avoid being flagged as AI was to use AI with a certain style prompt https://www.aiweirdness.com/dont-use-ai-detectors-for-anything-importa…

Hahah, there goes my evening... what a wonderful blog! E.g. https://www.aiweirdness.com/shaped-like-information/ – the prompt "Please generate a colorful guide to basic geometric shapes, as an aid to children learning to identify basic shapes" gave shape names like: CIRCLE CINCLE SQALE QUARE SQUARE CHALE ʤUARE TLIABLE TRINGLE SVAPE RESNTQON REUAGE RENATLE RECANGE HECTANBIE HOBOZ PSO SEOISUON HEXION FLOTN

It's funnier when you ask the LLM to explain something that isn't a meme and it hallucinates one anyways: https://www.aiweirdness.com/trolling-chatbots-with-made-up-memes/

And as a less dataset-quirk-specific example of AI fixating on backgrounds, here's Microsoft Azure dreaming of electric sheep: https://aiweirdness.com/post/171451900302/do-neural-nets-dream-of-elec…

I have the most bizarre sensation when reading through this. On one level, they're just boring brand names. But knowing that they were all created by Amazon for Purposes and their names chosen presumedly by marketing folks... Well, it sorta makes this read like the most genius piece of satire about today's retail marketplace. It would fit somewhere between McSweeny's lists[0] and AI-written paint color names[1] [0] https://www.mcsweeneys.net/columns/lists [1] https://aiweirdness.com/post/160776374467/new-paint-colors-invented-by…

The problem there isn't temporal consistency (although I agree that often sucks), it is over-reliance on context. The invisible sheep problem: http://aiweirdness.com/post/171451900302/do-neural-nets-dream-of-elect…

Related: the technology behind this has a harder time with cats than it does people, and the results are hilarious: http://aiweirdness.com/post/182633984547/gancats

Read some other entries in that blog, recipes are something to look at: http://aiweirdness.com/post/176589646292/its-time-for-cooking-with-neu… More of the same: https://www.reddit.com/r/SubredditSimulator/

Why the fuck does this link to the amp version of the site? Change http://aiweirdness.com/post/177091486527/this-ai-is-bad-at-drawing-but… to http://aiweirdness.com/post/177091486527/this-ai-is-bad-at-drawing-but… and it actually loads and looks proper.

I'm not so sure. Even prosaic, current AI, can be very unpredictable. See this blog post http://aiweirdness.com/post/172894792687/when-algorithms-surprise-us/a… , which is a somewhat more causal summary of this paper "The Surprising Creativity of Digital Evolution: A Collection of Anecdotes from the Evolutionary Computation and Artificial Life Research Communities" https://arxiv.org/abs/1803.03453v2 The authors of the paper explain a bunch of anecdotes when they wanted the AI to do one thing, but the measure they told it to optimise for wasn't actually what they intended, and so unexpected things happened.

Ah, good catch. It was discussed here: https://news.ycombinator.com/item?id=16600701. But maybe a second discussion isn't so bad. We'll change the URL to that from http://aiweirdness.com/post/172894792687/when-algorithms-surprise-us.

It seems that if you use Microsoft Azure for image recognition you would come to the conclusion that there are giraffe's everywhere: http://aiweirdness.com/post/171451900302/do-neural-nets-dream-of-elect… Maybe it's because Microsoft worked on a project to help conservationists protect giraffes: https://blogs.technet.microsoft.com/machinelearning/2016/04/11/how-mic… This might be an example of how a specialized dataset that was used to train an AI for a specific task, was later taken as part of another dataset to train a more general AI and the result was an AI with a "distorted view" of the world.

You can watch the lecture referred to in the article here: https://www.youtube.com/watch?v=rGgGOccMEiY The paper you quote is from 2022; it's also the last one Hinton posted to the arXiv: https://arxiv.org/search/cs?searchtype=author&query=Hinton%2C+G His resignation from Google and public warning about AI happened last month: https://www.bbc.com/news/world-us-canada-65452940 As LeCun is quoted saying in the article, "Geoff only started thinking about these things a couple of months ago". Finally, as already mentioned by others, agents finding unexpected ways to maximize their reward is a well known problem of reinforcement learning: https://www.aiweirdness.com/when-algorithms-surprise-us-18-04-13/ Fun quote from that 2018 article: "there was an algorithm that was supposed to figure out how to apply a minimum force to a plane landing on an aircraft carrier. Instead, it discovered that if it applied a huge force, it would overflow the program’s memory and would register instead as a very small force. The pilot would die but, hey, perfect score."

> Are you saying that GPT-6 would be less intelligent than a human, because we could give it an arbitrary deadline within which to answer the question, while not restricting a human that way? That doesn't sound like a fair test. The basic design of an ML model is just one really big matrix multiplication that always takes the same time - I'm saying we haven't given GPT the ability to not do that, so it always has that arbitrary default. It does have a system where it does beam search through the model instead of simply running it once, but it's not fully recurrent. > That's a reasonable intuition and a fair experiment, but I think it hinges on the question of "What does a human do when they go away and think about a problem?". Could it be that they are just clearing their mind of other distractions, and trying to look at the problem from multiple perspectives, which is something that a machine could do more efficiently? Well, some problems just take longer to think about or need knowledge that isn't already written down. > Putting it another way, if you met someone who had never performed an experiment before, would you think they were not intelligent? I think a prisoner or someone like that could demonstrate they're intelligent. If you're in the room with them, I mean, not like the Chinese room problem. But I think pure "intelligence" is just going to be less interesting that it looks, because it can't accomplish many tasks you'd expect it to if it can't actively…