I finally did some prodding at ChatGPT last week. I wrote a little bit before about some of my problems with the way these things are trained, though the problem is slightly less acute (though still present) with text-based models.
My first prompt was to have it write a bio for Olivia. Where it confidently declared she'd worked on several games that she hadn't, had attended a university several thousand miles away from the one she had, and had won several awards that she hadn't. The most correct thing about it came in the form of a sentence verbatim lifted from her website.
My experiments progressed to having it construct several dramatic scenarios, having it rewrite scenes from Shakespeare, but replace characters with pigeons, or Brian Blessed, or set them on a submarine.
BRIAN BLESSED PIGEON: (undeterred) And might I add, your beauty shines brighter than the moon in the sky at night!
(Strictly speaking, I'd asked it to change the dove in a previous draft into Brian Blessed, but I guess it technically followed the instructions.)
Or having it apply for a job as an assistant, but as a pigeon:
As a pigeon, I am highly adaptable and able to learn quickly. My ability to navigate my surroundings with precision and agility would make me an excellent researcher, helping you to gather information and inspiration for your stories. Additionally, my keen sense of observation allows me to pick up on small details that others may miss, which would be valuable in the editing and proofreading process.
I also have a unique perspective that would bring fresh ideas to the table. My avian nature allows me to see the world from a different angle and provide a unique perspective in the creative process.
I then tried teaching it to GM a TTRPG for me, which... did actually seem sorta possible in a slightly unexpected way, with steady coaching of the right prompts. It struggled with numbers, and I am suspicious that it wasn't just picking favourable outcomes for dice rolls and telling me that I had rolled an X. But if anything, that's a legitimate -- if not actually desired -- approach to the problem.
Act as the game master in a single player improvised tabletop roleplaying game set in the world of Buffy the Vampire Slayer. Answer as the game master. I am the only player. Do not write explanations. Start with character creation. ... Refer to me as if I’m Willow; don’t talk about her in the third person. However, I am not actually Willow, I am just playing the character of Willow. ... When I talk in character l, I will use speech marks “Like this.” When I talk in character, don’t repeat my speech in your reply, just paraphrase what I’ve said instead. Other character should still talk to me in direct speech.
(Some of the prompts that went into setting this up. I'd actually like to give this another go.)
My overall feeling from this play was that it was great for just that -- play. Seeing ChatGPT as a vector for entertainment felt good; it's applications beyond that much more tenuous.
Dan Hon:
Bard made a factual error in its first demo7. A point here is that the demo confidently asserts that the James Webb Space Telescope “took the very first pictures of a planet outside of our own solar system”, and, whooeee let me interpret it this way: there’s no way this demo would’ve gotten out without going past a bunch of (human) eyes checking it was okay. It clearly got past all those eyes. One reason why, I’m pretty sure, is that LLM-generated text is super easy to glaze over entirely because it’s so predictable and we’re already framed to predict what it’s going to say. Which is a major point of the criticism of these models: they’re great until (which is frequently) they assert something that we don’t check. Which people didn’t check sufficiently. And now you’ve got AIgg on your face, big disgrace, etc. Things That Caught My Attention s14e14
Interconnected:
Look, I know the code isn’t rocket science. I know I could do this, eventually, and you could probably smash this out without looking, but I don’t really know React - I can’t write it idiomatically - and I don’t know about webcams in the browser, and I don’t know about the MediaStream API.
So this was a day of work in 10 minutes.
What is meant was that I could spend that day integrating hand pose detection and noodling with the actual micro-interactions. And now I have opinions about all of that!
Now, none of that Copilot-supplied code remains in my app.
What happened what that it helped me frame my problem. I was able to rapidly explore the edges of my knowledge, and figure out how to structure my questions and what I need to learn. My learning requirement is not obviated obviously… Interconnected: AI-generated code helps me learn and makes experimenting faster
(I also read something recently about a well-known concept artist using AI art as a means of noodling at ideas/exploring surprising conceptual spaces, but I can't track that piece down again right now.)
Having played with it some, I can see the magnetic draw and fascination ChatGPT inculcates. The problem is that I think a big part of that comes from this very superficiality -- its ability to bluster confidently. Which is, again, great for entertainment value, and absolutely lethal for the uses I think most people imagine for it.
Setting aside my concerns around how this is implemented and rolled out – which are MASSIVE, and honestly I am more than a little scared by the ramp-up in recent days to make this ubiquitous because dear God are we not ready for that, technically or socially – but setting that aside, the value here for me seems to be for uses where unexpected or unreliable outputs are baked into the intent. So, creative experimentation, generative play, or rapid prototyping.
But I feel that, while they exist, those use cases are much more niche, limited in appeal to a much smaller number of people. Which is not what the companies shipping this things are banking on at all -- they're banking, literally, on it being the next big money spinner, and working to capture the next big information market. Unfortunate, then, that the information that market is selling is rather spoiled.