It seems that if you log into Poe.com, you get one free query to each of the new GPT-4 and Claude+ engines in each calendar day. Given the vital importance of Pony, I chose to use today's free query to re-run the test I gave ChatGPT in December. I used exactly the same prompt as I did then, so you can compare the outputs of each model.
Prompt: Please write a My Little Pony story in which Fluttershy and Rarity discuss a sensitive issue over a cup of tea. The two ponies are very long-standing friends and trust each other completely. Both of them are quiet conversationalists, so the conversation should have a restrained emotional tone. However, the much brasher Rainbow Dash barges into the cottage halfway through.
1. A Delicate Conversation [by GPT-4]
Once upon a time in the magical land of Equestria, Fluttershy and Rarity were enjoying a leisurely afternoon together. The sun was shining brightly as they sat in Fluttershy's cozy cottage, the air filled with the sweet fragrance of blooming flowers.
Fluttershy had prepared a delightful tea party for the two of them, complete with an array of delectable treats. The elegant china tea set, a gift from Rarity, gleamed in the sunlight as the two friends sipped their tea.
Fluttershy took a deep breath and spoke up softly. "Rarity, I've been wanting to talk to you about something that's been troubling me."
Rarity looked at her dear friend with concern. "Of course, darling. You know you can always confide in me."
Fluttershy hesitated for a moment, gathering her thoughts. "Well, you see, I've been noticing that Angel has been acting rather... distant lately. I'm worried that I might not be giving him enough attention."
Rarity nodded, understanding Fluttershy's concern for her beloved pet. "It's only natural to worry about those we love, dear. But I'm sure you're doing your best to make Angel happy."
"But what if my best isn't enough?" Fluttershy whispered, her eyes filling with tears. "I feel like I'm failing him as a friend."
Rarity reached out and patted Fluttershy's hoof reassuringly. "Fluttershy, you are one of the most caring and devoted friends I know. I'm certain that you're giving Angel all the love and attention he needs."
Before Fluttershy could respond, the door to her cottage burst open, and in flew Rainbow Dash, her wings flapping excitedly. "Hey, you two! Guess who just broke the Equestria Speed Record? This pony!"
Fluttershy and Rarity exchanged a glance, their delicate conversation interrupted by their boisterous friend.
Rainbow Dash finally noticed the somber atmosphere and sheepishly rubbed the back of her neck. "Uh, sorry. Did I interrupt something important?"
Rarity cleared her throat, trying to maintain the relaxed atmosphere from before. "It's quite alright, Rainbow Dash. We were simply discussing a sensitive issue."
Rainbow Dash's expression turned serious. "Oh, I see. Well, if you need any help, you can count on me!"
Fluttershy smiled at Rainbow Dash's earnest offer. "Thank you, Rainbow Dash. That means a lot to me."
The three friends spent the rest of the afternoon enjoying each other's company, sharing stories and laughter. As the sun began to set, Fluttershy felt her heart swell with gratitude for her friends and their unwavering support.
With Rarity and Rainbow Dash by her side, she knew that she would find the strength to face any challenge, no matter how delicate the situation may be.
And so, as the day came to a close, Fluttershy, Rarity, and Rainbow Dash shared a warm embrace, knowing that their friendship would always carry them through the toughest of times.
2. [Untitled] [by Claude+]
Fluttershy poured Rarity a cup of chamomile tea and slid a plate of daffodil sandwiches towards her friend. "Thank you for coming to talk, Rarity," she said softly. "I know how busy you are this time of year."
"Nonsense, dear, I will always make time for you." Rarity took a delicate sip of tea. Though she and Fluttershy had been close friends for years, Rarity still found their heart-to-heart conversations somewhat taxing. But Fluttershy had been there for her in her times of need, and Rarity would do the same. "Now, what seems to be on your mind?"
Fluttershy fidgeted with her hooves. "It's about Discord. We've been spending more and more time together, and I think my feelings for him are growing deeper. But so many ponies still don't trust him, and I worry what they might say."
"I understand this is difficult for you," Rarity said calmly. "However, you cannot let the petty judgments of others dictate who you care for. As long as Discord treats you well and makes you happy, that is all that matters."
"You really think so?" Fluttershy asked.
"Of course, darling. Love is too precious to--”
Suddenly the door burst open in a flash of rainbows. "Hey Fluttershy, are you busy? I was hoping we could--oh, sorry, didn't know you had company!" Rainbow Dash glanced between the two ponies. "Did I, uh, interrupt something?"
"Not at all, Rainbow Dash," Rarity said, with the faintest hint of annoyance. "We were just discussing the complexities of interspecies romance. But do join us."
Rainbow Dash blinked. "Come again?"
Fluttershy blushed brightly. "It's a long story..."
Oh god… that second one has picked up on fandom ships/the shiptease that is "The Last Problem"…!
ReplyDeleteIt also exposes the cracks in this, because Rarity's reaction to Fluttershy developing a crush on Discord would include at least some worry and nervousness, not this copy-and-paste-as-though-it's-any-old-pony schtick. These AI writing bots are not nearly at the level of parsing out individual character relationships and opinions on each other!
Otherwise, this doesn't feel markedly different from before, akin to just refreshing the result ChatGPT would have given you. It is amusin how the first one today has a similar "Dash crashes in, gets the lowdown, then leaves" result, albeit a bit more thoughtfully, while the second has Dash sticking around. Most curious…! I'm sure other folks more bothered then myself would make note of the other differences.
I definitely prefer Claude+'s attempt. GPT-4 disappointed me a bit, to be honest: it has a boringly clichéd opening and begins far too many paragraphs (11 out of 17) with a character's name. I'm not fond of "her boisterous friend", either, since anyone reading ponyfic knows what Dash is like.
DeleteClaude+'s story may not have a title and it's very dialogue-heavy, but I prefer it to read. It gets straight to the action rather than doing a weather report, it suggests something quite interesting (that Rarity might still find Fluttershy a taxing conversationalist even though they're so close) and it's willing to end without doing the "everypony hugs" thing. Breaking off a sentence for effect as Rainbow appears, too. It reads like a more confident writer, if you see what I mean.
This isn't to say that I adore either of them! But maybe part of the problem is that the bots (at least the versions I can access) won't do longer-form stories, so you end up with a fic being squashed into 500 words when it really needs three or four times that space at minimum. For example, I think Rarity's reaction would depend on exactly when this is set, and whether the two had talked about this subject before. We can't tell from the little we're given here.
We're still a good way from "AI can write a good story", though I think that may be almost a side-issue anyway. (All the more so now fully AI-produced material has been ruled ineligible for copyright in at least some jurisdictions.) As helpers for human writers, though, I think we're in more interesting territory already. Also, points to both bots for not asking Rarity to go for a fly this time! ;)
Argh, "their boisterous friend". Point stands, though.
DeleteOh, Claude+ is a better piece of writing for sure, both for the reasons you cite and those iisaw brings up. I didn't miss that. I was just lamenting it took bringing that ship into the mix for it to do so, is all. Also, I just felt they were still too far out from being usable for a human to bothering saying so.
DeleteIt'd be interesting to see if this is just the outcome of these specific results, or if Claude+ does consistently produce better results then GPT-4's. Food for thought, my friend.
GPT-4's reads very much like an author's first fic, and Claude+'s reads like just the intro to a more mature (in the sense of the author's capability*) fic.
ReplyDelete-------
* I hate how porn has ruined the utility of that word.
"Mature", too...
DeleteXD... I'm tired.
DeleteI want to see a video about international diplomacy that starts with the warning "FOR MATURE AUDIENCES ONLY".
DeleteTempted to feed in the teaser line for Traveling Tutor and see what it does.
ReplyDeleteDid you?
DeleteThis comment has been removed by the author.
ReplyDelete