Prompt transformation and the Gemini debacle

I heard a new term this week: “prompt transformation.”

This keeps popping up in discussions of the Google Gemini fiasco. It’s probably not brand new, but I heard it for the first time from Kevin Roose on the Hard Fork podcast.

I like it -- it's a succinct way of describing what’s happening under the hood when Gemini generates a racially diverse image of the U.S. founding fathers. Users’ prompts are being rewritten or modified before being sent to the model. In this case, it appears the prompts were modified to include a request for racial diversity in the output.

This practice isn’t new, or unique to Google. OpenAI in 2022 announced a “new technique” that improved the diversity of images generated by its DALL-E model; the new technique was almost certainly prompt transformation.

I’ve also heard this technique called “secret prompt engineering”, which underlines the fact that the transformation is usually hidden from users.

Roose made some great points:

▶ Keeping this secret from users makes it ripe for conspiracy theories and culture wars
▶ Instead of doing automatic prompt transformation, the system could be set up to ask follow-up questions before generating images or text
▶ At the very least, the system should show you the transformed prompt

However, I don't think companies will embrace the last point. They might see their prompt transformation recipe as a trade secret, or as a way to hide embarrassing limitations of their models.

What do you think about prompt transformation? Have you heard other names for it?