Discussion about this post

User's avatar
artworm's avatar

Prompting language is definitely something inbetween programming and natural language. Pure natural language does NOT work well with AI.

The best example of this, is how stable diffusion prompting radically changed over time. It started with people trying to type natural language, the AI did 'understand' the subject matter, but the results were terrible looking.

Then came the Greg Rutkowski era, of cargo-culting prompts, which improved results somewhat.

Then came the big revolution of the NovelAI model. What makes it special, is it uses the danbooru tagging system (definitely NSFW), where an image is tagged via 20-50 1-2 word tags. The tags are not arbitrarily decided upon, but is consistently understood and enforced by the community, avoiding descriptions that don't map to any direct visual feature.

So the novelAI prompts look like this: Masterpiece, 1girl, long hair, red hair, dress, street.

There's no longer any human grammar involved. And the results work way better.

Since then, even the photo-style SD models, are trained from the NovelAI dataset, and use the tagging system originally developed for anime pictures.

What this suggests, is that pure natural language prompting may not be the end result of AI. AI will prefer a subset of natural language that is more precisely and consistently defined, with different grammar rules. That becomes the new prompting language.

Expand full comment
Eric Nichols's avatar

This old novel called "A Working Theory of Love" is essentially about this guy in Silicon Valley hired by a Ray Kurzweil-esque figure to do lots of manual interaction with a chatbot; basically conversation-engineering, similar to prompt-engineering. Crazy to think this is now a real job instead of just fiction.

Expand full comment
4 more comments...

No posts