I've fumbled around previously with my own definitions of 'fiction-first' vs D&D* TTRPGs. I mentioned that those games generally have a different 'locus of simulation' -- what are the rules primarily interested in simulating? Which to me generally seems to be 'the actions of the actors in this fictional world' or 'a piece of fiction in this style or genre'.
For me, that's still a useful lens, particularly in terms of the clarity it's given me about what works in some and not others. Most obviously: D&D* doesn't really do anything in terms of teaching its players narrative/dramatic tools -- around pacing, framing scenes, getting to the interesting stuff, etc. If you bring those skills in from outside, there's a better chance of having a good time. Fiction-first TTRPGs often have explicit rules, tools, or guidance that gets people thinking along those lines ('you are not just playing a character, you are an actor in a story').
I bring this up again (again!) because I read this piece: Three layers of RPG rules, which I think approaches this in a more systematic and interesting way.
Go read the piece, but to summarise: the 'rules' in TTRPGs aren't really just the bits about maths and rolling dice and how abilities work. They are also the social rules of the table and the fictional rules of the world. Different TTRPGs put the focus in different places (what I'd call the 'locus of simulation', though I think these layers usefully collapse/muddy that a bit), but it's more productive to think about all of those things as being 'rules', because it helps us disentangle what we really need to understand to have a good experience of a game.
I also love the examples of choosing to keep rules in the fictional layer:
First, I welcome you to take a look at the “powers” list in Project Ikaros, or “talents” in Eos or Legends. Here are a few:
Superheating: Scald or melt with a touch.
Sincerity: People may or may not believe you’re right, but always believe you’re honest.
Abjuration: Enemies cannot approach or attack while you chant, so long as allies do not attack.These lists favor fictional rules over abstract rules. Thinking about what I need at the table, and what I find easy or hard to adjudicate, led me to fill pages with abilities I hoped would be self-evidently useful. They operate at the fictional layer because I find it easier to just describe the obvious result of being touched with a superheated hand than to I find it to track abstractions like hit points. And I still think of them as “rules” because they still provide guidance we need to abide by; I may well need my player to remind me, “I put on my most sincere smile, and count on him to believe me…” That governs what is allowed to happen next. It’s a rule.
This is another feature I associate favourably with fiction-first TTRPGs. Coming from a D&D background, coming across rules like this can feel vague on paper ('but... what am I allowed to do with it?'), but feel very empowering in play. (The flipside is usually around game balance, but that doesn't have the same concern where the game is less invested in the abstract rules layer.)
Closing thought: also check out the Indie RPG Newsletter #98: Play Culture of Why I Play How I Play for another perspective on these. Above all, I still enjoy architecting elaborate stories to delight and horrify my players, which pulls me back to D&D* games. But that's a point of differentiation, rather than something implicitly superior.