On The Value of Fresh Eyes

Evaluating works-in-progress, featuring dozens of pages on Poly Fighter

I recently played the invite-only demo¹ for upcoming game Poly Fighter and had some thoughts - 28 pages of thoughts.

I’m not going to discuss the writeup itself in depth — you can read it if you’re so inclined. But I’ll touch on my mindset with this sort of document while using it as an example of the value of fresh eyes on a project.

It’s 28 pages of issues with few compliments, so you might assume I’m negative on the game. But I could write this volume of text for any in-progress game. I think Poly Fighter, while early, is pretty cool.

If you make a long list of issues to fix, then fix them, you’ll probably end up with a game that’s good if not great. A long list of issues is daunting, and fixing them is way less fun than dreaming up new features. But that’s why game development is a job and not just a fun hobby. (Though it can be that as well)

Many great games don’t have exciting new features and aren’t uniquely inspired. They become great simply by virtue of tediously improving what needs it.

I would refer to 15 Years of Indie Dev In 4 Bits of Advice and this bit:

How did we make Wizards good? We asked players which bits were bad, then fixed them.

Redfall has a 57 on Opencritic. The exact same game with issues addressed would be a 77, maybe up to an 82. Pushing higher than that might involve more fundamental changes: ditching the animatic-styled cutscenes, making the unique character abilities more interesting, delivering on the Far Cry 2-style systems-driven open world that was teased but is nowhere in the shipped game. But creating a long list of issues to fix then tediously fixing them would have given the game a significant bump.

I view a writeup like this one for Poly Fighter as a list of issues to address. As such I don’t do “sandwich a complaint between two compliments.”

Too many negatives and no positives might be demoralizing. But a list of compliments, while motivating, isn’t actionable, and overly-kind feedback can be misleading. A common element of some of the most troubled projects is a “toxic positivity” mindset where negative feedback is both ignored and discouraged, or where the developers have an insular view of their game. (If nothing else, adding two compliments for every issue would make my document 84 pages long…)

“Toxic positivity” gets thrown around often now. Here’s Josh Sobel, a lead tech artist at Wildlight who was let go along with most of the staff immediately following release, describing Highguard:

“The future seemed bright. Everyone I knew who had any connection to the team or project had the same sentiments: ‘This is lightning in a bottle.’ ‘I trust this team wholeheartedly.’ ‘If there’s one project nobody in the industry is worried will fail, it’s yours.’ ‘This has mainstream hit written all over it.’ ‘There’s no way this will flop.’ ‘I could play this game all day,’” Sobel said.

I’ve played Highguard and I don’t think it’s a bad game. But I could make a long list of issues that could be improved or fixed, and that’s a released game, not an in-development one.

I don’t love the term “toxic positivity” as people apply it to teams without knowing anything about that team’s culture. Sometimes teams suppress negative feedback through a variety of cultural mechanisms, and reward the relentlessly positive as “team players.” But sometimes teams are just naturally optimistic and perhaps a smidge pollyannaish — that’s not “toxic positivity” that’s plain old positivity. But plain old positivity can still be a problem and result in the same conclusion.

As such I think following a “sandwich a negative between two positives” approach does the team a disservice, that spares their feelings at the risk of damaging their long-term success chance.

A last note about my Poly Fighter writeup is that, were I a new hire on the team, or a consultant rather than a total outsider, it would be much less talky, exclude many known issues, and cut down on the number of suggestions. Stating known issues can be valuable to underscore their importance, but I wouldn’t mention the end-of-match jingle, for example, if I knew it was a temporary asset.

Issue Blindness and Losing Perspective

Older games from the era before extensive playtesting often have glaring balance, understandability or control issues.

One reason for this is simple: while working on a game you eventually get used to the current state, in the same way someone becomes used to the particular smell of their house.

A game developer might have played their game for hundreds of hours. Not only are they used to enemy patterns, the controls, etc, but they designed those enemy patterns and controls. Game designers often vastly overestimate how understandable their systems are because they designed those systems and understand their underlying logic and motivation, whereas a player can only interpret that logic via what’s surfaced in game and can only guess at why that system exists at all.

As a developer you never have a legitimate new player experience on your own game, and to whatever extent you approximate it that quickly becomes a distant memory. Very few developers can put themselves into the shoes of a new player.

To use some specific examples here, players complain that the loot-finding stage of Highguard is dull. The developers understand the map layouts, the traversal and mining mechanics, and the relative strength of mined crystals and store-bought items vs loot found in treasure chests. So to them optimizing their paths to maximize preparation and economy is an interesting minigame. But to a new player there aren’t interesting tradeoffs because they don’t understand what those tradeoffs are.

2XKO has some odd control choices but after hundreds of hours they become muscle memory. The developers aren’t going to forget which button activates which move or accidentally get a super move instead of a special move because they pressed a button sequence too quickly.

I played the demo for Queen’s Domain, a fantastic-looking faux-retro Kings Field-like. The first enemies I ran into were insanely difficult — each of my hits did 10% damage to them and they did 30% damage to me, and I was outnumbered three-to-one.

The developers probably know that if you run and move in a certain way the enemies can’t hit you at all, and if anything think they’re too easy.

Low Prioritization of Avoidable Problems

Another element of blindness is voluntary: when developers know an issue exists but continually de-prioritize it because they know to avoid it.

To use a personal example, I’m working on a game with a 3D camera. The older implementation of the camera had some odd behavior: as the player rotated the camera down into the ground the camera would slide along the ground closer to the character, then as the player rotated the camera back up it would slide back along the ground instead of immediately moving upwards. It felt like the camera was magnetized to the ground. It didn’t honor the player’s input; the camera moved out when the player told it to move up. I’d planned to fix this issue eventually, but I didn’t consider it a big issue because I simply learned to avoid it. As the funny Doom tweet said “you control the buttons you press.” I stopped rotating the camera down into the ground so the resulting behavior stopped bothering me.

Coming back to the project after some time off I realized how incredibly annoying this issue actually is, and fixed it and a number of similar issues immediately.

The Burden of Historical Knowledge and Being Smart Enough to Rationalize Bad Decisions

A common game development failure pattern is when decisions are constantly revisited or undone — when flying in Anthem is added then removed then added again. But an opposite problem can also occur: a feature is tried and dismissed, and remains mentally categorized as a failed effort when the context (including office politics) around it changes.

It’s usually the case that when players make an obvious suggestion the developers already tried it or avoided it for good reason: the feature didn’t work out in practice or was too work-intensive. It’s rare that the single-most-obvious idea just never occurred to the team. But it’s not that rare that a feature that didn’t make sense at the time makes more sense now.

In that case knowledge of development history is a burden, and it can be clarifying to hear a naïve “just try this obvious thing” take.

Sometimes it’s useful to ask “what would a fresh player and absolute dummy (no offense) think?” An example of this is Nintendo’s insistence on displaying long lists of options as a scrolling single-line rather than a grid or radial-menu or matrix.

Presumably Nintendo has real reasons to stick to this design — they did some research that showed that players struggle with more complex UI, for example. But I suspect that those reasons are outdated, inapplicable and need revisiting. Maybe sticking to a single horizontal line makes sense when the UI only displays up to 15 icons, but breaks down when Echoes of Wisdom displays dozens of choices.

There’s a smart person at Nintendo who could make a sophisticated argument for this UI style, while any dummy can say “this single-line list is just too damn long!” But in this case the dummy is probably correct.

Similarly Nintendo’s sports games no longer have the story / RPG modes that people enjoyed. I assume there’s a good rationalization for that but that’s not the same as a good reason. Sometimes developers can outfox themselves and execute on a sophisticated philosophy that performs worse than “give the people what they want.”

Becoming Blind to Strengths

There are many ways to become blind to weaknesses in a game, or blind to fruitful potential avenues. It’s also possible to become blind to the strengths of an in-development game. I haven’t played it so this is an uninformed take, but my impression of Big Hops is that it has many features that don’t relate much to the core appeal. It has a Mario 64 move set and adds wall running, climbing, and swinging with a tongue — that’s more than enough for a full game. (It’s more than what Mario 64 offers) But it also has items (not just temporary powerups), an inventory and inventory management, swappable trinkets that provide different perks, NPCs, quests, and minigames where you use your tongue to solve Pipe Dream-style puzzles. Looking at these minigames I can’t help but wonder if the team, after hundreds of hours of play, grew too used to the core mechanics and worried that the game was too plain and needed more pizazz in the form of new features.

Game Developer Critique of In-Progress Games

In my first blog post, originally written for GameDeveloper.com in 2015 about Dark Souls, I explained my motivation this way:

This is inspired by Matt Zoller Seitz's Please, Critics, Write About the Filmmaking, the gist of which is that critics, rather than writing solely about how a film made them feel, should examine how the film made them feel that way.

I consider much of what I write craft critique.

Any player can write “I liked the music” or “the controls are unresponsive.” There’s nothing wrong with those observations but a game developer sharing them is functioning as a player, not as as a professional — like a director praising the work of another director with “I enjoyed the movie!”

I don’t expect you to read my 30-pages of Poly Fighter impressions (really it’s like 17 pages and some screenshots) but I’d highlight a few sections to underscore the difference between player and game developer feedback.

I don’t think many players would include drawing-order issues in their feedback. They might identify related symptoms: the action is harder to read than it should be, it can be hard to tell if a hit landed or where it landed, hits are less satisfying than they could be, some VFX can look wonky, etc. But they probably wouldn’t identify z-axis sorting issues as the culprit.

Lack of clarity around in-combat rules like air-juggling is another issue they might obliquely gesture at without identifying directly.

Craft Critique vs Other Type of Criticism

This sort of feedback isn’t a review. I often bristle when reviews contain suggestions. At that point the game has been released so (patches and live service games notwithstanding) that feedback isn’t actionable. Most game reviewers aren’t game developers so “here’s what I would have done” is silly. But a game developer giving suggestions on an in-progress game is potentially helpful.

Live Play Criticism

I would contrast this sort of criticism to criticism-as-content. I’ve watched videos where a game developer plays a game on Twitch or YouTube and shares criticism — but I’ve never seen a good one. That format isn’t conducive to useful, honest feedback.

Playing a game live there’s an incentive to just keep talking. If you keep asking play-testers for their impressions, after they’ve exhausted their initial feedback, they’ll often simply make things up to avoid appearing like they have nothing to say. In the same way a game developer doing running commentary has to fill silence with something, whether it’s pertinent or not.

The format discourages grace and consideration. You might play an RPG that introduces a new battle mechanic that’s initially hard to grasp — often RPGs are designed with the expectation that new systems are initially opaque but will become clearer over time. But if you’re playing a game live and doing running commentary as content there’s a strong incentive to declare that the tutorial is bad or that the game has “poor onboarding.”

My greatest objection to this sort of commentary is that it’s often ego-driven, where the developer providing the criticism is fundamentally motivated by the desire to show off and say “here’s how I would have done this better.”

This sort of criticism helps the person doing the play-through accumulate views and gives their audience something to chortle at, but doesn’t help the game’s developer at all. This sort of criticism delivers flexing, not real insights.

Arguably any criticism intended for any audience beyond the game’s developers is “content” — I’ve turned my Poly Fighter writeup into blog content. But I’d consider most live-plays “#content” — a pejorative.

Mining New Hires for Impressions

Not many game developers are going to write something like my Poly Fighter writeup — at least not for free. This kind of craft critique is valuable but in scarce supply.

As a developer you can do playtests, betas and early access, but while that provides useful feedback it doesn’t provide this type of feedback.

I’ve heard that some companies solicit this sort of feedback when interviewing game design candidates; but I’ve also heard that game designers don’t like being asked for this sort of feedback. I’m not sure what the exact objections are but I imagine it’s awkward to be asked to critique a game you’re hoping to get hired on — a sort of “describe your greatest weakness” question that dissuades honest answers. It might also come off as spec work. (Valuable but unpaid real work.)

At conventions other game developers might play your games and offer feedback, but those impressions are usually based on a few minutes of play. And I’ll be honest here: when I’ve shown up to E3 industry-only hours, played a game and been asked for feedback, I’ve always been positive, even when I wasn’t positive on the game. I pretended not to notice the screen-tearing in Killer is Dead and I said the cinematic QTE car chase sequences in Spider-Man: Web of Shadows² were “neat” when I found them dull. Conventions are fun atmospheres and players, including other game developers, shy away from being downers. If your game gets a tepid response at a show that’s a bad sign, but if it gets a good response at a show that’s probably a nothing sign.

You can ask game-developer friends to play your game and give feedback, but my impression is that that feedback, though it varies widely, is often quite succinct. And depending on the personality of your friend and the nature of that friendship it’s likely to be overly positive.

You can pay outside consultants for mock reviews and game evaluations. A mock review is when an ex-journalist writes a review as if they were still at a review website: “your game would get a 7.5 at Gamespot and here’s why.” A game evaluation is when they write “and here’s how it could get an 8.5 instead.” Mock reviews are valuable, but the people offering these review and evaluation services tend to be ex-journalists with no experience making games, so I don’t know why you’d take their advice on how to make yours. Game journalists consulting on a game are articulate gamers more than they’re game developers, even if they’re now technically in the game development business via consulting.

So while game developer feedback is valuable there aren’t many places to find it. Which brings me to new hires.

It can be awkward for new hires to be critical, but it’s less awkward than it is for job seekers. Asking a new hire to provide this feedback isn’t asking for a favor or spec work — it’s asking them to do their job. (As you define it)

Some people aren’t well-suited to offer this sort of feedback. They may be too polite, too narrow in interests, or have off-kilter opinions. But each new hire is at least a potential resource.

I don’t like to give advice as I hardly have life figured out. But I think it’s wise to treat new hires as a potentially valuable resource with a limited shelf-life — certain flaws fade to low-priority or invisibility within weeks or even days. Have them meet the team, get set up and familiarized with processes and software, go through all the HR rigamarole, then have them play the game and provide feedback before they’ve been subject to in-depth system and mechanics explanations, so that they’re getting an experience close to that of a new player. That gives you access to an expert, in-depth opinion.

And that opinion isn’t fire-and-forget, the way playtest sessions or outside consulting might be. I was once hired on a project and some of my first feedback was “the pathfinding stalls the entire game”; one of my first tasks was fixing the pathfinding stalls. Which makes sense — let the guy most bothered by that issue improve it till it meets his standards. Even if I wasn’t technical enough to fix the issue myself I could have been assigned to approve any fix. This is also a natural way to combat toxic negativity, by allowing people to be the change they want to see rather than encouraging fruitless griping.

1. As I put the finishing touches on this the team released a full demo, but my comments are based on an invite-only one from winter 2025

2. I think it was Web of Shadows, it might have been a different Spider-Man game