The ability of computers to semantically understand the world – and the humans in it – is next to nonexistent. Computers are wonderful at crunching numbers, transforming data, and analysing the organisation of bits and bytes in any forms, but only in so far as those bits and bytes are complete in themselves. When it comes to a holistic understanding of how that data relates to the world in which they live, computers remain inept, unable even to understand the rudimentary meanings of the English language. Try to ask Google a question, rather than just punching in words to search for, and watch how quickly Google will run into difficulties understanding what question you’re asking. It might know ‘What year was Obama born?’, but ask ‘What term is Obama in?’ and it cannot tell you ‘His second.’
In the near-term seeking such human-level understanding is a futile endeavour. In a previous article for The White Review, ‘Choose Your Own Formalism’, I wrote about the formal aspects of human-computer interaction in games: the ways in which the limitations of computer representation of characters and ideas place constraints on what can be done within the confines of digital interactivity. These delineate the difficulties computers have in simulating – and thus in representing – human character and interaction.
There is little cause for optimism in that regard. The ability to create interactive analogues for the complexities of human activity and emotion are limited by the need to specify them to the very finest detail. Unless a computer can ‘think’ like a human, which it certainly can’t, the only way for human-like interaction to take place is if the designers manually code up every variation and possibility, an endless, exhausting, and uncommercial task. But there is a much richer vein of development, one that promises far more surprises and advances in the year to come. I am talking about interactive modalities, the means by which we exert control over our player character counterpart within a game. There is one catch, however, which is that it is entirely sub-rational and non-cognitive.
When I speak of modalities, from controller to keyboard to mouse to joystick, it is important to realise what I am excluding, which is any notion of the degree of player choice. I am not concerned here with the means by which a player chooses one course of action over another in a game. As soon as we consider the complexity of player choice in a game, we are back to the issues which so vex game designers trying to mirror the richness and ambiguity of life in a formal representation.
Instead, the issue is one of agency. Just as the stage and film offer a different modality for empathy with a character than a written text, allowing a performer to convey the visceral person-ness to the audience without the scriptwriter needing to work quite so hard at it, so too does the relation of a game protagonist’s actions to the pushing of buttons and levers grant the player a deeper visceral connection to the acts of that protagonist. This mechanism exists regardless of the actual degree of player choice.
For example, there’s The Walking Dead, the post-apocalyptic zombie game developed by Telltale from Robert Kirkman’s comic book series of the same name. As an interactive experience, the game is, as the saying goes, ‘on rails’ – there is very little you can do that meaningfully affects the resolution of the overarching plot. Some characters may die earlier or later, but those marked for death will nonetheless die, at which point you (or someone else, but most likely you) will have to push the button and blow their brains out so that they don’t become zombies. When speaking to other players, there are usually at most two dialogue options, and the effects of picking one over the other are generally inconsequential, thus reducing the programming labour of coding up many divergent plot paths.
Kotaku described the game as putting players ‘through the emotional wringer,’ and a mass of angst-ridden Youtube videos of players gnashing their teeth over the fates of the characters is testament to that. Yet the emotional grip, then, does not come from any autonomous decision on the player’s part, but from two other factors: first, the believably realistic animation and voice acting, of a higher grade than the norm; and second, the requirement that the player act to advance the story. This is not really a choice per se, since the story simply won’t continue if you don’t do what the game wants you to do, but it is nonetheless a volitional act on the part of the player.
In one scene midway through the game, a preadolescent boy named Duck has been bitten by a zombie. You (the protagonist, Lee Everett) have been travelling with Duck and his parents for the entire game. He is still sentient and conscious, but he must be shot in the head before he dies or he will turn into a zombie and attack you. You can let his father do the deed (a real, if narratively irrelevant, choice), but you can also do it yourself. If you choose to do so, the gun’s crosshairs appear on the screen as you look through Lee’s eyes. You move the crosshair over Duck’s head, and push the action button to fire. There is no puzzle or challenge in the act, but Lee won’t fire until you have aimed his hand and pulled the trigger. As soon as you push the button, there is a loud gunshot, the controller vibrates with its force-feedback to simulate the recoil of the gun, and Duck’s head explodes.
The scenario is modelled to simulate, as best as possible given the input modalities, the act of the player him- or herself pulling the trigger. Although no actual ‘choice’ enters into it, the player nonetheless feels a physical agency for Duck’s death of a nature not possible if the player were simply watching a film shot in the first person. The brain processes the gunshot and vibration as a direct effect of the action taken by the player: pushing a button.
You can say that this is horribly manipulative on the part of the game designers. You’d be right. But there is no doubt it has been tremendously effective, not only establishing the game as among the best of recent years but. more importantly, convincing a great many people that they are having a deep and resonant aesthetic experience. Players’ sympathy for Lee and Duck is indisputably real. That a bit of ham-fisted technological sadism helped the process along is not relevant.
Moreover, that emotional effect was achieved without the computer requiring any formal model of the scenario, characters, or plot. It just needs to update the animation accordingly. Everything has been pre-animated and pre-recorded. Interactivity is minimised to a single, emotionally loaded point.
The simplest instance of such a point is termed a ‘quick time event’, or QTE, a phrase invented by Sega’s Yu Suzuki. In their original form, QTEs were moments in a game’s cinematic cut scenes where a prompt would appear to press a particular controller button. If the player failed to press the button within a designated time frame (less than a second, usually), the result would be instant failure, perhaps illustrated by a generic failure scene, and the game would end or the sequence would restart. If the player succeeded, the scene would continue as planned.
Over the past twenty years QTEs have become abundant in video games, though barely more sophisticated. The critically acclaimed Heavy Rain makes use of them, most hilariously in its sex scenes, which only comes off if the player manages to push the correct sequence of buttons. It is a more literal rendering of the common fantasy used in game advertising that the player’s skill at video games will get him (it’s almost always ‘him’, not ‘her’) laid.
While a game like The Walking Dead extends the QTE to a slightly more interactive and sophisticated level, the fundamental mechanism is the same: create the illusion of player agency by requiring a physical interaction at a point correlating to a particular action taken by the player-character protagonist. The QTE mechanism remains as long as two labour-intensive design approaches to player ‘choice’ are not taken by a game:
- Genuine, divergent plot pathing premised on different choices by the player.
- Semantic modelling of plot and character in the code of the game.
The former is not theoretically difficult so much as it is time-consuming. The Walking Dead could have contained dozens of story branches leading to wildly different outcomes, but the work involved would have been geometrically greater than for a linear narrative while yielding the same playtime for a single playthrough. Restricting narrative branching to local and inconsequential effects is vastly cheaper. As games have required increasingly complex and expensive graphics and acting, the cost of multiplying branches has become commensurately greater. Across action and adventure games, such divergent branching is usually restricted to the endgame, where past a ‘point of no return’ the player can make one or two key decisions that result in one of a handful of possible endings, thus containing the multiplicity of plot branches.
The latter approach is, simply, fiendishly difficult.
So with the cost of approach 1 and the difficulty of approach 2 both weighing on game designers, the QTE approach is the path of least resistance to the goal of exciting a sense of engaged involvement on the part of the player. Given the increasing costs involved in commercial game development, there is immense pressure upon designers to keep the level of branching and resources down. Every potential variation in action by a player requires different graphical and sonic resources. If these can be programmatically generated, as in first-person shooters or other action games, then ‘choices’ in the form of aiming and firing a gun or moving a player around a programmatically-represented landscape can be made dozens of times per second by the player. But programmatic generation of semantically meaningful content (plot, dialogue) is not currently possible, and will continue to be a distant dream for the foreseeable future.
Ironically, the assignment of emotional intensity via QTEs comes to complement the ever-decreasing involvement in modern technological combat, where drone pilots see their targets less as people and more as dots on a map. While the armed forces seek to remove soldiers from realities in which sudden bursts of empathy and humanity might interfere with the strict differentiation of friend and foe, using technology to vastly increase geographical distance between friend and foe, QTEs seek to plant a greater immediacy into an artificial situation and thus manipulate the player’s emotions in the opposite direction. Fiction is becoming more emotionally loaded even as reality is becoming less so.
Pre-scripted interaction can be taken to wittier and involving levels. The concertedly low-tech Papers, Please, animated with graphics reminiscent of games from the early 1990s, is not particularly interesting as a game, but as an interactive narrative experience, it’s skillfully done. The conceit is that you are a customs official in Arstotzka, a small Eastern Bloc country, in 1982, and you must examine travellers’ passports, entry permits, and other documentation for discrepancies in birth dates, photographs, expiration dates, and such. Designer Lucas Pope said that the game was intended to simulate tedium, but the game mitigates against this by working a real narrative into the tedium. A good three-quarters of the encounters are randomised and inconsequential, but the remainder play out, slowly, a story of insurgency and corruption. It helps that the interface is quite evocative in spite of and even because of its limitations (the grey, washed-out palette certainly evokes the Iron Curtain), and touches like the incomprehensible gibberish that plays whenever a character speaks, and the loud, sharp kerchunk of the visa stamps contribute to making this tedium simulator hypnotic.
It helps that the writing is exceptional and droll, as with the unfailingly upbeat smuggler Jorji Costava, who makes a number of hilariously incompetent attempts to cross the border, such as a hand-drawn passport from ‘Cobrastan’.
Jorji is sympathetic if you reject or even detain him: ‘Is ok! You just doing your job! But is getting expensive to bribe guards!’
Perhaps because of the strictly limited perspective, Papers, Please is more genuinely involving than The Walking Dead because the gameplay and the narrative mesh coherently. It achieves emotional resonance without requiring the player to shoot a convincing representation of a young boy in the head. Though the player in Papers, Please has a limited amount of agency, this is in keeping with the very conceit of the game, which is that the player character is a small functionary with very little power while larger governmental and rebel forces are making plans of which you have little knowledge. The interactive limitations match up with the narrative, in contrast to The Walking Dead, where the points of interaction are shoehorned into a story where you in fact have very little control over your player character’s behaviour and choices.
The outlook for the sort of success that Papers, Please achieves may seem grim because the limitations of the gameplay appear to circumscribe the limitations of the story: in effect, the stories that match up best to such interactive modalities are stories that make the player feel constrained to a small scope of action and a certain tedium. Papers, Please exploits these very features, but it’s not clear that a game must do so. As long as the game is over and done before the limitations of the gameplay become too much of a nuisance, telling a reasonably constrained story should be feasible in a variety of ways. What is not possible is the creation of a deep, richly-represented world. The richness of the world outside the direct gameplay must be evoked through textual, narrative, visual, and interactive sleight-of-hand – in other words, by traditional literary techniques.
Yet the avenues of invention are far more tractable than those requiring semantic innovation. They will take the form of greater interactive modalities, appealing to different forms of muscle movement and sensory stimulation in order to vary and enlarge the immersive experience. The footpads and motion sensors used for dancing and rhythm games (Dance Dance Revolution and the like) can be extended to allow the player to walk, move, gesture, speak, glance, and so forth, ideally in a way that makes some sense in the context of the game. Greater and more immersive sensory feedback will be provided through gloves, glasses, moving treadmills, and a whole host of peripherals which are no doubt in development.
None of this will result in greater player choice, but it will provide for a greater variety of game experience. Instead of moving the mouse over a zombie-infected boy and clicking the trigger to fire, one can imagine speaking ‘Fire’ to a firing squad to order the execution of a criminal, or wearing force feedback gloves to strangle the murderer who killed your in-game partner. There are less grisly possibilities. Increasingly immersive simulations of physical activities like rock climbing and rowing, or even physically impossible activities like winged flight or low-gravity environments. Or, perhaps inevitably, removing the bra from your in-game sex partner. Through clever manipulation of muscle responses, proprioception, and visual receptivity, it will be possible to create experiences that may not be comparable to its real world analogue, but will certainly be wildly different from anything the real world can offer.
At this point, however, we are moving more into the realm of simulation than game per se, and narrative takes a backseat to the simple immersive nature of the experience. Not that narrative will go away, but the amount of time and effort required to provide the simulation will dwarf that required to construct the narrative. Great stories may come not from the designers of such technology, but from the fans and small studios making small cult projects that use and abuse the technology used more generally to provide the good old experience of blowing things up or seducing hot chicks. This is already evident, for example, in some of the fanmade mods for Half-Life 2, whose Source engine could never have been developed by anything less than a blockbuster game company like Valve, but which offered users the opportunity to modify and abuse the basic mechanics in occasionally interesting ways. The most famous of these, ‘The Stanley Parable’, has a pompous British narrator telling a heavy-handed Twilight Zone–style story, but offers the player the opportunity to contradict the narrator’s story repeatedly (going down instead of up in an elevator, for example), to the narrator’s increasing irritation. It’s a minor achievement, but still an indication of a direction that could be pursued by creative minds in this context.
Or, perhaps, the best experiences will come completely out of left field. The independent Russian studio Icepick Lodge has produced what must be two of the craziest games ever: Pathologic (2005) and The Void (aka Turgor, aka Tension, 2008). Surreal, disturbing, unfriendly, and entirely free of any sort of American sense of ‘fair play’, they create experiences that will alienate the majority of gamers but have attracted a devoted following, despite Pathologic being burdened with a badly-dated graphics engine and frequently incomprehensible translation, and The Void having a gameplay system so difficult that a fan released an unofficial (though sanctioned) patch to make the game less damnably difficult. And while Pathologic is a grim tale of a plague-infested village with a huge mysterious structure called the Polyhedron, I can’t really improve on the broken-English of Icepick Lodge’s description of The Void:
The choice is either to support life in its horrifying Hunters or its mournful girls or strip them of it. The way you pass through The Void is the way it changes. The way it reflects upon you.
The surrealistic world of Tension is based on a single resource that grants the ability to live and act – Lympha. Each colour of Lympha possesses its own character and unique properties – both useful and dangerous. Only Lympha gives life, but it can take it back as well. Hunters’ blood, captured creature, gracious girls, tree brought to life, just everything is the source of Lympha. Any action you take: battling an enemy, talking to a Sister, hunting, searching, travelling – the price in exchange is Lympha.
What can we learn from this, other than the evident fact that the tolerance for artistic derangement in Russia continues to be far higher than in America?
It does not seem that any formal innovations are made (though it is difficult to be certain), but the general air is so aberrant, evoking the malady-ridden landscapes of Vladimir Sorokin, Yuri Olesha, and Fyodor Sologub, and with graphics closer to Giacometti than Giger, that the game sticks regardless.
Here the achievement is not necessarily in the modalities – though the interfaces in both games are so intentionally dysfunctional that they could be termed innovations – as with the classic process of deviant creative expression: working from commercial materials and elevating them to forms of idiosyncratic individual expression. One might say that such derangement is a trickery of its own sort. While a legitimate argument, it is beside the point here. That kind of trickery puts it on a continuum with any other media, interactive or no. One does not get the sense that The Void is leaning on interactive modality as a crutch to prop up derivative and conventional content. While The Walking Dead supercharges clichés through the simple device of demanding player agency for one character’s actions, The Void does the hard work of creative innovation, manipulative though it may be.
There is always room for debate over what constitutes creative artistic expression versus overrated hackwork, and the two have certainly coexisted. But The Walking Dead and Heavy Rain achieve their effects solely through this added crutch: subtract the immersive modality, and what remains would not be sufficient to move people in the way that these games have. There is no specificity to the interactivity in such games; force the player to kill a child by the push of a button, and it will be wrenching regardless of the surrounding content. Because this modality is merely grafted on to the content (with a minimum of true player interactivity) and is not a constitutive part of the narrative, it is very difficult to see how such works can rise above the level of being, at best, good stories that don’t let themselves coast on the added emotional involvement they gain not through their content, but through a purely independent aspect of presentation. The resulting art is incoherent, and reduced purely to the level of craft, the sort of art that satisfies the rote requirements Aristotle specifies for tragedy, in the words of Kenneth Rexroth:
It never occurs to Aristotle that the three great tragedians, Aeschylus, Sophocles, Euripides, are, from his point of view, themselves philosophers, and that together they form a philosophical school, a body of doctrine with insights into the meaning and end of life that have never been surpassed. Unlike the notions of the technical philosophers, their “tragic world view” is not subject to the changes of fashion, but remains true in all times and all societies. Of course Aeschylus, Sophocles, and Euripides are not primarily philosophers. They are artists. But Aristotle does not consider them as artists either. He considers them as craftsmen.
The hard work of a game such as The Void comes in its very integration of narrative and game into a unified aesthetic substance: the choices that make the constituent pieces fit together to add something beyond pure issues of craft. Which is to say that in such games, narrative art – shorn of its elitist, academic excesses of the last decades or even the last century, as well as a good deal of its élan and nuance – lives on, bruised and battered but not defeated.
