Haptic and Non-Visual Games Update

Halfway through a commission from Harvestworks’ Cultural Innovation Fund program, I am fully immersed in haptic technologies, digging into early virtual reality research, and thinking about ways of expanding traditional game space to include non-traditional senses, extending common haptic technologies (vibration motors), and removing the visual from gameplay while still creating beautiful, meaningful games like the ones where people use P4rgaming services. Never before had anyone carried off such a huge sum in one spin of the reels. In fact, up until then, the record-breaking win for Canada had been $1.7 million. One lucky gambler won it playing a Powerbucks machine also at Casino de Montreal, in April 2016 — it was a new slots record in Montreal.

I am particularly interested in activating senses that have been neglected in media: touch (texture, temperature), smell, and (to some extent) sound. Above is a screenshot of an early 4-channel vibration controller that uses a joystick to move vibration around a surface, much like sound can be mixed to appear in different locations. The circuit fits on top of the Arduino microcontroller for easy programming and connection.

Research links (a bit of a mess – mostly what tabs are open at the moment):

A game where the player has to ride a bike using sonar (click into the microphone, receive auditory feedback on where objects are in front of them) is approaching the play-testing phase. The game is inspired by a video I saw a long time ago about Daniel Kish, shown below:

Up next: smell tests, optimizing combinations in a circle for combinatorial smell “playback”, testing various materials for vibration transmission (damping vs transmission).

Haptic Games: Installing Android For Processing, Example Sketches

As the starting point for developing haptic and otherwise non-visual games for a project in collaboration with Harvestworks’ “Cultural Innovation Fund” program, I’ve been poking around the Android documentation for developing apps with Processing for tablets and mobile devices. Since this is a relatively new feature for Processing developers (and since the Android syntax is a bit weird), the first step was to figure out how to get everything working.

In the spirit of sharing, I’ve created a GitHub repository for these experiments, as well as a detailed set of instructions for getting started. I have also created a GitHub repository for this project, which will be a bit of a mess over the next 3-4 months of development but will hopefully get cleaned up as the project nears completion.

These examples and projects are being developed for the Google Nexus 10 tablet – it seemed the beefiest and most flexible for the price. If you have problems with any of these examples on your device, please let me know so I can update them!

In the pipeline:

  • Better control with the vibration motor
  • Research into the role that sound + other feedback (visual, tactile, etc) plays to clarify or intensify interaction
  • Consider trade-offs for games created for mobile devices vs custom hardware (accessibility, reliability, price, etc)

Above: a giant d-pad

“Secret of Mana” Intro

Slow, sprawling introduction to the SNES game “Secret of Mana” (1993).  Falling from a bridge, I’m particularly drawn to the “wander and discover” gameplay where it isn’t clear what you’re supposed to be doing or going (and wandering through creek-beds is a really nice touch, too).

Notes: “The Future Was Here” by Jimmy Maher

A few notes on The Future Was Here: The Commodore Amiga by Jimmy Maher, part of MIT Press’ “platform studies” series:

+ I am fascinated by the idea of the institutional computer (ie: mainframe that one logs into as a shared resource) versus the personal computer (pg 6) – a distinction that might be obvious to someone who did not grow up around PCs.

+ Images we see on the computer screen “also exist within the memory of the computer itself.  The latter image is in fact the original of the image that is mirrored to the monitor” (page 23).

+ Early digital artist Rodney Chang (top) in 1989: “My work helps document the progress the software makes” (pg 79 – originally from Mindy Skelton’s article “Rodney Chang: Artist on the Edge” in Info #25, pg 45).

+ For old-school ray-tracing, the Maher suggests Principles of Three-Dimensional Computer Animation by Michael O’Rourke (pg 89).  O’Rourke breaks the 3d rendering process into six pieces: object geometry, the camera, the lights, the surface characteristics, the shading algorithm, and the rendering algorithm.

+ Another old-school rendering technique and alternative to the slower ray-tracing is the “scanline snapshot” method (pg 105).  Little information is available online, but a brief mention in a Spanish page about the Amiga also has some cool screenshots of Amiga art.

+ The Digi-View which captured video frames but could be used as a rudimentary scanner as well – for color, one must capture the same frame four times with “a set of colored filters, one each for red, green, and blue (pg 122).  Since most subjects can’t sit still for the 30-seconds or so required for each image (of three), images were often captured from printed photographs.

+ One of the first known computer viruses was called the “Elk Cloner”, which would incidentally be a fantastic band name (think high school sci-fi punk, if such a genre exists).  Created in 1982 by “a 15-year-old American high school student named Rich Skrenta [using] an Apple II… spread from floppy disk to floppy disk, displaying a mocking limerick on every fiftieth infection”.  The virus, intended as an inside joke among friends, but was eventually seen as “threatening enough to be the subject of a presentation by the Department of Defense/National Bureau of Standards Computer Security Conference that year” (page 171).  For a full digital version of Fred Cohen’s paper from that conference, see: http://all.net/books/virus.

+ Finally, the 1984 game Elite, developed by David Braben and Ian Bell.  Fascinating because the designers were able to squeeze “eight galaxies, each of which houses 256 star systems… [each of which] has its own name, economy, even system of government” – all fitting into 32kb.  The game also featured algorithmically-created graphics, built in realtime to avoid being saved as assets (pg 216).