Global Game Jam is an annual event held at approximately 700 sites simultaneously across the world in which participants join teams to create a video game in 48 hours. I participated in the event for the first time this year in January at the North Sydney Institute of TAFE's site and had a blast! This year's theme was "Waves" and joining in an enthusiastic team with three other people I met at the event, we created our game "Jet Ski Jousting".
I was a little apprehensive when I turned up to the event on a Friday afternoon, not knowing anyone, and not being sure if I'd even be able to join a team. Luckily I ran into a pair who had done the event for at least the last three years and needed some audio. Overall there were about 60-70 people there (I guess) and everyone was super friendly and inclusive. There were a range of different games types and platforms, people doing standard PC games, tablet games, VR etc.
I did the sound and music for the game (and a little bit of UI art) which was a great experience, as I've mostly been involved in programming or game design in previous projects. I used Musescore to compose the background theme and a little victory jingle, and used Audacity to chop and post-process a range of open-licenced sounds I found online, mostly through Freesound.org for sound effects. I combined these all together and integrated them into our game (which was built in Unity) using FMod, which I had never used before; it was relatively easy to pick-up with a few pointers from one of our other team members who knew what he was doing.
I had a bit of spare time on one of the mornings so I also worked on a little side-project game. I had brought along my Wii Balance Board that I picked up from an Op Shop a few weeks ago, and a Raspberry Pi 3. I previously got the Wii Balance Board to talk to the RPi over Bluetooth using python, so I worked off this to build a game in python/pygame called "waverider". It's a sort of motion-racing game where you have to accelerate a particle along a wave function by leaning left and right to steer the particle up or down the hills in the wave function, racing another particle to get to the end of the course.
Good thing about using the RPi was that I could plug it into a ceiling mounted TV hanging above a busy thoroughfare and leave the balance board on the ground nearby, so passerbys could casually play. People seemed to enjoy the game; a bit of physical movement was probably a nice respite from intense work in front of a computer screen.
This is a continuation of a previous post on re-building the electronics for an old, broken electric piano keyboard. After getting the keyboard working as a MIDI device, I decided to add in a Raspberry Pi to act as a synthesiser to actually generate out different instrument sounds based on the MIDI data from a Teensy which is used to read the keyboard state.
I found some good instructions here for running Fluidsynth, an open-source, command-line based synth on the Pi that uses Soundfont sf2 files. The great thing about soundfont is that I can download and pick and choose form thousands of different instruments for free in the internets, which will make for a more interesting playing experience than the simple square waves that I can generate on the Teensy. After install Fluidsynth, I connected up the MIDI/USB connection from the Teensy into a Raspberry Pi 2 model B, and ran: "fluidsynth -a alsa /usr/share/sounds/sf2/FluidR3_GM.sf2". I then put Fluidsynth into the back ground using cmd-z and ran: "aconnect 20:0 128:0" to route the MIDI input from the Teensy (client 20, port 0) to Fluidsynth (client 128, port 0 on my machine). I could now hear a standard piano instrument when playing the piano, but the sound is fairly delayed through the HDMI audio (a latency of maybe 250 ms by my guess). Tried changing audio output to headphones using "amixer cset numid=3 1", and the delay is slightly less noticeable, but the sound quality is dreadful.
A bit of digging around on the internets regarding Raspberry Pi audio quality, and I came across this, an experimental firmware update for improved headphone jack audio quality, using a different DAC interface. I ran "sudo rpi_update" (I ran it on Oct 12th) and went and added two lines to my /boot/config.txt file: "audio_pwm_mode=2" and "dtparam=audio=on", rebooted and re-tested audio. The quality improvement is very noticeable; at least through my headphones, the quality of sound is now acceptable.
I turned my attention to trying to get amplified audio out of my setup. I connected up a class D audio amp I had left over from a previous project to the 3.5mm audio jack on the RPi, powered it using the 5V out from the RPi GPIO and connected it up to a 4 ohm 2.5W speaker that was inside the original piano keyboard. After configuring the RPi audio volume to 80% using "amixer cset numid=1 -- 80%" I was getting a reasonable sound coming out when playing. I ramped up the volume to 100% on the gain pot on the amplifier and played again: now quite loud and a little bit distorted. When I power cycled the RPi I was getting a fairly loud popping sound coming from the speaker, and also when the RPi booted up again. When I had the volume up to max I was getting a little lightning bolt brown out warning coming up on my screen too ... I decided to tune the volume down to a slightly more modest level of 80% again. Didn't notice the brown out issues again. I added a pot to the signal line coming out of the RPi and into the amplifier for volume control.
I did a bit of configuring in my /etc/rc.local file on the Pi to start everything up automatically:
The -c and -z flags are for controlling audio buffer sizes and number of buffers in fluidsynth/alsa: setting these I'm able to find an acceptable middle ground between low latency and infrequency of audio stuttering: I still sometimes get a little bit of stutter on instruments with complex, long lasting waveforms, but the latency is now at an acceptable level (I haven't figured out yet how to measure it precisely, but I mean from a playing/responsive feel perspective). I also setup a configuration file for Fluidsynth so I can automatically set the instruments for different channels (fs_config.txt).
Next thing I'm going to do is add some more controls onto the Teensy and then package the whole thing up in a new custom designed case. Stay tuned!
Recently, my father in law donated an old electronic piano keyboard for me and my son to play around with, an old Yamaha PSR 8 from the 1990s. When we brought it home, my son accidentally pulled off the tip of the 9V DC adaptor that came with it and when I plugged it back on I got the polarity wrong and we broke it (sad face). Yah, I know, I know ... I got lazy and thought I definitely had it the right way (who knew there was no standard for the polarity of DC power pack tips? Now I do) ... will always check with the multimeter from now on.
So I opened up the case and had a peak around. Can't see any obvious physical damage (no burnt out or broken components). Had a search online for some repair manuals for this keyboard and found a few pages, but to be honest wasn't that keen on trying to find replacements parts. So ... only one thing left to do: gut it and rebuild the electronics again from scratch!
I unscrewed and pulled away the actual keyboard assembly from the rest of the case. Had a look at the keyboard itself and contacts for keys. Running along the back of the keys is a single long PCB with contacts for each key and a bunch of diodes on it (one for each of the 49 keys) that all map back to a 15 pin connector. A bit of googling and it looks like this is a switch matrix. I found this helpful reference which seems to explain how this circuit works: keys are grouped into nine rows, each with a set of six keys (except for the last row which contains a single key), where each key in a row shares a common ground, and the first pin of each row shares a common contact on the other end of a diode, as does all the second pins etc. The state of each of the 49 keys can be read by selectively reading from 6 digital input pins while incrementally setting 9 output pins to ground one at a time.
I decided to use my Teensy 3.2, which I hadn't previously used, to do the reading of keyboard state. I installed the teensyduino add on for Arduino 1.6.11, tested out an LED blink code and got a 4 ohm speaker working with simple melody. I then got a simple, single switch working with a INPUT_PULLUP mode, all good, lighting up pin 13 LED. Then I wrote a sketch to flick back and forward between two pins, both in INPUT_PULLUP mode and to a common ground, by setting the other pin OUTPUT/HIGH while reading the other (diodes built into the keyboard to stop current pulling between pins when both keys are down). Working well.
I snipped and stripped the ribbon cable I found connected to the 15 pin connector coming off the key switch PCB, plugged it into my breadboard and connected to 15 digital pins on the Teensy. I wrote a sketch to implement a basic matrix to read out all 49 keys. I found I had to add in a 10 us delay between setting Teensy pins to INPUT_PULLUP mode and reading them (as suggested by the Teensy instructions on their website) and also found I had to add a 50us delay between subsequent reads of each key on the matrix to get things to work such that more than one key could be read at a time. Once all keys read, wrote a program to output the tone corresponding to the highest key. Basic monophonic piano implemented!
I decided to round off this design my modifying the Teensy built in tone function to produce four simultaneous tones using the 4 PIT timers available. I modified the standard Teensy tone implementation to use up to four timers simultaneously and prioritised timers to the higher keys on the piano. I used four pins each connected through a 330 ohm resistor back to the speaker and I also chucked on a potentiometer for volume control. Working fine, and I could now play polyphonic tunes pretty well (up to four simultaneous key presses). The square waves sounded pretty ordinary through the little 4 ohm/500mW speaker, so I also tried connecting the ends to an RCA co-axial cable and plugging it into an external amplifier, which was a little better.
At this point I decided to try an alternative direction for the design: instead of doing the sound synthesis on the Teensy, what if I used an additional computer or micro to do it for me? I decided to try and get the keyboard working as a MIDI device. I changed the code to output MIDI "noteon" and "noteoff" messages on key presses and configured the teensy to output MIDI, using the options available in the Arduino IDE. The teensy library in teensyduino makes this very easy: there are a bunch of send/receive midi functions built in, no need for any additional code or external libraries.
Once i got the keyboard working as a MIDI device, I downloaded and installed "Fluidsynth" on my macbook to read in the MIDI messages and play sounds. I used the "General User GS v1.47" font by S. Christian Collins and started up Fluidsynth from the command line using the following settings: ./fluidsynth GeneralUser_GS_v1.47.sf2 -o midi.driver="coremidi" -o audio.driver="coreaudio"
I then used a software package called "MIDI Patchbay" to patch the output from the Teensy to Fluidsynth, and voila! I can play synthesised piano through my laptop. Once in fluidsynth, I used "prog <channel> <inst>" to set different instruments.
So thus far I've managed to (a) make a simple synth (square waves) which is OK, but lets face it, my son will only find squares waves interesting for a short amount of time! and (b) make a MIDI keyboard, which can play all sorts of cool sounds, but only when connected to and setup on my laptop, so my son can't really go and turn it on and play himself. I am going to try and keep going down both design routes and (a) get a synth working on the teensy that can play a wider variety of tones/waveforms and (b) try connecting the MIDI keyboard up to a dedicated raspberry pi for doing the audio synthesis. Stay tuned!
Once I had the hardware finished up, I was ready to finalise the on-board software and load up a few games. I already had three ex-pyweek games ready to go: all I needed was some sort of main menu that would come up when the console was switched on and would allow the player to navigate options and choose games using the dpad and buttons. I wrote a quick and dirty main menu interface using pygame that allowed the player to choose between the three games (I will probably come back and upgrade this to something more visually impressive somewhere down the track). I added a line to the end of /etc/rc.local to run this on start-up.
To get the menu interface to quit itself and start up a selected game, I used "os.execv", a nice function that does just this: shuts down the current instance of python and finally runs another command on the shell with specified arguments. I set it up so when a game was selected, the code would run:
I used the same mechanism to get games to shift control back to the main menu when the player quit them. This was convenient, because it meant I had to make very little change to the existing game code.
All games were working nicely. I also adapted an existing game that used the mouse to work with the touchscreen input. Here's the console in action:
Current Games:
"The Wizard's Data": pyweek 20 entry by Team Chimera
"Adrift" pyweek 21 entry by Team Chimera
"Underworld": a beta version of a game I've worked with on and off for about a year :)
Overall I was super happy with the outcome for this project. There are however a few things that I learnt and would do differently for next time:
Buttons: I didn't really like the buttons in the end, they make an annoying loud clicky sound and don't depress far enough. To their credit they look nice and are responsive. They just don't feel as good as a proper games controller: next time I do this sort of thing I will definitely invest in some proper arcade buttons, or at least something that has a built-in spring or rubbery inside to produce a smooth, but still responsive feel.
Audio Volume: I don't know why, but the audio volume is super quiet. Not sure if I botched something or not. I'm pretty sure I've got the trimpot ramped up correctly ... will try to figure this out somewhere down the track.
When my case was fresh off the printer, I started lining up all the components to see that everything would fit as I'd planned. I think I had convinced myself that I would have gotten at least one measurement wrong somewhere and that this first print would be just a test run. I was happy to discover that I had actually not stuffed up anything too critical and that I could get it all to work. The thing that didn't really work as well as I had hoped was the standoffs for the smaller components (amp and boost converter): I had designed a cylinder to fit through the small mounting holes on these PCBs with a diameter of 1.6mm, but when they came out of the printer, these parts of the structure were just a stringy mess: I guess they were just a bit below the resolution the printer was capable of. The standoff cylinders for the pitft screen and RPi mounting holes were 2.5mm diameter, and these came out fine, but they were pretty fragile. In the end, I decided to break them off, drill 2 mm holes in their place and screw the pitft, RPi and larger holes on the boost converter onto the standoffs: lesson learnt for next time!
Once I was happy that everything would fit, I started assembling the buttons, RPi and screen into the case. I cut two square(ish) sections from perf board to hold the buttons (main buttons and dpad) on either side of the screen, and drilled 2 mm holes into these so I could screw them down to standoffs on the case. Buttons were then soldered on before screwing the perf board onto the case.
At this point I realised I'd miscalculated the heights of the standoffs for the dpad side: it was short by about 2 mm such that the buttons would be pressed down permanently upon screwing it in. I didn't have any washers that were small enough, so, because I'm super impatient, I improvised with snipped off bits of wall plugs :) ... Total hack, but hey, it did the job. A little bit of bluetack on the back of each button stopped them from have that slightly annoying rattle.
I then screwed in the tft screen to the case; the RPi would then slot on top of the 40 pin header; conveniently the tft also provides an additional breakout of this same header, which I connected a 40 pin ribbon cable connector to, chopped off at one end so I could feed out the GPIOs/ground to the buttons and 5V/ground in from the boost converter. I soldered up all those connections, plugged the pi on top and powered it up to check all the buttons were working well.
Next I wired up the amp to the speaker, and positioned these in the case. The standoff pins in the case for the amp hadn't come out properly in the 3d print, but it was ok with no further modification, found the board could sort of float in the case with no issue. Next step was to screw in the boost convertor to the case using two of the four mount holes. Soldered the power out pins to the RPi (via header cables) and across to the amp.
Placed the battery into the case, held in place with a tiny bit of gaffer. I fit the slide switch into its hole in the case and soldered it on to the boost convertor. Last step was to solder the signal pins from the amp to the solder points on the RPi PCB that sat adjacent to the 3.5 mm audio jack. Closed the case up, switched it on and everything was working a-ok!
After I finalised my case design, I shifted attention towards getting the Raspberry Pi configured to run games in the way I wanted. I basically wanted to be able to run existing python/pygame games on the device, making sure that the screen, audio and input from buttons via GPIO would all be working smoothly.
I started with the 27-05-2016 build of Raspbian, standard config for expanding the boot partition and configuring for a 104 key keyboard. I followed this guide for installing the kernel modules for the PiTFT 2.8 support. I wanted to use pygame and have support for the touchscreen, so I followed this guide to ensure the SDL version underneath pygame would support the screen (apparently needs to be SDL 1.2, there are reports of issues with SDL 2.0). I wrote a simple pygame app to test the touchscreen and test pre-installed gpio-connected buttons that came with the screen. I added a line at the top of the script before pygame.init() to ensure the SDL output was set to the framebuffer:
os.putenv('SDL_FBDEV', '/dev/fb1')
The touchscreen was working great, and very conveniently linked straight into the mouse drivers, such that touches come up as mouse click events in pygame. I added these lines in to get this happening:
I used the python RPi.GPIO package, that came pre-insalled, to try and read button presses and at this point discovered the performance was fairly underwhelming: I was using event detection, and tried a combination of logic rules to read whether the pin was high or low, to detect short presses and long button holds, also tried playing with the bouncetime argument: all to no avail. I ended up realising that Adafruit had actually written a really nice c-based daemon, "Adafruit-Retrogame" which would map gpio events to key strokes: exactly what I wanted! And it worked perfectly: configured it to the keys I wanted, compiled it and added it to /etc/rc.local to run in the background on boot up.
Wireless: I setup my RPi to connect automatically to my wireless access point using this guide. I really wanted to get wireless working, as I knew from the case shell I wanted, it would be hard to feed one of the USB ports to the side for a keyboard or usb drive etc. (the other alternative for debugging would be a Bluetooth keyboard, but I still also wanted to be able to copy games on/off the device). I found the wireless would drop out constantly: turned off the pre-installed wireless power management: single line "sudo iwconfig wlan0 power off" added to /etc/rc.local to turn off every time on boot: wireless then working perfectly.
The last thing I did was to test the performance of one of my previous pyweek entries to see how it would go on the pi. The first thing I noticed was considerable audio lag for sounds (music was ok, but sounds would play at least 300-400 ms after being triggered) ... Found out I could fix this by reducing the buffer size using the following pre initialisation line in all python/pygame code:
pygame.mixer.pre_init(44100,-16,1,512)
More details on this here. The game was set to run at 30 fps, but I was only getting about 22-23 fps and the graphics were pretty choppy. I tweaked the PiTFT frame rate and SPI frequency by editing the settings in /boot/config.txt editing the relevant line to:
After a bit of experimentation, these settings seemed the best balance for a not too choppy graphics and enough CPU left over to run the game smoothly.
Now that the Pi was setup, it was time to start putting all the hardware together!
Ever since I started programming games, I've always been keen on the idea of building my own custom hardware to run them. Its one thing to write a game that runs on a PC or laptop, but running on hardware specifically designed for games is just more exciting. I've found often when I'm writing a game I am imagining holding a gamepad in my hand, and that a keyboard is just a poor substitute. I'd previously played around making an Arduino-based video game console and I wanted to make something with a bit more performance (colour graphics!). Other design criteria were:
Ability to play games written in python: because I'd already written a few from previous pyweek comps and its the language I'm most comfortable with and hence most likely to want to develop more games in.
Portable: I wanted to try something portable ever since I saw the PiGRRL.
Another cool thing about this project is its a good chance to try out some 3D printing, which I've not had any experience with but am keen to try!
The first video game device I owned as a kid was the original gameboy ... it has a special place in my heart; so many hours spent playing Tetris, Metroid II, Super Mario Land, R-type, Adventure Island :) ... it was still working perfectly up until about 8 years ago when it reached a stage where it would chew through a pack of double As in about 180 seconds. Sadly it no longer powers up, but the memory lives on. The only other handheld I have is a DS Lite, which I did a bit of homebrew development on, but never finished anything.
Based on the inspiration provided by the PiGRRL, I decided to go with a Raspberry Pi 3 as the core of the console, as I'd had experience with it before, runs pygame reliably, could find a good screen (PiTFT 2.8 LCD screen capable of 320 x 240 pixels) and GPIOs make it easy to design custom buttons. I decided to go with the same audio amp and power boost converter/USB battery charger used in the PiGRRL2, as I hadn't had much experience with these before, and they seemed to play well with the RPi. Good thing about the RPi3 is that it also has built-in wifi and bluetooth, which means I can debug/copy code on/off without having to open up the case or worry about feeding out a USB port from the case, which would put constraints on the case shape.
So I started doing some research on different handheld consoles to get a feel for what I liked/didn't like. I was keen on a landscape format (i.e. DS Lite or PSP) rather than a portrait format (i.e. Gameboy), and I started sketching out some case shapes. I walked around JB HiFi for a bit trying out the new PSP Vita and the 3DS: the thing that gets me about these modern handhelds is that the case is thin, the buttons are super small and located pretty close to the edge of the device. I find them kind of uncomfortable to play after a while ... I dunno, I guess they are more designed for a younger market these days with smaller hands, but my hands aren't that big either. I pick up my old gameboy again and it's more comfortable: a combination of being slightly fatter and a bit heavier too.
So I finalise some sketches for the shape I like and it's time for some CAD. Last time I did any serious CAD was in undergrad, so I had some rehashing to do. From a random conversation I had with someone about 3D printing, I remember them mentioning parametric cad modelling and "OpenSCAD", in which objects are "programmed" ... the concept was appealing to me (as was the fact that I could get OpenSCAD free and open source). A week (and a bit) later, I've got the case design finished:
Man, it took me such a long time (learning curves in OpenSCAD, and "programming" the shape I wanted took a few goes, implementations that didn't work) but I'm happy with the end result, and because its parametric I can easily adjust anything that is not working out further down the line.
