[MUSIC PLAYING]
CAMILLE SIMON: Hi.
I'm Camille, a software engineer on the Flutter team.
KHANH NGUYEN: And I'm Khanh, developer relations
engineer also on the Flutter team.
And today we're talking to you about a fun project
that we've been working on with Flutter, Dart,
and Raspberry Pi.
CAMILLE SIMON: If you're not already familiar,
this is a Raspberry Pi.
I'm sorry if you were expecting a more delicious kind
of Raspberry Pi.
It's a single-board computer that
was released in 2012 to teach kids computer science.
It's grown in popularity since then.
And it's now used in a variety of fields for various use cases
because it's just so cost effective.
A Pi can range from $5 to $75.
And you can run all kinds of things on it
because, well, it's a computer.
KHANH NGUYEN: I caught the Pi bug myself a few years ago.
Inspired by the hashtag #NowPlaying trend back
in the day, I built a dashboard to display the song
that I was listening to on my music streaming
service and a dashboard to show the upcoming commuter train
status.
And since the Pi can be hooked up
to all kinds of gadgets and gizmos,
like sensors and buttons, I also built a disk drop game
with a digital UI.
You drop a little puck.
It'd fall down, hit a bunch of pegs along the way,
and trigger one of the sensors that was
installed in a slot down below.
The UI would update to show whatever prize
the player had won.
You could say I've been really into building my side
projects on Raspberry Pis for a while now.
Every time I build a new project,
a little voice in my head likes to ask, can I run this on a Pi?
Fast forward to last summer when a teammate and I
built a game in Flutter.
Doodle Dash, as we like to call it,
is a platformer game, where players
try to jump as high as possible and rack up points.
There's powerups, enemies, the whole nine yards.
The game is built with Flame, an open source game engine that's
been built on top of Flutter.
It was a lot of fun.
And that voice in the back of my head popped up again.
We started wondering, can we build a game console experience
with Dart, Flutter, and Raspberry Pi?
With some code, a Raspberry Pi, a screwdriver,
and a little Pi accessory called a game hat, turns out we can.
Before we jump into how we brought this Dream
project to life, here were our three big project requirements.
First, we wanted a gamepad.
This was a requirement because, at least to me,
a game console experience requires a gamepad.
Second, there are gamepads that plug in via USB.
But since we have access to the Pi's GPIO pins,
we wanted to use them.
Personally, my favorite Pi projects
have always included working with GPIO pins for input
or output, sometimes both.
As much as I love writing code, having the hands-on aspect
of putting the pieces together is always a ton of fun.
If you're not familiar with GPIO pins, don't worry.
We'll cover this topic in more detail in just a few moments.
And finally, we wanted as much of our code to be as reusable
and extensible as possible.
Given how community-driven both the Flutter and Pi
communities are, we want to share our work with everyone.
This means the code that we write
shouldn't just enable Doodle Dash to work with the game hat.
It should enable other Flutter developers
to integrate their games with their game hat as well.
CAMILLE SIMON: Awesome.
Now that we've covered our requirements, let's talk setup.
We installed a Debian-based operating system for the Pi,
Raspberry Pi OS 64-bit on a micro SD card
using the Raspberry Pi imager that's
available on the Raspberry Pi website.
Once that was loaded up, we popped the card into our Pi
and booted it up.
Then we used Snap, Canonical's app store for Linux,
to install Flutter.
Now for the fun part, assembling the game hat.
The first step was inserting the micro SD card into the Pi
and attaching the game hat module to the Pi's GPIO pins.
Next was connecting the game hat's pre-installed screen
to the Pi's micro HDMI out using the included adapter.
Finally, positioning the phase plates,
putting all the screws and support pieces in place,
and attaching the joystick to the game hat.
Our development cycle involved writing code on our laptops
and building, executing, and testing
the game on the Pi itself.
For portability during development,
we used a mini keyboard to run commands on the terminal.
And the built-in trackpad let us interact
with the Pi's desktop UI.
KHANH NGUYEN: Now that we've got everything set up,
let's get building.
First, reading the GPIO signals with Dart.
As you'll recall from a few moments ago,
we attached the game hat to the Pi's GPIO pins.
GPIO stands for general purpose input output.
They're signal pins on the Pi that
can be controlled with code.
As their name indicates, they can
be used for input or output.
Like I said, you can connect it to all kinds of things,
from buttons or sensors to, well, in this case, a game hat.
The game hat itself is really a pre-configured set of buttons
that we want input from.
Taking a look at the manual for the game hat,
it mapped out exactly what GPIO pin is used for which button.
We made this more manageable in code
by creating an enum to represent the game hash GPIO input.
Each enum has a pin with a game control event.
We then used the rpi GPIO package from pub.dev
to get the input values from the pins.
We created a new class called gamepad to represent, well,
a gamepad.
On initialization, the code loops
through the list of game hat GPIO enums,
initializes each pin, and starts listening for input.
The code listens for a change in pin state
from true to false or false to true.
A value of false means that it's been pressed.
And a value of true means that the button has been released.
The game will listen to the event
stream for new gamepad input.
So when there's a change in a pin state,
a new game input gets added to the event's stream controller.
CAMILLE SIMON: Now, let's talk about a small detail
that we glossed over before, turning the signals
into game input.
The question we need to answer is, how should our code
translate raw button press events from the hardware
into Doodle Dash commands my game can understand.
To translate between the two, we created a few classes
for the Doodle Dash game to use.
There's game control event, which
represents what game control was activated, left, right,
A, B, et cetera, and event state,
which represents whether the button was pressed or released.
Together they make up a game input event.
Like Khanh discussed before, the gamepad
will listen for events from the physical gamepad buttons
and add game input events to its stream controller.
To test this out, we can use the gamepad.
To use the gamepad we'll need to initialize it in the app.
And now we can run the app while logging the joystick movement,
button presses, and releases.
Since we're able to confirm that the button presses are coming
through, the remaining work left for the project
is having Doodle Dash react to the gamepad event stream.
KHANH NGUYEN: OK, so the button presses are being registered.
How do we ensure that Flame recognizes game input
and allows players to actually control the game?
Let's talk about Flame components.
Flame components are all the individual pieces
that make up a Flame game.
I like to think that components are to Flame as widgets
are to Flutter, like these assets
you see in the game, and the background, and even the things
that you don't see on the screen,
like the game manager, which is responsible for spawning
on screen objects, similar to state management widgets.
Flame components start off pretty bare.
But you can enhance their functionality
by using various mix-ins offered by the Flame package.
For example, a popular mix-in is the keyboard handler.
This mix-in gives Flame components
access to key press events.
You can add the keyboard handler mix-in to a component
and override the onKeyEvent method
to do whatever you'd like whenever
there's a new key press.
With Doodle Dash, if the left or right key has been pressed,
onKeyEvent will make Dash move left or right
based on the arrow key that's been pressed.
We followed a similar architecture
for extending Flame to work with the game hat
by creating a mix-in called GamepadInput.
This mix-in has three parts.
onMount initializes a stream subscription
that listens to the gamepad broadcast stream for new game
input events.
It calls a method named onGamepadEvent whenever
a new event comes through.
onRemove ensures that the subscription is cleaned up
when the component is removed.
And finally, onGamepadEvent which is called every time
there's a game input event from the gamepad.
Then in the Player class we add the gamepad input mix-in
and override onGamepadEvent.
When a new game input comes in and the player
has pressed the joystick either left or right,
then the character will move left or right,
just the key presses.
The only difference here is that we
listen to all game input events, not just button presses.
So we had to introduce a conditional
for EventState.PRESSED so that it filters
out EventState.RELEASED.
With all that coding done, we can now
move Dash left and right using the joystick on the game hat.
CAMILLE SIMON: Finally, the missing piece in our game
console is menu navigation.
Flame uses regular old Flutter widgets for overlays and menus.
That's great.
But we don't have a keyboard, so that
means we don't have access to Flutter's
built-in keyboard-based focus system.
We came up with our own solution backed by a class called
GamepadHandler.
A GamepadHandler has five callbacks
that can be called, on up, on down, on left, on right,
and on confirm.
These callbacks can be used to navigate the menu screen.
For example, this nextCharacter method that
changes the selected character.
You'll see it being passed to the GamepadHandler's
on left and on right callbacks in the next slide.
Within our menu widget, a new GamepadHandler
is initialized for each section of the UI
that requires focus and directional
control from the gamepad.
Upon game initialization, the characterSelector
GamepadHandler is active.
On left and on right calls the nextCharacter method
to change the selected character.
While on down changes the activeSelector
to the next section that wants focus,
the difficultySelector GamepadHandler below it.
The neat part is the activeSelector
can also be used to conditionally style
the section of the UI that is in focus.
We can check for which selector is currently
active to put a border on that section of the UI.
Finally, the widget initializes a subscription
to the gamepad's broadcast stream
and listens for game input events.
When a game input value comes through,
it activates the selected gamepad handler's callback.
And there you have it, a fully functional menu navigation
system.
You can use the gamepad controls to select a character,
change the difficulty level, and start a new game.
KHANH NGUYEN: Once we got the core functionality working,
we had a few things to work out, like making sure to adapt
the UI to the smaller screen.
And this is a game console after all.
So we wouldn't want to carry a keyboard with us everywhere we
go.
So we also made sure that the Doodle Dash
app launches on boot up.
And finally, for the finishing touch,
we also added a 3D-printed case.
How cool is that?
CAMILLE SIMON: So cool.
And there you have it, a Flutter, Dart,
and Raspberry Pi game console.
This project might be a prototype.
But it sure does show so much potential for Flutter and Dart
on Raspberry Pi.
KHANH NGUYEN: For sure.
Camille, this was such a fun project to work on.
I mean, I truly felt like I was five years old again,
getting my very first game console.
It also makes me very excited looking forward
to what's coming for Flutter and Linux in the future.
In fact, the new Ubuntu 23.04 desktop installer
is built with Flutter.
So that's pretty cool.
I also want to say get it to work with the open source
community throughout this project
has been such a great experience.
Plus, they're always doing awesome things like this.
I mean, I've seen a Flame game running on a smartwatch.
So that's all to say I'm so excited to see what the Flutter
community continues to build.
CAMILLE SIMON: Completely agreed.
But wait, Khanh, do we have the source code
to share with everyone?
KHANH NGUYEN: Good question.
You'll notice that this code was made to be very modular,
given our goal to make it reusable and extensible.
We'd love to see you use it with your own Flutter and Flame
games.
We're polishing the code for this project right now
and plan to share it on the Flutter Samples
repository in GitHub very soon.
Be sure to follow Flutter Dev for updates.
CAMILLE SIMON: That sounds great.
All that said, I think that's about it for us.
Thanks for joining us on this Flutter, Dart, and Raspberry Pi
journey.
I'm Camille.
KHANH NGUYEN: And I'm Khanh.
CAMILLE SIMON: See you later.
KHANH NGUYEN: Bye.
[MUSIC PLAYING]