Here is how IoT elevates my user interaction design for connected toys

As a result of the expeditious technology updates, especially on the Internet, cloud connectivity has been infused in almost every single corner of our lives and has transformed the way we interact with things and others. We can now feed and talk to our pets when at work, sprinkle our garden when we’re away on vacation, or be notified about our running performance after each workout session.

Since the physical and the digital worlds have become more and more inseparable, it is crucial for designers to understand how to apply technology to their designs to help augment the user experience they are designing. And inevitably, I’m one of these designers.

Early days: discovering my passion

I have a background in product design where I focused mostly on conducting user research and applying the insights I gathered to inform the physical product design. Through my study in product design, I discovered my passion for toy design.

With the persistence of pursuing my passion, I landed a job at Hasbro where I got to work on multiple projects for brands including furReal and Littlest Pet Shop. If you’re not familiar with the brand furReal, it’s a line of animatronic toys that look like the real pets kids often beg for and emulate those behaviors.

Aside from designing the physical character of the toy and accessories, as the product designer on the team, I was also responsible for designing the interaction of the toys and the play experience for the kids. Each furReal pet has not only mechanical movement but also sound effects that help enrich the play experience.

Due to cost concerns and a few other reasons, the microcontroller we were using for our toys at the time only allowed for limited offline data storage. This means that there’s a limited amount of interaction – sound effects or sensors we could store on the microcontroller. As you can imagine, it was quite challenging.

Product design years

As the product designer, I had to think through every interaction that the kids could receive when playing with the toy. I worked closely with cross-functional teams including sound designers, mechanical and electrical engineers to figure out how we could maximize the play experience to make each interaction the most physically engaging and emotionally responsive.

For example, if there’s a color sensor embedded in the toy to recognize different accessories, then I had to map out every response the toy would give to the kids each time they place a different accessory in front of the toy.

The biggest hurdle of toy design is that often times the kids get tired of their toys very quickly, especially after they figure out what kind of responses they will get from the toys after playing with it a few times. This becomes especially challenging if you are working with static data.

How can we make toys smarter?

Although I gained a tremendous amount of experience and knowledge while working on the furReal team, I kept asking myself these questions – can I make our toys smarter? How?

Keeping these questions in mind, I did a lot of market research and competitive landscape analysis to identify the trendiest toys on the market. I found that the products considered the most popular smart toys at the time, including Cozmo from Anki, all used dynamic data instead of static data in their products.

Dynamic data allows users to communicate with their toys via the internet, and receive timely responses allowing “infinite” (well there’s still a limit, let’s be completely honest) interaction between the users and toys. I realized that I needed more technical knowledge to understand how the communication between the Internet and the products is driven, and how I could implement this type of interaction into my designs to augment the user experience.

After spending some time researching and thinking about this, I decided to take a break from my job and join a master’s program to continue my education and expand my technical knowledge.

Mastering product design at the University of Pennsylvania

I began my masters in the Integrated Product Design program at the University of Pennsylvania, where I get to collaborate with people from different backgrounds to create meaningful solutions collaboratively, while exploring my interests and the career path that I would like to take moving forward. For me, although I knew I wanted to gain more knowledge and experience working with technology, I was more inquisitive about how to pick the right technology to work with and how to apply it to my design.

In the first semester, I took a class that introduced me to working with Arduino to create smart objects. During this class, I created an interactive storybook that allows you to engage with the elements of the story, a flower shaped music box that blooms revealing a singing fairy, as well as a mechanical flower that changes shape to communicate emotion to help children with autism communicate their feelings.

From a technical standpoint, the class was super helpful because it allowed me to get a taste of what it’s like to work with microcontrollers and sensors to create objects that are interactive (smart!), and helped me build some basic programming skills. Though this course allowed me to build a foundation of skills, I still felt constrained – partially because of my minimal programming experience, and partially because of the limitations of the microcontroller I was using.

My desire to explore more advanced and varied interactions led me to discover a class that focused on creating connected objects and experiences. It turned out to be on of the best classes I’ve ever taken.

I began using microcontrollers from Particle, including the Photon, which opened up so many more opportunities for me. The best advantage of using Photon is its cloud connectivity, which encouraged me to think outside of box of beyond traditional interaction design. When prototyping ideas with a Photon, I can leverage that flexibility and come up with different alternatives to implement my ideas. Here are a few project examples to help explain.

Making better food choices with Edie

Meet Edie – he’s a big fan of ice cream, but he has been become a little chubby since he lost control over his consumption of ice cream. As much as he hates it, Edie knows that he needs to change his diet up a bit, but he needs some motivation to achieve his goal. It is the user’s responsibility to help him get back to a healthy diet by feeding him with veggies – like broccoli.

As you can imagine, it is not going to be an easy process since broccoli doesn’t taste anything like ice cream, but it’s a necessary evil. As visualized below, Edie can be very moody, especially during the training process. One second he could be very excited, and the next second he can get very grumpy.

People all loved the interaction they had when playing with Edie, largely because all the interaction felt very organic. Everything was triggered by people communicating with him in written form instead of using a few buttons. The communication between the “trainer” and Edie was all done by email. With the combination of the usage of the Photon and IFTTT, I established a few hashtags to include in the subject of an email as the trigger, which then allows Edie to switch to different states to deliver various responses.

However, I still felt like challenging myself to maximize my use of the cloud connectivity functionality with the Photon. I was very lucky to work with a team of experienced programmers for another project — we had created a game that encourages interaction between the players by forcing real world cooperation and communication in order to beat the game.

Diving deeper with the Subsim Submarine

Subsim Submarine was created to challenge the fact that most games are largely impersonal. Even when games are made so people can play together, players ultimately end up playing as individuals on a team, not as a team that has to function as a whole.

Subsim Submarine is comprised of three play stations. The status of each station contributes to the overall game progress determining whether the players win or lose the game. Each play station has a command screen that tells the player what to complete in each round. The command displayed on your screen, however, is not the command for you, but for your fellow players.

In order for everyone to complete their task for the round, each player needs to communicate the command they’re reading on their screen to the rest of the team, while also catching the command that corresponds to their own station from the others. Only if all the players on the team complete their tasks do they get to contribute a point to their overall progress bar. If they fail to communicate and complete all the tasks, they lose a point and affect the status of the submarine at the main station.

The team must win 3 rounds of the game in order to save the submarine from sinking. When the team wins, the submarine plays a victorious sound effect, spins to show off the triumph, and blows some cheerful bubbles from its periscope to celebrate the winning of the game.

If the team loses, the submarine doesn’t hold back, but throws shame on all the players by playing a losing sound effect, spinning out of control, and blowing bubbles at the players’ faces – indicating that the submarine is sinking.

Want to witness all these glorious bubbles with some cheerful SFX? You better practice hard on this game with your fellow players!

In this project, the Particle Photon’s cloud connectivity was key. The central device (the submarine) sends out commands to the peripheral devices, creating time-dependent game elements. In return, the outcome of each peripheral device contributes to the end result of the game. Since the end result of the game is less predictable, it becomes more exciting and engaging to the players, drawing them back to the play over and over again.

Looking ahead

Now that I have experience working with IoT, I started to think of how I could apply this experience to my design. As I’m looking for the next step in my career, I am evaluating what kind of skills companies are looking for in a designer.

Through conversations with companies and in reading articles from other designers, I realize that there is an emerging trend of companies looking for designers with a more diverse, hybrid background, especially those with technical knowledge and experience.

Work environments have become more collaborative where designers are always working hand in hand with engineers and developers. Having technical knowledge and experience helps to foster communication and understanding between these roles, and in turn a more productive workflow. Designers with a good grasp of the most current technology trends can employ this knowledge when brainstorming and ideating solutions to create designs for their consumers and clients which lead the industry forward.