*Hear(Say)Think! is an experience design exploration questioning the legitimacy and competency of AI in translation softwares by physicalizing it’s process.



︎︎︎Background

I’ve always been intrigued by how language is affected by culture, and the ambitous task of translators to act as facilitators between two different linguistic worlds. I found an article︎︎︎ that claimed how AI and ML will make translators jobs redundant. That made me think whether language can really be understood through algorithmic structures. To understand language, one must also understand it’s cultural context. So then my question evolved to — can culture be quantified, and learned through computation?

︎︎︎Research 

Google Translate is at the forefront of language translation technology, so I started to look into it’s functionality and effectiveness in the real world. I started by imagining and mapping out different scenarios where understanding context would be significant for translation. This included — writing an essay in a foreign language, asking a local for a shortcut route, and in the case of a local being approached by a tourist for help.

Throughout these flows, the most apparent thing was that at no point in the step in the app, where context played a role in translating anything. To me, it seemed like Google Translate was skirting past by obscuring culture.


My suspicion was confirmed by the plethora of imagery I found online of Google Translate fails. Some of them were arbitary, and some pretty funny.


︎︎︎Insights


︎Translation softwares obscure culture,
as they do not understand context.
︎Even if AI were to understand context, will it be able to keep up with the pace of everyday culture, lingo, and individual expressions? Probably not. There is no “logic” to culture.

︎︎︎Opportunity 


To physicalize the process of obscurement of culture, that occurs in translation software, to prove that without context, communication becomes much harder even within the bounds of the same language.

︎︎︎Initial Prototypes 


I. Tattoo generator︎︎︎ is based on the commonly observed phenomenon of people getting phrases from other languages tattooed on them — primarily Asian languages — and being misinformed about the meaning, never really knowing what they got tattooed on them. 



II. The Window came as a literal attempt to obscure communication, and to prove that miscommunication can happen even within the same language if context is missing.



︎︎︎*Hear(Say)Think! — a game of communication through disruption.


Through a set of three mini-games, the *Hear(Say)Think! kit invites participation to disrupt communication by taking away the context from words.


Each mini-game comes with set-up instruction, rules and all the game pieces. The only thing to do is play! 


The Win-Woe
Utilizing a literal barrier in communication, two people have to talk to each other, one can only write, the other can only speak.

Speak & Spell Based on the simple concept of homophones (similar sounding words), one player says a word without revealing the spelling, while the other tries to guess which spelling it is!

Meaningfully Twisted Based on the concept of homonyms (same spelling, different meaning words), the two players write the meaning of the word on the card, comparing it to see if they wrote the same meaning.  

The discussions that followed after resulted in a lot of discourse about how important context is to the way we perceive things. Even with simple words like “rock”, one of us would think “music” and the other would think “stone”, or “rocking chair”.


Learnings This project was my first time designing a participatory experience. It was rewarding, and made me notice and really appreciate cultural nuance.
One of my takeaways was to not try to control the natural reactions and interactions people had while playing these games.