42
@ U S T . H K
It is perhaps no coincidence that it has taken a
woman engineer to build the virtual mind of a robot
that goes further than any other in the empathy it –
or rather she – can display.
Meet Zara the Supergirl, the virtual agent with
emotional intelligence created at HKUST by one
of the leading researchers in the field, Prof Pascale
Fung, and her diverse student team.
Unlike most of the current speech recognition
systems, Zara has been programmed to detect
meaning and intent during conversations and
respond to the emotion and personality type of her
user by observing their facial expression, manner
of speaking, language used, and the context. This
breakthrough has attracted global academic,
industry and popular media attention since
Zara was first unveiled at the 2015 World
Economic Forum in Davos. While some form
of emotional intelligence in avatars is not
new, Zara represents the first full integration of
multimodal emotional perception and response
in an artificial intelligence (AI) system.
The frontier development is a result of
advanced tools and algorithms devised by Prof
Fung, whose goal is to create virtual agents and
robots that can be viewed as buddies rather
than unfeeling machines. What makes Zara’s
extraordinary “mind” tick? “We came up with
an empathy module for recognizing emotion
from the way people talk, based on large
The concept
of empathetic
robots can go
far. In the next
few years you
are going to see
an explosion
of ideas
PROF PASCALE FUNG
Professor of Electronic
and Computer Engineering
Zara the Supergirl
Prof Pascale Fung
I KNOW HOW YOU FEEL
Zara the Supergirl in conversation
with her creator. Zara is multilingual
and can comprehend emotions from
how a person looks and sounds.
amounts of audio data from different nationalities
and cultures,” Prof Fung said. This required teaching
machines not only to recognize the meaning of
words but also comprehend acoustic signals and
facial expressions – just as a human would. “And by
learning, Zara can figure out what to do next. She
uses recognition of emotion and personality to enable
the systemto converse and feel what the user feels.”
Such advanced functionality builds on the
University’s expertise in multilingual spoken
language understanding developed over the
past two decades. Zara’s “talents” are achieved
through neural network machine learning.
Illustration: Zohar Lazar
