R E S E A R C H @ H K U S T
43
Prof Fung and her
team of students from
diverse backgrounds
work to enhance the
emotional intelligence
of their virtual agents.
The learned output is derived from
“neurons” firing signals across multiple layers
while simultaneously adjusting weighting
functions and biases across these layers.
Algorithms developed by the HKUST team enable
them to make Zara and related applications
increasingly responsive in real time.
The personality recognition capacity that Prof
Fung’s team has developed draws on common
classifications used by psychologists, including
the Big Five personality traits and Myers-Briggs
indicators. Personality of the user is assessed in a
question-and-answer session that takes less than five
minutes. Zara’s responses mimic how we intuitively
react with different audiences, whether a child or
adult, man or woman.
Since Davos, the team has refined multiple facets
of linguistic and emotional intelligence for robots
and avatars. As Zara interacts with more people
and gathers more data, the machine-learning
algorithms enable her to be more intelligent and more
empathetic. The technology behind Zara will take
the leap from screen to embodiment in the form of
a receptionist robot, who will interact with visitors to
HKUST, and guide them to their destinations.
The potential applications of empathetic
machines are endless. Prof Fung is particularly
excited about uses in healthcare, which range from
monitoring when an elderly person needs help to
assisting in the diagnosis of Alzheimer’s disease.
Prof Fung’s team has also developed a virtual
psychologist using similar technology to Zara,
focusing on detecting stress and depression in a
user, and providing counseling services. In human
resources, it can be used for initial job interviews
without the unconscious bias inherent in human
interviewers. For business and finance, the team is
extending speech and language data analytics to
sentiment analysis.
The technology is already available commer-
cially in a smart “robot speaker with a heart” that
senses your mood by interacting with you, and
responds with the music and lighting to match.
Music is suggested according to the time, weather,
and the user’s previous listening choices. The
classification of music is again based on a neural
network algorithm that recognizes the mood, genre
and even artist for each piece.
Prof Fung, a founding member of the Human
Language Technology Center at HKUST, established
in 1997, leads an international team of postdoctoral
researchers and students from Asia and Europe.
This is helpful for the research as students coming
from different cultural and educational backgrounds
bring different insights and perspectives to the
Provide assistance
to the elderly.
Carry out
sentiment
analysis for
business and
finance.
Serve as a social
companion to take
care of people
in need.
Conduct initial job
interviews without
unconscious bias.
Act as a virtual
psychologist to provide
counseling services.
Empathetic Robots, Wide-reaching Applications
interdisciplinary field of AI. The team is enhancing
AI capacities in areas such as conversational humor,
abusive language recognition, mood recognition,
music retrieval, and sentiment analysis. One of
her students is working on the algorithms that will
make the voice of virtual agents such as Zara more
emotional.
While Prof Fung is fascinated with humanistic
robots, she is alert to the potential societal challenges
that AI poses. She is a member of the Global Future
Council on AI and Robotics of the World Economic
Forum, which advises policymakers and CEOs to be
mindful about the usages of technology, and believes
laws and regulationswill be needed to prevent abuse.
