Growing up in the Middle East, one of my most vivid childhood memories was the Iraqi invasion of Kuwait in 1990. Overnight, our world changed. My parents lost their jobs, their home, their savings. In the aftermath, I felt a flood of emotions unlike anything I’d ever experienced, but I suppressed my feelings and my family did the same. Even in a time of immense disruption, fear, and uncertainty, we forged ahead, accepting our change in circumstance.
Thirty years later, as I watch the world reel from the COVID-19 pandemic, I can’t help but feel like that 12-year-old girl in Kuwait again. Our world has changed. Businesses, universities, and schools are now operating largely online. Conferences are canceled, postponed, or shifting to virtual events. This is not only a health crisis, but also an economic crisis that compounds feelings of angst, fear, and isolation for everyone. On top of all of that, we are in a moment of reckoning as our country deals with issues of injustice and systemic racism that have many of us wanting to stand up, share our perspectives, listen to others, and combat this together.
In challenging times we tend to crave human connection—but we’re being asked to do the opposite through social distancing. Where does that leave us?
Now, more than ever, we’re relying on technology to connect us. But there’s a big problem. Technology is emotion blind, and while we may feel a semblance of connection when we communicate in a digital world, it can often feel like something is missing.
When we communicate in person, we convey so much more than just the words we say: We emote using facial expressions, vocal intonations, and non-verbal cues. But technology simply wasn’t built to capture the nuances of how we interact when we’re face-to-face with loved ones, coworkers, teachers, or friends. While video conferencing platforms like Zoom have given us the foundation to stay connected online, I don’t think I’m alone in feeling the Zoom fatigue. There’s an element of in-person interaction and meaningful connection that just doesn’t come through the screen. Our emotions get lost in cyberspace.
As we increasingly live our lives online, we need to preserve the shared experience that’s felt at a live event, or in a classroom, or when visiting family members. One where, in a digital world, you can feel non-verbal expressions of excitement, empathy, curiosity, and sadness, rather than relying on an emoji or a text to get the message across.
To rectify that, I believe we need to teach technology to recognize emotions the way that people can. I’ve spent my career chasing that mission, building artificial emotional intelligence (Emotion AI): software that can understand nuanced human emotions and complex cognitive states from people’s facial and vocal expressions, in the same way that people understand one another.
It may seem counterintuitive to rely on something artificial to keep us human. But over the last few months I’ve spoken to business leaders, professors, healthcare professionals, and friends to envision what an emotion-enabled digital world could look like for our jobs, our classrooms, and our health.
AN EMOTION NEWSFEED FOR VIRTUAL EVENTS
Consider all of the conferences and events that are now going virtual. In a live event, you feed off of the energy in the room. For example, when I’m speaking at a live event, I can tell if people are engaged, if they’re bored, and if they’re laughing at my jokes (whether that’s out of politeness, or if I’m actually funny!). I can adapt my presentation based on that feedback.
But presenting virtually is painful. It’s an isolating experience both for the presenter—who has no idea whether or not their content is resonating—and for the audience, who’s missing out on how everyone else is responding. We lose the feeling of a shared experience.
Now, imagine if virtual events were able to capture all of the energy in a room within an online universe. If virtual conferences used technology with Emotion AI, some of that emotional energy could be restored. For example, attendees could opt in to turn on their device’s camera, and allow the platform to detect and share their emotional reactions throughout the presentation. The video of attendees wouldn’t need to be shared, but the technology could gather metrics on attendees’ emotional engagement and share that with the presenter and the “crowd.”
Picture an emotion newsfeed that could aggregate, moment-by-moment, the audience’s responses to a presentation, and visualize that energy; for example, imagine a graph that shoots up when people are engaged or enthusiastic, or displays the crying-laughing emoji when people are laughing. This kind of virtual emotional engagement could also be useful for large team meetings. As a CEO, one of my biggest concerns right now is making sure my team is still doing okay—that they’re engaged, motivated, and feeling connected. While a lot of that effort must be done in one-to-one interactions, large organizations could benefit from a boost in understanding the emotional health of an organization during large virtual team meetings.
EMOTION FEEDBACK FOR ONLINE LEARNING
Education is another area that’s suffering from this breakdown in connection. I have two kids at home who are in 11th and fifth grade. Like many of your kids, they’ve spent the last few months doing online learning programs for school. As a mom, I can’t help but wonder what more we can do to make sure they’re engaged in the same way as they would be when they’re in their classroom. Educators are feeling the same pressure.
For Sri Krishnamurthy, who teaches at Northeastern University and runs the data science consultancy and education program QuantUniversity, education should be collaborative, where students work in teams to replicate real-world projects. But that’s difficult to replicate online.
“The move to online has been a bit challenging and we are still adjusting,” Krishnamurthy says. “Students have been engaging in fewer discussions and I think that’s largely due to the fact that it’s harder to replicate the in-person experience online. It’s also been hard for me to gauge if my students are engaged, confused, or if they’re even following my lectures.”
We shouldn’t be left wondering if students are bored or engaged. Emotion AI could restore that sense of connection by sharing feedback with the teacher and with fellow classmates on how students are engaging in, or reacting to, a lesson—as long as students opt-in and consent.
TRACKING EMOTIONAL WELLBEING OVER TIME
But this isn’t just about our roles as professionals, students, or educators. Conducting our lives in an emotion-free, digital zone has implications for our health and wellbeing, too. This is especially true amidst the COVID-19 pandemic, as health organizations are advising people to consult with their doctors from home when possible. Medicare has expanded access to telehealth services during this time, but the experience will inevitably be different than speaking with your doctor in person.
A friend of mine who’s a physician echoed this. He told me that as his patients are being diagnosed with COVID-19, they’re unable to come in for risk of exposing others. Beyond their needs for physical care, there’s obviously a lot of stress and anxiety that comes with being sick right now, but the onus is currently on patients to self-report on their mental health.
Emotion AI could help collect this information longitudinally. For example, an app on a patient’s phone could track their emotional health over time, and share that with their healthcare provider as an additional source of data on their well being. Not only would that information be helpful for doctors, but it could also be shared with loved ones of patients or people whose relatives are high-risk. I know many of us (myself included) are worried about how our parents and grandparents are holding up, and while we can check in virtually, Emotion AI could help us feel assured that nothing is falling through the cracks when we inevitably have to be working or tending to the other needs of our family members.
Experts predict that, even as the world begins to reopen, the pandemic will have a long-term impact on the way we work, learn, and connect. Our reliance on technology to connect with the world will only grow stronger. Arming technology with artificial emotional intelligence might be the only way to preserve the things that make us human in the first place: empathy, emotion, and meaningful connection.
About the author: Rana el Kaliouby, PhD, is the cofounder and CEO of Affectiva and author of the book Girl Decoded.