Robots Read Your Emotions, The Next Shopping Frontier

Emotion-reading robot Pepper
Emotion-reading robot Pepper
Paul Molga

MARSEILLE — Have you met Pepper? This four-feet-tall emotion-reading robot is expected to hit stores soon in Tokyo, where technology lovers will be able to acquire one for the equivalent of $1,650. The child-faced robot, the latest invention of French start-up Aldebaran, was created to "live alongside humans." But household chores such as vacuuming or cooking are not among Pepper's abilities. Instead, this aristocrat of the robot tribe is more like Star Wars" C-3PO.

Like its golden movie counterpart, it's a protocol droid, "endearing and kind," says Aldebaran's founder and CEO Bruno Maisonnier. It doesn't move the same way C-3PO does, but its many sensors feed its algorithms with information about the people it talks to, making conversations with the robot rather entertaining.

"Pepper understands our primary emotions: joy, sadness, anger, surprise, neutrality," Maisonnier explains. "It can determine the sex and the age of a person, and therefore identify all members of a family. It can keep up with 70% of a conversation. By analyzing our facial expressions, our vocabulary and our body language, it guesses your mood and adapts to your behavior. If you frown, it'll understand that something's bothering you and can try to cheer you up by, for example, playing a song you really like."

After having spent several months with the people at SoftBank, a Japanese telecom company and Aldebaran's primary shareholder, Pepper is said to spark as much curiosity as good humor. "Our goal is to make kind, pet-like humanoid robots that will live with humans as an artificial species," Maisonnier says.

From pouts and frowns to grins and smiles, our expressions betray pretty much all our feelings. And thanks to the progress achieved in mathematical analysis, artificial intelligence specialists have exploited this metalanguage of facial expression so that, one day, machines can have a certain form of empathy.

"Our faces contribute to 55% of the global impact of the message we're expressing," explains Axel Boidin, founder of French start-up Picxel, which specializes in facial recognition. "From a physiological point of view, emotional responses translate into a combination of distortions of our facial features that inform the people in front of us of our real intentions, and so contribute to coordinating the conversation. Robots will soon be able to understand these rules."

A longtime pursuit

Scientists have been trying to turn our mimicking into equations for a long time. In the 1970s, psychologist Paul Ekman even made it his specialty by decrypting the Rosetta Stone of emotions (what he called the "Facial Action Coding System"), which is now the basis of the universal alphabet book of behavioral psychology. The dictionary they thus devised lists the 10,000 facial expressions our 43 facial muscles are capable of producing. Most of them are funny faces, and we're able to distinguish a tiny part of the 3,000 combination that actually mean something.

Will a robot do better? At this point, the few companies that have gone into automated emotional recognition are a little bit powerless. They all know more or less how to identify the characteristic elements of the seven sorts of basic emotions Ekman listed. Most of them use image libraries as a comparison tool. In the U.S., the start-up Affectiva, founded by researchers from the Massachusetts Institute of Technology, uses a database of thousands of emotional reactions that allows its algorithms to decipher what a camera records. The company hopes to equip future smartphones with software capable of analyzing our reactions when we're following an online course or playing a video game.

Aldebaran robots — Photo: Facebook page

Picxel has a different approach. Its algorithm works with ultrafast cameras to track the micro-expressions that reveal our most intimate emotions. These muscular contractions don't last longer than a few microseconds, are impossible to control and, most importantly, they don't lie. They're the Freudian slips of our body language.

Some of us are naturally able to perceive them, like Tim Roth's character in the series Lie To Me. Professor Ekman, who trains FBI and CIA agents, among others, also says that we can learn to read them — though the machine that can do this automatically hasn't been invented, yet. But "giving a camera the ability of tracking these unsaid emotions will revolutionize many fields," Boidin speculates.

In the future, stores could, for instance, use connected screens as a new sort of dressing room. These screens could analyze our reactions while suggesting different products to determine what we like most. In different sections of a store, they could also determine which products best attract consumers attention.

These powerful tools will eventually enable pollsters to collect impartial information on how a film, politician or advertisement is perceived. "With those tools, our computers will be able to automatically adapt the environments and luminosity depending on our mood, as well as adapting their behavior," Boidin explains. "They'll know they should be accommodating if they see we're angry, or stimulating if they think we're being apathetic."

In cars, emotion detectors will come in handy to anticipate the first signs of fatigue. Connected to surveillance cameras at border checkpoints, airports and public places, as Paul Ekman envisions, they'll be able to identify suspicious behavior to help locate terrorists.

"The collected data will enable us to build incredibly evolved models on how we behave, how we make decisions and engage," Ekman says. In other words, intuition will soon be a thing of the past.

Keep up with the world. Break out of the bubble.
Sign up to our expressly international daily newsletter!

7 Ways The Pandemic May Change The Airline Industry For Good

Will flying be greener? More comfortable? Less frequent? As the world eyes a post-COVID reality, we look at ways the airline industry has been changing through a pandemic that has devastated air travel.

Ready for (a different kind of) takeoff?

Carl-Johan Karlsson

It's hard to overstate the damage the pandemic has had on the airline industry, with global revenues dropping by 40% in 2020 and dozens of airlines around the world filing for bankruptcy. One moment last year when the gravity became particularly apparent was when Asian carriers (in countries with low COVID-19 rates) began offering "flights to nowhere" — starting and ending at the same airport as a way to earn some cash from would-be travelers who missed the in-flight experience.

More than a year later today, experts believe that air traffic won't return to normal levels until 2024.

But beyond the financial woes, the unprecedented slowdown in air travel may bring some silver linings as key aspects of the industry are bound to change once back in full spin, with some longer-term effects on aviation already emerging. Here are some major transformations to expect in the coming years:

Cleaner aviation fuel

The U.S. administration of President Joe Biden and the airline industry recently agreed to the ambitious goal of replacing all jet fuel with sustainable alternatives by 2050. Already in a decade, the U.S. aims to produce three billion gallons of sustainable fuel — about one-tenth of current total use — from waste, plants and other organic matter.

While greening the world's road transport has long been at the top of the climate agenda, aviation is not even included under the Paris Agreement. But with air travel responsible for roughly 12% of all CO2 emissions from transport, and stricter international regulation on the horizon, the industry is increasingly seeking sustainable alternatives to petroleum-based fuel.

Fees imposed on the airline industry should be funneled into a climate fund.

In Germany, state broadcaster Deutsche Welle reports that the world's first factory producing CO2-neutral kerosene recently started operations in the town of Wertle, in Lower Saxony. The plant, for which Lufthansa is set to become the pilot customer, will produce CO2-neutral kerosene through a circular production cycle incorporating sustainable and green energy sources and raw materials. Energy is supplied through wind turbines from the surrounding area, while the fuel's main ingredients are water and waste-generated CO2 coming from a nearby biogas plant.

Farther north, Norwegian Air Shuttle has recently submitted a recommendation to the government that fees imposed on the airline industry should be funneled into a climate fund aimed at developing cleaner aviation fuel, according to Norwegian news site E24. The airline also suggested that the government significantly reduce the tax burden on the industry over a longer period to allow airlines to recover from the pandemic.

Black-and-white photo of an ariplane shot from below flying across the sky and leaving condensation trails

High-flying ambitions for the sector

Joel & Jasmin Førestbird

Hydrogen and electrification

Some airline manufacturers are betting on hydrogen, with research suggesting that the abundant resource has the potential to match the flight distances and payload of a current fossil-fuel aircraft. If derived from renewable resources like sun and wind power, hydrogen — with an energy-density almost three times that of gasoline or diesel — could work as a fully sustainable aviation fuel that emits only water.

One example comes out of California, where fuel-cell specialist HyPoint has entered a partnership with Pennsylvania-based Piasecki Aircraft Corporation to manufacture 650-kilowatt hydrogen fuel cell systems for aircrafts. According to HyPoint, the system — scheduled for commercial availability product by 2025 — will have four times the energy density of existing lithium-ion batteries and double the specific power of existing hydrogen fuel-cell systems.

Meanwhile, Rolls-Royce is looking to smash the speed record of electrical flights with a newly designed 23-foot-long model. Christened the Spirit of Innovation, the small plane took off for the first time earlier this month and successfully managed a 15-minute long test flight. However, the company has announced plans to fly the machine faster than 300 mph (480 km/h) before the year is out, and also to sell similar propulsion systems to companies developing electrical air taxis or small commuter planes.

New aircraft designs

Airlines are also upgrading aircraft design to become more eco-friendly. Air France just received its first upgrade of a single-aisle, medium-haul aircraft in 33 years. Fleet director Nicolas Bertrand told French daily Les Echos that the new A220 — that will replace the old A320 model — will reduce operating costs by 10%, fuel consumption and CO2 emissions by 20% and noise footprint by 34%.

International first class will be very nearly a thing of the past.

The pandemic has also ushered in a new era of consumer demand where privacy and personal space is put above luxury. The retirement of older aircraft caused by COVID-19 means that international first class — already in steady decline over the last decades — will be very nearly a thing of the past. Instead, airplane manufacturers around the world (including Delta, China Eastern, JetBlue, British Airways and Shanghai Airlines) are betting on a new generation of super-business minisuites where passengers have a privacy door. The idea, which was introduced by Qatar Airways in 2017, is to offer more personal space than in regular business class but without the lavishness of first class.

Aerial view of Rome's Fiumicino airport

Aerial view of Rome's Fiumicino airport


Hygiene rankings  

Rome's Fiumicino Airport has become the first in the world to earn "the COVID-19 5-Star Airport Rating" from Skytrax, an international airline and airport review and ranking site, Italian daily La Repubblica reports. Skytrax, which publishes a yearly annual ranking of the world's best airports and issues the World Airport Awards, this year created a second list to specifically call out airports with the best health and hygiene standards.

Smoother check-in

​The pandemic has also accelerated the shift towards contactless traveling, with more airports harnessing the power of biometrics — such as facial recognition or fever screening — to reduce touchpoints and human contact. Similar technology can also be used to more efficiently scan physical objects, such as explosive detection. Ultimately, passengers will be able to "check-in" and go through a security screening anywhere at the airports, removing queues and bottlenecks.

Data privacy issues

​However, as pointed out in Canadian publication The Lawyer's Daily, increased use of AI and biometrics also means increased privacy concerns. For example, health and hygiene measures like digital vaccine passports also mean that airports can collect data on who has been vaccinated and the type of vaccine used.

Photo of planes at Auckland airport, New Zealand

Auckland Airport, New Zealand

Douglas Bagg

The billion-dollar question: Will we fly less?

At the end of the day, even with all these (mostly positive) changes that we've seen take shape over the past 18 months, the industry faces major uncertainty about whether air travel will ever return to the pre-COVID levels. Not only are people wary about being in crowded and closed airplanes, but the worth of long-distance business travel in particular is being questioned as many have seen that meetings can function remotely, via Zoom and other online apps.

Trying to forecast the future, experts point to the years following the 9/11 terrorist attacks as at least a partial blueprint for what a recovery might look like in the years ahead. Twenty years ago, as passenger enthusiasm for flying waned amid security fears following the attacks, airlines were forced to cancel flights and put planes into storage.

40% of Swedes intend to travel less

According to McKinsey, leisure trips and visits to family and friends rebounded faster than business flights, which took four years to return to pre-crisis levels in the UK. This time too, business travel is expected to lag, with the consulting firm estimating only 80% recovery of pre-pandemic levels by 2024.

But the COVID-19 crisis also came at a time when passengers were already rethinking their travel habits due to climate concerns, while worldwide lockdowns have ushered in a new era of remote working. In Sweden, a survey by the country's largest research company shows that 40% of the population intend to travel less even after the pandemic ends. Similarly in the UK, nearly 60% of adults said during the spring they intended to fly less after being vaccinated against COVID-19 — with climate change cited as a top reason for people wanting to reduce their number of flights, according to research by the University of Bristol.

At the same time, major companies are increasingly forced to face the music of the environmental movement, with several corporations rolling out climate targets over the last few years. Today, five of the 10 biggest buyers of corporate air travel in the US are technology companies: Amazon, IBM, Google, Apple and Microsoft, according to Taipei Times, all of which have set individual targets for environmental stewardship. As such, the era of flying across the Atlantic for a two-hour executive meeting is likely in its dying days.

Keep up with the world. Break out of the bubble.
Sign up to our expressly international daily newsletter!