Robots Read Your Emotions, The Next Shopping Frontier

Emotion-reading robot Pepper
Emotion-reading robot Pepper
Paul Molga

MARSEILLE — Have you met Pepper? This four-feet-tall emotion-reading robot is expected to hit stores soon in Tokyo, where technology lovers will be able to acquire one for the equivalent of $1,650. The child-faced robot, the latest invention of French start-up Aldebaran, was created to "live alongside humans." But household chores such as vacuuming or cooking are not among Pepper's abilities. Instead, this aristocrat of the robot tribe is more like Star Wars" C-3PO.

Like its golden movie counterpart, it's a protocol droid, "endearing and kind," says Aldebaran's founder and CEO Bruno Maisonnier. It doesn't move the same way C-3PO does, but its many sensors feed its algorithms with information about the people it talks to, making conversations with the robot rather entertaining.

"Pepper understands our primary emotions: joy, sadness, anger, surprise, neutrality," Maisonnier explains. "It can determine the sex and the age of a person, and therefore identify all members of a family. It can keep up with 70% of a conversation. By analyzing our facial expressions, our vocabulary and our body language, it guesses your mood and adapts to your behavior. If you frown, it'll understand that something's bothering you and can try to cheer you up by, for example, playing a song you really like."

After having spent several months with the people at SoftBank, a Japanese telecom company and Aldebaran's primary shareholder, Pepper is said to spark as much curiosity as good humor. "Our goal is to make kind, pet-like humanoid robots that will live with humans as an artificial species," Maisonnier says.

From pouts and frowns to grins and smiles, our expressions betray pretty much all our feelings. And thanks to the progress achieved in mathematical analysis, artificial intelligence specialists have exploited this metalanguage of facial expression so that, one day, machines can have a certain form of empathy.

"Our faces contribute to 55% of the global impact of the message we're expressing," explains Axel Boidin, founder of French start-up Picxel, which specializes in facial recognition. "From a physiological point of view, emotional responses translate into a combination of distortions of our facial features that inform the people in front of us of our real intentions, and so contribute to coordinating the conversation. Robots will soon be able to understand these rules."

A longtime pursuit

Scientists have been trying to turn our mimicking into equations for a long time. In the 1970s, psychologist Paul Ekman even made it his specialty by decrypting the Rosetta Stone of emotions (what he called the "Facial Action Coding System"), which is now the basis of the universal alphabet book of behavioral psychology. The dictionary they thus devised lists the 10,000 facial expressions our 43 facial muscles are capable of producing. Most of them are funny faces, and we're able to distinguish a tiny part of the 3,000 combination that actually mean something.

Will a robot do better? At this point, the few companies that have gone into automated emotional recognition are a little bit powerless. They all know more or less how to identify the characteristic elements of the seven sorts of basic emotions Ekman listed. Most of them use image libraries as a comparison tool. In the U.S., the start-up Affectiva, founded by researchers from the Massachusetts Institute of Technology, uses a database of thousands of emotional reactions that allows its algorithms to decipher what a camera records. The company hopes to equip future smartphones with software capable of analyzing our reactions when we're following an online course or playing a video game.

Aldebaran robots — Photo: Facebook page

Picxel has a different approach. Its algorithm works with ultrafast cameras to track the micro-expressions that reveal our most intimate emotions. These muscular contractions don't last longer than a few microseconds, are impossible to control and, most importantly, they don't lie. They're the Freudian slips of our body language.

Some of us are naturally able to perceive them, like Tim Roth's character in the series Lie To Me. Professor Ekman, who trains FBI and CIA agents, among others, also says that we can learn to read them — though the machine that can do this automatically hasn't been invented, yet. But "giving a camera the ability of tracking these unsaid emotions will revolutionize many fields," Boidin speculates.

In the future, stores could, for instance, use connected screens as a new sort of dressing room. These screens could analyze our reactions while suggesting different products to determine what we like most. In different sections of a store, they could also determine which products best attract consumers attention.

These powerful tools will eventually enable pollsters to collect impartial information on how a film, politician or advertisement is perceived. "With those tools, our computers will be able to automatically adapt the environments and luminosity depending on our mood, as well as adapting their behavior," Boidin explains. "They'll know they should be accommodating if they see we're angry, or stimulating if they think we're being apathetic."

In cars, emotion detectors will come in handy to anticipate the first signs of fatigue. Connected to surveillance cameras at border checkpoints, airports and public places, as Paul Ekman envisions, they'll be able to identify suspicious behavior to help locate terrorists.

"The collected data will enable us to build incredibly evolved models on how we behave, how we make decisions and engage," Ekman says. In other words, intuition will soon be a thing of the past.

Keep up with the world. Break out of the bubble.
Sign up to our expressly international daily newsletter!

The Olympic torch is lit at the Archaeological site of Olympia in Greece.

Anne-Sophie Goninet, Jane Herbelin and Bertrand Hauger

👋 Asham!*

Welcome to Tuesday, where Pyongyang test fires a suspected submarine-launched missile, Colin Powell is remembered, Poland-EU tensions rise, and yay (or yeesh): it's officially Ye. Meanwhile, our latest edition of Work → In Progress takes the pulse of the new professional demands in a recovering economy.

[*Oromo - Ethiopia and Kenya]


• North Korea fires missile off Japan coast: South Korea military reports that North Korea has fired a ballistic missile into the waters off the coast of Japan. The rocket, thought to have been launched from a submarine, is the latest test in a series of provocations in recent weeks.

• Poland/EU tensions: Polish Prime Minister Mateusz Morawiecki has accused the EU of "blackmail" and said the European Union is overstepping its powers, in a heated debate with EU Commission President Ursula von der Leyen over the rule of law. The escalation comes in the wake of a controversial ruling by Poland's Constitutional Tribunal that puts national laws over EU principles.

• Colin Powell remembered: Tributes are pouring for former U.S. Secretary of State Colin Powell, after his death yesterday at age 84. Although fully vaccinated, Powell died from complications from COVID-19 as he was battling blood cancer. A trailblazing soldier, he then helped shape U.S. foreign policy, as national security adviser to President Ronald Reagan, then chairman of the Joint Chiefs under Presidents George H.W. Bush and Bill Clinton, and served as the nation's top diplomat for George W. Bush. Powell's legacy is, by his own admission, "blotted" by his faulty claims of weapons of mass destruction to justify the U.S. war in Iraq.

• Russia to suspend NATO diplomatic mission amid tension: Russia is suspending its diplomatic mission to NATO and closing the alliance's offices in Moscow as relations with the Western military block have plunged to a new low. Foreign Minister Sergei Lavrov announced the move after NATO expelled eight diplomats from Russia's mission for alleged spying. Relations between NATO and Russia have been strained since Moscow annexed Ukraine's Crimean Peninsula in 2014.

• Ecuador state of emergency to battle drug crime: President Guillermo Lasso declared a state of emergency amid Ecuador's surge in drug-related violence. He announced the mobilization of police and the military to patrol the streets, provide security, and confront drug trafficking and other crimes.

• Taliban agrees to house-to-house polio vaccine drive: The WHO and Unicef campaign will resume nationwide polio vaccinations after more than three years, as the new Taliban government agreed to support the campaign and to allow women to participate as frontline health workers. Afghanistan and neighboring Pakistan are the last countries in the world with endemic polio, an incurable and infectious disease

• Kanye West officially changes name: Some say yay, some say yeesh, but it's official: The-artist-formerly-known-as-Kanye-West has legally changed his name to Ye, citing "personal reasons."


The Washington Post pays tribute to Colin Powell, the first Black U.S. Secretary of State, who died at 84 years old from complications from COVID-19.


Jashn-e Riwaaz

Indian retailer Fabindia's naming its new collection Jashn-e Riwaaz, an Urdu term meaning "celebration of tradition," has been met with severe backlash and calls for boycott from right-wing Hindu groups. They are accusing the brand of false appropriation by promoting a collection of clothes designed for Diwali, the Hindu festival of lights, but giving it a name in Urdu, a language spoken by many Muslims.


Work → In Progress: Where have all the workers gone?

After the economic slowdown brought on by the coronavirus pandemic, companies all over the world are taking advantage of loosened lockdowns and progress on the vaccine front to ramp up operations and make up for lost productivity. But the frenetic spurts of the recovery are getting serious pushback. This edition of Work → In Progress looks not only at the coming changes in our post-COVID economy, but also the ways our world is re-evaluating professional obligations.

🗓️ Hail the 4-day week Across the planet, the shorter work week trend is spreading like wildfire. Four is the new five. Spain began experimenting with the concept earlier this year. New Zealand launched a similar trial run in 2020. And in Iceland, efforts to curb working hours date all the way back to 2015, with significant results: 86% of the country's workforce gained the right to reduce work hours with no change in pay.

🚚 Empty seats In the United States, meanwhile, a severe lack of truck drivers has the country's transportation industry looking to hire from abroad. The only problem is … the shortage is happening worldwide, in part because of the e-commerce boom in the wake of worldwide quarantines. The Italian daily Il Fatto Quotidiano reports that companies will be scrambling to fill the jobs of 17,000 truck drivers in the next two years. The article blames low wages and the dangerous nature of the job, stating that Italian companies are making moves to employ foreign workers.

💼 Key help wanted It's all well and good to question current working conditions. But what about 20 years from now? Will we be working at all? A recent article in the French daily Les Echos posed just that question, and posits that by 2041 — and with the exception of a few select jobs — automation and digitalization will decimate employment. The piece refers to the lucky few as "essential workers," a concept that originated with COVID lockdowns when almost all labor halted and only a minority of workers capable of performing society's most crucial in-person tasks were allowed to carry on.

➡️ Read more on Worldcrunch.com


I'm worried for my Afghan sisters.

— Nobel Peace Prize recipient Malala Yousafzai Nobel Prize tells the BBC that despite the Taliban's announcement that they would soon lift the ban on girls' education in Afghanistan, she worries it "might last for years."

✍️ Newsletter by Anne-Sophie Goninet, Jane Herbelin and Bertrand Hauger

Are you more yay or yeesh about the artist currently known as Ye? Let us know how the news look in your corner of the world — drop us a note at info@worldcrunch.com!

Keep up with the world. Break out of the bubble.
Sign up to our expressly international daily newsletter!