How ‘Human’ Should Robots Be?

Choi, Sungwoo（崔成宇）

Hotel guests like interaction with devices that look and sound like them, but they can spark displeasure after service failures, new CUHK study shows

By Guy Haydon

The past decade has seen an increasing use of hi-tech automation and smart service robots – many with human-like features and behaviour. They use artificial intelligence (AI) to interact with people, offer directions and information, provide check-in services, and sort and transport baggage at leading airports, railway stations, hotels and restaurants in major cities around the world.

Despite this increasing use of robotics in frontline applications in the service industry, little remains known about how customers react when their performance fails to meet expectations. Do the words “I am sorry” hold the same emotional depth and meaning if it is said by a machine instead of a human being? This is a topic that a group of researchers, including at the Chinese University of Hong Kong (CUHK), decided to tackle.

“The use of more advanced technologies recognising physical or vocal signs, such as facial expressions or the tone of voice, can help service robots detect a customer’s dissatisfaction and respond to it more attentively.” – Prof. Choi Sungwoo

Consider that the continuing Covid-19 pandemic – which has seen countries around the world placed in lockdown, with air, boat and train travel halted while people have been forced to stay in their homes – has only heightened the interest of governments and businesses in using robotic devices to perform services in communal areas and buildings to help reduce the risk of the spread of the deadly coronavirus disease.

In the hotel industry, citizenM boutique hotel group, based in cities including Copenhagen Paris, New York and Shanghai has replaced front-desk staff with digital kiosks, while at the Mandarin Oriental in Las Vegas, in the U.S., a service robot handles guests’ requests and interacts with visitors while moving around the lobby. Three other lobby-based robots at the Sky Hotel in Johannesburg, South Africa, which opened in November 2020, can deliver room service, provide travel information, and each carry up to 74kg up to rooms.

When Robots Disappoint

A recent study found that about 70% of hotel managers expect to see the widespread use of service robots in customer contact service delivery, such as offering greetings and helping with check-in and check-out, by 2025. Yet new robot technology does not always work as planned. Japan’s Henn na, or “Strange”, Hotel group, is certified by Guinness World Records as the first to offer a hotel with working robots.

It opened its hotel to theme park guests in Nagasaki in 2015 using 243 smart robots – many with human-like features – accounting for about 90 per cent of its total staff. The robots were expected to interact and communicate with guests, including providing check-in services and sorting and transporting baggage to and from rooms.

However, by 2019 it had replaced more than half of them with people following complaints from unhappy guests. Technology malfunctions were blamed when the hotel’s doll-like voice assistants in rooms interrupted people’s conversations, messed up room service orders and repeatedly woke up guests during the night after mistaking snoring for requests for help. Concierge robots were also unable to perform tasks without the help of human staff.

“Although robots are known to make mistakes, we do not have much understanding about how people respond when service robots fail to perform tasks correctly, such as organising their hotel check-in or ordering room service food and drinks,” says Choi Sungwoo, Research Assistant Professor at the School of Hotel and Tourism Management at CUHK Business School, and co-author of a new study. “What do guests really think about these errors and what is the best way for hotels to try to resolve these problems and restore customer satisfaction?”

He and two co-researchers, Prof. Anna Mattila and Prof. Lisa Bolton, both based at Pennsylvania State University, addressed such questions by examining how social perceptions regarding the warmth and competence of service robots influence consumer reactions to service failures and recovery efforts by robots. Their findings were published in a report titled To Err Is Human(-oid): How Do Consumers React to Robot Service Failure and Recovery?

Projecting Service Warmth

Prof. Choi says service robots use technology, such as artificial intuitive and empathetic intelligence, to predict or detect a customer’s emotions – by recognising physical or vocal signs, such as facial expressions, hand gestures or the tone of voice – and respond so they appear to understand or feel what the person is experiencing, to enhance consumers’ service experiences. Such high levels of intelligence enable machines to deliver socially and emotionally interactive services, such as chatbots communicating with customers and frontline services delivered by robots.

The perceived “humanness” of service robots through their appearance can influence consumers’ attitudes and their willingness to interact. Prof. Choi says people tend to more strongly associate warmth – in the form of perceived friendliness or helpfulness – with humanoid service robots than those that are nonhumanoid, which has important implications for their increasing use in the future.

They found that consumer responses to service failure by robots depend upon the level of a robot’s human characteristics and personality – or anthropomorphisation – as well as the type of service failure and recovery. Their studies also uncovered the role of warmth and competence that underlie customer satisfaction following a service failure and recovery by robots.

Warmth perceptions are particularly important in driving post-failure satisfaction and determining the effectiveness of recovery efforts by both service robots and humans, Prof. Choi says.

He and his colleagues recruited 6,517 people in the U.S. to complete questionnaires involving robots that looked humanoid versus those which didn’t in various imagined scenarios, including service failures in restaurants and hotels.

One of the studies revealed consumers were more dissatisfied with humanoid robots when they encounter a process-related failure – that is, the service is delivered but falls short of expectations in some way. An example of this is inattentive or slow service, and people feel more dissatisfaction when such failure occurs when humanoid robots are involved because they have higher expectations for humanoid robots to show warmth due to their humanlike appearance, and the expectation is unmet.

However, when the service failure is outcome-related, then the differences in consumer responses between humanoid and nonhumanoid robots disappear. Outcome-related failures occur when a basic service need or core service is not performed, such as if a hotel room is unavailable due to overbooking or if an order is delivered to the wrong place, and is typically perceived as being tied to competence rather than the warmth of the service provider.

Two other studies showed humanoid robots could overcome a service failure by themselves by offering a sincere apology and explanation – to restore perceptions of warmth and customer satisfaction.

Yet in contrast, consumers were not receptive to an apology or an explanation by non-humanoid robots because their lack of warmth inhibited expressions of empathy. Another study highlighted the importance of human intervention to mitigate dissatisfaction after a service problem involving a nonhumanoid robot.

Taken together, the research offers theoretical implications for robot anthropomorphism and practical implications for firms employing service robots, Prof. Choi says.

Building a Better Machine

Although robot engineers try to design robots to look and behave similarly to humans so people can interact with robots more intuitively and naturally, the research suggests that the use of robots – be they humanoid or otherwise – should be done with caution, as they each can have disadvantages in certain situations, he says.

“The use of more advanced technologies recognising physical or vocal signs, such as facial expressions or the tone of voice, can help service robots detect a customer’s dissatisfaction and respond to it more attentively,” Prof. Choi says.

Prof. Choi says future research in this field could provide further insights into robots for the service industry by examining how consumer responses to robots vary depending on their physical design characteristics.

One past study found that people perceived male robots as being more proactive and intelligent than female robots, which were regarded as more communal and sociable – raising questions of whether gender stereotypes influence consumer responses to service failures and recovery attempts by service robots.

Another area of study could also investigate whether consumers react to service failure and recovery for robots with an extremely human-like appearance in a similar fashion, or if an “uncanny valley” effect takes hold, referring to the unease that some people feel when they encounter robots that are almost – but not quite – human.

“It’s the natural next step. We now have a better idea of how customers in a service environment respond to both robots that look nothing like and somewhat like humans. It would be interesting to see what the results are the closer we get to perfect mimicry of the human form,” Prof. Choi adds.