Naba Banerjee has devoted over three years to tackling party "collusion" by users, to recognizing "repeat party houses" and, mostly, to developing an AI system to oppose parties. The AI models are based on hundreds of criteria, such as the reservation's proximity to the user's birthday, the individual's age, the stay's duration, the location's distance to the user's home, whether it is a weekday or weekend, the type of property, and if the place is in a visited area. Between August 2020 and August 2022, Airbnb experienced a 55% decline in reported incidents of partying, and since the worldwide introduction of Banerjee's system in May, it has denied access or diverted more than 320,000 guests.
Naba Banerjee has been an integral part of Airbnb's effort to combat "collusion" among users and "repeat party houses" since joining the company's trust and safety team in May 2020. She has developed an anti-party AI system with sufficient training data to prevent high-risk reservations before checkout. By targeting open-invite gatherings accompanied by excessive noise, trash, visitors, parking issues, and other factors, Airbnb reports that reported parties have decreased by 55%, and over 320,000 guests were blocked from booking attempts since the system's global launch. Despite a post-pandemic surge, the company remains on track as evidenced by last month's earnings.
Airbnb cites the pandemic and fear of property damage as the major motivators behind its recent moves to curtail parties, though there have been darker incidents as well. For example, a Halloween bash at an Airbnb in 2019 ended with five fatalities, while five more people were killed at Airbnbs during Memorial Day and Labor Day weekends this year. Subsequently, the company faced a lawsuit from the family of an 18-year-old boy killed in a shooting at a 2021 Airbnb party.
When Banerjee joined the trust team in the summer of 2020, many around her posed the challenge: “How do you solve this problem?” Her anxiety was compounded as the company wrestled with a multi-faceted dilemma. As the mother of five, Banerjee was highly alert to potential risks related to parties, and used this insight to inform her work. For example, when her 17-year-old daughter’s friend planned an 18th birthday bash at an Airbnb, Banerjee’s warning about the platform’s AI safeguards was heeded.
Banerjee and Airbnb co-founder and chief strategy officer Nate Blecharczyk discussed strategies to address the problem within three timeframes: “right now”, a year, and in the future. To get immediate results, they used existing platform data to identify trends and signals associated with party reports. Following this, in July 2020, the company banned high-risk bookings for users under 25 and “redirected thousands” of guests worldwide. Then, two months later, it implemented a “global party pan”—though it was still largely symbolic.
Banerjee also organized workshops to engage Airbnb data scientists, machine learning engineers, customer support staff, and members of the operations and communications teams. They worked on removing the option for hosts to book out their homes to gatherings of more than 16 people, and created a 24/7 safety line and neighborhood support line for hosts to call.
Banerjee had an objective to construct an AI that was similar to a neighbor keeping an eye on the house when the teenagers are left unsupervised on the weekend. After hearing from Airbnb's Australia offices that disruptive parties at Airbnbs were on the rise due to the travel industry at a standstill and Covid, Banerjee thought about introducing a ban on those under 25 from renting, but then opted to experiment with a machine learning model to prevent parties instead. This left her feeling scared, so she contacted her father in Kolkata who is her biggest supporter, and he encouraged her not to be overconfident. Finally in October 2021, the pilot program was put into place and resulted in a 35% reduction in parties between those who had the program and those who did not. Afterwards, additional data, safety and property damage incidents, and records of user collusion were added to improve and upgrade the system.
As a 21-year-old planning a Halloween party in their hometown, the user runs into a problem: Airbnb's AI system is working against them. This algorithm looks at hundreds of factors from the reservation's closeness to the user's birthday, the user's age, length of stay, location of the listing, proximity to the user's address, whether it is a weekend or week, type of listing, and if the listing is in an overcrowded area or a rural one. These neural networks use deep learning, a subset of machine learning that takes in data and learns by example. One model focuses on the possibility of a party, while another looks at property damage. After the analysis is performed, every reservation is assigned a party risk and based on risk tolerance in the country or area, it will be either allowed or banned. Special evaluations are conducted for holiday weekends, such as the Fourth of July, Halloween, and New Year's Eve.
In some situations where the correct decision is not easily discernible, reservations are put through for human review and the message thread is inspected to determine party hazard. However, Banerjee stated that the organisation is paying "a huge amount" into expansive language models to interpret content and accurately predict party and fraudulent activities. She went on to say that these language models are quickly becoming a necessity, in the same way that the internet is. Banerjee added that her team had recognised higher levels of risk parties in the USA and Canada, followed by Australia and certain European countries, with Asia presenting the least risk. The algorithms are aided by data on tickets marked as parties and property damage, as well as hypothetical and recorded incidents prior to the system going live to see if they are picked up. Additionally, the models are trained on what is seen as “good” guest behaviour, such as punctual check-ins and check-outs, reviews given promptly, and no disturbances on the platform. Nevertheless, when creating these AI training datasets, disparities in the definition of “good” guests can occur. Airbnb has previously implemented experiments to reduce partiality such as obscuring guests’ photographs, preventing hosts from seeing a guest’s full name before booking has been confirmed and introducing a Smart Pricing tool to tackle discrepancies in income, resulting in the gap actually increasing. Airbnb has declared that their reservation-screening AI has been investigated by the company’s anti-discrimination team and is regularly monitored for correctness and recall.
Exactly one year ago, Banerjee happened to be at a nursery with her husband and mother-in-law when she received a call from Airbnb CEO Brian Chesky asking her about trust in the platform. He questioned her as to whether she would feel safe sending one of her college-bound children to stay at an Airbnb, and what would make her feel more secure. Banerjee's response resulted in the decision to launch her team's reservation screening AI around the world the following spring. Preparations were initiated, including TV spots in which Banerjee sought her daughter's advice on what to wear. Next, the team conducted a live demo of the AI with Chesky and afterwards, he suggested changing an existing message to a customer-friendly version. In April 2023, Banerjee visited her family in India for the first time in a year, and the following month, the rollout of the algorithm began. This past Labor Day, the AI blocked or redirected 5,000 potential party bookings. Moving forwards, Banerjee and her team will have to remain vigilant in monitoring and adjusting the systems as users seek new ways to circumvent the barriers. As Banerjee puts it, "The interesting part about the world of trust and safety is that it never stays static. As soon as you build a defense, some of these bad actors out there who are potentially trying to buck the system and throw a party, they will get smarter and they'll try to do something different."
Kommentare