When eBay rolled out an image-based search for shoppers nearly four years ago, it was an early pioneer of using computer vision technology in e-commerce. The company has learned many lessons since then, and it’s now working on bringing innovation to eBay sellers.
In a conversation at VentureBeat’s Transform 2021 virtual conference, eBay’s new chief AI officer, Nitzan Mekel-Bobrov, shared some insights with Maribel Lopez, founder, and principal analyst at Lopez Research. Mekel-Bobrov discussed the challenges of developing image-based search technology, the essential elements of any AI strategy, and what’s next in computer vision.
Data presented some of the hardest obstacles, especially with the novelty of items that eBay is known for.
“The adage that data is king is true, probably nowhere more than in training CD models for item matching,” Mekel-Bobrov said. “And for us, this was compounded at the time with the fact that we don’t control all the ways in which our sellers photograph their items.”
Hardware also posed issues in the first years, Mekel-Bobrov said, and the company had to find creative solutions.
“First, we did our training separately,” he explained. “We trained the models on the public cloud, where we could access [graphic processing units]. Then we did the inference and deployment locally. We also set up a series of what were essentially stateless systems, holding the entire index in memory.”
eBay was also one of the early users of product quantization. That reduced the dimensionality and the size of the problem during a search.
Organizational challenges remain eBay’s biggest undertaking. Launching a product that requires many different teams with “tremendous” upstream and downstream dependencies presents a lot of issues, Mekel-Bobrov said. “A lot of the workflows needed reworking to allow for integration of computer vision capabilities.”
Additionally, e-commerce category taxonomy changes based on the market and inventory. “As the taxonomy innovation progresses, we have to keep up with it on the image search site and continuously adapt and evolve not just our models but our entire approach sometimes,” Mekel-Bobrov said.
The essential elements of AI strategy
Having only been with the company for two months, Mekel-Bobrov said he’s been spending most of his days thinking about eBay’s AI strategy. His plan focuses on three broad categories: taking a customer-first approach, democratizing AI among teams, and nailing down the data infrastructure.
First, the question should never be what new technology can be adopted, but what problems can be solved for customers, Mekel-Bobrov said. “For us, as a two-sided marketplace … the mediation of that relationship, there’s a ton of opportunity and complexity there that AI can really transform,” he said.
The rapidly changing nature of AI also calls for company-wide upskilling and training. “We want to put these technologies into the hands of developers much more broadly than AI specialists, even product managers and analysts,” Mekel-Bobrov said. “The question is, how do we organize ourselves such that we can have a smaller cohort of AI specialists build differentiated core capabilities that can then enable teams across the company to leverage those and build unique products in their own domains?”
Finally, a key facet of any AI strategy is data. Mekel-Bobrov said a company must ask where the data is while offline and streaming, and how it’s collected, labeled, managed, and searched. “The more we’re doing at scale, the more we’re doing with real-time, the more we’re doing with dynamic learning and reinforcement learning, the more we have to push the boundaries of these infrastructure considerations,” Mekel-Bobrov added.
What’s next?
The e-commerce giant is now “turbocharging” to make computer vision an integral part of the experience not just for buyers but for sellers as well, Mekel-Bobrov said.
eBay recently launched an image scanning tool on its app, starting with trading cards that sellers can scan with their phones to auto-populate and create listings.
“If you think about an individual seller or small business that needs to continuously create new listings at a high volume, but wants to maintain the highest quality of information, selling tools based on computer vision can be a real game-changer,” Mekel-Bobrov said. The ultimate goal is to maintain trust between buyers and sellers.
During the COVID-19 pandemic, eBay has used computer vision detection and classification models on items like hand sanitizers and masks to identify listings with inflated prices and products that make false health claims.
Since November 2020, eBay has removed 50 million listings that violated its COVID-19 policies, Mekel-Bobrov said.
“We’re continuously making every effort to ensure that anyone who sells on our platform follows local laws, as well as eBay’s policies,” Mekel-Bobrov said. “Doing that at a scale requires a lot of creative thinking and advanced technologies like computer vision.”