According to a study from Dartmouth College, Dartmouth’s Tuck School of Business, and Indiana University, artificial intelligence systems can be trained to produce human-like product reviews that help consumers, marketers, and professional reviewers.
The study, which was published in the International Journal of Research in Marketing, also recognizes the ethical issues that computer-generated content raises.
“Review writing is challenging for humans and computers, in part, because of the overwhelming number of distinct products,” said Keith Carlson, a doctoral research fellow at the Tuck School of Business. “We wanted to see how artificial intelligence can be used to help people that produce and use these reviews.”
The Dartmouth researchers devised two challenges for the study. The first was to see if, after being trained on a collection of existing content, a machine could be taught to produce fresh, human-quality reviews using only a small number of product features. Second, the team wanted to test if machine learning techniques might be used to create syntheses of reviews for products that already have a large number of reviews.
“Using artificial intelligence to write and synthesize reviews can create efficiencies on both sides of the marketplace,” said Prasad Vana, assistant professor of business administration at Tuck School of Business. “The hope is that AI can benefit reviewers facing larger writing workloads and consumers that have to sort through so much content about products.”
Because there was so much material to train the computer algorithms on, the researchers concentrated on wine and beer reviews. These items’ descriptions also have reasonably focused vocabularies, which is beneficial when working with AI systems.
The researchers used around 180,000 existing wine ratings to train an algorithm to see if it could produce useful reviews from scratch. The machine-learning algorithm was also trained using metadata tags for parameters including product origin, grape variety, rating, and pricing.
When the research team compared machine-generated reviews to human reviews for identical wines, they discovered that the two versions were in accord. Even when the researchers tested the algorithms by adjusting the amount of input data available for reference, the results remained consistent.
After proving that artificial intelligence can write genuine wine evaluations, the researchers turned to beer reviews to see how well AI could produce “review syntheses.” The algorithm was charged with gathering aspects from previous evaluations of the same product, rather than being trained to generate fresh reviews. Based on a significant volume of varied perspectives, AI’s capacity to detect and give limited but important information about products was put to the test.
“Writing original review tests the computer’s expressive ability based on a relatively narrow set of data. Writing a synthesis review is a related but distinct task where the system is expected to produce a review that captures some of the key ideas present in an existing set of reviews for a product,” Carlson, a Ph.D. candidate in computer science at Dartmouth, conducted the research.
Researchers trained the system on 143,000 existing reviews of over 14,000 beers to see if it could generate review synthesis. The text of each review was matched with metadata such as the product name, alcohol concentration, style, and the original reviewers’ scores, just as in the wine dataset.
The study utilized independent study participants to assess whether the machine-written summaries gathered and summarized the viewpoints of numerous evaluations in a helpful, human-like manner, similar to the wine reviews.
The model, according to the paper, was successful in accepting product evaluations as input and producing a synthesis review for that product as output.
“Our modeling framework could be useful in any situation where detailed attributes of a product are available and a written summary of the product is required,” said Vana. “It’s interesting to imagine how this could benefit restaurants that cannot afford sommeliers or independent sellers on online platforms who may sell hundreds of products.”
To ingest, process, and output review language in both challenges, a deep learning neural net based on transformer architecture was deployed.
The computer systems, according to the research team, are not meant to replace expert writers and marketers, but rather to aid them in their work. For example, a machine-written review may be used as a first draft of a review that a human reviewer could subsequently revise.
Consumers may benefit from the research as well. Syntheses reviews, like the ones used in the study, can be used for a wide range of products and services in online marketplaces to help individuals who don’t have time to browse through a lot of product reviews.
Aside from the advantages of machine-written evaluations, the research team points out some of the ethical issues that come with utilizing computer algorithms to affect human customer behavior.
The team calls for openness when computer-generated reviews are presented, noting that marketers might improve acceptance of machine-generated reviews by fraudulently attributing them to humans.
“As with other technology, we have to be cautious about how this advancement is used,” said Carlson. “If used responsibly, AI-generated reviews can be both a productivity tool and can support the availability of useful consumer information.”
More Information: Keith Carlson et al, Complementing human effort in online reviews: A deep learning approach to automatic content generation and review synthesis, International Journal of Research in Marketing (2022). DOI: 10.1016/j.ijresmar.2022.02.004