Comparison of aggregation methods used in online reviews: a critical analysis

Maria Grazia Olivieri, Elisa Fiorenza

Abstract


With digitalization, online reviews have become a crucial component of consumer decisions and are an important source of feedback on products and services. Users can make informed choices using online review systems such as e-commerce platforms, social media, and others. However, the collection and analysis of these reviews raises a number of questions regarding reliability, representativeness and accuracy. Since they directly influence users' choices, the methods used to aggregate review scores are extremely important. In this article, we explore the most common aggregation methods for rating online reviews on the Amazon platform, with a particular focus on the arithmetic mean. Although these methods are easy to implement, they do not always accurately reflect the overall quality of a product. We then critically analyze how more sophisticated approaches can minimize some of the limitations of the mean, such as the so-called recent bias; propose more robust solutions and alternative approaches that can provide a more accurate representation of the overall rating, improving the quality of aggregated ratings.


Keywords


aggregation methods; arithmetic mean; online reviews; alternative methods; recent bias.

Full Text:

PDF

References


Abdel-Hafez, A., Xu, Y., & Jøsang, A. (2014, September). A rating aggregation method for generating product reputations. In Proceedings of the 25th ACM conference on Hypertext and social media (pp. 291-293).

Agarwal, B., Mittal, N., Bansal, P., & Garg, S. (2015). Sentiment analysis using common‐sense and context information. Computational intelligence and neuroscience, 2015(1), 715730.

Bączkiewicz, A., Kizielewicz, B., Shekhovtsov, A., Wątróbski, J., & Sałabun, W. (2021). Methodical aspects of MCDM based E-commerce recommender system. Journal of Theoretical and Applied Electronic Commerce Research, 16(6), 2192-2229.

Benlahbib, A. (2020). Aggregating customer review attributes for online reputation generation. IEEE Access, 8, 96550-96564.

Chevalier, J., & Goolsbee, A. (2003). Measuring prices and price competition online: Amazon. com and BarnesandNoble. com. Quantitative marketing and Economics, 1, 203-222.

Dahooie, J. H., Raafat, R., Qorbani, A. R., & Daim, T. (2021). An intuitionistic fuzzy data-driven product ranking model using sentiment analysis and multi-criteria decision-making. Technological Forecasting and Social Change, 173, 121158.

Decker, R., & Trusov, M. (2010). Estimating aggregate consumer preferences from online product reviews. International Journal of Research in Marketing, 27(4), 293-307.

Dellarocas, C., & Wood, C. A. (2008). The sound of silence in online feedback: Estimating trading risks in the presence of reporting bias. Management science, 54(3), 460-476.

Duan, W., Gu, B., & Whinston, A. B. (2008). Do online reviews matter?—An empirical investigation of panel data. Decision support systems, 45(4), 1007-1016.

Dwork, C., Kumar, R., Naor, M., & Sivakumar, D. (2001, April). Rank aggregation methods for the web. In Proceedings of the 10th international conference on World Wide Web (pp. 613-622).

Eryarsoy, E., & Piramuthu, S. (2014). Experimental evaluation of sequential bias in online customer reviews. Information & Management, 51(8), 964-971.

Ganu, G., Elhadad, N., & Marian, A. (2009, June). Beyond the stars: Improving rating predictions using review text content. In WebDB (Vol. 9, pp. 1-6).

Ge, Y., & Li, J. (2015). Measure and Mitigate the Dimensional Bias in Online Reviews and Ratings. In ICIS.

Ghose, A., & Ipeirotis, P. G. (2006, December). Designing ranking systems for consumer reviews: The impact of review subjectivity on product sales and review quality. In Proceedings of the 16th annual workshop on information technology and systems (pp. 303-310).

Gionis, A., Mannila, H., & Tsaparas, P. (2007). Clustering aggregation. Acm transactions on knowledge discovery from data (tkdd), 1(1), 4-es.

Han, S., & Anderson, C. K. (2020). Customer motivation and response bias in online reviews. Cornell Hospitality Quarterly, 61(2), 142-153.

Hellerstein, J. M., Haas, P. J., & Wang, H. J. (1997, June). Online aggregation. In Proceedings of the 1997 ACM SIGMOD international conference on Management of data (pp. 171-182).

Ikotun, A. M., Ezugwu, A. E., Abualigah, L., Abuhaija, B., & Heming, J. (2023). K-means clustering algorithms: A comprehensive review, variants analysis, and advances in the era of big data. Information Sciences, 622, 178-210.

Ivanova, O., & Scholz, M. (2017). How can online marketplaces reduce rating manipulation? A new approach on dynamic aggregation of online ratings. Decision Support Systems, 104, 64-78.

Jamadi Khiabani, P., Basiri, M. E., & Rastegari, H. (2020). An improved evidence-based aggregation method for sentiment analysis. Journal of Information Science, 46(3), 340-360.

Ji, F., Cao, Q., Li, H., Fujita, H., Liang, C., & Wu, J. (2023). An online reviews-driven large-scale group decision making approach for evaluating user satisfaction of sharing accommodation. Expert Systems with Applications, 213, 118875.

Jindal, N., & Liu, B. (2008, February). Opinion spam and analysis. In Proceedings of the 2008 international conference on web search and data mining (pp. 219-230).

Karaman, H. (2021). Online review solicitations reduce extremity bias in online review distributions and increase their representativeness. Management Science, 67(7), 4420-4445.

Lakshmanaprabu, S. K., Shankar, K., Gupta, D., Khanna, A., Rodrigues, J. J., Pinheiro, P. R., & de Albuquerque, V. H. C. (2018). Ranking analysis for online customer reviews of products using opinion mining with clustering. Complexity, 2018(1), 3569351.

Lin, S. (2010). Rank aggregation methods. Wiley Interdisciplinary Reviews: Computational Statistics, 2(5), 555-570.

Liu, B., & Zhang, L. (2012). A survey of opinion mining and sentiment analysis. In Mining text data (pp. 415-463). Springer, Boston, MA.

Liu, Y., Bi, J. W., & Fan, Z. P. (2017). Ranking products through online reviews: A method based on sentiment analysis technique and intuitionistic fuzzy set theory. Information Fusion, 36, 149-161.

Liu, B. (2022). Sentiment analysis and opinion mining. Springer Nature.

Mackiewicz, J. (2008). Reviewer motivations, bias, and credibility in online reviews. In Handbook of research on computer mediated communication (pp. 252-266). IGI Global.

Orlikowski, W. J., & Scott, S. V. (2014). What happens when evaluation goes online? Exploring apparatuses of valuation in the travel sector. Organization Science, 25(3), 868-891.

Pourghebleh, B., & Navimipour, N. J. (2017). Data aggregation mechanisms in the Internet of things: A systematic review of the literature and recommendations for future research. Journal of Network and Computer Applications, 97, 23-34.

Preethi, J. S., Abirami, A. M., Askarunisa, A., Priya, G. S., & Sankaragomathy, E. (2015). Applying MCDM techniques for ranking products based on online customer feedback. Int. J. Knowl. Based Comput. Syst, 3, 13-20.

Viviani, M., & Pasi, G. (2017). Quantifier guided aggregation for the veracity assessment of online reviews. International Journal of Intelligent Systems, 32(5), 481-501.




DOI: http://dx.doi.org/10.23755/rm.v54i0.1674

Refbacks

  • There are currently no refbacks.


Copyright (c) 2025 Elisa Fiorenza

Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.

Ratio Mathematica - Journal of Mathematics, Statistics, and Applications. ISSN 1592-7415; e-ISSN 2282-8214.