CFP last date
21 July 2025
Reseach Article

Adaptive Encoding for Scalable Segment Storage in Advertising and Recommendation Systems

by Xingpeng Xiao, Pengfei Gao
International Journal of Computer Applications
Foundation of Computer Science (FCS), NY, USA
Volume 187 - Number 19
Year of Publication: 2025
Authors: Xingpeng Xiao, Pengfei Gao
10.5120/ijca2025925275

Xingpeng Xiao, Pengfei Gao . Adaptive Encoding for Scalable Segment Storage in Advertising and Recommendation Systems. International Journal of Computer Applications. 187, 19 ( Jul 2025), 34-37. DOI=10.5120/ijca2025925275

@article{ 10.5120/ijca2025925275,
author = { Xingpeng Xiao, Pengfei Gao },
title = { Adaptive Encoding for Scalable Segment Storage in Advertising and Recommendation Systems },
journal = { International Journal of Computer Applications },
issue_date = { Jul 2025 },
volume = { 187 },
number = { 19 },
month = { Jul },
year = { 2025 },
issn = { 0975-8887 },
pages = { 34-37 },
numpages = {9},
url = { https://ijcaonline.org/archives/volume187/number19/adaptive-encoding-for-scalable-segment-storage-in-advertising-and-recommendation-systems/ },
doi = { 10.5120/ijca2025925275 },
publisher = {Foundation of Computer Science (FCS), NY, USA},
address = {New York, USA}
}
%0 Journal Article
%1 2025-07-09T01:07:44+05:30
%A Xingpeng Xiao
%A Pengfei Gao
%T Adaptive Encoding for Scalable Segment Storage in Advertising and Recommendation Systems
%J International Journal of Computer Applications
%@ 0975-8887
%V 187
%N 19
%P 34-37
%D 2025
%I Foundation of Computer Science (FCS), NY, USA
Abstract

Effective segment storage is an integral infrastructure problem in recommendation systems and targeted advertising systems. These systems need to store and retrieve enormous numbers of user-segment relationships on-the-fly in real-time while keeping latency low and storage overhead minimal. Conventional segment storage solutions have high storage overhead and low query performance, particularly as the number of user accounts and segments grows. In this work, we introduce an adaptive storage system for segment stores that automatically chooses among storing in an array form versus bitmaps versus run-length encoding depending on each user's segment listing's sparsity or density. Through optimized threshold computation, the system can automatically choose the most space-effective storage mechanism for each user segment listing without having to compare all compression options. Our approach is shown through experimentation to save significant storage space while also optimizing segment retrieval operations at a low latency. The new methods are well-suited for large-scale real-time recommendation engines and targeted advertising systems as well as for large-scale streaming services. A preliminary version of this encoding model was published as a U.S. patent [1].

References
  1. X. Xiao, Y. Shi, et al., “Efficient Storage for Segment Store,” U.S. Patent US20240403919A1, Apr. 2024. [Online]. Available: https://patents.google.com/patent/US20240403919A1/en
  2. A. Agarwal, R. Agrawal, P. Jain, V. Kashyap, R. Khandekar, and M. Sabharwal, “Scaling personalized advertising on LinkedIn,” in Proc. 23rd ACM SIGKDD Int. Conf. on Knowledge Discovery and Data Mining (KDD), 2017, pp. 1953–1961.
  3. B. Schwartz, E. Weinstein, and A. Roy-Chowdhury, “FLEDGE: Privacy-preserving interest-based advertising,” Google Ads Developer Blog, 2022. [Online]. Available: https://developer.chrome.com/en/docs/privacy-sandbox/fledge/
  4. K. Wu, E. Otoo, and A. Shoshani, “Compressing bitmap indexes for faster search operations,” In Proc. SSDBM, 2006, pp. 99–108.
  5. D. Lemire, O. Kaser, and K. Aouiche, “Compressed bitmap indexes: Beyond unions and intersections,” Software: Practice and Experience, vol. 40, no. 2, pp. 131–147, 2010.
  6. S. Chambi, D. Lemire, O. Kaser, and R. Godin, "Better bitmap performance with Roaring bitmaps," Software: Practice and Experience, vol. 46, no. 5, pp. 709–719, 2016.
  7. D. Lemire, L. Boytsov, and N. Kurz, "Roaring bitmaps: Implementation of an optimized compressed bitmap index," Software: Practice and Experience, vol. 48, no. 4, pp. 867–886, 2018.
  8. A. Colantonio and R. Di Pietro, “Concise: Compressed 'n' composable integer set,” Information Systems, vol. 38, no. 8, pp. 1084–1097, 2013.
  9. M. Zukowski, S. Heman, N. Nes, and P. Boncz, “Super-scalar RAM-CPU cache compression,” in Proc. IEEE Int. Conf. on Data Engineering (ICDE), 2006, pp. 59–70.
  10. D. Abadi, D. Carney, U. Çetintemel, M. Cherniack, C. Convey, S. Lee, M. Stonebraker, N. Tatbul, and S. Zdonik, “Aurora: a new model and architecture for data stream management,” The VLDB Journal, vol. 12, no. 2, pp. 120–139, 2003.
  11. P. Covington, J. Adams, and E. Sargin, “Deep neural networks for YouTube recommendations,” in Proceedings of the 10th ACM Conference on Recommender Systems (RecSys), 2016, pp. 191–198.
  12. A. Criminisi, J. Shotton, and E. Konukoglu, “Decision forests: A unified framework for classification, regression, density estimation, manifold learning and semi-supervised learning,” Foundations and Trends in Computer Graphics and Vision, vol. 7, no. 2–3, pp. 81–227, 2012.
Index Terms

Computer Science
Information Sciences

Keywords

Segment store adaptive encoding sparsity personalization advertising