As the volume of data generated and processed continues to grow exponentially, the demand for innovative and efficient sorting algorithms has become increasingly critical. Traditional sorting algorithms, while effective in certain scenarios, often struggle with the challenges posed by large-scale datasets, particularly in terms of memory usage and time complexity. This paper evaluates six primary sorting algorithms, identifying key limitations such as high memory consumption in merge sort and inefficiency in bubble sort for large datasets. To address these challenges, we introduce the Proximity-Based Pivot Sort (PBPS), an advanced sorting algorithm designed to optimize performance in real-time numerical data streams and big data applications. The proposed PBPS algorithm leverages proximity-based principles, such as absolute numerical difference, to group and sort similar elements efficiently, reducing unnecessary comparisons and computational overhead. By incorporating dynamic pivot selection (initially using the last element as the pivot, with plans to explore more advanced selection strategies in future work) and adaptive merging strategies, PBPS achieves significant improvements in both time complexity and memory efficiency. Experimental results demonstrate that PBPS outperforms traditional methods, including merge sort, radix sort, heap sort, and quicksort, particularly with datasets that exhibit a high degree of data locality. For instance, PBPS can process up to approximately 2000 sorted elements—twice the number managed by standard quicksort—while reducing execution time by up to 40% and improving memory efficiency by 30%. PBPS outperformed the other algorithms in tests measuring memory usage and execution time, making it a superior choice for handling large-scale datasets. The PBPS algorithm is particularly well-suited for real-time data processing and big data analytics, offering faster insights and streamlined data processing. Its ability to handle large-scale datasets with minimal latency makes it a valuable tool for applications such as financial trading, IoT sensor networks, and real-time analytics. By addressing the limitations of traditional sorting methods, PBPS represents a significant advancement in sorting algorithm design, providing a more efficient solution for modern data-intensive environments.

Big-O notation Big-data Applications Numerical Data Streams Real-Time Data Processing Sorting Algorithms Time Complexity Proximity-Based Pivot Sort