CFP last date
20 August 2026
Reseach Article

Towards Trustworthy Data Pipelines: A Maturity Model for Justified Reliance

by Kyriakos Kartas
International Journal of Computer Applications
Foundation of Computer Science (FCS), NY, USA
Volume 187 - Number 117
Year of Publication: 2026
Authors: Kyriakos Kartas
10.5120/ijcaa08a422f79a1

Kyriakos Kartas . Towards Trustworthy Data Pipelines: A Maturity Model for Justified Reliance. International Journal of Computer Applications. 187, 117 ( Jun 2026), 1-12. DOI=10.5120/ijcaa08a422f79a1

@article{ 10.5120/ijcaa08a422f79a1,
author = { Kyriakos Kartas },
title = { Towards Trustworthy Data Pipelines: A Maturity Model for Justified Reliance },
journal = { International Journal of Computer Applications },
issue_date = { Jun 2026 },
volume = { 187 },
number = { 117 },
month = { Jun },
year = { 2026 },
issn = { 0975-8887 },
pages = { 1-12 },
numpages = {9},
url = { https://ijcaonline.org/archives/volume187/number117/towards-trustworthy-data-pipelines-a-maturity-model-for-justified-reliance/ },
doi = { 10.5120/ijcaa08a422f79a1 },
publisher = {Foundation of Computer Science (FCS), NY, USA},
address = {New York, USA}
}
%0 Journal Article
%1 2026-07-01T03:09:49.977065+05:30
%A Kyriakos Kartas
%T Towards Trustworthy Data Pipelines: A Maturity Model for Justified Reliance
%J International Journal of Computer Applications
%@ 0975-8887
%V 187
%N 117
%P 1-12
%D 2026
%I Foundation of Computer Science (FCS), NY, USA
Abstract

Data pipelines now mediate dashboards, experimentation, reporting, and machine learning, yet their trustworthiness cannot be inferred from uptime or point estimates of data quality alone. Pipelines may continue to run while semantics drift, derivation becomes opaque, change history weakens, or access rights detach from responsibility. Relevant control domains are addressed across adjacent literatures on data quality, provenance, governance, observability, reliability, reproducibility, metadata, security, and DataOps, but those literatures typically stop at their own boundaries. What remains underdeveloped is a pipeline-specific conceptual framework that integrates these domains around a defensible answer to a single question: when is reliance on pipeline outputs justified? This article develops such a framework as a conceptual maturity model. Following a theory-synthesis approach, purposive literature assembly, and explicit maturity-model design guidance, the paper defines pipeline trustworthiness as the extent to which a pipeline can be justifiably relied upon because its outputs, behavior, and control arrangements remain intelligible, dependable, governable, reproducible, and auditable over time. On that basis, it retains eight dimensions of trustworthy pipeline maturity: data quality assurance, observability and monitoring, reliability and fault tolerance, lineage and traceability, governance and ownership, reproducibility and change management, metadata and documentation, and security and access control. It further specifies five heuristic maturity levels—Ad Hoc, Repeatable, Managed, Controlled, and Trustworthy–Optimized—and a profile-first interpretation in which overall maturity is constrained by the weakest dimension rather than estimated through a compensatory average. The contribution does not lie in claiming that the individual dimensions are unprecedented in isolation. It lies in integrating them into a pipeline-specific architecture centered on justified reliance and in specifying a non-compensatory, bottleneck-aware maturity logic. The result is a conceptual framework for distinguishing trustworthy pipeline maturity from governance-only, DataOps-only, provenance-centered, data-mesh, and generic enterprise capability approaches, while offering a more operationally inspectable basis for future empirical work. Its present payoff is conceptual as much as preparatory: it explains why looser, governance-centric, or compensatory architectures misclassify unevenly developed pipelines even before empirical calibration begins.

References
  1. Munappy AR, Bosch J, Holmstrom Olsson H. Data Pipeline Management in Practice: Challenges and Opportunities. In: Product-Focused Software Process Improvement. LNCS 12562. Cham: Springer; 2020. p. 168–184. https://doi. org/10.1007/978-3-030-64148-1_11
  2. Foidl H, Golendukhina V, Ramler R, Felderer M. Data pipeline quality: Influencing factors, root causes of data-related issues, and processing problem areas for developers. Journal of Systems and Software. 2024;207:111855. https://doi.org/10.1016/j. jss.2023.111855
  3. Simmhan Y, van Ingen C, Szalay A, Barga R, Heasley J. Building Reliable Data Pipelines for Managing Community Data Using ScientificWorkflows. In: Fifth IEEE International Conference on e-Science; 2009. p. 321–328. https://doi. org/10.1109/e-Science.2009.52
  4. Wang RY, Strong DM. Beyond accuracy: What data quality means to data consumers. Journal of Management Information Systems. 1996;12(4):5–33. https://doi.org/ 10.1080/07421222.1996.11518099
  5. Wand Y, Wang RY. Anchoring data quality dimensions in ontological foundations. Communications of the ACM. 1996;39(11):86–95. https://doi.org/10.1145/2404 55.240479
  6. Batini C, Cappiello C, Francalanci C, Maurino A. Methodologies for Data Quality Assessment and Improvement. ACM Computing Surveys. 2009;41(3):16:1–52. https://doi.org/10.1145/1541880.1541883
  7. International Organization for Standardization. ISO/IEC 25012:2008 Software engineering — Software product Quality Requirements and Evaluation (SQuaRE) — Data quality model. Geneva: ISO; 2008.
  8. Buneman P, Khanna S, Tan WC. Why and Where: A Characterization of Data Provenance. In: Database Theory – ICDT 2001. Berlin: Springer; 2001. p. 316–330. https: //doi.org/10.1007/3-540-44503-X_20
  9. Herschel M, Diestelkaemper R, Ben Lahmar H. A survey on provenance: What for? What form? What from? The VLDB Journal. 2017;26(6):881–906. https://doi.org/10.100 7/s00778-017-0486-1
  10. Simmhan YL, Plale B, Gannon D. A survey of data provenance in e-science. SIGMOD Record. 2005;34(3):31–36. https://doi.org/10.1145/10 84805.1084812
  11. Rupprecht L, Davis JC, Arnold C, Gur Y, Bhagwat D. Improving Reproducibility of Data Science Pipelines through Transparent Provenance Capture. Proceedings of the VLDB Endowment. 2020;13(12):3354–3368. https://doi.org/ 10.14778/3415478.3415556
  12. Khatri V, Brown CV. Designing data governance. Communications of the ACM. 2010;53(1):148–152. https://doi.org/10.1145/1629175.1629210
  13. Abraham R, Schneider J, vom Brocke J. Data governance: A conceptual framework, structured review, and research agenda. International Journal of Information Management. 2019;49:424–438. https://doi.org/10.1016/j.ijin fomgt.2019.07.008
  14. Alhassan I, Sammon D, Daly M. Data governance activities: an analysis of the literature. Journal of Decision Systems. 2016;25(sup1):64–75. https://doi.org/10.1080/1246 0125.2016.1187397
  15. Jahnke N, Otto B. Data Catalogs in the Enterprise: Applications and Integration. Datenbank-Spektrum. 2023;23:89–96. https://doi.org/10.1007/s132 22-023-00445-2
  16. Ross R, Pillitteri V, Dempsey K, Riddle M, Guissanie G. Security and Privacy Controls for Information Systems and Organizations. NIST Special Publication 800-53 Revision 5. Gaithersburg (MD): National Institute of Standards and Technology; 2020. https://doi.org/10.6028/NIST.S P.800-53r5
  17. Sandhu RS, Coyne EJ, Feinstein HL, Youman CE. Role-based access control models. Computer. 1996;29(2):38–47. https: //doi.org/10.1109/2.485845
  18. Hu VC, Ferraiolo DF, Kuhn DR, Friedman AR, Lang AJ, Schnitzer MM, Sandlin K, Miller R, Scarfone K. Guide to Attribute Based Access Control (ABAC) Definition and Considerations. NIST Special Publication 800-162. Gaithersburg (MD): National Institute of Standards and Technology; 2014. https://doi.org/10.6028/NIST.S P.800-162
  19. Mace J, Roelke R, Fonseca R. Pivot Tracing: Dynamic Causal Monitoring for Distributed Systems. ACM Transactions on Computer Systems. 2018;35(4):1–28. https://doi.org/ 10.1145/3208104
  20. Li Z, Chen J, Jiao R, Zhao N, Wang Z, Zhang S, Wu Y, Jiang L, Yan L, Wang Z, Chen Z, Zhang W, Nie X, Sui K, Pei D. Practical Root Cause Localization for Microservice Systems via Trace Analysis. In: IEEE/ACM International Symposium on Quality of Service; 2021. p. 1–10. https://doi.org/10 .1109/IWQOS52092.2021.9521340
  21. Munappy AR, Bosch J, Holmstrom Olsson H, Wang TJ. Towards automated detection of data pipeline faults. In: Asia-Pacific Software Engineering Conference; 2020. p. 346–355. https://doi.org/10.1109/APSEC51365.2 020.00043
  22. Polyzotis N, Roy S, Whang SE, Zinkevich M. Data Management Challenges in Production Machine Learning. In: Proceedings of the 2017 ACM SIGMOD International Conference on Management of Data. New York: ACM; 2017. p. 1723–1726. https://doi.org/10.1145/3035918.30 54782
  23. Munappy AR, Mattos DI, Bosch J, Holmstrom Olsson H, Dakkak A. From Ad-Hoc Data Analytics to DataOps. In: International Conference on Software and System Processes. New York: ACM; 2020. p. 165–174. https://doi.org/10 .1145/3379177.3388909
  24. DAMA International. DAMA-DMBOK: Data Management Body of Knowledge. 2nd ed. Basking Ridge (NJ): Technics Publications; 2017.
  25. de Bruin T, Freeze R, Kulkarni U, Rosemann M. Understanding the Main Phases of Developing a Maturity Assessment Model. In: ACIS 2005 Proceedings; 2005. Paper 109. https://aisel.aisnet.org/acis2005/109/
  26. Becker J, Knackstedt R, Poeppelbuss J. Developing Maturity Models for IT Management – A Procedure Model and its Application. Business and Information Systems Engineering. 2009;1(3):213–222. https://doi.org/10.1007/s12599 -009-0044-5
  27. Poeppelbuss J, Niehaves B, Simons A, Becker J. Maturity Models in Information Systems Research: Literature Search and Analysis. Communications of the Association for Information Systems. 2011;29:505–532. https://doi.or g/10.17705/1CAIS.02927
  28. Jaakkola E. Designing conceptual articles: four approaches. AMS Review. 2020;10:18–26. https://doi.org/10.100 7/s13162-020-00161-0
  29. Dehghani Z. Data Mesh: Delivering Data-Driven Value at Scale. Sebastopol (CA): O’Reilly Media; 2022.
Index Terms

Computer Science
Information Sciences

Keywords

Data pipelines; trustworthiness; maturity model; justified reliance; observability; governance; data lineage; reproducibility; metadata management; accountability