A Framework for the Design, Development, Testing and Deployment of Reliable Big Data Platforms
- Publisher:
- Institute of Electrical and Electronics Engineers (IEEE)
- Publication Type:
- Conference Proceeding
- Citation:
- Proceedings - 2022 IEEE International Conference on Big Data, Big Data 2022, 2022, 00, pp. 2660-2666
- Issue Date:
- 2022-01-01
Embargoed
| Filename | Description | Size | |||
|---|---|---|---|---|---|
| 2022 IEEE Big Data Methods - Camera Ready V1.pdf | Accepted version | 287.05 kB |
Copyright Clearance Process
- Recently Added
- In Progress
- Open Access
This item is currently unavailable due to the publisher's embargo.
We consider the problem of reliability in big data science projects that are comprised of multiple computing platforms and complex architectures that harness data. Specifically on their ability to capture, process and analyze streaming high frequency data from vast complex systems reliably with effective scalability for deployment in vast domains such as clinical care, smart cities or within extreme climatic work environments. This paper introduces a framework to enable reliable data science projects by integrating multiple computing principles of autonomy, local responsibility, fault tolerance, symmetry, decentralization, well-understood building blocks, and simplicity. The designed framework is applied in the development of a decoupled data pipeline demonstrated through a case study on pre-deployment acclimation strategies that is continuously monitored to ensure reliability and availability is effectively quantified.
Please use this identifier to cite or link to this item: