In today’s distributed computing environment, continuous integration and delivery (CI/CD) can be challenging given the multitude of dependencies that have to managed and replicated. In this post, we are going to develop a continuous integration workflow using Docker.
Data pipelines are common for any businesses that work with large amount of data. It is a crucial piece of infrastructure that fetches data from its source, transforms it and stores it for internal use. In this post we will build a distributed data pipeline using core services in Amazon Web Services (AWS).