Question
Data lakes can include at-rest and streaming data because these two data sources often need to be combined. By evaluating the tools within a big
Data lakes can include at-rest and streaming data because these two data sources often need to be combined. By evaluating the tools within a big data ecosystem, the combination of these can be used.
For this assignment, you will utilize data from both concepts to gather incremental changes as they occur. While the data lake will not be populated, the architecture and design will consume and operate so that the destination could be many sources, including a data lake.
The project deliverables include the following:
Consume data from this link.
Consume the data locally on your PC using Python.
Construct a query algorithm that produces an incremental number of changes in real time. The Python code will require the use of a streaming library of your choosing.
Step by Step Solution
There are 3 Steps involved in it
Step: 1
Get Instant Access to Expert-Tailored Solutions
See step-by-step solutions with expert insights and AI powered tools for academic success
Step: 2
Step: 3
Ace Your Homework with AI
Get the answers you need in no time with our AI-driven, step-by-step assistance
Get Started