Relational Databases Collection of data organized into a table structure where the tables can be linked, or related, based on data common to each. Build on the organizational principles of flat files such as spreadsheets, with data organized into r...
These questions have been curated based on the interviews of various data Professionals in the industry as stated in the IBM Data engineering specialization. They will be helpful to anyone attending interviews for DE roles, it also gives a glance to ...
The ecosystem exists for: Extracting data from disparate sources Architecting and managing data pipelines for transformation, integration, and storage Architecting and managing data repositories Automating and optimizing workflows and flow of data ...
Data Warehouses A data warehouse is a central repository of data integrated from multiple sources. It serves as a single source of truth that has data that is cleansed, conformed, and categorized. Data loaded in the warehouse is already modeled and ...
What is the type of data - Structured, Unstructured, semi- Structured? The schema of the data What are its performance requirements Whether you're working with data at rest or streaming data. Data Encryption needs Volume of data and whether you nee...
Data wrangling or Data Munging is a process that involves: data Exploration Transformation Validation Making Data available for credible and meaningful analysis Structuring data: The task includes actions that change the form and schema of your ...
V's of Big Data : Velocity: Velocity is the speed at which data accumulates. Data is being generated extremely fast, in a process that never stops. Near or real-time streaming, local, and cloud-based technologies can process information very quickl...
Architecting the Data Platform Layers of a data platform architecture, A layer represents functional components that perform a specific set of tasks in the data Data Ingestion or Data Collection layer Data Storage and Integration Layer Data Proces...
ETL has historically been used for batch workloads on a large scale. They're being used for real-time streaming as well. popular ETL tools: IBM Infosphere information server, AWS Glue, Improvado, Skyvia, Hevo, Informatica. ETL ( Extract Load Transfo...