Dl/glossary

Data Lake Knowledge Center

Dataframe

A basic data unit that carries data. For example:

• A dataframe in Apache Spark
• A dataframe in Pandas

Dataframe should be immutable once it is published -- the data it carries should never change.

Asset

Represent underneath data a dataframe is loaded from. For example:

• A parquet file in AWS S3 bucket
• A parquet file in HDFS
• A parquet file in local filesystem
• A table in DB (RDBMS)
• A subset of a table in DB (RDBMS) -- via a filter
• A collection in mongodb
• A subset of a collection in mongodb -- via a filter

Dataset

bbb