Dl/glossary: Difference between revisions

Revision as of 23:04, 5 March 2023

Dataframe

A basic data unit that carries data. For example:

Dataframe should be immutable once it is published -- the data it carries should never change.

Asset

Represent underneath data a dataframe is loaded from. For example:

Asset URI

A URI that uniquely identifies an asset, for example:

s3://bucket_name/foo.parquet -- represent a parquet file stored in AWS S3
mysql://myserver/mydb/foo -- represent a table in MySQL, server name is myserver, dbname is mydb, table name is foo
mysql://myserver/mydb/foo/?batch_id=1& -- represent a table in MySQL, server name is myserver, dbname is mydb, table name is foo, with a filter, which batch_id column need to match 1

Dataset

bbb

@@ Line 36: / Line 36: @@
 * <code>s3://bucket_name/foo.parquet</code>       -- represent a parquet file stored in AWS S3
 * <code>mysql://myserver/mydb/foo</code>          -- represent a table in MySQL, server name is myserver, dbname is mydb, table name is foo
-* <code>mysql://myserver/mydb/foo</?batch_id=1&</code> -- represent a table in MySQL, server name is myserver, dbname is mydb, table name is foo, with a filter, which batch_id column need to match 1
+* <code>mysql://myserver/mydb/foo/?batch_id=1&</code> -- represent a table in MySQL, server name is myserver, dbname is mydb, table name is foo, with a filter, which batch_id column need to match 1
 </div>
 </div>