Dl/DataApplication: Difference between revisions
From stonehomewiki
Jump to navigationJump to search
Stonezhong (talk | contribs) No edit summary |
Stonezhong (talk | contribs) No edit summary |
||
| (One intermediate revision by the same user not shown) | |||
| Line 1: | Line 1: | ||
<p> [[dl/home|Data Lake Knowledge Center]] </p> | |||
<div class="toccolours mw-collapsible mw-collapsed expandable"> | <div class="toccolours mw-collapsible mw-collapsed expandable"> | ||
<div class="mw-collapsible-preview">Data Application: for data ingestion</div> | <div class="mw-collapsible-preview">Data Application: for data ingestion</div> | ||
| Line 11: | Line 13: | ||
<div class="mw-collapsible-content"> | <div class="mw-collapsible-content"> | ||
* Consume one or multiple dataset and generates dataframes and update datasets | * Consume one or multiple dataset and generates dataframes and update datasets | ||
</div> | |||
</div> | |||
<p></p> | |||
<div class="toccolours mw-collapsible mw-collapsed expandable"> | |||
<div class="mw-collapsible-preview">Data Pipeline</div> | |||
<div class="mw-collapsible-content"> | |||
* Bunch of data application that has dependency | |||
* Runs on a regular basis | |||
It is a good idea to using Apache Airflow to orchestrate data pipeline | |||
</div> | </div> | ||
</div> | </div> | ||
<p></p> | <p></p> | ||
