Dl/DataApplication: Difference between revisions
From stonehomewiki
Jump to navigationJump to search
Stonezhong (talk | contribs) (Created page with "<div class="toccolours mw-collapsible mw-collapsed expandable"> <div class="mw-collapsible-preview">Brief</div> <div class="mw-collapsible-content"> </div> </div> <p></p>") |
Stonezhong (talk | contribs) No edit summary |
||
| (2 intermediate revisions by the same user not shown) | |||
| Line 1: | Line 1: | ||
<p> [[dl/home|Data Lake Knowledge Center]] </p> | |||
<div class="toccolours mw-collapsible mw-collapsed expandable"> | <div class="toccolours mw-collapsible mw-collapsed expandable"> | ||
<div class="mw-collapsible-preview"> | <div class="mw-collapsible-preview">Data Application: for data ingestion</div> | ||
<div class="mw-collapsible-content"> | <div class="mw-collapsible-content"> | ||
* An application that generates dataframes and update dataset. | |||
</div> | |||
</div> | |||
<p></p> | |||
<div class="toccolours mw-collapsible mw-collapsed expandable"> | |||
<div class="mw-collapsible-preview">Data Application: for data transformation</div> | |||
<div class="mw-collapsible-content"> | |||
* Consume one or multiple dataset and generates dataframes and update datasets | |||
</div> | |||
</div> | |||
<p></p> | |||
<div class="toccolours mw-collapsible mw-collapsed expandable"> | |||
<div class="mw-collapsible-preview">Data Pipeline</div> | |||
<div class="mw-collapsible-content"> | |||
* Bunch of data application that has dependency | |||
* Runs on a regular basis | |||
It is a good idea to using Apache Airflow to orchestrate data pipeline | |||
</div> | </div> | ||
</div> | </div> | ||
<p></p> | <p></p> | ||
