View source for Firebird/Concepts

<p>[[Firebird]]</p>

= Overview =
<div class="toccolours mw-collapsible mw-collapsed expandable">
<div class="mw-collapsible-preview">pipeline</div>
<div class="mw-collapsible-content">
[[Image:Pipeline.png]]

* A Pipe is a data processing unit, it may have zero or more input ports, and may have zero or more output ports
* A Generator is a Pipe that does not have any input ports
* A Sink is a Pipe that does not have any output ports
</div>
</div>
<p></p>

<div class="toccolours mw-collapsible mw-collapsed expandable">
<div class="mw-collapsible-preview">wiring</div>
<div class="mw-collapsible-content">
You can connect an output port to one ore more input port to set the data flow direction.
</div>
</div>
<p></p>

= Models =
<div class="toccolours mw-collapsible mw-collapsed expandable">
<div class="mw-collapsible-preview">Node</div>
<div class="mw-collapsible-content">
Represent a node in the pipeline. A node can have bunch of input ports and bunch of output ports. Each port has a unique id within the node.

<b>Properties</b>
<pre><nowiki>
id:              The id of the node, it is unique within a pipeline
title:           A human readable name of this node
description:     Detailed description of this node
pipeline:        The pipeline this node belongs to
input_port_ids:  tuple of all input port ids
output_port_ids: tuple of all output port ids
input:           The default input port
output:          The default output port
</nowiki></pre>

<b>method: get_port</b>
<pre><nowiki>
def get_port(self, id:str) -> Optional["Port"]
# each port has a unique id within the node.
# return a port given port id

# You can also use [] to get port by name
node["foo"]  # same as node.get_port("foo")
</nowiki></pre>

<b>method: on_message</b>
<pre><nowiki>
on_message(self, port_id:str, data:Any)

Called when this node receives data. port_id is the id of the port which receive the data, data is the JSON payload of the data.
</nowiki></pre>

<b>method: <nowiki>>> and <<</nowiki></b>
<pre><nowiki>
# You can use >>, << to connect nodes, here are examples:
nodeA >> nodeB               # connect default output port of nodeA to default input port of node B
nodeA["foo"] >> nodeB        # connect port "foo" of nodeA to default input port of node B
nodeA["foo"] >> nodeB["bar"] # connect port "foo" of nodeA to port "bar" of node B

# You can also use << as a reverse of >>
</nowiki></pre>

<b>method: emit</b>
<pre><nowiki>
def emit(self, data:Any, port_id:str=DEFAULT_OUTPUT_PORT_ID)

# emit data to it's output port specified by port_id
</nowiki></pre>
</div>
</div>
<p></p>

<div class="toccolours mw-collapsible mw-collapsed expandable">
<div class="mw-collapsible-preview">Generator</div>
<div class="mw-collapsible-content">
Represent a node that does not have input ports, it is derived from Node.

<b>methods: pump</b>
<pre><nowiki>
def pump(self)

# derived class must override this method to pump data to the pipeline.
</nowiki></pre>
</div>
</div>
<p></p>

<div class="toccolours mw-collapsible mw-collapsed expandable">
<div class="mw-collapsible-preview">Sink</div>
<div class="mw-collapsible-content">
Represent a node that does not have output ports, it is derived from Node.
</div>
</div>
<p></p>

<div class="toccolours mw-collapsible mw-collapsed expandable">
<div class="mw-collapsible-preview">Port</div>
<div class="mw-collapsible-content">
Represent an input port or output port of a node.

<b>Properties</b>
<pre><nowiki>
type:  Either PortType.INPUT or PortType.OUTPUT, represent it is a input port or output port
id:    ID of the port, it is unique within the node
owner: The node which this port belongs to
</nowiki></pre>

<b>method: <nowiki>>> and <<</nowiki></b>
<pre><nowiki>
# connect this port to other port
# see Node document.
</nowiki></pre>

<b>method: emit</b>
<pre><nowiki>
def emit(self, json_data:Any)
# emit JSON data to this port
</nowiki></pre>

</div>
</div>
<p></p>

<div class="toccolours mw-collapsible mw-collapsed expandable">
<div class="mw-collapsible-preview">Pipeline</div>
<div class="mw-collapsible-content">
Represent a pipeline, which has
* bunch of nodes
* A node's output port can connect to another node's input port

<b>Properties</b>
<pre><nowiki>
id:          The id of the pipeline, every pipeline has a unique id.
title:       A human readable short description of this pipeline.
description: Detailed description of this pipeline.
mq:          RabbitMQ instance
nodes:       Tuple containing all node belongs to this pipeline.
</nowiki></pre>

<b>method: message_loop</b>
<pre><nowiki>
def message_loop(self):

# enters a message loop, allow each node to process data
</nowiki></pre>

<b>method: <nowiki>[]</nowiki></b>
<pre><nowiki>
# You can use [] to get node by id
pipeline["foo"]  # get node with id "foo" of this pipeline
</nowiki></pre>

</div>
</div>
<p></p>

<div class="toccolours mw-collapsible mw-collapsed expandable">
<div class="mw-collapsible-preview">RabbitMQ</div>
<div class="mw-collapsible-content">
Represent a RabbitMQ connection
</div>
</div>
<p></p>