Firebird/GettingStarted

Preparation

• Assuming you are using a linux based machine. (I am using a Ubuntu 22.04)
• Assuming you can ssh to this machine with a private ssh key
• Create a docker network called "streaming"

docker network create -d bridge streaming

Step 1: checkout the example

git clone https://github.com/stonezhong/firebird.git

# then enters the example home directory
cd firebird/examples/getting-started/

Step 2: customize your ssh config

You need to edit app/ssh_config/config and app/ssh_config/keys/devbox

Step 3: start zookeeper and rabbitmq

docker-compose -f infra/docker-compose.yaml up -d

Step 4: build docker image for your streaming application

# Build docker image for the streaming application
docker-compose -f app/docker-compose.yaml build

# start the console container
docker-compose -f app/docker-compose.yaml up -d

step 5: register your pipeline

# Before we can use any pipeline, we need to register the pipeline with the system
# A pipeline module is a python module which implements function get_pipeline(..) which returns a pipeline
# see app/src/pipe.py, since we use pipe.py, so the pipeline module name is "pipe"
docker exec -it console pipeline register -pmn pipe

step 7: list pipelines

docker exec -it console pipeline list

test:
    module: pipe
    executors: None

• Now you can see pipeline "test" has been registered, the id is returned from app/src/pipe.py

step 6: execute pipeline

# We are executing the pipeline
# A pipeline module is a python module which implements function get_pipeline(..) which returns a pipeline
# see app/src/pipe.py, since we use pipe.py, so the pipeline module name is "pipe"
docker exec -it console pipeline execute -pid test -dhn devbox -dcn foo