Your first pipeline

This walkthrough builds a pipeline that generates sample log events, rewrites them with VRL, and prints them on the agent. It takes about ten minutes on a fresh install.

A pipeline runs on the agents enrolled in its environment, so you start with an environment and an agent.

Enroll an agent

1. Open Environments. Pick the environment you want to deploy to, or create one (for example staging).

2. On the environment’s page, choose Generate Token and copy the enrollment token.

3. Start an agent with that token. On the VectorFlow host itself, the installer can run one for you:

   veltro agent enroll
   (umask 077; printf '%s' '<enrollment-token>' > .suite/vf-agent/token)

   Run these from the install directory. The agent waits for the token file, enrolls, and appears under Fleet. For agents on other hosts, see Agents.

Build the pipeline

1. Open Pipelines, choose New pipeline, give it a name such as demo-pipeline, and select the environment your agent is in.

2. In the editor, search the component palette for Demo Logs and drag it onto the canvas. Select it and set its format to json.

3. Drag a Remap transform next to it and put this in its VRL source:

   .message = "processed: " + string!(.message)
   .processed_at = now()

4. Drag a Console sink next to the transform and set its encoding to json. The console sink writes to the agent’s own output, which VectorFlow shows in the pipeline’s logs.

5. Connect the three: drag from the right edge of Demo Logs to Remap, then from Remap to Console. The editor refuses connections Vector can’t run, such as a metrics source into a logs-only sink.

6. Choose Validate pipeline in the toolbar. VectorFlow generates the Vector configuration and has Vector check it. Fix whatever it reports, then save with Save pipeline (Cmd+S or Ctrl+S).

Deploy it

1. Choose Deploy in the toolbar. The dialog shows the generated configuration, or the change from what is running now.

2. Write a Deployment Reason, such as “First demo pipeline”. It’s required and stays in the pipeline’s history.

3. Choose Publish to Agents. If the environment needs a second person to approve deploys, the button asks for approval instead, and the deploy runs once someone else approves it.

Agents poll for new configuration every five seconds by default. Once yours picks the pipeline up, Fleet shows it running, and the pipeline’s logs show the rewritten events.

Every deploy saves a version of the configuration with its reason, so you can compare versions or go back to one later.