Real-Time Data Flow in Pega: A Step-by-Step Guide

 

In modern applications, real-time data synchronization is crucial for creating responsive and efficient systems. This blog post demonstrates how to set up and test a real-time data flow in Pega using built-in streaming capabilities.

Prerequisites:

Before you begin, ensure that your Pega environment has an available Stream node. You can verify this by navigating to:

In our application, a stream node is available, and its status is NORMAL, which means it's ready for use.

Step 1: Create Source and Target Data Types

Create two data types:

• Source
• Target

Ensure both data types have identical structures (same columns) to facilitate smooth data transformation during the flow.

 

Step 2: Create a Stream Data Set

Create a Stream Data Set under the Source class (e.g., O0XRR7-ExploreP-Data-Source).

You can choose from three stream types: Kafka, Kinesis, or Stream.

Since no external stream systems are used in this setup, select the built-in Stream type.

This Stream Data Set will serve as the source for your real-time data flow.

Below is the created stream data set; we will use this in the source of the data flow.

 

 Step 3: Design the Data Flow

Create a new Data Flow and configure the following:

 Source Component:

• Select the previously created Stream Data Set as the input source.

Transformation (Convert Component):

Add a Convert shape to transform records from the Source class to the Target class.

Target Component:

• Create a new Database Table Data Set in the Target class
• This will be used to insert the transformed records.
• Add this data set as the target in your data flow.

Selected the dataset created as the target component to insert the record

Step 4: Test the Real-Time Flow

Run the Flow:

Open the Data Flow rule.

Click Run to activate the real-time listener.

 

Create an Activity to Save a Record

Create and run an activity to insert a test record into the Source class.

    

 

Verify the Result

Once the activity runs, check the Target class.

You’ll find that the record has been moved in real time using the data flow you configured.