Task Types & Execution Modes

Tasks are the building blocks of a pipeline. Each task performs a specific job. Supported tasks include:

• Data Ingestion: Pull data from external systems like databases, APIs, or files using Airbyte (through Qualiz UI).
• Run Python Script: Execute custom Python logic.
• Run SQL Script: Run custom SQL queries using a secure backend.
• Data Cleansing: Standardize or enrich data using cleansing rules.
• Data Deduplication: Detect and remove duplicate records using algorithms.
• Invoke Another Job: Trigger a different job as part of the current pipeline.
• Run Python with Apache Beam (coming soon): Execute Python code on a distributed Beam cluster.
• Email Notification (coming soon): Send results or alerts via email post execution.

Each task can be customized from the UI with required parameters.