Skip to content

Projects

Projects in Qualiz serve as the primary organizational unit that brings together related data pipelines, validation rules, source and destination configurations, and audit logs under a unified context. This structure simplifies management, access control, and monitoring for teams working across multiple domains or tenants.

Key Features

Logical Grouping

Projects group pipelines, rules, connections, and audit logs, providing a clear separation between workflows that belong to different teams, departments, or customers.

Multi-Tenant and Multi-Domain Support

Projects enable seamless handling of data pipelines for multiple tenants or business domains by isolating resources such as storage, metadata, and configurations.

Contextual UI Experience

Once a project is selected in the Qualiz UI, only the jobs, audits, rules, connections, and related configurations belonging to that project are displayed. This ensures users interact with data and workflows relevant to their scope, reducing complexity and improving operational efficiency.

Data Isolation and Security

Projects help maintain data segregation by configuring separate buckets, schemas, and runtime environments for each project. This design supports compliance requirements and enhances data privacy.

Extensibility and Scalability

As new workflows or pipelines are created, they can be organized under existing projects or new ones, allowing the platform to scale smoothly while maintaining operational clarity.

Integrated Workflow Management

Projects serve as the foundation for orchestrating pipelines using Airflow, where each pipeline (DAG) is tied to the project context. Similarly, data ingestion setups, cleansing rules, deduplication strategies, and monitoring configurations are scoped to the project.

How Projects Work

  1. Create or Select a Project
    Users can create a new project or select an existing one through the UI. Each project can be associated with specific teams, datasets, or business units.

  2. Define Pipelines and Rules
    Within a project, users can define data ingestion pipelines, transformation workflows, data cleansing rules, and deduplication strategies tailored to their domain.

  3. Access Audit Logs and Monitoring
    Project-specific audit logs for jobs and tasks help track data pipeline executions, troubleshoot errors, and analyze performance metrics.

  4. Manage Configurations
    All configurations, including source/destination mappings, rules, and settings, are isolated per project, ensuring consistency and reducing configuration conflicts.

Benefits of Using Projects

  • Improved organization and collaboration
  • Simplified access control and security management
  • Enhanced observability and troubleshooting
  • Scalable architecture for handling multiple domains or tenants
  • Clear separation of data, metadata, and configurations