AI-enabled Features
Overview¶
The platform integrates AI capabilities to automate data preparation, monitoring, and quality assurance, reducing manual effort and improving accuracy.
Automated Cleansing Rule Generation¶
- Purpose: Automatically suggest data cleansing and transformation rules based on column profiling.
- How It Works:
- AI analyzes column names, data types, and value patterns.
- Suggests appropriate transformations such as:
- Standardizing formats (dates, phone numbers, currency).
- Handling missing values.
- Normalizing categorical values.
- Users can review, modify, and approve suggested rules before execution.
Intelligent Deduplication¶
- Purpose: Enhance duplicate detection accuracy with machine learning.
- How It Works:
- AI-based similarity scoring across records.
- Supports fuzzy matching, semantic similarity, and cross-field analysis.
AI-Driven Job Monitoring¶
- Purpose: Proactively identify job anomalies, failures, or performance bottlenecks.
- How It Works:
- Monitors historical execution metrics (duration, resource usage, error frequency).
- Flags abnormal job patterns in real time.
- Provides probable cause insights and potential fixes.