SmartConnectors

Audience: Administrators, Developers, Integrators, Solution Architects

Purpose: Introduces SmartConnectors as the batch data integration and ETL layer of the Kizen platform, establishes a mental model for how they work, and prepares readers to progress into Core Concepts, SmartConnector Types, SmartConnector SQL Processing, and API guidance.

Overview

SmartConnectors are the mechanism Kizen uses to process data in bulk, ingesting, transforming, and loading large datasets into Records reliably and efficiently.They accept data from spreadsheet files, webhooks, schedules, or external sources; optionally transform it using ClickHouse SQL; validate it through typed execution variables; and load it into one or more Kizen Objects. SmartConnectors are designed for batch operations, not real-time, single-Record processing.

SmartConnectors act only on data that is explicitly included in the current run's input. Existing Records in Kizen are unaffected unless they appear in the input data and match a load step's matching rules.

When to Use SmartConnectors

SmartConnectors are the right tool when you need to process data in bulk, apply transformation logic before ingestion, or build a repeatable Workflow that runs on a schedule or in response to incoming data. Common use cases include:

Loading data in bulk from an external system on a recurring schedule. For example, ingesting a weekly carrier file or a nightly database export
Transforming raw vendor or partner files before ingesting them into Kizen Records, using SQL to normalize, filter, or reshape the data
Batching inbound webhook data for efficient processing rather than handling each event individually as it arrives
Populating multiple related Objects from a single data source in a single run. For example, creating vendor Records and then creating product Records that reference them
Running scheduled logic against reference data without requiring a file upload and using the Scheduler SmartConnector type with existing Kizen data as the input

When to Use SmartConnectors vs. the Kizen API

Both SmartConnectors and the Kizen API can write data to Kizen, but they are optimized for different use cases. Use the table below to determine which tool is the right fit for your scenario.

SmartConnectors

Kizen API

Volume

Hundreds, thousands, or millions of Records

Single Records or small sets

Transformation

Complex SQL transformation before ingestion

Pass-through or simple field mapping

Frequency

Recurring imports, scheduled Workflows

Real-time, event-driven operations

Validation

Row-level type validation with per-row failure reporting

Immediate response per request

Note: If you are reading or writing more than ~1,000 records, SmartConnectors are almost always the right choice. For real-time events at high volume, SmartConnectors can also be used to reduce processing overhead by grouping incoming webhook data into batches for scheduled processing.

When to Use SmartConnectors vs. Agentic Workflows

SmartConnectors and Agentic Workflows are both powerful tools in Kizen, but they are built for fundamentally different kinds of work. Choosing the right one depends on how many Records you are operating on and whether the work is event-driven or batch-oriented.

SmartConnectors

Agentic Workflows

Volume

Hundreds, thousands, or millions of Records

One Record at a time, or no Record (global)

Trigger model

File upload, batched webhook queue, or time-based schedule

Record field changes, form submissions, individual webhook events, or time-based triggers on a single Record

Data transformation

Full SQL transformation using ClickHouse before ingestion

Field updates, conditions, and logic applied step by step

Execution model

Batch processing with chunked, sequential load steps

Asynchronous, Record-scoped or global step execution

Best for

Bulk imports, recurring data Workflows, and complex ETL Workflows

Workflow Agentic Workflow, notifications, field updates, and integrations on individual Records

Note: If you are operating on more than ~1,000 records at once, SmartConnectors are almost always the right choice. For single-Record, event-driven work, use Agentic Workflows. The two can also work together. A SmartConnector load step can explicitly trigger an Agentic Workflow on Records it creates or updates.

Key Differentiators

SmartConnectors are built for a specific kind of work. The following capabilities set them apart from other data integration approaches in Kizen.

Batch-Optimized: SmartConnectors process data in chunks designed for reliability and efficiency at scale, making them the right choice when operating on hundreds, thousands, or millions of Records. Processing continues chunk by chunk even if individual rows fail.
SQL-Powered Transformation: The integrated ClickHouse SQL environment gives developers and integrators the full power of a columnar analytical database to transform, join, and reshape data before it enters Kizen, without needing to pre-process files externally.
Type-Safe by Design: Execution variables enforce data type validation before any data reaches the load step, failing individual rows without halting the entire run and producing a full audit report of what was processed, skipped, and errored. When a variable is declared as a Number type, the system extracts numeric values from the input and ignores any non-numeric characters. For example, $16.99 USD becomes 16.99 and 41 Apples becomes 41. This coercion happens silently rather than failing the row, so confirm that your input data produces the expected numeric values by reviewing the execution variable tab in the output report before processing live data.
File-Oriented Workflow: Every SmartConnector type, whether triggered by a file upload, a webhook, a schedule, or a bulk action, ultimately passes data through the same file-oriented Workflow, meaning all SmartConnector types share identical behavior from variable mapping onward.