> For the complete documentation index, see [llms.txt](https://developer.kizen.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://developer.kizen.com/docs/concepts/smartconnectors.md).

# SmartConnectors

{% hint style="success" %}
**Audience:** Administrators, Developers, Integrators, Solution Architects

**Purpose:** Introduces <code class="expression">space.vars.smartconnectors</code> as the batch data integration and ETL layer of the <code class="expression">space.vars.Kizen\_company\_name</code> platform, establishes a mental model for how they work, and prepares readers to progress into Core Concepts, <code class="expression">space.vars.smartconnector</code> Types, <code class="expression">space.vars.smartconnector</code> SQL Processing, and API guidance.
{% endhint %}

## Overview

<code class="expression">space.vars.smartconnectors</code> are the mechanism <code class="expression">space.vars.Kizen\_company\_name</code> uses to process data in bulk, ingesting, transforming, and loading large datasets into <code class="expression">space.vars.entities</code> reliably and efficiently. They accept data from spreadsheet files, webhooks, schedules, or external sources; optionally transform it using ClickHouse SQL; validate it through typed execution variables; and load it into one or more <code class="expression">space.vars.Kizen\_company\_name</code> <code class="expression">space.vars.objects</code>. <code class="expression">space.vars.smartconnectors</code> are designed for batch operations, not real-time, single-<code class="expression">space.vars.entity</code> processing.

<code class="expression">space.vars.smartconnectors</code> act only on data that is explicitly included in the current run's input. Existing <code class="expression">space.vars.entities</code> in <code class="expression">space.vars.Kizen\_company\_name</code> are unaffected unless they appear in the input data and match a load step's matching rules.

### When To Use SmartConnectors

<code class="expression">space.vars.smartconnectors</code> are the right tool when you need to process data in bulk, apply transformation logic before ingestion, or build a repeatable <code class="expression">space.vars.workflow</code> that runs on a schedule or in response to incoming data. Common use cases include:

* Loading data in bulk from an external system on a recurring schedule. For example, ingesting a weekly carrier file or a nightly database export
* Transforming raw vendor or partner files before ingesting them into <code class="expression">space.vars.Kizen\_company\_name</code> <code class="expression">space.vars.entities</code>, using SQL to normalize, filter, or reshape the data
* Batching inbound webhook data for efficient processing rather than handling each event individually as it arrives
* Populating multiple related <code class="expression">space.vars.objects</code> from a single data source in a single run. For example, creating vendor <code class="expression">space.vars.entities</code> and then creating product <code class="expression">space.vars.entities</code> that reference them
* Running scheduled logic against reference data without requiring a file upload and using the Scheduler <code class="expression">space.vars.smartconnector</code> type with existing <code class="expression">space.vars.Kizen\_company\_name</code> data as the input

#### When To Use SmartConnectors VS. The Kizen API

Both <code class="expression">space.vars.smartconnectors</code> and the <code class="expression">space.vars.Kizen\_company\_name</code> API can write data to <code class="expression">space.vars.Kizen\_company\_name</code>, but they are optimized for different use cases. Use the table below to determine which tool is the right fit for your scenario.

|                    | SmartConnectors                                                                         | Kizen API                                                                |
| ------------------ | --------------------------------------------------------------------------------------- | ------------------------------------------------------------------------ |
| **Volume**         | Hundreds, thousands, or millions of <code class="expression">space.vars.entities</code> | Single <code class="expression">space.vars.entities</code> or small sets |
| **Transformation** | Complex SQL transformation before ingestion                                             | Pass-through or simple field mapping                                     |
| **Frequency**      | Recurring imports, scheduled <code class="expression">space.vars.workflows</code>       | Real-time, event-driven operations                                       |
| **Validation**     | Row-level type validation with per-row failure reporting                                | Immediate response per request                                           |

{% hint style="info" %}
**Note:** If you are reading or writing more than \~1,000 <code class="expression">space.vars.entities</code>, <code class="expression">space.vars.smartconnectors</code> are almost always the right choice. For real-time events at high volume, <code class="expression">space.vars.smartconnectors</code> can also be used to reduce processing overhead by grouping incoming webhook data into batches for scheduled processing.
{% endhint %}

#### When To Use SmartConnectors VS. Agentic Workflows

<code class="expression">space.vars.smartconnectors</code> and <code class="expression">space.vars.automations</code> are both powerful tools in <code class="expression">space.vars.Kizen\_company\_name</code>, but they are built for fundamentally different kinds of work. Choosing the right one depends on how many <code class="expression">space.vars.entities</code> you are operating on and whether the work is event-driven or batch-oriented.

|                         | SmartConnectors                                                                                                                                         | Agentic Workflows                                                                                                                                                                                                           |
| ----------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Volume**              | Hundreds, thousands, or millions of <code class="expression">space.vars.entities</code>                                                                 | One <code class="expression">space.vars.entity</code> at a time, or no <code class="expression">space.vars.entity</code> (global)                                                                                           |
| **Trigger model**       | File upload, batched webhook queue, or time-based schedule                                                                                              | <code class="expression">space.vars.entity</code> field changes, form submissions, individual webhook events, or time-based triggers on a single <code class="expression">space.vars.entity</code>                          |
| **Data transformation** | Full SQL transformation using ClickHouse before ingestion                                                                                               | Field updates, conditions, and logic applied step by step                                                                                                                                                                   |
| **Execution model**     | Batch processing with chunked, sequential load steps                                                                                                    | Asynchronous, <code class="expression">space.vars.entity</code>-scoped or global step execution                                                                                                                             |
| **Best for**            | Bulk imports, recurring data <code class="expression">space.vars.workflows</code>, and complex ETL <code class="expression">space.vars.workflows</code> | <code class="expression">space.vars.workflow</code> <code class="expression">space.vars.automation</code>, notifications, field updates, and integrations on individual <code class="expression">space.vars.entities</code> |

{% hint style="info" %}
**Note:** If you are operating on more than \~1,000 records at once, <code class="expression">space.vars.smartconnectors</code> are almost always the right choice. For single-<code class="expression">space.vars.entity</code>, event-driven work, use <code class="expression">space.vars.automations</code>. The two can also work together. A <code class="expression">space.vars.smartconnector</code> load step can explicitly trigger an <code class="expression">space.vars.automation</code> on <code class="expression">space.vars.entities</code> it creates or updates.
{% endhint %}

### Key Differentiators

<code class="expression">space.vars.smartconnectors</code> are built for a specific kind of work. The following capabilities set them apart from other data integration approaches in <code class="expression">space.vars.Kizen\_company\_name</code>.

* **Batch-Optimized:** <code class="expression">space.vars.smartconnectors</code> process data in chunks designed for reliability and efficiency at scale, making them the right choice when operating on hundreds, thousands, or millions of <code class="expression">space.vars.entities</code>. Processing continues chunk by chunk even if individual rows fail.
* **SQL-Powered Transformation:** The integrated ClickHouse SQL environment gives developers and integrators the full power of a columnar analytical database to transform, join, and reshape data before it enters <code class="expression">space.vars.Kizen\_company\_name</code>, without needing to pre-process files externally.
* **Type-Safe by Design:** Execution variables enforce data type validation before any data reaches the load step, failing individual rows without halting the entire run and producing a full audit report of what was processed, skipped, and errored. When a variable is declared as a Number type, the system extracts numeric values from the input and ignores any non-numeric characters. For example, `$16.99 USD` becomes `16.99` and `41 Apples` becomes `41`. This coercion happens silently rather than failing the row, so confirm that your input data produces the expected numeric values by reviewing the execution variable tab in the output report before processing live data.
* **File-Oriented Workflow:** Every <code class="expression">space.vars.smartconnector</code> type, whether triggered by a file upload, a webhook, a schedule, or a bulk action, ultimately passes data through the same file-oriented <code class="expression">space.vars.workflow</code>, meaning all <code class="expression">space.vars.smartconnector</code> types share identical behavior from variable mapping onward.

***

## SmartConnectors Mastery Checklist

Explore the following topics to understand how <code class="expression">space.vars.smartconnectors</code> are configured and used in <code class="expression">space.vars.Kizen\_company\_name</code>.

**Core Knowledge**

* [ ] [What a SmartConnector is and how it differs from the Kizen API and Agentic Workflows](#when-to-use-smartconnectors-vs.-the-kizen-api)
* [ ] [The `Input → SQL Processing → Execution Variables → Load Steps → Kizen Records` mental model and how to apply it to a real Workflow](/docs/concepts/smartconnectors/smartconnector-core-concepts.md#conceptual-model)
* [ ] [The five SmartConnector types and when to use each](/docs/concepts/smartconnectors/smartconnector-core-concepts.md#smartconnector-types)

**SQL Processing**

* [ ] [How the ClickHouse SQL processor works and what input tables are available](/docs/concepts/smartconnectors/smartconnector-sql-processing.md#provided-input-tables)
* [ ] [How to write, test, and publish SQL within a SmartConnector](/docs/concepts/smartconnectors/smartconnector-sql-processing.md#writing-and-testing-sql)
* [ ] [How to use reference data, SQL parameters, and integration secrets in SQL](/docs/concepts/smartconnectors/smartconnector-sql-processing.md#sql-parameters)
* [ ] [How to select the right CHDB runtime version and avoid the `SELECT *` anti-pattern](/docs/concepts/smartconnectors/smartconnector-sql-processing.md#chdb-runtime-versions)
* [ ] [How to connect to external data sources including PostgreSQL, MySQL, and S3](/docs/concepts/smartconnectors/smartconnector-external-data-sources.md#supported-external-sources)

**Execution Variables**

* [ ] [What execution variables are and why every SQL output column must be declared with an explicit type](/docs/concepts/smartconnectors/smartconnector-execution-variables.md#what-are-execution-variables)
* [ ] [How type validation works at the row level and what happens when a required variable is invalid or blank](/docs/concepts/smartconnectors/smartconnector-execution-variables.md#type-validation-behavior)
* [ ] [How variable scopes relate to SQL output tables](/docs/concepts/smartconnectors/smartconnector-execution-variables.md#variable-scopes)

**Load Steps**

* [ ] [How matching rules identify whether to create or update a Record](/docs/concepts/smartconnectors/smartconnector-load-steps.md#matching-rules)
* [ ] [How mapping rules write execution variable values to fields and how conflict resolution works](/docs/concepts/smartconnectors/smartconnector-load-steps.md#mapping-rules)
* [ ] [How relationship fields work in SmartConnectors and why missing related Records are skipped rather than created](/docs/concepts/smartconnectors/smartconnector-load-steps.md#relationship-fields)&#x20;
* [ ] [How to configure post-load Agentic Workflow triggers and the entity variable pattern for multi-step Workflows](/docs/concepts/smartconnectors/smartconnector-load-steps.md#trigger-automations)

**Running and Operating**

* [ ] [How diff checking works, when to use it, and when a custom diff approach is more appropriate](/docs/concepts/smartconnectors/smartconnector-diff-checking.md)&#x20;
* [ ] [How to activate a SmartConnector, run a dry run, interpret the XLS output report, and troubleshoot failed rows](/docs/concepts/smartconnectors/running-a-smartconnector.md#activation)
* [ ] [How to configure concurrent executions, notification emails, and the execution ended webhook](/docs/concepts/smartconnectors/smartconnector-settings.md)&#x20;

**Webhook Connectors**

* [ ] [What makes Webhook SmartConnectors different](/docs/concepts/smartconnectors/webhook-smartconnectors.md#what-makes-webhook-smartconnectors-different)
* [ ] [How to access webhook payload data in SQL using `input.webhooks` and `input.webhooks_raw`](/docs/concepts/smartconnectors/webhook-smartconnectors.md#access-to-webhook-data-in-sql)&#x20;
* [ ] [How to capture sample webhooks during development using the inactive state](/docs/concepts/smartconnectors/webhook-smartconnectors.md#capturing-sample-webhooks-for-development)

**Advanced Topics**

* [ ] [How permissions govern SmartConnector access](/docs/concepts/smartconnectors/smartconnector-permissions.md)
* [ ] [How to export, import, and duplicate SmartConnectors across businesses and environments](/docs/concepts/smartconnectors/smartconnector-export-import-and-duplication.md)
* [ ] [How connectors, executions, and output are structured as data model entities](/docs/concepts/smartconnectors/smartconnector-data-model.md#smartconnector-data-model-foundations)

**APIs & Developer Tooling**

* [ ] [How to trigger, monitor, and cancel SmartConnector executions programmatically via API](/docs/concepts/smartconnectors/smartconnector-apis.md#how-smartconnectors-are-exposed-programmatically)
* [ ] How to upload an input file and trigger a run via API — see File Upload (Trigger Run) API **(Topic Coming Soon)**
* [ ] [How to deliver webhook payloads to a Webhook SmartConnector programmatically](/docs/concepts/smartconnectors/smartconnector-apis/smartconnector-webhook-trigger-api.md)
* [ ] How to use the `sc` CLI and dev package for local SQL development and deployment — see Build with SmartConnectors **(Topic Coming Soon)**

**Design Best Practices**

* [ ] [How to design reliable, maintainable SmartConnectors that avoid common data integrity pitfalls](/docs/concepts/smartconnectors/smartconnector-design-best-practices.md)
* [ ] [When to use SQL parameters, integration secrets, and the entity variable pattern](https://developer.kizen.com/docs/~/changes/316/concepts/smartconnectors/smartconnector-design-best-practices#sql-design)

***

## What's Next

Continue to [SmartConnector Core Concepts](/docs/concepts/smartconnectors/smartconnector-core-concepts.md) to build the foundational understanding you need before configuring <code class="expression">space.vars.smartconnectors</code>, designing load steps, or working with the API. The Core Concepts establishes the shared vocabulary, <code class="expression">space.vars.workflow</code> model, and component overview that every subsequent topic builds on.

<details>

<summary>Related Topics</summary>

* [SmartConnector Core Concepts](/docs/concepts/smartconnectors/smartconnector-core-concepts.md)
* [SmartConnector Types](/docs/concepts/smartconnectors/smartconnector-types.md)
* [SmartConnector SQL Processing](/docs/concepts/smartconnectors/smartconnector-sql-processing.md)

</details>


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://developer.kizen.com/docs/concepts/smartconnectors.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.