SmartConnector Core Concepts
Audience: Administrators, Developers, Integrators, and Solution Architects
Purpose: Defines the structural foundation of SmartConnectors.
Overview
SmartConnectors move your data from point A to point B reliably, and in a predictable order. No matter what type of SmartConnector you're working with, data always follows the same path: it comes in, gets transformed, and gets written to Records.
Why SmartConnectors Matter
At scale, manual data entry and one-Record-at-a-time Agentic Workflows break down. SmartConnectors are built for the operations that can't be handled any other way, moving large amounts of data reliably, repeatedly, and without human intervention.
A well-designed SmartConnector integration enables:
Reliable bulk data ingestion from external systems
Consistent, repeatable Record creation and updates at scale
Predictable load behavior with full execution logging and reporting
Clean data governance through type enforcement and diff checking
Reusable Workflow logic across multiple SmartConnector types
Without SmartConnectors, teams often resort to fragile workarounds: manual CSV uploads, overloaded Agentic Workflows, or custom scripts that are difficult to maintain and impossible to audit.
Foundational Definitions
The following terms are used throughout all SmartConnector documentation.
SmartConnector
A batch data integration tool that reads structured input, optionally transforms it via SQL, maps it to typed variables, and writes Records to one or more Kizen Objects. Designed for operations on hundreds to millions of Records.
The mechanism by which a SmartConnectorreceives its input data. All three connector types: Spreadsheet, Webhook, Scheduler, share identical Workflow behavior from the variable mapping step onward.
Execution (Topic Coming Soon)
A single end-to-end run of a SmartConnector, from data intake through load step completion. Each execution is logged with status, timing, and a downloadable report.
Execution Variable (Topic Coming Soon)
A typed variable used by load steps. Every column that exits the SQL processor (or the input file, for Spreadsheet connectors without SQL) must be declared as an execution variable before it can be used in a load step.
Additional Variable (Topic Coming Soon)
A typed variable that is not derived directly from the input data but can be used within the connector's Workflow for supplemental logic.
Load Step (Topic Coming Soon)
A configured stage within a connector that writes execution variable data to a Kizen Object. A SmartConnector may contain multiple load steps; they always run sequentially.
Matching Rule (Topic Coming Soon)
Logic applied within a load step to determine whether an incoming row corresponds to an existing Kizen Record. If no match is found, a new Record can be created; if one match is found, it is updated; if multiple are found, the row is skipped.
Mapping Rule (Topic Coming Soon)
Logic applied within a load step that controls how execution variable values are written to fields on a matched or newly created Record; only Update Blank, Overwrite (Clear Field if Blank), Overwrite (But Skip if Blank)
Dry Run (Topic Coming Soon)
A test execution mode that processes the full Workflow and produces a detailed XLS report without writing any data to Kizen. Use dry runs to validate configuration before going live.
Live Run (Topic Coming Soon)
A full execution that processes input data and writes Records to Kizen. A SmartConnector must be active before a live run can execute.
Diff Check (Topic Coming Soon)
An efficiency feature that compares each incoming row against a hash of execution variable values from the most recent successful execution. Rows that have not changed are automatically skipped.
Reference Data (Topic Coming Soon)
Existing Kizen Object data made available inside the SQL processing environment. Reference data can be filtered using saved filter groups and queried alongside input file data.
SQL Processing (Topic Coming Soon)
An optional (Spreadsheet) or required (all other SmartConnector types) transformation layer that processes the input file before variable mapping. Runs in a CHDB (local ClickHouse) environment.
Active / Inactive (Topic Coming Soon)
The activation state of a SmartConnector. Only active SmartConnectors can execute live runs. Activating a SmartConnector triggers full configuration validation. A SmartConnector is automatically deactivated when its SQL configuration is changed and not yet re-validated.
Conceptual Model
Every SmartConnector moves data through the same Workflow, regardless of SmartConnector type:
The diagram above illustrates this flow. Here is what each stage means in practice:
Input is the starting point for every SmartConnectors run. Depending on the SmartConnectors type, this is either a file uploaded manually (Spreadsheet), a batched payload from an external system (Webhook), or a generated timestamp (Scheduler). Regardless of source, all input is treated as a structured file before it moves to the next stage.
SQL processing is optional for Spreadsheet SmartConnectors and required for all other SmartConnector types. Non-spreadsheet SmartConnectors receive data in formats such as JSON (Webhook) or a generated timestamp (Scheduler); SQL is the mechanism that extracts and shapes that data into structured rows before it can be mapped.
Execution variables are the typed interface between SQL output and the rest of the workflow. Every column that is returned by SQL must be declared as a variable with an explicit data type. This is the point where type validity is enforced: invalid required variables fail the row, not the entire run.
Load steps consume execution variables and write Records to Kizen Objects. A single SmartConnector can contain multiple load steps, and they always run in order. Sequence matters: if one Object depends on Records from another, the parent Records must be created first.
A single SmartConnector can write to multiple Objects by chaining load steps, each targeting a different Object
Note: For structural and relational detail on how these components interact, see SmartConnector Data Model (Coming Soon).
High-Level Components
The table below introduces each component at a glance. Each row links to the full topic where that component is documented in depth.
SQL Processor (Topic Coming Soon)
A transformation layer that runs before variable mapping.
Accepts the input file as a table in a CHDB (ClickHouse) environment. Queries write results to the output schema; those tables become the input to variable mapping.
Reshape, filter, join, or enrich raw input data, including JSON from webhooks or reference data from Kizen, before it reaches the load step.
Execution Variables (Topic Coming Soon)
Typed declarations that represent each column exiting the SQL processor or input file.
Every column must be explicitly declared with a data type. Invalid required variables fail the row, not the entire run.
Enforce type safety and provide a stable, named interface that load steps reference by variable name.
Additional Variables (Topic Coming Soon)
Variables not sourced directly from input columns.
Defined separately from execution variables and available for use within the Workflow.
Inject supplemental values or support more complex Workflow logic without requiring them in the source file.
Transformation Flow (Topic Coming Soon)
The ordered sequence of steps between data intake and Record writing.
Moves data through SQL processing, variable mapping, and load steps in a defined order.
Provides a clear, auditable path from raw input to committed Kizen Records.
Load Step (Topic Coming Soon)
A configured stage that writes data to a Kizen Object.
Applies matching rules to find or create Records, then applies mapping rules to write field values. Multiple load steps run sequentially within a single SmartConnector.
Target multiple Objects or apply different write logic to different subsets of data within one SmartConnector run.
Matching Rules (Topic Coming Soon)
Logic that determines how an incoming row is matched to an existing Kizen Record.
Rules are evaluated in order. No match → create new; one match → update; multiple matches → row is skipped. Archived Record handling is configurable.
Prevent duplicate Records and control exactly how the SmartConnector resolves ambiguous or missing matches.
Mapping Rules (Topic Coming Soon)
Logic that controls how execution variable values are written to Record fields.
Applied after a Record is matched or created. Options include Only Update Blank, Overwrite (Clear Field if Blank), Overwrite (But Skip if Blank)
Fine-tune write behavior field by field. For example, never overwrite a value that already exists, or always apply the latest file value.
Diff Check (Topic Coming Soon)
A row-level deduplication feature based on hashed execution variable values.
Compares each row against the hash from the last successful execution. Unchanged rows are skipped automatically.
Avoid redundant processing when running a complete dataset file repeatedly — only changed rows consume processing time.
SmartConnector Types
SmartConnectors support three SmartConnector types, each with a different trigger mechanism and data intake behavior. All three share identical Workflow behavior from the variable mapping step onward. Once data has been shaped into execution variables, the SmartConnector type no longer matters.
The three SmartConnector types are:
Spreadsheet
Webhook
Scheduler
Note: For more details, see SmartConnector Types.
SmartConnector Settings
Each SmartConnector has a set of configurable settings that control its identity, execution behavior, and notification handling.
The configurable settings are as follows:
Description
Concurrent Executions
Notification Emails
Execution Ended Webhook
For more information, see SmartConnector Settings (Topic Coming Soon).
Note: A SmartConnector API Name is defined via original SmartConnector name and cannot be changed once initially created.
Guiding Principles
These are the behavioral rules that govern how SmartConnectors operate. Understand these before configuring anything.
Batch-oriented: SmartConnectors are not intended for real-time or single-Record operations. Use Agentic Workflows for single-Record Workflows.
Execution variables enforce type validity at the row level: A failed type conversion is treated as blank; if the variable is required, the row is skipped. Partial success is a valid execution outcome, and individual rows may be skipped while the rest of the run completes.
Diff checking operates on a hash of execution variable values per row: Unchanged rows are skipped automatically — no additional configuration required beyond enabling the feature.
Canceling a run does not roll back already-processed data: Any Records written before cancellation remain committed in Kizen.
Load steps always execute sequentially: Ordering matters when one Object depends on another — create parent Records before child Records.
Dry run mode produces a full XLS report without writing anything to Kizen: Always validate configuration with a dry run before executing live.
SmartConnectors must be activated before a live run can execute: Activation triggers full configuration validation; a SmartConnector is automatically deactivated when its SQL is changed and not yet re-validated.
Agentic Workflows do not fire automatically on Records created or updated by a SmartConnector run unless explicitly configured in the load step. Default trigger suppression is available as a load step option.
Every SmartConnector type is fundamentally file-oriented: Non-spreadsheet SmartConnectors generate a file internally and pass it through the same Workflow. All SmartConnector types share identical behavior from the variable mapping step onward. This generated input file is available for review from the three-dot menu on any execution; just select Download Input File.
What's Next
The next step is SmartConnector Types. That topic covers each of the trigger mechanisms in detail: what distinguishes its data intake behavior, when to use it, and what it produces as input to the SmartConnector Workflow.
Last updated
Was this helpful?