Webhook SmartConnectors

Audience: Developers, Integrators, Solution Architects

Purpose: Explains how Webhook SmartConnectors queue and batch inbound HTTP requests, how to access payload data using input.webhooks and input.webhooks_raw, and how to capture, test, and reprocess webhook data during development.

Overview

A Webhook SmartConnector accepts incoming HTTP requests from an external system, queues them, and processes them in batches on a schedule you set. When the SmartConnector runs, it applies your SQL to the queued payloads and loads the results into a Kizen Object.

This batching model makes Webhook SmartConnectors significantly more efficient than processing each inbound event individually through an Agentic Workflow webhook trigger, particularly when volume is high. SmartConnectors are typically the better fit when you need to batch-process payloads and transform them into the schema of your Objects representing your business use cases.

What Makes Webhook SmartConnectors Different

Spreadsheet SmartConnectors process a file you supply, either uploaded manually or delivered via API. Webhook SmartConnectors work differently: instead of waiting for a file, they expose an HTTP endpoint that accepts inbound POST requests. Those requests are queued as they arrive and then processed together on a schedule you configure.

The key implications of this model are:

• Batching over real-time: Inbound webhooks are not processed one at a time as they arrive. They accumulate in a queue and are processed together when the next scheduled execution runs.
• The inactive state is a development tool: While a Webhook SmartConnector is inactive, incoming requests are still queued but not processed. This makes the inactive state useful for capturing real sample data during development without running it against live Kizen Records.

---

Sending Webhook Data

Note: Webhooks need to be authenticated using Kizen credentials. For more information, check out the Authentication topic.

Send inbound data to the webhook URL as an HTTP POST request.

• Request body: JSON is the recommended format. input.webhooks_raw provides the same data as input.webhooks but with simpler, less strictly typed columns. Use this table when you need to parse the payload manually or when the typed columns in input.webhooks do not suit your use case.
• Query strings: Query string parameters are accepted and available in the querystring column of input.webhooks and via extractURLParameter in input.webhooks_raw. Avoid passing sensitive data in query strings, as they may be captured in server logs and network traces.

Size limits

Parameter	Limit
Request body	256 KB
Query string	approximately 4 KB

Requests that exceed these limits will be rejected.

---

Access to Webhook Data in SQL

Each queued webhook is available as a row in one of two input tables. Use input.webhooks for most use cases. Use input.webhooks_raw when your payloads are not JSON or when you need full control over how the body is parsed.

input.webhooks

input.webhooks contains the webhook request body (if the body was formatted as JSON).

Field	Type	Description
timestamp	DateTime64	The time the webhook was received.
employee_id	UUID	The identifier of the employee associated with the request.
querystring	String	The raw query string from the request URL.
body	JSON	The parsed JSON request body, accessible via dot notation.

Access JSON body values using dot notation, for example body.customer_email. In ClickHouse, JSON columns store sub-columns as the Dynamic type, so cast values to an explicit type for use in SQL. For example, body.age::Int32."

Extract query string values using the extractURLParameter function:

extractURLParameter(querystring, 'param_name')

Example input.webhooks (recommended for use)

The following example reads from input.webhooks, accesses JSON body values using dot notation, and writes the results to a named output table.

Given a POST body of the form:

{"table": 7, "order": {"item": "fries", "toppings": ["ketchup"]}}

The following SQL extracts the relevant fields:

create table output.food_orders engine Log() as
select
    body.table::String                              as table_number,
    body.order.item::String                         as order_item,
    arrayStringConcat(body.order.toppings::Array(String), ', ') as topping_list
from
    input.webhooks;

This produces:

table_number	order_item	topping_list
7	fries	cheese, ketchup

input.webhooks_raw

input.webhooks_raw provides the same data as input.webhooks, except columns are typed as String rather than cast to their specific types. For example, body is the raw JSON payload as a string rather than a JSON Object, and employee_id is a string rather than a UUID. Use input.webhooks_raw when you need to parse the payload manually or when the typed columns in input.webhooks don't suit your use case.

Example input.webhooks_raw

The following example reads from input.webhooks_raw, extracts values from a JSON body and a query string using explicit ClickHouse JSON functions, and writes the results to a named output table. Use this approach when your payloads are not JSON or when you need full control over parsing.

Given a request of the form:

POST /webhook?source=regression&campaign_id=abc123
Content-Type: application/json

{"event_id":"evt_001","customer_email":"[e-mail]","amount":"49.99"}

The following SQL extracts the relevant fields:

create table output.events engine Log() as
select
    fromUnixTimestamp64Milli(toInt64(timestamp))     as timestamp,
    employee_id,
    JSONExtractString(body, 'event_id')              as event_id,
    JSONExtractString(body, 'customer_email')        as customer_email,
    JSONExtractFloat(body, 'amount')                 as amount,
    extractURLParameter(querystring, 'source')       as source,
    extractURLParameter(querystring, 'campaign_id')  as campaign_id
from
    input.webhooks_raw;

Note: In this payload, amount is sent as a string ("49.99"). JSONExtractFloat is used to cast it to a float on extraction. Always cast extracted values to the type your output table and execution variables expect.

---

Capturing Sample Webhooks for Development

Before activating a Webhook SmartConnector and running executions against live Kizen Records, use the inactive state to capture and inspect real webhook payloads. While the SmartConnector is inactive, inbound requests are queued but not processed, making this a safe environment for development.

Follow these steps to capture sample data:

1. Leave the Webhook SmartConnector in the inactive state.
2. Send sample HTTP requests to the webhook URL.
3. Open the executions list and locate the queued execution holding your sample requests.
4. Download the queued execution's input file. Use this file as a SQL test input to develop and validate your query against real payload data.
5. Before activating the SmartConnector, cancel the queued execution so that your sample data is not processed against live Records.

Note: Cancel the queued execution before activating the SmartConnector. If you activate without cancelling first, the queued execution will run and process your sample data against live Kizen Records.

---

Failed Batch Reprocessing

If an execution fails or produces incorrect output, you can reprocess the original batch of webhooks without re-sending the original HTTP requests.

From the executions list, download the input file from the failed execution. This file contains the exact set of webhooks that were processed. Re-upload it to trigger a new execution against the same data.

This is particularly useful when the failure was caused by a SQL error or a mapping misconfiguration rather than a problem with the inbound payload data. Correct the issue, then reprocess the original file to confirm the corrected behavior.

---

Variables, Load Steps, and Post-Processing

Once your SQL runs, all subsequent behavior is identical across SmartConnector types. Execution variables, load steps, field mapping, and Agentic Workflow triggers work the same way for a Webhook SmartConnector as they do for a Spreadsheet SmartConnector.

---

What's Next

You now have a working understanding of how Webhook SmartConnectors receive, queue, and process inbound data. The next step is to create one.

Continue to Configuring Webhook SmartConnectors for a step-by-step walkthrough of the configuration wizard.

Related Topics

• SmartConnector Settings
• SmartConnector Execution Variables
• SmartConnector Load Steps
• SmartConnector Diff Checking