Validate Data Structure/Schema v1.0.0 Help ​

Validates the structure and values of the input data against the defined validation rules.

How can I use the Step? ​

The Step lets you validate the structure and values of incoming data. You can use the Step to validate data payloads from your Flows or external systems to ensure compliance with data schema requirements and enforce data integrity.

How does the Step work? ​

You provide a data payload using the Merge field, define input data structure, and set validation rules. The Step applies validation rules to the incoming data object to check for compliance and returns the validation result. For a comprehensive understanding, refer to the Output example.

Input settings ​

To set up this section, follow these steps:

  • For Data to validate, input a Merge field variable containing the payload data to be validated.
  • For Data structure example, enter the structure of the incoming payload, adhering to the JSON syntax. The field auto-populates if there's an output example for the Merge field in the selected Step.

For manual data structure input, define properties and their types as follows:

  • For text type:
json
{ "property_name": "string" }
{ "property_name": "string" }
  • For boolean type:
json
{ "property_name": "boolean" }
{ "property_name": "boolean" }
  • For number type:
json
{ "property_name": "number" }
{ "property_name": "number" }
  • For date type:
json
{ "property_name": "date" }
{ "property_name": "date" }
  • For object type:
json
{
    "property_name": {
       "nested_property": "string"
    },
    "property_name_1": {},
    "property_name_2": "object"
}
{
    "property_name": {
       "nested_property": "string"
    },
    "property_name_1": {},
    "property_name_2": "object"
}
  • For array type:
json
{
    "property_name": [
       "string"
    ],
    "property_name_1": [],
    "property_name_2": "array",
    "property_name_3": [["array"],["array"]]
}
{
    "property_name": [
       "string"
    ],
    "property_name_1": [],
    "property_name_2": "array",
    "property_name_3": [["array"],["array"]]
}

Validation rules ​

The validation rules enable you to check and enforce the data structure of the payload. You can add multiple groups of validation rules with specific conditions that meet your task.

Adding and configuring groups of rules ​

To add and configure a group of validation rules, follow these steps:

  1. To add a new group of validation rules, click +Add New Group. The first group of validation rules is added by default.
  2. Name the group (the default is New Group 1).
  3. Optionally: Add a condition description for the group.
  4. For Match type, select one of the following match types to decide how to evaluate the group's rules:
    • any (default): If at least one rule in the group passes validation, the group is considered valid.
    • all: All rules in the group must pass validation. If even one rule fails, the group is deemed invalid.

Setting up rule conditions ​

To set up a condition, do the following:

  1. From the list of properties, select the property to validate. The list of properties auto-populates based on the Data structure example input.
  2. Change the rule type if needed. By default, it is automatically updated based on the selected property type.
  3. From the comparisons list, select a value-comparison operation. If the chosen operation requires a second comparison value, enter this value or set it via a Merge field.
  4. For Error message, enter a custom error message. You can do it manually or via a Merge field. This field is required.
  5. Optionally: Turn off Auto-detect properties and enter the name or path of the property you want to validate. For the path, use the name.first format. To denote each item of an array, use _each, e.g., tasks._each or tasks._each.label. The property name you enter overwrites any auto-detected property name.

Rule types ​

You can use the following validation rule types:

  • General: Common rules applicable to any value.
  • Text: Rules for string type values.
  • Number: Rules for number type values.
  • True/False: Rules for boolean type values.
  • List (array): Rules for array type values.
  • Object: Rules for object type values.
  • Date: Rules for date type values.
  • Advanced: Option to enter validation rules in JavaScript syntax. If the code execution returns true, the data structure is valid, and vice versa.

Warning: Property names are not auto-detected for Advanced rules. If the Advanced rule is the single rule in the group, you must turn off Auto-detect properties and manually enter the property name. Otherwise, the Step throws an error.

Unexpected parameters handling ​

Unexpected parameters are properties of the incoming data that were not defined as related to any validation rule.

To manage unexpected parameters, follow these steps:

  1. Select one of the handling options:

    • Allow all at any depth: Allows all unexpected parameters at any depth.
    • Disallow all at any depth: Disallows all unexpected parameters at any depth.
    • Disallow in root: Disallows unexpected parameters at the root level.
    • Disallow at depths less than: Disallows unexpected parameters at depths less than the specified depth. For this option, you also must provide the depth number, where 0 corresponds to the root level.

  2. Optionally: Turn on Add property-specific handling to specify the handling option for particular properties of input data.

  3. Optionally: Turn on Include unexpected parameters in output data to include properties without validation rules in the Step's output.

Note: Allowed parameters do not affect the summary result. Disallowed parameters interpreted as invalid affect the summary result, even if expected parameters pass validation.

Advanced settings ​

The section contains the option for automatically generating API documentation:

  • Use settings for API docs generator: Includes the Step's configuration in the automatic API documentation generator for HTTP flows.

Merge field settings ​

The Step returns the result as a JSON object and stores it in the Merge field variable. So you can use the retrieved data in subsequent Steps or other Flows.

Output example ​

The following output example demonstrates the structure of the JSON object returned by the Step:

json
{
    "overall": {
        "valid": false,
        "errorCount": 1,
        "invalidParamCount": 1,
        "invalidParams": [
            "phone"
        ]
    },
    "byParam": {
        "property_name": {
            "__valid": false,
            "__path": "property_name",
            "nested_property": {
                "__valid": false,
                "__path": "property_name.nested_property",
                "__errors": {
                    "count": 1,
                    "all": {
                        "0": {
                            "message": "error message",
                            "rule": {
                                "name": "rule name",
                                "description": "rule description",
                                "key": "ruleName",
                                "relatedParams": [
                                    "nested_property"
                                ]
                            }
                        }
                    },
                    "first": {
                        "message": "error message",
                        "rule": {
                            "name": "rule name",
                            "description": "rule description",
                            "key": "ruleName",
                            "relatedParams": [
                                "nested_property"
                            ]
                        }
                    }
                }
            },
            "byRules": {
                "ruleName": {
                    "message": "error message",
                    "rule": {
                        "name": "rule name",
                        "description": "rule description",
                        "key": "ruleName",
                        "relatedParams": [
                            "nested_property"
                        ]
                    }
                }
            },
            "unexpectedParams": [
                {
                    "__valid": false,
                    "__path": "property_name"
                }
            ]
        }
    }
}
{
    "overall": {
        "valid": false,
        "errorCount": 1,
        "invalidParamCount": 1,
        "invalidParams": [
            "phone"
        ]
    },
    "byParam": {
        "property_name": {
            "__valid": false,
            "__path": "property_name",
            "nested_property": {
                "__valid": false,
                "__path": "property_name.nested_property",
                "__errors": {
                    "count": 1,
                    "all": {
                        "0": {
                            "message": "error message",
                            "rule": {
                                "name": "rule name",
                                "description": "rule description",
                                "key": "ruleName",
                                "relatedParams": [
                                    "nested_property"
                                ]
                            }
                        }
                    },
                    "first": {
                        "message": "error message",
                        "rule": {
                            "name": "rule name",
                            "description": "rule description",
                            "key": "ruleName",
                            "relatedParams": [
                                "nested_property"
                            ]
                        }
                    }
                }
            },
            "byRules": {
                "ruleName": {
                    "message": "error message",
                    "rule": {
                        "name": "rule name",
                        "description": "rule description",
                        "key": "ruleName",
                        "relatedParams": [
                            "nested_property"
                        ]
                    }
                }
            },
            "unexpectedParams": [
                {
                    "__valid": false,
                    "__path": "property_name"
                }
            ]
        }
    }
}

Error handling ​

By default, the Handle error toggle is on, and the Step handles errors with a separate exit. If any error occurs during the Step execution, the Flow proceeds down the error exit.

If the Handle error toggle is disabled, the Step does not handle errors. In this case, if any error occurs during the Step execution, the Flow fails immediately after exceeding the Flow timeout. To prevent the Flow from being suspended and continue handling errors, you can place the Flow Error Handling Step before the main logic or your Flow.

Reporting ​

After the Step completes, it generates a report that includes its execution status and other details. You can customize the report by adjusting the Step's log level and adding tags.

Log level ​

By default, the Step's log level matches that of the Flow. You can change the Step's log level by selecting an appropriate option from the Log level dropdown.

Tags ​

Tags provide a way to classify and search for sessions based on their attributes. To create a new tag, specify its category, label, and value. You can then use tags to filter and group the sessions in the report.

Service dependencies ​

  • flow builder - v2.62.1
  • event-manager - v2.0.1
  • deployer - v2.10.0
  • library v2.11.3
  • studio v2.64.1

Release notes ​

v1.0.0

  • Initial release

v0.1.3

  • Fix unexpected parameters handler (add all parameters to expect, even when the rule was not processed, rely on Match type)

v0.1.2

  • Fix unexpected parameters handler (Disallow at depths less than option)

v0.1.1

  • Update Condition Builder v1.0.3.
  • Fix the Advanced rule issue to use variables that do not exist but are expected in the incoming payload

v0.0.22

  • Update default value of Data structure example
  • Check the output example of the selected Merge field (Data to validate) and, if it exists, use it as an initial value for Data structure example
  • Hide Data structure example before Data to validate has being selected

v0.0.21

  • Fix Use specific settings for separate parameters in the step logic
  • Minor UI fixes

v0.0.18

  • Fix Detect related params wildcard logic