Validating custom fields

When importing entities into the product catalog, Newstore validates all imported entities.

However, some of the product fields can contain properties like additional_attributes or external_identifiers, the values of which NewStore cannot validate other than explicit validation rules. For example, if a number has been entered for a field additional_attributes that only accepts strings, NewStore will return an error. However, no other validation rules can be applied to check if a string entered as the value for the field is valid.

For retailers, these arbitrary fields can hold information critical for selling activities. For example, external_identifiers contain the ean value that are used to scan items when selling them. If an ean is missing or invalid, the item will not be found by the scanner.

To prevent such cases Newstore provides an option to attach a custom schema for each imported product.

{
    "head": {
        "locale": "en-us",
        "shop": "storefront-catalog-en",
        "is_master": true
    },
    "items": [
        {
            "product_id": "TEST_EGE",
            "validation": {
                "schema_id": "ISd8f52b9f83c360cbc1c5"
            },
            "external_identifiers": [
                {
                    "type": "sku",
                    "value": "TEST_RDYWUNVSRCTKFQK"
                },
                {
                    "type": "ean0",
                    "value": "34362890646072"
                }
            ],
            "variant_group_id": "TEST_STYLE_UK",
            "extended_attributes": [
                {
                    "name": "material",
                    "value": "Steel"
                },
                {
                    "name": "gender",
                    "value": "male"
                }
            ]
        }
    ]
}

In the example above, every product contains a validation object that holds the schema ID. It is used as a reference to the schema for product validation.

Important

The schema must exist in the platform before you actually run an import with the attached schema. Missing validation schema will cause the product import to fail.

Creating a custom schema

You can create a custom schema via the Catalog Schema API. You can also find a list of fields to use for validation and more details about the format of the schema.

JSON Schema is an industry standard. You can use various free tools to check the schema before deploying to production.

A custom created schema checks for:

• An existing external_identifiers property
• The external_identifiers property must contain 2 items
• The external_identifiers property must only contain unique items
• 1 item must be sku with the value being a minimum length of 1 and maximum 255
• 1 item must be ean with the value being a minimum length of 1 and maximum 255

curl --location 'https://{tenant}.p.newstore.net/catalog/import-schemas' \
--header 'Content-Type: application/json-schema' \
--header 'Authorization: ••••••' \
--data '{
    "$schema": "http://json-schema.org/draft-07/schema#",
    "type": "object",
    "required": ["external_identifiers"],
    "properties": {
        "external_identifiers": {
            "type": "array",
            "minItems": 2,
            "uniqueItems": true,
            "items": {
                "oneOf": [
                    {
                        "type": "object",
                        "required": ["type", "value"],
                        "properties": {
                            "type": {
                                "type":"string",
                                "minLength":1,
                                "pattern": "^sku$"
                            },
                            "value": {
                                "type":"string",
                                "minLength":1,
                                "maxLength": 255
                            }
                        }
                    }, {
                        "type": "object",
                        "required": ["type", "value"],
                        "properties": {
                            "type": {
                                "type":"string",
                                "minLength":1,
                                "pattern": "^ean$"
                            },
                            "value": {
                                "type":"string",
                                "minLength":1,
                                "maxLength": 255
                            }
                        }
                    }
                ]
            }
        }
    }
}'

This API call responds with the schema ID, which you can then use in your import jobs.

{
    "id": "ISd8f52b9f83c360cbc1c5",
    "last_updated_at": "2024-08-02T14:02:13.416604803Z"
}

Important

Authorisation token for this request requires two additional scopes.

• catalog:import-schemas:read
• catalog:import-schemas:write

Schema updates are reflected in import jobs after 1 hour. Take this into consideration when making updates to the schema. These updates may not be reflected immediately.

Checking the import status

After you have created and deployed a custom schema, you can check the status of the import.

The import fails if any of the products do not match against the schema. You can find the reason for failure in the error responses of an import. Access these responses via the Import API.

curl --location 'https://{tenant}.p.newstore.net/v0/d/import/89b70a65-2413-4ecd-8e0b-42cc838f19e0/errors' \
--header 'Authorization: ••••••'

This call returns the errors:

{
    "items": [
        {
            "created_at": "2024-07-23T07:48:04.350065Z",
            "entity_id": "TEST_EGE",
            "entity_type": "products",
            "reason": "Field: external_identifiers.1, Must validate one and only one schema (oneOf);
                       Field: external_identifiers.1.type, Does not match pattern '^sku$'"
        }
    ]
}

In the above example, you can see the message containing a validation error.

In this case element 1 in the array (arrays are indexed from 0, and at 0 we have sku, at 1 we have ean0) does not match any of requested patterns either for sku or for ean.

To resolve the error, the external identifier at index 1 must be changed to type ean and not ean0.