Understanding batch data files

This section describes the batch data files used for the batch apply process. The following subjects are described:

  • Required import order
  • Data files
  • Control files

Required import order

Sample batch data may be used as a model for retailer-specific data. In the sample batch data, there are three groups of data: Taxonomy, User, and Product. Each group contains specific JSON files for batch apply. The batch apply process uses the Batchapply API, and the control file contains the details required to apply the files in the required order. Data must be applied in a specific order due to data dependencies throughout the process.

Data must be loaded into the database in the following order:

  1. Taxonomy
    • Enterprise taxonomy
  2. User
    • Roles
    • Users
    • User roles
  3. Product
    • Tax types
    • Tax location
    • Taxes
    • Tax class tax rules
    • Tax class tax types
    • Unit of Measure (UOM) details
    • Age codes
    • Groups
    • Vendors
    • Restrictions
    • Hierarchy levels
    • Hierarchies
    • Products
    • Linked products
    • Product age codes
    • Group details
    • Product prices
    • Product org features

Data files

Batch data files are JSON-formatted batch files that contain data records to be inserted into the database tables.

Format

Batch data files have the following format:

Copy

JSON

{
    "batchId": "<batchId>",
    "originalFilename": "<batch data filename>",
    "action": "<action value>",
    "records":
 [
    {
        "dataType": "<datatype value>", "data":
        {
            "<table field name>": "<value>", "<table field name>": "<value>">
        }
    },
    {
        "dataType": "<datatype value>", "data":
        {
            "<table field name>": "<value>", "<table field name>": "<value>">
        }
    },
 ]
}

Data

Batch data files contain the following data.

DataDescription
batchId

Specifies an alphanumeric ID used to identify and track the batch file.

Length: 1–20

Note

The batchId is optional. If specified, the batchId in the data file should match the batchId in the corresponding control file. Otherwise, the system automatically generates a batchId for the control file and checks the originalFilename in the data file.

originalFilenameSpecifies the name of the data file.
action

Indicates whether the data should be added to, updated, or deleted from the database. The following are the possible actions:

  • add—inserts new data to the database. This action restricts duplicate records from being added to the database.
  • update—modifies existing data in the database. This action allows duplicate records to be added, but prompts a warning to the user.
  • delete—removes existing data from the database.
Note

The update and delete actions cannot be used with certain record data types. If the data cannot be deleted by batch, then remove the data directly from the database tables.

recordsContains the data records to be added, deleted, or updated.
dataType

Indicates the type of batch data file. A data file can contain multiple data types.

Following are the base dataType values:

  • Roles
  • Users
  • UserRoles
  • TaxTypes
  • Taxes
  • TaxClassTaxTypes
  • TaxClassTaxRules
  • TaxonomyLevels
  • TaxLocation
  • Taxonomies
  • UOM
  • AgeCode
  • Groups
  • GroupDetails
  • Vendors
  • Restriction
  • Hierarchies
  • HierarchyLevels
  • Products
  • ProductPrices
  • ProductAgeCode
  • ProductVendor
  • WarrantyAddOn
  • WarrantyEligible
  • PreOrder
  • RequiredFeeItem
  • LinkedItem
Note

The dataType value must match a supported value for the data to be processed.

Control files

Control files are JSON-formatted text files that define when and in what order the batch data files are imported. These files can be opened using any text editor, such as Windows Notepad. The content of the control files can be copied and pasted into the NCR DSR Web API page.

Note

Control files are optional, and can be used as a convenient shortcut.

Format

Batch control files have the following format:

Copy

JSON

{
  "batchId": "<batchID>",
  "applyDateTime": "<Coordinated Universal Time (UTC)>",
  "fileCount": "<batch data file count>",
  "fileType": "<thread name>",
  "fileFormat": "application/json",
  "filesList": [
    {
      "fileOrder": "<order number>",
      "fileName": "<batch data filename>",
      "recordContainsInFile": "<record count>",
      "checkSumValue": "<checksum hash algorithm-SHA256 value>"
    },
  ],
  "batchChecksumValidation": "<Yes or No>",
  "autoStoreDataLoad": "<DataLoadAction>"
}

Data

The Control file contains the following data:

DataDescriptionRequired
batchId

Contains an alphanumeric unique ID used to identify and track the batch file.

Length: 1–20

Note

The batchId is optional. If specified, the batchId in the control file should match the batchId in the corresponding data file. Otherwise, the system automatically generates a batchId for the control file and checks the OriginalFilename in the data file.

No
applyDateTime

Indicates the date and time (UTC) to process the Control File.

  • If the applyDateTime is current, or in the past, the batch application processes the data at the next polling interval.
  • If the applyDateTime is in the future, the pre-check process occurs at the next polling interval; however, the batch remains pending with a Pre-checked failed status or Future status until the date reaches the current date.
Yes
fileCountIndicates the total number of payload files included in the control data.Yes
fileType

Specifies the thread to use to apply the files so that multiple files can be processed simultaneously.

The following file types are used in the control file:

  • taxonomy—loads data relating to the organizational structure.
  • user—loads data relating to the users.
  • product—loads data relating to the Product Catalog.

The batch maintenance feature can handle threads organized by the type of data being loaded. Different threads can be processed simultaneously; however, threads of the same file type are processed one at a time.

Yes
fileFormat

Indicates the format of the data file.

Note

The Advanced Store solution supports the application/JSON file format.

Yes
payloadType
Note

This data is reserved for future functionality.

No
filesList

Contains the list of data files and the order in which to load the files. The control file can point to multiple data files.

The following File List data is included in the control file:

  • fileOrder—specifies the sequence for the files to be applied.
  • filename—indicates the name of the payload file.
  • recordContainsInFile—specifies the total records contained in the file for validation.
  • checkSumValue—specifies the checksum value using SHA256 hash algorithm for validation.
  • batchChecksumValidation—enables or disables checksum validation. By default, the value for this data is set to No. To enable checksum validation, set this value to Yes.
Yes
batchCheckSumValidationEnables or disables checksum validation. By default, the value for this data is set to No.No
autoStoreDataLoad

Indicates what data should be pushed to the store automatically through the StoreDataLoad process after the Batch Apply process has finished.

Note

This field and value are optional. When the field is omitted from the control file, has no value specified, or has a value of NULL, zero, or greater than 3, the software will behave as it did in previous releases.

Valid values:

  • 0 or NULL—send only updated data down to the affected store(s). This is the default behavior if the parameter is not included.
  • 1—send all data down to the affected store(s) after the batch is complete. This is the equivalent of running the Store Data Push Utility.
  • 2—do not send any data down to the store(s). No message is published to the queue; data is only inserted into the database.
No