Data exchanged between systems must be structured so it can be processed automatically, without manual intervention. When the source and destination data are organized in the same way, this is easily accomplished. When they differ, Integration Studio can transform the data by adding or removing hierarchy, changing how the data is organized.
For example:
- The GET Time Data and Quick Entry APIs are based on time entry records or quick entries. Each record in the response represents one time entry or one quick entry. There is no nesting or hierarchy.
- The Employee Bulk and Ongoing Benefit Carrier Export APIs are based on an employee. Each record in the response represents one employee. Employee data, such as Work Assignment and Employment Statuss, are nested inside the main data array.
This means each API response is organized around a specific “unit” (time entry, quick entry, employee, etc), and every record in that response follows that same structure. The top level array with this information is referred to as the main data array.
Adding and removing hierarchy changes the “unit” that the data is organized around. For example, grouping quick entry data by employee or work location or flattening the employee bulk API so that each WorkAssignment is represented alongside employee data.
Note: For inbound integrations, the main data array must be defined in source setup.
For more information about structuring your data, see Working with APIs and The Importance of Understanding Your Data Structure
Grouping Fields
Important: For best results, configure as much of your integration as possible before you enable grouping. This includes setting up the destination structure, destination field formats, and field mappings.
Some target systems require the output file to follow a specific hierarchy, which may differ from the source file.
For example:
- A payroll report integration might need data to be grouped by employee before it’s sent to the vendor.
- An accounting system might require a GL summary file grouped by region, even though the source data is organized by employee.
After you define the destination structure, you can turn on the grouping functionality on an appropriate parent array. Depending on what you’re reporting, you may also need to configure grouping details on the child fields.
To enable grouping in Integration Studio:
- In your mapping step, find the parent field you want to group by.
- Click the Action icon (vertical ellipsis) next to that parent field.
- Click Edit.
- In the Edit <Field Name> Element dialog box, turn on the Is Array switch if disabled.
- Turn on the Group Fields by Key switch.
- Click Save.
When grouping is enabled, all child fields under a grouped parent and all child fields of an object parent that is below a grouped parent field are either Keys or Totals. You can then review and adjust those settings as needed. By default, all nonnumeric fields default to Keys, and all numeric fields that use Aggregation as their source default to Totals. All other numeric fields are defaulted to Keys. These designations are shown in Integration Studio for your review and can be changed, if needed.
If a numeric Key field should be a Total, update its source to Transformation, Conditional Mapping, Token, or Expression as appropriate.
The Transformation, Conditional Mapping, and Expression configuration screens include a Mark as Total toggle. This toggle is enabled by default for Sum transformations and disabled by default for all other transformations including expressions and conditionally mapped fields.
To change a field's designation after configuration, click the Action icon beside the field and select Mark as Total or Mark as Key.
You can also use custom tokens as grouping totals for fields that are configured to use Number as their Data Type and Token as their Source.
Important: Keys cannot be configured for a source that uses Aggregation. However, Keys that are configured for a source other then Aggregation must result in a single value. For example, a key configured as the concatenation of an employee’s work assignment location and badge number would cause the integration to fail if multiple work assignments exist in the source and a Choose Record filter was not applied to limit the key to a specific work assignment. Your integration can also fail or generate unexpected results when the Keys and Totals are not properly assigned.
Flatten Your Data (Remove Hierarchy)
Some target systems require a flat file with no hierarchy, but your source data might be organized by employee or another parent level. After you define the destination structure, you can enable flattening on the top‑level array to remove the hierarchy and then map the fields you need.
To flatten your data in Integration Studio:
-
Create the required destination structure, including:
- Fields that will be mapped from the array you want to flatten.
- Any other fields or arrays required by the vendor.
- On the top‑most array parent, change the Source from Source Array to Flattened Array.
- In Source Details drop-down list, select the array you want to flatten.
The selected array is flattened into the main data array.
For example, if you are using the Employee Bulk API to flatten the EarningElections array into the main data array so that employee name and election details can be reported without nested JSON, you would do the following:
- Create the destination structure with the required fields.
- Change the Source of the top-most array parent to Flattened Array.
- Select EarningElections in Source Details drop-down list.
How flattening affects your data
Flattening an array into the main array increases the number of items in the main array.
Continuing the Employee Bulk API example:
- The main data array is in the context of an employee (one employee per object).
-
If your source has 2 employees:
- Employee A with 3 elections
- Employee B with 2 elections
Then, after flattening EarningElections to the main data array, the result contains 5 items—one for each unique Employee + Election combination.
Limitations
Only one array can be flattened at a time. If you need to flatten multiple arrays, create additional maps and flatten each array in its own map. If there are no arrays, create additional maps and flatten each array in its own map. No options are shown in the Source Details drop-down list when you select Flattened Array as the source and there are no arrays in the source.
Note: Flattened arrays can’t be used with grouping or parent‑level expressions. If you need grouping or parent‑level expressions after flattening, use linked maps to perform those operations in a separate step.
Multiple Structural Changes with Linked Maps
Some integrations require several structural changes to the raw source data to meet vendor requirements. You can define multiple maps in a single integration to make apply iterative mapping changes to the data so that the data is in the format and structure required by the consuming system.
When you create a new integration, the mapping step only contains a single final map, which defines the output file structure. When you add another map, it becomes an intermediate map that transforms the source data and passes its output to the text map. So, the output of the intermediate map becomes the input for the text map. The final map always defines the final file or payload structure.
Linked maps are required when the source data of the integration can only be transformed into the target vendor’s specifications through multiple, iterative transformation steps.
For example, source data must first be flattened when hierarchical, and once flattened can only then be grouped. Similarity, only one array can be flattened at a time. Understanding the structure of your source data and the structure of your destination data is required to determine whether you need linked maps in your integration configuration. This is discussed in more detail in the Real World Examples in the Field Availability Across Maps section.
For more information about data structures, arrays, and the significance of the main data array, see Working with APIs and The Importance of Understanding Your Data Structure
The following table outlines some high-level scenarios that would require linked maps:
| Task | Procedure |
|---|---|
|
Flattening, then Filtering |
Flatten an array to the main data array, then filter based on the flattened array |
| Flattening, then Grouping | Flatten an array to the main data array, then group the flattened data based on one or more fields pulled from the sub-array. |
| Flattening Array1, then Flattening Array2 | Flatten an array to the main data array, then flatten another array to the main data array. |
| Consecutive Grouping | Group the data by a value that exists within the main data array, then group the output by another value that exists in the new main data array. |
Important: All intermediate maps produce JSON output. The output type you select in the Define the Parameters section of the configuration only applies to the final map.
Add an Additional Map
To add an additional map in Integration Studio:
- In the mapping step, click Add Mapping.
- Click the Action icon beside the new map and then click Edit.
- In the Mapping Summary dialog box, enter a name for your map in the Mapping Name field and description for your map in the Mapping Description field.
- Note: The name entered into the Mapping Name field shows in the error logs when that map fails at runtime.
- Configure the fields for the first intermediate map, then continue configuring each intermediate map in order, ending with the final map.
- Note: You can reorder intermediate maps using Move Up and Move Down from the action icon.
Only one mapping step can be open at a time and each map has its own:
- Tokens
- View IDL screen and IDL mapping
- Schema
- Destination file
If needed, you can convert individual maps to full IDL while leaving the remaining maps in the UI, which gives you the flexibility to create complex integration logic using IDL where needed and the freedom to use the UI.
The final mapping step opens automatically when you access the integration.
How Sources Work Across Maps
When your integration only contains one map, its source is the selected API. Once you add an additional map, the first map’s source is still the selected API; however, each subsequent map uses the output of the previous intermediate map as its source data.
When you rename fields or change field settings (for example, data type) in an intermediate map, you may need to update mappings in the maps that follow.
Important: When the final map already has configured fields and you add a new intermediate map, those field configurations are cleared. You’ll need to remap the fields after the intermediate map is configured. You can save the integration with partially configured maps and invalid mappings, but you cannot run it until mappings are valid.
You also have the ability to auto-fill the destination schema from the source of a mapping step. This helps you to create a destination schema that matches the source schema hierarchy, field names, and data types, which gives you a strong starting point for restructuring source data.
The following auto-fill options are shown in the mapping step:
| Auto-fill Options | Auto-fill Options Availability |
|---|---|
|
Auto-fill from prior mapping step |
The Auto-fill from prior mapping option shows in the mapping toolbar on the final map when the output format is XML or JSON, and on intermediate maps between the first and final map. |
| Auto-fill from API schema | The Auto-fill from API schema option shows in the mapping toolbar on any intermediate map and on the final map when the output type is XML or JSON. When the source is a Dayforce Report, the Auto-fill from Report option shows instead. |
Field Availability Across Maps
Only fields included in the previous map’s output are available to the next map.
For example, if the final map needs to filter by an employee’s home address, that home address must be:
- Present in the first map.
- Carried forward through each subsequent intermediate map, even if it isn’t used until the final map.
The following tables cover some real-world scenarios:
| Scenario One: Send Pay Summary data grouped by Employee Number and Docket Ledger Code |
|---|
|
Connector: WFM Pay Transmit Data Using WFM Pay Transmit Data (Pay Summaries) as the source
Why two maps are required here: Flattening and grouping cannot be configured on the same map, but both are required in order to transform the data to meet target specifications. WFM Pay Transmit Data is in the context of Employees. So, the main data array contains unique values for each employee, such as the Employee Reference Code, First Name, Last Name, etc., and each object in the arrays nested inside the main data array represent additional data. For example, each object in the PaySummaries paysummaries array represents a block of time of an employee’s timesheet. When the target system requires the data to be in the context of data points at different levels of hierarchy in the source, the source data must first be flattened to allow the necessary grouping. |
| Scenario 2: Send Earning Elections per Employee as a flat XML file |
|---|
|
Connector: Build My Own (Outbound) Using the Employee Bulk API as the source:
Why two maps are required here: Because of the hierarchy in the Employee Bulk API, flattening and filtering must be separated into linked maps. If you try to filter on the top‑level array for elections with a specific ShortName before flattening:
By flattening first, each object in the main data array represents a single Employee + Election. The filter then applies to each election, so only elections that match the criteria are included. |