Dataloader Pro

Dataloader Pro is an advanced feature ARM provides for transferring data from the source sandbox to the destination sandbox more conveniently and automatically handles the parent-child relationship. Migrating the Salesforce data/objects to more than one object-supporting hierarchy can be easily achieved using the Dataloader Pro feature in ARM.

Before you Begin

While performing Dataloader Pro on the objects for the first time, ensure you perform Dataloader Configuration among the same orgs on all the objects included in your Dataloader Pro job. This is a one-time operation.

Dataloader plays an essential role in data migration from source sandbox to destination sandbox. However, in this data migration process, the chances of duplicate records being created always exist. To avoid this, ARM has developed a new feature that allows synchronizing between the orgs with the help of the ARM external ID AutorabitExtid__c field.

How to perform a Dataloader Pro operation

1. Log in to your ARM account.

2. Hover your mouse over the Dataloader module and select Dataloader Pro.

3. Click on Create New Job.

   
4. On the next screen, choose the Source Org and the Destination Org that automatically populate the selected sandbox's details.

5. Click Login and Fetch Objects.

   
6. Next, Select Master Object.

   
7. View the relationship between child objects/ancestor objects and the master object in the Schema (Grid View) section.

   
8. Change the grid view to a graph view by clicking the Switch to Graph View button. Click on the icon to view the graphical representation on full screen.

   

Filters and Mappings

For each object displayed, the user can view the list of fields related to the corresponding object.

A. Filters

You can extract records within a specified limit using specifying criteria in the Filters section. You can even filter the details via Date or Date Literals. A date literal is a fixed expression representing a relative range of time, such as last month, this week, or next year. (Refer here for the list of data literals supported)

Upload the CSV file (if required) if there is a large amount of data and requires the filter for those records. The max file size supported is 10 MB. Once the CSV is uploaded, click Auto-Populate. The filters are auto-populated for the selected field and operator based on the values of the chosen field in the uploaded CSV file.

Format for CSV file to filter records:

• ARM Dataloader Pro accepts CSV (comma-separated values) files. Use a spreadsheet program such as Microsoft Excel to create your CSV file.

• Ensure you have a column header and rows of data populated for all system-required fields, such as Account Name or Contact Last Name.

• There can be only one column header.

For more information, see Preparing the CSV file for Dataloader.

The Record Count Limit box limits the number of records extracted from the source, giving a value in this field.

Click Validate to fetch the number of records transferred from the source sandbox to the destination sandbox. Finally, click Apply.

To skip migrating an object, click on the Skip Records icon and then click on Set Records to 0.

Click Yes on the confirmation screen. If there is an existing record limit value, it will be overridden, and the new value will be set to 0.

B. Mappings

Map the object fields between the source and destination sandboxes.

Using the Auto-map feature, you can map the fields automatically based on fetched object fields with destination fields. To set up manual mappings, the auto mapping needs to be disabled. Click on Clear Mappings to remove the automapping and set up the desired manual mappings.

External Id Field Mapping

In this section, you can use an external ID instead of a related record's Salesforce record ID to relate or associate records to each other as you process the Dataloader Pro operation. For example, if Object B has a lookup field to another Object A, you can use the values in a field marked as an External ID on Object A to relate the two (Object B to Object A records).

In the Source field, select the source whose values will be populated in the destination external ID field.

In the Destination field, select the required destination org whose values will remain unique for all the records.

Important Note: ARM does not support the automatic creation of an ExternalUniqueID. The user has to create this manually on both the Source Org and Destination Org.

Process Details

Here in this section, fill in the process details listed below:

1. Give the job a Name.

2. Select the category in the Job Group field. This is important if you'd like to group related jobs into a single category. You can also create a new group and assign your job to this group.

3. Master Object and External ID are auto-populated.

4. Enter the User Name Suffix for the Source Org and the Destination Org. Below are some examples of usernames and the corresponding suffixes:

   Case 1	User Name	User Name Suffix
   Source	[e-mail]	src
   Destination	[e-mail]	dest

   Case 2	User Name	User Name Suffix
   Source	[e-mail]	qan.com
   Destination	[e-mail]	aws.com

   Case 3	User Name	User Name Suffix
   Source	[e-mail]	Empty (leave it blank)
   Destination	[e-mail]	dest

   Case 4	User Name	User Name Suffix
   Source	[e-mail]	src
   Destination	[e-mail]	Empty (leave it blank)

5. Additionally, users can ignore certain records related to community users by selecting the Ignore Community Users checkbox.

   

Masking Wizard

Data Masking refers to changing certain data elements within a data store so the structure remains similar while the information is altered to protect sensitive information. It ensures sensitive customer information is unavailable beyond the permitted production environment.

Under the Masking Wizard section, click on New to add a masking rule.

In the Masking Form screen, do the following:

1. Select Object for masking and choose the Field Type.

2. Choose the Masking Style:

   1. Prefix: This option adds the character before the selected field name. For example, if the field value in source org is ABC, and the prefix masking value is kept as 123, then the final field value being deployed will be 123.ABC.

   2. Suffix: This option adds the character after the selected field name. For example, if the field value in source org is ABC, and the suffix masking value is kept as 123, then the final field value being deployed will be ABC.123.

   3. Replace: This option replaces the selected field name with the character you enter in the Masking Value field. For example, if the field value in source org is ABC, and the replace masking value is kept as 123, then the final field value being deployed will be 123.

   4. Shuffle: Shuffles the data in the column (like a deck of cards) and leaves the other columns untouched. For example, if the field value in source org is ABCDE, then the final field value being deployed will be DCBEA.

   5. Generate Random: This option helps mask the original value with a random value within a specified range. For example, if the field value in source org is ABC, and the random string length value is set to 7, then the deployed field value would be similar to 15d3aRG.

Important Note:Masking is not applicable if the field value for the record is empty.

Process Schedule

In the Scheduling procedure, the user can schedule the process it must run.

1. Daily: The process will run every day at the scheduled or set time interval.

2. Weekly: The process will run weekly on the scheduled day and time.

3. No schedule: The process is only saved; you can run it when required.

   

Finally, click on Save to complete the initial process. You will be redirected to the Dataloader Pro Summary page, where the dataloader process initiated can be seen at the top of the list.

Running the Dataloader Pro Job

Select your job from the Dataloader Pro Summary screen and click on Run. This option allows you to run the processes created in the selected category.

The table below lists the configurations to choose from, along with their descriptions:

Serial Number	Configurations	Description
1	Disable workflows	The workflows of the Salesforce objects are deactivated, and the data is transferred from the source to the destination sandbox. Once the migration is complete, workflows are reactivated.
2	Disable validation rules	Validation rules verify that the data a user enters in a record meets the criteria you specify before the user can save the record. On selection, all the validation rules of the Salesforce objects are deactivated, and the data is transferred from the source to the destination sandbox. Once the migration is complete, validation rules are reactivated.
3	Use Bulk API	The Bulk API is based on REST principles and optimized for inserting, updating, and deleting large data sets. You can use the Bulk API to process jobs in serial or parallel mode. Processing batches serially means running them one after another, while processing batches in parallel means running multiple batches simultaneously. Note: When you run a Bulk API job, processing more batches in parallel means giving that job a higher degree of parallelism, providing your overall run better data throughput.
4	Insert/update with null values	Insert/update with null values
5	Minimize multiple references between the same objects	When there are multiple references between the same objects, unnecessary API calls are not triggered upon selecting this option.
6	Automatically apply master object filter on other dependency objects	All objects in the hierarchy are calculated based on the Master Object filters; this option avoids extra records addition due to self-references and multiple references.
7	Data encryption for data files	Data encryption for data files
8	Incremental data migration	After a dataloading process is done, only the newly added records are transferred into the destination sandbox.

More Options

1. 1. Edit: Edits the processes in the selected category to rerun them.

   2. Abort: Aborts the process.

   3. Schedule: Schedules the data loading process for the selected category.

   4. Clone: Clones the respective Dataloader Pro job.

   5. Log: Provides information about the process execution.

      

Schema (Grid View)

This section lets you view the master object and its related information for the test environment setup job. For the setup run with disabled validation/workflow rules, ARM lists all the validation rules set under VR/WFR section. The UI lists all the workflow/validation rules; users must enable them for the disabled rules (if required).