====== Data Import ======

CRM365 allows the import of data organized in special Excel templates for entities of the types //Account//, //Contact// and //Lead//.

There are several caveats when importing data into CRM, therefore, **use great caution when you want to import data!**

**Note: **Importing Datasets is currently limited to a single record type only: It is not possible to import both //Accounts// and //Contacts// in a single import process, and it is also not possible to link an import record to another record such as linking a //Contact// to an //Account//. Basically, when importing //Contacts// or //Leads// they will become **floating** records and need attention lateron.
 
The process of importing data consists of several steps and caution should be taken to make sure that every step is fully completed before commencing to the next step.

Steps:
  - Copy the Import Template
  - Prepare the Source Dataset in the original workbook
  - Copy the Source Dataset to the Import Workbook
  - Import the Data
  - Check for import failures
  - Assign the imported records to a Marketing List

===== 1. Select the proper Template =====

Data in CRM is organized in Fields of various types. In order to eliminate import failures, you first must make sure that you have the right (and recent) Excel template for the record type you want to import/create. The current templates can be found in:

**S:\Sales\CRM\Data_Import\Templates**

Select the template that matches your target entity type and **copy it to a new workbook** in this folder:

**S:\Sales\CRM\Data_Import**

Rename your Workbook to reflect the source or purpose of the data you want to import. (Your workbooks will be uploaded to CRM. You will own a list of all import files, so it is handy to use a filename that matches fi. the Marketing List to which you will assign the imported records).

===== 1A. Prepare the Source Dataset =====

In CRM,  many fields are //Selection fields//: the data you can enter is limited to a list or //Lookup//. The data in those fields in the import dataset need to match those criteria 100%. This may seem trivial, but fi the //Gender// field can either be //Male or Female//, so data like //Mr. of Mrs.// are invalid and need to be converted in the source dataset. 

Also, remember that we use CRM in the English language, so fi all //Country//names need to be in the proper English language.

You can check valid entry values in the Template you copied for //Option sets// and in CRM for //Lookup// fields.


===== 2. Fill Excel =====

If relevant, copy each column of the originating dataset into the corresponding column of the import workbook. **Do not copy and overwrite the column header row!**

The column headers can have one of these color codings:
  * **Red**: This column is mandatory; each cell must contain a value (otherwise the import fails).
  * **Green**: This column is preferred. Most datasets usually contain most of these columns.
  * **Blue**: This column is optional.

**Note: ** Some fields contain an //Option Set//. Only a value from a Drop Down List will be accepted during import or the record fails. Excel will give an error when you paste values in cells that contain //Option Set// vaules that do not exist.
 
==== Mandatory Fields ====

=== Country1Name ===

One important mandatory field that is not obvious to see in the Template is **Country1Name**. This is a Lookup Field. It needs to match exactly with a record in the //Countries// entity in CRM.  The Entity can be found through the main Navigation menu: Sales > Extensions > Countries. 
**The data entered in this field must be copied to the field //Address 1: Country/Region//.**

=== Company ===

This is a text field, unlike the Field **Company Name** which is a //Lookup// field. The latter cannot be resolved during the import and should not contain any data. Instead use the **Company** field. This is used in the Spotler Forms when the person itself updates its data.
===== 3. Create Marketing List =====

Usually when you import //Contacts// or //Leads// from a list, a //Marketing Campaign// is the source for the data. In these cases it is good practice to create a //Marketing List// in CRM before you import the data. This speeds up the process of adding the imported records to the //ML//. Since you are importing a fixed dataset, or datasets, the //Marketing List// for an import should be of type //Static// and match the record type of the import (//Contact / Lead//).

===== 4. Import Excel =====

  - In CRM go to Settings > Data Management > Imports.
  - Select //IMPORT DATA// from the Menu
  - Choose the Excel file you want to import or drag it to the designated area and press //Next//.
  - CRM will perform a basic validation of the import format.
  - If the validation fails, correct the problem as specified in the failure message and start over again.
  - If the validation succeeds you will be prompted to enter the //Owner// of the records that will be imported. This can be a //User// or a //Team// responsible to follow up on the imported data, fi: if the List is from a specific Supplier and a single Sales person needs to follow up, select the //User//. If it is a list of //Leads// for various sales persons to follow up, select //Sales Team//, and if it is a list of //Contacts// without specific follow up, select //Marketing Team//. 
  - Select //Submit// and then //Finish//

The data import is now started asynchonously as can be seen in the list (use the //Refresh// button) and once finished, its //Status Reason// changes to //completed//. 

===== 5. Check and Correct =====

Once the import job is //Completed//, open the import record (//Import Name// object). On the //General// tab a //Results// section shows how many records were successfully imported, had partial failures or were not imported.

In case of (Partial) Failures, it is important to check why they failed and if this can be corrected in the source data. If the failures are only few compared to the Successes, it usually is faster to add or update the records manually. If there are many failures, this usually is an indication that there is a generic issue in one or more Columns in the source data which should be corrected and a new import should be done.


===== 6. Undo an Import =====

In thoses cases that there are a lot of failures, or field mismatches, it is best to undo the import all together. This will wind-down any changes to existing records and delete any created records.

  - Go to Settings > Data Management > Imports
  - Select the Row of the Import Job you want to delete
  - In the Menu under //Delete//, select //All Records Imported to This Entity//.
  - A window appears where you can give the Delete Job a name. Usually this is not required.
  - If Many records need to be deleted, you can have CRM send you an email when finished
  - Select //In addition to deleteing imported records, also delete import job history//.
  - Press OK.

The Job is now submitted and once completed the job is disappeared from the List and you will receive an email if applicable. 

===== 7. Add to Marketing List =====

When all records are successfully imported, they can be added to the //Marketing List//.

Standard procedure (Functionality broken: July 2023):
  - Open the Import Job
  - Open the section //Fully Imported//
  - Select all Rows
  - The Ribbon in the Import Window will now enable a number of buttons. Select //"Custom"// when adding //Contacts//, or //"Add to Marketing List"// when adding //Leads//.
  - A //Lookup// window opens 
  - Select the Row of the ML you want to use, or alternatively (not preferred), press //New//.
  - Press //Add//.
  - Confirm in the //Add Contacts// Dialog.
  - By default, only the records in the current window are added to the list, so repeat the "Add to ML" steps for the number of pages displayed in the bottom right and corner. Keep in mind that the window refreshes to the fits page after each cycle.....

Workaround (from July 2023):
  - Open the //Marketing List//
  - In the Menu Bar, Select the // Manage Members// drop down and choose //Add using Advanced Find//.
  - Set these Selection Criteria:

And
   Add Row -> Created On  - Today
   Add Row -> Created By - Equals - <your name>
   Add Row -> Created On - Last x hours - 1
   
The last Row is only needed if you import more than 1 dataset that day, or if you already manually created records today.

  - Click Find
  -  In the List, select all, or only the relevant records and press //Add all// or //Add only selected//.


===== 8. Final Notes =====

Once all these steps have been completed, two issues remain unresolved: 

  - All imported //Contacts// and //Leads// are not associated with an //Account//. They are **floating**. If the Account name was known during the data import, a reference can be found in the Marketing Section of the record where we keep track of data provided by the person itself when submitting an update from a Spotler Form. If all imported records are for the same Account, then you can select all Marketing List members and us the //Edit// button to relate them to the //account//.
  - Duplicate Records may be created if there is a slight difference in name or email address.