Data migration is the one-time or ongoing movement of data from one system to another. It is common for companies to undergo a CRM data migration process but often times this is easier said than done. It is crucial you understand the complexity of the data you are trying to migrate and the limitations imposed by your source and destination CRM.
Careful planning is essential for ensuring that this process is smooth and secure. To minimize chances of data loss, I will use Kingsway Soft SSIS Integration toolkit for Microsoft Dynamics 365/CRM for CRM 2011 data migration process.
Kingsway Soft is very easy-to-use tool for running SSIS-based packages and fetching data from Dynamics CRM and saving it into another instance of Dynamics CRM. It has connectors which use Dynamics CRM web services to fetch and save the data. Furthermore, there are several extremely useful features that make the data migration project a lot easier than using some other tools. A few of these are:
- Optionset field mapping
- Date/time field value handling
- Ability to do several different actions on the target CRM for the data (not just creates operations but also updates, upserts etc.)
- Very robust error handling mechanisms
For simplicity, I will only migrate two accounts from CRM 2011 to Dynamics365 CRM Online in this blog post using Kingsway Soft SSIS Integration toolkit for Microsoft Dynamics 365/CRM .
This blog post assumes that you have Visual Studio 2015/2017 with Microsoft SQL Server Data Tools – Business Intelligence template and Kingsway Soft SSIS Integration toolkit for Microsoft Dynamics 365/CRM already installed on your machine. So let’s get started:
1) Open up Visual Studio and create new project then navigate to Business Intelligence template and select Integration Services Project, after naming it click on OK button.
2) In the Control Flow tab of the package design view, drag and drop Data Flow Task from SSIS Toolbox and named it as Accounts.
3) Double click on the recently added Data Flow Task component, now you should be on the Data Flow tab. At the bottom under the Connection Managers tab, right click and select New Connection option, a popup window would appear with number of connection managers. Select DynamicsCRM connection manager and click on Add button, a window would appear in which information about source CRM will be provided as shown below, Once you’re done with all the fields, click on OK button and rename the connection manager as your source CRM name.
4) Add another DynamicsCRM connection manager but this one would be configured as you destination CRM. Once you are done with all of the fields, click on OK button and rename the connection manager as your destination CRM name.
5) In the Data Flow tab design view, drag and drop Dynamics CRM Source component from SSIS Toolbox and rename it as per your choice.
6) Double click on Dynamics CRM Source component, a window would appear with General tab already selected. Select Source CRM Connection Manager from the Connection Manger dropdown, Source Type as FetchXML, Output Timezone as UTC (Default) and in the editor window paste the FetchXML for reading two accounts from source CRM as shown below:
7) Similarly drag and drop Dynamics CRM Destination component in the Data Flow design view and rename it as per your choice. Connect the output of Dynamics CRM Source component to Dynamics CRM Destination component.
8) Double click on Dynamics CRM Destination component, a window would get open with General tab already selected. Select destination CRM connection manager from the CRM Connection Manager dropdown, Action as Upsert, Entity as account and in the Optional settings select the first checkbox for sending time to destination CRM in UTC format. As the Action is Upsert so it’s better to select the second checkbox also which would ignore the unchanged fields while migrating data and hence good for performance.
9) In the Columns tab, most of the columns get auto-mapped. Verify the mappings, specifically ones with Text Lookup.
10) In the Error Handling tab, select second radio button as shown below as we would be redirecting errors to Flat File Destination component so that we could identify errors easily by analyzing the tracing logs when the package execution gets failed. Once we are done with Dynamics CRM Destination component, click on OK button.
11) Drag and drop Flat File Destination component from SSIS Toolbox in the Data Flow design view and rename it as per your choice.
12) Double click on the Flat File Destination, a window would get open with Connection Manager tab selected. Under Flat File connection manager click on New button.
13) As the New button is clicked another window would get open with General tab selected, name the connection manager and select the destination of the text file in which you want the errors to be save and leave rest of the options as they are as shown below:
14) In the Advanced tab, Add three columns of type string and named them as Errorcode, ErrorColumn and CRMErrorMessage so that they could be map from its source component. Once we are done with the configuration of Flat File Destination Connection Manager click on OK button.
15) Being in the Flat File Destination component, Select the Mappings tab and map the newly created error columns of the Flat File Destination from the error columns of CRM Destination Component and click on OK button. Note that before you do mapping the error output of Dynamics CRM Destination component should be connected to the input of Flat File Destination component.
Once we are done with all of the above steps, our Data Flow design view would look like just below as shown. Now just click on Start button and our SSIS package get executed, migrating two accounts from source to destination CRM successfully if everything is done as defined in all above steps or else you would be notify for any errors which you could just identify them from the error log file.
KingswaySoft is an integration toolkit built on SQL Server Integration Services (SSIS) and it is best for data migration project as it comprises of powerful features which I do really like personally. Hope you guys found this blog post really useful in your data migration project. I will be back soon with more enthusiastic blog, Thanks.