Data migration and integration in Microsoft Dynamics 365 can be complicated because data is often more complex than people realize. When delivering a new implementation of Dynamics 365, or when considering making your CRM system communicate with other IT systems, a proper design for data migration and/or integration is key to success.
Let’s start by giving some definitions before we analyze some design patterns for system integration:
- Data Migration is the process of moving data from one system to another, which is often necessary to set up a new system with the data that already exists in the current systems.
- Data Integration is the process of building and maintaining the synching of data between IT systems over time.
While data migration and integration are two different things, the processes and design principles are very similar. In this article, we’ll focus on the best practices and design patterns to model your data migration and integration strategies for Dynamics 365.
The Integration Journey
Effective and efficient Dynamics 365 integration with systems like web portals, data warehouse / BI applications, or ERP systems is critical to making it a valuable business tool. Many organizations are making their CRM system the center of their technology portfolio, supporting many business processes like sales automation, marketing, or case management. To provide a single view of their customers, though, it is necessary to integrate the CRM with other business applications and share relevant information.
There are numerous touchpoints and opportunities for Dynamics 365 integrations to provide value for an enterprise, such as connecting to legacy systems, developing a partner ecosystem, or building new company initiatives. As the “journey” towards uncovering new opportunities and obtaining a single view of customers evolves over time, so do the integrations, often resulting in convoluted and over-engineered solutions. These types of point-to-point connections are neither practical nor sustainable. Separate teams involved, different designs and approaches, a variety of technologies and multiple vulnerability points to secure - all these make the integration architecture a complex ecosystem that does not scale.
Spaghetti-style Integration Architecture
Point-to-point integrations grow a tangled architecture over time. You end up with different technologies, different message formats, and multiple endpoints, each needing to be secured. Maintenance is a nightmare, and scalability is at stake. Say that you want to replace your SMS system, you would need to update all the connections to other systems. This is what I call a “spaghetti-style” integration architecture. For as much I love spaghetti being Italian myself, in software architecture, this is an anti-pattern to avoid at all costs.
What’s the alternative? We want to avoid point-to-point connections and make sure that each endpoint can be (relatively) easily replaced, by abstracting its communication channel, message format, and centralizing its security (i.e. one identity and access management system). A good approach to solving these challenges is introducing an enterprise system that conveys messages, exposes a common communication interface, and represents the single point of contact for all our IT systems. This system typically goes under the name of
Connect Bridge Integration Middleware
Connect Bridge is a software integration platform that allows you to quickly build your custom integrations in any programming language. The speed of development comes from not having to learn and use the specific API of the systems you want to connect to. Connect Bridge uses standard SQL syntax instead. Its hundreds of connectors translate your SQL statements into API calls, so all you need to do is simply create a SQL connection to CB – as we call it – et voila the connection is established.
The following illustration shows how clean an integration architecture based on Connect Bridge is.
Move on to IPaaS
Do you have everything in the cloud, and therefore you also want your integration projects in the cloud? Then you want to use the Connect Bridge Integration Platform as a Service (IPaaS). IPaaS is a set of cloud services that enable connecting any combination of on-premises and cloud-based processes, services, applications, and data.
Even if you are not entirely on the cloud yet, Connect Bridge is also a good solution. You can begin by running Connect Bridge as an integration middleware on-premises. When you are ready, you can move to your cloud servers on the Microsoft Azure-hosted SaaS platform managed by Connecting Software. Plus, this works for both integration design patterns we looked at the beginning: data migration and data integration.
By Stefano Tempesta