Simplify Your Software Development: A Smart Approach to Building Flexible Ecosystems that Prioritize Customer Experience.

Traditionally all the approaches come at a significant investment of time, money and skill, and are always focused on reducing complexity, often at the cost of customer experience and flexibility. At the same time, internal as well as external change is constant and inevitable within every organisation. So how do you build software ecosystems that remain core to what the business needs and at the same time add value for the customers when everything is constantly changing? You do it the smart way, integration via abstraction.

Summary

Create a generic API engine to abstract all inter-system events and interactions. This API engine uses workflow to transform the interactions into complex transactions, and stores data in a data lake for future analysis. This generic design allows for app interchange and overcomes legacy app issues as workflow lives within the engine, and is not locked away in a third-party application.

Once an organisation have adopted this new way of integration it will be able to easily modify its business logic, swap out core systems and gain a competitive edge over competitors, all whilst giving customers a better experience through end-to-end integration, and lowering technology costs by leveraging best of breed modular systems.

Now for the details. Why would organisations want to do this?

No man is an island, and the same goes for business systems. Virtually all enterprise applications exchange data with other systems, often via custom integrations. This is often complex and costly, and the older or less actively maintained the application, the harder this integration becomes to maintain.

The smart way involves a new API engine that becomes the gateway for all integrations. All interactions between systems go through this gateway, where these integrations are expanded and modified as need be, without the source systems requiring change.

The intelligent approach has some significant benefits over the traditional approach of direct integration. These include:

• Workflow is built for each integration, and lives inside the API engine, making it easy to adapt and expand it for future business needs, without the need to always get the third-party app supplier involved. This means even apps that have a limited set of APIs or are no longer being maintained can be extended.
• Every interaction is logged, making global auditing and compliance across applications easier.
• Data for every interaction or event is stored in the data lake, making future analysis in the data warehouse simpler and more comprehensive.
• Scheduled interactions can be driven from outside the individual applications, and complex multi-app interactions are now possible without third-party engagement.

So what exactly does this mean and how can it be done?

All the applications talk to the gateway and the gateway alone. The gateway uses its internal workflow engine to transform the API payloads and drive integration amongst the various systems.

This gateway approach enables an organisation to switch out individual applications without having to rebuild complex integrations, enabling an almost ‘plug-and-play’ approach to enterprise applications. As all the systems talk to the gateway, replacing one system only requires new integration between the gateway and that system.

For this approach to be used to its full potential, the applications that talk to the gateway need to support webhooks and a full set of APIs. Webhooks should be configured so that all object (customer, product, invoice, etc) modifications and key events trigger a webhook call to the API gateway to inform it of the change. The API gateway then uses its workflow to interrogate the system that fired the webhook to fetch information about the change, and then decide the appropriate downstream actions to perform.

What is required from participating systems to work in this design?

Compliant systems need to have the ability to attach webhooks to data modifications and other key events (to notify the API gateway of changes) as well as a standard set of RESTful APIs to perform standard CRUD (create, read, update & delete) operations. Most modern applications conform to these requirements.

This eliminates the need to do extensive and continuous modifications to the individual systems, as the maintenance of integration and business logic will rather be invested in the API gateway and its workflow engine.

So what does this look like in a real-world scenario?

Let’s say we have a CRM system where customer information is maintained, a stock system where products and stock levels are managed, a billing system where invoicing is performed and an ecommerce website where products are sold online.

Traditionally integrating all these systems would require serious effort, cost and time, and will require continuous maintenance as the business adapts its workflows. Often this is a significant barrier that prevents organisations from rapid change.

With the central API gateway design, this becomes much simpler. When a new customer signs up on the website an API call is fired to the API gateway, which in turn will add the customer information to both the CRM & billing systems. Once the customer completes his online purchase, the API gateway generates the invoice through the billing system, updates the stock levels in the stock system, and adds the transaction, invoice and statement (from the billing system) to the customer’s profile in the CRM system.

At the same time, all key business information is logged in the data lake by the API gateway, where the data warehouse transforms it into relational data for use by the business intelligence (BI) system for downstream reporting and decision-making.

If the business decides to introduce an additional step in this workflow, like firing off an automatic purchase order once stock levels drop below a specific threshold, it can be easily inserted into the gateway’s workflow, without the need to get the stock system suppliers or third party developers to amend the stock system itself.

Another example could be the business deciding to email the customer an up-to-date statement after each transaction. This would simply require expanding the API engine’s workflow to pull a statement from the billing system, attach it to the customer’s profile in the CRM system and generate a notification email to the customer.

This rapid change is often a game changer, as the ability to adapt to changing environments and take advantage of new opportunities is a serious competitive edge over those businesses that operate complex traditional integrations.

This sounds too good to be true. What is the catch?

This approach requires vision, and commitment from the executive team, as well as a long-term perspective. Adopting this approach requires an internal software team dedicated to the app ecosystem, as they need to ensure that the app ecosystem’s integrity is maintained.

Instead of focusing on traditional software development, this team will focus on the relationship and interactions between the business applications and their associated information, as well as the workflow that drives the various business processes across the organisation.

The bottom line

Once an organisation have adopted this new way of integration it will be able to easily modify its business logic, swap out core systems and gain a competitive edge over competitors, all whilst giving customers a better experience through end-to-end integration, and lowering technology costs by leveraging best of breed modular systems.