What should I check in my CRM system to ensure it is performing as it should? Against which metrics should I check it to ensure it is alive and kicking?

How should I audit my CRM system?

Audit

This article is not about the ‘audit’ Dynamics CRM feature.  Nor about the consulting engagement on the ‘CRM’ strategy of a given organization.

This article is about the technical and functional audit of a Dynamics CRM system which ensures that it will perform as expected over the time because it is properly built and maintained.

General architecture and system documentation

The first task in such an auditing engagement is validating that the system is built on a solid foundation and is well dimensioned to support the requirements against it has been built.

This is not only about stating that the building blocks of the system are adequate for the requirements but also they are properly sized. Moreover, it is also about ensuring that the system is properly documented.

It is all about answering these questions:

• Are the services that build the system adequately architected?
• Are the servers supporting the system properly sized?
• Is there any bottleneck affecting the overall system performance?
• Is it secured and does not expose any vulnerability to external attacks beyond the strictly necessary?
• Is it adequately documented? Is there a detailed technical and functional document? Is there an operations manual? Are the use cases documented? Is there a manual for the system administrators?

Technical audit of an on-premise system

From a technical point of view it is about answering this question:

Do the systems and services that support the CRM system work as they should and will be working properly in the future?

This question requires measuring some of the following items:

• The operating system in the servers that support the CRM system: available disk space, CPU usage, available RAM, the appropriate size of the paging file and the latest updates.
• The CRM database system and the reporting system: the size of the data file and transaction log, locks or waiting tables, the maintenance plan, the availability of the services -Reporting Services, Agent, Browser and Server-, growth and consistency of the DDBB and latest updates.
• Specific CRM services : IIS services -W3SVC, HTTPFilter, IISADMIN-, asynchronous processing service, and email router service.

Functional auditing of a CRM system

From a functional perspective the question gets harder to answer because it heavily depends on understanding what the system is designed for and what the user’s expectations are. In any case a functional audit should answer the following key issues:

• Are integrations with other systems operating properly?
• Are the web services exposed to third party systems operating properly?
• Are the CRM tables growing as expected? Is there any design flaw in the security system or in the workflows?

Detailed Table Analysis

• Is there any critical error when tracing messages to the core platform?
  

  CRM Diagnostics Tool

• Are there any critical errors related to CRM services in the event viewer of the servers that support the system?
• Is the email routing service properly configured and working?

Email Router Test

• Are the systems jobs ending OK? Is there any ‘canceled’ or ‘completed with errors’ job? What errors are causing cancellation of the jobs?
• Is there any system to relaunch canceled jobs because of temporary ‘time-outs’ in the SQL Server DDBB?
• Is there any mechanism to ‘housekeep’ and erase the completed jobs?
• Is the javascript code behaving as defined?
• Is the source code available (plug-ins, reports, integrations and web services …)?

Metrics

From the system architecture perspective is quite difficult to define metrics apart from those recommended by the manufacturer.

As a solution to obtain a measure of the implemented architecture, from a black box approach, the proposal is to conduct stress tests at different levels (IIS, databases, Web services layer of the CRM system and web/mobile clients). These results will demonstrate that there is no component bottlenecking others and negatively affecting the overall system performance.

Technically speaking, the metrics are more objective because the indicators to be measured are so:

• Operating systems that support the CRM services should be 100% available.
• The available hard disks space should be not less than 30% of their capacity.
• The use of the CPU and the RAM should not exceed an average of 80% of these resources.
• The server should have the latest updates.
• The growth of the data file and transaction file should not exceed a ratio of 5% per month.
• The process of restoring the DDBB is properly tested and documented.
• The DDBB services and specific CRM services are 100% available.

From a functional point of view, the metrics are not as objective and they depend largely on how the system has been designed, for instance:

• The system does not trace any critical error on the platform layer.
• No system job is queued in a wrong state.
• Systems interfacing with CRM should be 100% available.
• The web services exposed by CRM are 100% available.
• No table grows more than expected, especially the asynchronous operations table (AsyncOperationBase) and the security operations table (PrincipalObjectAccess).
• There is no critical error related to the CRM services in the event viewer of the servers that support the system.
• Every message sent from CRM reach its recipients.
• Every message sent to the monitored mailboxes are created successfully in their correspondent work queue.

Conclusion: a proposal for an audit Project

Auditing a CRM system is not a ‘one shot’ task. Rather, it is an ongoing process that have to measure the defined indicators in the context of a audit project for both suggesting possible improvements to the system and also acting proactively just in case any indicator is out of range.

An audit project requires an initial analysis phase to define the indicators and metrics to measure.

A development phase, where monitoring tools are built and automated as far as possible. And also are the events that will fire as soon as any indicator is out of the expected range.

An implementation of the audit system phase where the system automatically warns of possible malfunctions and prevents when certain indicators out of range may have negative consequences for the system and its users.

And finally a monthly report stating all the indicators and the recommendations on which preventive actions should be taken to ensure that the system is stable and is ready to support both technical and functional requirements of its users.