At CIO Insights, we often get calls from CRM project teams asking us for our opinion on the various commercially available CRM systems. Having completed their evaluation, these groups are often confused as to how these applications differ. When you compare these alternatives feature to feature, you are often left with the impression that the tools are pretty much identical.

In response to these requests for our opinion, we explain that CSO Insights does not do product comparisons or reviews, as do Sell More Now, ISM or Gartner. However, having benchmarked thousands of firms who implemented these applications, we have a wealth of feedback from these system users. The information these organizations provide make it clear that CRM vendors are not all alike.

In preparation for publishing our most recent Sales Effectiveness Insights Study earlier this year, we surveyed 1,040 companies worldwide to assess what challenges their sales teams were encountering, how they were leveraging people, process, technology and knowledge to address those issues, and what results these initiatives were leading them to achieve. Users who have implemented CRM applications as part of their sales effectiveness initiatives do find their results vary depending on which vendor’s product they implemented.

We first asked survey participants to assess what types of results they were achieving by using the applications. By aggregating all the responses we received to this query, we found that 67.1% of the firms were seeing measurable improvements in their operating performance, 19.2% were seeing no measurable improvement, and 13.7% did not know. This roughly two-third’s project success rate tracks with other studies released this year.

For comparison purposes, we next segmented the study data based on class of CRM application. In Group A we included CRM systems typically installed by larger enterprises: Oracle, PeopleSoft, SAP and Siebel. In Group B we included CRM systems installed across enterprises of all sizes: Maximizer, Onyx, Pivotal, SalesForce.com and SalesLogix. (See Figure 1.)

The ratings reveal clear differences. When we began to look at individual products, we found that even within classes, the ratings differ widely.

What does all this mean to a firm evaluating CRM alternatives? Simply doing a feature comparison is not enough. Prior to making a final selection, project teams should do an extensive amount of reference checking to determine how firms like theirs are faring with their implementations. You need to go beyond comparing what these applications do and fully understand how they do it. Failure to do so could be hazardous to your project’s health.

Jim Dickie is a partner with CSO Insights, a benchmarking firm that specializes in analyzing now companies are effectively leveraging people, process, technology, and knowledge to optimize customer relationships. He can be reached via e-mail at jim.dickie@csoinsights.com.