You’ve reached out to a few different vendors. Based on your statement of work and requirements, they’ve submitted their proposals. How do you go about evaluating the different vendors and picking a winner?

PineapplesIt can be hard to pick a vendor when they all seem the same.

You’ve reached out to a few different vendors. Based on your statement of work and requirements, they’ve submitted their proposals. How do you go about evaluating the different vendors and picking a winner?

One way is to use the Software Engineering Institute’s Capability Maturity Model (SW-CMM). Yes, it’s a mouthful, but what is it for? It provides a way to measure the maturity and effectiveness of a software organization. It evaluates an organization's ability to deliver quality software on budget time after time.

You can leverage the model to create a decision matrix. The matrix will rely on the evaluation criteria of your choosing. But how do you use the matrix?

You might use it to exclude some vendors because of low scores. You might exclude others due to poor scores in specific categories. For example, you might remove any vendor with low scores in project management. Remember the goal of picking a vendor is not to choose the lowest bid. Instead, it is to choose the vendor that will provide the best value.

All scores are subjective based on your team's opinion. Evaluation criteria ideas include:

Requirements Management

How will the vendor establish and manage the requirements for the project?

Project Planning

What processes do they have in place to establish a plan for carrying out and managing the project. This involves developing estimates, and defining the plan to perform the work. It also includes steps to estimate the size of the software, and the resources needed. Other steps include creating a schedule, identifying and assessing software risks.

Project Tracking

How will they provide visibility into the project's progress? So that both parties can take action should the project deviate from the plan.

Subcontract

How qualified are they to select subcontractors and manage them?

Quality Assurance

How will the team review and audit the product to ensure it complies with standards?

Configuration Management

How is the integrity of the product’s configuration established and maintained?

Training / Scaling

How does the vendor develop the skills and knowledge of individuals so they can perform their roles to a standard? How will they assess their current and future skill needs? How will they fill their missing skills?

Product Engineering

Does the organization have a well-defined engineering process to produce correct, consistent software products. For example, analyzing the system requirements, developing the requirements, developing the architecture, designing the software, implementing it, integrating components, and testing it.

Defect Prevention

How is the organization setup to identify the cause of defects? How do they prevent them from recurring? Do they use peer reviews to remove defects from the software?

Software Quality Management

How is the organization prepared to understand and achieve specific quality goals?