How to Select and Manage Software Consulting Firms

GUIDE TO OUTSOURCING: TEchnology development

In the medical device industry, there are a variety of technical consulting firms that provide hardware design, field programmable gate array (FPGA) development, embedded systems development, mechanical engineering, user interface design, and various types of software development. A healthy relationship between these providers and the medical device OEM is vital to the success of product development. And such health comes only from choosing the right service provider and then engaging in open communication between the two companies.

It's not as easy as it sounds. In a recent survey conducted by Full Spectrum Software, 319 senior engineering managers from medical device OEMs rated their overall satisfaction level with software and other technical service providers. Only 49% rated their experience as “acceptable,” “good,” or “very good.” When asked to choose from a list of problems that contributed to their dissatisfaction with technology service providers, a stunning 81% said that communication issues were the number one source of problems.

Such data highlight a general trend that technology service providers need to do a better job for their clients. However, communication between consultants and clients is a two-way street. Either or both parties can contribute to a breakdown or failure in communication.

To select an appropriate technical consulting firm for an outsourced project, medical device companies should use a series of guidelines to help them. In addition, manufacturers must take steps to mitigate risk and facilitate communications so that the service arrangement results in a good outcome.

Selecting the Right Provider

Most OEM engineering managers keep a file on various types of service providers. When a service provider contacts them, or if one is found in the course of an Internet search, the engineers should retain information about that firm for future use. Internal referrals are also a good way to locate highly regarded consulting firms. Other engineering managers and staff engineers can be a good source of information on firms that they have worked with successfully.

In general, at least three outsourcing firms should be identified and contacted to ensure that they have the expertise required by the project. A standard nondisclosure agreement (NDA) should be executed with each firm. Ultimately the OEM should receive a proposal from each firm that includes the approach to the project, the timeline for completion, and the budget. It should also include a description or list of similar projects that the vendor has successfully delivered, any areas of risk, open issues and what is expected of the OEM for successful delivery. To get to that point, however, an OEM needs to provide the consulting firms with sufficient information to provide those proposals.

Many engineering managers believe they must develop exquisitely detailed specifications in order for the consulting firm to provide an accurate proposal. In fact, some managers believe this is a downside to outsourcing, that they must spend additional (and highly valuable) time developing detailed hardware or software specifications, whereas their in-house teams could work from looser specifications.

In most cases, such an assumption is not true. Good consulting firms that work on FDA-controlled devices are usually very good at developing technical specifications. Therefore, all a manager really needs, as a starting point, is a solid functional specification. However, this is assuming that the manager is willing to collaborate with the consulting firm to develop the final specifications. Although doing so does require some time, with the consulting firm taking the lead, it takes far less time than doing all the specification work up front and without collaboration.

Requirements Gathering. To proceed, the manager should provide each consulting firm with the same functional specification. Carefully note how each firm reacts to being asked to provide a proposal based on a functional specification. Recognize those firms that are asking progressively more-detailed questions that distinguish them as capable of understanding the breadth and depth of the specific demands of your project. This is a first indication as to how effective the consulting firm is regarding communication.

There are two factors to consider at this point. The most important factor is the nature of the questions the consultants ask during subsequent technical meetings. A highly competent firm will ask a series of qualifying questions to make sure it understands the nature and scope of the project. It wants to know the OEM's goals, objectives, and deliverables for the project.

Once that is done, the consultants should start asking questions about the implementation aspects of the project. A strong vendor will ask about business requirements such as future feature expansion, integration with IT infrastructure systems distributed capability, and Software As Service. That is an indication that they have carefully reviewed the functional specification, but more importantly, it indicates that they have relevant experience in the technical aspects of the project. The nature of the questions that software engineers ask is a good way to tell very quickly whether a firm has the appropriate technical expertise. The firm should also be able to grasp the domain-specific details of the project requirements.

The second factor is executive involvement. If an OEM is considering working with large consulting firms, it should ensure that a senior program manager from the service provider has been involved in and led the information-gathering meetings. If a mid-sized firm is being considered, the CTO or VP of engineering should perform the function. The point is that a good consulting firm treats the project with the utmost respect by assigning an executive or an appropriately high-level technology expert to lead the requirements-gathering phase of the project. An OEM should also expect that expert to maintain oversight throughout the course of the project.

Request a Quote. The requirements-gathering meetings give OEM managers a few options. If all selected firms have done an excellent job of asking intelligent questions and an executive has led these discussions, then chances are that each firm is qualified for the project. The next step is to request a preliminary estimate of time and cost along with a description of the methodology and approach. By reviewing these preliminary estimates, the OEM project leader can get a sense of which firm has the best understanding of the tasks involved and whether time estimates are realistic based on the OEM's experiences. In addition, it is important to ensure that the quote includes detailed technical specifications as task items.

Now is also the time to check professional references. An OEM should expect that all references would be positive, so it's important to construct detailed questions to ask of those references. Sample questions can include inquiries on the number of projects a firm has finished for the reference company, and how accurate the firm has been in price and time estimates.

Questions should also include the following:

How is the quality of the work overall?

Is interacting with the firm easy and is communication good with the firm?

Does the firm collaborate in terms of your business priorities, as well as the primary function of delivering completed products or systems?

The next step is to audit the software firm's quality system. If a product is FDA controlled, it is extremely important that the quality system is fully developed and provides traceability throughout the development life cycle. A firm's quality system must be judged against the standards by which it operates, including, among others, IEC 62304 and ISO 13485. Medical device companies usually have a self-audit form that is filled out by prospective vendors. Some companies go so far as to hire outside consultants to qualify a prospective vendor's quality system.

Awarding the Project

If the prospective software consulting companies have all performed admirably, with no single firm clearly standing out, it may be best to simply choose one. The OEM can award a 1-phase proposal for project to the selected candidate. A 1-phase proposal involves development of the specifications and project plan. This step is low risk and low cost, and it serves as a further test for the consulting firm. If the consulting firm does not produce an acceptably detailed technical specification, the OEM has spent only a small amount of the budget and at least some of the specification work can be reused to complete the specifications in-house or by an alternate firm. At this point, the OEM may choose not to continue the work with the firm that completes phase 1, but has still received some value.

On the other hand, a single firm might clearly demonstrate detailed knowledge of the project, and the engineers might feel comfortable working with the consultants through a series of technical meetings. If so, it is appropriate to enact a 2-phase proposal. The first phase is the development of detailed technical specifications and project plan as illustrated in the previous paragraph. The second part is the development of detailed time and cost estimates for project implementation.

An OEM's comfort level is a very important consideration in selecting a vendor. It is perhaps one of the most important factors. In the scenario of this article, engineers and senior staff will be working collaboratively with the firm to develop the detailed specifications. From a budgeting perspective, the additional funding for the development of the specifications and project plan will be offset by more efficient and timelier implementation. In addition, because the consulting firm has worked with the OEM to develop the specifications, it will have a more intimate understanding of the nature and scope of the project.

An OEM mitigates risk by making the acceptance of the deliverable of phase 1 the deciding factor for awarding phase 2. However, the expectation should be that with a truly outstanding consulting firm, there will be a much shorter acceptance period and the project will move rapidly and efficiently into a phase-2 implementation.

Effective Communication and Management

There are various ways to ensure that communication between the two companies is open, honest, and most importantly, accurate. Establishing contact points, and knowing what to do if those individuals have problems is key.

Choosing Contact Points. In any consulting engagement, there should always be a single point of contact on each side. This ensures that an individual can identify and prevent conflicting information that may be passed in either direction. This does not preclude multiple personnel participating in team meetings or technical exchanges. In fact team meetings between the OEM and vendor are highly desirable. However, to avoid conflicts, technical exchanges should not occur in a vacuum.

A single contact prevents conflicting information from being passed in either direction. Telephone conversations between the key contacts should be immediately summarized in an e-mail, effectively establishing an audit trail on all technical decisions.

The OEM should assign a technical contact that has both the significant technical expertise in the project domain and the ability to clearly articulate that expertise. This individual must be capable of making technical decisions that are consistent with the goals of the project. This is not a job for a marketing person or a product manager (unless these people are also experienced engineers). Projects can fail if the device firm does not assign a highly experienced engineer to be the point of contact. In addition to being experienced, the contact person should be articulate and able to make decisions on key technical issues in a timely fashion. The contact should also be empowered to make those necessary decisions that prevent project delays and aid in consistent productivity.

Likewise, the consulting firm must also assign an individual with excellent communication skills to lead the project team. This is often less of an issue for a consulting firm, as they tend to have program managers or project leaders who are trained specifically for this role. Both contact individuals must develop a healthy respect for the other person's technical skills. In other words, they must be able to have or develop a productive working relationship.

Status Meetings. There should be an agreed-upon plan for status meetings. These meetings can be ad hoc, formal, or based on certain milestones. Depending on the size, scope, and complexity of the project, two formal status meetings per month are generally sufficient. In no case should these meetings happen less than once a month. The purpose of a status meeting is to determine progress, identify problem issues and their resolution, and agree upon the goals and objectives for the next status meeting.

Communication Breakdowns. If the working relationship is not achieved within the first month, one or both of the two individuals must be replaced. If the two technical leads cannot work with each other, the project will very likely fail.

Generally the consulting firm will first attempt to reassign a project leader. However, if the OEM has assigned an inappropriate contact person on its side, then a somewhat awkward duty arises for the consulting firm to request that a different individual be put in charge of the project. It may seem untenable for a consulting firm to ask for a replacement. The client, after all, is the client. However, a consulting firm with experience will have its executive team go to the OEM's upper management with a documented list of reasons that necessitate a change of technical lead, or risk project failure.

Reporting Problems. Throughout the course of the project, straightforward communication is key to success. If the consulting firm runs into a problem or technical issue, it should bring this to the client's attention sooner rather than later. Good consulting firms will be completely open with their clients about both successes and problems. A clear indication of a poor consulting firm is one that tells the OEM it has never encountered a single problem that it was not able to quickly resolve. Such a claim is never the whole story. No technical project in the history of humankind has been executed flawlessly and without a single problem that took time to resolve. Although not documented, it's likely that even the invention of the wheel encountered technical problems.

Part of the process is to discuss technical issues that are blocking progress and figure out how to address them. If a project is midway through and the consultant is not reporting problems, it is the OEM's responsibility to ask specific questions. It's better to get any difficulties out on the table as soon as possible and work towards resolution than to let them run to the end of the project when they become expensive to resolve and increase the risk of failure.

A True Partnership. A successful OEM-provider relationship should feel like a true partnership. The device firm should feel that the consulting team is an extension of its own team. Good technical service providers will adapt and learn how OEMs like to work, how they like to communicate, and how the decision-making process works. If it doesn't feel that way, the device company should spend time considering why it is not so. If the consulting firm is falling behind schedule, call a meeting to openly and honestly discuss it and work together to find a solution. If the working relationship feels adversarial, meet with the internal teams and solicit opinions as to why this is happening. Then approach the executive team at the consulting firm to discuss the problems.

Conclusion

Many engineering managers are reluctant to outsource any aspect of an FDA-controlled product. However, this article describes several steps a medical device company can take to mitigate risk and ensure a successful working relationship. What surprises most engineering managers is that when they follow these basic guidelines, they often adopt an outsourcing partner as part of their business model.

A trusted service provider can help relieve situations when internal teams are overloaded. They can be used to perform engineering services that are noncore to the company's business. They can be used to lead new product initiatives and enable internal teams to focus on existing products. The most surprising statistic from the survey is that 90% of formerly reluctant engineering managers continue to use their consulting partners on an ongoing basis.

Andrew Dallas is president and CTO of Full Spectrum Software (Southborough, MA).

Copyright ©2009 Medical Device & Diagnostic Industry