9.4. The Stages of an Interview9.4.1. Opening the InterviewAn interview facilitator (usually the Lead Assessor) begins the interview and makes introductions and opening remarks. He or she briefly explains the purpose of the assessment and explains that confidentiality rules and non-attribution policies apply. Each person interviewed is asked to give his name and say what he does. This practice has proven useful for several reasons. It puts people at ease by allowing them to talk about what they do, how long they have been doing it, and so on. It also alerts the team members about the kind of information they should expect from the interviewee. Some team members might not perform the specific activity that the interview has been scheduled to investigate, and it is useful for the team to know this as soon as possible and to elicit why the organization has asked him or her to participate. (An interview is almost never a waste of time, even if the people being interviewed at first seem inappropriate.) 9.4.2. Phases of the InterviewThe interview facilitator usually begins by asking questions from a set of questions previously prepared by assessment team members. Time should be set aside, however, for additional questions as well. After "open-ended" questions about the interviewee's job(s) and the processes he or she uses, the interviewers then probe for known problems and strengths within the organization or the process. General questions are asked about the organization's environment. Individual team members ask questions related to their assigned areas (KPAs/PAs). The interview session typically concludes with open-ended questions about the areas within the organizationboth model-related and non-model-relatedthat the interviewees believe could be improved. An example of a concluding question might be: "Is there anything you would like us to look at during the assessment?" During the interview, any team members can conduct a "spot request" for necessary documents. A typical interview session spends more time on guided, KPA/PA-specific questions than on open-ended dialogue. However, the time spent on each depends on the comfort level of the people being interviewed and on the maturity level of the organization. In an initial assessment, interviewees usually need more open-ended dialogue. It is important to remind people being interviewed to tell the assessment team if they do not understand a question or if they cannot answer it. Sometimes people being interviewed do not understand the model terminology or feel defensive because they do not know the answer to a question that is out of their field. It is in fact very useful to hear that "I'm not the one you should talk to about thatyou need to ask Mr. Y." No one is expected to know everything. (In fact, if everyone interviewed seems to know all the answers, the team should start to be suspicious.) 9.4.3. Typical Sequences of Questions in an InterviewThe interview facilitator usually starts with general questions and then asks more specific ones. At the end of the interview, the facilitator returns to more general questions to provide closure for the interview. Moving from general to specific helps the team achieve required coverage of the reference model and also improves the accuracy of the information collected. An interview usually starts with context-free, open-ended questions in order to encourage the people being interviewed to explain things from their own perspective and in their own words. These questions provide a wealth of information about the organization, the jobs of the people being interviewed, and any problems in the organization. Direct questions are then used to ensure coverage of the areas of the model and to probe more deeply into any problem areas that have been identified. Direct questions are used to fill in specifics after the open-ended questions have provided sufficient context to ensure that the questions are relevant. A sample sequence of questions might include:
In this example, the first two questions are organization-focused. The next questions focus specifically on the Requirements Management KPA/PA in more detail. This set of questions would most likely be used in a group interview, and everyone in the group would be expected to answer all the questions because each individual might be representing a different project. Everyone being interviewed must be encouraged to be as specific as possible during the interview. Although the team at times may ask open-ended questions, they must ultimately elicit specific answers: examples of how, when, where, and how often a given activity occurs. People who have had previous experience with assessments, health checks, or gap analyses know the questions they will be asked because they have been through similar interviews before. They may regard the questions as self-evident and become bored, requiring the team members to remind them that (for example) new KPAs/PAs are being investigated. 9.4.4. Typical Time Allotments for Interview QuestionsThe following charts illustrate time allotments for 60-minute interviews, 90-minute interviews, and 2-hour group interviews. Typical Time AllotmentsInterview Schedule 60 Minutes
Typical Time AllotmentsInterview Schedule 90 Minutes
Typical Time AllotmentsInterview Schedule Two Hours for Group Interviews
Example of How 85 Minutes for Process Area Questions Are Allocated for Requirements
The PAs that are discussed and the amount of time spent on each depend on the group of people who are being interviewed. 9.4.5. Looking for Specific Information: Kinds of Interview QuestionsPeople being interviewed should expect to be asked about three different areas: process, product, and culture:
Follow-up questions are essential. The following are useful examples:
Follow-up questions allow people being interviewed to speak from their personal perspective, and their answers may influence the rest of their interview and future interviews as well. 9.4.6. Direct or Open-Ended Questions in an InterviewAnother way of thinking about the kinds of typical questions asked during interviews is to divide them into "direct" and "open-ended" questions. A direct question asks for specific information and often results in a one-word answer. These frequently have to do with team members trying to verify information they have already received or attempting to clarify points that are still unclear. Open-ended questions usually ask "how" or "what." The person being interviewed is free to interpret an open-ended question and provide the details in any way he or she likes. Examples of open-ended questions are:
Open-ended questions are preferred during an interview because they reflect the perspective of the people being interviewed. They also make the people being interviewed feel like they are part of the interview process. Open-ended questions can concern processes, products, or organizational cultures and obviously can follow up more specific inquiries. They can be either context-free or guided. A context-free question applies to any development activity or organization. For example, a context-free question about the environment may ask, "If you could fix any problem in this organization, what would it be?" A guided question on the other hand might ask, "Can you explain how you establish X?" Open-ended questions can also be specifically related to the CMM/CMMI models. For example, activity X in the question "How is activity X planned?" could refer to configuration management, requirements management, or any other area where the CMM/CMMI calls for planning. The question is open-ended because the details of the planning process are not assumedthose being interviewed are free to answer in any way they wish. Although open-ended questions are preferred during the beginning of interviews, interviewers need to ask direct questions as well. Direct questions are used to get specific information, but there is a high risk that they will not be answered unless open-ended questions are asked first and the proper context for the questions has been established. For example, the question "Have your software development folders ever been configuration audited?" could provide useful information about the software configuration management process, but only if it has been established that the organization has software development folders and that the people being interviewed know what a configuration audit is. Without a context, the terminology involved in direct questions may render the questions unanswerable. The team must carefully adapt questions to the language of the organization in order to be understood by the interviewees [Dunaway 01c]. 9.4.7. Additional Sample QuestionsExamples of exploratory questions for specific technical groups (all related to some level of the CMM/CMMI) are provided in the following charts. It is important to ask appropriate questions to the people who are most familiar with the topic and who are therefore most likely to provide the assessment team with valuable information. Here, for example, are some typical questions the team might ask different jobholders about specific process areas. Requirements Management and Requirements Development Process Areas
| ||||||||||||||||||||||||||||||||||||||||||||||||||
To Project Managers: | ||||||||||||||||||||||||||||||||||||||||||||||||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||
To Implementers: | ||||||||||||||||||||||||||||||||||||||||||||||||||
|
Project Planning Process Area
To Project Managers: |
|
To Implementers:
What is your role in estimating the size of the product?
How do you determine whether your part of the implemented software satisfies the plan?
How do you estimate how long and how much effort your particular piece of the job will take? Can you walk us through a recent example?
To Configuration Management Practitioners:
How do you plan your part (configuration management) of the project?
To Quality Assurance Practitioners:
How do you plan what you will do on the projects?
Can you explain what happens if a project team says it has too little money to perform quality assurance?
Organizational Process Definition (ML 3) and Integrated Project Management (ML3) Process Areas
To Project Managers: |
|
To Implementers:
What role did you have in developing the engineering processes for the organization (i.e., the engineering handbook)?
To Members of the Process Improvement Group:
How were the organization's management and engineering processes developed?
How do project teams know what processes they must follow on their project?
What criteria did you use to select the best documents, templates, and so on from projects?
How does the organization collect measurements from the projects?
To the CM Staff:
Where do you find the configuration management processes you use on projects?
To the QA Staff:
Where do you find the quality assurance processes you use on projects?
Quantitative Project Management (ML 4) Process Area
To Project Managers: |
|
To Implementers:
What role do you play in analyzing the control charts?
How often do you review the charts?
To Configuration Management Practitioners:
What role do you play in analyzing the control charts used on projects?
To Quality Assurance Practitioners:
Do quality assurance people use statistical process control (e.g., control charts, Pareto diagrams) to understand how they can improve their performance?
Middle and senior managers are not ordinarily asked PA-specific questions, but it might be useful to ask them general questions, such as the following:
To Middle and Senior Managers: |
|
9.4.8. More Details: Sample Technical Questions Concerning the Area of Software Project Planning (Similar Questions Must Be Formulated for Every Technical Area)
Software project planning involves developing estimates for the work to be performed, establishing the necessary commitments, and defining the plan to perform the work. The software planning begins with a statement of the work to be performed and other constraints and goals that define and bound the software project (those established by the practices of the Requirements Management key process area). The software planning process includes steps to estimate the size of the software work products and the resources needed, produce a schedule, identify and assess software risks, and negotiate commitments.
KPA-Level Questions
Would you please describe how you plan for software development on your project? |
|---|
Listen Fors: |
|
Follow-Up Questions:
How do allocated requirements play a role in your software planning activities?
Could you describe how your software planning activities better prepare you for tracking progress of the work?
Goal Satisfaction:
How realistic are the software plans for developing your project's software?
Goal-Level Questions
Goal 1: Would you please describe how you derive your software project's estimates? |
Listen Fors: |
|
Look Fors:
Documented procedures for deriving estimates.
Follow-Up Questions:
What does your project do to ensure the software engineers that develop estimates are adequately trained?
Goal Satisfaction:
Do the practices you just described ensure that you have realistic and usable estimates with which to plan and track your project's software activities?
Goal 2: Would you please describe how you plan software activities for your project?
Listen Fors:
The project is managed using a software life-cycle model with manageably sized stages.
The software development plan is documented, reviewed, and agreed to by all affected groups and is controlled.
The software development plan is based on customer standards, allocated requirements, approved SOW, and project standards.
Software technical, cost, resource, and schedule risks are identified, assessed, and documented for tracking purposes.
Look Fors:
Software development plan.
Documented procedure for developing the project's software development plan.
Follow-Up Questions:
What is SQA's involvement in your project's planning activities?
How does your project deal with late requirements?
Goal Satisfaction:
Do these activities ensure that your project has a development plan that is suitable/feasible for the software work that is to be accomplished?
Goal 3: Would you please describe how you obtain agreements with planned commitments on your project?
Listen Fors:
Project groups participate in planning activities for which they are responsible.
Project groups may negotiate their commitments before they are agreed to.
Commitments are formally documented for tracking purposes.
Software technical, cost, resource, and schedule risks are identified, assessed, and documented for tracking purposes.
Look Fors:
Sample documented agreements between project groups.
Documented procedure for senior management's review of external commitments.
Follow-Up Questions:
What is SQA's involvement in your project's commitment process?
Can you please describe an example of when you have had to assist in negotiating a commitment?
Goal Satisfaction:
Do these activities ensure that your project has commitments that are mutually agreed to by all affected groups and individuals?
[Dunaway 01c]
9.4.9. Judgmental Statements Are to Be Avoided
Interviewers should avoid judgmental words such as "right," "wrong," "good," "bad," "should," "ought," "must," and so on. Interviewees should feel free to make such judgmental statements, but the assessment team should always follow up by asking why the interviewee has made those judgments. For example, if the person being interviewed says, "The managers here don't care about programmers," an assessment team member might ask, "What actions have you seen that make you feel this way?" People being interviewed need to express themselves and vent frustration. The team, however, always needs evidence to either substantiate or refute such statements. Follow-up questions of this kind will usually turn up data about practices that have had a negative impact on productivity [Dunaway 01c].
All information must be corroborated. That is, the team must confirm all data with a second independent source. Follow-up questions provide a good way to corroborate and clarify issues without imposing outside interpretations. Assessment team members are ultimately required to make judgments when they come to the rating process (see Chapter 11, "The Final Stages of an Onsite Assessment"), but those judgments are always based on objective data supplied by the organization. (They also must always be the result of a consensus of the entire assessment team.)
9.4.10. Closing the Interview
The facilitator often concludes the interview by asking an open-ended question, such as: "Do you think the team understands enough about your job and the strengths and weaknesses of your organization?"
In closing interview segments, each person being interviewed must be given time to answer one or more of the following closing questions:
"If you could change anything in this organization to improve quality, what would it be?"
"If you could change anything about the organization, what would you change?"
"What do you believe are the greatest strengths of this organization?"
If there is time at the end of the interview, the facilitator may ask:
"Is there anything you feel was inadequately discussed?"
"Was there anything you feel the team did not understand?"
People who are interviewed finally need to be reminded about the documents that they should submit after the interview is over. They are also reminded that the confidentiality rule must be respected and that the assessment team will respect it in their turn. The interviewers should specifically request that nothing said during the interview be repeated and that under no conditions should statements be attributed to an individual participant. They are also reminded to come to the Draft Findings Briefing.
No comments:
Post a Comment