Are the dependent variables clearly defined? Consider the operational definition, the appropriateness of the measurement system, and reliability (e.g., data collectors, IOA).

Are the independent variables clearly defined? Consider the procedural details, procedural fidelity, and measures of social validity

given the following information:

The dependent variable of this study was the percentage of social niceties (i.e., initiating and closing the interaction) correctly emitted by participants in one session (i.e., three work scenarios). We decided on polite interaction skills concerning work as a correct response because such polite responses are important for working with others in Japan. The first social nicety was the initiating statement. Concretely, when speaking to others, this was to say "Do you have a minute?" before saying the business-related matters. We recorded a correct response when a participantwas 1.5m or closer to the actor, said "Do you have a minute?" within 5 s of approaching the actor without emitting any other remarks. If the participant emitted the social nicety more than 5 s after having approached the actor, and if they emitted it too far away from the actor, we recorded it as an incorrect response. In addition, if the participant did not approach the actor or made no remark, we recorded it as an incorrect response. Furthermore, if the participant made additional work-related statements or requests before the boss or colleague responded to the social nicety, we recorded it as an incorrect response. The second social nicety was the closing statement. Concretely, this was to say "Thank you for your time" to close an interaction. We recorded it as a correct response when a participant responded before departing from an interaction (i.e., within 5 s after the actor responded to the participant's request but remained within about 1.5 m).

Responses that functioned similarly to the social niceties described previously were also recorded as correct responses. For example, the remarks "Do you have a little time?" and "Is it a goodtime to talk ?" seemed to have the same function as "Do you have a minute?" In addition, "Thank you for the help" and "I am sorry I interrupted you" were considered functionally equivalent to "Thank you for your time." Impolite responses, such as knocking on the boss's desk, or impolite statements, such as saying "Stop working and listen to me," were recorded as incorrect responses.

To provide performance feedback, the trainers recorded a circle for a correct response or a triangle for an incorrect response on a copy of the performance feedback sheet, out of the participants' sight. The reason for the use of circles and triangles was that in Japan, a circle is understood to refer to something positive and a triangle negative; this scoring system was appropriate to provide performance feedback for the participants.

The trainers scored correct and incorrect responses throughout each session. Figure Figure22 summarizes the data recorded by the trainers. For interobserver agreement (IOA), one trained observer independently recorded data in the corner of the room where this study was conducted during each session. The observer independently scored the dependent variables during a subset of 53% of opportunitiesacross all phases. For each of the sessions sampled for IOA, the experimenters randomly selected two opportunities to score one initiation and its closing response per participant. Because there were five participants, the total number of opportunities assessed for IOA was 10 per session. In addition, the number of opportunities for each social nicety was the same number per session. Thus, observers collected data for 80 opportunities. Experimenters defined an agreement as the trainer and the observer independently scoring the same performance on the same opportunity. IOA was calculated by dividing the total number of agreements by the number of agreements plus disagreements and multiplying that by 100%. The mean IOA for "Do you have a minute?" was 97%, and the mean IOA results for each participant were as follows: for Yoshitomi, 96%; for Joichi, 100%; for Hiromu, 97%; for Mebae, 96%; and for Naohide, 97%. The mean IOA for "Thank you for your time" was 97%, and the mean IOA results for each participant were as follows: for Yoshitomi, 97%; for Joichi, 98%; for Hiromu, 98%; for Mebae, 98%; and for Naohide, 95%.

Fig. 2

Percentage of Correct Responses for Each Social Nicety