LORANDOS: Now doctor, your assessment work obligates you to comply with the 1999 edition of the standards for educational and psychological testing, correct?
Witness: Yes.
LORANDOS: And the 1999 testing standards specifically state, and I'll quote, "The applicability of the standards to an evaluation device or method is not altered by the label applied to it, ie test, assessment, scale, inventory." Now my question; given the statement that I just cited from the 1999 testing standards, those standards apply directly to you and your professional work, correct?
Witness: One of good psychological testing, yes.
LORANDOS: Please consider standard 2.10 of the 1999 standards for educational and psychological testing, which state, "When subjective judgment enters into test scoring, evidence should be provided on both inter-rater consistency in scoring and within examinee consistency over repeated measures." Now my question; your work in this case obligates you to comply with this standard, correct?
Witness: Correct.
LORANDOS: Evidence of inter-rater consistency and scoring also refers to what is generally recognized and accepted in your field as inter-rater reliability, correct?
Witness: Also correct, yes.
LORANDOS: Psychologists using the Rorschach often claim that Exner's system has resolved the problems of inter-rater reliability, correct?
Witness: Oh indeed it has, yes.
LORANDOS: In using the Rorschach in this case, did you rely on Exner's TRACS, or what is otherwise known as the comprehensive system?
Witness: Yes, yes I did.
LORANDOS: For research purposes we know that the Rorschach can potentially be scored in a reliable manner, correct?
Witness: Oh yes, all kinds of research studies demonstrate that.
LORANDOS: And, doctor, since 1991, the Journal of Personality Assessment has required a minimum inter-rater reliability of eighty percent before publishing an article submitted to it, correct?
Witness: Yes, yes.
LORANDOS: But that requirement corresponds to what we might call the research reliability of the Rorschach, correct?
Witness: Yes, you could make that designation, yes.
LORANDOS: And psychologists participating in those research studies underwent specialized training to reach acceptable levels of inter-rater reliability, correct?
Witness: Yes, functioning together as research teams.
LORANDOS: Yes, and with respect to those research teams, what specialized training have you undergone to assist you in scoring the Rorschach in an acceptable manner?
Witness: Well, I've attended various workshops offered by John Exner and his group in North Carolina.
LORANDOS: Ok. And those are not inside of a research facility, correct?
Witness: No.
LORANDOS: Ok. So use of the Rorschach in the field is something we could describe as field reliability, correct?
Witness: Now I understand. Yes, yes that distinction...
LORANDOS: And the field reliability of the Rorschach refers to how practicing psychologists such as yourself use it, correct?
Witness: Yes, you can define it that way.
LORANDOS: Can you cite any data published in a peer review journal reporting the field reliability of Exner's system?
Witness: Well, Exner himself in his various volumes reports high levels of inter-rater reliability.
LORANDOS: Can you cite any published data in a peer review journal reporting the field reliability of Exner's system?
Witness: Well I'm sorry; I haven't committed that data to memory.
LORANDOS: Thank you. In other words, the extent to which practicing psychologists fully comply with Exner's scoring procedure remains unknown, correct?
Witness: Not according to Exner himself.
LORANDOS: I see. And that's can you cite any data published in a peer review journal that says that the extent to which Exner's scoring procedures are fully complied with is actually known?
Witness: No.
LORANDOS: If the extent to which practicing psychologists carefully comply with Exner's scoring procedures remains unknown, we do not know how carefully you complied with those scoring procedures, correct?
Witness: If those levels of inter-rater reliability are unknown, that would be the case.
LORANDOS: If you resorted to your own idiosyncratic scoring method, your interpretations of the Rorschach in this case could be mistaken, correct?
Witness: If I resorted to my own idiosyncratic scoring systems, which I don't think I did.
LORANDOS: Ok. Then let me ask you this. Because assumptions about the interpretive accuracy of the Rorschach assume that it was scored properly, correct?
Witness: Correct.
LORANDOS: Other than accepting your claims, we have no way of knowing if you scored the Rorschach accurately in this case, correct?
Witness: Well, I would offer my many years of clinical experience, including administering and scoring the Rorschach to support the fact that I'm confident I did it accurately.
LORANDOS: Now my question; other than accepting your claims, we have no way of knowing if you scored the Rorschach accurately in this case, correct?
Witness: Well if you just want to disregard the explanation I offered...
LORANDOS: Your honor, would you direct the witness to answer the question that's asked?
Judge: You understand that you are to answer questions asked. Would you please do so?
Witness: Yes, your honor.
LORANDOS: Now my question; other than accepting your claims, we have no way of knowing if you scored the Rorschach accurately in this case, correct?
Witness: When you define it to the terms that you've defined it, one can argue that position, yes.
LORANDOS: Thank you. The Exner system that you say you used here includes fifty-four different ratios and percentages in the scoring procedure, correct?
Witness: I knew the number was high but I didn't know it was that high.
LORANDOS; And because the Exner system includes fifty-four different ratios and percentages in its scoring procedure, the potential for error is enormous, correct?
Witness: One can argue that, yes.
LORANDOS: Now given the burden of scoring fifty-four different ratios and percentages, do you really think that you did it with no errors whatsoever?
Witness: I'm confident that I did it accurately enough to support my testimony.
LORANDOS: But you cannot cite any data published in a peer review journal demonstrating that there is a substantial likelihood of your scoring these fifty-four different ratios and percentages in an accurate manner, correct?
Witness: Not off the top of my head as I sit here, no.
LORANDOS: Thank you.
