Using the framework method for the analysis of triadic interview data : process notes from a curriculum review

We document and reflect on modifying the framework method for the qualitative data analysis (QDA) of large-scale triadic interviews with education actors. The QDA is part of a curriculum revision's early assessment and impact evaluation, which used a mixed methods design including a teacher survey, key informant interviews, and focus group discussions. The participants work in schools sampled for the assessment using cluster randomized design that assigned schools to either the treatment or control curriculum. Considering the amount of interview data and the diversity of our team, we aimed for a very high intercoder and intracoder reliability (ICR) in creating and applying our analytical framework or coding tree during data condensation and analysis. Clearly defining the codes with inclusion and exclusion boundaries, segmenting the transcripts, and memoing were the key processes that increased our ICR. Assessing the codes application and performance using Fleiss Kappa improved the working analytical framework's communicability, consistency, and transparency. Memoing helped trace the points of interpretive divergence among the coding team. After coding, we indexed the segmented data in a framework matrix, using relatively more accessible or free tools to organize, display, and interpret the data. With these procedures, we aimed to meet the consolidated standards for reporting qualitative data (COREQ).