Qualitative Analytic Design #1: Factors underlying our approach

In her recent post, Rose commented on the variety in the responses to the 2008 Your Life Line Directive. This variety has shaped the way we are approaching the qualitative analysis of this and the 1990 Social Divisions Directives. So I thought I’d outline our analytic design and share how we are implementing it within MAXQDA (see here for an explanation of our choice of CAQDAS package). I’m doing that in a series of 4 posts, this is the first, check back over the next few weeks and months as our analysis proceeds for the next 3 posts, which detail the way we are going about each analytic phase.

Framed by the projects’ overall objectives, research questions and methodology, our analytic design evolved out of a pilot analysis phase when different approaches were trialled on a sub-sample of narratives from both Directives.

The resulting design involves three phases: i) high-level mapping of semantic content, ii) thematic prioritisation, and iii) in-depth latent thematic analysis. Each phase will be the subject of separate blog posts.

Here, though, I’m briefly discussing four factors that underlie this approach:

1) the nature of the data

2) the need to both keep separate and to integrate the analysis of the two Directives

3) the practicalities of the project

4) the need to develop a transparent and transferable process

 

The nature of writing for the Mass Observation Project (MOP) and the data that is generated

Because the writings of Mass Observation volunteers are only loosely guided by the questions in the Directive, we did not have an overview of the general content of the material at the start. This is symptomatic of this type of secondary qualitative data analysis. The narratives were not generated for the purpose of this study and therefore we have had no influence on the nature or content of the material we are analysing.

This is very different to the type of situation where researchers design and undertake interviews or focus-groups, or observe naturally occurring settings or events. Had we been involved in designing the Directive questions for a specific substantive purpose we might have had some ‘control’ but even then, the very nature of the MOP results in very varied responses to Directives. Some are short, others much longer; some specifically seek to answer Directive questions, others attend only very loosely to the Directive questions; some are written in a quite structured form, for example using bullet points, others are longer more free-flowing, discursive-style narratives; some are written in the first person and reveal detailed insights into personal experiences and opinions, others are more cursory, brief descriptions that upon first reading appear to reveal little about the feelings and opinions of the writers.

This varied nature means we have a very rich set of materials – just what qualitative data analysts love – but when we started out we had no idea of the content of this large body of varied writings. We therefore needed to design an approach that first provides us with an overview, so that we can evaluate the extent to which our research questions are answerable by the data. We could have achieved this by first reading all the Directive responses, but with almost 600 Social Divisions and almost 200 My Life Lines responses, some of which are many pages long, and a short-time frame, we couldn’t do this. We needed to be coding whilst reading. Had the analytic team done the transcribing, we would have had the broad content view we’re looking for from that process, but again, the project resources didn’t allow for that and temporary typists were employed to transcribe the responses. We did informally ‘interview’ the transcribers about their impressions of the MOP writers when they had finished, though, and this informed our thinking. But we could not solely rely on their opinions.

Analysing different sets of responses separately then integrating our analyses

Initially our intention had been to analyse responses to both Directives together, because one of our objectives is to explore the extent to which perceptions and lives have changed between 1990 and 2008 amongst writers who responded to both Directives (that we call ‘serial responders’). However, the pilot demonstrated that despite the need to analyse the ‘serial responders and despite the synergies across the two Directives’, starting out with all the data together would be impractical and would affect our ability to maintain focus whilst coding each Directive.

In addition, it became clear in our pilot coding that we cannot know at the outset which areas of our substantive interest offer potentials for looking across the two Directives. There are many potential synergies, and the longitudinal element of exploring the identities of writers that have responded to both Directives is an important part of our work. However, the Directives are very different and the context of the time in which these were responded to is important.

The practicalities of the project – number of coders and time-frame

Any project needs to attend to ensuring coding is consistent. The involvement of three coders and the short time-scales mean that we needed to design an approach that maximizes consistency from the outset. We could have mapped out the content of the Directive responses by undergoing an “open” coding exercise as a means of initial theme generation; this would have been similar to the usual first stage of Grounded Theory-informed projects. Indeed we did this in our pilot work and this informed the focus of phase one. However, the time available and the uncertainty of content means it is more systematic to focus first on the descriptive content and undertake more interpretive work once we have a clear idea of content.

The need to develop a transparent and transferable process 

One of the objectives of this project is to open up possibilities for using MOP as a source of secondary longitudinal qualitative data. This means that verified and assured processes for analysing MOP data that can be adopted or adapted by other researchers are amongst the projects outputs. The three-phased approach not only serves our analytic purposes but also offers a method that can be easily documented and illustrated.

 

Like qualitative research design in general, ours is iterative and emergent – we expect to need to refine our initial research questions as we progress – in light of our growing understandings. I will outline the three phases of the design (high-level mapping of semantic content, thematic prioritisation, and in-depth latent thematic analysis) and how they are being implemented in the CAQDAS package MAXQDA, in future posts.

 

 

Prescient Observers

It is very difficult to predict the future in politics. In recent months the Conservative Party won a parliamentary majority in the General Election whilst almost every polling organisation predicted a hung parliament. Meanwhile, the Scottish National Party lost an independence referendum but nevertheless gained enormous electoral support. Latterly, and perhaps most surprisingly of all – certainly for political pundits and bookmakers alike – veteran Labour MP Jeremy Corbyn was elected leader of the Labour Party despite beginning the contest as a 100-1 also-ran. Political expertise, it seems, is overrated.

So are the ‘ordinary’ Mass Observation (MO) writers any better at this? Much like contemporary commentators in both print and social media, MO writers are not shy of making political predictions about the future of their society. This was especially true of the Social Divisions directive in 1990. This directive, as the October 12 blog referred to, touched on broad issues of social, economic and demographic change, issues which encouraged some MO writers to respond in a ‘state of the nation’ manner which considered political ideologies and human nature. Take, for example, H1806’s thoughts on Communist societies:

‘It is virtually impossible to create a society of equal opportunity… A uniform society would produce a most boring way of living similar to Mao’s China or Stalin’s Russia. The human race needs variety and stimulation from different sources. We need competition in our lives which inevitably create rich and poor. As long as we do not go to the extremes this state would prove more colourful and exciting for as I have previously stated life would be dull if all our speech, colour, finances were roughly equal.’

The inevitability of competition and inequality in human societies due to our nature remains a commonplace view. For example, in a recent article for the Guardian, the campaigner and author George Monbiot, quotes a soon to be published report by the Common Cause Foundation which found that 78% of people believe others to be more selfish than they really are.

However, amongst the general political trends foreshadowed in the MO archive there are occasional moments of prescience in which our present appears to be repeating the past with uncanny similarities. The Conservative Government’s support for academies over local authority schools and the recent decision to approve a new grammar school has received an enormous amount of media coverage and analysis from proponents and critics. The latter though needed only to consult R470 who outlined the same critiques twenty-five years earlier:

‘There is an inbred desire to get back to the Grammar School , the nursery of the middle class. The opted out school is the future school of the ‘haves’.   The remaining Council schools will be just as the Council schools of my childhood, the place to dump kids to comply with the descendants of the 1877 Education Act. With minimum staff, facilities and equipment to educate the future working class to read and write (not too well).’

R470’s perspective may be more emotive than the mainstream press but its ideological foundation is much the same. This shows that you don’t need to be a highly educated Guardian journalist to critique public policy. Sometimes, like R470, you just have to live it. MO writers like R470 often used their personal accounts to reflect on the politics of their own period as well as the problems they surmised correctly would persist in our contemporary society. Thus, they do not see their role as just ‘observers’ of their everyday lives but of their whole society.