Qualitative Analytic Design #1: Factors underlying our approach

In her recent post, Rose commented on the variety in the responses to the 2008 Your Life Line Directive. This variety has shaped the way we are approaching the qualitative analysis of this and the 1990 Social Divisions Directives. So I thought I’d outline our analytic design and share how we are implementing it within MAXQDA (see here for an explanation of our choice of CAQDAS package). I’m doing that in a series of 4 posts, this is the first, check back over the next few weeks and months as our analysis proceeds for the next 3 posts, which detail the way we are going about each analytic phase.

Framed by the projects’ overall objectives, research questions and methodology, our analytic design evolved out of a pilot analysis phase when different approaches were trialled on a sub-sample of narratives from both Directives.

The resulting design involves three phases: i) high-level mapping of semantic content, ii) thematic prioritisation, and iii) in-depth latent thematic analysis. Each phase will be the subject of separate blog posts.

Here, though, I’m briefly discussing four factors that underlie this approach:

1) the nature of the data

2) the need to both keep separate and to integrate the analysis of the two Directives

3) the practicalities of the project

4) the need to develop a transparent and transferable process

 

The nature of writing for the Mass Observation Project (MOP) and the data that is generated

Because the writings of Mass Observation volunteers are only loosely guided by the questions in the Directive, we did not have an overview of the general content of the material at the start. This is symptomatic of this type of secondary qualitative data analysis. The narratives were not generated for the purpose of this study and therefore we have had no influence on the nature or content of the material we are analysing.

This is very different to the type of situation where researchers design and undertake interviews or focus-groups, or observe naturally occurring settings or events. Had we been involved in designing the Directive questions for a specific substantive purpose we might have had some ‘control’ but even then, the very nature of the MOP results in very varied responses to Directives. Some are short, others much longer; some specifically seek to answer Directive questions, others attend only very loosely to the Directive questions; some are written in a quite structured form, for example using bullet points, others are longer more free-flowing, discursive-style narratives; some are written in the first person and reveal detailed insights into personal experiences and opinions, others are more cursory, brief descriptions that upon first reading appear to reveal little about the feelings and opinions of the writers.

This varied nature means we have a very rich set of materials – just what qualitative data analysts love – but when we started out we had no idea of the content of this large body of varied writings. We therefore needed to design an approach that first provides us with an overview, so that we can evaluate the extent to which our research questions are answerable by the data. We could have achieved this by first reading all the Directive responses, but with almost 600 Social Divisions and almost 200 My Life Lines responses, some of which are many pages long, and a short-time frame, we couldn’t do this. We needed to be coding whilst reading. Had the analytic team done the transcribing, we would have had the broad content view we’re looking for from that process, but again, the project resources didn’t allow for that and temporary typists were employed to transcribe the responses. We did informally ‘interview’ the transcribers about their impressions of the MOP writers when they had finished, though, and this informed our thinking. But we could not solely rely on their opinions.

Analysing different sets of responses separately then integrating our analyses

Initially our intention had been to analyse responses to both Directives together, because one of our objectives is to explore the extent to which perceptions and lives have changed between 1990 and 2008 amongst writers who responded to both Directives (that we call ‘serial responders’). However, the pilot demonstrated that despite the need to analyse the ‘serial responders and despite the synergies across the two Directives’, starting out with all the data together would be impractical and would affect our ability to maintain focus whilst coding each Directive.

In addition, it became clear in our pilot coding that we cannot know at the outset which areas of our substantive interest offer potentials for looking across the two Directives. There are many potential synergies, and the longitudinal element of exploring the identities of writers that have responded to both Directives is an important part of our work. However, the Directives are very different and the context of the time in which these were responded to is important.

The practicalities of the project – number of coders and time-frame

Any project needs to attend to ensuring coding is consistent. The involvement of three coders and the short time-scales mean that we needed to design an approach that maximizes consistency from the outset. We could have mapped out the content of the Directive responses by undergoing an “open” coding exercise as a means of initial theme generation; this would have been similar to the usual first stage of Grounded Theory-informed projects. Indeed we did this in our pilot work and this informed the focus of phase one. However, the time available and the uncertainty of content means it is more systematic to focus first on the descriptive content and undertake more interpretive work once we have a clear idea of content.

The need to develop a transparent and transferable process 

One of the objectives of this project is to open up possibilities for using MOP as a source of secondary longitudinal qualitative data. This means that verified and assured processes for analysing MOP data that can be adopted or adapted by other researchers are amongst the projects outputs. The three-phased approach not only serves our analytic purposes but also offers a method that can be easily documented and illustrated.

 

Like qualitative research design in general, ours is iterative and emergent – we expect to need to refine our initial research questions as we progress – in light of our growing understandings. I will outline the three phases of the design (high-level mapping of semantic content, thematic prioritisation, and in-depth latent thematic analysis) and how they are being implemented in the CAQDAS package MAXQDA, in future posts.

 

 

Advertisements

Prescient Observers

It is very difficult to predict the future in politics. In recent months the Conservative Party won a parliamentary majority in the General Election whilst almost every polling organisation predicted a hung parliament. Meanwhile, the Scottish National Party lost an independence referendum but nevertheless gained enormous electoral support. Latterly, and perhaps most surprisingly of all – certainly for political pundits and bookmakers alike – veteran Labour MP Jeremy Corbyn was elected leader of the Labour Party despite beginning the contest as a 100-1 also-ran. Political expertise, it seems, is overrated.

So are the ‘ordinary’ Mass Observation (MO) writers any better at this? Much like contemporary commentators in both print and social media, MO writers are not shy of making political predictions about the future of their society. This was especially true of the Social Divisions directive in 1990. This directive, as the October 12 blog referred to, touched on broad issues of social, economic and demographic change, issues which encouraged some MO writers to respond in a ‘state of the nation’ manner which considered political ideologies and human nature. Take, for example, H1806’s thoughts on Communist societies:

‘It is virtually impossible to create a society of equal opportunity… A uniform society would produce a most boring way of living similar to Mao’s China or Stalin’s Russia. The human race needs variety and stimulation from different sources. We need competition in our lives which inevitably create rich and poor. As long as we do not go to the extremes this state would prove more colourful and exciting for as I have previously stated life would be dull if all our speech, colour, finances were roughly equal.’

The inevitability of competition and inequality in human societies due to our nature remains a commonplace view. For example, in a recent article for the Guardian, the campaigner and author George Monbiot, quotes a soon to be published report by the Common Cause Foundation which found that 78% of people believe others to be more selfish than they really are.

However, amongst the general political trends foreshadowed in the MO archive there are occasional moments of prescience in which our present appears to be repeating the past with uncanny similarities. The Conservative Government’s support for academies over local authority schools and the recent decision to approve a new grammar school has received an enormous amount of media coverage and analysis from proponents and critics. The latter though needed only to consult R470 who outlined the same critiques twenty-five years earlier:

‘There is an inbred desire to get back to the Grammar School , the nursery of the middle class. The opted out school is the future school of the ‘haves’.   The remaining Council schools will be just as the Council schools of my childhood, the place to dump kids to comply with the descendants of the 1877 Education Act. With minimum staff, facilities and equipment to educate the future working class to read and write (not too well).’

R470’s perspective may be more emotive than the mainstream press but its ideological foundation is much the same. This shows that you don’t need to be a highly educated Guardian journalist to critique public policy. Sometimes, like R470, you just have to live it. MO writers like R470 often used their personal accounts to reflect on the politics of their own period as well as the problems they surmised correctly would persist in our contemporary society. Thus, they do not see their role as just ‘observers’ of their everyday lives but of their whole society.

First looks at the Defining Mass Observation Project database

In recent weeks, Ian Waldock and Jason Sadler (both based at Geodata at the University of Southampton) from the project team have started work on the Defining Mass Observation Project database and we now have a website to test.

The database brings together biographical data submitted by the Mass Observation writers upon joining the MO Project with their individual writing record. Using this database, researchers will be able to sample writers based on biographical characteristics and discover more about the writing behaviours of the Mass Observation Project Panel members.

The data in this database has been collected by the Mass Observation Archive and processed by a team of volunteers, who have generously given their time to the Defining Mass Observation project. It was gathered from the biographical forms completed by the Mass Observers when they join the Project and from the catalogue records produced by Mass Observation staff as part of the process to archive new material which has been submitted to Mass Observation.

The information in this database has always been available to researchers, but not in such an accessible form. In the past, to find out more about the biographical profile of the writers, researchers were required to visit the Archive to trawl through thousands of non-electronic records; a very time consuming task! Researchers wanting to know about the response rate to each Directive, were encouraged to use The Keep catalogue. This catalogue contains over a million records relating to the thousands of diverse collections that The Keep cares for, so it is easy for a researcher to get lost amongst all of records!

We hope that this database will enable researchers, for the first time, to discover more about the Mass Observation Project and its writers quickly and remotely. We are excited about the opportunities and projects this will open the Mass Observation Project up to.

We are interested in feedback about the website and database and in getting some users to test the new site. We will update the blog once we have moved on to this stage of the project.

Analysing writers’ responses to the 2008 Your Life Line directive

We’ve been fascinated by the variety of responses to the 2008 Your Life Line directive, and the different ways in which writers have described life events that are significant to them.

Some writers who have contributed to this directive have listed key life events chronologically, and in very neutral terms. Events such as the birth of a child, death of a parent are understated, and it can be difficult to understand how the writer feels about these, requiring a very close reading of the text. This has prompted huge discussions amongst the research team about neutral language use, the context of an individual’s writing, and how evocative very subtle changes in language and terminology can be. Writing, that at a first glance can appear to be a relatively neutral list,  can have the capacity to move the reader through its brevity and subtle changes in tone, register  and content.

Other writers have written full-blown autobiographies, detailing their feelings about events, everyone involved, giving us insights into the weft and texture of their lives. We thought these would be much easier to analyse.  But, as we have started to analyse these we became aware of ourselves as readers, and aware that the writer has produced this writing at one timepoint – 2008 – with hindsight. It has made us wonder whether the writer felt differently at the time that  these events were happening?

Writer M388, a female writer who responded to this directive at the age of 77 sums up some of the difficulties of writing a life, and picking out significant events:

As I think about the key events of my life, I wonder if age has a marked effect on perceptions. I think there are fewer ‘key events’ I’d list now then perhaps 20 years ago. Things/events that mattered great deal at the time have merged with other things since to take a less important role in my life line.

This is a point that our transcribers made when they were transcribing the handwritten scripts into electronic documents. They felt that the significant life-events that younger writers were describing were not the same of those as middle-aged and older writers.

We intend to look at some of the earlier writing of some writers who have been writing for the archive for a long-time, and compare these to their more recent writing, to test  M388’s thoughts on age and what constitutes a significant life-event.

Mass Observation in the ‘New Times’

At the start of the 1990s, the Mass Observation Archive sent out two directives. The first was titled ‘Social Divisions’ and the second ‘A Retrospective View of the Eighties’. Though different in content, both directives marked a chronological and social turning point. The Social Divisions directive began by asking Mass Observation (MO) writers:

‘Are there some major divisions in your own environment – class, race, gender, religion, “culture” etc – that invite comment and are typical of contemporary society?’[1]

Page 1 of the Social Divisions directive

Page 1 of the Social Divisions directive

This question, and those that followed, conveyed a sense that changes had taken place in British society in the preceding decade, which were being discussed by influential writers from the political Left, particularly in publications like Marxism Today. Theorists like Stuart Hall and Martin Jacques argued that the end of the 1980s seemed to mark a point at which longer trends, such as deindustrialisation, the replacement of blue collar jobs with white, immigration and rising ethnic and cultural diversity, and struggles for women’s rights, could be taken stock of. The world, as Hall and Jacques asserted in 1989, had changed, ‘not just incrementally but qualitatively… Britain and other advanced capitalist societies [were] increasingly characterised by diversity, differentiation and fragmentation, rather than homogeneity, standardisation and the economies of scale which characterised modern mass society.’[2] In short, these were ‘New Times’.

The wording of the Social Divisions directive interest, and the way in which it reflected the contemporary debates on social change, has really excited the project team. ‘Class’, for example, long thought to be the key division in British society is no longer the only social division to be recognised. Instead, race, gender, religion and ‘culture’ now sit alongside class as potential forms of division, thereby mirroring the demographic, economic and social changes that had taken place in the 1980s and before. The wording of this 1990 directive, and the MO writers’ responses to it, therefore gives us a fascinating insight into how these New Times were received by the British people. The MO writers came from a variety of backgrounds across many of these social divisions and they can consequently help us explore how these New Times were understood, or indeed whether they were seen to be ‘New’ at all.

In looking at how ‘class’ was constructed and defined by the MO writers, we can consider whether they accepted or disputed the occupational models of class being developed in the same period by social scientists. We can see whether class, as well as other divisions like race and gender, was understood as cultural, economic or political, or an inconsistent and amalgamated version of all three. Within the diverse responses there are opinions and views on accent, patterns of consumption and styles of dress sitting alongside discussions of political identity and Thatcherism; virulent racism alongside hopes for ethnically diverse utopias; and tangential personal stories alongside pointed revelations of discrimination and inequality. As researchers for the ‘Defining Mass Observation’ project, it is our role to explore these narratives and understand which themes the MO writers believed important, both individually and collectively. What is clear at even this early stage in the process is that the rich and varied perspectives of the MO writers on these issues provide us with a complex tableaux of Britain at a point in which many of the social and political fault lines of our contemporary society were becoming visible.  Thus, as with all research concerned with the past, it is hard to shake the feeling we are not also examining the present.

Footnotes

[1] See image.

[1] Stuart Hall and Martin Jacques, ‘Introduction’, in New Times: The Changing Face of Politics in the 1990s (London, 1990), p. 11.

A New Team Member and a New World

 A Brief Introduction

My name is George Stevenson I recently joined the team on the Defining Mass Observation project as a research fellow. I’ll be analysing MO scripts from the 1990 ‘Social Divisions’ directive focusing on how MO writers responded to questions about class and race and what these responses can tell us about identity and inequality around the period. Prior to this, I have just submitted my PhD thesis, entitled ‘The Women’s Liberation Movement and the intractable problem of class, 1968-79’, at Durham University. In this research I explored the importance of class in its many different forms to feminists involved in the Women’s Liberation Movement and tried to recover some of the previously absent voices of working-class women active in and around the movement. I’m currently awaiting my viva.

Alien and yet familiar…

Starting work on the Defining Mass Observation project as a historian has been a very interesting and revealing experience. To my historian’s eyes, the world of social science research at first seemed an alien one. The literature and approaches to research, with their clearly demarcated terms and concepts – strewn through nearly every page and action – challenged my conception of what we do as researchers in a quite fundamental way. Whilst history is certainly not shorn of theory in its analysis and conclusions, the reflexivity of social science, particularly on methodology, is rare. When I’ve analysed themes I’ve simply being doing one of the things that historians do, not ‘thematic analysis’. For a social scientist, the latter is not so generic but is instead a particular and specific type of analytical approach amongst others that should be carefully selected.

To begin with, this depth of theory felt quite off-putting. Historians are clear and careful with concepts but we tend to use far fewer of them, and, as I’m also finding, we tend to use different labels to talk about much the same things. The dichotomy, for example, between essentialist/realist and constructionist approaches in social science will usually be described as poststructuralist and structuralist in history (if mentioned at all). More problematically, terms like ‘top down’ and ‘bottom up’, used to denote deductive or inductive approaches in social science also have other meanings in history. A ‘bottom up’ approach is usually associated with Marxist, feminist, subaltern or other more explicitly politicised types of history in which the past is understood through the experiences of the oppressed and dispossessed rather than the elite. For these ‘historians from below’, a group I identify with, a ‘top down’ approach is a slur rather than a consideration of the merits of deductive or inductive theories.

Nevertheless, it hasn’t taken long for this new world to seem more familiar. After all, although I might not be used to detailing my analytical approach so overtly, I – and every other historian – was still doing the same thing. My PhD thesis was structured thematically and I was in perpetual dialogue with myself, my supervisors and my research data about whether these themes were being decided by me (deductively) or were arising from the sources (inductively). This was especially true when I was analysing the oral histories of the interview participants in my project. In the field of oral history, there is as rich and vast a seam of theory and concepts as any in social science, and again, though the terms used across the two disciplines are different, the issues they discuss are greatly shared. Thus, in contrast to the discomfort of the alien world I had first encountered, I have begun to feel very much at home and I’ve started to enjoy the challenge of a new language and style of thought. The gaps between the disciplines are more slits than chasms and I am certain that the interdisciplinary nature of the Defining Mass Observation team will produce insights that would not have occurred in social science or history independently. We hope that this will also enable the outputs we produce to be as accessible to those arriving from one discipline as any other.

‘They speak for themselves’

Transcript of

Transcript of “They Speak For Themselves” (BBC, June 1939)

On the evening of 1st June 1939 ‘They speak for themselves’ was broadcast on BBC Radio. The programme was presented by the founders of Mass Observation: Charles Madge and Tom Harrisson. In the broadcast, Madge and Harrisson announced Mass Observation to the world and set out the organisations key aims (a transcript of the broadcast can be viewed at The Keep). One of these being that ‘ordinary’ people would have a chance to describe their own lives in their own words and ‘speak for themselves’. While it could be argued that the first phase of Mass Observation was not always true to this, it is a principle that the current Mass Observation Project has tried to hold at the forefront of its mind when recruiting volunteer writers to the Panel.

One way the MO Project has enabled Mass Observers to self define themselves is to us biographical sheets. These sheets, which were introduced into the administration of the Project in 1991, ask the Observers to provide basic information about their age, gender, living situation and occupation. Since the beginning of this year, I have been working with a team of volunteer to transform the information on these sheets onto an spreadsheet that will be analysed by the Defining Mass Observation project team. Five volunteers have been working on the project. These are:

  • Claire Chevalier Nash
    University of Sussex History MA student
  • Monica Burchill
    Mature student and long term MO volunteer
  • Robyn Long
    University of Brighton Sociology BA student
  • Samantha MacComack
    University of Sussex History BA Student
  • Salah Seoudi
    University of Sussex Sociology BA student
Example of a biographical sheet completed by a Mass Observer

Example of a biographical sheet completed by a Mass Observer

On the face of it, the project looked to be simple; just a matter of transferring the information across. In practice, however, we came across many complications. The most complicated of which relate to the occupation of the Observer and their partner.

Observers who had completed the forms in the early 1990s would frequently list jobs that no longer exist or have changed how they are described. Often the volunteers (some of whom were born after 1995 or not in the UK) had never heard of the occupation listed. Some notable ones being: telegraphist, typist, miner, cahier and … Bunny Girl. There was also the problem of defining which sector the Observer worked in; are train drivers still public sector workers after the railways were nationalised in 1993?

We also found that Observers would frequently list multiple jobs, responsibilities, duties and hobbies. K2657 describes her job as:

“At home by choice (although divorced) until youngest child starts school. Part time degree student … Help in library at local comp school 2 mornings a week…Prior to marriage, customer service clerk at Yorkshire Water HQ”
Lives are complicated and the Mass Observers push at the boundaries of categorisation. This is something that the Defining Mass Observation Project team continue to discuss as we consider how to define the Observers as well as continuing to allow them to ‘speak for themselves’.

By Jessica Scantlebury