Here at jpub, we believe that transparency is a key attribute of good scientific research. That’s one reason why we write this blog. And that’s also the reason why we decided to publish in detail the questionnaires we used in our standardized surveys among journalists and audience members. You find the original German versions as well as English translations of the questionnaires underneath this post for download. But if you want to know a little more about the context in which they were used, read on to learn about our methodological approach:
As you might have read in one of our publications (e.g., Heise/Loosen/Reimer/Schmidt 2013) or heard about in one of our presentations, the jpub-project (Re-)Discovering the audience consists of four case studies of different news outlets and their audiences (s. fig. 1):
Fig. 1: The four case studies
In each of the case studies, we investigate the journalists’ as well as the audience members’ performances and expectations with regard to audience inclusion as described in the following heuristic model (see fig. 2) developed by Wiebke Loosen & Jan-Hinrik Schmidt (2012; you can download a pre-print version of the paper here):
Fig. 2: Heuristic model developed by Loosen/Schmidt (2012)
By comparing performances on both sides, we can determine the inclusion level which can be high or low on either side as well as even or uneven. By comparing inclusion expectations on both sides, we find out about the inclusion distance between journalists and their audience: It is small if expectations are rather congruent; it is large, if expectations differ to a greater extent.
Researching both manifest performances and expectations among journalists as well as audience members requires a multi-method approach with a fixed set of methods applied in every case study (see fig. 3 for the example of the newscast case study):
Fig. 3: Methods applied in every case study (example case study: newscast Tagesschau)
For instance, we conduct in-depth interviews with members of the newsroom in different positions (from leading editors to community managers) as well as with audience members who participate to varying degrees (from those who only read or watch the parent medium to those who comment heavily on the medium’s website or run their own user blog there).
Two very important components of our approach are the standardized surveys which we conduct in every case study: one among the respective news outlet’s news staff and one among the users of its online presence.
In addition to some general questions on their position and work in the newsroom as well as on socio-demographic characteristics, the journalists are asked
- which sources their image of the audience relies on,
- how often they are in contact with their audience for different purposes (e.g., for research, to moderate online discussions, to promote new content, to make editorial decisions transparent, …),
- how often they use social media features for journalistic activities (e.g., to find sources or eye witnesses of an event, to receive notifications of errors, to obtain an impression of the public opinion towards a particular topic, …),
- which roles they ascribe to their audience (e.g., passive viewers/readers, commentators, experts, providers of topics, producers of content, …).
The audience members are asked
- which journalistic products and offers of the respective outlet they use and how often,
- which participatory features they use and how often (e.g., sending audience mail, “liking” posts on the outlet’s Facebook page, commenting in the outlet’s discussion forum, sharing website content, …),
- if they use their real name or a pseudonym when they participate,
- whom they address when they comment publicly,
- whom they speak for when they comment publicly,
- to what extent they want journalists to moderate and engage in online discussions,
- how they evaluate audience members’ contributions in the respective outlet’s different offers,
- and – if they do not or only rarely participate – what keeps them from participating.
A special feature of the surveys is that four more item batteries which deal with important dimensions of inclusion expectations are included in both the questionnaire for journalists and the questionnaire for audience members. This allows us to compare directly both sides’ ratings of these items and calculate inclusion distance in the four dimensions (see fig. 4):
Fig. 4: Operationalisation of inclusion distance in the standardised online surveys
In detail, we ask an outlet’s audience members what motivates them to participate; and we ask the outlet’s journalists what they think their audience’s motivations for participation are. Furthermore, we ask journalists which goals they pursue in their journalistic work; and we ask their audience which goals the respective outlet’s journalists in their view should pursue. Moreover, we ask audience members how much importance they ascribe to certain participatory functions; and we ask journalists how important they assume these functions are to their audience. Finally, we ask both sides to rate some general statements on audience inclusion at the respective news outlet, e.g. with regard to strategic rationales and the impact of participation on the journalistic process and products.
If you are interested in how a comparison of journalists’ and audience members’ (expected) inclusion expectations looks like in detail, you might want to read Including the audience. Comparing the attitudes and expectations of journalists and users towards participation in German TV news journalism., the article we recently published in Journalism Studies.