METHODOLOGY

This study analyses the RSS feeds and websites of 19 major news outlets in the United States, the United Kingdom and the Arab World: 

The Study's Background

The research team originally set out to evaluate reporting on the Middle East.  In order to regularize and simplify the collection of news coverage from the 19 online outlets, the team decided to use RSS feeds to simultaneously capture the coverage by using Yahoo! Pipes beta technology.  (see What is RSS?)  In the subsequent analysis of the media coverage of Iran and Iraq captured through the feeds, it quickly became apparent that there were problems in the type and number of articles and multimedia elements that came through the RSS feeds. In short, the research team found the stories collected through the feeds were quite different than the stories collected by conducting searches on the websites of the news outlets that were sending out the RSS feeds.

Further investigation over a series of discrete quantitative and qualitative searches discovered that the problems were not generic across the field.  RSS feeds were used differently by the various outlets.  Some, for example, fed all the stories that were accessible on their websites through their RSS feeds, including articles and other content that was not staff-generated; other news outlets only sent their own bylined material through the feeds, even though AP and Reuters wire-service stories, for example, were prominently featured on their websites. 

A further, and perhaps more significant problem also became evident.  The RSS feeds did not seem to cover the “fields” of news that they were supposed to.  In other words, a feed labeled “International” might not return all international stories on a given website—even those posted the day of the search and written or produced by the staff of the news outlet.  That observation gave rise to a second round of investigation.  A series of searches were conducted using the set keyword terms:  “Iraq,” “Iran,” and “Sudan”—all countries in the news during the period of analysis. Since all the searches that were being conducted were international, the default RSS that was added was the “International” feed (variously called the “World” feed, or similar).  If stories were returned on website searches that did not appear on the basic international RSS feed initially linked through the Yahoo! Pipes, then the research team went back to the missing stories to discover what feeds they were “supposed” to be on.  Then those additional feeds were added to the Yahoo! Pipes.  As a result, for the final analysis, some media outlets were represented by as many as 24 RSS feeds, while others were represented by as few as two. 

After several rounds of searches which prompted RSS additions to the Yahoo! Pipes, it appeared that the percentage of appropriate stories coming through the Yahoo! Pipes searches essentially stabilized.   Stories that came through the website searches but not the Yahoo! Pipes when checked were listed as being tracked in the same RSS feeds already added.  It remains possible—even probable—that we could have continued to marginally improve our percentage of returns had we continued to add additional RSS feeds, especially as the news changed and stories took on additional angles that perhaps might merit tagging in other RSS feeds.

However, when the percentage of stories returned stabilized, the research team moved to conduct its final rounds of quantitative coding and qualitative analysis.  Each website and its corresponding RSS feeds were repeatedly checked over two days using the search terms “Iraq” and “Sudan” by two different coders.

How the Research Team Operated

After extensive training on quantitative and qualitative content analysis, 18 members of the research team conducted the website and RSS searches, and coded and analyzed the results using an online codebook of 33 close-ended and open-ended questions. 

The codebook was developed by Dr. Susan Moeller and Jad Melki and extensively tested for reliability.  To account for variation among different international stories, three keywords were used in the searches: “Iraq,” “Iran” and “Sudan” (“Iran” was only used in the pre-testing phase). The team also ran the searches on different days during the week and different times during the day. In addition, each media outlet was analyzed by at least two members of the team to account for reliability.

The research team also used Google's free online collaboration tool Google Docs and Spreadsheets and Yahoo! Pipes (see What is RSS? for details). One key advantage to using these online tools was their instant accessibility to all members of the research team, which allowed for efficient collaboration and real-time feedback from the research team leaders.

The team used the Google spreadsheet feature in two ways. First, web technology expert Jeanne Kramer-Smyth created a spreadsheet to list all the RSS feeds the team used for research. The researchers could log in and edit the spreadsheet and list new RSS feeds to be added to the Yahoo! Pipe's input list. By the end of the research project, Kramer-Smyth devised a way that the beta Yahoo! Pipes could pull its RSS feeds directly from a Google Docs spreadsheet, which allowed the researchers to add a new RSS feed to the pipe by simply editing the spreadsheet.  Once the spreadsheet was saved (republished), the updates were automatically included the next time the Pipe was run. The Google Docs innovation also made it easy for anyone on the team to verify that a specific RSS feed was in fact included in the pipe.

The Research Team

Dr. Susan Moeller supervised the research team at the Philip Merrill College of Journalism at the University of Maryland, College Park. Jad Melki conducted the quantitative data analysis and Jeanne Kramer-Smyth built and up-dated all the web tools.

Seniors Meagan Bond, Jackie Cutler and Matthew Johnson, all journalism majors, and Rafael Lorente, a visiting professor in the College of Journalism, all contributed significantly to the analysis of the quantitative and qualitative data. Rafael Lorente helped to edit the narrative evaluations. Meagan Bond gathered the background information on each news organization.

Eighteen undergraduate journalism majors, (17 seniors and one junior) were part of the quantitative coding and the qualitative analysis team.  The students were:  Nicole Alberico, Catherine Citroni, Leah Cowdrey, Kristina Ellingsworth, Graham Fitts, Danielle Hayner, Chelsea Jones, Hadass Kogan, Mark Leff, Sarah Merkey, Carrie Peirce, Nicole Porcaro, Lisa Rassenti, Lauren Scott, Nicholas Sohr, Lauren Vitrano, Christopher Williams and Michael Wish (See Authors for details).