QDR Qualitative Data Repository
Not a member yet
110 research outputs found
Sort by
Adverse psychological effects amongst self-tested individuals living in couples in urban Blantyre, Malawi
No full textWe investigated adverse psychological effects and coping mechanisms following HIV-positive and HIV-discordant test results amongst self-tested individuals living in couples in urban Blantyre, Malawi. 35 qualitative in-depth interviews with self-tested individuals living in couples were conducted
Data for: Authoritarian apprehensions: Ideology, judgment, and mourning in Syria
No full textProject Summary
In Authoritarian Apprehensions, Lisa Wedeen draws on her decades-long engagement with Syria to analyze the revolutionary exhilaration of the Syrian uprising’s initial days and then the devastating violence that shattered hopes of any quick undoing of dictatorship. Developing a theoretical approach within the frame of emerging forms of authoritarianism worldwide, Wedeen asks: What led a sizable part of the Syrian citizenry to stick by the regime through one atrocity after another? What happens to political judgment in a context of pervasive misinformation? And what does the Syrian example tell us about how authoritarian leaders exploit digital media to create uncertainty, political impasses, and fractures among their citizens? Based on extensive fieldwork and drawing on material from a variety of Syrian artistic practices, Wedeen lays bare the ideological investments that sustain ambivalent attachments to established organizations of political power and contribute to the ongoing challenge of pursuing political change.
Data Abstract
The project contains three forms of data:
Selected video-clips from the popular television comedy series, Day`a daay`a (A Forgotten Village [2008, 2010]) and from the sketches Amal—ma fi (Hope—There Isn’t Any, [2004]) captioned in English by Lisa Wedeen with the help of Osama Esber, Sofia Fenner, and the staff at Sama Art International. These series are available without English subtitles at various sites online. The episodes are discussed in chapter two, “Humor in Dark Times.”
Statistical materials analyzed in the appendix to chapter 1 of the book.
A list of all URLs used in the book with archived versions of the links in perma.cc, webrecorder.io, or the Internet Archive
</p
Replication Data for: Inconsistency and Indecision in the United States Supreme Court
No full textData Overview
All data and documents contained in this entry were collected for a project that sought to measure and understand a particular phenomenon of decision-making on collegial courts, with a focus on the United States Supreme Court. What happens when a high court, composed of several justices or judges, issues a ruling on the merits without also issuing a clear precedent for lower courts to follow? Why does this happen? How frequently does it happen? What are the consequences of this phenomenon for American law and society? This inquiry led to two publications:
Hitt, Matthew P. 2019. Inconsistency and Indecision in the United States Supreme Court. Ann Arbor, MI, University of Michigan Press.
Hitt, Matthew P. 2016. “Measuring Precedent in a Judicial Hierarchy.” Law & Society Review 50(1): 57-81.
The main argument advanced in these works is: The Supreme Court historically emphasized the goal of dispute resolution, evolving into a Court that now prioritizes an alternative goal of logically consistent legal doctrine. As a result, the Court today fails to resolve more underlying questions in law and society, in part in order to minimize criticism of its output from other elites. In so doing, the modern Court often fails to live up to its Constitutional obligation.
Data Organization
There are two main forms of data available in this entry.
Archival materials of the justices’ deliberations in cases that resulted in unreasoned judgments in the modern era. These materials are organized by case and justice. Materials were collected from the various justices’ papers, archived at the Library of Congress in Washington, D.C. Materials include draft opinions with marginalia, formal memos between justices, informal, private memos between justices and clerks, and letters from justices to external third parties. These materials can be used to replicate the analysis found in the 2019 book, in particular the theory expounded in Chapter 1. Files explicitly featured in the book include their respective chapter number in the filename and are stored in a separate ‘Book’ folder; supplemental files include ‘Sup’ in the filename and are stored in separate folders organized by case names.
Quantitative datasets and code in Stata 15.0 format to replicate the statistical analyses found in the 2019, organized by chapter. The details of these statistical analyses are found in the Technical Appendix B in the 2019 book.</ol
Institutional Moral Hazard in the Regulation of Unemployment
No full textData Overview
The data collected for this dissertation consists of 1) interviews, 2) policy documents, 3) legislation and 4) descriptive statistics. The interviews are conducted in a semi-structured way and have been collected by the researcher. They have been anonymized, transcribed, coded and analyzed using NVIVO. All anonymized transcripts and reports are subject to depositor-approved access, according to conditions specified in the QDR Depositor-approved Access Control Policy and the Data Narrative file. The policy documents have been gathered through publicly available means or have been gifted to the researcher by informants. In several cases where copyright applied, permission has been obtained from the copyright holder. Legislation and descriptive statistics are gathered from publicly available sources.</p
Data for: Regional governance: The evolution of a new institutional form
No full textData Overview
Data set consists of all legally binding treaties for the universe of regional economic institutions as defined in the data paper. The universe of regional economic institutions includes all multilateral, multi-issue regional organizations that have at least one or more legally binding treaty. Each legally binding treaty is coded on a variety of dimensions as given by the coding sheets.
The basic data are contained in the coding sheets for each
treaty. Treaties are almost always available by themselves. Treaty language is not always English: often the language of the REI is adopted (e.g. Spanich, French, etc.), for the post-Soviet REIs they are almost all in Russian
Data for: Forbearance as redistribution: The politics of informal welfare in Latin America
No full textOverview
Why do governments tolerate the violation of their own laws and regulations, and when do they enforce them? Conventional wisdom is that state weakness erodes enforcement, particularly in the developing world. In contrast, this book project, "Forbearance as Redistribution: The Politics of Informal Welfare in Latin America," highlights the understudied political costs of enforcement. Governments choose not to enforce laws and regulations, or forbearance, when it is in their electoral interest.
Focusing on laws that the poor violate, the manuscript shows how a simple distributive logic can account for enforcement patterns over space and time.
Politicians forbear when formal welfare policies are inadequate and they depend on the poor’s votes to win office. Forbearance both indirectly signals a politician’s class affinities and directly functions as a form of informal welfare provision to the poor. Unlike these informal transfers, many state benefits accrue to the middle class in developing countries. The poor therefore vote and mobilize for forbearance, while expecting little from tax-based redistribution. Forbearance thus offers much-needed support when governments fail, yet the book also show how it can perpetuate the same exclusionary welfare policies from which it originates.
The primary—but not exclusive—empirical focus is Latin America, a middle-income region where many governments have the money and manpower to enforce their laws. Latin America is the region with the most unequal income distribution in the world, which means that poverty rates are much higher than would be expected at similar development levels. Sharp inequality and residential segregation create different incentives to enforce depending on where politicians seek office within a city. I select city cases that vary along the principal independent variables under both my theory and competing state capacity-based explanations. The cases span a city known for its capable institutions (Santiago, Chile) and cities with more middling capacities that either group all voters into a single catchall district (Bogotá, Colombia) or divide voters into many income-segregated districts (Lima, Peru).
The decision to compare enforcement across cities reflects substantive and methodological concerns. City governments have become increasingly important sites of policymaking and electoral contestation after a wave of decentralizing reforms in the 1980s and 1990s. Methodologically, there are ways in which the biggest cities across countries—Lima and Santiago, or Lagos and Accra—are more similar to one another than to the secondary cities or rural regions to which they are compared in more common within-nation subnational research designs. Therefore, in this book I focus on Latin American capitals as a way to make valid inferences across similar units, and I use enforcement patterns both within and across cities to expand the number of testable observations. I underscore the comparability of major cities by extending the argument to Istanbul, Turkey. Common tensions around law enforcement emerge in a quite distinct national context.
I take a multi-method approach in which I use observations about how a variety of actors—citizens, bureaucrats, mayors, and presidents—behave to distinguish my theory from dominant alternatives moored in state weakness.
These included public opinion data that reveal that poor support forbearance and candidates who advocate it; qualitative interviews with local politicians, bureaucrats, street vendors, and squatters; local government data on enforcement actions and legal violations; an archive of all newspaper articles on squatting and street vending; and a range of administrative documents, such as government reports, campaign platforms, and correspondence with squatter settlements. I use a combination of methods, including 1) statistical analysis of public opinion data and enforcement data, 2) process-tracing using interviews, government documents, and newspaper reports, and 3) content analysis of newspaper reports.
My findings underscore the strategic—and deeply democratic—nature of enforcement of laws that the poor violate. “Weak” enforcement does not necessarily imply a weak state that cannot regulate the behavior of its citizens. To the contrary, forbearance can indicate healthy electoral democracy in which politicians are responsive to poor voters and choose not to enforce laws that conflict with local preferences. This theory naturally suggests counterintuitive policy conclusions: reforms to strengthen the welfare state may do more to build the rule of law than additional funding for police and bureaucrats. Successful democratization and reforms to increase the poor’s political power, if unaccompanied by improvements in social policy, can erode enforcement.Technical Details
Compilation of existing materials: I reviewed all newspaper articles published about street vending and squatting between roughly 1990 and 2011.
Given that online archives were not available (or not for the entire period), I hired RAs to identify the set of newspaper articles that discussed these issues. RAs were trained to pull any article that related to street vending or squatting. In Peru, this task involved working from one of the main newspaper’s (El Comercio) physical archive. RAs photocopied each newspaper article located. In Colombia, the newspaper archive is online for El Tiempo, but the search algorithm is imprecise so the RA identified that each article actually related to street vending or squatting after doing a key term search for related words (for street vending, these included “vendedores ambulantes,” “comercio informal,” “espacio público”; for squatting, these included “ocupaciones ilegales,” “ocupaciones informales,” “barrios informales,” “subdivisiones informales,”
“piratas,” “terreros,” and “vivienda informal”).
Content analysis: Using the newspaper article database, I coded (along with an RA) the topics discussed. The articles first were classified as one of four types: 1) news item, 2) letter to the editor, 3) short note, and 4) platforms, speeches, or interviews with politicians. Articles were then given a primary coding based on their dominant theme (sympathetic narrative, sympathetic polemic, unsympathetic narrative, and unsympathetic polemic), as well as secondary markers for the specific types of issues mentioned. Given the fact that the newspaper articles were not digitized, the articles were coded by hand (rather than through a machine learning procedure).
Focus group: I also conducted informal focus groups with street-vending associations and squatter-settlement leaders to understand how organized groups interact with politicians. In the case of street vendors, I located these association leaders through organized labor confederations and non-governmental organizations. The focus groups took place in the headquarters of one of the main labor confederation in each city (CUT in Colombia, CTP in Peru). In the case of squatters, I located these associations through local politicians in two districts of Lima, Villa María del Triunfo and Villa El Salvador.
Interviews (individual, face-to-face): Three main types of respondents were interviewed for this project: 1) bureaucrats, 2) politicians, and 3) leaders of street-vending and squatting associations. I personally interviewed 149 district bureaucrats in Bogotá, Lima, and Santiago. These were all “street-level bureaucrats” in Lipsky’s (1980) sense that they interact with citizens directly and have discretion over decisions that matter for citizens’ lives. To conduct the survey, I went to every district government office, sometimes multiple times, and asked to speak with the director or sub-director of the office in charge of street vending and housing issues. All interviews were conducted in the local government offices. Interviewes lasted between 30 and 120 minutes and involved both a largely open-ended section of questions about the municipality’s enforcement activities (using an interview guide) and then a structured survey of the bureaucrats’ attitudes and experiences in the municipality.
I also gathered demographic information on each bureaucrat, such as their education, work history, and political engagement.
The sample does not include districts that are classified as more than 75 percent rural or have populations under 5000. This means that the sample excludes Sumapaz in Bogotá, and Ancón, Pucusana, Punta Hermosa, Punta Negra, San Bartolo, and Santa Rosa in Lima. In Santiago, I include the 34 districts that comprise Greater Santiago (32 comunas in Santiago, plus San Bernardo and Puente Alto in the provinces of Maipo and Cordillera, respectively). In Istanbul, I conducted the survey only in a subset of the city’s 39 districts. I first stratified the sample into lower, mixed, and upper-income districts. I selected four districts from each category (although I was unable to complete one interview in the low-income district of Çatalca). The districts surveyed included: Gaziosmanpaşa, Sultanbeyli, and Beyoğlu (lower); Arnavutköy, Beykoz, Fatih, and Kadıköy (mixed); and Beşiktaş Maltepe, Sarıyer, and Sultangazi (upper). All interviews were conducted in Turkish with the assistance of a translator.
During my survey of district offices, I always attempted to speak with politicians as well. I conducted unstructured interviews with 62 politicians (mayors, city councilors, and local councilors). Local politicians are hard to track down given that their political careers depend in large part on constituency service and local councilors often have second jobs. I therefore only managed to interview politicians in a subset of my cases. My main concern in sampling was to complete interviews with politicians in districts with different socioeconomic profiles. These interviews generally were less structured than those with bureaucrats because I tried to ask about specific cases that bureaucrats had mentioned as challenges for the district.
Finally, I also interviewed the leaders of squatter settlements and street vending associations. In some cases, I located these leaders simply during my visits to local government offices. I spent countless hours in municipal government offices waiting to meet with bureaucrats and politicians. In these waiting rooms, I often encountered the leaders of squatter and street-vending associations and asked whether I could interview them about the types of concerns that they were bringing to the mayor. This approach introduced a potential bias in that the most active leaders tended to be those who showed up in local government offices. I therefore complemented this strategy with visits to specific squatter settlements and street vending markets. It was difficult to conduct these visits unaccompanied so most of these occurred with the assistance of a local NGO or a local politician.
Interviews with leaders then were conducted in the headquarters of the NGO or in the homes/community spaces.
Participant observation: I shadowed roughly a dozen bureaucrats in their everyday tasks to understand the constraints on enforcement firsthand. In my interviews, I asked bureaucrats where I might be able to accompany them in their enforcement activities. Many objected or said they needed additional permission from the mayor. In the case that they permitted me to accompany them, I spent a half or full day watching them do their job and took field notes on the procedures. Those who allowed me to accompany them probably were not a random sample of all districts, but interestingly, many of those who wanted me to come along were from the most challenging districts where bureaucrats felt overwhelmed and wanted me to “see” the constraints that they faced.
Survey (face-to-face): My interviews with bureaucrats ended with a short survey questionnaire about their experiences and attitudes (taking about
15-30 minutes). Through this survey, I collected data on offenses, enforcement actions, government resources, citizen complaints, decision-making, and bureaucrats’ perceptions of enforcement politics. See above on the selection of districts to survey.
The survey guides are included in the data deposit. Data Narrative
This data collection includes three major types of materials, 1) a collection of newspaper articles, 2) a selection of interview transcripts, and 3) survey questionnaires and coded responses of bureaucrats.
First, the newspaper articles constitute a curated collection on specific topics, street vending and squatting, which may be relevant to other researchers, from Peru’s newspaper, El Comercio. This archive is not digitized and is not easily accessible for the early part of the time period studied (particularly the 1990s). The articles are organized by topic (street vending and squatting) and chronologically to make them most useful to other researchers.
Second, the interview transcripts constitute a sample of the interviews that I conducted. Due to the effort involved in de-identification and translation, only a set of 28 interviews from Lima is shared. Additional interviews may be available on request from the depositor. Where interviewees agreed to be identified, the transcripts include the name of the interview subject. I have included a full list of interviewees who agreed to be identified, in the case that a researcher is interested in a particular interview.In general, each interview includes the place and date where it was conducted consists of my detailed notes on the questions asked and the responses. In most cases, the interview does not constitute a complete transcription of the interview. Instead, I noted down key moments in the interview (the approximate time when a topic came up) which I then transcribed from the audio recording, or asked the interviewee to repeat to make sure that I wrote down the relevant quote appropriately. For pedagogical purposes, this approach may be interesting to other researchers. Many researchers (and especially graduate students) do not have the funds to transcribe each interview conducted, especially when doing a large number of interviews. They also may not have theoretical reasons to care about the specific language used by interview subjects (as was the case for many of the interviewees in this project). I did not transcribe every interview because I cared more about the general interactions and events described, rather than the specific wording used by the bureaucrat. In terms of replication, this approach to interviews obviously has downsides. The specific passages transcribed reflect my particular interests and hypotheses, and therefore may be less useful for future researchers hoping to test other hypotheses or wanting to see if they arrive at the same interpretation of the evidence.
Given the confidentiality concerns discussed, I am not allowed to share the audio recordings (when they exist) for the full interview and therefore it may be difficult for other researchers to reconstruct the full course of the interview to test other hypotheses. However, the detailed field notes do allow other researchers to see the questions and topics discussed that they could then follow up in the course of their own research or see if they find consistent with my interpretation of the evidence.
Third, I include the interview guides used for local-level bureaucrats, as well as a spreadsheet coding their responses to the structured questions included in the interview. The interview guides further help to understand the questions asked and the potential uses of the data collected beyond the original research project. Data Sensitivity
The data are sensitive because they include interviews with bureaucrats who admit that they do not enforce the law as required for their jobs, as well as street vendors without licenses and squatters without property titles. In the case of bureaucrats, although it would be very rare, some of their admissions may put them at risk of losing their job (particularly in Bogotá where the Comptroller investigates whether bureaucrats follow legal orders).
The economic vulnerability of street vendors and squatters also raises particular concerns about revealing their identity. For these reasons, Harvard’s IRB suggested that I maintain the confidentiality of bureaucrats and leaders of street vending and squatting associations that agreed to be interviewed. I cite my interviews only based on the office of the individual in charge, as well as the district where they work (or where street vendors and squatters are located). In addition, Harvard’s IRB was concerned that releasing the audio recordings of my interviews would cause unnecessary risks to participants and asked that I delete these recordings as soon as they were transcribed.
I have included the approved IRB application to give a sense of the risks and precautions taken.
I also interviewed local politicians and higher-level bureaucrats. While I offered them anonymity, most agreed to be interviewed in their public capacity and therefore these interviews do not raise sensitivity questions like those discussed above. The list of identified interviews includes information on the interviewees who agreed to be cited by name.</p
Data for: Debating Algorithmic Fairness
No full textThis is an Annotation for Transparent Inquiry (ATI) data project. The annotated article can be viewed on the Publisher's Website.
Data Generation
The research project engages a story about perceptions of fairness in criminal justice
decisions. The specific focus involves a debate between ProPublica, a news organization, and Northpointe, the owner of a popular risk tool called COMPAS. ProPublica wrote that COMPAS was racist against blacks, while Northpointe posted online a reply rejecting such a finding. These two documents were the obvious foci of the qualitative analysis because of the further media attention they attracted, the confusion their competing conclusions caused readers, and the power both companies wield in public circles. There were no barriers to retrieval as both documents have been publicly available on their corporate websites. This public access was one of the motivators for choosing them as it meant that they were also easily attainable by the
general public, thus extending the documents’ reach and impact. Additional materials from ProPublica relating to the main debate were also freely downloadable from its website and a third party, open source platform. Access to secondary source materials comprising additional writings from Northpointe representatives that could assist in understanding Northpointe’s main document, though, was more limited. Because of a
claim of trade secrets on its tool and the underlying algorithm, it was more difficult to reach Northpointe’s other reports. Nonetheless, largely because its clients are governmental bodies with transparency and accountability obligations, some of Northpointe-associated reports were retrievable from third parties who had obtained them, largely through Freedom of Information Act queries. Together, the primary and (retrievable) secondary sources allowed for a triangulation of themes, arguments, and conclusions.
The quantitative component uses a dataset of over 7,000 individuals with information that was collected and compiled by ProPublica and made available to the public on github. ProPublica’s gathering the data directly from criminal justice officials via Freedom of Information Act requests rendered the dataset in the public domain, and thus no confidentiality issues are present. The dataset was loaded into SPSS v. 25 for data analysis.
Data Analysis
The qualitative enquiry used critical discourse analysis, which investigates ways
in which parties in their communications attempt to create, legitimate, rationalize, and control mutual understandings of important issues. Each of the two main discourse documents was parsed on its own merit. Yet the project was also intertextual in studying how the discourses correspond with each other and to other relevant writings by the same authors. Several more specific types of discursive strategies were of interest in attracting further critical examination:
Testing claims and rationalizations that appear to serve the speaker’s self-interest
Examining conclusions and determining whether sufficient evidence supported them
Revealing contradictions and/or inconsistencies within the same text and intertextually
Assessing strategies underlying justifications and rationalizations used to promote a
party’s assertions and arguments
Noticing strategic deployment of lexical phrasings, syntax, and rhetoric
Judging sincerity of voice and the objective consideration of alternative perspectives
Of equal importance in a critical discourse analysis is consideration of what is not addressed, that is to uncover facts and/or topics missing from the communication. For this project, this included parsing issues that were either briefly mentioned and then neglected, asserted yet the significance left unstated, or not suggested at all. This task required understanding common practices in the algorithmic data science literature.
The paper could have been completed with just the critical discourse analysis. However, because one of the salient findings from it highlighted that the discourses overlooked numerous definitions of algorithmic fairness, the call to fill this gap seemed obvious. Then, the availability of the same dataset used by the parties in conflict, made this opportunity more appealing. Calculating additional algorithmic equity equations would not thereby be troubled by irregularities because of diverse sample sets.
New variables were created as relevant to calculate algorithmic fairness equations. In
addition to using various SPSS Analyze functions (e.g., regression, crosstabs, means), online statistical calculators were useful to compute z-test comparisons of proportions and t-test comparisons of means.
Logic of Annotation
Annotations were employed to fulfil a variety of functions, including supplementing the main text with context, observations, counter-points, analysis, and source attributions. These fall under a few categories.
Space considerations. Critical discourse analysis offers a rich method for studying speech and text. The discourse analyst wishes not simply to describe, but to critically assess, explain, and offer insights about the underlying discourses. In practice, this often means the researcher generates far more material than can comfortably be included in the final paper. As a result, many draft passages, evaluations, and issues typically need to be excised. Annotation offered opportunities to incorporate dozens of findings, explanations, and supporting materials that otherwise would have been redacted. Readers wishing to learn more than within the four corners of the official, published article can review these supplementary offerings through the
links.
Visuals. The annotations use multiple data sources to provide visuals to explain, illuminate, or otherwise contextualize particular points in the main body of the paper and/or in the analytic notes. For example, a conclusion that the tool was not calibrated the same for blacks and whites could be better understand with reference to a graph to observe the differences in the range of risk scores comparing these two groups. Overall, the visuals deployed here include graphs, screenshots, page extracts, diagrams, and statistical software output.
Context. The data for the qualitative segment involved long discourses. Thus, annotations were employed to embed longer portions of quotations from the source material than was justified in the main text. This allows the reader to confirm whether quotations were taken in proper context, and thus hold the author accountable for potential errors in this regard.
Sources. Annotations incorporated extra source materials, along with quotations from them to aid the discussion. Sources that carried some indication that they may not be permanently available in the same form and in available formats were more likely to be archived and activated. This practice helps ensure that readers continue to have access to third party materials as relied upon in the research for transparency and authentication purposes.</p
Data for: Shifting Legal Visions: Judicial Change and Human Rights Trials in Latin America
No full textData Overview
All documents contained in this entry were collected for a project that sought to explain variation in the scope and effectiveness of human rights prosecutions in contemporary Latin America. The project led to two main publications:
Shifting Legal Visions: Judicial Change and Human Rights Trials in Latin America (Cambridge University Press, 2016);
“Persuade Them or Oust Them: Crafting Judicial Change and Transitional Justice in Argentina” in Comparative Politics, 46(4): 279-298.
The main argument developed in these publications is that a crucial pre-condition for widespread and sustained human rights trials is a change in the legal preferences of judges and prosecutors, such that they become more knowledgeable and accepting of international human rights law and innovative investigative techniques attuned to the specificities of human rights crimes. These changes are brought about by victims and their lawyers when they deploy very specific strategies of legal mobilisation. The book traces changes or continuities in legal preferences in three countries: Argentina, Peru, and Mexico.
Data Organization
This project includes some of the primary data used in the aforementioned publications. Specifically, three different types of files, distinguished by filename:
Rulings handed down by Argentine courts. One type of document (Argentina_CSJ) contains a selection of rulings handed down by the Argentine Supreme Court on important questions such as the constitutionality of amnesty laws or pardons. I also included briefs on key rulings published by the Supreme Court. A second type (Argentina_TOF) groups some of the rulings handed down by Federal Oral Tribunals (Tribunales Orales Federales) during the wave of trials that started in the mid-2000s. A third type of document (Argentina_CABA) groups rulings handed down by an appeals court in the City of Buenos Aires in the mid-1990s, which I used to trace changes in the way judges understood the “right to truth” as litigants stepped up their strategies of legal preference change.
Documents from the archives of the Centro de Estudios Legales y Sociales (CELS), a leading human rights NGO in Argentina (https://www.cels.org.ar/web/). These documents contain Argentina_CELS in their filenames. I used these files to trace victims’ efforts to transform legal preferences between 1992 and 1996. These efforts consisted of pedagogical activities designed to spread awareness of international human rights law among judges, prosecutors, and their clerks. The documents include: correspondence between CELS, donors, and participants in preparation for the seminars; list of panels and talks; transcripts of lectures and seminar discussions; reports written after the events took place; and lists of participants.
News articles from several Mexican outlets. I collected a series of news articles from a variety of Mexican outlets to trace the main contours of the debate about transitional justice during the 2000 presidential campaign, the transition period once President Fox was elected, and the early years of the Fox administration. The news archive should be useful for anyone interested in process tracing the Mexican transition to democracy. The articles are stored in spreadsheets that contain their title, date of publication, URL, and a link to a copy of the article in an internet archive (perma.cc). These files contain Mexico_NewsArchive in their filename
</ol
Data for: The structure of description: Evaluating historical description and its role in theorizing
No full textThis is an Annotation for Transparent Inquiry (ATI) data project. The annotated article can be viewed on the publisher's website.
This data overview first briefly discusses the paper’s data generation and analysis elements that Annotation for Transparency Inquiry (ATI) tries to foreground. It then elaborates more fully on the paper’s logic of annotation by identifying two broad annotation categories that encompass seven distinct annotation sub-categories.
1.1. Data Generation:
The generation of data can take several forms. A conventional hypothesis testing paper will generate data to test the empirical implications derived from its theoretical predictions as well as those of its main challengers. Such data is generated by drawing on existing data sets, conducting experiments, or doing original field or archival research. This paper takes a slightly different path given that it does not test hypotheses. It pursues what could be call a quasi meta-analysis and uses as its data the book reviews, scholarly exchanges, historiographical assessments, and archival “replications” that together comprised the so-called Goldhagen controversy. It treats the qualitative findings of other scholars as evidence and thus can be said to resemble a meta-analysis. As a result, my date generation involved little more than downloading PDF or articles or checking books from the library.
The paper’s reliance on such secondary, easily accessible, and copy-righted material also means that it makes little sense to included original textual passages as is the standard in the ATI. However, such a meta-analysis involves a sufficient number of different research judgments that should be made transparent because their complexity makes them unsuitable for conventional footnotes. For those instances, I choose to add ATI. The logic of annotation section below elaborates more fully on the nature of those judgements.
1.3. Data Analysis:
I reviewed 45 distinct contributions to the debate that varied in length from short book reviews to detailed, longish journal articles. Of those 45 works, I ended up citing nine (20%). The selection was largely dictated by what I deemed to be their relevance in helping me evaluate Goldhagen and Browning’s descriptive inferences. It might be worth recalling that I am not trying to settle every last facet of the Goldhagen debate nor am I interested in the wider public reception of his work which received a lot of attention. I am strictly interested in using the debate to illustrate a narrow, strictly methodological point about the nature of description. I used citation scores as a second criterion for their selection. I used those scores as a proxy measure for the scholarly consensus on the validity of the assessment of Goldhagen and Browning’s work. Using the term in strictly figurative sense, these selection criteria constituted my “sampling” strategy.
It should add that the works from which I could sample from where heavily skewed in favor of Browning and against Goldhagen. Of the 39 works (excluding Goldhagens self-defenses), only one scholar defended Goldhagen (Markovits 1998) and one article remained neutral. (Mahoney and Ellsberg 1999) I attribute this skewedness to the scholarly consensus on the flaws in Goldhagen’s analysis. I cannot confirm this consensus because I did not analyze the primary materials myself. Goldhagen critics, for example, could have been motivated less by his factual errors and more by his oftentimes condescending tone of earlier scholars’ work, professional animosity towards a non-historian (Goldhagen was a political scientist and most of his critics historians), or sheer envy of the book’s commercial success, and very positive public (i.e. none scholarly) reception, particularly in Germany. (Deak 1997, 297)
1.4. Logic of Annotation:
Broadly speaking, this paper’s ATI tries to increase confidence in its claims by foregrounding two sets of research judgments that are usually not acknowledged in conventional footnotes. The first set of judgments relate to the selection, and interpretation of empirical evidence used to support my claims. These are the very judgments that the conventional ATI is trying to foreground. I will refer to the foregrounding of those judgements as research transparency. The second set of judgments veers a bit farther from this more conventional use of ATI. These judgments related to broader elements of knowledge production that also influence the confidence in our research findings independently on how the evidence was handled. I will refer to the foregrounding of those judgments as research integrity. The rest of this section elaborates on these two logics and identifies the specific annotation sub-categories it uses. Each bold-faced term in the text indicates a distinct sub-category of either research transparency or research integrity. Each subsequent annotation in turn is assigned on of these sub-categories.
Parenthetically, I should point out that this paper does not aspire to a comprehensive ATI annotation in which every element of research transparency and research integrity are addressed and foregrounded. The invitation to annotate this paper came after it was completed, thus making it impossible to retrace each of its research judgments. I instead selected those judgements that were both important for the paper, and where I still could retrace in my notes the original judgements. The paper thus practices only part of what it preaches. But I still hope that those annotations will both increase the confidence in my findings, and more importantly, illustrate the range of possible ATI applications.
Research Transparency: The vast majority of my annotations follow the standard logic of using ATI to make transparent three key judgments related the evidence used to evaluate theoretical claims. These claims follow closely the three categories outlined by the Data Access and Research Transparency (DA-RT) initiative: data access, production transparency, and analytical transparency. It places particular attention on analytical transparency because those criteria are still evolving with respect to qualitative textual or ethnographic evidence. (Buthe & Jacobs, forthcoming, 2015; Golden & Golden 2016) This paper’s effort to specify criteria to evaluate descriptive inferences is part of this broader effort to make more explicit the criteria for qualitative evidence that analytical transparency should address. For example, the current specifications of analytical transparency don’t ask authors to be explicit about the conceptualization, ontological parameter, epistemological presuppositions, or cross-level inferences, four element this paper highlights. In another paper, I tried to be more specific elements of analytical transparency that apply to causal as opposed to descriptive inferences. Those elements include assessment of prior research, the specificity of theoretical predictions, units of analysis, and ontological assumptions. (Kreuzer, no date) Based on this prior work, I would like to add conceptual transparency as an additional ATI category to create the possibility to elaborate more fully on terminological or conceptual ambiguities.
Research Integrity: The paper looks at research integrity as broader category than research transparency that affects confidence in scholarly findings. Research integrity is not as clearly established as research transparency. The term was first proposed by Peter Hall (2016) to underscore the limitations of the narrower, more test-centric DA-RT research transparency guidelines. It has been espoused by scholars in the recent Qualitative Transparency Deliberations (QTD) to point to a wider range of research practices that affect the integrity of the research process even though they are distinct from the final testing stage. (Buthe Jacobs, forthcoming) Research integrity sees research transparency as one but not the exclusive focus for Annotation for Transparency Inquiry. (ATI) As Figure 1 shows, research integrity raises the possibility that ATI also address question related to research ethics, knowledge production, and epistemological priors. Various journals require scholars to disclose conflicts of interests or funding sources to make potential ethical conflicts more transparent. ATI would permit more substantial discussions in instances where such conflicts exist. I do not include any ATI related to ethical conflicts.
Production of knowledge is a broad umbrella term used in history of science or sociology of knowledge to explore how the personal, professional, economic, and political circumstances shape the research process independently of the final, empirical testing stage. This category is fairly broad but could include elements like the publication process, disciplinary variations in standards for research transparency, professional rank of authors relative to each other, the positionality of the scholar, the broader, public reception of the work, or funding agencies. Finally, epistemological priors try to explicate broader philosophical presuppositions that might cast doubt on research findings because they are not widely shared or incommensurate with other works that a scholar engages. For this paper, these epistemological priors were sufficiently important that it addresses them in section 1.2. rather than relegating them to an ATI.
Sources Cited:
Büthe, Tim and Alan Jacobs, eds. “Qualitative Transparency Deliberations Report,” Perspectives on Politics, forthcoming.
Büthe, Tim, and Alan Jacobs, eds. 2015. “Symposium: Transparency in Qualitative and Multi-Method Research.” Qualitative and Multi-Method Research 13(1).
Deák, István. 1997. “Holocaust Views: The Goldhagen Controversy in Retrospect.” Central European History 30(2): 295–3
Golden, Matt, and Sona Golden, eds. 2016. “Symposium: Data Access and Research Transparency.” Comparative Politics Newsletter 26(1).
Kreuzer, Marcus. Unpublished paper. “DATA FILES, FOOTNOTES, AND EDITORS: Bridging Quantitative, Qualitative, and Editorial Transparency Practice
Hall, Peter. 2016. “Transparency, Research Integrity, and Multiple Methods.” Comparative Politics Newsletter 26(1 (Spring)): 28–32
Mahoney, James, and Michael Ellsberg. 1999. “Goldhagen’s Hitler’s Willing Executioners: A Clarification and Methodological Critique.” Journal of Historical Sociology 12(4): 422–436
Markovits, Andrei. 1998. “Discomposure in History’s Final Resting Place.” In Unwilling Germans? The Goldhagen Debate, ed. Robert R. Shandley. Minneapolis: University of Minnesota Press.
Data for: Pastoralist decision-making on the Tibetan plateau
No full textThis is an Annotation for Transparent Inquiry (ATI) data project. The annotated article can be viewed on the publisher's website.
You will need to use the Chrome browser with the Hypothesis extension installed to view the ATI annotations
Data Generation & Analysis
This article is one of six resulting from an interdisciplinary, collaborative NSF-funded project that sought to investigate the multiple (social and ecological) correlates of rangeland status on the Tibetan Plateau. The project was motivated by the fact that rigorous studies of putative causes of rangeland degradation on the Tibetan Plateau have been lacking, and as a result, policies are being implemented without clear rationale or evidence. These same policies are often both ecologically ineffective and socially and culturally detrimental to Tibetan communities. The project combined repeated vegetation sampling on fixed plots, an exclosure experiment, and GPS tracking of livestock with intensive interviews with herders. While several of the articles resulting from the study are very interdisciplinary in nature, this article is primarily an interpretive article based on qualitative data, though some of the arguments are made in relation to the results of the vegetation studies.
More specifically, this article identifies three major assumptions of the Chinese state’s grassland policies, and uses qualitative evidence to refute each one. Data Analysis
Because the primary goal of the larger project was to understand specific management practices of pastoralist households and correlate them to their vegetation outcomes, and because of the intensive nature of the vegetation sampling, we chose as participating households the ones on whose pastures the biophysical measurements were being done. Coauthor Gaerrang conducted at least two sets of interviews with each household from June through August 2009, as well as follow-up interviews in summer 2010. He also collected numerous village-level statistics and documents relevant to grassland management. Coauthors Yeh and Harris also conducted interviews with Gaerrang in July 2009. Gaerrang followed up with phone calls in 2011 to households with clarifying questions after data analysis began. Coauthor Samberg made a visit to Gouli in 2013 and coauthor Volkmar conducted a final round of follow-up interviews in summer 2014. Most of the interviews were conducted in the local dialect of Tibetan by Gaerrang, and in other cases through translation by him or one of several Tibetan field assistants. Some of the Tibetan interviews were recorded and then transcribed into English, while others took notes directly in English based on translation.
In hindsight, for this article it would have been helpful to have interviewed more households. However, we were focused at the time of the design of the study on teasing out differences in management on the pastures on which we were collecting biophysical data. In making some of the arguments presented here, I drew upon broader contextual knowledge of processes across Tibetan pastoral areas, such as about ideas of the “essence” of the soil (sa bcud).
Analysis of the qualitative interview data used for this article was done through organizing and coding of interview transcript files and policy text files. I used descriptive and analytical codes to analyze the data. Most of the data were interview transcripts, but they also included tables of livestock holdings over time (often with incomplete entries), as well as summaries of management over time by pasture (rather than herder) written by Gaerrang based on his extended observations and first round of interviews. Particular attention was given to contradictory statements, such as when herders seemed to invoke a non-equilibrium understanding of rangeland ecology in some interviews, but not in others. Important coding categories also included precipitation/drought, snowstorms, livestock number, labor, and numerous types of contracts. Discourse analysis of policy documents resulted in the identification of the three arguments around which the article is based.
Logic of Annotation
The vast majority of annotations are to source excerpt translations: English excerpts of interview transcripts. This is for several reasons: the secondary sources are all already fully cited and easily available, and second, most of the data collected and analyzed was in the form of interviews, rather than written documents. Interviews are most relevant to examine how we drew conclusions, but space limitations for the article meant that only a small number could be selected. The annotations thus largely provide further examples in support of the observations made or conclusions drawn. While Tibetan language audio recordings do exist in some cases, only a very small fraction of potential readers would be able to make use of them. For annotation purposes, the excerpt translations had to have herders’ names removed, and they were also edited for grammar.
Though most of the annotations are interview excerpts, I also strove where possible to provide other forms of documentation that would provide greater transparency. In particular, I included our initial, lengthy semi-structured interview guide for the reader to be able to see the instrument used to collect data. I also included a photograph of the field site for further contextualization, as well as two difficult-to-obtain documents, one in Chinese and one in Tibetan, that provide further context and evidence on the policy assumptions around which the paper is framed.
For the most part, I did not feel that the arguments made in the article were particularly controversial or contested in the research community, and thus I did not use analytical notes to refute potential criticisms. Given the nature of argumentation in this article, they largely support descriptive claims.</p