16 research outputs found
Internet Appendix to: The Road to False Positives: Sample Selection and Specification Choice in Randomized and Natural Experiments
No full textThis Appendix contains additional results and discussions to accompany Black, Desai, Litvak, Yoo, and Yu, The Road to False Positives: Sample Selection and Specification Choice in Randomized and Natural Experiments. It contains sample details; graphs for each of the specifications and regression details for our specification, the best-match specifications for the re-examined papers and results for step-by-step moves from our specification to the best-match specification along with reported results for each of the 10 re-examined papers
Pandemic GPAs
No full textA student’s grade point average (GPA) is very important to most scholarship committees, potential employers, and graduate school selection committees. Some students place such importance on grades that they actively manipulate their GPA higher by taking actions that raise grades without additional study and learning. Students can seek out easy professors, transfer grades for difficult topics from community colleges, try to gain favoritism with their professors, or cheat on assignments. During the Covid-19 pandemic, universities offered grading leniency, such as allowing students to switch grades to credit / no credit after the semester’s end, which artificially inflated grades. We apply Felton and Koper’s (2005) proposal for having two grades on a transcript, Nominal GPA and Real GPA, as an easy to understand and apply method for dealing with some of the methods of GPA manipulation
The Impact of COVID-19 on Commodity and S&P 500 Sector Return Volatility
No full textWe investigate the impact of COVID-19 on commodity return volatility. We find that the impact of COVID-19 on return volatility is different across different markets. Unlike S&P 500 sector indices, commodity return volatility is less sensitive to the impact of COVID-19. The impact of vaccination programs on return volatility is weak for both commodity and financial markets. We employ Fama-French 3 Factor Model and APARCH (1,1) for return volatility estimation. The variation in COVID-19’s impact across different markets has an important implication for return volatility hedging
Specification Choice in Randomized and Natural Experiments: Lessons from the Regulation SHO Experiment
No full textDuring 2005-2007, the SEC conducted a randomized trial in which it removed short-sale restrictions from one-third of the Russell 3000 firms. Early studies found modest microstructure-related effects of removing the restrictions but no effect on short interest or share prices. More recently, however, many studies have attributed a wide range of substantive indirect outcomes to this experiment. We revisit the principal findings in four recent studies in major journals, using a sample that closely matches the actual experiment and a pre-specified research design, and find no support for any of the reported results. If we instead match their specifications as best we can based on the published descriptions, we still obtain quite different results; only two of 13 outcomes are statistically significant, barely so, and even those results are tenuous. For two papers, we have the authors’ original data and code. We can technically replicate two (different) results, but those results are highly sensitive to specification. Our findings have implications for the robustness of other studies finding indirect effects of the short-sale experiment. More broadly, they have implications for the credibility of “causal” research designs which rely on randomization or on natural experiments. Researchers retain extensive discretion over both the sample and model specification, even for a true randomized experiment. The choices in these studies produced significant results, when other reasonable choices would not
The SEC\u27s Short-Sale Experiment: Evidence on Causal Channels and Reassessment of Indirect Effects
No full textDuring 2005–2007, the Securities and Exchange Commission conducted a randomized trial in which it removed short-sale restrictions from one third of the Russell 3000 firms. Early studies found modest market microstructure effects of removing the restrictions but no effect on short interest, stock returns, volatility, or price efficiency. More recently, many studies have attributed a wide range of indirect outcomes to this experiment, mostly without assessing the causal channels for those outcomes. We examine the three most often cited causal channels for these indirect effects: short interest, returns, and managerial fear. We find no evidence to support these channels. We then reexamine the principal findings in four recent studies using a sample that closely matches the actual experiment and a common research design and find minimal support for the reported indirect effects. Our findings highlight the importance of confirming a causal channel or an economic mechanism and show that sample selection and specification choices can produce statistical significance even without an underlying economic mechanism
The Road to False Positives: Sample Selection and Specification Choice in Randomized and Natural Experiments
No full textDuring 2005-2007, the SEC conducted a randomized trial in which it removed shortsale restrictions from one-third of the Russell 3000 firms. Early studies found no effect of removing the restrictions on short interest, share returns, volatility, or price efficiency. In prior work, we confirm the lack of evidence for a natural causal channel that could explain these results (Black et al., 2024). Yet over 80 studies report evidence for a wide range of indirect effects on firms from the experiment. Given the lack of a causal channel, many of these results are likely to be false positives. We confirm that suspicion by closely reexamining the principal results from 10 of these studies using a simple specification with firm and year fixed effects. None of the results survive. We then examine best-match specifications that closely follow the sample selection, methodology and specification reported in the respective papers. Again, we mostly obtain null results. The gaps between reported and best match results reflects reported coefficients being much larger in magnitude, reported standard errors being much smaller than best-match, or both. The large gaps between best-match and reported results implies that the study authors made choices that are not evident from the explanation of their research design, which strongly affected the reported results. Our results suggest that researchers retain extensive discretion over the sample and model specification, even (as here) for a true randomized experiment. The choices in these studies produced statistically significant results when other reasonable choices would not. We draw lessons from our analysis for empirical practice and for ways in which researchers can find false positives results
Solution-free and simplified H&E staining using a hydrogel-based stamping technology
No full textHematoxylin and eosin (H&E) staining has been widely used as a fundamental and essential tool for diagnosing diseases and understanding biological phenomena by observing cellular arrangements and tissue morphological changes. However, conventional staining methods commonly involve solution-based, complex, multistep processes that are susceptible to user-handling errors. Moreover, inconsistent staining results owing to staining artifacts pose real challenges for accurate diagnosis. This study introduces a solution-free H&E staining method based on agarose hydrogel patches that is expected to represent a valuable tool to overcome the limitations of the solution-based approach. Using two agarose gel-based hydrogel patches containing hematoxylin and eosin dyes, H&E staining can be performed through serial stamping processes, minimizing color variation from handling errors. This method allows easy adjustments of the staining color by controlling the stamping time, effectively addressing variations in staining results caused by various artifacts, such as tissue processing and thickness. Moreover, the solution-free approach eliminates the need for water, making it applicable even in environmentally limited middle- and low-income countries, while still achieving a staining quality equivalent to that of the conventional method. In summary, this hydrogel-based H&E staining method can be used by researchers and medical professionals in resource-limited settings as a powerful tool to diagnose and understand biological phenomena
