166 research outputs found
The Effect of Perceived Task Difficulty on Operator Reliance and Trust in Decisions from AFRS
No full textCarragher and Hancock (2022) reported that knowing the identification decisions of a highly accurate Automated Facial Recognition System (AFRS) could improve the face matching performance of individual participants compared to their unassisted performance – but that such AFRS-assisted performance failed to reach the level achieved by the AFRS alone. A second study (Carragher, Sturman, & Hancock, in prep) shows that an operator’s trust in automation is positively correlated with gains in collaborative performance when working with the assistance of an AFRS. Participants who had greater trust in the AFRS benefited most when completing the task with the assistance of the AFRS, compared to participants with greater self-confidence (and less trust in the AFRS).
In these previous studies, participants have generally been told the accuracy of the AFRS they will be working with before the task begins (Carragher & Hancock, 2022). Yet, participants have not received any instruction about the difficulty of the face matching task itself. As is often written, perceptual face matching is a “surprisingly difficult” task, as the average observer achieves accuracy of around 80% in ideal conditions (Burton et al., 2010). In support of this perception, Carragher and colleagues (In Prep) noted that participant’s (n = 160) estimates of their likely accuracy in a face matching task fell from 71.9% before attempting the task, to 64.2% after the task. The perceived difficulty of the face matching task is one factor that may contribute to use or disuse of the AFRS, with Lee and See (2004) noting that automation use is higher when self-confidence is low (see also Lee & Moray, 1994). The aim of the current experiment is to investigate whether the perceived difficulty of the task influences the way operators rely on the AFRS as a decision-aid, in the context of a one-to-one face matching task. As part of this study, we will explore whether individual differences - such as trust in automation, self-confidence, and ability - modulate this effect.
The current pre-registration draws on methods and procedures that have previously been pre-registered in separate, but related, projects:
• Carragher, Sturman & Hancock - The effect of trust in automation on human use of simulated Automated Facial Recognition Systems as decision-aids in one-to-one forensic face matching tasks: https://osf.io/rjfup
• Hua & Carragher - A new procedure to measure operator reliance on identification decisions made by simulated Automated Facial Recognition Systems in one-to-one face matching tasks: https://osf.io/aevw
The Effect of Perceived Task Role on Operator Reliance and Trust in Decisions from AFRS
No full textOur previous research (Carragher & Hancock, 2022) found that although participants significantly improved their own face matching performance when shown the identification decisions of a highly accurate AFRS, the performance of the collaborative human-AFRS pair was sub-optimal, failing to reach the level of performance that the AFRS achieved alone. Human operators often failed to detect errors, while also overturning many of the system’s correct decisions. Our follow up study (Carragher, Sturman, & Hancock, in prep) showed that participants who express a higher degree of trust in automation appear to achieve greater gains in collaborative performance when assisted by the AFRS compared to those who report greater self-confidence (and therefore, less trust in automation).
In these previous studies, participants have received general instructions that they “should follow the decision from the AFRS when you believe that it is correct” and that “it is also important that you overturn the decisions from the AFRS when you believe it has made an error”. However, these instructions do not provide clear guidance to the participant (“operator”) about their precise role in this pairing. In some operational contexts, such as passport control, the human performs oversight over the AFRS (MacLeod & McLindin, 2021; FRONTEX, 2015). Conversely, it is possible that other roles might provide humans with an AFRS for assistance with their decision making. As such, the aim of the current experiment is to examine whether the perceived role of the operator might influence the performance of human-algorithm teams in the context of a one-to-one face matching task. A secondary aim is to investigate whether the individual attributes of the operator - such as their trust in automation, underlying ability, or confidence - modulate these results.
The current pre-registration draws on methods and procedures that have previously been pre-registered in separate, but related, projects:
• Carragher, Sturman & Hancock - The effect of trust in automation on human use of simulated Automated Facial Recognition Systems as decision-aids in one-to-one forensic face matching tasks: https://osf.io/rjfup
• Hua & Carragher - A new procedure to measure operator reliance on identification decisions made by simulated Automated Facial Recognition Systems in one-to-one face matching tasks: https://osf.io/aevwz
• Vohra, Sturman & Carragher - Does perceived task difficulty influence operator reliance and trust in decisions from a simulated Automated Facial Recognition System in a one-to-one face matching task? https://osf.io/ardp
Validating a New Trial Procedure for Assessing Human-Algorithm Teaming in One-to-One Face Matching Tasks
No full textA handful of recent studies have investigated human performance on one-to-one face matching tasks when assisted by an Automated Facial Recognition System (AFRS). The aim of the current study was to validate a new experimental trial procedure, which builds on that first introduced by Carragher and Hancock (2023). Here we recruited a final sample of 74 participants who completed the short version of the Glasgow Face Matching Test 2 (GFMT2-S), using our “new” trial procedure. These participants made an “independent” unassisted identification decision (“same” or “different”) for each pair of faces, before seeing the identification decision made by a simulated AFRS (accuracy = 95%). These participants then made a final “assisted” identification decision for the same trial. We compared the performance achieved by participants using this new trial procedure to that of a randomly selected subsample of 74 participants from Carragher et al. (2023), who completed the same face matching task with the same simulated AFRS, but using the “old” trial procedure introduced by Carragher and Hancock (2023). These participants made a single identification decision for each pair of faces, which was either independent or with assistance from the AFRS, depending on the experimental block. Our results show that the performance of the two trial procedure conditions did not differ on the primary outcome measure of overall accuracy. We conclude that our new trial procedure is suitable for further experimental investigation of human-algorithm teaming in the context of the one-to-one face matching task
Measurement of Fructose, Glucose, Maltose and Sucrose in Barley Malt Using Attenuated Total Reflectance Mid-infrared Spectroscopy
No full textThe objective of the present study was to develop a simple, rapid and accurate method for the determination of glucose, fructose, maltose and sucrose in barley malt using attenuated total reflectance (ATR) mid-infrared (MIR) spectroscopy. A total of 100 malt samples were analysed using an ATR-MIR instrument and the concentration of individual sugars determined using HPLC. Partial least squares (PLS) regression models yielded a coefficient of determination in cross validation (R2) and standard error in cross validation (SECV) of 0.64 (1.38 mg mL−1), 0.84 (0.12 mg mL−1), 0.80 (8.3 mg mL−1), and 0.60 (0.91 mg mL−1) for glucose, fructose, maltose and sucrose, respectively. This study demonstrated the potential benefits of ATR-MIR spectroscopy for the rapid measurement of the concentration of individual sugars in malt samples sourced from different commercial barley varieties, harvest seasons and localities.Yichao Huang, John Carragher, Daniel Cozzolin
Experiment 2: Wisdom of the Crowd for matching unfamiliar masked faces
No full textThis pre-registration plans our attempt to replicate the Wisdom of the Crowds (WOC) effects we first found in the re-analysis of the data reported in Carragher & Hancock (2020). Here we will collect new data to investigate whether we can replicate these WOC effects in a new face matching test (Kent Face Matching Test: Fysh & Bindemann, 2018), and using a different primary measure of sensitivity (Area Under the Curve). The aim of this experiment is to investigate whether the WOC effect can improve performance on an unfamiliar face matching task with masked faces
Limited evidence of hierarchical encoding in the cheerleader effect
No full textThis project holds the data files reported in the upcoming manuscript "Carragher, Thomas, Gwinn & Nicholls. (In Press). "Limited evidence of hierarchical encoding in the cheerleader effect, Scientific Reports"
The Effect of Time Pressure on AFRS Reliance in a Face Matching Task
No full textParticipants can significantly improve their own face matching performance when assisted by a highly accurate simulated Automated Facial Recognition System (AFRS; Carragher & Hancock, 2022). However, the performance of these human-AFRS teams was suboptimal, such that the AFRS achieved higher accuracy alone. This result points toward issues around operator misuse and/or disuse of the AFRS (Parasuraman & Riley, 1997). The aim of the current study is to investigate the operational factors that might influence reliance on the AFRS decision aid. One factor known to increase reliance on automated decision aids is task difficulty (Schwark et al., 2010). In the context of face matching tasks, time pressure (the amount of time that the stimulus is visible) is known to have a significant effect on accuracy. Fish and Bindemann (2017) reported that performance on a face matching task was highest when participants were given 10 seconds to view each face pair, and lowest when the presentation time was decreased to 2 seconds. Alenezi et al. (2015) reported that the average time taken to respond in a face matching experiment without time pressure was around 5 seconds. Here we investigate whether time pressure influences reliance on decisions from an AFRS in a one-to-one forensic face matching task
The dissociable influence of social context on judgements of facial attractiveness and trustworthiness
No full textThis OSF project contains the complete set of data files analysed and reported in Carragher, D. J., Thomas, N. A., & Nicholls, M. E. R. (2021). The dissociable influence of social context on judgements of facial attractiveness and trustworthiness. British Journal of Psychology. A link to the published article will be added here shortly. This project investigates whether "the cheerleader effect" also occurs for judgments of facial trustworthiness. Additionally, this project also investigates how the attributes of the target face, and those of the group, contribute to the final size of the cheerleader effect
Can feature instructions improve human face matching performance for masked faces?
No full textThis OSF project contains the pre-registration and data set reported in the Supplementary Materials of Carragher, D. J., Towler, A., Mileva, V. R., White, D., & Hancock, P.J.B. "Masked face identification is improved by diagnostic feature training". This project is currently undergoing peer-review. Updates to the project will be made periodically
Can feature instructions improve human face matching performance for masked faces?
No full textThis OSF project contains the pre-registration and data set reported in the Supplementary Materials of Carragher, D. J., Towler, A., Mileva, V. R., White, D., & Hancock, P.J.B. "Masked face identification is improved by diagnostic feature training". This project is currently undergoing peer-review. Updates to the project will be made periodically
- …
