139,364 research outputs found
Guideline aggregation: web accessibility evaluation for older users
No full textWeb site evaluation methodologies and validation engines take the
view that all accessibility guidelines must be met to gain compli-
ance. Problems exist in this regard as contradictions within the
rule set may arise, and the type of impairment or its severity is not
isolated. The Barrier Walkthrough (BW) method goes someway
to addressing these issues by enabling barrier types derived from
guidelines to be applied to different user categories such as motor
or hearing impairment, etc. In this paper, we use set theory to cre-
ate a validation scheme for older users by combining barrier types
specific to motor impaired and low vision users, thereby creating a
new “older users” category from the results of this set addition. To
evaluate this approach, we have conducted a BW study with four
pages, 19 expert and 49 non-expert judges. This study shows that
the BW generates reliable data for the proposed aggregated user
category and shows how experts and non-experts evaluate pages
differently. The study also highlights a limitation of the BW by
showing that a better aggregated user category would have been
created by having a severity level of disability for different impair-
ment types. By extending the BW with these impairment levels, we
argue that the BW would become more useful for validating Web
pages when dealing with users which multiple disabilities and thus
we would be able to create a “Personalised Validation and Repair”
method
The Expertise Effect on Web Accessibility Evaluation Methods
No full textWeb accessibility means that disabled people can effectively perceive, understand, navigate, and interact with the web. Web accessibility evaluation methods are needed to validate the accessibility of web pages. However, the role of subjectivity and of expertise in such methods is unknown and has not previously been studied. This article investigates the effect of expertise in web accessibility evaluation methods by conducting a Barrier Walkthrough (BW) study with 19 expert and 57 nonexpert judges. The BW method is an evaluation method that can be used to manually assess the accessibility of web pages for different user groups such as motor impaired, low vision, blind, and mobile users.
Our results show that expertise matters, and even though the effect of expertise varies depending on the metric used to measure quality, the level of expertise is an important factor in the quality of accessibility evaluation of web pages. In brief, when pages are evaluated with nonexperts, we observe a drop in validity and reliability. We also observe a negative monotonic relationship between number of judges and reproducibility: more evaluators mean more diverse outputs. After five experts, reproducibility stabilizes, but this is not the case with nonexperts. The ability to detect all the problems increases with the number of judges: With 3 experts all problems can be found, but for such a level 14 nonexperts are needed. Even though our data show that experts rated pages differently, the difference is quite small. Finally, compared to nonexperts, experts spent much less time and the variability among them is smaller, they were significantly more confident, and they rated themselves as being more productive. The article discusses practical implications regarding how BW results should be interpreted, how to recruit evaluators, and what happens when more than one evaluator is hired.
Supplemental materials are available for this article. Go to the publisher's online edition of Human–Computer Interaction for statistical details and additional measures for this article
Web Accessibility Guideline Aggregation for Older Users and Its Validation
No full textWeb site-evaluation methodologies and validation engines take the view that all accessibility guidelines must be met to gain compliance. Problems exist in this regard, as contradictions within the rule set may arise, and the type of impairment or its severity is not isolated. The Barrier Walkthrough (BW) method goes someway to addressing these issues, by enabling barrier types derived from guidelines to be applied to different user categories such as motor or visual impairment, etc. However, the problem remains of combinatorial explosion of possibilities when one has to consider users with multiple disabilities. In this paper, a simple set theory operation is used to create a validation scheme for older users by aggregating barrier types specific to motor impaired and low-vision users, thereby creating a new “older users” category from the results of this set union. To evaluate the feasibility and validity of this aggregation approach, two BW experiments were conducted. The first experiment evaluated the aggregated results by focusing on quality attributes and showed that aggregation generates data whose quality is comparable to the original one. However, this first experiment could not test for validity, as the older users category was not included. To remedy this deficiency, another BW experiment was conducted with expert judges who evaluated a web page in the context of older users. In this second experiment, it was found that there is no significant difference between the aggregated and the manually evaluated (by experts) barrier scores, and that the same barriers are identified using experts and aggregation, even though there are differences in how severity scores are distributed. From these results, it is concluded that the aggregation of barriers is a viable alternative to expert evaluation, when the target of that aggregation could not be evaluated manually or it would not be feasible to do so. It is also argued that aggregation is a technique that can be used in combination with other evaluation methods, like user testing or subjective assessments
Barriers common to mobile and disabled web users
No full textWorld Wide Web accessibility and best practice audits and evaluations are becoming increasingly complicated, time consuming, and costly because of the increasing number of conformance criteria which need to be tested. In the case of web access by disabled users and mobile users, a number of commonalities have been identified in usage, which have been termed situationally-induced impairments; in effect the barriers experienced by mobile web users have been likened to those of visually disabled and motor impaired users. In this case, we became interested in understanding if it was possible to evaluate the problems of mobile web users in terms of the aggregation of barriers-to-access experienced by disabled users; and in this way attempt to reduce the need for the evaluation of the additional conformance criteria associated with mobile web best practice guidelines. We used the Barrier Walkthrough (BW) method as our analytical framework. Capable of being used to evaluate accessibility in both the disabled and mobile contexts, the BW method would also enable testing and aggregation of barriers across our target user groups.
We tested 61 barriers across four user groups each over four pages with 19 experts and 57 non-experts focusing on the validity and reliability of our results. We found that 58% of the barrier types that were correctly found were identified as common between mobile and disabled users. Further, if our aggregated barriers alone were used to test for mobile conformance only four barrier types would be missed. Our results also showed that mobile users and low vision users have the most common barrier types, while low vision and motor impaired users experiencing similar rates of severity in the barriers they experienced. We conclude that the aggregated evaluation results for blind, low vision and motor impaired users can be used to approximate the evaluation results for mobile web users
Spoken Language, Conversational Assistive Systems for People with Cognitive Impairments? – Yes, If.
No full textYaghoubzadeh R, Kopp S. Spoken Language, Conversational Assistive Systems for People with Cognitive Impairments? – Yes, If. In: Yesilada Y, ed. ASSETS '15: Proceedings of the 17th International ACM SIGACCESS Conference on Computers & Accessibility. New York, NY: ACM; 2015
Group versus Individual Web Accessibility Evaluations: Effects with Novice Evaluators
No full textWe present an experiment comparing performance of 20 novice evaluators of accessibility carrying out Web Content Accessibility Guidelines 2.0 conformance reviews working individually to performance obtained when they work in teams of two. They were asked to first carry out an individual assessment of a web page. Later on, they were matched randomly to constitute a group of two and they were asked to revise their initial assessment and to produce a group assessment of the same page. Results indicate that significant differences were found for sensitivity (inversely related to false negatives: +8%) and agreement (when measured in terms of the majority view: +10%). Members of groups exhibited strong agreement on the evaluation results among them and with the group outcome. Other measures of validity and reliability are not significantly affected by group work. Practical implications of these findings are that, for example, when it is important to reduce the false-negative rate, then employing a group of two people is more useful than having individuals carrying out the assessment. Openings for future research include further explorations of whether similar results hold for groups larger than two or what is the effect of mixing people with different accessibility background. RESEARCH HIGHLIGHTS When novice accessibility evaluators work in groups, their ability to identify all the true problems increases (by 8%). Likewise, reliability of group evaluations increases (by 10%). Individual or group evaluations can be considered as equivalent methods with respect to false positives (if differences up to 8% in correctness are tolerated). Individual or group evaluations can be considered as equivalent methods with respect to overall effectiveness (if differences up to 11% in F-measure are tolerated)
Education
No full textThis chapter explores the main issues concerned with the task of designing accessible e-learning systems. While important steps forward have been accomplished to provide students with digital, easy to manage didactical resources, yet, barriers for learners with disabilities remain. Actually, even if such users are not completely ruled out from virtual classrooms, it is a fact that they can only partially enjoy the offered didactical experiences. With this in view, the current trend in research is concerned with the development of smart elearning systems, able to dynamically customize learning contents based on the specific user characteristics and needs. We provide a survey of current solutions and present our opinion of
the field
Education
No full textThis chapter explores the main issues concerned with the task of designing accessible e-learning systems. While important steps forward have been accomplished to provide students with digital, easy to manage didactical resources, yet, barriers for learners with disabilities remain. Actually, even if such users are not completely ruled out from virtual classrooms, it is a fact that they can only partially enjoy the offered didactical experiences. With this in view, the current trend in research is concerned with the development of smart elearning systems, able to dynamically customize learning contents based on the specific user characteristics and needs. We provide a survey of current solutions and present our opinion of
the field
Testability and Validity of WCAG 2.0: The Expertise Effect
No full textWeb Content Accessibility Guidelines 2.0 (WCAG 2.0) require that success criteria be tested by human inspection. Further, testability of WCAG 2.0 criteria is achieved if 80% of knowledgeable inspectors agree that the criteria has been met or not. In this paper we investigate the very core WCAG 2.0, being their ability to determine web content accessibility conformance. We conducted an empirical study to ascertain the testability of WCAG 2.0 success criteria when experts and non-experts evaluated four relatively complex web pages; and the differences between the two. Further, we discuss the validity of the evaluations generated by these inspectors and look at the differences in validity due to expertise.
In summary, our study, comprising 22 experts and 27 non-experts, shows that approximately 50% of success criteria fail to meet the 80% agreement threshold; experts produce 20% false positives and miss 32% of the true problems. We also compared the performance of experts against that of non-experts and found that agreement for the non-experts dropped by 6%, false positives reach 42% and false negatives 49%. This suggests that in many cases WCAG 2.0 conformance cannot be tested by human inspection to a level where it is believed that at least 80% of knowledgeable human evaluators would agree on the conclusion. Why experts fail to meet the 80% threshold and what can be done to help achieve this level are the subjects of further investigation
Scientific Documents
No full textScientific documents are a very specialised type of literature not only in terms of their topics and intended audience, but also in terms of their content and how it is presented. They generally use highly topical vernacular, mathematical formulas, diagrams, data visualisations, etc. While any single one of these features on its own poses a considerable accessibility problem, their combination makes the accessibility of scientific literature particularly challenging. However, with nearly all aspects of learning, teaching, and research moving to the web, there is a need to specifically address this problem for science on the web. In this chapter, we present an overview of the main challenges that arise when making scientific texts accessible. We will particularly concentrate on the accessibility problem for scientific diagrams and discuss the more common techniques for making them accessible via screenreading, sonification and audio-tactile presentation. This chapter gives an overview of the current state of the art, sketches some of the technical details on how to create accessible diagrams and closes with some open research question
- …
