Task Aware Browsing: AI Assisted Web Access for Blind Users

Date & Time

Thursday, March 16, 2023 - 9:20 AM PST  

Location

Platinum 2  

Description

A completely new approach for blind users to access web pages is presented. It uses machine learning and artificial intelligence to overcome the problems encountered by these users with conventional browsers and screen readers, particularly on an unfamiliar web site. Finding the location of key features, when accessing Internet web sites, is difficult and time consuming. This approach provides a method for finding active controls, and tracking and managing the results from invoking these controls. The new approach is called TAB: Task Aware Browsing, For a given vertical domain, e.g. shopping, travel, banking, etc., a set of keyboard commands is created that allow the user to perform the essential functions in that domain. The browser then uses a model generated from machine learning to convert each command into the appropriate actions for a learned web site. It filters the results of page updates to present the screen reader with only the essential portion of rendered page that is responsive to the command, eliminating excess clutter. Work performed to date indicates that machine learning techniques will allow this approach to work on a significant number of web sites. User testing demonstrates that it provides an appreciable improvement in the speed with which blind users can accomplish desired tasks.

This Presentation Link is provided by the Presenter(s) and not hosted by the Center on Disabilities at CSUN. The Center on Disabilities has confirmed, as of March 2, 2023, content linked is relevant to the presentation, but has not been reviewed for accessibility nor will the Center on Disabilities attempt to remediate any accessibility issues in the linked content. Please contact the Presenter(s) with any accessibility concerns.

 

Audience

• Higher Education
• Information & Communications Technology
• Research & Development

Audience Level

Intermediate  

Session Summary (Abstract)

AI can provide blind users with a command line interface to perform tasks on many web sites. This solves the common problem these users experience in navigating the web. Experimental results demonstrate the feasibility of this approach.  

Primary Topic

Blind/Low Vision  

Secondary Topics

• Artificial Intelligence (AI) & Machine Learning (ML)
• Engineering
• Web

 

Session Type

Journal Track  

Presenter

• David Cane
  Perkins School for the Blind

Add to Outlook/iCal (How it works)

Back to Session List