UIST 2026 Doctoral Symposium poster

Visual Communication for Low-Vision Information Access: From Personalized Highlights to AI Agents

Yotam Sechayk, The University of Tokyo

My thesis. Personalized AI that communicates visually, supporting the independence and agency of low-vision people.

About me

Yotam Sechayk, PhD candidate at the University of Tokyo, Igarashi Lab. BS in Computer Science and Film, Tel Aviv University; MSc, University of Tokyo. Graduating March 2027.

“Motivated by my lived experience with low vision”

Motivation

295 million people have low vision, and many prefer using their residual vision. Yet most research turns visuals into other modalities like speech or touch. Magnification techniques support residual vision, but they fragment the view and increase cognitive effort.

How can we communicate visual information in a visually accessible way?

Two directions of visual communication

  • Tool → UserProjects 1 and 2
    Tools that visually communicate visual information to users, making the experience of visual perception accessible.
  • User → AIProject 3
    Users communicate visually with AI tools by sharing their view or indicating a point of interest, and get personalized support based on how they use their assistive tools.
  • Both waysProject 4 (future)
    AI systems that share the user’s visual context and also communicate visually with users, making visuals visually accessible.

Towards ability-based AI systems that build on the residual vision and assistive tool use of people with low vision, for personalized, in-situ visual support through visuals.

Project 1: VeasyGuide

Published at ASSETS 2025 · Direction: Tool → User · Paper

Three screenshots of VeasyGuide. Left, the VeasyGuide player: a lecture slide titled Machine Learning where the instructor's pointer action is marked with a red bounding box and a pointer icon. Middle, the zoomed-in view of the same slide, toggled on demand with the Z key. Right, personalization settings for the highlight's fill color, opacity, border color, and border width.

Problem. Instructors visually communicate by pointing, marking, or sketching on slides, but this communication is often hard to access and follow for people with low vision.

Approach. Personalized highlights that follow instructor actions, co-designed with 3 low-vision partners.

Result. Learners noticed 88% of actions vs 61% without, with lower cognitive load, and more agency.

Lesson. Making visual communication accessible using visual cues aids both low-vision and sighted learners.

Project 2: On-Cursor Visual Context

Published at CHI 2026 · Direction: Tool → User · Paper

Left: a bar chart labeled with what provides context in chart reading: the overview, axes, legend, and grid lines. Middle, Dynamic Context: under magnification, the axis values, legend, and a crosshair are projected into a compact area around the pointer. Right, Mini-map: a small overview of the whole chart with a position indicator, shown next to the magnified region.

Problem. Magnification makes chart details visible, but out-of-view context is fragmented and hard to access.

Approach. Bring chart context to the pointer: Dynamic Context and Mini-map project context (axes, legend, grid lines, overview) around it.

Result. For 22 low-vision participants, Dynamic Context improved access and reduced effort.

Lesson. There is a delicate tradeoff between visual information cues and overwhelming amount of visuals.

Project 3: Visual Context for AI

Ongoing · Direction: User → AI

Diagram of the chart assistant. Left: a magnified bar chart of monthly government budget surplus and deficit, with the pointer on a green surplus bar, next to a chat panel with a [new point] message, the reply “This is the surplus in July 2025, with value about 20B”, and the question “What is the second highest surplus?”. Middle: the AI input context, the full chart with a red box marking the user's view and a yellow dot at the pointed bar, plus a crop of the user's view, tagged with spatial and semantic guidance. Right: the two AI answers, given in full in the caption.

Answers to “What is the second highest surplus?”

General AI (GAI): “The second highest surplus is 205B, in October 2025.”

Low-vision AI (LVAI): “The second highest surplus is 205B, in October 2025. It is outside your view, to the right, about one viewport width. The green bar you are pointing is the smallest surplus, 20B, and the highest, 257B, is visible at the left edge of your view.”

Idea. Visual communication as context for AI agents, with responses adapted to low vision, grounded in that context.

How it works

  1. User shows their magnified view, pointer, and intent.
  2. AI connects that view to the full chart, and reasons on it.
  3. AI response is adapted to low vision, relative to their view.

Lesson. Grounding the AI in the user’s view is necessary, but not sufficient: its answers should also adapt to specific low-vision needs.

Project 4: Visual Dialogue with AI

Future work · Direction: Both ways

Open questions

Tell, highlight, guide, or show?
Which visual feedback fits which user and task?
Independence or dependence?
Does visual dialogue with AI breed a new dependence?
How do we measure agency?
Would AI create a new type of AI-supported agency?

Next: the AI answers visually

  • Highlight a region
  • Guide along a path
  • Move the view

Making visual communication visually accessible for people with low vision and beyond.