AgentFixboard
Fixed (verified)Last checked browser-use#browser

Image-only click targets are serialized as a bare img

A grid of image-only click targets is serialized as bare tags such as [11514]<img />, so the agent cannot tell the targets apart. The linked issue does not give an exception name.

From GitHub

browser-use/browser-use

IssuePull request

Difficulty
Easy
Time
~5 min
Steps
2
Fixed in
0.13.9
Last checked

Verified fix: PR #5541 merged on . Fixed in 0.13.9 (first published release whose tag contains the merge commit). Last checked .

Symptom

A grid of image-only click targets is serialized as bare tags such as [11514]<img />, so the agent cannot tell the targets apart. The linked issue does not give an exception name.

Evidence what you'll see

  • The report's element list is a repeated pattern of a div, another div, and <img />, with no alt text, title, or accessible name on the image.
  • Turning on dom_highlight_elements=True draws numbered badges, and the report says those badges land on top of each other, so the overlay is unreadable and clicks land on the wrong target.
  • The report names browser-use 0.12.1. The operating system field is empty.

Why it happens

The text sent to the agent is a serialization of the DOM. An interactive parent whose only content is an image was written out without that image's alt, title, accessible name, or filename, so every card looked like the same empty tag. Numbered highlight badges were a separate overlay on the page, and on a tight grid they occupied the same spot.

Affected versions

browser-use 0.12.1 in the original report. The linked issue does not name an operating system or a browser version.

Fix steps

0 of 2 done

Your check marks stay in this browser only.

How to check it worked

  • The element list for a card whose image file is nested-card.png includes image_src=nested-card.png instead of only [11514]<img />.
  • alt maps to image_alt and aria-label maps to image_label. The merged example has alt Payment card, title Choose card, and aria-label Primary payment method, written as image_alt=Payment card image_title=Choose card image_label=Primary payment method. An image whose accessible name is Payment card via aria-label shows image_label=Payment card, not image_alt. A data: source does not appear.

If it still fails

  • A data: image with no alt, title, or aria-label still serializes with an empty image context. The same is true when the only filename would have come from an attribute longer than 4096 characters.
  • The fourth image under one parent is dropped. Pull request #5541 keeps image-0.png, image-1.png, and image-2.png, and leaves out image-3.png.
  • Numbered badges can still overlap. https://github.com/browser-use/browser-use/pull/4315 (Avoid overlapping highlight badges for dense image grids) was closed without merging. The linked issue does not give a released change that separates those badges.

Sources and credit

Summarized in our own words from these public reports. Credit goes to their authors.

Provided as is, without warranty. Try changes in a non-production environment and back up first. Security-related steps: verify against the official docs. Terms · Request removal or attribution change

Hit an error we don't cover? Submit it