源码教程：点击、输入、键盘与鼠标

本章解释定位如何转为动作。核心辅助方法是 resolveIndex、resolveActionTarget、clickWithMode 和输入执行逻辑；遮挡与重复提交的排障步骤见第 08 篇。

1. 索引、CSS 与语义定位

命令	定位和执行方式
`click_element_by_index`	从快照解析 Locator，记录动作前状态，交给点击模式执行器
`click_element_by_selector`	在指定 Frame 解析 CSS；多匹配优先选择可接收事件的可见元素
`click_element_by_text`	按文本查找并选择实际目标，结合命中信息判断是否点到过大的父容器
`click_element_by_role`	使用无障碍 role 和可选 name 定位，不是按 CSS 类名
`hover_and_click`	定位后先 hover，等待可选延时，再执行点击；适合悬停菜单
`double_click_element_by_index`	对索引目标执行双击语义，不能简单宣称是两次业务提交
`hover_element_by_index`	调用 Locator.hover，可能触发浮层和新控件
`focus_element_by_index`	调用 Locator.focus，后续键盘事件仍需确认实际焦点
`check_element_by_index`、`uncheck_element_by_index`	使用勾选/取消勾选语义，避免不加判断地切换状态

resolveActionTarget 先 count，0 个匹配立即报告不存在；多个匹配扫描前 20 个候选，通过可见性和中心点命中优先选前景控件。它不是业务语义解析器，仍应写能限定目标的选择器。

2. 点击执行不是无条件重试

auto 先尝试原生点击。在普通失败且未发现遮挡时，根据配置尝试真实鼠标和 JS；已发现遮挡则停止，避免点击背景提交按钮。原生调用遇到对象释放异常时不补点，因为先前操作可能已经生效。

显式 native、mouse、js 分别约束执行方式。鼠标坐标点击不能自动证明目标就是该元素，JS 事件也可能被框架拒绝。回执应记录实际 mode、命中对象与观察结果；不能把“派发成功”写成“业务成功”。

json

{"id":"1001","method":"click_element_by_selector","params":{"selector":"#preview-button","mode":"native","timeoutMs":3000}}

3. 输入与框架状态

命令	实现要点
`input_text`	索引定位后执行覆盖输入，支持 mode；默认优先真实交互
`input_text_by_selector`	CSS 定位，可指定 frame，复用输入执行逻辑
`input_text_by_label`	按标签定位表单控件，再执行输入
`type_text`	调用 pressSequentially 逐字符输入，不先清空现有内容；适合依赖按键事件的控件
`clear_text`	按 index 或 selector 定位并清空；要检查框架是否接受修改

真实输入会参与浏览器事件流程；JS 降级通过属性 setter 和 input/change 事件设置值，但不保证框架模型接受，因此有 committed 等说明。只读、禁用、隐藏及不可编辑是不同失败原因，不应全部换成 JS 强写。

json

{"id":"1001","method":"input_text_by_selector","params":{"selector":"#display-name","text":"示例用户","mode":"type"}}

输入后用 get_form_state 回读，尤其是在 Windows shell 传中文时。传参完整性与浏览器是否成功填值是两个问题。

4. 下拉框、滚动与拖拽

get_dropdown_options 从原生 select 的 options 读取文本；select_dropdown_option 按 label 选择。由 div 模拟的下拉框没有原生 options，应展开后对实际选项点击，不能硬套 select 接口。

scroll 的参数是 down、numPages 和可选 index。没有 index 时循环发 PageDown/PageUp；有 index 时对对应容器执行 scrollBy，距离按视口高度计算。它不是接收任意像素数的接口。scroll_to_text 则定位文字并滚动到可见范围。

drag_element_by_index 解析源索引和 targetIndex，调用拖拽操作。布局变化后要重取两端索引，不能只更新源元素。

5. 键盘与鼠标是更低层的工具

命令	底层机制与边界
`send_keys`	按输入形态分流：单个键名与「若干修饰键 + 恰好一个键」的组合走 `press`，其余（整段文本、中文、空格）走逐字符 `type`；回执给 `mode` 与 `focused`，按 Enter 不代表业务提交成功
`key_down`、`key_up`	保持/释放键状态，应成对使用，避免残留修饰键
`mouse_move`	移动到页面坐标，可能触发 hover
`mouse_down`、`mouse_up`	按下/释放指定鼠标键，用于连续操作；「按住不放」读图表某一点就是 `mouse_move` → `mouse_down` → 读文本 → `mouse_up`，漏掉 `mouse_up` 会让后续点击变成拖拽
`mouse_wheel`	按 deltaY 发滚轮事件
`mouse_click`	按 x、y 发真实鼠标点击，可指定 button 与 clickCount；坐标点击不依赖 DOM 节点句柄，是元素持续变化、选择器频繁撞伪故障时的稳定退路
`mouse_click_by_selector`	找到目标盒子中心后发真实鼠标点击，不等于原生可操作性检查

坐标按浏览器页面坐标使用，不能直接复制经过缩放的截图像素。窗口尺寸、页面视口与截图像素差异见第 10 篇。

6. 回归测试设计

准备背景提交按钮、覆盖层和前景按钮，分别记录点击次数。验证 auto 不触发背景按钮，重复选择器选到前景控件；再注入对象释放异常，确认没有第二次点击。输入测试应检查框架监听器收到的值，而不是仅检查 DOM attribute。原生 select 和自定义下拉框使用不同 fixture，避免错误实现只在一种控件上通过。

注册参数与 Java 入口

以下按 CommandTable 实际读取参数整理。* 表示注册层使用必填读取器；其余字段省略后由服务决定默认行为。带条件的入口仍需满足正文说明，例如上传文件来源、元素定位二选一。外层 id 不重复列出。

命令	params 字段	Java 入口
`click_element_by_index`	`index`*、`mode`、`timeoutMs`	`clickElementByIndex`
`double_click_element_by_index`	`index`*、`mode`	`doubleClickElementByIndex`
`hover_element_by_index`	`index`*	`hoverElementByIndex`
`focus_element_by_index`	`index`*	`focusElementByIndex`
`check_element_by_index`	`index`*、`mode`	`checkElementByIndex`
`uncheck_element_by_index`	`index`*、`mode`	`uncheckElementByIndex`
`input_text`	`index`、`text`、`mode`	`inputTextByIndex`
`type_text`	`index`、`text`	`typeText`
`drag_element_by_index`	`index`、`targetIndex`	`dragElementByIndex`
`send_keys`	`keys`*（单个键名、组合键，或整段文本）	`sendKeys`
`press_key`（别名）	`key` 或 `keys`（二选一，均映射到 `sendKeys`）	`sendKeys`
`key_down`	`keys`*	`keyDown`
`key_up`	`keys`*	`keyUp`
`get_dropdown_options`	`index`*	`getDropdownOptions`
`select_dropdown_option`	`index`、`text`	`selectDropdownOption`
`scroll`	`down`、`numPages`*、`index`	`scroll`
`scroll_to_text`	`text`*	`scrollToText`
`click_element_by_selector`	`selector`*、`mode`、`timeoutMs`、`frame`	`clickElementBySelector`
`input_text_by_selector`	`selector`、`text`、`mode`、`frame`	`inputTextBySelector`
`click_element_by_text`	`text`*、`mode`	`clickElementByText`
`click_element_by_role`	`role`*、`name`、`mode`	`clickElementByRole`
`input_text_by_label`	`label`、`text`、`mode`	`inputTextByLabel`
`clear_text`	`index`、`selector`	`clearText`
`hover_and_click`	`index`、`selector`、`hoverDelayMs`、`mode`	`hoverAndClick`
`mouse_move`	`x`、`y`	`mouseMove`
`mouse_down`	`button`	`mouseDown`
`mouse_up`	`button`	`mouseUp`
`mouse_wheel`	`deltaY`*	`mouseWheel`
`mouse_click`	`x`、`y`、`button`、`clickCount`	`mouseClick`
`mouse_click_by_selector`	`selector`*、`button`、`clickCount`、`timeoutMs`	`mouseClickBySelector`

当前源码：命令注册与执行

先在本章上半部分理解行为，再按命令展开实现。注册代码说明 JSON 参数如何传给 Java；服务方法展示实际浏览器操作。方法依赖共享类中的字段和辅助函数，不应脱离原类直接粘贴编译。

`click_element_by_index`

展开参数注册

源码：playwright-server/src/main/java/nexus/io/ai/browser/actions/registry/CommandTable.java。以下为当前实现，可放回原类中阅读；依赖同类字段和辅助方法，并非独立编译单元。

java

    put("click_element_by_index",
        (svc, id, a) -> svc.clickElementByIndex(id, reqInt(a, "index"), optStr(a, "mode"), a.getInteger("timeoutMs")));

展开 clickElementByIndex 实现

源码：playwright-server/src/main/java/nexus/io/ai/browser/service/PlaywrightService.java。以下为当前实现，可放回原类中阅读；依赖同类字段和辅助方法，并非独立编译单元。

java

  public RespBodyVo clickElementByIndex(Long browserId, int index, String mode, Integer timeoutMs) {
    BrowserInstance inst = INSTANCES.get(browserId);
    if (inst == null) {
      return RespBodyVo.fail("没有找到对应的浏览器实例：" + browserId);
    }
    ActionOutcome outcome = new ActionOutcome();
    return indexAction(inst, index, "click_element_by_index", CLICKABLE_SELECTOR,
        (locator) -> clickWithMode(locator, mode, outcome, () -> locator.click(clickOptions(timeoutMs))), true,
        outcome.extra);
  }

`double_click_element_by_index`

展开参数注册

java

    put("double_click_element_by_index", (svc, id, a) -> svc.doubleClickElementByIndex(id, reqInt(a, "index"),
        optStr(a, "mode")));

展开 doubleClickElementByIndex 实现

java

  public RespBodyVo doubleClickElementByIndex(Long browserId, int index, String mode) {
    return clickLike(browserId, index, "double_click_element_by_index",
        (locator) -> locator.dblclick(new Locator.DblclickOptions().setTimeout(actionTimeoutMs())), true, mode);
  }

`hover_element_by_index`

展开参数注册

java

    put("hover_element_by_index", (svc, id, a) -> svc.hoverElementByIndex(id, reqInt(a, "index")));

展开 hoverElementByIndex 实现

java

  public RespBodyVo hoverElementByIndex(Long browserId, int index) {
    return clickLike(browserId, index, "hover_element_by_index",
        (locator) -> locator.hover(new Locator.HoverOptions().setTimeout(actionTimeoutMs())));
  }

`focus_element_by_index`

展开参数注册

java

    put("focus_element_by_index", (svc, id, a) -> svc.focusElementByIndex(id, reqInt(a, "index")));

展开 focusElementByIndex 实现

java

  public RespBodyVo focusElementByIndex(Long browserId, int index) {
    return clickLike(browserId, index, "focus_element_by_index",
        (locator) -> locator.focus(new Locator.FocusOptions().setTimeout(actionTimeoutMs())));
  }

`check_element_by_index`

展开参数注册

java

    put("check_element_by_index",
        (svc, id, a) -> svc.checkElementByIndex(id, reqInt(a, "index"), optStr(a, "mode")));

展开 checkElementByIndex 实现

java

  public RespBodyVo checkElementByIndex(Long browserId, int index, String mode) {
    return clickLike(browserId, index, "check_element_by_index",
        (locator) -> locator.check(new Locator.CheckOptions().setTimeout(actionTimeoutMs())), false, mode);
  }

`uncheck_element_by_index`

展开参数注册

java

    put("uncheck_element_by_index",
        (svc, id, a) -> svc.uncheckElementByIndex(id, reqInt(a, "index"), optStr(a, "mode")));

展开 uncheckElementByIndex 实现

java

  public RespBodyVo uncheckElementByIndex(Long browserId, int index, String mode) {
    return clickLike(browserId, index, "uncheck_element_by_index",
        (locator) -> locator.uncheck(new Locator.UncheckOptions().setTimeout(actionTimeoutMs())), false, mode);
  }

`input_text`

展开参数注册

java

    put("input_text",
        (svc, id, a) -> svc.inputTextByIndex(id, reqInt(a, "index"), reqStr(a, "text"), optStr(a, "mode")));

展开 inputTextByIndex 实现

java

  public RespBodyVo inputTextByIndex(Long browserId, int index, String value, String mode) {
    BrowserInstance inst = INSTANCES.get(browserId);
    if (inst == null) {
      return RespBodyVo.fail("没有找到对应的浏览器实例：" + browserId);
    }
    Locator locator = resolveIndex(inst, index, INPUT_SELECTOR);
    if (locator == null) {
      return RespBodyVo.fail("input_text 索引越界: " + index + indexHint(inst));
    }
    ActionOutcome outcome = new ActionOutcome();
    try {
      fillWithMode(locator, value, mode, outcome);
    } catch (PlaywrightException e) {
      return RespBodyVo.fail(actionFailure("input_text", e));
    }
    return inputResult(outcome);
  }

`type_text`

展开参数注册

java

    put("type_text", (svc, id, a) -> svc.typeText(id, reqInt(a, "index"), reqStr(a, "text")));

展开 typeText 实现

java

  public RespBodyVo typeText(Long browserId, int index, String text) {
    BrowserInstance inst = INSTANCES.get(browserId);
    if (inst == null) {
      return notFound(browserId);
    }
    Locator locator = locatorOf(inst, index);
    if (locator == null) {
      return RespBodyVo.fail("type_text 索引越界: " + index + indexHint(inst));
    }
    try {
      requireEditable(locator);
      locator.pressSequentially(text, new Locator.PressSequentiallyOptions().setTimeout(actionTimeoutMs()));
    } catch (PlaywrightException e) {
      return RespBodyVo.fail(actionFailure("type_text", e));
    }
    return RespBodyVo.ok();
  }

`drag_element_by_index`

展开参数注册

java

    put("drag_element_by_index",
        (svc, id, a) -> svc.dragElementByIndex(id, reqInt(a, "index"), reqInt(a, "targetIndex")));

展开 dragElementByIndex 实现

java

  public RespBodyVo dragElementByIndex(Long browserId, int index, int targetIndex) {
    BrowserInstance inst = INSTANCES.get(browserId);
    if (inst == null) {
      return notFound(browserId);
    }
    Locator from = locatorOf(inst, index);
    if (from == null) {
      return RespBodyVo.fail("drag_element_by_index 源索引越界: " + index + indexHint(inst));
    }
    Locator to = locatorOf(inst, targetIndex);
    if (to == null) {
      return RespBodyVo.fail("drag_element_by_index 目标索引越界: " + targetIndex + indexHint(inst));
    }
    try {
      from.dragTo(to, new Locator.DragToOptions().setTimeout(INDEX_ACTION_TIMEOUT_MS));
    } catch (PlaywrightException e) {
      return RespBodyVo.fail(actionFailure("drag_element_by_index", e));
    }
    return RespBodyVo.ok();
  }

`send_keys`（别名 `press_key`）

按键别名兼容说明：为了适配调用端与习惯写法，系统支持 press_key 作为 send_keys 的完全等价别名，且支持参数 key 或 keys。命令行客户端 dsb run press_key -p key=Enter 会自动重定向映射。

展开参数注册

java

    put("send_keys", (svc, id, a) -> svc.sendKeys(id, reqStr(a, "keys")));
    put("press_key", (svc, id, a) -> svc.sendKeys(id, a.containsKey("key") ? reqStr(a, "key") : reqStr(a, "keys")));

展开 sendKeys 实现

java

  public RespBodyVo sendKeys(Long browserId, String keys) {
    BrowserInstance inst = INSTANCES.get(browserId);
    if (inst == null) {
      return notFound(browserId);
    }
    boolean asText = typesAsText(keys);
    try {
      if (asText) {
        inst.page.keyboard().type(keys);
      } else {
        inst.page.keyboard().press(keys);
      }
    } catch (PlaywrightException e) {
      return RespBodyVo.fail("send_keys 失败：" + briefMessage(e.getMessage()));
    }
    // 按键**发给谁**必须回报:实测「填完标签输入框 → send_keys Enter」会悄悄什么也不做
    // (焦点已经不在那个 input 上,或者框架刚把输入框重置了),而回执只有一句 ok:true ——
    // 调用方只能靠「标签没多出来」反推,白跑一轮。把焦点元素写进回执,空操作一眼可见。
    Kv data = new Kv();
    data.set("mode", asText ? "type" : "press");
    if (asText) {
      data.set("text", keys);
    } else {
      data.set("keys", keys);
    }
    Kv focused = focusedElement(inst);
    if (focused != null) {
      data.set("focused", focused);
      if (Boolean.TRUE.equals(focused.getBoolean("isBody"))) {
        data.set("focusNote", (asText ? "这段文字" : "按键") + "发出时焦点在 <body> 上(不在任何输入控件里):"
            + "内容不会进入任何输入框 —— 先 click 目标输入框,再送字/送键");
      }
    }
    return RespBodyVo.ok(data);
  }

展开 typesAsText 与分流判据

java

  static boolean typesAsText(String keys) {
    if (keys == null || keys.isEmpty()) {
      return false;
    }
    if (SINGLE_KEY.matcher(keys).matches() || MODIFIER_COMBO.matcher(keys).matches()) {
      return false;
    }
    return true;
  }

  /** 功能键、编辑键、方向键、翻页键这类「键名」;只有它们才该走 press */
  private static final String KEY_NAME =
      "(?i)(Enter|Escape|Esc|Tab|Backspace|Delete|Insert|Home|End|PageUp|PageDown|ArrowUp|ArrowDown|ArrowLeft"
          + "|ArrowRight|Space|CapsLock|NumLock|ScrollLock|PrintScreen|ContextMenu|F([1-9]|1[0-2]))";

  /** 可叠加的修饰键:它们后面**必须**还有且只有一个真正的键 */
  private static final String MODIFIER = "(?i)(Control|Ctrl|Alt|Shift|Meta|Command|Cmd)";

  /** 单个键名 */
  private static final Pattern SINGLE_KEY = Pattern.compile(KEY_NAME);

  /** 组合键:「若干修饰键 + 恰好一个键」 */
  private static final Pattern MODIFIER_COMBO = Pattern.compile(
      "(?i)(" + MODIFIER + "\\+)*" + MODIFIER + "(\\+([A-Za-z0-9]|" + KEY_NAME + "))?");

keys 收两种形态，按输入形态分流，不猜意图：

输入	走哪条路	回执 `data.mode`
`Enter` / `Tab` / `ArrowDown` / `F5` 等单个键名	`press`	`press`
`Control+A` / `Shift+Enter` / `Control+Shift+T`（若干修饰键 + 恰好一个键）	`press`	`press`
其余一切文本（`CRCL`、`hello world`、中文）	`type`，逐字符打字并触发 `keydown/keypress/input`	`type`

回执另有 data.keys（按键形态）或 data.text（文本形态），以及始终存在的 data.focused。

需要区分对待的两点：

形如 Control+A+B 的输入不算组合键（press 只接受「若干修饰键 + 恰好一个键」），会按文本打字；
文本走的是逐字符按键，因此 Vue / React 这类受控输入框同样能收到值；这也意味着「隐藏输入框没有选择器、又不想先取快照拿索引」时，可以直接用这条命令把整段文本送进去。

`key_down`

展开参数注册

java

    put("key_down", (svc, id, a) -> svc.keyDown(id, reqStr(a, "keys")));

展开 keyDown 实现

java

  public RespBodyVo keyDown(Long browserId, String keys) {
    BrowserInstance inst = INSTANCES.get(browserId);
    if (inst == null) {
      return notFound(browserId);
    }
    try {
      inst.page.keyboard().down(keys);
    } catch (PlaywrightException e) {
      return RespBodyVo.fail("key_down 失败：" + briefMessage(e.getMessage()));
    }
    return RespBodyVo.ok();
  }

`key_up`

展开参数注册

java

    put("key_up", (svc, id, a) -> svc.keyUp(id, reqStr(a, "keys")));

展开 keyUp 实现

java

  public RespBodyVo keyUp(Long browserId, String keys) {
    BrowserInstance inst = INSTANCES.get(browserId);
    if (inst == null) {
      return notFound(browserId);
    }
    try {
      inst.page.keyboard().up(keys);
    } catch (PlaywrightException e) {
      return RespBodyVo.fail("key_up 失败：" + briefMessage(e.getMessage()));
    }
    return RespBodyVo.ok();
  }

`get_dropdown_options`

展开参数注册

java

    put("get_dropdown_options", (svc, id, a) -> svc.getDropdownOptions(id, reqInt(a, "index")));

展开 getDropdownOptions 实现

java

  public RespBodyVo getDropdownOptions(Long browserId, int index) {
    BrowserInstance inst = INSTANCES.get(browserId);
    if (inst == null) {
      return RespBodyVo.fail("没有找到对应的浏览器实例：" + browserId);
    }
    Locator locator = resolveIndex(inst, index, SELECT_SELECTOR);
    if (locator == null) {
      return RespBodyVo.fail("get_dropdown_options 索引越界: " + index + indexHint(inst));
    }
    try {
      Object options = locator.evaluate("el => Array.from(el.options).map(o => o.textContent)");
      return RespBodyVo.ok(Kv.by("options", options));
    } catch (PlaywrightException e) {
      return RespBodyVo.fail(actionFailure("get_dropdown_options", e));
    }
  }

`select_dropdown_option`

展开参数注册

java

    put("select_dropdown_option", (svc, id, a) -> svc.selectDropdownOption(id, reqInt(a, "index"), reqStr(a, "text")));

展开 selectDropdownOption 实现

java

  public RespBodyVo selectDropdownOption(Long browserId, int index, String text) {
    BrowserInstance inst = INSTANCES.get(browserId);
    if (inst == null) {
      return RespBodyVo.fail("没有找到对应的浏览器实例：" + browserId);
    }
    Locator locator = resolveIndex(inst, index, SELECT_SELECTOR);
    if (locator == null) {
      return RespBodyVo.fail("select_dropdown_option 索引越界: " + index + indexHint(inst));
    }
    try {
      locator.selectOption(new SelectOption().setLabel(text),
          new Locator.SelectOptionOptions().setTimeout(INDEX_ACTION_TIMEOUT_MS));
    } catch (PlaywrightException e) {
      return RespBodyVo.fail(actionFailure("select_dropdown_option", e));
    }
    return RespBodyVo.ok();
  }

`scroll`

展开参数注册

java

    put("scroll", (svc, id, a) -> svc.scroll(id, optBool(a, "down"), reqInt(a, "numPages"), a.getInteger("index")));

展开 scroll 实现

java

  public RespBodyVo scroll(Long browserId, boolean down, int numPages, Integer index) {
    BrowserInstance inst = INSTANCES.get(browserId);
    if (inst == null) {
      return RespBodyVo.fail("没有找到对应的浏览器实例：" + browserId);
    }
    for (int i = 0; i < numPages; i++) {
      if (index == null) {
        inst.page.keyboard().press(down ? "PageDown" : "PageUp");
      } else {
        Locator locator = resolveIndex(inst, index, "*");
        if (locator == null) {
          return RespBodyVo.fail("scroll 索引越界: " + index + indexHint(inst));
        }
        locator.evaluate("(el, down) => el.scrollBy(0, down ? window.innerHeight : -window.innerHeight)", down);
      }
    }
    return RespBodyVo.ok();
  }

`scroll_to_text`

展开参数注册

java

    put("scroll_to_text", (svc, id, a) -> svc.scrollToText(id, reqStr(a, "text")));

展开 scrollToText 实现

java

  public RespBodyVo scrollToText(Long browserId, String text) {
    BrowserInstance inst = INSTANCES.get(browserId);
    if (inst == null) {
      return notFound(browserId);
    }
    try {
      inst.page.locator("text=" + text).first().scrollIntoViewIfNeeded();
    } catch (PlaywrightException e) {
      return RespBodyVo.fail("scroll_to_text 失败：" + briefMessage(e.getMessage()));
    }
    return RespBodyVo.ok();
  }

`click_element_by_selector`

展开参数注册

java

    put("click_element_by_selector",
        (svc, id, a) -> svc.clickElementBySelector(id, reqStr(a, "selector"), optStr(a, "mode"),
            a.getInteger("timeoutMs"), optStr(a, "frame")));

展开 clickElementBySelector 实现

java

  public RespBodyVo clickElementBySelector(Long browserId, String selector, String mode, Integer timeoutMs,
      String frame) {
    BrowserInstance inst = INSTANCES.get(browserId);
    if (inst == null) {
      return notFound(browserId);
    }
    int tabCountBefore = pagesOf(inst).size();
    ActionTarget resolved;
    Frame probeFrame;
    try {
      probeFrame = frame == null || frame.isBlank() ? null : frameOf(inst, frame);
      resolved = resolveActionTarget(frameOf(inst, frame), selector, "click_element_by_selector");
    } catch (IllegalArgumentException e) {
      return RespBodyVo.fail("click_element_by_selector 失败：" + e.getMessage());
    } catch (PlaywrightException e) {
      // 匹配 0 个 / 全都不可见:当场说清,不等到可操作性超时(那时报的是 ACTION_TIMEOUT,方向全错)
      return RespBodyVo.fail(locateFailure("click_element_by_selector",
          "选择器 " + selector + frameSuffix(frame), e));
    }
    Locator target = resolved.locator;
    Kv before = stateProbe(inst, probeFrame);
    ActionOutcome outcome = new ActionOutcome();
    outcome.extra.set(resolved.describe());
    try {
      Locator locator = target;
      clickWithMode(locator, mode, outcome, () -> locator.click(clickOptions(timeoutMs)));
    } catch (PlaywrightException e) {
      return actionErrorOrEffect("click_element_by_selector", before, inst, probeFrame, e,
          locateFailure("click_element_by_selector", "选择器 " + selector + frameSuffix(frame), e)
              + resolved.failureSuffix() + blockerSuffix(target));
    }
    adoptNewTab(inst, tabCountBefore);
    return okWithReceipt(before, inst, "click_element_by_selector", outcome.extra, probeFrame);
  }

`input_text_by_selector`

展开参数注册

java

    put("input_text_by_selector", (svc, id, a) -> svc.inputTextBySelector(id, reqStr(a, "selector"),
        reqStr(a, "text"), optStr(a, "mode"), optStr(a, "frame")));

展开 inputTextBySelector 实现

java

  public RespBodyVo inputTextBySelector(Long browserId, String selector, String value, String mode, String frame) {
    BrowserInstance inst = INSTANCES.get(browserId);
    if (inst == null) {
      return notFound(browserId);
    }
    ActionOutcome outcome = new ActionOutcome();
    ActionTarget resolved;
    try {
      resolved = resolveActionTarget(frameOf(inst, frame), selector, "input_text_by_selector");
    } catch (IllegalArgumentException e) {
      return RespBodyVo.fail("input_text_by_selector 失败：" + e.getMessage());
    } catch (PlaywrightException e) {
      return RespBodyVo.fail(locateFailure("input_text_by_selector",
          "选择器 " + selector + frameSuffix(frame), e));
    }
    outcome.extra.set(resolved.describe());
    try {
      fillWithMode(resolved.locator, value, mode, outcome);
    } catch (PlaywrightException e) {
      return RespBodyVo.fail(locateFailure("input_text_by_selector",
          "选择器 " + selector + frameSuffix(frame), e) + resolved.failureSuffix());
    }
    return inputResult(outcome);
  }

`click_element_by_text`

展开参数注册

java

    put("click_element_by_text", (svc, id, a) -> svc.clickElementByText(id, reqStr(a, "text"), optStr(a, "mode")));

展开 clickElementByText 实现

java

  public RespBodyVo clickElementByText(Long browserId, String text, String mode) {
    BrowserInstance inst = INSTANCES.get(browserId);
    if (inst == null) {
      return notFound(browserId);
    }
    Kv before = stateProbe(inst);
    TextHit hit = resolveByText(inst, text);
    if (hit == null) {
      return RespBodyVo.fail("click_element_by_text 失败：没找到文本为「" + text + "」的元素");
    }
    try {
      Kv info = describe(hit.locator, hit.marker);
      info.set(hit.describeMatch());
      ActionOutcome outcome = new ActionOutcome();
      try {
        clickWithMode(hit.locator, mode, outcome, () -> hit.locator.click(clickOptions()));
      } catch (PlaywrightException e) {
        return actionErrorOrEffect("click_element_by_text", before, inst, null, e,
            locateFailure("click_element_by_text", "文本 " + text, e));
      }
      info.set(outcome.extra);
      return okWithHit(before, inst, "click_element_by_text", info);
    } finally {
      // 定位用的临时属性必须等动作做完再摘:定位器是惰性求值的,提前摘掉就选不中元素了
      hit.cleanup();
    }
  }

`click_element_by_role`

展开参数注册

java

    put("click_element_by_role",
        (svc, id, a) -> svc.clickElementByRole(id, reqStr(a, "role"), optStr(a, "name"), optStr(a, "mode")));

展开 clickElementByRole 实现

java

  public RespBodyVo clickElementByRole(Long browserId, String role, String name, String mode) {
    BrowserInstance inst = INSTANCES.get(browserId);
    if (inst == null) {
      return notFound(browserId);
    }
    Kv before = stateProbe(inst);
    Locator locator = inst.page
        .getByRole(AriaRole.valueOf(role.toUpperCase()), new Page.GetByRoleOptions().setName(name)).first();
    Kv info = describe(locator);
    ActionOutcome outcome = new ActionOutcome();
    try {
      clickWithMode(locator, mode, outcome, () -> locator.click(clickOptions()));
    } catch (PlaywrightException e) {
      return actionErrorOrEffect("click_element_by_role", before, inst, null, e,
          locateFailure("click_element_by_role", "角色 " + role + "[name=" + name + "]", e));
    }
    info.set(outcome.extra);
    return okWithHit(before, inst, "click_element_by_role", info);
  }

`input_text_by_label`

展开参数注册

java

    put("input_text_by_label",
        (svc, id, a) -> svc.inputTextByLabel(id, reqStr(a, "label"), reqStr(a, "text"), optStr(a, "mode")));

展开 inputTextByLabel 实现

java

  public RespBodyVo inputTextByLabel(Long browserId, String label, String value, String mode) {
    BrowserInstance inst = INSTANCES.get(browserId);
    if (inst == null) {
      return notFound(browserId);
    }
    ActionOutcome outcome = new ActionOutcome();
    try {
      fillWithMode(inst.page.getByLabel(label).first(), value, mode, outcome);
    } catch (PlaywrightException e) {
      return RespBodyVo.fail(locateFailure("input_text_by_label", "标签 " + label, e));
    }
    return inputResult(outcome);
  }

`clear_text`

展开参数注册

java

    put("clear_text", (svc, id, a) -> svc.clearText(id, a.getInteger("index"), optStr(a, "selector")));

展开 clearText 实现

java

  public RespBodyVo clearText(Long browserId, Integer index, String selector) {
    BrowserInstance inst = INSTANCES.get(browserId);
    if (inst == null) {
      return notFound(browserId);
    }
    Locator locator;
    String target;
    if (index != null) {
      locator = locatorOf(inst, index);
      target = "index=" + index;
      if (locator == null) {
        return RespBodyVo.fail("clear_text 索引越界: " + index + indexHint(inst));
      }
    } else if (selector != null && !selector.isEmpty()) {
      locator = inst.page.locator(selector).first();
      target = "选择器 " + selector;
    } else {
      return RespBodyVo.fail("clear_text 需要 index 或 selector");
    }
    Kv data = Kv.by("value", null);
    try {
      fillEditable(locator, "");
    } catch (PlaywrightException e) {
      return RespBodyVo.fail(locateFailure("clear_text", target, e));
    }
    try {
      data.set("value", locator.inputValue());
    } catch (PlaywrightException e) {
      log.debug("clear_text 读回值失败:{}", briefMessage(e.getMessage()));
    }
    return RespBodyVo.ok(data);
  }

`hover_and_click`

展开参数注册

java

    put("hover_and_click", (svc, id, a) -> svc.hoverAndClick(id, a.getInteger("index"), optStr(a, "selector"),
        a.getInteger("hoverDelayMs"), optStr(a, "mode")));

展开 hoverAndClick 实现

java

  public RespBodyVo hoverAndClick(Long browserId, Integer index, String selector, Integer hoverDelayMs, String mode) {
    BrowserInstance inst = INSTANCES.get(browserId);
    if (inst == null) {
      return notFound(browserId);
    }
    Locator locator;
    String target;
    if (index != null) {
      locator = locatorOf(inst, index);
      target = "index=" + index;
      if (locator == null) {
        return RespBodyVo.fail("hover_and_click 索引越界: " + index + indexHint(inst));
      }
    } else if (selector != null && !selector.isEmpty()) {
      locator = inst.page.locator(selector).first();
      target = "选择器 " + selector;
    } else {
      return RespBodyVo.fail("hover_and_click 需要 index 或 selector");
    }
    Kv before = stateProbe(inst);
    Kv info = describe(locator);
    ActionOutcome outcome = new ActionOutcome();
    try {
      locator.hover(new Locator.HoverOptions().setTimeout(actionTimeoutMs()));
      sleepQuietly(hoverDelayMs == null ? 300 : hoverDelayMs);
      clickWithMode(locator, mode, outcome, () -> locator.click(clickOptions()));
    } catch (PlaywrightException e) {
      return actionErrorOrEffect("hover_and_click", before, inst, null, e,
          locateFailure("hover_and_click", target, e));
    }
    info.set(outcome.extra);
    return okWithHit(before, inst, "hover_and_click", info);
  }

`mouse_move`

展开参数注册

java

    put("mouse_move", (svc, id, a) -> svc.mouseMove(id, reqDouble(a, "x"), reqDouble(a, "y")));

展开 mouseMove 实现

java

  public RespBodyVo mouseMove(Long browserId, double x, double y) {
    BrowserInstance inst = INSTANCES.get(browserId);
    if (inst == null) {
      return notFound(browserId);
    }
    inst.page.mouse().move(x, y);
    return RespBodyVo.ok();
  }

`mouse_down`

展开参数注册

java

    put("mouse_down", (svc, id, a) -> svc.mouseDown(id, optStr(a, "button")));

展开 mouseDown 实现

java

  public RespBodyVo mouseDown(Long browserId, String button) {
    BrowserInstance inst = INSTANCES.get(browserId);
    if (inst == null) {
      return notFound(browserId);
    }
    inst.page.mouse().down(new Mouse.DownOptions().setButton(mouseButton(button)));
    return RespBodyVo.ok();
  }

`mouse_up`

展开参数注册

java

    put("mouse_up", (svc, id, a) -> svc.mouseUp(id, optStr(a, "button")));

展开 mouseUp 实现

java

  public RespBodyVo mouseUp(Long browserId, String button) {
    BrowserInstance inst = INSTANCES.get(browserId);
    if (inst == null) {
      return notFound(browserId);
    }
    inst.page.mouse().up(new Mouse.UpOptions().setButton(mouseButton(button)));
    return RespBodyVo.ok();
  }

`mouse_wheel`

展开参数注册

java

    put("mouse_wheel", (svc, id, a) -> svc.mouseWheel(id, reqDouble(a, "deltaY")));

展开 mouseWheel 实现

java

  public RespBodyVo mouseWheel(Long browserId, double deltaY) {
    BrowserInstance inst = INSTANCES.get(browserId);
    if (inst == null) {
      return notFound(browserId);
    }
    inst.page.mouse().wheel(0, deltaY);
    return RespBodyVo.ok();
  }

`mouse_click`

展开参数注册

java

    put("mouse_click",
        (svc, id, a) -> svc.mouseClick(id, reqDouble(a, "x"), reqDouble(a, "y"), optStr(a, "button"),
            a.getInteger("clickCount")));

展开 mouseClick 实现

java

  private static boolean mouseClick(Locator locator) {
    try {
      Locator target = locator.first();
      try {
        target.scrollIntoViewIfNeeded(new Locator.ScrollIntoViewIfNeededOptions().setTimeout(2_000));
      } catch (PlaywrightException ignored) {
        // 滚不动也继续:盒子可能本来就在视口里
      }
      BoundingBox box = target.boundingBox();
      if (box == null || box.width <= 0 || box.height <= 0) {
        return false;
      }
      target.page().mouse().click(box.x + box.width / 2, box.y + box.height / 2);
      return true;
    } catch (PlaywrightException e) {
      // The click may already have been delivered. Do not fall through to a JS click.
      if (ActionError.isSpuriousDispatch(e.getMessage())) {
        throw e;
      }
      return false;
    }
  }

`mouse_click_by_selector`

展开参数注册

java

    put("mouse_click_by_selector",
        (svc, id, a) -> svc.mouseClickBySelector(id, reqStr(a, "selector"), optStr(a, "button"),
            a.getInteger("clickCount"), a.getInteger("timeoutMs")));

展开 mouseClickBySelector 实现

java

  public RespBodyVo mouseClickBySelector(Long browserId, String selector, String button, Integer clickCount,
      Integer timeoutMs) {
    BrowserInstance inst = INSTANCES.get(browserId);
    if (inst == null) {
      return notFound(browserId);
    }
    Locator locator = inst.page.locator(selector).first();
    Kv before = stateProbe(inst);
    ActionOutcome outcome = new ActionOutcome();
    recordBlocker(locator, outcome);
    if (!mouseClickAt(locator, button, clickCount)) {
      return RespBodyVo.fail("mouse_click_by_selector 失败：拿不到元素盒子（选择器 " + selector
          + " 没有命中可见元素）" + blockerSuffix(locator));
    }
    outcome.mode = "mouse";
    outcome.record();
    return okWithReceipt(before, inst, "mouse_click_by_selector", outcome.extra);
  }

点击与定位共用的辅助实现

resolveIndex 从快照中解析元素；clickWithMode 负责执行模式和降级。这些函数解释了上层命令为什么不能简单等同于一次 locator.click。

展开 resolveIndex

java

  private Locator resolveIndex(BrowserInstance inst, int index, String selector) {
    if (inst.snapshotInvalidated) return null;
    DOMState state = inst.domState;
    if (state != null) {
      DOMElementNode node = state.getSelectorMap().get(index);
      if (node == null) {
        return null;
      }
      // 带 frame 的快照里,索引可能属于某个 iframe:xpath 必须**相对该 frame 的 document** 求值。
      // 索引里已经带了 frame 信息,所以调用方不必传 frame 参数(见 get_browser_state 的 includeFrames)
      Frame frame = frameForIndex(inst, state, index);
      if (frame == null) {
        return inst.page.locator("xpath=/" + node.getXpath()).first();
      }
      return frame.locator("xpath=/" + node.getXpath()).first();
    }
    if (selector == null) {
      return null;
    }
    int size = inst.page.locator(selector).count();
    if (index < 0 || index >= size) {
      return null;
    }
    return inst.page.locator(selector).nth(index);
  }

展开 clickWithMode

java

  private static void clickWithMode(Locator locator, String mode, ActionOutcome outcome, Runnable nativeAction) {
    String resolved = normalizeMode(mode);
    boolean covered = recordBlocker(locator, outcome);
    if ("js".equals(resolved)) {
      jsClick(locator);
      outcome.mode = "js";
      outcome.record();
      return;
    }
    if ("mouse".equals(resolved)) {
      if (!mouseClick(locator)) {
        throw new PlaywrightException("mouse 模式失败:拿不到元素盒子(元素可能不可见或已从页面移除)");
      }
      outcome.mode = "mouse";
      outcome.record();
      return;
    }
    try {
      nativeAction.run();
      outcome.mode = "native";
    } catch (PlaywrightException nativeFailure) {
      if (!"auto".equals(resolved) || ActionError.isSpuriousDispatch(nativeFailure.getMessage())) {
        throw nativeFailure;
      }
      outcome.fallbackReason = briefMessage(nativeFailure.getMessage());
      if (covered || recordBlocker(locator, outcome)) {
        // 不穿透弹窗点击背景提交按钮；页面可能在原生操作等待期间新增遮挡层。
        outcome.extra.set("mouseSkipped", "target-covered");
        throw new PlaywrightException("ELEMENT_OBSCURED 目标被遮挡，auto 模式不穿透点击；请重新读取当前弹窗并定位控件");
      } else if (mouseFallbackEnabled() && mouseClick(locator)) {
        outcome.mode = "mouse";
      } else if (jsFallbackEnabled()) {
        jsClick(locator);
        outcome.mode = "js";
      } else {
        throw nativeFailure;
      }
    }
    outcome.record();
  }

源码教程：点击、输入、键盘与鼠标 ​

1. 索引、CSS 与语义定位 ​

2. 点击执行不是无条件重试 ​

3. 输入与框架状态 ​

4. 下拉框、滚动与拖拽 ​

5. 键盘与鼠标是更低层的工具 ​

6. 回归测试设计 ​

注册参数与 Java 入口 ​

当前源码：命令注册与执行 ​

click_element_by_index ​

double_click_element_by_index ​

hover_element_by_index ​

focus_element_by_index ​

check_element_by_index ​

uncheck_element_by_index ​

input_text ​

type_text ​

drag_element_by_index ​

send_keys（别名 press_key） ​

key_down ​

key_up ​

get_dropdown_options ​

select_dropdown_option ​

scroll ​

scroll_to_text ​

click_element_by_selector ​

input_text_by_selector ​

click_element_by_text ​

click_element_by_role ​

input_text_by_label ​

clear_text ​

hover_and_click ​

mouse_move ​

mouse_down ​

mouse_up ​

mouse_wheel ​

mouse_click ​

mouse_click_by_selector ​

点击与定位共用的辅助实现 ​

源码教程：点击、输入、键盘与鼠标

1. 索引、CSS 与语义定位

2. 点击执行不是无条件重试

3. 输入与框架状态

4. 下拉框、滚动与拖拽

5. 键盘与鼠标是更低层的工具

6. 回归测试设计

注册参数与 Java 入口

当前源码：命令注册与执行

`click_element_by_index`

`double_click_element_by_index`

`hover_element_by_index`

`focus_element_by_index`

`check_element_by_index`

`uncheck_element_by_index`

`input_text`

`type_text`

`drag_element_by_index`

`send_keys`（别名 `press_key`）

`key_down`

`key_up`

`get_dropdown_options`

`select_dropdown_option`

`scroll`

`scroll_to_text`

`click_element_by_selector`

`input_text_by_selector`

`click_element_by_text`

`click_element_by_role`

`input_text_by_label`

`clear_text`

`hover_and_click`

`mouse_move`

`mouse_down`

`mouse_up`

`mouse_wheel`

`mouse_click`

`mouse_click_by_selector`

点击与定位共用的辅助实现