Repository navigation
feat(capability-registry): onboard the infra products, and the mechanisms they needed - #491
Open
sarve-shreyas wants to merge 19 commits into
Open
sarve-shreyas wants to merge 19 commits into
sarve-shreyas wants to merge 19 commits into
Conversation
…d App Automate
A product's capabilities all resolve against one `base_url`. App Automate does not
fit that: its account-level API (devices, app builds, plan, session records) answers
on api-cloud and the Appium hub that drives the device answers on hub-cloud, and
neither serves the other's paths.
Splitting them into two products would split the `app_session` entity with them —
the session the hub creates is the session the REST API reports on, and a caller
holding the id would have to know it had changed products to use it.
So a product may now declare `hosts: {name: url}` and a capability may name one with
`host`. Unset means the product's own `base_url`, so every existing index is
unaffected. Override with CAPABILITY_REGISTRY_BASE_URL_<PRODUCT>_HOST_<NAME>; the
`_HOST_` segment is there because the plain `_<SUFFIX>` form already means an
environment, and either reading of a bare `_HUB` would send calls to the wrong
origin silently. A host the index does not declare is refused by name rather than
falling back, because the fallback returns a 404 that reads like the caller's
mistake. `host` stays off the describe surface, as `method` and `path` already do.
This is NOT `base_urls`, which is a list of candidates probed to find one
region-sharded host. These are several hosts serving the product at once.
The App Automate index grows from 9 capabilities to 45:
api-cloud device catalogue, private devices, plan and parallel usage, app
build upload by link, build listings, session records, session
status, mid-session network profiles
hub-cloud screenshot, back, orientation, contexts, keyboard state and
dismissal, geolocation, file push and pull, key codes and iOS
hardware buttons, device properties, and the app lifecycle —
launch, close, uninstall, app state, biometric prompts, camera
injection and session annotations
with three new entities: device, app_build and plan.
Every endpoint was probed on a real device before its contract was written, then
re-run through the registry's own resolve/bind/auth/egress path on Android 12 and
iOS 27. iOS was where the contracts were wrong, every time because behaviour
verified on Android had been written up as if it were platform-neutral:
- launch_app, terminate_app, uninstall_app, get_app_state and is_app_installed
declared only `appId`. iOS requires `bundleId` and answered 400 to all five.
- get_geo_location is Android-only; iOS reads its position through
get_simulated_location, added here.
- get_current_app is Android-only; get_active_app_info is the iOS form and
carries the bundleId the lifecycle capabilities need.
- hide_keyboard reports success on iOS and leaves the keyboard up. A newline
through send_keys closes it; tapping an empty area does not.
- press_back works through a standard navigation bar and does nothing on a
screen that draws its own header.
- set_orientation returns the orientation on Android and null on iOS.
- send_keys REPLACES a field's contents; it was documented as appending.
read_device_property exposes `getprop` and nothing else: its `pattern` covers the
whole executor line, so a command that lists apps or system state cannot be
expressed through it and is refused before a request is built. A public phone is
shared between customers, and BrowserStack applies a second allow-list of its own
on top.
egress: a failed body read is no longer reported as an empty success. `fetch`
resolves when the headers arrive, so the abort timer can fire while the body is
still streaming; that error was swallowed and the call returned its status with a
null body. A slow session create therefore came back as a 200 that named no
session, while the device really had been allocated — one was left running until
its idle timeout. It now reports a failed call, and create_session's guidance says
that a start which fails or names no session may still have started one.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…emote server Some capabilities only mean anything when the person and the process share a computer. One that reads a file the caller names is the clear case: on a remote server that path resolves against the OPERATOR's disk, so honouring it would read one user's files on another's behalf. The capability does not become unreliable there, it becomes wrong. `local_only` is a claim about where a capability can run, as against `disabled`, which withholds it everywhere. The same artifact then serves both deployments: a local server publishes it and a remote one does not. It joins the existing filter, so it inherits the whole `capability_disabled` path: the capability is absent from searchCapability, describeCapability, describeEntity and listProducts, while a caller already holding the handle is told the absence is deliberate rather than hunting for a synonym. It fails closed on any truthy value, for the same reason `disabled` does. REMOTE_MCP is read at call time rather than captured at module load, because the remote server rebuilds its config per session and a captured value would outlive the deployment it described. No index sets the flag yet; this is the mechanism. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… machine Until now a capability was always a JSON request, so an endpoint taking the bytes of a file could only be published in whatever reference form it also accepted — for app builds, a public download link. A build sitting on the user's laptop had nowhere to go. A capability may now declare `encoding: "multipart"` and a parameter of type `file`. The bound value is streamed with `openAsBlob` rather than buffered: the cap is 4 GB, and reading that into memory to post it is how a server falls over. The JSON content-type is dropped for those requests, because fetch writes its own with the boundary and leaving the old one on sends a body the product cannot parse under a header claiming it can. THE PATH IS CHECKED IN `bind`, which register.ts runs dry before the write-consent gate. A path that does not exist, or a base directory nobody configured, is therefore refused before the user is asked to approve an upload that was never going to happen, and before anything is read from disk. The rules are `validateUploadPath`'s, already applied to every other upload this server does: confinement to MCP_UPLOAD_BASE_DIR, an extension allow-list, a size cap, and a hidden-directory block that stops ~/.ssh even after symlink resolution. Both refusals name what to do — set the variable and restart, or move the build into the directory, which the message gives as a resolved absolute path. upload_app_build_from_file is a SEPARATE capability rather than a file parameter on upload_app_build, because `local_only` withholds a whole capability: a shared one would have meant either losing the link form on a remote server or publishing a path that names the operator's disk. The link form stays available everywhere. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…t as withheld Search, describe, describeEntity and listProducts already behave as though a withheld capability does not exist — it is dropped when the registry loads, from the product's capabilities and from the entity records that named it. Invocation was the one place that contradicted them, answering `capability_disabled` and explaining that the capability was real but suppressed here. That explanation describes the artifact rather than this surface, and every action it invites fails: retrying, searching for a synonym, or asking the user to enable something no setting on this deployment turns on. It also discloses the existence of a capability that every other surface is at pains not to mention. So both lookups now raise their ordinary not-found error. The handle reads as absent wherever a caller meets it. The distinction is kept where it is actually useful. InvocationError carries a `withheld` flag, and the telemetry reason still resolves to `capability_disabled`, so how often a withheld handle is reached for stays countable — which is how we learn whether withholding it was right — without the message admitting to it. A genuinely absent name leaves the flag unset, so the two remain distinguishable in the data and indistinguishable to the caller. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…nce where it cannot run Withholding it entirely answered the wrong question. An agent asked to upload a build from the user's machine, talking to a remote server, found nothing — and concluded no such route exists, when one does: the user uploads the file themselves and passes back the url. The capability was the natural match for that query and the only place the answer could have come from. So on a remote server a `local_only` capability is now published for reference. It keeps its name, its entity and its intent — the intent is what the search matched on — and loses everything a caller would need to invoke it. The parameters and response shapes are stripped at load, because a model handed a body schema fills it in, and its guidance is replaced by `unavailable_note`: what to do instead, written as an instruction rather than an apology. invokeCapability refuses it with that note attached, under a new reason, `capability_not_invocable`. This is the ONE place describe and invoke diverge, and the code says elsewhere that such divergence is its own bug class — so it is commented as the deliberate exception it is. A capability published to be read and not called is exactly the case the rule did not anticipate. The three states are now distinct, and the tests pin them apart: disabled absent everywhere; answers as unknown local_only, local fully callable local_only, remote found and described, not callable `disabled` continues to win over `local_only`, so a capability carrying both is absent rather than merely uncallable. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ser and device work
App Automate drives an installed app; Automate drives a browser. Same shape, two
hosts again — api.browserstack.com for the catalogue, plan and session records, and
hub.browserstack.com for WebDriver — so the named-host mechanism carries it with no
further change. It is a separate product rather than more capabilities on appautomate:
the entity is a browser session, not an app session, and "session", "click" and
"screenshot" are shared vocabulary that would make routing between them a coin toss.
32 capabilities: the browser catalogue, plan and session records; sessions on desktop
and on real phones; navigation with history; finding one element or many; click, type
and clear; reading text, properties, attributes and visibility; page source; cookies;
JavaScript; screenshots; window geometry; tabs and frames.
Everything was probed on a real browser before its contract was written, and the
probing is most of what the guidance says:
- get_element_text is EMPTY for an input. What a user typed is a property, so
get_element_property on "value" is the one that reads a field.
- find_elements answers an empty list and a 200 where find_element answers 404,
which is how to test for absence without handling an error.
- clear_element genuinely empties a field here, unlike on App Automate where it
reports success and changes nothing. Both indexes now say so.
- switch_to_frame with null is how to get back to the top-level page, and not doing
it is the usual reason an element plainly on screen cannot be found.
- On mobile the browser is named after the platform: Chrome on Android is "android",
Safari is "iphone" or "ipad". Twelve names, read from the API rather than guessed.
- set_window_rect is accepted on Android and refused with a 405 on iOS.
Two registry changes the web hub forced, both kept narrow:
send_empty_body, per capability, for a parameterless POST. Chrome answers 400
"invalid argument" to one with the body omitted and 200 to the same call carrying
`{}`; Appium tolerates the omission, which is why the identical click_element needs
this on one product and not the other. Opt-in rather than a rule for every bodyless
POST, because 27 of those ship across the indexes and this cannot verify what `{}`
does to them.
`any` now passes a value through coerceType untouched. It was falling to the string
branch, so a frame id of null arrived as "null" and was refused as an invalid
argument — a failure that reads like the caller chose the wrong frame.
check-contract.py gains an envelope check. A hand-authored index with
schema_version "0.1" was refused by the loader, and because loading is a merge across
files it took tm, tra and loadtesting down with it. One assertion prevents a total
outage from a typo in a new file.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Mapping the 144 eval cases onto the two indexes leaves 21 capabilities no case reaches: on App Automate the device features — GPS, network profile, biometric, camera injection, file transfer, uninstall — and on Automate the DOM readers and window geometry. Those are withheld for now, so the published surface is the one under test rather than everything that was authored. 33 of 46 stay live on App Automate and 24 of 32 on Automate. This is a deliberate narrowing, not a quality judgement: all 78 were probed against real devices and browsers, and each withheld record keeps its contract and guidance intact, so re-enabling one is a single flag. It does cost something, and the cost is worth stating. Reading a screen is now screenshot-only on both products: get_page_source is withheld on App Automate, where it was the way an agent saw the whole UI tree, and get_page_source, get_element_text, get_element_attribute and is_element_displayed are withheld on Automate, where they were how an agent found a selector and checked a result without an image. The eval suite asserts visually, so it does not miss them; an agent doing the same work will pay for every look. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A minor rather than a patch: this adds capabilities and two products, and changes no existing contract. 2.2.2 is already released, so a beta built on it would sort below what is on `latest` and anyone tracking `@beta` would be handed something older than the stable release. The Beta Release workflow derives `<base>-beta.<n>` from this field and increments the counter itself, so this is the only version that needs setting by hand. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The guidance said a trailing \n submits, which is what the specification implies and what desktop Chrome and Chrome for Android both do. On Safari for iOS it does not: the text lands in the field and the page stays where it is, whether the newline is sent with the text or on its own. Found by eval case E105, which is the first thing in that suite to catch a published claim rather than confirm one. Clicking the form's submit button works, and so does arguments[0].form.submit() through execute_script with the field as the argument; both are named in the guidance because both were measured. App Automate's own send_keys keeps its line about a newline closing the keyboard on iOS. That was measured against a native text field, which is a different surface from a web form that happens to share the platform's name. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…orrections the eval run found find_elements was on Automate and missing from App Automate, which left no way to ask whether an element is absent: find_element answers that question with a 404, so a caller had to catch an error to learn a fact. Verified on a device before authoring — many matches, one match, and none, where none is an empty list and a 200. The web index's wording about an empty list not proving absence does NOT carry over. Through this surface it can be trusted: a strategy App Automate does not accept is refused before the request is sent, and an iOS strategy aimed at an Android screen comes back as a 400 "invalid selector" rather than as nothing found. The raw endpoint does answer an unsupported strategy with an empty list; the declared enum means a caller never reaches that. Three corrections, each from an eval case rather than a probe: resolution takes one of a fixed set per operating system, not arbitrary dimensions. 1920x1080 is accepted on Windows 11 and 2560x1440 is refused outright with "not compatible with the chosen OS version", and no session starts. The viewport that results is around 930x860, well short of the screen, so the request is not the usable area. (E085) launch_app answers 400 when an app genuinely cannot be brought forward, and get_app_state reads 4 once it really is in front. Sampled too early it can report the home screen mid-switch — which is what led me to accuse the capability of reporting false success before checking. It does not. (E067) get_app_state's backgrounded value is 3 or 2 rather than one number: the system suspends an app a moment after it stops being visible, so anything below 4 means not on screen. (E060) Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ct again Withholding these two did not cost coverage, it cost RECOVERY. Running the eval suite without them, four cases failed and stayed failing — E047 three times, E058 four — and in every instance the capability under test was working. The failures were wrong selectors and assumed screen bounds, and neither could be diagnosed, because the only move available was to guess again and watch the answer come back empty. With them published, both passed on the first attempt. The page source showed the product ids are add-to-cart-11, -12 and -16 in this build, where I had been guessing -1, -13 and -17 from a build probed earlier; the scroll had worked every time. get_window_rect returned 1080x2400, which is what the gestures had been assuming all along. So the argument is not that reading a screen is expensive without them. It is that an agent cannot recover from a wrong selector or an unknown viewport at all: page source is how a locator is discovered and how a failed one is debugged, and the window rectangle is what every coordinate gesture is measured against. 36 of 47 now publish on App Automate and 26 of 32 on Automate. The 17 still withheld are genuinely peripheral — GPS, biometric, camera injection, file transfer, cookies, element text, window resizing — and none of them stand between a caller and understanding what went wrong. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…l, and strip the examples back appautomate and automate are BrowserStack's own product names, and they say nothing about what the capabilities do. mobiledevicecontrol drives an installed app on a real phone; devicecontrol drives a browser on a desktop or a phone. The product key comes from the filename, so the `product` enum on describeCapability and invokeCapability follows without being edited. The marketing names stay in the summaries and the routing terms, because they are what a person says and what the plan, the minutes and the dashboard are still called. Dropping them would make the products unfindable by the words customers use. Each summary now names the other product, so a router can tell an app from a page. The hand-written App Automate tools under src/tools/appautomate* are untouched. They are a separate surface that happens to share the old name. THE EXAMPLES CARRIED REAL SESSION ARTEFACTS. An element reference and a window handle from a live run had been copied in as the canonical examples for those parameters — meaningless to a reader, and one account's internals shipped to npm. Both are now zeroed placeholders that still show the shape. The rest came from BrowserStack's own demo shop: bstackdemo.com as the example URL, .shelf-item__buy-btn and nav-cart as example selectors, a product name as the text to type. An agent reading those learns one test site rather than the shape of a selector, so they are now example.com, #add-to-cart and cart-button. Device, OS and browser names stay real — those have to match the catalogue exactly, so a plausible-looking placeholder there would be worse than none. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A cold agent run against the published surface found every one of these. Each is a line that sends a caller to something it cannot reach, which costs a round trip and then a guess. - the product summaries advertised cookies and element text on devicecontrol, and file transfer, GPS, network profiles, uninstall, biometrics and the camera on mobiledevicecontrol. All of those are withheld; a summary is read before anything else, so it was the first thing to mislead. - four capabilities referred the caller to `get_element_text`, which is withheld. The reads that answer the same question — `get_title`, `get_element_property` — say it instead, and the one line explaining why an input has no text now explains it without naming a capability that is not there. - `create_session` guidance said idle_timeout is "300 at most" against a declared maximum of 900, so the contract and the prose disagreed about the same number. - `create_session` and `execute_script` told the caller to mark a result with the raw `browserstack_executor` setSessionStatus action. `set_session_status` is the capability for that; the executor form is now named only for the case it is still needed, from inside a running session. - `list_apps` claimed uninstall_app needs the package id it returns. uninstall_app is withheld too. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The handler built its answer by spreading the capability and removing the route, so every field since added to `Capability` has been published to callers by default. `host: "api"` was in a shipped describe response — a caller cannot choose which of a product's origins serves a route, and showing it invites reasoning about plumbing it has no say in. Counted across the indexes: `disabled` on 64 capabilities, `host` on 15, `send_empty_body` on 6, `encoding` on 1, `local_only` on 1. `invocable` stays, and is the one of these that is not plumbing. Presence in the index means published, which is not the same as callable: a reference-only entry is published precisely so the search that should find it still does, and stripping the flag would leave that fact in a prose guidance line alone. The reference-only tests assert against the registry, so none of this was visible to them. The new tests go through the registered handler, which is what a caller sees; reverting the destructure fails two of them on the exact `host: "api"` that was reported. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Two separate causes, one symptom: listProducts put a clash in front of the user over a word nothing in the request was about, and the question it generated — "devicecontrol or mobiledevicecontrol or tm or tra?" — was unanswerable. OUR OWN NAME WAS ROUTING VOCABULARY. The camelCase split in `terms` runs before stopword filtering, so "BrowserStack" arrived as `browser` + `stack`, and `browser` is a devicecontrol entity and a tm configuration alias. "Which Android apps have I uploaded to BrowserStack?" therefore clashed over `browser` across four products. It reached past the requests that name us, too: the tm and tra summaries say "BrowserStack", so both became permanent claimants of `browser`, and "what browsers can I use" — which names us nowhere — clashed four ways over a noun only devicecontrol models. The brand is now removed after the split and the fold, so one pattern covers "BrowserStack", "browserstack", "browser stack" and `bstack`. A SUMMARY SENTENCE THAT DISOWNS A WORD WAS READ AS CLAIMING IT. Each of these summaries carries a sentence saying what is NOT in it and where to go instead, which is the most useful sentence in the paragraph: "Browsers are not here — those are device control" made mobiledevicecontrol a claimant of `browser`, and "Mobile APPS are not here" made devicecontrol a claimant of `app`. Both products were then offered to the user for a word their own summary disowns. The claimant pass now skips disclaiming sentences; since prose cannot introduce a term, dropping one can only narrow a clash, never misdirect it. Measured over the 198 search-eval and 77 routing-eval queries plus nine paraphrases that name us: 28 queries that clashed now settle, 14 clashes narrow to fewer products, and no query's settled product changed. One goes the other way — "before I build the run from this folder, show me which cases the selection resolves to" was settling on tra, because `build` is a tra entity and was the only single-owner term in the sentence; it is now an honest clash whose candidates include tm, which is the product the fixture expects. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…s response
`list_browsers` was published and unusable. `/automate/browsers.json` returns every
operating system, browser and version combination the account can run — 5,251 rows,
604 KB — and accepts no parameters at all: `?os=Windows`, `?browser=chrome` and
`?zzz=1` all come back byte-identical to the bare call, which is measured, not
assumed. Its own guidance told the caller to "narrow it yourself", which is the one
thing it had no means to do. 604 KB of JSON is most of a context window spent to
learn that Chrome runs on Windows 11.
So a capability may now declare `narrowable`, and invokeCapability grows a `select`
argument — `where`, `distinct`, `fields`, `limit` — applied to the response after
the call. Three rules keep it from becoming a second API:
1. Opt-in. `select` on a capability that declares nothing is refused, so the
untouched-response promise holds everywhere it held before.
2. Only the declared fields, and an undeclared one is refused BY NAME with the set
listed — a caller that guesses `platform` learns the vocabulary instead of
silently receiving everything.
3. The answer reports it: `narrowed: {from, to, by}`. A smaller body that does not
announce itself cannot be told from a product that returned less than it has.
It is not a query language and should not grow into one. Equality, membership,
distinct tuples, projection, a cap — anything beyond that is a reason to look for an
endpoint that filters, not to add an operator. Narrowing is applied to a 2xx only: a
4xx body is the diagnosis, and a selection matching none of its keys would empty it.
Against production, through the registered tool:
bare call 603,879 bytes
distinct browser 259 bytes 5251 -> 12
distinct os, os_version 1,473 bytes 5251 -> 41
Chrome versions on Windows 11 136 bytes 5251 -> 5
does Safari 18.0 run on Sequoia? 2 bytes 5251 -> 0 (it is 18.4)
The guidance was also wrong where it was not impossible: it named six browser values
and the API returns twelve, omitting ie, samsung, android, iphone, ipad and
chromeForTesting. It now says what the endpoint does not do, how to ask what values
exist, that `os` is capitalised in two of its four spellings, and that rows arrive
oldest-first so a `limit` is the wrong way to find the newest build — it hands back
Chrome 37 for Windows 11.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Important Review skippedAuto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the ⚙️ Run configuration
You can disable this status message by setting the Use the checkbox below for a quick retry:
Comment |
`Provenance.build_id` is documented as `<commit>_<UTC timestamp>`, which is what the generated indexes carry — loadtesting `43eba70_2026-09-02T14:26:51Z`, tm `cb63a4a5_2026-09-16T18:02:57Z`. These two were hand-authored and were stamped `automate-authored` and `appautomate-demo`, which are not that shape, still carry the pre-rename product names, and in the second case say "demo" about an index that ships on npm and answers against production. The stamp names the commit whose CONTENT it describes, not the commit that writes the stamp — the same relationship a generator has to its source, and it means `git show <commit>:capability/<product>.capability-index.json` reproduces the file the id refers to. Nothing in the repo writes build_id; every reference reads it. `version` goes 0.1 -> 1.0 with it. The field is the dotted content-generation counter and is documented as absent in the pre-release shape, so a version that is present and below 1 says two contradictory things about an index that is shipping. loadtesting's first release is 1.0. check-contract.py only asserted a non-empty string, which is why neither was caught. Content is byte-identical either side of this; only the envelope moved. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
# Conflicts: # package-lock.json # package.json # server.json
Measured on a real iPhone, three routes, text in the field before each: hide_keyboard 400 keyboard up field kept mobile: hideKeyboard keys ["Done"] 400 keyboard up field kept perform_actions tap above the keyboard 200 keyboard up field kept Both 400s are the same error — "invalid element state", "Did not know how to dismiss the keyboard" — and the guidance described that as the opposite: it said hide_keyboard "frequently leaves the keyboard open while still answering successfully, so always confirm with is_keyboard_shown". A caller following that writes a success path with a check after it, and takes an error branch instead. A second line recommended, as the iOS fallback, "tapping an empty area or the keyboard's own Done key" — both of which fail, and both of which the NEXT line in the same block already said fail. Two guidance lines disagreeing about one capability; the wrong one is gone. What was missing is the cost of the route that does work. keys ["return"], and a newline through send_keys, close the keyboard BY PRESSING RETURN: the field is submitted and reset to its placeholder and the value moves wherever the app sends it. Both were published as ways to dismiss the keyboard with no note that they change the app. The keys argument also names a key on the APP'S own keyboard, which is why ["Done"] fails on a keyboard that only offers Return — so the working route is not portable between apps either. So: on iOS there may be no way to put the keyboard away without submitting, and the guidance now says that outright rather than listing fallbacks that do not work. send_keys carried a related error. It led with "REPLACES whatever the field already holds rather than adding to it — measured on Android", and a later line hedged that other platforms may append. iOS appends: typing twice left routeA, then routeArouteB, then routeArouteBrouteC in one field. The headline is now platform-split and the hedge is replaced by the measurement. Since nothing on this surface can empty a field, an iOS field typed into twice cannot be corrected. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Onboards BrowserStack's two infrastructure products to the capability registry —
devicecontrol(web browsers) andmobiledevicecontrol(real Android and iOS devices) — and adds the registry mechanisms they needed, none of which existed because every product onboarded so far has been API-shaped.Jira: AIMCP-242 (POC), AIMCP-243 (mobile), AIMCP-244 (web), under epic AIMCP-196.
Why infra needed new mechanisms
TM, TRA and Load Testing are one host, one request, one JSON answer. Infra is not: it spans two origins, holds live sessions that cost money while they are open, uploads files from the caller's disk, and has endpoints that cannot safely run from a remote server at all.
hosts: {name: url}, a capability names oneapi-cloud(account) andhub-cloud(the Appium hub); neither serves the other's pathslocal_onlyunavailable_noteas its guidanceencoding: "multipart"+ atype: "file"parameter, validated before the write-consent gateMCP_UPLOAD_BASE_DIRsend_empty_body{}; Appium tolerates omission, so this could not be a blanket rule — 27 bodyless write-verb capabilities ship across the indexesnarrowableon the contract,selecton invokelist_browsersreturns 5,251 rows and accepts no filteringReference-only publishing is the one place
describeCapabilityandinvokeCapabilitydeliberately diverge, and it is commented as such.Defects this found, which were not infra-specific
egressswallowed an aborted body read.await response.text().catch(() => "")turned an abort into an empty body, so a slowcreate_sessionreturned 200 with a null body while the device was really allocated — one session ran to its idle timeout before anyone noticed. It now re-throws, giving status 0. This affected every product.A withheld capability told a direct
invokeCapabilitythat it was disabled. "Disabled" invites a retry, a synonym search, or a question to the user, none of which can succeed. It now reads as absent; the distinction survives in telemetry only.describeCapabilitypublished wire fields. The handler spread the capability record, so every field ever added toCapabilityreached callers by default:disabledon 64 capabilities,hoston 15,send_empty_bodyon 6,encodingon 1,local_onlyon 1. A caller cannot choose which origin serves a route, andhost: "api"was in a shipped response.invocabledeliberately stays — presence means published, which is not the same as callable.Product routing claimed words it does not own. Two causes, same symptom — a clash put to the user over a word nothing in the request was about:
browser+stack, andbrowseris a devicecontrol entity and a tm configuration alias. It reached past requests that name us: the tm and tra summaries say "BrowserStack", so both became permanent claimants, and"what browsers can I use"— which names us nowhere — clashed four ways.browser; "Mobile APPS are not here" did the same to devicecontrol forapp.Measured over the 198 search-eval and 77 routing-eval queries plus nine paraphrases that name us: 122 → 95 clashes, 155 → 182 settled, and no query's settled product changed.
The envelope guard in
check-contract.pyaccepted aschema_versionthe loader refuses. Loading is a merge, so one bad index would have taken tm, tra and loadtesting down with it.list_browsers/automate/browsers.jsonreturns every OS, browser and version combination the account can run — 5,251 rows, 604 KB — and accepts no parameters:?os=Windows,?browser=chromeand?zzz=1all come back byte-identical to the bare call. Its guidance told the caller to "narrow it yourself", the one thing it had no means to do.selectis opt-in (where,distinct,fields,limit), refuses a field the capability did not declare by name with the set listed, and reportsnarrowed: {from, to, by}— a smaller body that does not announce itself cannot be told from a product that returned less than it has. It is applied to a 2xx only; a 4xx body is the diagnosis. Against production:It is not a query language and should not become one. Anything beyond equality, membership, distinct tuples, projection and a cap is a reason to look for an endpoint that filters.
Published surface
mobiledevicecontrol47 authored / 36 published,devicecontrol32 authored / 26 published. The rest are deliberately withheld rather than shipped untested, and the summaries and guidance were corrected so nothing points at a capability that is not there.Both products are named for what they control rather than for the BrowserStack product brand; the marketing names stay in the summaries and
routing_terms, because that is what users say.Testing
Everything below was measured against production, not read off a spec.
bundleId. Five lifecycle capabilities declared onlyappIdand returned 400 on all five.get_app_statereports a backgrounded app as 3 or 2 — the system suspends it shortly after — so anything below 4 means not on screen.resolutiontakes a fixed per-OS set:1920x1080accepted on Windows 11,2560x1440refused as "not compatible with the chosen OS version".find_elementsresult can be trusted: a strategy the enum rejects never leaves the machine, and an iOS strategy on Android is a 400 rather than a silent miss.Eval coverage: mobile P0 24/24, mobile REST and error 11/11, Android remainder 11/11, iOS remainder 11/11, web REST 8/8, three full web sessions 30/30. Cases needing app builds or account state this account does not have are recorded as blocked rather than passed.
Suite is green at 1345 tests across 89 files, with new files for each mechanism. The tests are not vacuous — reverting each fix was checked to fail its own cases, including the exact
host: "api"that was reported.🤖 Generated with Claude Code