π«π· Version franΓ§aise
The firefox tool
See also: Tool inventory Β· Configuration reference Β· RaggableTree Β· Known limitations Β· Back to index
firefox gives an agent a browser: the Firefox installed on the machine, driven over
WebDriver BiDi on a WebSocket. No driver binary, no
vendor package, no browser download. The agent opens a page, reads it as an outline of sections,
links and controls, clicks, types, presses keys, searches the page by meaning, runs a script and
saves a screenshot. Pages run their JavaScript: what web_scrape cannot see, firefox can.
- One browser per agent run. Firefox starts at the run's first action, keeps its pages
from one action to the next, and closes with the run. Agents running side by side have their
own browsers, or share one with tabs of their own when the host says so (
SharedInstance). Outside any run, an action gets a browser of its own, closed right after it. A host that wants the browser to outlive the run keeps it withLifetime: host(below). - A disposable profile by default: a fresh folder, removed when the browser closes. The host may start the browser on a profile of the Firefox installed on the machine instead, and name the profiles an agent may ask for and the ones no browser ever uses (Profiles).
- The window is visible by default β the operator sees what the agent does.
Headlesshides it; a machine without a display starts headless anyway, with a warning in the log. - Every address is checked by the host's URL policy (
IUrlValidator), or by the fail-closed default guard when the host registers none: private and loopback addresses are refused before the browser starts. The address a page lands on after a click, a key or a script is checked the same way: a refused one is left for a blank page, and the answer says so. - The page is a graph. A snapshot projects the page into a
RaggableTreeof its own β never the code index of the host β and each node carries an id the next actions take.
Giving it to an agent
agents:
researcher:
role: "Researcher"
goal: "Find and summarise sources on the web"
tools: [firefox]
Firefox must be installed on the host (firefox on the PATH, or ExecutablePath). The tool is
registered by AddOrkeonWebTools(), which every shipped host calls; a library host also needs an
IFileSystemService (the screenshot is written through it) and, for search,
AddOrkeonEphemeralSearch() with an embedding provider configured β without one, every other
action works and search says so.
The actions
One tool, one action argument. The permission class of a call depends on the action: the
reading actions pass in plan mode, screenshot is an edit, the others are executions.
| Action | Arguments | Class | What it does |
|---|---|---|---|
navigate |
url |
Read | Opens an http or https page and returns its outline |
back |
β | Read | One page back in the tab's history |
snapshot |
max_nodes, offset |
Read | The outline of the page shown: sections, links, controls |
read |
node_id, offset |
Read | The text of the page, or of one node and the nodes it holds |
find |
kind, name, node_id |
Read | The nodes of a kind (section, link, control), whose name holds a text, under a node |
search |
query, max_nodes |
Read | The sections closest in meaning to a question (RAG) |
screenshot |
save_to_path |
Edit | Writes the PNG of the visible part of the page to a virtual path |
click |
node_id |
Execute | Clicks an element, scrolled into view first |
type |
node_id, text |
Execute | Replaces what a field holds; a line feed presses Enter |
press |
key |
Execute | A key or a chord on the focused element: Enter, Tab, Control+a |
evaluate |
script |
Execute | A JavaScript expression run in the page; a promise is awaited |
tabs |
β | Read | The tabs of the agent, in the order they were opened: id, url, title, current |
new_tab |
url |
Read | Opens a tab, on url when given, and drives it from now on |
switch_tab |
tab |
Read | Drives the tab tab (t1, t2) from now on, shown in front of the others |
close_tab |
tab |
Execute | Closes a tab, the current one by default; the last tab is never closed (close is for that) |
close |
profile |
Execute | Closes the run's browser; the next action starts a new one. Outside any run, under Lifetime: host, closes the host's browser of profile, the default's without (profile counts there only) |
Every answer carries the action, url and title of the page shown, tab, the handle of the
tab the actions drive now, and profile, the one the browser runs with. dialogs comes with
it when the page opened alert, confirm or prompt boxes during the action: what they said,
dismissed so the page never freezes (a beforeunload question is accepted: leaving was what the
agent asked for). The lists an action did not produce - nodes, hits, tabs - are absent from
its answer, never empty.
Every action takes an optional profile: it counts at the first action of a run, when the
browser starts; a browser already open keeps its profile, and an action that names another one
is refused until close. The rule holds under Lifetime: host: the run is bound to the profile
of its first action, and an action without profile goes to that browser, whatever the host's
default.
Tabs
An agent drives one tab at a time and sees only its own: the ones it opened with new_tab,
and the ones a link of one of its pages opened β the agent follows such a tab, as a user's eyes
would. A tab the user or another agent opened is never listed, read or closed. close_tab on
the tab driven moves the agent to the one before it.
Node ids
snapshot, find and search name nodes p<reading>#<index> β p2#7 is element 7 of the
second reading of a page. The ids stay valid while the page does not change: typing into a field
then clicking a button needs no new snapshot. After a navigation, or when an action changed the
page, an old id is refused with the reason, and the agent calls snapshot again. A listing is
cut at MaxNodes (Orkeon:Tools:Firefox, 40 by default, 200 at most): truncated and
total_nodes say so, and offset reads the next part.
A typical flow
navigate url=https://example.org/docs β outline: p1#0 section "Getting started" β¦
search query="how is a crew configured?" β hits: p1#5 (0.82) "Configuration" β¦
read node_id=p1#5 β the text of that section
find kind=control name=search β p1#12 textbox "Search"
type node_id=p1#12 text="crew yaml\n" β the page after Enter, p2#β¦
click node_id=p2#3 β the page the link opened
screenshot save_to_path=/output/docs.png β saved_path
search sends the text of each section of the page to the embedding provider as an inline
source of an ephemeral collection named after the page (prefix firefox): a page read twice is
embedded once. Each hit names the section's node, so the agent reads or clicks what the search
found: that is the link between the RAG subsystem and the page graph.
What never goes through the tool
- A secret: the text of
typeis a tool argument, which the model writes and the transcript keeps. A page that asks for a password is a page the agent should not fill in. - A page outside the policy: the sub-resources a page loads (images, scripts, frames) are fetched by Firefox itself and are not checked by the URL policy; only the pages the tab shows are. Keep the browser on an isolated network when the policy matters.
- The text of a page is not screened for prompt injection: an agent reads it as it reads any tool result. The host's Guardian applies to what the agent then says and does.
Profiles
A browser starts on one of three kinds of profile, and the answer's profile says which:
disposable, the default: a fresh folder underProfileRoot, with the preferences a quiet first run needs, removed when the browser closes. Nothing survives the browser.- A profile of the Firefox installed on the host, by the name
profiles.inigives it (about:profilesshows them): used in place, with its cookies, sessions, extensions and history, and never removed. The agent sees what the profile's user sees, signed in where they are: give an agent a profile made for it, never the operator's own. A profile another Firefox holds open is refused: close that Firefox, or use another profile. - A profile folder, by its path: the same, for a profile
profiles.inidoes not list.
Who chooses: Orkeon:Tools:Firefox:Profile names the profile a browser starts on when the
agent names none. An agent may name disposable, that profile, or one of AllowedProfiles β
nothing else. DeniedProfiles wins over all of them: a profile listed there starts no browser,
whoever asks. Names are compared without regard to case.
"Orkeon": { "Tools": { "Firefox": {
"Profile": "agent-work",
"AllowedProfiles": ["agent-reviews"],
"DeniedProfiles": ["default", "default-release"]
} } }
One browser for several runs
By default every agent run has a browser of its own. SharedInstance: true gives the host one
browser per profile, shared between the runs that use it: each run drives tabs of its own in it,
sees only those, and the browser closes when the last run leaves it β unless Lifetime: host
keeps it (below) β or when the host stops. close closes the
tabs of the run that calls it, not the browser the other runs share. A named
profile is what makes sharing useful: several agents on one signed-in session, side by side.
A browser that outlives the run
Lifetime: host keeps the browser once the run ends: its window, its tabs and its profile β the
disposable one included β stay as the last action left them, and the next run, or an action
outside any run, finds them. One browser per profile: an action names the browser it addresses
with profile, or gets the host's default. The browser closes when an agent calls close,
when the user closes the window, or when the host stops; the disposable folder goes with it. An
action that fails or is cancelled leaves it open. A browser the user closed, or that has died, is
noticed at the next action, which starts a new one. The session is the host's as well: tabs
lists the tabs the earlier runs left, and the next run drives the one the last was on.
Runs that overlap on one profile queue their actions on its browser, one at a time: the host's
browser is shared by every run of the host, which fits a console with one operator β an agent
that picks up where the last one left, a page the user reads between two runs β not agents that
browse side by side. For those, SharedInstance stays the answer, and the two combine: each run
still sees tabs of its own, which close with the run, and the window stays open for the next one
on a blank tab; should the user close that tab, the next run opens one of its own.
{ "Orkeon": { "Tools": { "Firefox": { "Lifetime": "host" } } } }
Inside a run, close closes the run's browser, as always β or, when the run has not used one
yet, the host's browser of the profile named, the default's without; outside any run β a console,
a script β close closes the host's browser the same way. Each time a disposable profile is
created, the tool first removes the disposable folders under ProfileRoot that no Firefox holds
and that are more than ten minutes old: what a host killed with its browsers left behind.
Settings
Orkeon:Tools:Firefox β see the configuration reference:
ExecutablePath, Headless, Profile, AllowedProfiles, DeniedProfiles, SharedInstance,
Lifetime, ProfileRoot, StartupTimeoutSeconds, CommandTimeoutSeconds, MaxNodes,
MaxTextLength. When Firefox is a snap, the default profile root is ~/snap/firefox/common/orkeon/profiles: a
snap cannot read the system's temp folder.
A browser that opened its remote protocol but leaves the first command, session.new, without an
answer for CommandTimeoutSeconds is killed, its profile removed when disposable, and started again
once, with a warning in the log; a second silence fails the start with an error that says Firefox was
started again.