Percolate An AI workflow engine in Postgres GitHub llms.txt

Install

There are three ways in, they install the same thing, and you can move between them later.

Docker Compose

Here we get a working database and services with nothing to compile and no ordering to get right.

curl -fsSL https://raw.githubusercontent.com/Percolation-Labs/get-percolate/main/compose/docker-compose.yml \
  -o docker-compose.yml
docker compose up -d

If you have a model key, put it in a .env beside the file before that up, because each container reads its environment once, when it starts:

echo 'OPENAI_API_KEY=sk-...' > .env      # agent turns
docker compose up -d                     # again, if the stack was already running

The Agent Runtime reads OPENAI_API_KEY, and the workers' embedding credential LLM_API_KEY falls back to it for a hosted embedding model, if you register one. A key exported in some other shell after the stack is up reaches neither. Nothing else needs one. The database, workflows, graph and text search run without it, and so does semantic search: the workers carry an embedding model, bge-small-en-v1.5, which runs on the CPU with no key and no bill. The same .env moves ports that are already taken, most often 5432 by a local Postgres: P8_PG_PORT=5433, and P8_REST_PORT, P8_AGENT_PORT, P8_CONTENT_PORT, P8_QUERY_MCP_PORT, P8_MINIO_PORT and P8_MINIO_CONSOLE_PORT for the others.

Seven services come up beside the database, and that is a list of roles rather than a recommended process count: PostgREST for the REST surface, a worker for steps that leave the machine, an ingest-worker for reading uploaded files, the Content Server for uploads, the Agent Runtime for model turns, query-mcp for MCP clients, and an S3 store for bytes. (docker compose ps -a also lists minio-perms, a one-shot that hands an existing volume to the store and exits.) Five of those are one image under five commands, so a real deployment usually runs fewer — often just a worker, since sql and p8ql steps need no process at all — or many more, a pool per queue. The one pairing not to collapse is the two workers: --queue takes a single queue, so merging them means choosing which of outbound calls and ingestion stops happening, with nothing to say so.

The S3 store is RustFS, under the service name minio, so endpoints and MINIO_* variables in an older .env keep working. MinIO's free images no longer pull from Docker Hub or quay.io. A volume an earlier MinIO install wrote is readable by RustFS: the one-shot minio-perms service hands its files to RustFS's user before the store starts, and does nothing on a fresh volume.

A second install needs a project name of its own. The compose file is project percolate, so docker compose up in a new directory reattaches the last install's volumes, database included. Put a name in the .env beside the file to keep them apart:

echo 'COMPOSE_PROJECT_NAME=my-second' >> .env
docker compose up -d                     # volumes are now my-second_pgdata, my-second_miniodata

The db image is pinned to a version, so docker compose up -d gives every reader the same database rather than whatever their machine last pulled. Updating therefore means changing the tag, not pulling a moving one:

echo 'P8_DB_IMAGE=percolationlabs/percolate-postgres:19-<new>' >> .env
docker compose up -d

P8_DB_IMAGE also runs a local build (P8_DB_IMAGE=percolate-postgres:local). Unset, the file's pin applies.

That is the chosen half of a trade. A floating :19 makes "works on my machine" literally true and unfalsifiable: a stale local copy serves an older extension while every file in the repository says otherwise, and the bug report it produces is about a defect that is already fixed. ci/versions.py enforces the pinned tag against versions.toml, so the number moves in one place and every file that repeats it moves with it.

The new image brings the new extension files; one more statement installs them into the database that already exists:

set role app_owner;
alter extension percolate update;

set role app_owner is not decoration. An extension update creates its new objects as whoever runs it, so running this as a superuser would hand every new table a superuser owner and leave the policies on it inert — the one failure this collection is built to refuse. Updating as app_owner is what keeps ownership and row-level security exactly as a fresh install has them.

Every release ships percolate--<old>--0.2.36.sql for each version published since 0.1.1 — a 0.1.0 database has no upgrade path and is replaced, not updated — and it is the whole schema replayed: every statement in it is written to survive being applied twice, and build-sql.py refuses one that is not. An upgraded database was checked against a fresh install of the same version object by object — 629 of them, identical.

Give it a minute, then ask it whether it is really there. If you have no psql on the host — and nothing above installed one — the database container has its own:

docker compose exec db psql -U p8 -d percolate -c "select * from workflow.compiler_capabilities()"
# or, with a local client -- p8:p8 unless .env sets POSTGRES_USER or POSTGRES_PASSWORD:
psql postgres://p8:p8@localhost:5432/percolate -c "select * from workflow.compiler_capabilities()"
docker compose exec db psql …, on a fresh volume
{"accepts": {"sql": true, "p8ql": true, "rest": true, "when": true,
             "work": true, "agent": true, "embed": true, "timer": true,
             "matrix": true, "repeat": true, "signal": true,
             "batching": true, "rate_key": true,
             "continue_on": true, "sub_workflow": true,
             "output_schema": true, "p8ql_vector_desugar": true},
 "missing": [], "available": true}

And ask it what it was built from, which is a different question:

select * from percolate_build();
 component | version | installed_version | commit_sha | artifact                              | built_at             | stamp_current
-----------+---------+-------------------+------------+---------------------------------------+----------------------+---------------
 parser    | 0.2.36  | 0.2.36            |            | v0.2.36, amd64 .so sha256:…, arm64 .so sha256:… | <build time>         | t
 schema    | 0.2.36  | 0.2.36            | <commit>   |                                       | <build time>         | t

The two rows answer differently, because the two halves are made differently. The schema is generated from a commit, so commit_sha identifies it and artifact is empty. The compiled parser is downloaded from a release — no commit of the schema's repository produced that binary — so artifact names the release and both architectures' .so digests, and commit_sha is empty rather than filled with the nearest plausible commit. A -dirty suffix on a commit means that build came from a working tree that matched no commit at all.

stamp_current compares what the image stamped against what the database is running now. It goes f after ALTER EXTENSION percolate UPDATE, which moves the database without touching the stamp — so the row still describes the image you started from and no longer describes this database. That is the case worth catching, because everything else in the row is then about the wrong build.

It answers from the image and the Helm chart; an install from the release files returns no rows, because the build is recorded when the image is built and the files carry no such record.

A database you did not just create says so here. Compose reattaches the volume of an earlier install that used the same project name, and the new image then starts on the old database without a word. Compare the two versions:

docker compose exec db psql -U p8 -d percolate -tAc "select extversion from pg_extension where extname='percolate'"
docker compose images db              # the image tag this file started

On a fresh install they match. An older extversion means the database predates the image: update it as above, or start clean with docker compose down -v, which deletes that database and its uploads.

Why it works — the database installs itself, and `missing` is the field to read

Two images and nothing to compile. The bootstrap is baked into the image, so the database installs itself the first time it starts — there is no init directory to fetch alongside the compose file and no ordering for you to get right.

missing is what to look at rather than a version string. The compiled parser and the SQL schema ship independently and will eventually disagree, so this probes the installed build with one canary per feature. If it lists anything you have version skew, which otherwise shows up looking like a syntax error in a workflow that is not wrong.

percolate_build() answers the other axis, and the two are not interchangeable. Capabilities detect the parser and the schema drifting apart from each other; both halves can agree perfectly and both be several commits behind the source that produced them. That second case cost a day once — a test suite failing on two parser bugs that were already fixed in the tree, against a .so compiled before the fix — so the build now says what it came from rather than leaving it to be inferred.

Helm

Here we install the whole stack into a cluster, with passwords we generate ourselves.

cat > percolate-values.yaml <<EOF
secrets:
  postgresPassword: $(openssl rand -hex 24)
  authenticatorPassword: $(openssl rand -hex 24)
  workerPassword: $(openssl rand -hex 24)
  jwtSecret: $(openssl rand -hex 32)
  s3Key: $(openssl rand -hex 16)
  s3Secret: $(openssl rand -hex 24)
  openaiApiKey: "${OPENAI_API_KEY:-}"   # the Agent Runtime's model key
  llmApiKey: "${OPENAI_API_KEY:-}"      # what embedding steps resolve
EOF
helm install percolate oci://ghcr.io/percolation-labs/charts/percolate \
  -n percolate --create-namespace -f percolate-values.yaml

You get the database, PostgREST, the three services, two worker pools — http and ingest — and an S3 store (RustFS, under the name minio, as in compose).

Keep percolate-values.yaml. Upgrades take it with -f; generating the passwords again writes new ones into the Secret while the database keeps the ones it started with, and every service then fails to log in. jwtSecret is also the secret you sign tokens with. The passwords are hex because the services connect with a postgres:// URL, and the / that base64 produces about four times in ten ends the password inside it. There are no CRDs and no operator — the only custom resources are KEDA's, and only if you turn autoscaling on.

Why it works — an OCI artefact, and four passwords the chart refuses to generate

The chart is an OCI artefact, so there is no chart repository, no index.yaml and no DNS involved, and each of those is a thing that can break on its own. helm repo add percolate https://percolation-labs.github.io/get-percolate/charts works too if you prefer a classic repo, and Flux and Argo can both point straight at charts/percolate in git with nothing published at all.

The four passwords are required and the chart will not generate them for you, which looks unhelpful until you see the failure it avoids. A generated value comes out different between helm template and helm install, and between a GitOps dry run and the apply after it — so a controller that diffs continuously will either report drift forever or "fix" it by rewriting the database password underneath a running StatefulSet.

Into a Postgres 19 you already run

Here we add the extensions to a database that already exists, without a service anywhere.

pgvector first, from your platform's packages, upgraded together with the server: apt install postgresql-19 postgresql-19-pgvector from the PGDG repository. While Postgres 19 is pre-release, PGDG builds pgvector against its newest beta or rc, and a pgvector built for one does not load into another: create extension vector fails with did not return an IndexAmRoutine struct. CI runs this path on postgres:19rc1-bookworm. Then the extension files, and then bootstrap.sql, which install.sh downloads into the current directory — run it as a superuser, in the database that will hold Percolate:

curl -fsSL https://raw.githubusercontent.com/Percolation-Labs/get-percolate/main/install.sh | sh

export P8_AUTH_PW=$(openssl rand -hex 24) P8_WORKER_PW=$(openssl rand -hex 24)
psql -d yourdb -v ON_ERROR_STOP=1 \
     -v auth_pw="$P8_AUTH_PW" -v worker_pw="$P8_WORKER_PW" -f bootstrap.sql

Keep the two passwords: PostgREST connects as authenticator and the workers as worker. Add -v app_provisioner_pw="$(openssl rand -hex 24)" only if an application will provision its own users as app_provisioner; without it that role is not created. A bare CREATE EXTENSION percolate does not work on this path — as a superuser the extension refuses to load, and as anyone else the roles it needs do not exist yet — which is what bootstrap.sql sequences. It is safe to run again.

Where this works, and where it does not

This path needs two things a managed Postgres service usually withholds: PostgreSQL 19, the only major the install is tested on, and a superuser for one step. bootstrap.sql creates the roles and installs pgvector and percolate_parser, a compiled C extension. percolate itself then installs as an ordinary owner, with pg_trgm and pgcrypto, which are trusted. So it works on a server you run yourself: a VM, a container, or bare metal.

Checked against each vendor's own documentation on 21 October 2026, while PostgreSQL 19 is a release candidate (GA is planned for 19 November):

Service PostgreSQL 19 A C extension of your own pgvector, pg_trgm So
Amazon RDS preview only (betas and RC 1) no. Only its listed extensions, and pg_tle for trusted languages, which a C extension is not yes not yet
Cloud SQL no (lists up to 18) no: "You cannot create your own extensions in Cloud SQL" yes no
Supabase no no on the hosted platform; self-hosted only by building an image Supabase calls "unofficial and unsupported" yes self-hosted only, on your own image
Neon no (lists 14 to 18) only by request through Neon support yes no

Every row fails on the same item, the parser, because the dialect and the graph algorithms are compiled code. Until a service loads it, run Percolate's database yourself and point your application at it.

Updating is the same two commands again, then one statement. install.sh installs the release's percolate--<old>--<new>.sql scripts beside the new version and says so when it finds an earlier one, and bootstrap.sql makes the grants a newer extension refuses to update without. A database that already has the extension keeps its version until it is updated, as app_owner for the reason the compose update gives:

set role app_owner;
alter extension percolate update;
Why it works — two extensions that ship differently, because they are different kinds of thing
Extension What it is How it ships
percolate the whole system — schemas, tables, functions, RLS pure SQL, one file, the same on every platform
percolate_parser the P8QL and YAML compilers a Rust .so, prebuilt per platform

A shared library is compiled against one ABI, so there is no portable form of the parser and we build it per platform instead. install.sh reads your pg_config — the only thing that knows where this particular Postgres keeps its extensions — checks the major version, and puts both files where they belong.

We publish parser builds for linux/amd64, linux/arm64 and macos/arm64. On anything else the script installs the SQL extension, tells you that define_yaml and p8ql: steps will not resolve until the parser is built, and exits non-zero, so a half install does not look like a successful one.

bootstrap.sql has two halves. As the superuser it creates the cluster roles and the two extensions that are not trusted, vector and percolate_parser; then it switches to app_owner and runs CREATE EXTENSION percolate CASCADE, which creates the trusted pgcrypto and pg_trgm and everything percolate itself holds. So all of it is owned by a non-superuser, which is not a detail: a superuser bypasses RLS unconditionally, so a superuser-owned install would leave every policy in the collection inert while looking correct. The extension asserts this about itself rather than trusting the installer to get it right.

pg_cron, if you want schedules

This one is optional and its absence is quiet, so it is worth setting up consciously.

Here we give the engine a clock, which it needs for three separate things.

Install the package (apt install postgresql-19-cron), then in postgresql.conf:

shared_preload_libraries = 'pg_cron'   # added to any list already there
cron.database_name = 'yourdb'          # the database bootstrap.sql ran in
cron.use_background_workers = on       # jobs run in-process, with no login

Restart the server and run bootstrap.sql again. It creates pg_cron and schedules the three jobs — workflow.tick(), workflow.reap_stale_tasks() and workflow.promote_due_timers(), every minute — as the scheduler role rather than as a superuser. The compose image and the Helm chart do all of this themselves.

Why it works — and what does not happen without it, unreported

pg_cron runs as a background worker, so there is no CREATE EXTENSION that can add it after startup. Without it, scheduled workflows never fire, the stale-task reaper never runs so a crashed worker's tasks are never recovered, and every timer step waits forever. None of those produce an error; they produce a system that looks idle.

One set of jobs covers the whole deployment however many schedules you have, because a schedule is a row in workflow.schedules rather than a cron entry.

cron.database_name is the usual reason a correctly-created job never runs — it can only be one database, and the default is postgres. cron.use_background_workers is the other: without it each job logs in over libpq as scheduler, which has no password, so wherever pg_hba.conf asks for one every run fails with connection failed while cron.job still shows the job active.

The first user, and a token

A fresh install has no users, no roles and no permissions — those tables are empty, because the alternative is a default administrator with a known password. Nothing over HTTP works until you create one: PostgREST answers permission denied for function upsert_agent with a 401, and the agent runtime answers a verified bearer token is required.

Here we get from an empty rbac to a bearer token that the REST interface and the agent runtime both accept.

Run it in psql on the database, docker compose exec db psql -U p8 -d percolate; \gset keeps the id it returns in :uid for that session.

select rbac.bootstrap_admin('me@example.com', 'a long passphrase') as uid \gset

That is the whole step. It creates the user, creates an admin role, grants it the permissions the system actually checks, and makes the first user_roles row — and it refuses to run once any role has been granted to anyone, which is what keeps it a bootstrap rather than a back door.

Use it rather than writing the insert statements yourself. A grant names a (resource, action) pair, and only the pairs some policy or function checks are accepted: rbac.role_permissions references rbac.permission_kinds, so granting workflow_runs/read where the policies ask for select is refused by a foreign key at the insert, rather than giving you an administrator who sees zero rows. select * from rbac.permission_kinds lists every pair.

Then the token. The core image the stack runs carries the percolate CLI and the stack's signing secret, so it signs one for you: nothing to install, and no token to build by hand. Run it in the directory with the compose file:

uid=$(docker compose exec -T db psql -U p8 -d percolate -Atc \
        "select id from rbac.users where email = 'me@example.com'")
export TOKEN=$(docker compose exec -T content percolate auth token --sub "$uid" \
                 --orgs d0000000-0000-0000-0000-00000000000a)

--orgs narrows the token to the sample's first tenant, which the sample makes you a member of. A token without --orgs sees every org you belong to; one naming an org you do not belong to sees none of its rows, and no error says why, so leave it out until you have loaded the sample. The token lasts an hour; run the block again for a new one. Without the compose stack, percolate auth token --sub <uid> from pip install percolate-core, with P8_JWT_SECRET set to your secret, signs the same token.

That is the $TOKEN every curl in these docs wants.

Your own agent, over MCP

Here we connect the agent you already use, such as Claude Code, to Percolate as yourself.

The query MCP server is on port 8090 of this machine only (127.0.0.1). It holds no credential of its own: it forwards your token to PostgREST, so every tool call runs as you, under your row policies. Beside query it offers the tools a connected agent judges and remembers with: list_judgements, claim_judgement, decide_judgement and release_judgement for what dream left for a judge, and read_session and land_message_facts for a conversation's facts.

claude mcp add --transport http percolate http://localhost:8090 \
  --header "Authorization: Bearer $TOKEN"

Claude Desktop reaches a local HTTP server through a bridge, configured in claude_desktop_config.json:

{"mcpServers": {"percolate": {"command": "npx",
  "args": ["mcp-remote", "http://localhost:8090",
           "--header", "Authorization: Bearer <your token>"]}}}

The token expires (exp), and an expired one makes every call fail at PostgREST; mint a new one and update the header. A call with no token is refused: the MCP response is an HTTP 200 whose tool result is an error carrying PostgREST's 401 (permission denied for schema aiq), so look in the result, not at the status. P8_QUERY_MCP_PORT in .env moves the port if 8090 is taken.

the same call, before and after
$ curl -s localhost:3000/rpc/upsert_agent -H 'Content-Profile: agentic' \
       -H 'Content-Type: application/json' -d '{"p_spec":{"name":"harbourmaster"}}'
{"code":"42501","message":"permission denied for function upsert_agent"}  # HTTP 401

$ curl -s localhost:3000/rpc/upsert_agent -H 'Content-Profile: agentic' \
       -H "Authorization: Bearer $TOKEN" \
       -H 'Content-Type: application/json' -d '{"p_spec":{"name":"harbourmaster"}}'
{"id":"4dc4e929-…","name":"harbourmaster", …}                          # HTTP 200
Why the first grant cannot go through `grant_role`, and why `psql` could author an agent all along

rbac.grant_role is gated on rbac.has_permission(current_user_id(), 'users', 'admin'), and on an empty database nobody holds that — including the person trying to bootstrap. So the first user_roles row cannot come from grant_role, and every grant after it can. That is a real bootstrap step rather than an oversight: the alternative is a function that grants users/admin to whoever asks first.

bootstrap_admin is that step with the escalation closed rather than left to the caller. It refuses the moment rbac.user_roles holds anything, so it is reachable only while the system has no administrator — the one window in which handing out users/admin is not a privilege escalation. It is also not granted to web_anon or authenticated, so it does not exist over PostgREST; like the CREATE EXTENSION before it, it is run from a privileged local session.

It is also why the SQL on the agents page works from psql before any of this. agentic.may_author permits the call when rbac.current_user_id() is null and the session is a privileged local one — the migration path — so psql is authoring as the database owner, not as a user. The moment the same call arrives over HTTP there is a JWT and therefore a user, and the permission is checked.

Passwords are for rbac.login, which exchanges them for a refresh token; it does not mint the access token, because whatever holds P8_JWT_SECRET does. There is no HTTP endpoint that hands you one, so a token comes from something holding the secret — percolate auth token, in the image or installed, or your own auth provider signing with the same secret and the same claims.

The sample, which nothing loads for you

A fresh install is empty, and every worked example in these pages reads data. There is a sample for that, and loading it is a step you take rather than something a container did while you were not looking.

Here we get the domain the rest of this documentation queries, and be able to tell it apart from our own data afterwards.

The sample needs the percolate CLI from percolate-core on your own machine. The compose stack runs that image; it does not put the command on your PATH. It needs Python 3.11 or newer; on a Mac the system python3 is older than that and pip will report the package as simply not existing rather than as unsupported.

The extras are not optional decoration: sample is the YAML reader and agent is what turns plugin.yaml's agents — which are JSON Schema documents, not prompt strings — into rows. Without them the load refuses before it writes anything, which is the right behaviour and still a stop.

It goes in a virtualenv, because a bare pip install stops at externally-managed-environment on Homebrew's Python and on current Debian and Ubuntu:

python3 -m venv ~/.percolate && . ~/.percolate/bin/activate   # 3.11 or newer
pip install 'percolate-core[sample,agent]>=0.1.12'

. ~/.percolate/bin/activate puts it back on your PATH in a new shell.

samples/harbour is a path inside this repository, so it needs to be on disk — a compose install has only the one file you curled. The DSN has to be the owner's, because the load writes rbac.*; the CLI names the compose one in its own error if you forget:

git clone https://github.com/Percolation-Labs/get-percolate
cd get-percolate
export P8_ADMIN_DSN=postgres://p8:p8@localhost:5432/percolate   # p8:p8 unless .env changed them
export P8_JWT_SECRET=change-me-a-long-random-string-at-least-32-chars
percolate sample load samples/harbour --as-email me@example.com

If .env moved P8_AGENT_PORT or P8_CONTENT_PORT, tell the loader too: --agent-url http://localhost:<agent port> and --content-url http://localhost:<content port>. It defaults to 8080 and 8081 and does not read .env, so with a moved agent port the load still exits 0 but prints tool server harbour-query: not synced, http://localhost:8080 is unreachable, and the plugin's tools are missing from the catalogue.

P8_JWT_SECRET is the compose file's default signing secret — use your own if you set one in .env — and the loader signs the token it uploads the corpus with. It ends loaded, and every document is embedded. once the stack's ingest worker has embedded the corpus.

A port-operations company: two tenants, four operators, five vessels, three ports, inspections, a corpus of reports and a graph tying them together.

harbour 1.0.0
  sql schema.sql
  embedding model bge-small-en-v1.5 (384d)
  entity operator <- harbour.operators
  entity vessel <- harbour.vessels
  entity port <- harbour.ports
  ...
  edge operator:MERB -subsidiary_of-> operator:MERI
  plugin harbour (1 servers, 1 skills, 1 agents)
  workflow harbour_deficiencies

Most of it is tenanted. The token from the first user names Meridian (...000a); for Kestrel, run that block again with --orgs d0000000-0000-0000-0000-00000000000b. A token that names neither sees only the shared tier, and its queries return no tenant rows.

It needs no key. The corpus goes in through POST /files and is embedded by the running pipeline with bge-small-en-v1.5, the model the workers carry, so SEMANTIC and SEARCH have vectors to rank as soon as the load returns. --dry-run says what a load would do without writing anything. To embed with a hosted model instead, uncomment its entry in samples/harbour/sources.yaml, point ingest_pipelines at it, and put its key in .env before the stack starts. A second load uploads the documents again and those copies fail to parse, so remove the first ones before loading again — the statement is in the sample's README.

Why it works — a directory of the documents you would have written anyway, and why the vectors are not in it

The sample is samples/harbour/ in the repository, and it is worth opening before you run it. schema.sql is the company's own tables — nothing in it mentions Percolate, which is the point, because Percolate indexes tables you already have. Everything else is a document in the format you would author by hand: sources.yaml is what is addressable, graph.yaml is the relation vocabulary with edges named vessel:Meridian Dawn rather than by uuid, plugin.yaml is skills and agents as one removable bundle, workflows/ are workflow documents and documents/ is markdown. Reading it teaches the formats; a .sql dump would have hidden all of them behind four hundred INSERTs.

The vectors are not in the files. An earlier version of this fixture shipped literal four-dimension vectors so it would load with nothing running. It reproduced beautifully and taught the wrong thing: a reader who copied the pattern had a corpus no model had ever seen and rankings that meant nothing. Here the documents are embedded by the same pipeline yours will be, so what you search is what a model produced. The model runs in the worker, so that costs CPU time rather than a key.

Nothing loads on first boot for the same reason. Rows that appear because a container started are rows nobody chose, and a demo you cannot tell apart from a deployment is a bad demo. Applying the plugin with an empty manifest takes the skills and agents back out and says what it removed, and drop schema harbour cascade takes the rest:

select agentic.apply_plugin('{"name":"harbour","version":"1.0.0"}'::jsonb);
drop schema harbour cascade;

Uninstalling is the same call as installing rather than a second function, because the pruning path is the one that knows what else references a row — which is why the plugin columns are on delete restrict.

Next: agents, which is the shortest useful thing to do against a fresh install: write one spec, save it, and call it.