Running the Framework Test Suite

This page is for people hacking on Wheels itself — the code in vendor/wheels/ of the framework repo. If you’re testing your application, you want Running Tests Locally; nothing on this page exists in a scaffolded app.

The fast inner loop is Lucee 7 + SQLite via tools/test-local.sh. The Docker matrix exists for the other job: proving a change works across every engine and database before you push.

The one-shot script — tools/test-local.sh

tools/test-local.sh is what every framework contributor runs before pushing. It creates the two SQLite test databases (wheelstestdb.db and wheelstestdb_tenant_b.db), starts a disposable server on port 8080 if one isn’t already up, downloads the SQLite JDBC driver if missing, runs the suite, and prints coloured pass/fail output. On exit it stops any server it started.

your shell — in the wheels repo root

bash tools/test-local.sh              # run all core tests
bash tools/test-local.sh model        # run model specs only
bash tools/test-local.sh security     # run security specs only
bash tools/test-local.sh browser      # run browser specs (Playwright JARs required)

The script accepts these short aliases as the first argument: model / models, controller / controllers, view / views, security, middleware, dispatch, migrator. Anything else is passed through as a literal directory path (dot-notation, rooted at wheels.tests.specs.).

Environment knobs:

• PORT=9090 bash tools/test-local.sh — use a non-default port (handy when 8080 is in use).
• DB=mysql bash tools/test-local.sh — run against a different database; bring up the container yourself first (see the Docker matrix below).
• WHEELS_BROWSER_TEST_BASE_URL — auto-set to http://localhost:${PORT} so browser specs hit the same server the script starts.

You can also run the framework suite through the CLI: wheels test --core, with --db=<sqlite|h2|mysql|postgres|sqlserver|oracle|cockroachdb> selecting between the wheelstestdb_* datasources the matrix wires up (--db is only honoured with --core). The CLI throws Wheels.TestsFailed (non-zero exit) on Fail/Error, on directoryRejected / bundlesDiscovered=0 (#3083), and on compile-skipped *Spec.cfc files. Do not treat a curl of /wheels/core/tests as pass/fail from totalFail/totalError alone — check those honesty fields too.

The core test-runner URL

The fastest iteration loop is hitting the runner directly over HTTP — this is what the script calls under the hood:

direct HTTP access (dev server running)

# Run everything
curl "http://localhost:60007/wheels/core/tests?db=sqlite&format=json"

# Filter by directory
curl "http://localhost:60007/wheels/core/tests?db=sqlite&format=json&directory=wheels.tests.specs.model"

# Run a single spec file: its dotted path runs it as one bundle
curl "http://localhost:60007/wheels/core/tests?db=sqlite&format=json&directory=wheels.tests.specs.model.callbacksSpec"

# Skip populate when iterating on one spec and the schema is already in place
curl "http://localhost:60007/wheels/core/tests?db=sqlite&format=json&populate=false&directory=wheels.tests.specs.model.callbacksSpec"

# Scope to a single installed package's tests (e.g. wheels-sentry, wheels-i18n)
curl "http://localhost:60007/wheels/core/tests?db=sqlite&format=json&directory=vendor.wheels-sentry.tests"

directory= takes a directory or the dotted path of one spec file, which runs as a single bundle (#3759). bash tools/test-local.sh accepts the same, as dispatch/TestScopeVisibilitySpec or in dotted form. The core endpoint only accepts directory values starting with wheels.tests (or vendor.<package>.tests); anything else is rejected and the full suite runs instead — the JSON flags this with directoryRejected: true and a warnings[] entry rather than silently reporting green (#3083). populate=false is the other inner-loop trick: populate drops and recreates every test table, so skipping it once the schema exists shaves the run down to just your specs.

Scope-visibility fields in the JSON response

The payload includes directoryRequested, directoryResolved, directoryRejected, bundleResolved (the spec file’s dotted path, when directory= named one), bundlesDiscovered, and warnings[]. A path that names neither a folder of specs nor a spec file yields bundlesDiscovered: 0 and a warning, and wheels test exits non-zero.

Docker cross-engine matrix

compose.yml at the repo root ships a service per supported engine, bind-mounted to your checkout — edit code, hit the port, no rebuilds. tools/test-matrix.sh wraps it end-to-end (tools/test-matrix.sh lucee6,adobe2025 sqlite mirrors CI’s compat matrix); or drive it by hand:

your shell — in the wheels repo root

# Start Lucee 7 + Adobe 2025 (minimum for a cross-engine check)
docker compose up -d lucee7 adobe2025

# Wait ~60s for startup, then run both
curl -s -o /tmp/lucee7.json   "http://localhost:60007/wheels/core/tests?db=sqlite&format=json"
curl -s -o /tmp/adobe2025.json "http://localhost:62025/wheels/core/tests?db=sqlite&format=json"

# Summarize — fail closed on #3083 honesty fields, not just Fail/Error
for f in /tmp/lucee7.json /tmp/adobe2025.json; do
  python3 -c "
import json, sys
d = json.load(open('$f'))
print('$f', d.get('totalPass',0), 'pass', d.get('totalFail',0), 'fail', d.get('totalError',0), 'error', 'rejected=', d.get('directoryRejected'), 'bundles=', d.get('bundlesDiscovered'))
bad = d.get('directoryRejected') or d.get('bundlesDiscovered', 1) == 0 or d.get('totalFail',0) or d.get('totalError',0)
sys.exit(1 if bad else 0)
"
done

Engine	Service	Port
Lucee 5	lucee5	60005
Lucee 6	lucee6	60006
Lucee 7	lucee7	60007
Adobe 2018	adobe2018	62018
Adobe 2021	adobe2021	62021
Adobe 2023	adobe2023	62023
Adobe 2025	adobe2025	62025
BoxLang	boxlang	60001

Tip

At minimum, run one Lucee and one Adobe before pushing — their runtimes diverge in ways pure-Lucee tests don’t catch. Do not iterate on Adobe failures through CI; reproduce locally (round-tripping GitHub Actions burns ~20 minutes per attempt).

Testing against specific databases

Each engine ships with H2 and SQLite baked in. Everything else is a compose service you bring up alongside the engine: mysql, postgres, sqlserver, cockroachdb, oracle.

your shell — multi-database

docker compose up -d lucee7 mysql
curl -sf "http://localhost:60007/wheels/core/tests?db=mysql&format=json"

Supported db values: sqlite (script default), h2, mysql, postgres, sqlserver, oracle, cockroachdb. Oracle is soft-failed in CI — failures log warnings but don’t block the build. (The docker-compose.db-*.yml files at the repo root are dev-stack overlays for the demo app, not test-matrix services.)

Cross-engine gotchas

The runtime differences that catch first-time framework contributors — each one has stung a real PR. The full list lives in Coding Standards; the ones that bite in tests specifically:

• struct.map() resolves to the built-in struct member function on Lucee/Adobe, not your CFC method.
• application scope members — Adobe CF rejects function members on the application scope.
• Closure this capture — share references across closures with var ctx = { ref: obj }.
• obj["key"]() inside closures crashes Adobe 2021/2023’s parser — split into var fn = obj["key"]; fn();.
• Arrays copy by value in struct literals on Adobe — reference through a parent struct instead.
• private mixin functions are not integrated — $integrateComponents() copies only public methods; helpers in vendor/wheels/model/*.cfc and friends must be public with a $ prefix.

Contributor failure modes

• Docker daemon not running — start Docker Desktop (macOS/Windows) or systemctl start docker (Linux).
• Port already in use — lsof -i :60007 and stop the holder; or PORT= a different one for the script.
• Populate SQL errors — “Table already exists” means a stale table from a previous engine-specific run (add DROP TABLE IF EXISTS); “No such function: NOW” means cross-engine SQL drift — use CURRENT_TIMESTAMP.
• Stale SQLite files — rm -f wheelstestdb.db wheelstestdb_tenant_b.db between weird runs.
• Server not responding after compose up — engines cold-start in 30–90 seconds; check docker compose logs <service> if it stays down past two minutes.

cleanup

docker compose down
rm -f wheelstestdb.db wheelstestdb_tenant_b.db
wheels stop

Related

Coding Standards The cross-engine rules in full, plus the pre-push test bar.

Submitting Pull Requests What CI runs and what reviewers expect.

Running Tests Locally Testing your application (not the framework).