Running the Framework Test Suite

This page is for people hacking on Wheels itself — the code in vendor/wheels/ of the framework repo. If you're testing your application, you want Running Tests Locally; nothing on this page exists in a scaffolded app.

The fast inner loop is Lucee 7 + SQLite via tools/test-local.sh. The Docker matrix exists for the other job: proving a change works across every engine and database before you push.

The one-shot script — tools/test-local.sh

tools/test-local.sh is what every framework contributor runs before pushing. It creates the two SQLite test databases (wheelstestdb.db and wheelstestdb_tenant_b.db), starts a disposable server on port 8080 if one isn't already up, downloads the SQLite JDBC driver if missing, runs the suite, and prints coloured pass/fail output. On exit it stops any server it started.

your shell — in the wheels repo root

bash tools/test-local.sh              # run all core tests
bash tools/test-local.sh model        # run model specs only
bash tools/test-local.sh security     # run security specs only
bash tools/test-local.sh browser      # run browser specs (Playwright JARs required)

The script accepts these short aliases as the first argument: model / models, controller / controllers, view / views, security, middleware, dispatch, migrator. Anything else is passed through as a literal directory path (dot-notation, rooted at wheels.tests.specs.).

Environment knobs:

• PORT=9090 bash tools/test-local.sh — use a non-default port (handy when 8080 is in use).
• DB=mysql bash tools/test-local.sh — run against a different database; bring up the container yourself first (see the Docker matrix below).
• WHEELS_BROWSER_TEST_BASE_URL — auto-set to http://localhost:${PORT} so browser specs hit the same server the script starts.

You can also run the framework suite through the CLI: wheels test --core, with --db=<sqlite|h2|mysql|postgres|sqlserver|oracle|cockroachdb> selecting between the wheelstestdb_* datasources the matrix wires up (--db is only honoured with --core).

The core test-runner URL

The fastest iteration loop is hitting the runner directly over HTTP — this is what the script calls under the hood:

direct HTTP access (dev server running)

# Run everything
curl "http://localhost:60007/wheels/core/tests?db=sqlite&format=json"

# Filter by directory
curl "http://localhost:60007/wheels/core/tests?db=sqlite&format=json&directory=wheels.tests.specs.model"

# Filter to a single spec file — use testBundles, not directory
curl "http://localhost:60007/wheels/core/tests?db=sqlite&format=json&testBundles=wheels.tests.specs.model.callbacksSpec"

# Skip populate when iterating on one spec and the schema is already in place
curl "http://localhost:60007/wheels/core/tests?db=sqlite&format=json&populate=false&testBundles=wheels.tests.specs.model.callbacksSpec"

# Scope to a single installed package's tests (e.g. wheels-sentry, wheels-i18n)
curl "http://localhost:60007/wheels/core/tests?db=sqlite&format=json&directory=vendor.wheels-sentry.tests"

directory= only filters down to a directory — pointing it at a single spec file matches zero bundles and runs nothing; use testBundles= for one file. The core endpoint only accepts directory values starting with wheels.tests (or vendor.<package>.tests); anything else is rejected and the full suite runs instead — the JSON flags this with directoryRejected: true and a warnings[] entry rather than silently reporting green (#3083). populate=false is the other inner-loop trick: populate drops and recreates every test table, so skipping it once the schema exists shaves the run down to just your specs.

Scope-visibility fields in the JSON response

The payload includes directoryRequested, directoryResolved, directoryRejected, bundlesDiscovered, and warnings[]. A path naming a single spec file yields bundlesDiscovered: 0 and a warning — reach for testBundles= instead.

Docker cross-engine matrix

compose.yml at the repo root ships a service per supported engine, bind-mounted to your checkout — edit code, hit the port, no rebuilds. tools/test-matrix.sh wraps it end-to-end (tools/test-matrix.sh lucee6,adobe2025 sqlite mirrors CI's compat matrix); or drive it by hand:

your shell — in the wheels repo root

# Start Lucee 7 + Adobe 2025 (minimum for a cross-engine check)
docker compose up -d lucee7 adobe2025

# Wait ~60s for startup, then run both
curl -s -o /tmp/lucee7.json   "http://localhost:60007/wheels/core/tests?db=sqlite&format=json"
curl -s -o /tmp/adobe2025.json "http://localhost:62025/wheels/core/tests?db=sqlite&format=json"

# Summarize
for f in /tmp/lucee7.json /tmp/adobe2025.json; do
  python3 -c "
import json
d = json.load(open('$f'))
print('$f', d['totalPass'], 'pass', d['totalFail'], 'fail', d['totalError'], 'error')
"
done

| Engine | Service | Port | |--------|---------|------| | Lucee 5 | lucee5 | 60005 | | Lucee 6 | lucee6 | 60006 | | Lucee 7 | lucee7 | 60007 | | Adobe 2018 | adobe2018 | 62018 | | Adobe 2021 | adobe2021 | 62021 | | Adobe 2023 | adobe2023 | 62023 | | Adobe 2025 | adobe2025 | 62025 | | BoxLang | boxlang | 60001 |

Tip

At minimum, run one Lucee and one Adobe before pushing — their runtimes diverge in ways pure-Lucee tests don't catch. Do not iterate on Adobe failures through CI; reproduce locally (round-tripping GitHub Actions burns ~20 minutes per attempt).

Testing against specific databases

Each engine ships with H2 and SQLite baked in. Everything else is a compose service you bring up alongside the engine: mysql, postgres, sqlserver, cockroachdb, oracle.

your shell — multi-database

docker compose up -d lucee7 mysql
curl -sf "http://localhost:60007/wheels/core/tests?db=mysql&format=json"

Supported db values: sqlite (script default), h2, mysql, postgres, sqlserver, oracle, cockroachdb. Oracle is soft-failed in CI — failures log warnings but don't block the build. (The docker-compose.db-*.yml files at the repo root are dev-stack overlays for the demo app, not test-matrix services.)

Cross-engine gotchas

The runtime differences that catch first-time framework contributors — each one has stung a real PR. The full list lives in Coding Standards; the ones that bite in tests specifically:

• struct.map() resolves to the built-in struct member function on Lucee/Adobe, not your CFC method.
• application scope members — Adobe CF rejects function members on the application scope.
• Closure this capture — share references across closures with var ctx = { ref: obj }.
• obj["key"]() inside closures crashes Adobe 2021/2023's parser — split into var fn = obj["key"]; fn();.
• Arrays copy by value in struct literals on Adobe — reference through a parent struct instead.
• private mixin functions are not integrated — $integrateComponents() copies only public methods; helpers in vendor/wheels/model/*.cfc and friends must be public with a $ prefix.

Contributor failure modes

• Docker daemon not running — start Docker Desktop (macOS/Windows) or systemctl start docker (Linux).
• Port already in use — lsof -i :60007 and stop the holder; or PORT= a different one for the script.
• Populate SQL errors — "Table already exists" means a stale table from a previous engine-specific run (add DROP TABLE IF EXISTS); "No such function: NOW" means cross-engine SQL drift — use CURRENT_TIMESTAMP.
• Stale SQLite files — rm -f wheelstestdb.db wheelstestdb_tenant_b.db between weird runs.
• Server not responding after compose up — engines cold-start in 30–90 seconds; check docker compose logs <service> if it stays down past two minutes.

cleanup

docker compose down
rm -f wheelstestdb.db wheelstestdb_tenant_b.db
wheels stop

Related

Coding Standards The cross-engine rules in full, plus the pre-push test bar.

Submitting Pull Requests What CI runs and what reviewers expect.

Running Tests Locally Testing your application (not the framework).