This directory contains utilities for continuous benchmarking of libc++ with LNT. This can be done locally using a local instance, or using a public instance like http://lnt.llvm.org.

Running a benchmark bot

The run-benchbot script is the main entry point for running benchmarks. That script is where libc++'s pre-defined LNT bot configurations are defined. To benchmark specific commits:

libcxx/utils/ci/lnt/run-benchbot --llvm-root <monorepo> <builder> -- <commit1> <commit2> ...

Results are stored as JSON files in <llvm-root>/build/<builder>/ by default. Use --build-dir <dir> to override the output directory.

To continuously poll for un-benchmarked commits and submit results to a LNT instance:

libcxx/utils/ci/lnt/run-benchbot --llvm-root <monorepo> --lnt-url http://lnt.llvm.org <builder>

Setting up a local LNT instance

# Create an instance and run a server
lnt create my-instance
echo "api_auth_token = 'example_token'" >> my-instance/lnt.cfg
lnt runserver my-instance

# In another terminal, create the libcxx test suite on the locally-running server
cat <<EOF > lnt-admin-config.yaml
lnt_url: "http://localhost:8000"
database: default
auth_token: example_token
EOF
lnt admin --config lnt-admin-config.yaml --testsuite libcxx test-suite add libcxx/utils/ci/lnt/schema.yaml

# Then run the benchbot against the local instance
libcxx/utils/ci/lnt/run-benchbot --llvm-root <monorepo> --lnt-url http://localhost:8000 <builder>