Benchmarking Rust Code on OSX
Today I went through the rust wasm tutorial and learned a bunch especially regarding performance via this section.
This tutorial explains benchmarking using browser tools like
console.time, but I'll focus on
the lower level benchmarking part here.
The author uses
cargo bench to visualize which doesn't exist on OSX, but I
discovered cargo instruments as well as rust
flamegraph. The latter worked a bit better to
just find bottlenecks.
I had to install this tool via
brew install cargo-instruments as it didn't build with cargo.
You can list all available templates via
cargo instruments --list.
/benches/bench.rs, a typical run through would be:
cargo instruments -t alloc --bench bench open target/instruments/my_bin_YYYY-MM-DD-THH:MM:SS.trace
time template did not result in any data showing in Instruments, but
sys did show some data which is why I am using the former for this example even though it
actually measures memory allocations instead of CPU usage and timing.
Once in the Instruments UI make sure to scroll down to see the app in some cases.
However in the same redit thread rust flamegraph was mentioned which I then tried.
cargo install flamegraph
Add this to
Cargo.toml to get better symbols in the flamegraph.
[profile.bench] debug = true
Run the below.
sudo is necesary to usd dtrace properly.
sudo cargo flamegraph --bench bench
Then open the flamegraph.
Unfortunately after running with
sudo you end up with a file in your
target/release/.fingerprint folder which causes permission problems later. It'll look something like
drwxr-xr-x 7 user admin 224B Jan 30 14:05 project-name-5063881d9843ce56/ drwxr-xr-x 7 root admin 224B Jan 30 14:26 project-name-58809faea7027018/
As you can see the second one is only accessible via
root and you need to manually remove it
before cargo is happy again.
However another one like this exists, i.e.
So in the end I just did
sudo rm -rf target/ to clean things up in one go. However that
incurred a recompile so improvement is possible here.
Back to the tutorial, if we don't need to see visualizations but just compare benchmarks after a chnage we'd proceed as follows.
Install the tool:
cargo install cargo-benchcmp
cargo bench | tee before.txt # Make changes you think imporve perf cargo bench | tee after.txt
Once you have that perf data just run the following to get a summary of differences.
cargo benchcmp before.txt after.txt
It'll look something like the below and in this case shows a 10x speedup.
name before.txt ns/iter after.txt ns/iter diff ns/iter diff % speedup universe_ticks 862,931 82,187 -780,744 -90.48% x 10.50