92 lines
3.3 KiB
Groff
92 lines
3.3 KiB
Groff
|
.Dd $Mdocdate: September 22 2011 $
|
||
|
|
.Dt WORLDSTONE 1
|
||
|
|
.Os
|
||
|
|
.Sh NAME
|
||
|
|
.Nm worldstone
|
||
|
|
.Nd shell script to consistently execute benchmarks
|
||
|
|
.Sh SYNOPSIS
|
||
|
|
.Nm worldstone
|
||
|
|
.Op Fl s
|
||
|
|
.Op Fl n Ar iterations
|
||
|
|
.Op Fl c Ar command
|
||
|
|
.Op Fl p Ar command
|
||
|
|
.Op Fl t Ar tag
|
||
|
|
.Sh DESCRIPTION
|
||
|
|
The
|
||
|
|
.Nm
|
||
|
|
utility is a shell script and takes care of some of the
|
||
|
|
grunt work around benchmarking, in order to make it easier
|
||
|
|
to get consistent and comparable benchmark results. Its basic
|
||
|
|
operation is: execute the precommand, then execute and time
|
||
|
|
the command, and do this a set number of iterations, and record
|
||
|
|
the times in a logfile.
|
||
|
|
|
||
|
|
Its features are:
|
||
|
|
.Bl -tag -width Ds
|
||
|
|
.It -
|
||
|
|
It executes the precommand and command once without timing it
|
||
|
|
in order to mitigate cold cache effects.
|
||
|
|
.It -
|
||
|
|
It allows a precommand to run before the command, so that the initial
|
||
|
|
state can be set up by the precommand without it being part of the timing
|
||
|
|
(e.g. make clean).
|
||
|
|
.It -
|
||
|
|
It redirects the stdout and stderr to /dev/null so that lots of output
|
||
|
|
going over a network connection doesn't influence timing.
|
||
|
|
.It -
|
||
|
|
It does a sync before running the timed command, and makes sure a final
|
||
|
|
sync is part of the timed command, to make the i/o more consistent.
|
||
|
|
.It -
|
||
|
|
It logs the times of each iteration in an easy-to-parse logfile.
|
||
|
|
.It -
|
||
|
|
It tries to guess a sensible log file name based on the current git
|
||
|
|
branch in /usr/src.
|
||
|
|
.El
|
||
|
|
|
||
|
|
The options are as follows:
|
||
|
|
.Bl -tag -width Ds
|
||
|
|
.It Fl s
|
||
|
|
If set, perform statistical profiling by invoking
|
||
|
|
.Xr profile 1
|
||
|
|
and executing sprofalyze, suitable for feeding to sprofdiff. This
|
||
|
|
requires building world with SPROFILE.
|
||
|
|
.It Fl n Ar iterations
|
||
|
|
Set the number of iterations to perform, after the initial run.
|
||
|
|
The default is 5.
|
||
|
|
.It Fl c Ar command
|
||
|
|
Set the command to run to be timed. This is passed to sh -c, so shell constructs
|
||
|
|
like loops etc. are okay to do. Default: make all.
|
||
|
|
.It Fl p Ar command
|
||
|
|
Set the pre-command to run. This command gets run before the timed command in order
|
||
|
|
to make the timed command get a consistent state before it starts.
|
||
|
|
Default: make clean.
|
||
|
|
.It Fl t Ar tag
|
||
|
|
Use the given tag name to modify the logfile that the utility uses
|
||
|
|
to write its results in. The default is just 'time' plus the git branch you
|
||
|
|
are currently on in /usr/src. In order for this to be useful you have to make sure the
|
||
|
|
git branch you are on reflects the system you wish to benchmark of course.
|
||
|
|
The script checks /usr/src/.git even if you are outside the /usr/src hierarchy
|
||
|
|
(such as in pkgsrc).
|
||
|
|
.El
|
||
|
|
|
||
|
|
The script executes the commands the set number of iterations, redirecting stdout
|
||
|
|
and stderr to /dev/null, and records the timed results in the logfile tagged with
|
||
|
|
the given tag.
|
||
|
|
.Nm
|
||
|
|
executes
|
||
|
|
.Xr time 1
|
||
|
|
with the -C option, resulting in printing the 64-bit cpu cycle counter
|
||
|
|
for both HZ-independent high resolution and an easy way not to have to convert minutes
|
||
|
|
and seconds to seconds when parsing the results.
|
||
|
|
.Pp
|
||
|
|
You can feed the two separate logfiles directly to
|
||
|
|
.Xr ministat 1
|
||
|
|
to have it tell you the statistical properties of the two datasets, and judge whether
|
||
|
|
there is a statistically significant difference.
|
||
|
|
.Sh ENVIRONMENT
|
||
|
|
The default commands, i.e. make all and make clean, can be modified by supplying a MAKE
|
||
|
|
environment variable, so e.g. MAKE=bmake worldstone still does something sensible
|
||
|
|
by default in /usr/pkgsrc directories.
|
||
|
|
.Sh SEE ALSO
|
||
|
|
.Xr ministat 1
|