Quick Start
===========

This tutorial will walk you through creating a simple project that
uses PHPBench as a dependency.

Create your project
-------------------

Create a directory for the tutorial:

.. code-block:: bash

    $ mkdir phpbench-tutorial

And create the following Composer_ file within it:

.. code-block:: javascript

    {
        "name": "acme/phpbench-test",
        "require-dev": {
            "phpbench/phpbench": "^1.0"
        },
        "autoload": {
            "psr-4": {
                "Acme\\": "src/"
                "Acme\\Tests\\": "tests/"
            }
        }
    }

Now perform a Composer install:

.. code-block:: bash

    $ composer install

PHPBench should now be installed. Now create two directories, ``benchmarks``
and ``src`` which we will need further on:

.. code-block:: bash

    $ mkdir -p tests/Benchmark
    $ mkdir src

PHPBench configuration
----------------------

In order for PHPBench to be able to autoload files from your library, you
should specify the path to your bootstrap file (i.e. ``vendor/autoload.php``).
This can be done in the PHPBench :doc:`configuration <configuration>`.

Create a ``phpbench.json`` file in the projects root directory:

.. code-block:: javascript

    {
        "bootstrap": "vendor/autoload.php"
    }

.. note::

    PHPBench does not **require** a bootstrap (or a configuration file for
    that matter). You may omit it if you do not need autoloading, or you want
    to include files manually.

.. warning::

    Some PHP extensions such as Xdebug will affect the performance of your
    benchmark subjects and you may want to disable them, see :ref:`Disabling
    the PHP INI file <configuration_disable_php_ini>`.

Creating and running a benchmark
--------------------------------

You will need some code to benchmark, create a simple class in ``lib`` which
consumes *time itself*:

.. code-block:: php

    <?php

    namespace Acme;

    class TimeConsumer
    {
        public function consume()
        {
            usleep(100);
        }
    }


In order to benchmark your code you will need to execute that code within
a method of a benchmarking class. By default the class name **must**
have the ``Bench`` suffix and each benchmark method must be prefixed
with ``bench``.

Create the following class in file ``tests/Benchmark/TimeConsumerBench.php``:

.. code-block:: php

    <?php

    use Acme\TimeConsumer;

    class TimeConsumerBench
    {
        public function benchConsume()
        {
           $consumer = new TimeConsumer();
           $consumer->consume();
        }
    }

Now you can execute the benchmark as follows:

.. code-block:: bash

    $ ./vendor/bin/phpbench run benchmarks/TimeConsumerBench.php --report=default

And you should see some output similar to the following:

.. code-block:: bash

    Running benchmarks.

    \TimeConsumerBench

        benchConsume                  I0 P0         [μ Mo]/r: 173.00μs   [μSD μRSD]/r: 0.00μs 0.00%

    1 subjects, 1 iterations, 1 revs, 0 rejects
    ⅀T: 173μs μSD/r 0.00μs μRSD/r: 0.00%
    min [mean mode] max: 173.00 [173.00 1732.00] 173.00 (μs/r)

    +-------------------+---------------+-------+--------+------+------+-----+----------+------------+---------+-------+
    | benchmark         | subject       | group | params | revs | iter | rej | mem      | time       | z-score | diff  |
    +-------------------+---------------+-------+--------+------+------+-----+----------+------------+---------+-------+
    | TimeConsumerBench | benchConsume  |       | []     | 1    | 0    | 0   | 265,936b | 173.0000μs | 0.00σ   | 1.00x |
    +-------------------+---------------+-------+--------+------+------+-----+----------+------------+---------+-------+

The code was only executed once (as indicated by the ``revs`` column). To
achieve a better measurement increase the revolutions:

.. code-block:: php

    <?php

    // ...

    class TimeConsumerBench
    {
        /**
         * @Revs(1000)
         */
        public function benchConsume()
        {
            // ...
        }
    }

:ref:`Revolutions <revolutions>` in PHPBench represent the number of times
that the code is executed consecutively within a single measurement.

Currently we only execute the benchmark subject a single time, to build
confidence in the result increase the number of :ref:`iterations <iterations>`
using the ``@Iterations`` annotation:

.. code-block:: php

    <?php

    // ...

    class TimeConsumerBench
    {
        /**
         * @Revs(1000)
         * @Iterations(5)
         */
        public function benchConsume()
        {
            // ...
        }
    }

Now when you run the report you should see that it contains 5 rows. One
measurement for each iteration, and each iteration executed the code 1000
times.

.. note::

    You can override the number of iterations and revolutions on the CLI using
    the ``--iterations`` and ``--revs`` options.

At this point it would be better for you to use the :ref:`aggregate
<report_aggregate>` report
rather than :ref:`default <report_default>`:

.. code-block:: bash

    $ php vendor/bin/phpbench run benchmarks/TimeConsumerBench.php --report=aggregate

Increase Stability
------------------

You will see the columns `stdev` and `rstdev`. `stdev` is the `standard
deviation`_ of the set of iterations and `rstdev` is `relative standard
deviation`_.

Stability can be inferred from `rstdev`, with 0% being the best and anything
about 2% should be treated as suspicious.

To increase stability you can use the ``--retry-threshold`` to automatically
:ref:`repeat the iterations <retry_threshold>` until the `diff` (the
percentage difference from the lowest measurement) fits within a given
threshold:

.. note:

    You can see the `diff` value for each iteration in the `default` report.

.. code-block:: bash

    $ php vendor/bin/phpbench run benchmarks/TimeConsumerBench.php --report=aggregate --retry-threshold=5

.. warning::

    Depending on system stability, the lower the ``retry-threshold`` the
    longer it will take to resolve a stable set of results.

Customize Reports
-----------------

PHPBench also allows you to customize reports on the command line, try the
following:

.. code-block:: bash

    $ ./vendor/bin/phpbench run benchmarks/TimeConsumerBench.php --report='{"extends": "aggregate", "cols": ["subject", "mode"]}'

Above we configure a new report which extends the :ref:`default
<report_default>` report that we have already used, but we use only the
``subject`` and ``mode`` columns.  A full list of all the options for the
default reports can be found in the :doc:`report-generators` chapter.

Configuration
-------------

To finish off, add the path and new report to the configuration file:

.. code-block:: javascript

    {
        ...
        "path": "benchmarks",
        "reports": {
            "consumation_of_time": {
                "extends": "default",
                "title": "The Consumation of Time",
                "description": "Benchmark how long it takes to consume time",
                "cols": [ "subject", "mode" ]
            }
        }
    }

Above you tell PHPBench where the benchmarks are located and you define a new
report, ``consumation_of_time``, with a title, description and sort order.

We can now run the new report:

.. code-block:: bash

    $ php vendor/bin/phpbench run --report=consumation_of_time

.. note::

    Note that we did not specify the path to the benchmark file, by default all
    benchmarks under the given or configured path will be executed.

Summary
-------

In this tutorial you learnt to 

- :doc:`Configure <configuration>` PHPBench for a project
- Create a benchmarking class
- Use :ref:`revolutions <revolutions>` and :ref:`iterations <iterations>` to more accurately profile your code
- Increase stability with the :ref:`retry threshold <retry_threshold>`
- Use :doc:`reports <reports>`

.. _composer: http://getcomposer.org
.. _Relative standard deviation: https://en.wikipedia.org/wiki/Coefficient_of_variation
.. _standard deviation: https://en.wikipedia.org/wiki/Standard_deviation