o
scalqa

Util.Benchmark

object Benchmark

Benchmark is a basic comparison testing tool

JVM technology makes it difficult to get absolute performance data

Benchmark allows to run several targets side by side, comparing their performance against each other

Let's benchmark 'append' performance for List, Vector and Buffer

import scala.collection.mutable.Buffer

scalqa.Util.Benchmark(
  ("List  ", () => { var v: List[Int]   = Nil;          for (i <- 1 to 1000) v = i :: v; v.size }),
  ("Vector", () => { var v: Vector[Int] = Vector.empty; for (i <- 1 to 1000) v = v :+ i; v.size }),
  ("Buffer", () => { val v: Buffer[Int] = Buffer.empty; for (i <- 1 to 1000) v += i;     v.size }))

// Output

Final Result.  Total Duration ~ 12 sec
--- ------ ------- --- ------ --- -----------------
Num Name   Ops/Sec %   Memory %   Control Value Avg
--- ------ ------- --- ------ --- -----------------
1   List   60.9k   53  38.0k  17  1000
2   Vector 13.5k   11  217.1k 100 1000
3   Buffer 113.2k  100 20.4k  9   1000
--- ------ ------- --- ------ --- -----------------

The results present three values:

  • 'Ops/Sec' is the number of times the target executed per second
  • 'Memory' is the average memory increase for one execution
  • 'Control Value Avg' is the average of target returns for validity check. Generally, all targets should arrive to the same value in different ways

Note. Absolute values are less informative than following percentages of the largest value

From the above example, the following conclusions can be made:

  • List is twice and Vector is ten times slower than Buffer
  • Vector is seriously heavier on memory
Ordering
  1. Alphabetic
Inherited
  1. Benchmark
  2. scala.AnyRef
  3. scala.Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Variable

  1. var verbose: Boolean

    Global verbose switch

    Global verbose switch

    Default value is true

    If false, only final result will be presented

    Definition Classes
    Benchmark

Method

  1. def apply[A](targets: (String, () ⇒ A)*)(implicit arg0: Numeric[A]): Unit

    Run with defaults

    Run with defaults

    Runs 4 trials 3 seconds each

    All trials are accumulated into final result

    targets

    a list of tuples, each representing target name and target function to run. The function return will be converted to Long and average value will be displayed in the results

    Definition Classes
    Benchmark
    Note

    All targets must return same numeric type (Int, Long, Double, etc.)

  2. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  3. def clone(): AnyRef
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  4. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  5. def equals(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  6. def finalize(): Unit
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  7. def hashCode(): Int
    Definition Classes
    AnyRef → Any
  8. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  9. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  10. final def notify(): Unit
    Definition Classes
    AnyRef
  11. final def notifyAll(): Unit
    Definition Classes
    AnyRef
  12. final def synchronized[T0](arg0: ⇒ T0): T0
    Definition Classes
    AnyRef
  13. def toString(): String
    Definition Classes
    AnyRef → Any
  14. def trials[A](trialCount: Int, trialLengthMillis: Long, targets: (String, () ⇒ A)*)(implicit arg0: Numeric[A]): Unit

    Run specified number of trials

    Run specified number of trials

    Benchmarking is split in several trials, which are equal sub-tests, and their results should be in the same ball park

    If some trial appears to be off the line, then something unusual happened, and it is a good reason to re-run the entire test

    All trials are accumulated into final result

    trialCount

    - number of trials to run

    trialLengthMillis

    - length of each trial

    targets

    - a list of tuples, each representing target name and target function to run. The function return will be converted to Long and average value will be displayed in the results

    Definition Classes
    Benchmark
    Note

    All targets must return same numeric type (Int, Long, Double, etc.)

  15. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  16. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  17. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

Operator

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
Linear Supertypes
AnyRef, Any
Source: Benchmark.scala