clojure - OutOfMemoryError: GC overhead limit exceeded Criterium -
i trying benchmark expression using criterium library. expression is
(vec (range 10000000))
to benchmark type
(criterium.core/bench (vec (range 10000000)))
and after while
outofmemoryerror gc overhead limit exceeded java.lang.long.valueof (long.java:840)
as have seen here means maximum size of heap (1 gb) not enough data fit , garbage collector tries free space unable so. however, microbenchmarking expression below doesn't produce error
(dotimes [i 60] (time (vec (range 10000000))))
by way, set 60 times because have seen here bench
macro 60 executions default.
the question why happening when using criterium.
edit: when starting fresh repl code below
{:max (.maxmemory (runtime/getruntime)), :total (.totalmemory (runtime/getruntime))}
outputs
{:max 922746880, :total 212860928}
after run (dotimes [i 60] (time (vec (range 10000000))))
or (criterium.core/bench (vec (range 10000000)))
it outputs
{:max 922746880, :total 922746880}
i able reproduce behavior using test:
;project.clj :profiles {:test {:jvm-opts ["-xms1024m" "-xmx1024m"]}} (:require [clojure.test :refer :all] [criterium.core :as ben]) (deftest ^:focused ben-test (is (ben/with-progress-reporting (ben/bench (vec (range 10000000))))))
the stack trace looks this:
estimating sampling overhead warming jit optimisations 10000000000 ... compilation occurred before 377618 iterations ... estimating execution count ... sampling ... final gc... checking gc... finding outliers ... bootstrapping ... checking outlier significance warming jit optimisations 10000000000 ... compilation occurred before 1 iterations criterium.core$execute_expr_core_timed_part$fn__40395.invoke (core.clj:370) criterium.core$execute_expr_core_timed_part.invokestatic (core.clj:366) criterium.core$execute_expr_core_timed_part.invoke (core.clj:345) criterium.core$execute_expr.invokestatic (core.clj:378) criterium.core$execute_expr.invoke (core.clj:374) criterium.core$warmup_for_jit.invokestatic (core.clj:428) criterium.core$warmup_for_jit.invoke (core.clj:396) criterium.core$run_benchmark.invokestatic (core.clj:479) criterium.core$run_benchmark.invoke (core.clj:470) criterium.core$benchmark_star_.invokestatic (core.clj:826) criterium.core$benchmark_star_.invoke (core.clj:812)
we can see here error occurs in jit-warning-up step. interesting point function execute-expr-core-timed-part (core.clj:345). functions performs expression (vec (range 10000000)) n times , saves returned value every time so-called mutable place. hypothesis have memory leak here.
(time-body (loop [i (long (dec n)) v (f)] ==> (set-place mutable-place v**) (if (pos? i) (recur (unchecked-dec i) (f)) v)))
Comments
Post a Comment