blob: 3e038c9d34ac92573847c30fe7fbf5801ced0e0c [file] [log] [blame]
2013-05-03 Cosmin Truta <>
SunSpider 1.0: 3d-morph: use epsilon to check result
Reviewed by George Staikos.
SunSpider/3d-morph should be more lenient in validating its results, since they depend on Math.sin, which isn't formally specified.
* tests/sunspider-1.0/3d-morph.js:
2013-04-19 Filip Pizlo <>
SunSpider/string-tagcloud should be more lenient in validating its results, since they depend on Math.log, which isn't formally specified
Reviewed by Michael Saboff.
* tests/sunspider-1.0/string-tagcloud.js:
2013-04-18 Filip Pizlo <>
Whenever it is cheap and non-invasive, SunSpider tests should validate their results to ensure that the browser runs them correctly
Reviewed by Geoffrey Garen.
This adds low-overhead checking of the results of each SunSpider tests. This is not
meant to be an exhaustive test that everything was executed correctly, but rather a
simple sanity check that will catch glaring mistakes. The philosophy here is that we're
not trying to prevent all forms of cheating, or that we're trying to prove the
browser's correctness. Moreover, these checks are meant to incur minimal overhead;
they currently clock in at <2% slow-down in SunSpider 1.0.
The test expectations were generated using the LLInt on command-line, and I've verified
that Firefox 20, Chrome 26, IE 10, Safari 6, and ToT with all of the JITs agree. Note
that some tests cannot be precisely validated because they use Math functions, which
ECMAScript chooses to not formally specify - those functions like sin() and friends are
allowed to return implementation-dependent results. Also some tests cannot be validated
at all because their behavior is either intentionally random or is timezone-dependent.
But 23 out of 26 tests now have some kind of validation.
I've updated the harnesses to show erroneous runs when displaying results.
* resources/TEMPLATE.html:
* resources/driver-TEMPLATE.html:
* resources/sunspider-analyze-results.js:
* resources/sunspider-compare-results.js:
* resources/sunspider-standalone-driver.js:
* tests/sunspider-1.0/3d-cube.js:
* tests/sunspider-1.0/3d-morph.js:
* tests/sunspider-1.0/3d-raytrace.js:
* tests/sunspider-1.0/access-binary-trees.js:
* tests/sunspider-1.0/access-fannkuch.js:
* tests/sunspider-1.0/access-nbody.js:
* tests/sunspider-1.0/access-nsieve.js:
* tests/sunspider-1.0/bitops-3bit-bits-in-byte.js:
* tests/sunspider-1.0/bitops-bits-in-byte.js:
* tests/sunspider-1.0/bitops-bitwise-and.js:
* tests/sunspider-1.0/bitops-nsieve-bits.js:
* tests/sunspider-1.0/controlflow-recursive.js:
* tests/sunspider-1.0/crypto-aes.js:
* tests/sunspider-1.0/crypto-md5.js:
* tests/sunspider-1.0/crypto-sha1.js:
* tests/sunspider-1.0/date-format-tofte.js:
* tests/sunspider-1.0/date-format-xparb.js:
* tests/sunspider-1.0/math-cordic.js:
* tests/sunspider-1.0/math-partial-sums.js:
* tests/sunspider-1.0/math-spectral-norm.js:
* tests/sunspider-1.0/regexp-dna.js:
* tests/sunspider-1.0/string-base64.js:
* tests/sunspider-1.0/string-fasta.js:
* tests/sunspider-1.0/string-tagcloud.js:
* tests/sunspider-1.0/string-unpack-code.js:
* tests/sunspider-1.0/string-validate-input.js:
2013-04-11 Filip Pizlo <>
Get rid of the 10ms delay between test executions
Reviewed by Darin Adler and Geoffrey Garen.
More details on this in the bug. Long story short, I don't know why the 10ms delay is
there, and it appears to be the root of some noisyness and unreliability in the results.
Removing it makes results more consistent, which is a good thing.
* resources/driver-TEMPLATE.html:
2013-01-16 Eric Seidel <>
Remove --shark* support from sunspider/run-sunspider now that Shark is dead
Reviewed by Darin Adler.
I tried to use --instruments, but that also seems broken (in the same way before/after this patch).
* sunspider:
2012-08-02 Filip Pizlo <>
Unreviewed, revert debugging stuff I landed in
by accident.
* tests/v8-v6/v8-crypto.js:
2012-03-11 Maciej Stachowiak <>
Add --instruments option to sunspider to profile with Instruments
Reviewed by Filip Pizlo.
* resources/TimeProfile20us.tracetemplate: Added.
* sunspider:
2011-11-15 Andy Wingo <>
Extend sunspider driver to be able to run kraken
Reviewed by Filip Pizlo.
* resources/sunspider-standalone-driver.js: Try to load a -data
file. If that succeeds, we have a kraken-like test, so we time
the test using `load'. Otherwise fall back to using `run'.
2011-08-11 Mark Rowe <>
Fix sunspider-compare-results to default to the same suite version as the sunspider script.
Rubber-stamped by Maciej Stachowiak.
* sunspider-compare-results:
2011-07-02 Maciej Stachowiak <>
Update SunSpider versioning to 1.0
Reviewed by Dan Bernstein.
* hosted/sunspider.html:
* hosted/versions.html:
* resources/driver-TEMPLATE.html:
* resources/results-TEMPLATE.html:
* sunspider:
2011-07-02 Maciej Stachowiak <>
Sunspider 0.9.1 harness doesn't actually close() its test documents
Reviewed by Daniel Bates.
* hosted/versions.html:
* resources/driver-TEMPLATE.html:
2011-07-02 Maciej Stachowiak <>
Not Reviewed.
Fix obviou typo in previous commit.
* tests/sunspider-1.0/bitops-bitwise-and.js:
2011-07-02 Maciej Stachowiak <>
Reviewed by Anders Carlsson.
SunSpider: all four bitops benchmarks can be replaced with NOP
* tests/sunspider-1.0/bitops-3bit-bits-in-byte.js:
(TimeFunc): Save the result.
* tests/sunspider-1.0/bitops-bits-in-byte.js:
(TimeFunc): Save the result.
* tests/sunspider-1.0/bitops-bitwise-and.js: More explicitly
save the results.
* tests/sunspider-1.0/bitops-nsieve-bits.js:
(sieve): Save the result.
2011-07-02 Maciej Stachowiak <>
SunSpider: string-validate-input.js uses an undeclared variable named 'name', which is a DOM API
Reviewed by Dan Bernstein.
* tests/sunspider-1.0/string-validate-input.js:
(doTest): Rename the variable to username
2011-07-02 Maciej Stachowiak <>
Reviewed by Darin Adler.
SunSpider: The main function of math-cordic is dead code
Try to prevent math-cordic from being eliminated as dead code or
compiled down to a constant.
* tests/sunspider-1.0/math-cordic.js:
2011-06-30 Maciej Stachowiak <>
Reviewed by Adam Barth.
Create sunspider-1.0 directory in preparation for fixing a bunch of bugs
* make-hosted: Learn about the new directory.
* tests/sunspider-1.0: Copied from PerformanceTests/SunSpider/tests/sunspider-0.9.1.
2010-11-29 Geoffrey Garen <>
Reviewed by Gavin Barraclough.
Improved accuracy of command-line SunSpider.
* resources/sunspider-standalone-driver.js: Use the returned number of
elapsed milliseconds from "run" and "checkSyntax" instead of measuring
ourselves, for slightly more accurate numbers.
2010-09-16 Gavin Barraclough <>
Reviewed by Mark Rowe.
Modify the SunSpider harness to allow a path to be passed to the --suite
argument, supress the normal prepend-"tests/" behaviour for suite names
containing a '/'.
Also wrap the code in an anonymous function to move variables out of global
scope (currently the harness cannot handle running tests the overwrite the
variable 'j').
* resources/sunspider-standalone-driver.js:
* sunspider:
2010-08-27 Michael Saboff <>
Reviewed by Stephanie Lewis.
Added version 5 and version 6 of V8 tests with modifications similar
to the version 4 tests so they'll work with the SunSpider test harness.
Note that these files are external source and therefore don't conform
to WebKit coding standards.
* tests/v8-v5: Added.
* tests/v8-v5/LIST: Added.
* tests/v8-v5/v8-crypto.js: Added.
* tests/v8-v5/v8-deltablue.js: Added.
* tests/v8-v5/v8-earley-boyer.js: Added.
* tests/v8-v5/v8-raytrace.js: Added.
* tests/v8-v5/v8-regexp.js: Added.
* tests/v8-v5/v8-richards.js: Added.
* tests/v8-v5/v8-splay.js: Added.
* tests/v8-v6: Added.
* tests/v8-v6/LIST: Added.
* tests/v8-v6/v8-crypto.js: Added.
* tests/v8-v6/v8-deltablue.js: Added.
* tests/v8-v6/v8-earley-boyer.js: Added.
* tests/v8-v6/v8-raytrace.js: Added.
* tests/v8-v6/v8-regexp.js: Added.
* tests/v8-v6/v8-richards.js: Added.
* tests/v8-v6/v8-splay.js: Added.
2010-07-15 Stephanie Lewis <>
Reviewed by Geoff Garen.
Output location of results file.
* sunspider:
2010-05-17 Maciej Stachowiak <>
Reviewed by Mark Rowe.
Browser-hosted version of SunSpider runs tests twice as many times as advertised
* make-hosted: Make sure to reset test list after every test suite.
2010-05-07 Adam Roben <>
Fix typo in sunspider-compare-results.js that caused incorrect
two-sample t statistics to be computed
Fixes <> SunSpider computes incorrect
two-sample t statistics
Reviewed by Darin Adler.
* resources/sunspider-compare-results.js:
(sunspiderCompareResults.resultLine): Fixed the calculation of the t
variable to match the definition of a two-sample t statistic. Looks
like this was just a simple typo.
2010-04-05 Darin Adler <>
Updated the ignore property for changes to the contents here.
* .: Modified property svn:ignore.
2010-02-20 Maciej Stachowiak <>
Reviewed by Adam Barth.
Add sunspider-0.9.1 to the site in preparation for announcing it.
* hosted/sunspider.css: Remove tabs, which I noticed where present while
copying to the WebKitSite directory.
2010-01-11 Oliver Hunt <>
Reviewed by Darin Adler.
Sunspider parse-only tests don't run
Fix parse-only test list
* tests/parse-only/LIST:
2009-12-15 Maciej Stachowiak <>
Reviewed by Adele Peterson.
Make SunSpider version more prominent in the title
* hosted/sunspider.html:
* hosted/versions.html:
* resources/driver-TEMPLATE.html:
* resources/results-TEMPLATE.html:
2009-12-15 Maciej Stachowiak <>
Reviewed by Darin Adler.
Make SunSpider refuse to compare results across content versions
The results URL now includes a version indicator, and compare mode
will refuse to compare between versions.
* resources/driver-TEMPLATE.html:
* resources/results-TEMPLATE.html:
2009-12-14 Maciej Stachowiak <>
Reviewed by Darin Adler.
Some Browser-hosted SunSpider files are not valid HTML5
Made various fixes to validate as HTML5 with no warnings (mainly removing bogus
close tags and adding a meta charset declaration).
* hosted/sunspider.html:
* hosted/versions.html:
* resources/TEMPLATE.html:
* resources/driver-TEMPLATE.html:
* resources/results-TEMPLATE.html:
2009-12-14 Maciej Stachowiak <>
Reviewed by Eric Seidel.
Make sunspider-0.9.1 the default content set (both command-line and hosted)
* hosted/sunspider.html: Change default suite link.
* sunspider: Change default suite.
* sunspider-compare-results: Likewise.
2009-12-14 Maciej Stachowiak <>
Reviewed by Sam Weinig.
in-browser SunSpider suffers excessive penalty under power management
I have made a few changes to address this:
1) Change how browser-hosted SunSpider loads tests - preload the content and write it in with
document.write to reduce triggering of progress UI.
2) Reduce time between tests to 10ms from 500ms, so that power management doesn't keep the CPU
stepped all the way down the whole time.
3) Run the test cycle an extra time for warmup before the runs that count.
* hosted/sunspider-record-result.js: Removed.
* make-hosted:
* resources/TEMPLATE.html:
* resources/driver-TEMPLATE.html:
2009-12-13 Maciej Stachowiak <>
Reviewed by Gavin Barraclough.
SunSpider/tests/string-base64.js does not compute a valid base64 encoded string
Based on a patch by Eric Seidel.
Fix the base64 computation to actually compute correct results. The impact on runtime of
the test is pretty small, but noticeable for some browsers. But at least it's not
doing a wrong and meaningless computation any more.
* tests/sunspider-0.9.1/string-base64.js:
2009-12-13 Maciej Stachowiak <>
Fixing commit error...
I accidentally committed my last patch in a form that broke Web-hosted SunSpider. Fixing.
* resources/driver-TEMPLATE.html:
2009-12-13 Maciej Stachowiak <>
Reviewed by Sam Weinig.
Use JSON.parse instead of eval for Web-hosted SunSpider results processing
* hosted/json2.js: Added. Incorporated from
* resources/driver-TEMPLATE.html: Fix an HTML validation bug I noticed.
* resources/results-TEMPLATE.html: Use JSON.parse instead of eval to process
2009-12-12 Maciej Stachowiak <>
Reviewed by Oliver Hunt.
Enable Web-hosted version of SunSpider to handle multiple versions
* make-hosted: Now generate the hosted version to be able to run both the
0.9 and the 0.9.1 test suites.
* hosted: Modified property svn:ignore.
* hosted/sunspider-driver.html: Removed.
* hosted/sunspider-results.html: Removed.
* hosted/sunspider.html:
* hosted/versions.html: Added.
* resources/TEMPLATE.html:
* resources/driver-TEMPLATE.html: Copied from hosted/sunspider-driver.html.
* resources/driver-TEMPLATE.html: Copied from hosted/sunspider-results.html.
2009-12-12 Maciej Stachowiak <>
Reviewed by Oliver Hunt.
Give command-line sunspider the ability to handle multiple suites and versions
Each suite/version's set of tests are now in a separate subdirectory of tests/, with a file named LIST
enumerating the tests. Current suites are:
sunspider-0.9 - riginal SunSpider content
sunspider-0.9.1 - SunSpider suite with a few tests fixed (and a few more to come)
ubench - "ubench" microbenchmark suite
v8-v4 - version 4 of the v8 benchmark
parse-only - some parser tests
Sunspider can now be run with the --suite= parameter to select a
suite. The default is --suite=sunspider-0.9.
* .: Modified property svn:ignore to ignore new results directories.
* make-hosted: Generate based on sunspider-0.9 suite.
* resources/TEMPLATE.html: Fix an HTML compliance problem.
* resources/sunspider-standalone-driver.js: Become aware of suite names.
* sunspider: Modified to handle multiple suites as described above.
* sunspider-compare-results: ditto
* tests/LIST: Moved to tests/sunspider-0.9/ and tests/sunspider-0.9.1/
* tests/LIST-PARSE-ONLY: Moved to parse-only/LIST
* tests/LIST-UBENCH: Removed.
* tests/LIST-V8: Removed.
* tests/3d-cube.js: Moved to tests/sunspider-0.9 and tests/sunspider-0.9.1
* tests/3d-morph.js: ditto
* tests/3d-raytrace.js: ditto
* tests/access-binary-trees.js: ditto
* tests/access-fannkuch.js: ditto
* tests/access-nbody.js: ditto
* tests/access-nsieve.js: ditto
* tests/bitops-3bit-bits-in-byte.js: ditto
* tests/bitops-bits-in-byte.js: ditto
* tests/bitops-bitwise-and.js: ditto
* tests/bitops-nsieve-bits.js: ditto
* tests/controlflow-recursive.js: ditto
* tests/crypto-aes.js: ditto
* tests/crypto-md5.js: ditto
* tests/crypto-sha1.js: ditto
* tests/date-format-tofte.js: ditto
* tests/date-format-xparb.js: ditto
* tests/math-cordic.js: ditto
* tests/math-partial-sums.js: ditto
* tests/math-spectral-norm.js: ditto
* tests/regexp-dna.js: ditto
* tests/string-base64.js: ditto
* tests/string-fasta.js: ditto
* tests/string-tagcloud.js: ditto
* tests/string-unpack-code.js: ditto
* tests/string-validate-input.js: ditto
* tests/sunspider-0.9/: Added
* tests/sunspider-0.9/*.js: Moved from one level up
* tests/sunspider-0.9.1: Added.
* tests/sunspider-0.9.1/*.js: Moved from one level up.
* tests/v8-crypto.js: Moved to v8-v4/ directory.
* tests/v8-deltablue.js: ditto
* tests/v8-earley-boyer.js: ditto
* tests/v8-raytrace.js: ditto
* tests/v8-regexp.js: ditto
* tests/v8-richards.js: ditto
* tests/v8-splay.js: ditto
* tests/v8-v4: Added.
2009-09-30 Csaba Osztrogonac <>
Reviewed by Darin Adler.
Platform specific null device replaced with a platform independent.
* sunspider: Using File::Spec->devnull() instead of /dev/null and NUL.
2009-09-21 Csaba Osztrogonac <>
Reviewed by Maciej Stachowiak.
[Qt] Make sunspider script work on Windows platform.
* sunspider:
Using platform specific null device instead of /dev/null.
2009-06-19 Adam Treat <>
Reviewed by Oliver Hunt.
Currently the SunSpider test driver lacks an option to run a test suite that
will test JavaScriptCore parsing performance only. This patch adds just such
a test suite and option to SunSpider as well as the jsc test shell. I've included
three large javascript source files found in the wild: jquery, mootools and prototype.
Combined with the concatenation of all three, these form a new testsuite to measure
and test pure JavaScriptCore parsing performance.
* resources/sunspider-standalone-driver.js:
Load the js with parse-only flag if it is part of the parse-only suite.
* sunspider:
Add the parse-only suite to the list of options.
* sunspider-compare-results:
Add the parse-only suite to the list of options.
* tests/LIST-PARSE-ONLY: Added.
* tests/parse-only/concat-jquery-mootools-prototype.js: Added.
A very large concatenation of the scripts found below. This js takes ~30ms to parse on my machine.
* tests/parse-only/jquery-1.3.2.js: Added.
Latest version from
* tests/parse-only/mootools-1.2.2-core-nc.js: Added.
Latest version from
* tests/parse-only/prototype- Added.
Latest version from
2009-06-13 Steve Falkenburg <>
Update V8 benchmark to version 4.
Reviewed by Cameron Zwarich.
* tests/LIST-V8:
* tests/v8-crypto.js:
* tests/v8-deltablue.js:
* tests/v8-earley-boyer.js:
* tests/v8-raytrace.js:
* tests/v8-regexp.js: Added.
* tests/v8-richards.js:
* tests/v8-splay.js: Added.
2009-05-26 Olivier DOLE <>
Reviewed by Maciej.
Store sunspider test suite results in a specific directory for each
different type of benchmark (sunspider, V8 or ubench).
* sunspider:
* sunspider-compare-results:
2009-05-12 Steve Falkenburg <>
Build fix.
* SunSpider.make:
2009-05-11 Steve Falkenburg <>
Include SunSpider tests in production builds.
Reviewed by Adam Roben.
* SunSpider.make: Added.
* xcopy.excludes: Added.
2009-03-04 Oliver Hunt <>
Added a few more tests to ubench
RS=Gavin Barraclough
Adding a couple of test to cover too few and too many arguments. This
is needed to prevent regressions in all call paths from my current refactoring.
* tests/LIST-UBENCH:
* tests/ubench/function-correct-args.js: Added.
* tests/ubench/function-excess-args.js: Added.
2008-10-30 Zoltan Horvath <>
Fix a bug where when V8's or UBENCH's test suite list file is missing
the error message is suggesting that SunSpider's list file is missing.
Reviewed by Sam Weinig.
* sunspider:
(loadTestsList): Fix the error message to use the right list name.
2008-10-07 David Hyatt <>
Remove global substitution from SunSpider regexp test, since only Firefox was honoring it and
we want all browsers to be doing the same thing on the test.
Reviewed by Maciej
* tests/regexp-dna.js:
2008-09-09 Cameron Zwarich <>
Rubber-stamped by Maciej Stachowiak.
The bleeding edge branch of V8 now supports passing fileanmes after
'-f', so the SunSpider script should be changed to do this again.
This allows all 3 major JavaScript engines to be used with the
unmodified SunSpider script.
* sunspider:
2008-09-04 Cameron Zwarich <>
Reviewed by Maciej Stachowiak.
Add a '--args' option to SunSpider, so that we can pass '-j' to TraceMonkey
and '--expose-gc' to V8. Also, rename the '--v8' option to '--v8-suite' and
correct the punctuation in its documentation.
* sunspider:
2008-09-04 Cameron Zwarich <>
Reviewed by Oliver Hunt.
Bug 20616: Incorporate V8 benchmarks in testing
This is a first attempt at running the V8 benchmarks within SunSpider,
with the --v8 option. The number of iterations is fixed for each
benchmark, and was chosen to make each of the benchmarks run between
500 ms and 600 ms in V8 on my machine, a 2.16 GHz MacBook Pro.
* sunspider:
* tests/LIST-V8: Added.
* tests/v8-crypto.js: Added.
* tests/v8-deltablue.js: Added.
* tests/v8-earley-boyer.js: Added.
* tests/v8-raytrace.js: Added.
* tests/v8-richards.js: Added.
2008-09-02 Cameron Zwarich <>
Not reviewed.
Remove a local change that was accidentally committed in r36034.
* resources/sunspider-standalone-driver.js:
2008-09-02 Cameron Zwarich <>
Rubber-stamped by Oliver Hunt.
Make SunSpider work with JavaScript shells that don't take filenames
after a '-f' argument.
* sunspider:
2008-06-07 Cameron Zwarich <>
Reviewed by Timothy.
Remove the --squirrelfish option from SunSpider, as it is no longer needed.
* sunspider:
* tests/LIST-SQUIRRELFISH: Removed.
2008-06-03 Alexey Proskuryakov <>
Rubber-stamped by Mark Rowe.
Roll out r31215 - with Mac OS X 10.5.3, there is no need to pause update daemon any more.
* sunspider:
2008-04-15 Maciej Stachowiak <>
Rubber stamped by Oliver.
- add newly working tests to squirrelfish list
* tests/LIST-SQUIRRELFISH: Add 3d-raytrace and string-unpack-code
2008-04-14 Maciej Stachowiak <>
Rubber stamped by Oliver.
- move ubench tests properly.
* tests/ubench: Added.
* tests/ubench/function-closure.js: Added.
* tests/ubench/function-empty.js: Added.
* tests/ubench/function-missing-args.js: Added.
* tests/ubench/function-sum.js: Added.
* tests/ubench/loop-empty-resolve.js: Added.
* tests/ubench/loop-empty.js: Added.
* tests/ubench/loop-sum.js: Added.
2008-04-14 Maciej Stachowiak <>
Rubber stamped by Oliver.
* sunspider: Don't force --runs=1 in squirrelfish mode any more.
2008-04-14 Maciej Stachowiak <>
Reviewed by Oliver.
- make --squirrelfish mode use the real harness, but a reduced set of tests
Also add a new ubench mode which runs the older squirrelfish microbenchmarks.
* sunspider:
* tests/LIST-UBENCH: Added.
* tests/squirrelfish: Moved to tests/ubench
2008-04-14 Geoffrey Garen <>
Reviewed by Maciej Stachowiak.
Removed an unused test.
* tests/squirrelfish/loop-resolve.js: Removed.
2008-04-10 Maciej Stachowiak <>
Reviewed by Oliver.
Add newly runnable tests to --squirrelfish mode.
2008-04-06 Geoffrey Garen <>
Reviewed by Maciej Stachowiak.
A little more love for --squirrelfish mode:
Fixed a misplaced sort. Results properly sort now.
Added a test list just for squirrelfish, and updated it to include
all currently passing SunSpider tests.
Fixed the pruning regexp to match 3d-morph.js.
* sunspider:
2008-03-26 Geoffrey Garen <>
Reviewed by Oliver Hunt.
--squirrelfish mode: pared down tests for squirrelfish to chew on.
* sunspider:
* tests/LIST:
* tests/squirrelfish: Added.
* tests/squirrelfish/function-closure.js: Added.
* tests/squirrelfish/function-empty.js: Added.
* tests/squirrelfish/function-missing-args.js: Added.
* tests/squirrelfish/function-sum.js: Added.
* tests/squirrelfish/loop-empty-resolve.js: Added.
* tests/squirrelfish/loop-empty.js: Added.
* tests/squirrelfish/loop-resolve.js: Added.
* tests/squirrelfish/loop-sum.js: Added.
2008-03-21 Alexey Proskuryakov <>
Reviewed by Geoff.
* sunspider: Pause update daemon on Darwin to make results reliable
again (workaround for <rdar://problem/5811127>).
2008-02-07 Eric Seidel <>
Reviewed by Sam.
* sunspider: add an error message when ./tests/LIST is not found
2007-12-29 Alp Toker <>
Fix typo.
* hosted/sunspider.html:
2007-12-29 Alp Toker <>
Reviewed by Maciej.
Provide the current URL in the SunSpider results page. Useful for
copying benchmark results out of browsers without a location bar or
where the location bar can't handle long URLs.
* hosted/sunspider-results.html:
2007-12-20 Eric Seidel <>
Reviewed by mjs.
* sunspider: increase default number of runs for more accurate numbers
2007-12-18 Mark Rowe <>
Fix typo.
* hosted/sunspider.html:
2007-12-18 Maciej Stachowiak <>
Reviewed by Eric.
- give an extra digit of precision for small differences.
* resources/sunspider-compare-results.js:
2007-12-17 Maciej Stachowiak <>
Reviewed by Eric.
- work around IE layout bugs
* hosted/sunspider.css:
2007-12-17 Maciej Stachowiak <>
Reviewed by Mark.
- fix compare feature
* hosted/sunspider-results.html:
2007-12-17 Maciej Stachowiak <>
Reviewed by Mark.
- Change the format of the output URL to be shorter, to meet IE constraints
* hosted/sunspider-driver.html:
* hosted/sunspider-results.html:
2007-12-13 Maciej Stachowiak <>
Reviewed by Oliver.
- Add some explanatory test and a dumb logo to SunSpider web hosted version
* hosted/sunspider-driver.html:
* hosted/sunspider-results.html:
* hosted/sunspider.css:
* hosted/sunspider.html:
2007-12-13 Maciej Stachowiak <>
Reviewed by Oliver.
- Make the web-hosted version of SunSpider look a little bit less drab.
* hosted/sunspider-driver.html:
* hosted/sunspider-results.html:
* hosted/sunspider.css: Added.
* hosted/sunspider.html:
* resources/TEMPLATE.html:
2007-12-12 Maciej Stachowiak <>
Reviewed by Sam.
- Change sunspider-compare-results output from showing "% faster" values to "x as fast" values.
A sample of what new-style results would look like:
** TOTAL **: 1.41x as fast 6613.6ms +/- 0.1% 4703.0ms +/- 0.1% significant
3bit-bits-in-byte: *1.36x as slow* 88.8ms +/- 0.6% 121.1ms +/- 0.6% significant
* resources/sunspider-compare-results.js:
2007-11-14 Eric Seidel <>
Reviewed by Sam.
* sunspider: add --shark-cache for L2 Cache Miss profiling
2007-11-09 Alexey Proskuryakov <>
Reviewed by Darin Adler.
SunSpider should check for Shark being installed
* sunspider: Explicitly test that shark command line tool is installed.
2007-11-04 Darin Adler <>
* hosted/sunspider.html: Tweaked formatting a bit.
2007-11-03 Maciej Stachowiak <>
Reviewed by Darin Adler.
- Add compare mode and a description of the benchmark to the hosted version.
* hosted/sunspider-results.html: Add compare mode - you can just paste in another results URL.
* hosted/sunspider.html: Added intro text.
* make-hosted: Copy new file.
* resources/sunspider-compare-results.js: Made more reusable by putting the implementation in
a function.
* resources/sunspider-standalone-compare.js: Added. Calls the function in the way standalone mode
* sunspider-compare-results: Use sunspider-standalone-compare.js.
2007-11-02 Darin Adler <>
* sunspider: Changed "--base" to "--set-baseline".
2007-11-02 Darin Adler <>
Reviewed by Maciej.
* sunspider: Added a "--base" option that records the current test as a baseline.
Moved the code to find the newest .mshark file into a function (since I needed to use
it in the other script and I prefer to copy/paste a function).
* sunspider-compare-results: Added logic to compare the baseline with the last
run when you don't pass any parameters.
2007-10-29 Maciej Stachowiak <>
Reviewed by Eric.
- Added list of ECMAScript3 features that SunSpider does not cover currently.
2007-10-29 Maciej Stachowiak <>
Reviewed by Oliver.
- Give better error messages when sunspider-compare-results fails
* sunspider-compare-results:
2007-10-29 Maciej Stachowiak <>
Reviewed by Darin Adler.
- fix Date tests to work in command-line SpiderMonkey
* tests/date-format-tofte.js:
* tests/date-format-xparb.js:
2007-10-29 Darin Adler <>
Reviewed by Maciej.
* sunspider: Added --shark20 option, to run Shark at its highest sample
resolution instead of its default.
2007-10-28 Adam Roben <>
Fix SunSpider on Windows
Reviewed by Eric.
* sunspider: Don't use colons in filenames.
2007-10-28 Darin Adler <>
Reviewed by Adam.
* resources/sunspider-analyze-results.js: Tweak the output format a little.
Change so that when there's only one run we don't write out confidence
intervals at all rather than writing "NaN" over and over again.
* sunspider: Use the time and date as a suffix on the results file, that way
all the results are left behind in the tmp directory. This will make it easy
for us to add features that compare past results in the future. Also rename
the .mshark files using the same scheme.
* sunspider-compare-results: Relax the parsing rules so we can parse the
results file exactly as it's emitted from sunspider.
2007-10-25 Adam Roben <>
Strip carriage returns from results
Carriage returns were screwing up sunspider-compare-results on
Rubberstamped by Oliver.
* sunspider:
2007-10-24 Darin Adler <>
Reviewed by Eric.
* sunspider: Open the profile in shark after completing a run.
* sunspider-compare-results: Tiny perl tweaks.
2007-10-23 Maciej Stachowiak <>
Reviewed by Darin Adler.
- Don't hardcode my path to testkjs
* sunspider-compare-results:
2007-10-23 Maciej Stachowiak <>
Reviewed by Darin Adler.
- Add a compare script to compare results, and improve formatting of standard results a bit.
* resources/sunspider-analyze-results.js:
* resources/sunspider-compare-results.js: Added.
* sunspider-compare-results: Added.
2007-10-23 Maciej Stachowiak <>
Reviewed by Darin Adler.
* resources/sunspider-analyze-results.js: Do the error range
properly, using the t-distribution instead of 1.96 (which was
based on the normal distribution).
* sunspider: Print results in a way that will be friendlier to the
soon-to-come compare mode.
2007-10-22 Maciej Stachowiak <>
Reviewed by Review.
- add more tests (probably the final set for now)
Already balanced. Added date, regexp, control flow, and a few
more string and object/array access tests.
* tests/LIST:
* tests/access-fannkuch.js: Added.
* tests/access-nbody.js: Added.
* tests/controlflow-recursive.js: Added.
* tests/date-format-tofte.js: Added.
* tests/date-format-xparb.js: Added.
* tests/regexp-dna.js: Added.
* tests/string-validate-input.js: Added.
2007-10-21 Eric Seidel <>
Reviewed by mjs.
* sunspider: make --shark only sample from testkjs
2007-10-21 Maciej Stachowiak <>
Rubber stamped by Eric.
- added crypto tests (already balanced)
* tests/LIST:
* tests/crypto-aes.js: Added.
* tests/crypto-md5.js: Added.
* tests/crypto-sha1.js: Added.
2007-10-21 Maciej Stachowiak <>
Reviewed by Adam.
- rebalance new tests
Scaled the new tests so they take 400-600ms on a reasonably normal user setup:
Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
This balances them with the existing tests.
* tests/access-binary-trees.js:
* tests/access-nsieve.js:
* tests/bitops-nsieve-bits.js:
* tests/math-partial-sums.js:
* tests/math-spectral-norm.js:
* tests/string-fasta.js:
2007-10-20 Eric Seidel <>
Reviewed by darin.
Add improved argument handling to sunspider, including
--runs=<number>, --tests=<pattern, --shark, and --help
Also re-factor code into subroutines
* sunspider: improved argument handling
2007-10-20 Maciej Stachowiak <>
Reviewed by Mark.
- Add more new tests, mostly from the computer language shootout. Not normalized yet.
* tests/LIST:
* tests/access-binary-trees.js: Added.
* tests/access-nsieve.js: Added.
* tests/bitops-nsieve-bits.js: Added.
* tests/math-partial-sums.js: Added.
* tests/math-spectral-norm.js: Added.
* tests/string-fasta.js: Added.
2007-10-20 Maciej Stachowiak <>
Reviewed by Darin Adler.
* TODO: Updated to note areas that are now well-covered.
- Change the tests and the driver to avoid leaving large object
graphs hanging around, since that throws off the subsequent tests.
* resources/sunspider-standalone-driver.js:
* tests/3d-cube.js:
* tests/3d-morph.js:
* tests/3d-raytrace.js:
* tests/bitops-bitwise-and.js:
* tests/string-base64.js:
* tests/string-tagcloud.js:
2007-10-20 Maciej Stachowiak <>
Reviewed by Nikolas.
- Rebalance test complexity.
Scaled all the tests so they take 400-600ms on a reasonably normal user setup:
Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
However, for some tests, IE7 was a huge outlier compared to other
major browsers. For those tests, I used Firefox on the
same OS and hardware as a normalization basis.
* tests/3d-morph.js:
* tests/3d-raytrace.js:
* tests/bitops-3bit-bits-in-byte.js:
* tests/bitops-bits-in-byte.js:
* tests/bitops-bitwise-and.js:
* tests/math-cordic.js:
* tests/string-base64.js:
* tests/string-tagcloud.js:
* tests/string-unpack-code.js:
2007-10-20 Maciej Stachowiak <>
Reviewed by Oliver.
- Fix the browser-hosted driver to be compatible with Safari 2
* hosted/sunspider-driver.html:
2007-10-20 Maciej Stachowiak <>
Rubber stamped by Hyatt.
- Add BSD license notice to cordic test, the original author said to do so.
* tests/math-cordic.js:
2007-10-18 Maciej Stachowiak <>
Reviewed by Eric.
- Support Windows and Windows browsers.
* hosted/sunspider-results.html: Display results correctly in IE.
* sunspider: tweaks to work on Windows, and with spaces in the path.
2007-10-18 Maciej Stachowiak <>
Rubber stamped by Oliver.
* tests/string-tagcloud.js: Remove stray print()
2007-10-17 Maciej Stachowiak <>
Reviewed by Oliver.
Add some string test cases.
* tests/string-base64.js: Added. Does base64 encoding and decoding.
* tests/string-tagcloud.js: Added. Parses a JSON string and
generates tagcloud HTML markup based on the data.
* tests/string-unpack-code.js: Added. Unpack compressed versions of
four of the most popular JavaScript libraries.
* tests/LIST: Added the new tests.
2007-10-16 Maciej Stachowiak <>
Reviewed by Anders.
* TODO: Updated with info on how to do the stats properly.
* hosted/sunspider-driver.html: Reduce timeout from 500 to 200 to
avoid overly slowing down the benchmark.
Added some 3D-related test cases.
* tests/3d-cube.js: Added.
* tests/3d-morph.js: Added.
* tests/3d-raytrace.js: Added.
* tests/LIST:
2007-10-14 Maciej Stachowiak <>
Reviewed by Darin Adler.
- New JavaScript benchmark
A start on a new JavaScript benchmark, with standalone and
browser-hosted drivers.
The test content is still incomplete and the drivers could use
some more features but there is enough here to make a good start.
The drivers compute a 95% confidence interval on the mean for the
whole test, each category, and each individual test to make it
easier to tell whether differences are statistically
significant. The confidence interval can be narrowed by running
the test more times.
* TODO: Added. Remaining things that need to be done.
* sunspider: Added. Perl script that acts as the standalone test driver.
* resources/sunspider-standalone-driver.js: Added. JavaScript part
of standalone test driver.
* resources/sunspider-analyze-results.js: Added. JavaScript statistical
analysis code.
* resources/TEMPLATE.html: Added. Template for browser-hosted tests.
* make-hosted: Added. Script to generate browser-hosted tests.
* hosted/sunspider.html: Added. Start page for browser-hosted test.
* hosted/sunspider-driver.html: Added. Driver for browser-hosted tests.
* hosted/sunspider-results.html: Added. Results page for browser-hosted
* hosted/sunspider-record-result.js: Added. Helper file for browser-hosted
* tests/LIST: Added. List of tests to use.
* tests/bitops-3bit-bits-in-byte.js: Added. Some initial test content.
* tests/bitops-bits-in-byte.js: Added. Ditto.
* tests/bitops-bitwise-and.js: Added. Ditto.
* tests/math-cordic.js: Added. Ditto.