| <?xml version="1.0" encoding="UTF-8" standalone="no"?> |
| <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"><html xmlns="http://www.w3.org/1999/xhtml"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /><title>Test</title><meta name="generator" content="DocBook XSL-NS Stylesheets V1.78.1" /><meta name="keywords" content="ISO C++, test, testsuite, performance, conformance, ABI, exception safety" /><meta name="keywords" content="ISO C++, library" /><meta name="keywords" content="ISO C++, runtime, library" /><link rel="home" href="../index.html" title="The GNU C++ Library" /><link rel="up" href="appendix_porting.html" title="Appendix B. Porting and Maintenance" /><link rel="prev" href="internals.html" title="Porting to New Hardware or Operating Systems" /><link rel="next" href="abi.html" title="ABI Policy and Guidelines" /></head><body><div class="navheader"><table width="100%" summary="Navigation header"><tr><th colspan="3" align="center">Test</th></tr><tr><td width="20%" align="left"><a accesskey="p" href="internals.html">Prev</a> </td><th width="60%" align="center">Appendix B. |
| Porting and Maintenance |
| |
| </th><td width="20%" align="right"> <a accesskey="n" href="abi.html">Next</a></td></tr></table><hr /></div><div class="section"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a id="manual.intro.setup.test"></a>Test</h2></div></div></div><p> |
| The libstdc++ testsuite includes testing for standard conformance, |
| regressions, ABI, and performance. |
| </p><div class="section"><div class="titlepage"><div><div><h3 class="title"><a id="test.organization"></a>Organization</h3></div></div></div><div class="section"><div class="titlepage"><div><div><h4 class="title"><a id="test.organization.layout"></a>Directory Layout</h4></div></div></div><p> |
| The directory <span class="emphasis"><em>libsrcdir/testsuite</em></span> contains the |
| individual test cases organized in sub-directories corresponding to |
| chapters of the C++ standard (detailed below), the dejagnu test |
| harness support files, and sources to various testsuite utilities |
| that are packaged in a separate testing library. |
| </p><p> |
| All test cases for functionality required by the runtime components |
| of the C++ standard (ISO 14882) are files within the following |
| directories. |
| </p><pre class="programlisting"> |
| 17_intro |
| 18_support |
| 19_diagnostics |
| 20_util |
| 21_strings |
| 22_locale |
| 23_containers |
| 25_algorithms |
| 26_numerics |
| 27_io |
| 28_regex |
| 29_atomics |
| 30_threads |
| </pre><p> |
| In addition, the following directories include test files: |
| </p><pre class="programlisting"> |
| tr1 Tests for components as described by the Technical Report on Standard Library Extensions (TR1). |
| backward Tests for backwards compatibility and deprecated features. |
| demangle Tests for __cxa_demangle, the IA 64 C++ ABI demangler |
| ext Tests for extensions. |
| performance Tests for performance analysis, and performance regressions. |
| </pre><p> |
| Some directories don't have test files, but instead contain |
| auxiliary information: |
| </p><pre class="programlisting"> |
| config Files for the dejagnu test harness. |
| lib Files for the dejagnu test harness. |
| libstdc++* Files for the dejagnu test harness. |
| data Sample text files for testing input and output. |
| util Files for libtestc++, utilities and testing routines. |
| </pre><p> |
| Within a directory that includes test files, there may be |
| additional subdirectories, or files. Originally, test cases |
| were appended to one file that represented a particular section |
| of the chapter under test, and was named accordingly. For |
| instance, to test items related to <code class="code"> 21.3.6.1 - |
| basic_string::find [lib.string::find]</code> in the standard, |
| the following was used: |
| </p><pre class="programlisting"> |
| 21_strings/find.cc |
| </pre><p> |
| However, that practice soon became a liability as the test cases |
| became huge and unwieldy, and testing new or extended |
| functionality (like wide characters or named locales) became |
| frustrating, leading to aggressive pruning of test cases on some |
| platforms that covered up implementation errors. Now, the test |
| suite has a policy of one file, one test case, which solves the |
| above issues and gives finer grained results and more manageable |
| error debugging. As an example, the test case quoted above |
| becomes: |
| </p><pre class="programlisting"> |
| 21_strings/basic_string/find/char/1.cc |
| 21_strings/basic_string/find/char/2.cc |
| 21_strings/basic_string/find/char/3.cc |
| 21_strings/basic_string/find/wchar_t/1.cc |
| 21_strings/basic_string/find/wchar_t/2.cc |
| 21_strings/basic_string/find/wchar_t/3.cc |
| </pre><p> |
| All new tests should be written with the policy of one test |
| case, one file in mind. |
| </p></div><div class="section"><div class="titlepage"><div><div><h4 class="title"><a id="test.organization.naming"></a>Naming Conventions</h4></div></div></div><p> |
| In addition, there are some special names and suffixes that are |
| used within the testsuite to designate particular kinds of |
| tests. |
| </p><div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p> |
| <span class="emphasis"><em>_xin.cc</em></span> |
| </p><p> |
| This test case expects some kind of interactive input in order |
| to finish or pass. At the moment, the interactive tests are not |
| run by default. Instead, they are run by hand, like: |
| </p><pre class="programlisting"> |
| g++ 27_io/objects/char/3_xin.cc |
| cat 27_io/objects/char/3_xin.in | a.out |
| </pre></li><li class="listitem"><p> |
| <span class="emphasis"><em>.in</em></span> |
| </p><p> |
| This file contains the expected input for the corresponding <span class="emphasis"><em> |
| _xin.cc</em></span> test case. |
| </p></li><li class="listitem"><p> |
| <span class="emphasis"><em>_neg.cc</em></span> |
| </p><p> |
| This test case is expected to fail: it's a negative test. At the |
| moment, these are almost always compile time errors. |
| </p></li><li class="listitem"><p> |
| <span class="emphasis"><em>char</em></span> |
| </p><p> |
| This can either be a directory name or part of a longer file |
| name, and indicates that this file, or the files within this |
| directory are testing the <code class="code">char</code> instantiation of a |
| template. |
| </p></li><li class="listitem"><p> |
| <span class="emphasis"><em>wchar_t</em></span> |
| </p><p> |
| This can either be a directory name or part of a longer file |
| name, and indicates that this file, or the files within this |
| directory are testing the <code class="code">wchar_t</code> instantiation of |
| a template. Some hosts do not support <code class="code">wchar_t</code> |
| functionality, so for these targets, all of these tests will not |
| be run. |
| </p></li><li class="listitem"><p> |
| <span class="emphasis"><em>thread</em></span> |
| </p><p> |
| This can either be a directory name or part of a longer file |
| name, and indicates that this file, or the files within this |
| directory are testing situations where multiple threads are |
| being used. |
| </p></li><li class="listitem"><p> |
| <span class="emphasis"><em>performance</em></span> |
| </p><p> |
| This can either be an enclosing directory name or part of a |
| specific file name. This indicates a test that is used to |
| analyze runtime performance, for performance regression testing, |
| or for other optimization related analysis. At the moment, these |
| test cases are not run by default. |
| </p></li></ul></div></div></div><div class="section"><div class="titlepage"><div><div><h3 class="title"><a id="test.run"></a>Running the Testsuite</h3></div></div></div><div class="section"><div class="titlepage"><div><div><h4 class="title"><a id="test.run.basic"></a>Basic</h4></div></div></div><p> |
| You can check the status of the build without installing it |
| using the dejagnu harness, much like the rest of the gcc |
| tools.</p><pre class="programlisting"> make check</pre><p>in the <span class="emphasis"><em>libbuilddir</em></span> directory.</p><p>or</p><pre class="programlisting"> make check-target-libstdc++-v3</pre><p>in the <span class="emphasis"><em>gccbuilddir</em></span> directory. |
| </p><p> |
| These commands are functionally equivalent and will create a |
| 'testsuite' directory underneath |
| <span class="emphasis"><em>libbuilddir</em></span> containing the results of the |
| tests. Two results files will be generated: <span class="emphasis"><em> |
| libstdc++.sum</em></span>, which is a PASS/FAIL summary for each |
| test, and <span class="emphasis"><em>libstdc++.log</em></span> which is a log of |
| the exact command line passed to the compiler, the compiler |
| output, and the executable output (if any). |
| </p><p> |
| Archives of test results for various versions and platforms are |
| available on the GCC website in the <a class="link" href="http://gcc.gnu.org/gcc-4.3/buildstat.html" target="_top">build |
| status</a> section of each individual release, and are also |
| archived on a daily basis on the <a class="link" href="http://gcc.gnu.org/ml/gcc-testresults/current" target="_top">gcc-testresults</a> |
| mailing list. Please check either of these places for a similar |
| combination of source version, operating system, and host CPU. |
| </p></div><div class="section"><div class="titlepage"><div><div><h4 class="title"><a id="test.run.variations"></a>Variations</h4></div></div></div><p> |
| There are several options for running tests, including testing |
| the regression tests, testing a subset of the regression tests, |
| testing the performance tests, testing just compilation, testing |
| installed tools, etc. In addition, there is a special rule for |
| checking the exported symbols of the shared library. |
| </p><p> |
| To debug the dejagnu test harness during runs, try invoking with a |
| specific argument to the variable RUNTESTFLAGS, as below. |
| </p><pre class="programlisting"> |
| make check-target-libstdc++-v3 RUNTESTFLAGS="-v" |
| </pre><p> |
| or |
| </p><pre class="programlisting"> |
| make check-target-libstdc++-v3 RUNTESTFLAGS="-v -v" |
| </pre><p> |
| To run a subset of the library tests, you will need to generate |
| the <span class="emphasis"><em>testsuite_files</em></span> file by running |
| <span class="command"><strong>make testsuite_files</strong></span> in the |
| <span class="emphasis"><em>libbuilddir/testsuite</em></span> directory, described |
| below. Edit the file to remove the tests you don't want and |
| then run the testsuite as normal. |
| </p><p> |
| There are two ways to run on a simulator: set up DEJAGNU to point to a |
| specially crafted site.exp, or pass down --target_board flags. |
| </p><p> |
| Example flags to pass down for various embedded builds are as follows: |
| </p><pre class="programlisting"> |
| --target=powerpc-eabism (libgloss/sim) |
| make check-target-libstdc++-v3 RUNTESTFLAGS="--target_board=powerpc-sim" |
| |
| --target=calmrisc32 (libgloss/sid) |
| make check-target-libstdc++-v3 RUNTESTFLAGS="--target_board=calmrisc32-sid" |
| |
| --target=xscale-elf (newlib/sim) |
| make check-target-libstdc++-v3 RUNTESTFLAGS="--target_board=arm-sim" |
| </pre><p> |
| Also, here is an example of how to run the libstdc++ testsuite |
| for a multilibed build directory with different ABI settings: |
| </p><pre class="programlisting"> |
| make check-target-libstdc++-v3 RUNTESTFLAGS='--target_board \"unix{-mabi=32,,-mabi=64}\"' |
| </pre><p> |
| You can run the tests with a compiler and library that have |
| already been installed. Make sure that the compiler (e.g., |
| <code class="code">g++</code>) is in your <code class="code">PATH</code>. If you are |
| using shared libraries, then you must also ensure that the |
| directory containing the shared version of libstdc++ is in your |
| <code class="code">LD_LIBRARY_PATH</code>, or equivalent. If your GCC source |
| tree is at <code class="code">/path/to/gcc</code>, then you can run the tests |
| as follows: |
| </p><pre class="programlisting"> |
| runtest --tool libstdc++ --srcdir=/path/to/gcc/libstdc++-v3/testsuite |
| </pre><p> |
| The testsuite will create a number of files in the directory in |
| which you run this command,. Some of those files might use the |
| same name as files created by other testsuites (like the ones |
| for GCC and G++), so you should not try to run all the |
| testsuites in parallel from the same directory. |
| </p><p> |
| In addition, there are some testing options that are mostly of |
| interest to library maintainers and system integrators. As such, |
| these tests may not work on all cpu and host combinations, and |
| may need to be executed in the |
| <span class="emphasis"><em>libbuilddir/testsuite</em></span> directory. These |
| options include, but are not necessarily limited to, the |
| following: |
| </p><pre class="programlisting"> |
| make testsuite_files |
| </pre><p> |
| Five files are generated that determine what test files |
| are run. These files are: |
| </p><div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p> |
| <span class="emphasis"><em>testsuite_files</em></span> |
| </p><p> |
| This is a list of all the test cases that will be run. Each |
| test case is on a separate line, given with an absolute path |
| from the <span class="emphasis"><em>libsrcdir/testsuite</em></span> directory. |
| </p></li><li class="listitem"><p> |
| <span class="emphasis"><em>testsuite_files_interactive</em></span> |
| </p><p> |
| This is a list of all the interactive test cases, using the |
| same format as the file list above. These tests are not run |
| by default. |
| </p></li><li class="listitem"><p> |
| <span class="emphasis"><em>testsuite_files_performance</em></span> |
| </p><p> |
| This is a list of all the performance test cases, using the |
| same format as the file list above. These tests are not run |
| by default. |
| </p></li><li class="listitem"><p> |
| <span class="emphasis"><em>testsuite_thread</em></span> |
| </p><p> |
| This file indicates that the host system can run tests which |
| involved multiple threads. |
| </p></li><li class="listitem"><p> |
| <span class="emphasis"><em>testsuite_wchar_t</em></span> |
| </p><p> |
| This file indicates that the host system can run the wchar_t |
| tests, and corresponds to the macro definition <code class="code"> |
| _GLIBCXX_USE_WCHAR_T</code> in the file c++config.h. |
| </p></li></ul></div><pre class="programlisting"> |
| make check-abi |
| </pre><p> |
| The library ABI can be tested. This involves testing the shared |
| library against an ABI-defining previous version of symbol |
| exports. |
| </p><pre class="programlisting"> |
| make check-compile |
| </pre><p> |
| This rule compiles, but does not link or execute, the |
| <span class="emphasis"><em>testsuite_files</em></span> test cases and displays the |
| output on stdout. |
| </p><pre class="programlisting"> |
| make check-performance |
| </pre><p> |
| This rule runs through the |
| <span class="emphasis"><em>testsuite_files_performance</em></span> test cases and |
| collects information for performance analysis and can be used to |
| spot performance regressions. Various timing information is |
| collected, as well as number of hard page faults, and memory |
| used. This is not run by default, and the implementation is in |
| flux. |
| </p><p> |
| We are interested in any strange failures of the testsuite; |
| please email the main libstdc++ mailing list if you see |
| something odd or have questions. |
| </p></div><div class="section"><div class="titlepage"><div><div><h4 class="title"><a id="test.run.permutations"></a>Permutations</h4></div></div></div><p> |
| To run the libstdc++ test suite under the <a class="link" href="debug_mode.html" title="Chapter 17. Debug Mode">debug mode</a>, edit |
| <code class="filename">libstdc++-v3/scripts/testsuite_flags</code> to add the |
| compile-time flag <code class="constant">-D_GLIBCXX_DEBUG</code> to the |
| result printed by the <code class="literal">--build-cxx</code> |
| option. Additionally, add the |
| <code class="constant">-D_GLIBCXX_DEBUG_PEDANTIC</code> flag to turn on |
| pedantic checking. The libstdc++ test suite should produce |
| precisely the same results under debug mode that it does under |
| release mode: any deviation indicates an error in either the |
| library or the test suite. |
| </p><p> |
| The <a class="link" href="parallel_mode.html" title="Chapter 18. Parallel Mode">parallel |
| mode</a> can be tested in much the same manner, substituting |
| <code class="constant">-D_GLIBCXX_PARALLEL</code> for |
| <code class="constant">-D_GLIBCXX_DEBUG</code> in the previous paragraph. |
| </p><p> |
| Or, just run the testsuites with <code class="constant">CXXFLAGS</code> |
| set to <code class="constant">-D_GLIBCXX_DEBUG</code> or |
| <code class="constant">-D_GLIBCXX_PARALLEL</code>. |
| </p></div></div><div class="section"><div class="titlepage"><div><div><h3 class="title"><a id="test.new_tests"></a>Writing a new test case</h3></div></div></div><p> |
| The first step in making a new test case is to choose the correct |
| directory and file name, given the organization as previously |
| described. |
| </p><p> |
| All files are copyright the FSF, and GPL'd: this is very |
| important. The first copyright year should correspond to the date |
| the file was checked in to SVN. |
| </p><p> |
| As per the dejagnu instructions, always return 0 from main to |
| indicate success. |
| </p><p> |
| A bunch of utility functions and classes have already been |
| abstracted out into the testsuite utility library, <code class="code"> |
| libtestc++</code>. To use this functionality, just include the |
| appropriate header file: the library or specific object files will |
| automatically be linked in as part of the testsuite run. |
| </p><p> |
| For a test that needs to take advantage of the dejagnu test |
| harness, what follows below is a list of special keyword that |
| harness uses. Basically, a test case contains dg-keywords (see |
| dg.exp) indicating what to do and what kinds of behavior are to be |
| expected. New test cases should be written with the new style |
| DejaGnu framework in mind. |
| </p><p> |
| To ease transition, here is the list of dg-keyword documentation |
| lifted from dg.exp. |
| </p><pre class="programlisting"> |
| # The currently supported options are: |
| # |
| # dg-prms-id N |
| # set prms_id to N |
| # |
| # dg-options "options ..." [{ target selector }] |
| # specify special options to pass to the tool (eg: compiler) |
| # |
| # dg-do do-what-keyword [{ target/xfail selector }] |
| # `do-what-keyword' is tool specific and is passed unchanged to |
| # ${tool}-dg-test. An example is gcc where `keyword' can be any of: |
| # preprocess|compile|assemble|link|run |
| # and will do one of: produce a .i, produce a .s, produce a .o, |
| # produce an a.out, or produce an a.out and run it (the default is |
| # compile). |
| # |
| # dg-error regexp comment [{ target/xfail selector } [{.|0|linenum}]] |
| # indicate an error message <regexp> is expected on this line |
| # (the test fails if it doesn't occur) |
| # Linenum=0 for general tool messages (eg: -V arg missing). |
| # "." means the current line. |
| # |
| # dg-warning regexp comment [{ target/xfail selector } [{.|0|linenum}]] |
| # indicate a warning message <regexp> is expected on this line |
| # (the test fails if it doesn't occur) |
| # |
| # dg-bogus regexp comment [{ target/xfail selector } [{.|0|linenum}]] |
| # indicate a bogus error message <regexp> use to occur here |
| # (the test fails if it does occur) |
| # |
| # dg-build regexp comment [{ target/xfail selector }] |
| # indicate the build use to fail for some reason |
| # (errors covered here include bad assembler generated, tool crashes, |
| # and link failures) |
| # (the test fails if it does occur) |
| # |
| # dg-excess-errors comment [{ target/xfail selector }] |
| # indicate excess errors are expected (any line) |
| # (this should only be used sparingly and temporarily) |
| # |
| # dg-output regexp [{ target selector }] |
| # indicate the expected output of the program is <regexp> |
| # (there may be multiple occurrences of this, they are concatenated) |
| # |
| # dg-final { tcl code } |
| # add some tcl code to be run at the end |
| # (there may be multiple occurrences of this, they are concatenated) |
| # (unbalanced braces must be \-escaped) |
| # |
| # "{ target selector }" is a list of expressions that determine whether the |
| # test succeeds or fails for a particular target, or in some cases whether the |
| # option applies for a particular target. If the case of `dg-do' it specifies |
| # whether the test case is even attempted on the specified target. |
| # |
| # The target selector is always optional. The format is one of: |
| # |
| # { xfail *-*-* ... } - the test is expected to fail for the given targets |
| # { target *-*-* ... } - the option only applies to the given targets |
| # |
| # At least one target must be specified, use *-*-* for "all targets". |
| # At present it is not possible to specify both `xfail' and `target'. |
| # "native" may be used in place of "*-*-*". |
| |
| Example 1: Testing compilation only |
| // { dg-do compile } |
| |
| Example 2: Testing for expected warnings on line 36, which all targets fail |
| // { dg-warning "string literals" "" { xfail *-*-* } 36 } |
| |
| Example 3: Testing for expected warnings on line 36 |
| // { dg-warning "string literals" "" { target *-*-* } 36 } |
| |
| Example 4: Testing for compilation errors on line 41 |
| // { dg-do compile } |
| // { dg-error "no match for" "" { target *-*-* } 41 } |
| |
| Example 5: Testing with special command line settings, or without the |
| use of pre-compiled headers, in particular the stdc++.h.gch file. Any |
| options here will override the DEFAULT_CXXFLAGS and PCH_CXXFLAGS set |
| up in the normal.exp file. |
| // { dg-options "-O0" { target *-*-* } } |
| </pre><p> |
| More examples can be found in the libstdc++-v3/testsuite/*/*.cc files. |
| </p></div><div class="section"><div class="titlepage"><div><div><h3 class="title"><a id="test.harness"></a>Test Harness and Utilities</h3></div></div></div><div class="section"><div class="titlepage"><div><div><h4 class="title"><a id="test.harness.dejagnu"></a>Dejagnu Harness Details</h4></div></div></div><p> |
| Underlying details of testing for conformance and regressions are |
| abstracted via the GNU Dejagnu package. This is similar to the |
| rest of GCC. |
| </p><p>This is information for those looking at making changes to the testsuite |
| structure, and/or needing to trace dejagnu's actions with --verbose. This |
| will not be useful to people who are "merely" adding new tests to the existing |
| structure. |
| </p><p>The first key point when working with dejagnu is the idea of a "tool". |
| Files, directories, and functions are all implicitly used when they are |
| named after the tool in use. Here, the tool will always be "libstdc++". |
| </p><p>The <code class="code">lib</code> subdir contains support routines. The |
| <code class="code">lib/libstdc++.exp</code> file ("support library") is loaded |
| automagically, and must explicitly load the others. For example, files can |
| be copied from the core compiler's support directory into <code class="code">lib</code>. |
| </p><p>Some routines in <code class="code">lib/libstdc++.exp</code> are callbacks, some are |
| our own. Callbacks must be prefixed with the name of the tool. To easily |
| distinguish the others, by convention our own routines are named "v3-*". |
| </p><p>The next key point when working with dejagnu is "test files". Any |
| directory whose name starts with the tool name will be searched for test files. |
| (We have only one.) In those directories, any <code class="code">.exp</code> file is |
| considered a test file, and will be run in turn. Our main test file is called |
| <code class="code">normal.exp</code>; it runs all the tests in testsuite_files using the |
| callbacks loaded from the support library. |
| </p><p>The <code class="code">config</code> directory is searched for any particular "target |
| board" information unique to this library. This is currently unused and sets |
| only default variables. |
| </p></div><div class="section"><div class="titlepage"><div><div><h4 class="title"><a id="test.harness.utils"></a>Utilities</h4></div></div></div><p> |
| </p><p> |
| The testsuite directory also contains some files that implement |
| functionality that is intended to make writing test cases easier, |
| or to avoid duplication, or to provide error checking in a way that |
| is consistent across platforms and test harnesses. A stand-alone |
| executable, called <span class="emphasis"><em>abi_check</em></span>, and a static |
| library called <span class="emphasis"><em>libtestc++</em></span> are |
| constructed. Both of these items are not installed, and only used |
| during testing. |
| </p><p> |
| These files include the following functionality: |
| </p><div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p> |
| <span class="emphasis"><em>testsuite_abi.h</em></span>, |
| <span class="emphasis"><em>testsuite_abi.cc</em></span>, |
| <span class="emphasis"><em>testsuite_abi_check.cc</em></span> |
| </p><p> |
| Creates the executable <span class="emphasis"><em>abi_check</em></span>. |
| Used to check correctness of symbol versioning, visibility of |
| exported symbols, and compatibility on symbols in the shared |
| library, for hosts that support this feature. More information |
| can be found in the ABI documentation <a class="link" href="abi.html" title="ABI Policy and Guidelines">here</a> |
| </p></li><li class="listitem"><p> |
| <span class="emphasis"><em>testsuite_allocator.h</em></span>, |
| <span class="emphasis"><em>testsuite_allocator.cc</em></span> |
| </p><p> |
| Contains specialized allocators that keep track of construction |
| and destruction. Also, support for overriding global new and |
| delete operators, including verification that new and delete |
| are called during execution, and that allocation over max_size |
| fails. |
| </p></li><li class="listitem"><p> |
| <span class="emphasis"><em>testsuite_character.h</em></span> |
| </p><p> |
| Contains <code class="code">std::char_traits</code> and |
| <code class="code">std::codecvt</code> specializations for a user-defined |
| POD. |
| </p></li><li class="listitem"><p> |
| <span class="emphasis"><em>testsuite_hooks.h</em></span>, |
| <span class="emphasis"><em>testsuite_hooks.cc</em></span> |
| </p><p> |
| A large number of utilities, including: |
| </p><div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: circle; "><li class="listitem"><p>VERIFY</p></li><li class="listitem"><p>set_memory_limits</p></li><li class="listitem"><p>verify_demangle</p></li><li class="listitem"><p>run_tests_wrapped_locale</p></li><li class="listitem"><p>run_tests_wrapped_env</p></li><li class="listitem"><p>try_named_locale</p></li><li class="listitem"><p>try_mkfifo</p></li><li class="listitem"><p>func_callback</p></li><li class="listitem"><p>counter</p></li><li class="listitem"><p>copy_tracker</p></li><li class="listitem"><p>copy_constructor</p></li><li class="listitem"><p>assignment_operator</p></li><li class="listitem"><p>destructor</p></li><li class="listitem"><p>pod_char, pod_int and associated char_traits specializations</p></li></ul></div></li><li class="listitem"><p> |
| <span class="emphasis"><em>testsuite_io.h</em></span> |
| </p><p> |
| Error, exception, and constraint checking for |
| <code class="code">std::streambuf, std::basic_stringbuf, std::basic_filebuf</code>. |
| </p></li><li class="listitem"><p> |
| <span class="emphasis"><em>testsuite_iterators.h</em></span> |
| </p><p> |
| Wrappers for various iterators. |
| </p></li><li class="listitem"><p> |
| <span class="emphasis"><em>testsuite_performance.h</em></span> |
| </p><p> |
| A number of class abstractions for performance counters, and |
| reporting functions including: |
| </p><div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: circle; "><li class="listitem"><p>time_counter</p></li><li class="listitem"><p>resource_counter</p></li><li class="listitem"><p>report_performance</p></li></ul></div></li></ul></div></div></div><div class="section"><div class="titlepage"><div><div><h3 class="title"><a id="test.special"></a>Special Topics</h3></div></div></div><div class="section"><div class="titlepage"><div><div><h4 class="title"><a id="test.exception.safety"></a> |
| Qualifying Exception Safety Guarantees |
| <a id="idm269992903504" class="indexterm"></a> |
| </h4></div></div></div><div class="section"><div class="titlepage"><div><div><h5 class="title"><a id="test.exception.safety.overview"></a>Overview</h5></div></div></div><p> |
| Testing is composed of running a particular test sequence, |
| and looking at what happens to the surrounding code when |
| exceptions are thrown. Each test is composed of measuring |
| initial state, executing a particular sequence of code under |
| some instrumented conditions, measuring a final state, and |
| then examining the differences between the two states. |
| </p><p> |
| Test sequences are composed of constructed code sequences |
| that exercise a particular function or member function, and |
| either confirm no exceptions were generated, or confirm the |
| consistency/coherency of the test subject in the event of a |
| thrown exception. |
| </p><p> |
| Random code paths can be constructed using the basic test |
| sequences and instrumentation as above, only combined in a |
| random or pseudo-random way. |
| </p><p> To compute the code paths that throw, test instruments |
| are used that throw on allocation events |
| (<code class="classname">__gnu_cxx::throw_allocator_random</code> |
| and <code class="classname">__gnu_cxx::throw_allocator_limit</code>) |
| and copy, assignment, comparison, increment, swap, and |
| various operators |
| (<code class="classname">__gnu_cxx::throw_type_random</code> |
| and <code class="classname">__gnu_cxx::throw_type_limit</code>). Looping |
| through a given test sequence and conditionally throwing in |
| all instrumented places. Then, when the test sequence |
| completes without an exception being thrown, assume all |
| potential error paths have been exercised in a sequential |
| manner. |
| </p></div><div class="section"><div class="titlepage"><div><div><h5 class="title"><a id="test.exception.safety.status"></a> |
| Existing tests |
| </h5></div></div></div><div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p> |
| Ad Hoc |
| </p><p> |
| For example, |
| <code class="filename">testsuite/23_containers/list/modifiers/3.cc</code>. |
| </p></li><li class="listitem"><p> |
| Policy Based Data Structures |
| </p><p> |
| For example, take the test |
| functor <code class="classname">rand_reg_test</code> in |
| in <code class="filename">testsuite/ext/pb_ds/regression/tree_no_data_map_rand.cc</code>. This uses <code class="classname">container_rand_regression_test</code> in |
| <code class="filename">testsuite/util/regression/rand/assoc/container_rand_regression_test.h</code>. |
| |
| </p><p> |
| Which has several tests for container member functions, |
| Includes control and test container objects. Configuration includes |
| random seed, iterations, number of distinct values, and the |
| probability that an exception will be thrown. Assumes instantiating |
| container uses an extension |
| allocator, <code class="classname">__gnu_cxx::throw_allocator_random</code>, |
| as the allocator type. |
| </p></li><li class="listitem"><p> |
| C++11 Container Requirements. |
| </p><p> |
| Coverage is currently limited to testing container |
| requirements for exception safety, |
| although <code class="classname">__gnu_cxx::throw_type</code> meets |
| the additional type requirements for testing numeric data |
| structures and instantiating algorithms. |
| </p><p> |
| Of particular interest is extending testing to algorithms and |
| then to parallel algorithms. Also io and locales. |
| </p><p> |
| The test instrumentation should also be extended to add |
| instrumentation to <code class="classname">iterator</code> |
| and <code class="classname">const_iterator</code> types that throw |
| conditionally on iterator operations. |
| </p></li></ul></div></div><div class="section"><div class="titlepage"><div><div><h5 class="title"><a id="test.exception.safety.containers"></a> |
| C++11 Requirements Test Sequence Descriptions |
| </h5></div></div></div><div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p> |
| Basic |
| </p><p> |
| Basic consistency on exception propagation tests. For |
| each container, an object of that container is constructed, |
| a specific member function is exercised in |
| a <code class="literal">try</code> block, and then any thrown |
| exceptions lead to error checking in the appropriate |
| <code class="literal">catch</code> block. The container's use of |
| resources is compared to the container's use prior to the |
| test block. Resource monitoring is limited to allocations |
| made through the container's <span class="type">allocator_type</span>, |
| which should be sufficient for container data |
| structures. Included in these tests are member functions |
| are <span class="type">iterator</span> and <span class="type">const_iterator</span> |
| operations, <code class="function">pop_front</code>, <code class="function">pop_back</code>, <code class="function">push_front</code>, <code class="function">push_back</code>, <code class="function">insert</code>, <code class="function">erase</code>, <code class="function">swap</code>, <code class="function">clear</code>, |
| and <code class="function">rehash</code>. The container in question is |
| instantiated with two instrumented template arguments, |
| with <code class="classname">__gnu_cxx::throw_allocator_limit</code> |
| as the allocator type, and |
| with <code class="classname">__gnu_cxx::throw_type_limit</code> as |
| the value type. This allows the test to loop through |
| conditional throw points. |
| </p><p> |
| The general form is demonstrated in |
| <code class="filename">testsuite/23_containers/list/requirements/exception/basic.cc |
| </code>. The instantiating test object is <code class="classname">__gnu_test::basic_safety</code> and is detailed in <code class="filename">testsuite/util/exception/safety.h</code>. |
| </p></li><li class="listitem"><p> |
| Generation Prohibited |
| </p><p> |
| Exception generation tests. For each container, an object of |
| that container is constructed and all member functions |
| required to not throw exceptions are exercised. Included in |
| these tests are member functions |
| are <span class="type">iterator</span> and <span class="type">const_iterator</span> operations, <code class="function">erase</code>, <code class="function">pop_front</code>, <code class="function">pop_back</code>, <code class="function">swap</code>, |
| and <code class="function">clear</code>. The container in question is |
| instantiated with two instrumented template arguments, |
| with <code class="classname">__gnu_cxx::throw_allocator_random</code> |
| as the allocator type, and |
| with <code class="classname">__gnu_cxx::throw_type_random</code> as |
| the value type. This test does not loop, an instead is sudden |
| death: first error fails. |
| </p><p> |
| The general form is demonstrated in |
| <code class="filename">testsuite/23_containers/list/requirements/exception/generation_prohibited.cc |
| </code>. The instantiating test object is <code class="classname">__gnu_test::generation_prohibited</code> and is detailed in <code class="filename">testsuite/util/exception/safety.h</code>. |
| </p></li><li class="listitem"><p> |
| Propagation Consistent |
| </p><p> |
| Container rollback on exception propagation tests. For |
| each container, an object of that container is constructed, |
| a specific member function that requires rollback to a previous |
| known good state is exercised in |
| a <code class="literal">try</code> block, and then any thrown |
| exceptions lead to error checking in the appropriate |
| <code class="literal">catch</code> block. The container is compared to |
| the container's last known good state using such parameters |
| as size, contents, and iterator references. Included in these |
| tests are member functions |
| are <code class="function">push_front</code>, <code class="function">push_back</code>, <code class="function">insert</code>, |
| and <code class="function">rehash</code>. The container in question is |
| instantiated with two instrumented template arguments, |
| with <code class="classname">__gnu_cxx::throw_allocator_limit</code> |
| as the allocator type, and |
| with <code class="classname">__gnu_cxx::throw_type_limit</code> as |
| the value type. This allows the test to loop through |
| conditional throw points. |
| </p><p> |
| The general form demonstrated in |
| <code class="filename">testsuite/23_containers/list/requirements/exception/propagation_coherent.cc |
| </code>. The instantiating test object is <code class="classname">__gnu_test::propagation_coherent</code> and is detailed in <code class="filename">testsuite/util/exception/safety.h</code>. |
| </p></li></ul></div></div></div></div></div><div class="navfooter"><hr /><table width="100%" summary="Navigation footer"><tr><td width="40%" align="left"><a accesskey="p" href="internals.html">Prev</a> </td><td width="20%" align="center"><a accesskey="u" href="appendix_porting.html">Up</a></td><td width="40%" align="right"> <a accesskey="n" href="abi.html">Next</a></td></tr><tr><td width="40%" align="left" valign="top">Porting to New Hardware or Operating Systems </td><td width="20%" align="center"><a accesskey="h" href="../index.html">Home</a></td><td width="40%" align="right" valign="top"> ABI Policy and Guidelines</td></tr></table></div></body></html> |
