Changes between Version 1 and Version 2 of Building/RunningTests/Adding

Timestamp:

07/08/11 15:49:38 (23 months ago)

Author:

dterei

Comment:

--

Building/RunningTests/Adding

@@ -28 +28 @@
 
  Under each main directory there are usually up to three subdirectories:
-       '''should_compile''':    
-           test cases which need to compile only
-       '''should_fail''':    
-           test cases which should fail to compile and generate a particular error message
-       '''should_run''':
-           test cases which should compile, run with some specific input, and generate a particular output.
+       '''should_compile''': test cases which need to compile only [[BR]]
+       '''should_fail''': test cases which should fail to compile and generate a particular error message [[BR]]
+       '''should_run''': test cases which should compile, run with some specific input, and generate a particular output.
     
  We don't always divide the test cases up like this, and it's not
@@ -361 +358 @@
 compiler version.  For example, the sample `stderr` for GHC 6.8.x would go in the file
 `T.stderr-ghc-6.8`.
-
-== The details ==
-
-The testsuite driver is just a set of Python scripts, as are all of
-the .T files in the testsuite.  The driver (driver/runtests.py) first
-searches for all the .T files it can find, and then proceeds to
-execute each one, keeping a track of the number of tests run, and
-which ones succeeded and failed.
-
-The script runtests.py takes several options:
-
-  --config <file>
-  
-       <file> is just a file containing Python code which is 
-       executed.   The purpose of this option is so that a file
-       containing settings for the configuration options can
-       be specified on the command line.  Multiple --config options
-       may be given.
-
-  --rootdir <dir>
-
-       <dir> is the directory below which to search for .T files
-       to run.
-
-  --output-summary <file>
-
-       In addition to dumping the test summary to stdout, also
-       put it in <file>.  (stdout also gets a lot of other output
-       when running a series of tests, so redirecting it isn't  
-       always the right thing).
-
-  --only <test>
-
-       Only run tests named <test> (multiple --only options can
-       be given).  Useful for running a single test from a .T file
-       containing multiple tests.
-
-  -e <stmt>
-
-       executes the Python statement <stmt> before running any tests.
-       The main purpose of this option is to allow certain
-       configuration options to be tweaked from the command line; for
-       example, the build system adds '-e config.accept=1' to the
-       command line when 'make accept' is invoked.
-
-Most of the code for running tests is located in driver/testlib.py.
-Take a look.
-
-There is a single Python class (TestConfig) containing the global
-configuration for the testsuite.  It contains information such as the
-kind of compiler being used, which flags to give it, which platform
-we're running on, and so on.  The idea is that each platform and
-compiler would have its own file containing assignments for elements
-of the configuration, which are sourced by passing the appropriate
---config options to the test driver.  For example, the GHC
-configuration is contained in the file config/ghc.
-
-A .T file can obviously contain arbitrary Python code, but the general
-idea is that it contains a sequence of calls to the function test(),
-which resides in testlib.py.  As described above, test() takes four
-arguments:
-
-      test(<name>, <opt-fn>, <test-fn>, <args>)
-
-The function <opt-fn> is allowed to be any Python callable object,
-which takes a single argument of type TestOptions.  TestOptions is a
-class containing options which affect the way that the current test is
-run: whether to skip it, whether to expect failure, extra options to
-pass to the compiler, etc. (see testlib.py for the definition of the
-TestOptions class).  The idea is that the <opt-fn> function modifies
-the TestOptions object that it is passed.  For example, to expect
-failure for a test, we might do this in the .T file:
-{{{
-   def fn(opts):
-      opts.expect = 'fail'
-
-   test(test001, fn, compile, [''])
-}}}
-so when fn is called, it sets the instance variable "expect" in the
-instance of TestOptions passed as an argument, to the value 'fail'.
-This indicates to the test driver that the current test is expected to
-fail.
-
-Some of these functions, such as the one above, are common, so rather
-than forcing every .T file to redefine them, we provide canned
-versions.  For example, the provided function expect_fail does the
-same as fn in the example above.  See testlib.py for all the canned
-functions we provide for <opt-fn>.
-
-The argument <test-fn> is a function which performs the test.  It
-takes three or more arguments:
-
-      <test-fn>( <name>, <way>, ... )
-
-where <name> is the name of the test, <way> is the way in which it is
-to be run (eg. opt, optasm, prof, etc.), and the rest of the arguments
-are constructed from the list <args> in the original call to test().
-The following <test-fn>s are provided at the moment:
-
-           compile
-           compile_fail
-           compile_and_run
-           multimod_compile
-           multimod_compile_fail
-           multimod_compile_and_run
-           run_command
-           run_command_ignore_output
-           ghci_script
-
-and obviously others can be defined.  The function should return
-either 'pass' or 'fail' indicating that the test passed or failed
-respectively.