Re: [gentoo-perl] test

[Chris White]

On Thursday 07 September 2006 10:29, Michael Cummings wrote:
> dams wrote:
> > test
>
> NAME
>     Test - provides a simple framework for writing test scripts
>
> SYNOPSIS
>       use strict;
>       use Test;
>
>       # use a BEGIN block so we print our plan before MyModule is loaded
>       BEGIN { plan tests => 14, todo => [3,4] }
>
>       # load your module...
>       use MyModule;
>
>       # Helpful notes.  All note-lines must start with a "#".
>       print "# I'm testing MyModule version $MyModule::VERSION\n";
>
>       ok(0); # failure
>       ok(1); # success
>
>       ok(0); # ok, expected failure (see todo list, above)
>       ok(1); # surprise success!
>
>       ok(0,1);             # failure: '0' ne '1'
>       ok('broke','fixed'); # failure: 'broke' ne 'fixed'
>       ok('fixed','fixed'); # success: 'fixed' eq 'fixed'
>       ok('fixed',qr/x/);   # success: 'fixed' =~ qr/x/
>
>       ok(sub { 1+1 }, 2);  # success: '2' eq '2'
>       ok(sub { 1+1 }, 3);  # failure: '2' ne '3'
>
>       my @list = (0,0);
>       ok @list, 3, "\@list=".join(',',@list);      #extra notes
>       ok 'segmentation fault', '/(?i)success/';    #regex match
>
>       skip(
>         $^O =~ m/MSWin/ ? "Skip if MSWin" : 0,  # whether to skip
>         $foo, $bar  # arguments just like for ok(...)
>       );
>       skip(
>         $^O =~ m/MSWin/ ? 0 : "Skip unless MSWin",  # whether to skip
>         $foo, $bar  # arguments just like for ok(...)
>       );
>
> DESCRIPTION
>     This module simplifies the task of writing test files for Perl modules,
>     such that their output is in the format that Test::Harness expects to
>     see.
>
> QUICK START GUIDE
>     To write a test for your new (and probably not even done) module,
> create a new file called t/test.t (in a new t directory). If you have
> multiple test files, to test the "foo", "bar", and "baz" feature sets, then
> feel free to call your files t/foo.t, t/bar.t, and t/baz.t
>
>   Functions
>     This module defines three public functions, "plan(...)", "ok(...)", and
>     "skip(...)". By default, all three are exported by the "use Test;"
>     statement.
>
>     "plan(...)"
>              BEGIN { plan %theplan; }
>
>         This should be the first thing you call in your test script. It
>         declares your testing plan, how many there will be, if any of them
>         should be allowed to fail, and so on.
>
>         Typical usage is just:
>
>              use Test;
>              BEGIN { plan tests => 23 }
>
>         These are the things that you can put in the parameters to plan:
>
>         "tests => *number*"
>             The number of tests in your script. This means all ok() and
>             skip() calls.
>
>         "todo => [*1,5,14*]"
>             A reference to a list of tests which are allowed to fail. See
>             "TODO TESTS".
>
>         "onfail => sub { ... }"
>         "onfail => \&some_sub"
>             A subroutine reference to be run at the end of the test script,
>             if any of the tests fail. See "ONFAIL".
>
>         You must call "plan(...)" once and only once. You should call it in
>         a "BEGIN {...}" block, like so:
>
>              BEGIN { plan tests => 23 }
>
>     "ok(...)"
>           ok(1 + 1 == 2);
>           ok($have, $expect);
>           ok($have, $expect, $diagnostics);
>
>         This function is the reason for "Test"'s existence. It's the basic
>         function that handles printing ""ok"" or ""not ok"", along with the
>         current test number. (That's what "Test::Harness" wants to see.)
>
>         In its most basic usage, "ok(...)" simply takes a single scalar
>         expression. If its value is true, the test passes; if false, the
>         test fails. Examples:
>
>             # Examples of ok(scalar)
>
>             ok( 1 + 1 == 2 );           # ok if 1 + 1 == 2
>             ok( $foo =~ /bar/ );        # ok if $foo contains 'bar'
>             ok( baz($x + $y) eq 'Armondo' );    # ok if baz($x + $y)
> returns # 'Armondo'
>             ok( @a == @b );             # ok if @a and @b are the same
> length
>
>         The expression is evaluated in scalar context. So the following
> will work:
>
>             ok( @stuff );                       # ok if @stuff has any
> elements
>             ok( !grep !defined $_, @stuff );    # ok if everything in
> @stuff is
>                                                 # defined.
>
>         A special case is if the expression is a subroutine reference (in
>         either "sub {...}" syntax or "\&foo" syntax). In that case, it is
>         executed and its value (true or false) determines if the test
> passes or fails. For example,
>
>             ok( sub {   # See whether sleep works at least passably
>               my $start_time = time;
>               sleep 5;
>               time() - $start_time  >= 4
>             });
>
>         In its two-argument form, "ok(*arg1*, *arg2*)" compares the two
>         scalar values to see if they match. They match if both are
>         undefined, or if *arg2* is a regex that matches *arg1*, or if they
>         compare equal with "eq".
>
>             # Example of ok(scalar, scalar)
>
>             ok( "this", "that" );               # not ok, 'this' ne 'that'
>             ok( "", undef );                    # not ok, "" is defined
>
>         The second argument is considered a regex if it is either a regex
>         object or a string that looks like a regex. Regex objects are
>         constructed with the qr// operator in recent versions of perl. A
>         string is considered to look like a regex if its first and last
>         characters are "/", or if the first character is "m" and its second
>         and last characters are both the same non-alphanumeric
>         non-whitespace character. These regexp
>
>         Regex examples:
>
>             ok( 'JaffO', '/Jaff/' );    # ok, 'JaffO' =~ /Jaff/
>             ok( 'JaffO', 'm|Jaff|' );   # ok, 'JaffO' =~ m|Jaff|
>             ok( 'JaffO', qr/Jaff/ );    # ok, 'JaffO' =~ qr/Jaff/;
>             ok( 'JaffO', '/(?i)jaff/ ); # ok, 'JaffO' =~ /jaff/i;
>
>         If either (or both!) is a subroutine reference, it is run and used
>         as the value for comparing. For example:
>
>             ok sub {
>                 open(OUT, ">x.dat") || die $!;
>                 print OUT "\x{e000}";
>                 close OUT;
>                 my $bytecount = -s 'x.dat';
>                 unlink 'x.dat' or warn "Can't unlink : $!";
>                 return $bytecount;
>               },
>               4
>             ;
>
>         The above test passes two values to "ok(arg1, arg2)" -- the first a
>         coderef, and the second is the number 4. Before "ok" compares them,
>         it calls the coderef, and uses its return value as the real value
> of this parameter. Assuming that $bytecount returns 4, "ok" ends up testing
> "4 eq 4". Since that's true, this test passes.
>
>         Finally, you can append an optional third argument, in
>         "ok(*arg1*,*arg2*, *note*)", where *note* is a string value that
>         will be printed if the test fails. This should be some useful
>         information about the test, pertaining to why it failed, and/or a
>         description of the test. For example:
>
>             ok( grep($_ eq 'something unique', @stuff), 1,
>                 "Something that should be unique isn't!\n".
>                 '@stuff = '.join ', ', @stuff
>               );
>
>         Unfortunately, a note cannot be used with the single argument style
>         of "ok()". That is, if you try "ok(*arg1*, *note*)", then "Test"
>         will interpret this as "ok(*arg1*, *arg2*)", and probably end up
>         testing "*arg1* eq *arg2*" -- and that's not what you want!
>
>         All of the above special cases can occasionally cause some
> problems. See "BUGS and CAVEATS".
>
>     "skip(*skip_if_true*, *args...*)"
>         This is used for tests that under some conditions can be skipped.
>         It's basically equivalent to:
>
>           if( $skip_if_true ) {
>             ok(1);
>           } else {
>             ok( args... );
>           }
>
>         ...except that the ok(1) emits not just ""ok *testnum*"" but
>         actually ""ok *testnum* # *skip_if_true_value*"".
>
>         The arguments after the *skip_if_true* are what is fed to "ok(...)"
>         if this test isn't skipped.
>
>         Example usage:
>
>           my $if_MSWin =
>             $^O =~ m/MSWin/ ? 'Skip if under MSWin' : '';
>
>           # A test to be skipped if under MSWin (i.e., run except under
> MSWin)
>           skip($if_MSWin, thing($foo), thing($bar) );
>
>         Or, going the other way:
>
>           my $unless_MSWin =
>             $^O =~ m/MSWin/ ? '' : 'Skip unless under MSWin';
>
>           # A test to be skipped unless under MSWin (i.e., run only
> under MSWin)
>           skip($unless_MSWin, thing($foo), thing($bar) );
>
>         The tricky thing to remember is that the first parameter is true if
>         you want to *skip* the test, not *run* it; and it also doubles as a
>         note about why it's being skipped. So in the first codeblock above,
>         read the code as "skip if MSWin -- (otherwise) test whether
>         "thing($foo)" is "thing($bar)"" or for the second case, "skip
> unless MSWin...".
>
>         Also, when your *skip_if_reason* string is true, it really should
>         (for backwards compatibility with older Test.pm versions) start
> with the string "Skip", as shown in the above examples.
>
>         Note that in the above cases, "thing($foo)" and "thing($bar)" *are*
>         evaluated -- but as long as the "skip_if_true" is true, then we
>         "skip(...)" just tosses out their value (i.e., not bothering to
>         treat them like values to "ok(...)". But if you need to *not* eval
>         the arguments when skipping the test, use this format:
>
>           skip( $unless_MSWin,
>             sub {
>               # This code returns true if the test passes.
>               # (But it doesn't even get called if the test is skipped.)
>               thing($foo) eq thing($bar)
>             }
>           );
>
>         or even this, which is basically equivalent:
>
>           skip( $unless_MSWin,
>             sub { thing($foo) }, sub { thing($bar) }
>           );
>
>         That is, both are like this:
>
>           if( $unless_MSWin ) {
>             ok(1);  # but it actually appends "# $unless_MSWin"
>                     #  so that Test::Harness can tell it's a skip
>           } else {
>             # Not skipping, so actually call and evaluate...
>             ok( sub { thing($foo) }, sub { thing($bar) } );
>           }
>
> TEST TYPES
>     * NORMAL TESTS
>         These tests are expected to succeed. Usually, most or all of your
>         tests are in this category. If a normal test doesn't succeed, then
>         that means that something is *wrong*.
>
>     * SKIPPED TESTS
>         The "skip(...)" function is for tests that might or might not be
>         possible to run, depending on the availability of platform-specific
>         features. The first argument should evaluate to true (think "yes,
>         please skip") if the required feature is *not* available. After the
>         first argument, "skip(...)" works exactly the same way as "ok(...)"
>         does.
>
>     * TODO TESTS
>         TODO tests are designed for maintaining an executable TODO list.
>         These tests are *expected to fail.* If a TODO test does succeed,
>         then the feature in question shouldn't be on the TODO list, now
>         should it?
>
>         Packages should NOT be released with succeeding TODO tests. As soon
>         as a TODO test starts working, it should be promoted to a normal
>         test, and the newly working feature should be documented in the
>         release notes or in the change log.
>
> ONFAIL
>       BEGIN { plan test => 4, onfail => sub { warn "CALL 911!" } }
>
>     Although test failures should be enough, extra diagnostics can be
>     triggered at the end of a test run. "onfail" is passed an array ref of
>     hash refs that describe each test failure. Each hash will contain at
>     least the following fields: "package", "repetition", and "result". (You
>     shouldn't rely on any other fields being present.) If the test had an
>     expected value or a diagnostic (or "note") string, these will also be
>     included.
>
>     The *optional* "onfail" hook might be used simply to print out the
>     version of your package and/or how to report problems. It might also be
>     used to generate extremely sophisticated diagnostics for a particularly
>     bizarre test failure. However it's not a panacea. Core dumps or other
>     unrecoverable errors prevent the "onfail" hook from running. (It is run
>     inside an "END" block.) Besides, "onfail" is probably over-kill in most
>     cases. (Your test code should be simpler than the code it is testing,
>     yes?)
>
> BUGS and CAVEATS
>     *   "ok(...)"'s special handing of strings which look like they might
> be regexes can also cause unexpected behavior. An innocent:
>
>             ok( $fileglob, '/path/to/some/*stuff/' );
>
>         will fail, since Test.pm considers the second argument to be a
>         regex! The best bet is to use the one-argument form:
>
>             ok( $fileglob eq '/path/to/some/*stuff/' );
>
>     *   "ok(...)"'s use of string "eq" can sometimes cause odd problems
> when comparing numbers, especially if you're casting a string to a number:
>
>             $foo = "1.0";
>             ok( $foo, 1 );      # not ok, "1.0" ne 1
>
>         Your best bet is to use the single argument form:
>
>             ok( $foo == 1 );    # ok "1.0" == 1
>
>     *   As you may have inferred from the above documentation and examples,
>         "ok"'s prototype is "($;$$)" (and, incidentally, "skip"'s is
>         "($;$$$)"). This means, for example, that you can do "ok @foo,
> @bar" to compare the *size* of the two arrays. But don't be fooled into
> thinking that "ok @foo, @bar" means a comparison of the contents of two
> arrays -- you're comparing *just* the number of elements of each. It's so
> easy to make that mistake in reading "ok @foo, @bar" that you might want to
> be very explicit about it, and instead write "ok scalar(@foo),
> scalar(@bar)".
>
>     *   This almost definitely doesn't do what you expect:
>
>              ok $thingy->can('some_method');
>
>         Why? Because "can" returns a coderef to mean "yes it can (and the
>         method is this...)", and then "ok" sees a coderef and thinks you're
>         passing a function that you want it to call and consider the truth
>         of the result of! I.e., just like:
>
>              ok $thingy->can('some_method')->();
>
>         What you probably want instead is this:
>
>              ok $thingy->can('some_method') && 1;
>
>         If the "can" returns false, then that is passed to "ok". If it
>         returns true, then the larger expression
>         "$thingy->can('some_method') && 1" returns 1, which "ok" sees as a
>         simple signal of success, as you would expect.
>
>     *   The syntax for "skip" is about the only way it can be, but it's
>         still quite confusing. Just start with the above examples and
> you'll be okay.
>
>         Moreover, users may expect this:
>
>           skip $unless_mswin, foo($bar), baz($quux);
>
>         to not evaluate "foo($bar)" and "baz($quux)" when the test is being
>         skipped. But in reality, they *are* evaluated, but "skip" just
> won't bother comparing them if $unless_mswin is true.
>
>         You could do this:
>
>           skip $unless_mswin, sub{foo($bar)}, sub{baz($quux)};
>
>         But that's not terribly pretty. You may find it simpler or clearer
>         in the long run to just do things like this:
>
>           if( $^O =~ m/MSWin/ ) {
>             print "# Yay, we're under $^O\n";
>             ok foo($bar), baz($quux);
>             ok thing($whatever), baz($stuff);
>             ok blorp($quux, $whatever);
>             ok foo($barzbarz), thang($quux);
>           } else {
>             print "# Feh, we're under $^O.  Watch me skip some tests...\n";
>             for(1 .. 4) { skip "Skip unless under MSWin" }
>           }
>
>         But be quite sure that "ok" is called exactly as many times in the
>         first block as "skip" is called in the second block.
>
> ENVIRONMENT
>     If "PERL_TEST_DIFF" environment variable is set, it will be used as a
>     command for comparing unexpected multiline results. If you have GNU
> diff installed, you might want to set "PERL_TEST_DIFF" to "diff -u". If you
> don't have a suitable program, you might install the "Text::Diff" module
> and then set "PERL_TEST_DIFF" to be "perl -MText::Diff -e 'print
> diff(@ARGV)'". If "PERL_TEST_DIFF" isn't set but the "Algorithm::Diff"
> module is available, then it will be used to show the differences in
> multiline results.
>
> NOTE
>     A past developer of this module once said that it was no longer being
>     actively developed. However, rumors of its demise were greatly
>     exaggerated. Feedback and suggestions are quite welcome.
>
>     Be aware that the main value of this module is its simplicity. Note
> that there are already more ambitious modules out there, such as Test::More
> and Test::Unit.
>
>     Some earlier versions of this module had docs with some confusing
> typoes in the description of "skip(...)".
>
> SEE ALSO
>     Test::Harness
>
>     Test::Simple, Test::More, Devel::Cover
>
>     Test::Builder for building your own testing library.
>
>     Test::Unit is an interesting XUnit-style testing library.
>
>     Test::Inline and SelfTest let you embed tests in code.
>
> AUTHOR
>     Copyright (c) 1998-2000 Joshua Nathaniel Pritikin. All rights reserved.
>
>     Copyright (c) 2001-2002 Michael G. Schwern.
>
>     Copyright (c) 2002-2004 and counting Sean M. Burke.
>
>     Current maintainer: Sean M. Burke. <sburke@××××.org>
>
>     This package is free software and is provided "as is" without express
> or implied warranty. It may be used, redistributed and/or modified under
> the same terms as Perl itself.
>
>
>
> --
>
> -----o()o----------------------------------------------
> Michael Cummings   |    #gentoo-dev, #gentoo-perl
> Gentoo Perl Dev    |    on irc.freenode.net
> Gentoo/SPARC
> Gentoo/AMD64
> GPG: 0543 6FA3 5F82 3A76 3BF7  8323 AB5C ED4E 9E7F 4E2E
> -----o()o----------------------------------------------

I agree.  (sorry couldn't help it ;p)

-- 
Chris White
Gentoo Developer aka:
ChrisWhite		cpw
ChrisWhite|Work	WhiteChocolate
VanillaWhite		Whitey
WhiteLight		WhiteCheese
WhiteSugar		WhiteButter
WhiteWall		WhiteLemon
WhiteApple		WhiteBlanket
WhiteEnergy		WhiteWhite