Re: [PHP-DEV] 4.0.2 Release date From: Stig Bakken (ssb <email protected>)
Date: 08/06/00

Manuel Lemos wrote:
>
> Hello Stig,
>
> On 04-Aug-00 09:48:20, you wrote:
>
> >PHP has been in need of a fully automated regression testing system
> >again for a while. We used to have one, but most people find tests
> >almost as boring to write as documentation, so it was dissolved by the
> >time we released 3.0.
>
> >I've started experimenting with a framework for this in the pear subdir
> >(see pear/run-tests.in). It's modular by defining a subdir in every dir
> >called "tests" where test scripts (written in PHP) and their expected
> >output are stored in files called "testname.t" and "testname.o" (plus
> >maybe a "testname.i" file if the test needs file input).
>
> >What if the QA team members, whenever using test scripts to verify that
> >a bug is really gone, put their test code into this or a similar
> >structure, so we can "re-use" your efforts automatically? That way we
> >at least get regression tests for all bugs that are fixed.
>
> >I assume I'm not the only one who finds pleasure in seeing screenfull
> >upon screenfull of "test passed" ;-).
>
> That's a great thing to have regression tests back in PHP. I have
> developed my own test suite to verify the conformance of Metabase database
> interface driver classes and it has been great not only find the many bugs
> in the driver classes, but also in PHP and some times some less known
> limitations the the database servers themselves.
>
> Now, a greater challenge: many existing bugs lead PHP to crash. How do
> you (plan to) handle crashes in your test suite? Since you use PHP scripts
> to test features I supposed you have to use something external, like shell
> scripts or PHP scripts running independently of the actual test scripts, to
> detect failures that lead to crashes.

The idea is very simple: a PHP/shell script runs the PHP interpreter
(with a dedicated php.ini file and/or customization directives on the
command line) once for each test. The stdout of the test is then
compared to the expected output, if they differ, the test fails. If it
gets harder than this, I fear people won't bother writing tests.

 - Stig 

-- 
PHP Development Mailing List <http://www.php.net/>
To unsubscribe, e-mail: php-dev-unsubscribe <email protected>
For additional commands, e-mail: php-dev-help <email protected>
To contact the list administrators, e-mail: php-list-admin <email protected>