4. Configure Test Systems Properly

In the course of a data-flow analysis, missing URL encodings or double-escaping of SQL statements become apparent. Multiple escapes can also arise through inattentiveness when, in your own test environment, PHP is run with disabled magic quotes that are then enabled on the production system. Production and test systems should both be run with the same versions and configuration files. A cron job that runs a simple diff via an SSH connection can ensure that the configurations are consistent.

5. Remember Debug and Error Messages

Security problems often arise from forgotten debug messages. The more detailed these messages are, the greater their usefulness for attackers. For instance, an attacker could discover database and table names from a debug message. A simple remedy against forgotten debug messages is to use a keyword prefix, which will make the messages easy to discover through grep. Alternatively, developers can integrate a special debug output function into their test system via include that is left blank in the production system.

Like debug messages, error messages that find their way to the outside world also help attackers. In addition to revealing security problems to intruders, technical messages confuse users, which has a negative effect on usability. Detailed log entries or automatic email sent to the admin are a better solution. Good documentation, which is a prerequisite for software testing, includes creating a list of possible error messages and deciding under what conditions the web application will receive them.

6. Construct a Test Scenario

Test scenarios are combinations of specific initial conditions designed to illicit predictable behavior the program should demonstrate. Such a test scenario for a calculator could be 2+2 with the expected output of 4. The addition scenario is obvious for the tester and thus easy to apply, but who adds (231 – 1)+1 to see whether a 32-bit signed integer overflows?

An organized test, therefore, includes determining the limits of the system and testing them. As a rule, it is not very effective if the same developers who wrote the application define the test scenario – their familiarity with the code makes it likely that they will be blind to their own mistakes.

Tests should be extensive; there are test scenarios for each individual function, for each module, and for the completed system. These tests should also deliberately cover extreme situations. Developers should also include tests for every known bug. If the calculator had ignored the multiplication before addition rule at some time in the past because of a coding error, this bug should be covered by a test scenario.

A measure for the quality of the testing is the amount of code covered. A coverage of 100 percent is desirable but not always possible. In event-driven programming, or with exception handlers for errors that are difficult to provoke, some branches of code will remain virtually unreachable.