Glossary

Term	Meaning
Ad hoc testing	Informal software testing that is outside a formal test plan. It can be used for the purpose of learning the system. A common use is for functional subject matter experts to “beat” on the system with odd scenarios that may hit in the real world (such as user errors and exceptional conditions) with the intent to try to “break the system.”
Black box testing	Testing based on business requirements and functionality without regard to internal design or code.
Boundary testing	Testing to determine if the system handles values that are outside set boundaries, for example: - Maximum negative, maximum positive, and “0” inputs or outputs - Input or output strings at size limits or one character beyond size limits - Empty input files and files with only one character in them
Capacity testing	Testing to determine capacity of hardware, servers
Defect	A fault or error. See software bug.
Dynamic testing	Testing involving execution of code
End-to-end testing	Testing a complete application environment from beginning to end with all programming, interfaces, network communications, hardware and applications operating to mimic a real world environment.
Function point	A measure for estimating software size
Functional testing	Black box type testing based on business requirements and functionality, without regard to internal design or code.
Integration testing	Testing integrated software where multiple units or modules are aggregated to create a larger component
Load testing	Testing the system by feeding it large tasks, such as: executing code that would cause generation of a large volume of forms; feeding an unusually large document to a word processor; sending empty tasks through (i.e. a report creation when no data can be found to populate it); executing a search that would require the system to read a high number of records.
Performance testing	Testing to eliminate bottlenecks and establish a baseline for future regression testing, such as testing for acceptable response times and ability to handle concurrent users
Recovery or Failover testing	Testing how well the system recovers from crashes, hardware failures or other catastrophic events.
Regression testing	Retesting the system after programming modifications to ensure that no new “bugs” have been introduced.
Requirements traceability	A proven engineering technique that links system requirements: — Backward to their sources (e.g., stakeholders, federal and State laws and regulations) — Forward to the resulting system development work products
Requirements traceability matrix (RTM)	A tool to implement requirements traceability during the SDLC. Can be manual or automated.
Risk	The possibility of suffering harm or loss. Risks have three elements: 1. Chance: What are the chances of risk happening? 2. Choice: What choices do you have to avoid or reduce risk? 3. Consequence: What are the consequences of risk happening?
Sanity or Smoke testing	Initial testing to determine whether a new software version is performing well enough to accept for a major testing effort.
Scalability testing	Testing whether the application can be scaled easily for increases or reductions in workload.
Security testing	Testing how well the system protects against unauthorized internal or external access and attempts made to compromise data, software or hardware.
Software bug	An error or mistake in a computer program that prevents it from working correctly or causes it to produce the wrong result. It may be a defect in the source code or the design.
Static testing	Walkthroughs and Inspections. Manual examination of project documentation such as requirements, use cases, design documents, program specifications, dataflow diagrams, architectural diagrams, etc. Most effective if completed by independent reviewer who was not involved in creation of deliverable.
Stress testing	Testing to check system performance in response to a failure – does it fail and recover gracefully? Test conditions could include: doubling the baseline number for concurrent users/HTTP connections; randomly shutting down and restarting ports on the network switches/routers that connect the servers; taking the database offline, then restarting it.
Structured testing	An approach to software testing with several characteristics: 1. Integrated into overall SACWIS Project Plan 2. Supported with strong organizational structures, defined processes, and appropriate resources, including budget, staff, and tools 3. Implemented throughout the SACWIS System Development Lifecycle (SDLC) 4. Is a proven strategy to manage risks to the quality of State SACWIS systems
System Development Lifecycle (SDLC)	A framework for system development with defined phases. A typical SACWIS SDLC has seven phases: Requirements, Analysis, Design, Code, Test, Implementation, and Maintenance.
System testing	Testing the whole system in its actual hardware environment with full integration
Test case	Describes the input, action, or event to be executed in a system and the expected system response. Is developed to test application features to verify that a system works as expected, based on the requirements. Is sometimes used interchangeably with “test script.”
Test scenario	A set of test cases that ensure that the business process flows are tested from end to end. They may be independent tests or a series of tests that follow each other, each dependent on the output of the previous one.
Test script	Describes (scripts out) the step-by-step process for executing a test case. Describes the expected system response to each individual step. Contains a place to log and explain the Pass/Fail status of the executed test. Is sometimes used interchangeably with “test case.”
Test strategy	The organization’s overall approach to testing. Includes the organization’s defined test methods, techniques, and tools.
Unit testing	Testing the smallest testable piece of software
Usability testing	Testing to determine if the system is user friendly and easy to navigate. Should be completed by functional staff.
Use case	Captures who (actor) does what (interaction) with the system, for what purpose (goal), without dealing with internal system information
User Acceptance testing	Testing by the functional subject matter experts (the customer) to ensure that the system (software and hardware) meets all user requirements and is ready for acceptance and deployment
White box testing	Testing based on knowledge of the internal logic of the application code. Also called structural testing or glass box testing.