Software Testing & Quality Assurance Slides - Week 4
Date Published: 05/05/2019
These slides explain the important concept of different testing types i.e. Black and White box testing and when and how we do it with an example. We will learn about the functional and non-functional testing type and will then understand the different Black box (functional and non-functional) testing types e.g. Integration, Stress, Regression, and Integration testing.

Summary of the Last Class
  • Snagit Tool
  • Test Cases Execution
  • Defect Report
  • Defect Life Cycle
  • Simple Web Architecture

    • Software Testing Types
      • Black Box Testing
      • White Box Testing
      • Functional Vs Non-Function Testing
      • integration Testing
      • Regression Testing
      • Stress Testing
    • V Model

      Software Testing Types
      • There are well over 150 types of software testing. i.e. A software can be tested in 150 different ways
      • You do not need to memorize all software testing types
      • We will learn a few important types and will practice them.

        Black Box Testing
        • Black Box testing means what is going into the system and what is coming out. i.e. what does system do when a specific action is taken.
        • The functionality of the Software Under Test (SUT) is tested without looking at the code or implementation details.
        • Focus in only on inputs and outputs

        • For example, You don’t care about code or programming language when testing the login functionality of Photo-World application with valid Email/Password.

          Black Box Testing Types

          Generally, there are three classifications of testing:  (General Classification)

            Functional Testing
            • Functional Testing verifies that each function of the software application works according to the requirements.
            • Each and every functionality of the system is tested by providing appropriate input, verifying the output and comparing the actual results with the expected results.
            • This testing involves checking of User Interface, APIs, database, security, client/server applications and functionality of the Application Under Test.
            • Can be done either Manually or by using Automation Tools.

              What do you test in Functional Testing?
              • Mainline functions:  Testing the main functions of an application
              • Basic Usability: It involves basic usability testing of the system. In simple words, is application easy to use?. 
              • Error Conditions: Usage of testing techniques to check for error conditions.  It checks whether suitable error messages are displayed. E.g. for Login page, if your Email address is incorrect. The appropriate message can be:

                    invalid Email Address!

                Functional Testing Process (how to)
                • Understand the Requirements. (From BRD & SRS documents)
                • Identify test input (test data).
                • Compute the expected outcomes with the selected test input value.
                • Execute test cases. 
                • Comparison of expected and computed an expected result.
                • If the expected output is equal to computed, the test is Passed, otherwise Failed.
                • This is what we have been doing so far.

                  Non-Functional Testing
                  • Non-functional requirements like performance, usability.
                  • Many times you have come across to long load time messages while accessing an application.

                  • To address this issue, performance testing is carried out to check & fine tune system response times.
                  • The goal of performance testing is to reduce the response time of application to an acceptable level. 

                    Functional Testing Types
                    • There are many types of functional testing.
                    • We will explore some of them briefly today.
                      • Unit Testing
                      • Integration Testing
                      • Smoke / Sanity Testing
                      • User Acceptance
                      • Localization
                      • Globalization
                      • Etc ….

                      Integration Testing
                      • A typical software project consists of multiple software modules, coded by different programmers.
                      • Integration testing focuses on checking data communication amongst these modules.
                      • Example
                        • Photo-World application has Login, Register, Admin Area, Download pic, etc modules
                        • Each module may have been coded by a different programmer.
                        • Integration testing case may be to add a gallery by logging in as admin and verify that it shows up for end users
                      • It is also termed as 'I & T' (Integration and Testing), 'String Testing' and sometimes 'Thread Testing'.

                        Why Integration Testing is Important?
                        • A Module, in general, is designed by an individual software developer whose understanding and programming logic may differ from other programmers. 
                          • Integration testing becomes necessary to verify the software modules work in unity.
                        • At the time of module development, there are wide chances of change in requirements by the clients. 
                          • These new requirements may not be unit tested by the developer and hence integration testing becomes necessary.

                          System Testing
                          • System testing is the testing of a complete and fully integrated software product.
                            • The software may be built as individual components and integrated later once all component builds successfully to make complete software application.
                          • System testing is a type of Black Box Testing.
                          • What do you verify in System Testing?
                            • Testing the fully integrated applications including external peripherals in order to check how components interact with one another and with the system as a whole. This is also called End to End scenario testing.
                            • Verify thorough testing of every input in the application to check for desired outputs.
                            • Testing of the user's experience with the application. (How easy to use and navigate application)

                            System Testing Types
                            • Usability Testing: Test how easy to use application 
                              • E.g. Google search engine has simple UI and very easy to use.
                            • Load Testing: Test how the software will behave under real-world traffic load.
                              • E.g. Amazon, eBay, Google may have millions of users at a time.
                            • Regression Testing: Test if there is no new defect/bug introduced after the developer fixes the previous defect/bug.
                            • Functional Testing: Test to check software meets the client requirements.

                              Regression Testing
                              • The purpose of regression testing is to confirm that a recent program or code change has not affected existing features.
                              • This testing is done to make sure that new code changes should not have side effects on the existing functionalities.
                              • It ensures that the old code still works once the new code changes are done.
                              • When to do Regression Testing?
                                • Change in requirements and code is modified according to the requirement.
                                • A new feature is added to the software.
                                • Defect fixing.
                                • Performance issue fix. 
                              • Regression Testing Technique
                                • Retest All: Test All test cases, expensive and need more time & resources.
                                • Regression Test Selection: Select part of the test case to be executed.
                                • Prioritize Test case: Prioritize test case based on business impact, critical & frequently used functionalities.

                                GUI Testing
                                • GUI stands for Graphical User Interface.
                                • Graphical User Interface (GUI) testing is the process of testing the system's GUI of the System Under Test (SUT).
                                • GUI testing involves checking if font size and style are consistent, colors are consistent, controls are aligned properly, controls sizes are consistent, etc
                                • GUI Elements:

                                  GUI Testing Checklist
                                  • Check all the GUI elements for size, position, width, length, and acceptance of characters or numbers. For instance, you must be able to provide inputs to the input fields.
                                  • Check you can execute the intended functionality of the application using the GUI.
                                  • Check Error Messages are displayed correctly.
                                  • Check for Clear separation of different sections on the screen.
                                  • Check Font used in the application is readable.
                                  • Check the alignment of the text is proper.
                                  • Check the Color of the font and warning messages are aesthetically pleasing.
                                  • Check that the images have good clarity.
                                  • Check that the images are properly aligned.
                                  • Check the positioning of GUI elements for different screen resolution

                                    Stress Testing
                                    • Stress testing is used to test the stability & reliability of the system.
                                    • This test mainly determines the system on its robustness and error handling under extremely heavy load conditions.
                                    • A most prominent use of stress testing is to determine the limit, at which the system or software or hardware breaks.
                                    • For example, Notepad is a small application that is designed to store a few MBs of data, if you try to copy 5 GB data in Notepad, it will show the following error message:

                                      Why Stress Testing is Required?
                                      • To check whether the system works under abnormal conditions.
                                      • Displaying appropriate error message when the system is under stress.
                                      • For example, you might have seen the message “we are experiencing heavy traffic this time, please come back later” message.
                                      • System failure under extreme conditions could result in huge revenue loss
                                      • For example, online shopping portals.
                                      • It is better to be prepared for extreme conditions by executing Stress Testing. 

                                      The goal of stress testing is to analyze the behavior of the system after a failure. For stress testing to be successful, the system should display an appropriate error message while it is under extreme conditions.

                                        Smoke and Sainty Testing
                                        • Smoke testing is done after the project is complete to make sure critical functionalities are working before handing over the project to the QA team for testing.
                                          • The purpose is to reject a badly broken application so that the QA team does not waste time installing and testing the software application.
                                          • For example, you start a website and found a totally blank page. No need to move further.
                                          • Done by Developers.
                                        • Sanity testing is done to make sure that after fixing a bug, no further issues are introduced due to that fix.
                                          • The goal is to determine that the proposed functionality works roughly as expected. If sanity test fails, the build is rejected to save the time and costs involved in more rigorous testing.
                                          • For example, after fixing the bug if you result like 2+2 = 5. There is no need to do further testing to verify what is the value of sin10+cos120*10+log20 etc.
                                          • Done by Testers.

                                          Static Testing vs Dynamic Testing
                                          • Under Static Testing, the code is not executed. Rather it manually checks the code, requirement documents, and design documents to find errors. 
                                          • The main objective of this testing is to improve the quality of software products by finding errors in the early stages of the development cycle. This testing is also called a Non-execution technique or verification testing.
                                            • For example, Requirement, Design, Test Plan, Test Cases reviews.
                                          • Under Dynamic Testing, the code is executed. It checks for functional behavior of software system, memory/CPU usage and overall performance of the system.

                                            White Box Testing
                                            • White Box Testing is the testing of a software solution's internal coding and infrastructure.
                                            • It focuses primarily on improving security, the flow of inputs and outputs through the application, and improving design and usability (how easy to use the application).
                                            • Also known as clear, open, structural, and glass box testing.
                                            • Usually done by Developers.
                                            • Example
                                              • Unit Testing

                                              White Box Testing - Unit Testing
                                              • Unit testing is done during actual development (coding) of software.
                                              • Unit testing is used to test only one module or function. It is used to check the correctness of an individual piece of code.
                                              • Proper unit testing is a great asset to avoid lots of bugs that introduce during integration and system testing
                                              • Done by Developers.
                                              • Software Testers do not need to worry about it.

                                                Software Development Life Cycle [SDLC] – Water Fall Model

                                                  Waterfall Model Problems
                                                  • The SDLC in the previous slide is called “Waterfall Model”.
                                                    • Contains sequential activities. 
                                                    • You cannot go back to the previous activity.
                                                    • Assumes requirements are frozen and do not change frequently. 
                                                  • “Verification” phase involves all QA and testing phase where you check for defects and verify system according to client requirements. 
                                                  • The problem in the Waterfall Model
                                                    • In a complex project, if you miss any requirement or design something wrong then you cannot get to know until the “Verification” phase.
                                                    • Research proves that “defects introduced during requirements & design make up close to half of the total number of defects”.

                                                    • The costs of fixing a defect increases across the development life cycle. The earlier in the life cycle a defect is detected, the cheaper it is to fix it.
                                                    • To address the Waterfall Model problem, the V model of testing was developed for every phase, in the Development life cycle, there is a corresponding Testing phase.

                                                      V Model Testing

                                                      • The left side is SDLC and the right side is STLC (Software Test Life Cycle).
                                                      • V Model: Every phase should be tested until the final product development.

                                                        V Model Testing (V-Shaped SDLC)
                                                        • Advantages
                                                          • Simple and easy to use.
                                                          • Testing activates happen parallel to development, save time and a higher chance of success over Waterfall SDLC.
                                                          • Early defect tracking & avoid downwards flow of the defects.
                                                          • Good for small project with well-understood client’s requirements.
                                                        • Disadvantages
                                                          • Less flexible and rigid.
                                                          • No prototype since the software is developed in the implementation phase.
                                                          • Requirements change also affect the corresponding testing documents update.
                                                        • When to Use
                                                          • Small to medium size project with well-understood requirements.
                                                          • Have highly skilled technical resources.

                                                          Keywords: black and white box testing, functional and non functional testing, integration testing, stress testing, regression testing, v model, software testing and quality assurance