CPT203-Software Engineering

1. Introduction

1.1 Computer Software:

Computer software is the product that software professionals build and then support over the long term. It encompasses computer programs and associated documentation that execute within a computer of any size and architecture . Software products may be developed for a particular customer or may be developed for a general market.

Why is it important?

Software is important because it affects nearly every aspect of our lives and has become pervasive in our commerce, our culture, and our everyday activities.

• National infrastructures and utilities are controlled by computer-based systems and most electrical products include a computer and controlling software.
• Industrial manufacturing and distribution is completely computerized, as is the financial system.
• Entertainment, including the music industry, computer games, and film and television, is software intensive.

1.1.1 Software system

Software systems are abstract and intangible. They are not constrained by the properties of materials, governed by physical laws, or by manufacturing processes.

The implication of software system:
This simplifies software engineering, as there are no natural limits to the potential of software. However, because of the lack of physical constraints, software systems can quickly become extremely complex, difficult to understand, and expensive to change.

1.2 Software Failure & Success

Software engineering is criticized as inadequate for modern software development. Software failures are a consequence of two factors:

• Increasing demands
• Low expectations

1.3 Professional Software Development

• Amateur
  • People in business write spreadsheet programs to simplify their jobs.
  • Scientists and engineers write programs to process their experimental data.
  • Hobbyists write programs for their own interest and enjoyment.
• Professional
  • developed for specific business purposes, for inclusion in other devices, or as software products.
  • Intended for use by someone apart from its developer.
  • It is usually developed by teams rather than individuals.
  • It is maintained and changed throughout its life.

Software engineering is intended to support professional software development, rather than individual programming. It includes techniques that support program specification, design, validation, and evolution. Professional software usually has the following properties:

• Strict user requirements
• Required accuracy and data integrity
• Higher security standard
• Stable performance for heavy load
• Required technical support, etc.

Some product characteristics:

1. Maintainability: Software should be written in such a way so that it can evolve to meet the changing needs of customers. This is a critical attribute because software change is an inevitable requirement of a changing business environment.
2. Dependability and Security: Software dependability includes a range of characteristics including reliability, security, and safety. Dependable software should not cause physical or economic damage in the event of system failure. Malicious users should not be able to access or damage the system.
3. Efficiency: Software should not make wasteful use of system resources such as memory and processor cycles. Efficiency therefore includes responsiveness, processing time, memory utilization, etc.
4. Acceptability: Software must be acceptable to the type of users for which it is designed. This means that it must be understandable, usable, and compatible with other systems that they use.

Two kinds of software products

• Generic software products: These are systems that are produced by a development organization and sold on the open market to any customer who is able to buy them.
• Customized software products: These are systems that are commissioned by a particular customer. A software contractor develops the software especially for that customer.

1.4 Software Deterioration

Software is a logical rather than a physical system element. Therefore, software has one fundamental characteristic that makes it considerably different from hardware: Software doesn’t “wear out.”

Software doesn’t wear out. But it does deteriorate. During its life, software will undergo change. As changes are made, it is likely that errors will be introduced, causing the failure rate curve to spike as shown in the “actual curve”

Before the curve can return to the original steady-state failure rate, another change is requested, causing the curve to spike again.

1.5 Software Engineering

Software engineering is an engineering discipline that is concerned with all aspects of software production from the early stages of system specification through to maintaining the system after it has gone into use.

Engineering discipline
Engineers make things work. They apply theories, methods, and tools where these are appropriate. However, they use them selectively and always try to discover solutions to problems even when there are no applicable theories and methods. Engineers also recognize that they must work to organizational and financial constraints so they look for solutions within these constraints.

All aspects of software production
Software engineering is not just concerned with the technical processes of software development. It also includes activities such as software project management and the development of tools, methods, and theories to support software production.

Why software engineering important?
More and more, individuals and society rely on advanced software systems. We need to be able to produce reliable and trustworthy systems economically and quickly. It is usually cheaper, in the long run, to use software engineering methods and techniques for software systems rather than just write the programs as if it was a personal programming project. For most types of systems, the majority of costs are the costs of changing the software after it has gone into use.

1.5.1 Software Engineering Approaches

The systematic approach that is used in software engineering is sometimes called a software process. A software process is a sequence of activities that leads to the production of a software product.

There are four fundamental activities that are common to all software processes:

1. Software specification
2. Software development
3. Software validation
4. Software evolution

1.5.2 General Issues That Affect Most Software

• Heterogeneity: Increasingly, systems are required to operate as distributed systems across networks that include different types of computer and mobile devices.
• Business and social change: Business and society are changing incredibly quickly as emerging economies develop and new technologies become available. They need to be able to change their existing software and to rapidly develop new software.
• Security and trust: As software is intertwined with all aspects of our lives, it is essential that we can trust that software.

1.5.3 Software Engineering Diversity

Different types of systems need different development processes. Software engineering is a systematic approach to the production of software that takes into account practical cost, schedule, and dependability issues, as well as the needs of software customers and producers.

Two major approaches:

• Sequential
• Iterative

Application Types

1. Stand-alone applications: These are application systems that run on a local computer, such as a PC. They include all necessary functionality and do not need to be connected to a network.
2. Interactive transaction-based applications: Applications that execute on a remote computer and are accessed by users from their own PCs or terminals. These include web applications such as e-commerce applications.
3. Embedded control systems: These are software control systems that control and manage hardware devices. Numerically, there are probably more embedded systems than any other type of system.
4. ** Batch processing systems:** These are business systems that are designed to process data in large batches. They process large numbers of individual inputs to create corresponding outputs.
5. Entertainment systems: These are systems that are primarily for personal use and which are intended to entertain the user.
6. Systems for modeling and simulation: These are systems that are developed by scientists and engineers to model physical processes or situations, which include many, separate, interacting objects.
7. Data collection systems: These are systems that collect data from their environment using a set of sensors and send that data to other systems for processing.
8. Systems of systems: These are systems that are composed of a number of other software systems.

1.5.4 Software Engineering Fundamentals

Some fundamental principles apply to all types of software system, irrespective of the development techniques used:

• Systems should be developed using a managed and understood development process. Of course, different processes are used for different types of software.
• Dependability and performance are important for all types of system.
• Understanding and managing the software specification and requirements (what the software should do) are important.
• Where appropriate, you should reuse software that has already been developed rather than write new software.

1.5.5 Software Engineering and The Web

The Web is now a platform for running application and organizations are increasingly developing web-based systems rather than local systems.

Web services allow application functionality to be accessed over the web. Cloud computing is an approach to the provision of computer services where applications run remotely on the ‘cloud’.

Software reuse is the dominant approach for constructing web-based systems. When building these systems, you think about how you can assemble them from pre-existing software components and systems.

Web-based systems should be developed and delivered incrementally. It is now generally recognized that it is impractical to specify all the requirements for such systems in advance.

User interfaces are constrained by the capabilities of web browsers. Technologies such as AJAX allow rich interfaces to be created within a web browser but are still difficult to use. Web forms with local scripting are more commonly used.

Web-based systems are complex distributed systems but the
fundamental principles of software engineering discussed
previously are as applicable to them.

1.5.6 Software Engineering Ethics

Like other engineering disciplines, software engineering is carried out within a social and legal framework that limits the freedom of people working in that area.

As a software engineer, you must accept that your job involves wider responsibilities than simply the application of technical skills. You must also behave in an ethical and morally responsible way if you are to be respected as a professional engineer. You should not use your skills and abilities to behave in a dishonest way or in a way that will bring disrepute to the software engineering profession.

There are areas where standards of acceptable behavior are not bound by laws but by the more tenuous notion of professional responsibility. Some of these are:

• Confidentiality
• Competence
• Intellectual property rights
• Computer misuse

1.5.7 Software Development Risk

Many software development projects run into difficulties

• Does not work as expected
• Over budget
• Late delivery

Much of the functionalities in the software is wasted

• Wrong requirement
• User dislike it
• There are no customer

Many software projects fail because the software developers build the wrong software. The software development team must:

• Fully understand requirement
• Validate requirement

The developer will often add technical insights and suggestions, but remember the client satisfaction is the primary measurement of success in software project.

How to minimize risk
Communication with the client

• Feasibility studies
• Separation of requirement from design
• Milestones and releases
• Acceptance and user testing

---

2. Software process models

A software process model is a simplified representation of a software process. Discuss very general process models and present these from an architectural perspective, framework of the process rather than the details of specific activities.

These generic models are not definitive descriptions of software processes, they are abstractions of the process that can be used to explain different approaches to software development.

2.1 The waterfall model

This takes the fundamental process activaties of specification, development, validation, and evolution and represents them as separate process phases such as requirements specification, software design, implementation, testing, and so on.

Because of the cascade from one phase to another, this model is known as the ‘waterfall model’ or software life cycle. The waterfall model is an example of a plan-driven process.

In principle, the result of each phase is one or more documents that are approved (‘signed off’). The following phase should not start until the previous phase has finished. In practice, these stages overlap and feed information to each other. The software process is not a simple linear model but involves feedback from one phase to another. Documents produced in each phase may then have to be modified to reflect the changes made.

Because of the costs of producing and approving documents, iterations can be costly and involve significant rework. Therefore, after a small number of iterations, it is normal to freeze parts of the development, such as the specification, and to continue with the later development stages. Problems are left for later resolution, ignored, or programmed around. This premature freezing of development tasks may mean that the system won’t deliver what the users are expecting.

During the final life cycle phase (operation and maintenance) the software is put into use. Errors and omissions in the original software requirements are discovered. Program and design errors emerge and the need for new functionality is identified. The system must therefore evolve to remain useful. Making these changes (software maintenance) may involve repeating previous process stages.

Advantages

• In the Waterfall Model, documentation is produced at each phase. This makes the process visible so managers can monitor progress against the development plan.

Drawbacks

• Its major problem is the inflexible partitioning of the project into distinct stages.
• Commitments must be made at an early stage in the process, which makes it difficult to respond to changing customer requirements.

Summary
In principle, the waterfall model should only be used when the requirements are well understood and unlikely to change radically during system development.

2.2 Incremental development

This approach interleaves the activities of specification, development, and validation. The system is developed as a series of versions (increments), with each version adding functionality to the previous version.

Incremental development is based on the idea of developing an initial implementation, exposing this to user comment and evolving it through several versions until an adequate system has been developed. Specification, development, and validation activities are interleaved rather than separate, with rapid feedback across activities. Incremental software development, which is a fundamental part of agile approaches, is better than a waterfall approach for most business, e-commerce, and personal systems.

By developing the software incrementally, it is cheaper and easier to make changes in the software as it is being developed. Each increment or version of the system incorporates some of the functionality that is needed by the customer. Generally, the early increments of the system include the most important or most urgently required functionality. This means that the customer can evaluate the system at a relatively early stage in the development to see if it delivers what is required. If not, then only the current increment has to be changed and, possibly, new functionality defined for later increments.

Incremental development has three important benefits, compared to the waterfall model:

• The cost of accommodating changing customer requirements is reduced.
• It is easier to get customer feedback on the development work that has been done.
• More rapid delivery and deployment of useful software to the customer is possible, even if all of the functionality has not been included.

From a management perspective, the incremental approach has two problems:

• The process is not visible.
• System structure tends to degrade as new increments are added.
• large organizations have bureaucratic procedures that have evolved over time and there may be a mismatch between these procedures and a more informal iterative or agile process.
• Formal procedures are required by external regulations (e.g., accounting regulations)

Summary
The problems of incremental development become particularly acute for large, complex, long-lifetime systems, where different teams develop different parts of the system.

Large systems need a stable framework or architecture and the responsibilities of the different teams working on parts of the system need to be clearly defined with respect to that architecture.

This has to be planned in advance rather than developed incrementally.

2.3 Reuse-oriented software engineering

This approach is based on the existence of a significant number of reusable components. The system development process focuses on integrating these components into a system rather than developing them from scratch.

In the majority of software projects, there are some forms of informal software reuse. This informal reuse takes place irrespective of the development process that is used. In the 21st century, software development processes that focus on the reuse of existing software have become widely used. Reuse-oriented approaches rely on a large base of reusable software components and an integrating framework for the composition of these components.

There are three types of software component that may be used in a reuse-oriented process:

• Web services that are developed according to service standards and which are available for remote invocation.
• Collections of objects that are developed as a package to be integrated with a component framework such as .NET or J2EE.
• Stand-alone software systems that are configured for use in a particular environment.

Adavantages

• reducing the amount of software to be developed and so reducing cost and risks.
• usually also leads to faster delivery of the software.

Drawbacks

• Requirements compromises are inevitable, and this may lead to a system that does not meet the real needs of users.
• Some control over the system evolution is lost as new versions of the reusable components are not under the control of the organization using them.

---

3. Software process activities

Real software processes are interleaved sequences of technical, collaborative, and managerial activities with the overall goal of specifying, designing, implementing, and testing a software system.

The four basic process activities of specification, development, validation, and evolution are organized differently in different development processes. In the waterfall model, they are organized in sequence. In incremental development they are interleaved. How these activities are carried out depends on the type of software, people, and organizational structures involved.

3.1 Software Specification

Software specification or requirements engineering is the process of understanding and defining

• what services are required from the system
• identifying the constraints on the system’s operation and development.

Requirements engineering is a particularly critical stage of the software process as errors at this stage inevitably lead to later problems in the system design and implementation.

There are four main activities in the requirements engineering process:

The requirements engineering process aims to produce an agreed requirements document that specifies a system satisfying takeholder requirements.

Requirements are usually presented at two levels of detail:

• End-users and customers need a high-level statement of the requirements
• system developers need a more detailed system specification.

3.2 Software design and implementation

The implementation stage of software development is the process of converting a system specification into an executable system.

It always involves processes of software design and programming but, if an incremental approach to development is used, may also involve refinement of the software pecification.

A software design is a description of the structure of the software to be implemented, the data models and structures used by the system, the interfaces between system components and, sometimes, the algorithms used.

Design activities

• Architectural design: where you identify the overall structure of the system, the principal components (sometimes called sub-systems or modules), their relationships, and how they are distributed.
• Interface design: where you define the interfaces between system components. This interface specification must be unambiguous.
• Component design: where you take each system component and design how it will operate.
  • a simple statement of the expected functionalities
  • a list of changes to be made to a reusable component
  • a detailed design model (model-driven approach).
• Database design: where you design the system data structures and how these are to be represented in a database.

Designers do not arrive at a finished design immediately but develop the design iteratively. They add formality and detail as they develop their design with constant backtracking to correct earlier designs.

The detail and representation of the design output vary considerably.

• For critical systems, detailed design documents setting out precise and accurate descriptions of the system must be produced.
• If a model-driven approach is used, these outputs may mostly be diagrams.
• Where agile methods of development are used, the outputs of the design process may not be separate specification documents but may be represented in the code of the program.
• Model-driven development (MDD) or model-driven engineering (Schmidt, 2006), where models of the software are created at different levels of abstraction
• The models are developed in sufficient detail so that the executable system can be generated from them
• Software development tools may be used to generate a skeleton program from MDD. This includes code to define and implement interfaces, and, in many cases, the developer need only add details of the operation of each program component

3.3 Software validation

Software validation or, more generally, verification and validation (V&V) is intended to show that

• a system both conforms to its specification
• it meets the expectations of the system customer

Program testing, where the system is executed using simulated test data, is the principal validation technique.

Validation may also involve checking processes, such as
inspections and reviews, at each stage of the software process
from user requirements definition to program development.

A three-stage testing process involved testing for system components then the testing for an integrated system and, finally, the testing of the system with the customer’s data

The stages in the testing process are:

• Development testing: The components making up the system are tested by the people developing the system. Each component is tested independently, without other system components.
• System testing: System components are integrated to create a complete system. This process is concerned with finding errors that result from unanticipated interactions between components and component interface problems.
• Acceptance testing: This is the final stage in the testing process before the system is accepted for operational use. The system is tested with data supplied by the system customer rather than with simulated test data.

Normally, component development and testing processes are interleaved. Programmers make up their own test data and incrementally test the code as it is developed.

If an incremental approach to development is used, each increment should be tested as it is developed, with these tests based on the requirements for that increment.

When a plan-driven software process is used (e.g., for critical systems development), testing is driven by a set of test plans. An independent team of testers works from these pre-formulated test plans.

3.4 Software evolution

Once a decision has been made to manufacture hardware, it is very expensive to make changes to the hardware design. However, changes can be made to software at any time during or after the system development. Even extensive changes are still much cheaper than corresponding changes to system hardware

Historically, there has always been a split between the process of software development and the process of software evolution (software maintenance)

People think of software development as a creative activity in which a software system is developed from an initial concept through to a working system. However, they sometimes think of software maintenance as dull and uninteresting

Rather than two separate processes, it is more realistic to think of software engineering as an evolutionary process

---

4. Agile methods

4.1 Rapid software development

The software is not developed as a single unit but as a series of increments, with each increment including new system functionality.

Fundamental characteristics:

• The processes are interleaved.
• Minimum documentation
• Developed in a series of versions, or increments, with system stakeholders involvement.
• System user interfaces are often developed using an interactive development system that allows the interface design to be quickly created

Other approaches to RAD includes:

• Adaptive software development
• Agile methodologies
• Spiral model
• Unified software development process

4.2 Plan-driven development approach

Large, long lived software - careful project planning, formalized quality assurance, the use of analysis and design methods supported by CASE tools, and controlled and rigorous software development processes.

• Work of multiple development teams has to be well coordinated
• The system is a critical system. A critical system is a system that refers to the systems that are efficient and retain this efficiency as they change without prohibitive costs being incurred.
• Many different people will be involved in maintaining the software over long period of time

4.3 Agile methods

• Agile allowed the development team to focus on the software itself rather than on its design and documentation.
• universally rely on an incremental approach to software specification, development, and delivery.
• They are best suited to application development where the system requirements usually change rapidly during the development process
• They are intended to deliver working software quickly to customers, who can then propose new and changed requirements to be included in later iterations of the system.
• The philosophy behind agile methods is reflected in the agile manifesto that was agreed on by many of the leading developers of these methods.
  • Individuals and interactions over processes and tools
  • Working software over comprehensive documentation
  • Customer collaboration over contract negotiation
  • Responding to change over following a plan

Principle

• Customer involvement: Customers should be closely involved throughout the development process. Their role is provide and prioritize new system requirements and to evaluate the iterations of the system.
• Incremental delivery: The software is developed in increments with the customer specifying the requirements to be included in eacch increment.
• People not process: The skills of the development team should be recognized and exploited. Team members should be left to develop their own ways of working without prescriptive processes.
• Embrace change: Expect the system requirements to change and so design the system to accommodate these changes.
• Maintain simplicity: Focus on simplicity in both the software being developed and in the development process. Wherever possible, actively work to eliminate complexity from the system.

Some of the agile methods:

• Extreme programming (Beck, 1999; Beck, 2000)
• Scrum (Cohn, 2009; Schwaber, 2004; Schwaber and Beedle, 2001)
• Crystal (Cockburn, 2001; Cockburn, 2004)
• Adaptive Software Development (Highsmith, 2000)
• DSDM (Stapleton, 1997; Stapleton, 2003)
• Feature Driven Development (Palmer and Felsing, 2002)

These agile methods are all based around the notion of incremental development and delivery with different processes. However, they share a set of principles, based on the agile manifesto, and so have much in common.

The Challenges

• In practice, the principles underlying agile methods are sometimes difficult to realize:
  • Its success depends on having a customer who is willing and able to spend time with the development team and who can represent all system stakeholders
  • Individual team members may not have suitable personalities for the intense involvement
  • Prioritizing changes can be extremely difficult, especially in systems for which there are many stakeholders
  • Maintaining simplicity requires extra work
  • It is difficult for some organization to accept informal processes defined by development teams
• The software requirements document is usually part of the contract between the customer and the supplier (software company)
• Because incremental specification is inherent in agile methods, writing contracts for this type of development may be difficult.
• Agile methods have to rely on contracts in which the customer pays for the time required for system development rather than the development of a specific set of requirements
• Agile methods have to rely on contracts in which the customer pays for the time required for system development rather than the development of a specific set of requirements

Maintenance

• There are two questions that should be considered when considering agile methods and maintenance:
  • Are systems that are developed using an agile approach maintainable, given the emphasis in the development process of minimizing formal documentation?
  • Can agile methods be used effectively for evolving a system in response to customer change requests?
• Formal documentation is supposed to ease system evolution and maintenance. In practice, however, formal documentation is often not kept up to date and so does not in system maintainability
• Agile methods supporters argue that it is a waste of time to write this out-of-date documentation
• The key to implementing maintainable software is to produce high-quality, readable code.
• The key document is the system requirements document, which tells the
  software engineer what the system is supposed to do.
• The main difficulty after software delivery is likely to be keeping customers involved in the process.
• The other problem that is likely to arise is maintaining continuity of the development team. Agile methods rely on team members understanding aspects of the system without having to consult documentation.
• If an agile development team is broken up, then this implicit knowledge is lost and it is difficult for new team members to build up the same nderstanding of the system and its components.

Plan-driven and agile development

Most software projects include practices from plan-driven and agile approaches. To decide on the balance between a plan-based and an agile approach, you have to answer a range of technical, human, and organizational questions:

• Detail specification and design needed?
• Is incrementally strategy realistic?
• How large is the system?
• what type of system being developed?
• System life span?
• Available technologies and tools?
• Organization of the team?
• Cultural issues?
• Available skillsets?
• External regulation?