Software Engineering for Artificial Intelligence Projects

Fall 2001
Chip Seitter, Nisha Sheth, Alex Strehl, Sue Tseng, Chris Wagner

College of Charleston
Computer Science Department
Room 216, JC Long Building
9 Liberty Street
Charleston, SC 29401
Chip029@yahoo.com, nishacs@yahoo.com, astrehl@edisto.cofc.edu, Sft_831@hotmail.com, chris@kainaw.com

Abstract

There are many popular software engineering models used to aid in the successful completion of software projects. Most models do not take into account the problems caused by the inclusion of artificial intelligence in the software project. This paper discusses four major areas of artificial intelligence and the effect of each area on the common areas of software engineering.

Categories and Subject Descriptors
D.2 [Software]: Software Engineering
I.2 [General]: Artificial Intelligence

General Terms
Design

Keywords
Artificial Intelligence
Software Engineering

1. Introduction

Over the past forty years many well-known software engineering models have been developed to handle large projects. However, most software engineering models do not account for complications caused by the inclusion of artificial intelligence in the software project. With the growing interest in the popularity of artificial intelligence, such complications should be addressed.

While artificial intelligence is a vast field with many different types of implementation and uses, it is possible to generalize four major areas of artificial intelligence. Then, each area may be analyzed in order to predict possible problems that may occur during a software engineering project.

The remainder of this document is concerned with identifying major classes of artificial intelligence, how each class changes the software engineering model, and suggestions for identifying which class of artificial intelligence a project may require.

2. Engineering AI Projects

It is very useful to generalize software engineering models by the stages they incorporate. For many purposes, one can consider all stages to be one of five fundamental types. Taken together, these five types of activities can be considered an abstraction of any software engineering model, and therefore can be considered a generic software engineering model or life cycle. These types are typically referred to as Requirements Analysis, Design, Implementation, Testing, and Maintenance. Besides the infamous waterfall model, most models do not prescribe these stages to occur one after another during ideal software development. However, it is generally true that "many other, more complex models are really just embellishments of the waterfall, incorporating feedback loops and extra activities"^[5]. In fact, almost every stage of any theoretically feasible software engineering model will overlap with one or more of these activities.

The use of artificial intelligence in academic and non-academic settings will almost certainly impact every software engineering model. To describe these changes implicitly, without referring directly to any specific model, the changes to the fundamental stages previously mentioned may be enumerated. In general, the use of complex artificial intelligence in a software engineering project will cause a noticeable change in the project's life cycle. The amount of change is, for the most part, directly related to the complexity of the artificial intelligence.

2.1. The AI of Pong

In the earliest days of computing, there was an interest in playing games against a computer. From tic-tac-toe to chess, many programs were written to give computers human-like intelligence. Pong contained a very simple form of this artificial intelligence. The goal of the computer is to move a paddle in the path of an oncoming ball. The artificial intelligence may be summarized in two lines of code that translate to, "If the ball is high, move up; otherwise, move down."

if(ball.y > paddle.y) paddle.y++; else paddle.y--;

Figure 1: Pong game design with included AI code.

Skill is an important factor in game play. The computer must be able to display a low level of skill or a high level of skill as selected by the user. In Pong, the skill level was easily selected by adjusting how fast the paddle could move. If the speed were too fast, the computer would never miss the ball. If it was too slow, the computer could only hit the ball when it was aimed directly at the paddle.

In the end, skill level is the only adjustment available for this form of artificial intelligence. At any specific skill level, the computer always reacts in the same manner. This is a form of monotonic logic. For any specific action, the reaction is always the same.

2.2. Monotonic Logic

A logic is monotonic "if the truth of a proposition does not change when new information (axioms) are added to the system"^[1]. This is well suited to computers because digital logic, or Boolean logic, is a monotonic logic. Any problem that can be simplified to some form of Boolean logic may be implemented in a computer. In our Pong example, there was a simple true/false check to decide which direction the paddle should move. As the environment advances in complexity, such as making a move in Tic-Tac-Toe, a series of checks will be needed to make a decision. When the situation is relatively simple, a short linear program may be written to make a decision. In more complicated situations, recursive or state space searches may be used. Such solutions involve searching a large set of possible states for one or more specific solutions. Many methods used to solve this type of problem have already been developed, documented, and are fundamental to artificial intelligence.

Many of the different methods used to solve state space search problems rely on graph theory. The first problem is to determine how the states will be represented within the computer. Next, an initial, or start, state is required. Once that is complete, a strategy must be chosen which begins with the initial state and attempts to traverse the set of states in a specified manner until a solution is found. One commonly used strategy is breadth-first search, which examines the states closer to the initial state before it examines states farther from the initial state. Another strategy used is depth-first search, which searches one path from the initial state to completion before examining other paths. The depth-first search limits computer resource usage by limiting the number of states that are being examined (assuming the state tree is fairly balanced). The breadth-first search sacrifices resources in order to find a solution involving the least number of transitions between states.

Breadth First Search Order: A B C D E F G

Depth First Search Order: A B E G F C D

Figure 2: State Space Diagram of a Tic-Tac-Toe game.

Regardless of the search strategy used, the state space approach should only be used if the problem has certain properties that allow the computer to generate the states and determine the transitions between them. The states should be discrete, and the transitions should be well defined. In other words, there must be exact rules to determine all states that can be immediately arrived at (in one move) from some specified state. Keep in mind that the number of possible states cannot be too large or the search will require more resources (time and space) than the computer can provide. One way of overcoming this problem is to limit the number of states that are searched.

There are two methods of limiting the number of states searched. One is to limit the number of states in the design phase. In Tic-Tac-Toe, it is apparent that all possible configurations of X's and O's on the three by three board will not be used. Once a win is made, any further states branching from that pattern will not be valid and may be omitted from the program's design. Another method of state limitation is to mathematically determine a subset of the state tree while the program is running. This is known as a heuristic search because it does not guarantee that all states will be checked, but provides some level of certainty in a successful decision.

Common problems, such as chess, have so many possible states that it is not practical to represent each and every state with the computer's limited resources. A state space search could be used to determine a move that will allow the next player in a chess game to always win. However, "it is a safe bet that we will never in our lifetimes know the answer; the algorithm requires fantastically large amounts of time for its execution, even though it is finite"^[2]. In such cases, heuristic searches may be used if there are properties of the states that can be analyzed to determine their likeliness to lie on a path to the solution^[3]. A limited state-space search is used which continually alters the state tree to only represent a certain number of moves from the present board position. The processing time increases in a limited state-space search as a result of continually changing the tree. However, the reduction in memory usage is often worth the increase in processing time. Theorem proving and many other games are well suited to this form of artificial intelligence.

The use of the state space search approach to artificial intelligence can be related to the general software engineering cycle. During the requirement analysis phase, the practical software engineer will keep an eye out for requirements, which seem well suited to the state space search solution. This is important because the characteristics of the problem, which are based upon the requirements, determine whether or not a state space algorithm can be used. Secondly, it must be officially determined in the design phase whether a state space search procedure is appropriate. The implementation of the state space search should also be allotted a distinct module during the design phase, allowing the state space search to be tested individually from the application. It should be noted that many of these implementations rely on recursion or recursion-like programming. This is generally true because the solution requires the repeated traversal and analysis of states.

During the testing phase of the software engineering cycle, some additional considerations must be made. It would be impractical, if not impossible, to test all possible states and searches that may be encountered. Also, it would not be advisable to only do a limited number of tests with random data. It is unlikely that this would detect most errors, and there are usually more efficient methods available. In some cases, each rule used to traverse the set of states could be individually tested. For example, when testing a game that plays chess, rather than test every configuration of the board, it would be more feasible just to test the possible moves of the chess pieces. Generally, if such a rule tests successfully several times, it will work under all conditions. However, tricky test cases need to be considered. In the chess playing game, one would test the eight different moves a knight could make. One test case would be to ensure that the program does not move the knight off the board when it is located near the edge of the board. Once these considerations are made, the software engineering cycle can proceed normally.

While many software engineers opt for this type of artificial intelligence, it is definitely not the best use of resources for all problems. Consider attempting a complete state tree for chess. In a linear series of checks, there would need to be a check for every possible situation the board may be in. That program would be prohibitively long. Using simple recursion, the computer may only look ahead a certain number of moves until it uses up the available memory resources. Near the end of a game, this is acceptable, but in the middle of a game, it is far from perfect. A tree of preferred board situations is helpful to guide the decisions, but it is a small fix for the general problem. Monotonic logic approaches to artificial intelligence depend on vast amounts of storage and processing resources for problems that the human brain can easily handle.

Once the testing phase of the project is complete, it is tempting to assume that the artificial intelligence development is complete. Unfortunately, the artificial intelligence development often continues long after the general project is complete. While the computer may appear to have some advanced intelligence at first, most people will learn to predict the decisions the computer will make and take advantage of them to out-think the computer. Then, it is up to the software engineers to go back and increase the apparent power of the computer. Of course, an advance in the computer's ability will result in a requirement for more disk space, more memory, and more processing power.

The main reason for the unimpressive performance of monotonic logic is that each situation creates a unique and unchanging response. If situation A makes a response X and situation B makes a response Y, the computer cannot make a decision about having situation A before B or B before A unless specifically programmed to do so. Regardless of the order of the situations, the computer always responds the same and only then when given the response it should make.

2.3. Can Bob Fly

Consider the following statements:

• Many animals cannot fly.
• Most birds can fly.
• Penguins cannot fly.

In a computer, only the values of 'true' and 'false' are available. So, these would be represented as:

• AnimalFly = false;
• BirdFly = true;
• PenguinFly = false;

Now, consider Bob. Bob is an animal. Can Bob fly? The human would answer, "Bob probably cannot fly."; The computer would answer, "False". Bob is also a bird. Can Bob fly? The human would simply reply, "Bob probably can fly." The computer would hit a logical error. Being an animal makes BobFly=false, but being a bird makes BobFly=true. Complicate it further by showing that Bob is a Penguin. Still unflustered, the human answers, "Bob definitely cannot fly." The computer now has to deal with a situation that is true by one rule and false by two.

The situation could become more complicated. What if Bob was the first penguin trained to hang glide? The answer to Bob's flight capability changes once again.

Words such as maybe, probably, and definitely create levels of certainty. In the world of digital or Boolean logic, there is only one level of certainty: 100%. In order to handle real-world problems in a manner similar to humans, a computer must be able to take conflicting situations and create a best guess as well as some level of certainty.

2.4. Non-Monotonic Reasoning

When the computer does not have all of the information required to provide an answer to a question, which is generally the case in real-world problems, it uses what is known as non-monotonic reasoning. This is the process of generating the most probable answer or educated prediction to the given question, "as long as no contradictory evidence is presented"^[6]. Non-monotonic reasoning is required in humans but difficult to imitate in machines because it involves three problems: Incompleteness, inconsistencies, and change. Incompleteness requires reasoning systems to compensate for lack of knowledge. Inconsistencies require reasoning systems to resolve ambiguities and contradictions. Change requires reasoning systems to be able to update its world knowledge base over time. The statistical approach to artificial intelligence can cope with these uncertainties through the use of statistical methods.

Given a situation, the computer can make a best guess at a proper reaction. If the situation is added to, such as placing a new X in a tic-tac-toe board or adding the population of a city, the computer's best guess may change. It is also possible to have multiple answers, all of which are true at different levels of certainty.

Fuzzy logic focuses on ambiguities in describing events rather than uncertainty about the occurrence of an event. It is a superset of conventional (Boolean) logic that has been extended to handle the concept of partial truth -- truth-values between "completely true" and "completely false". Traditional logic systems assume that things are either in one category or the other. A fuzzy logic system is a system, which attempts to reflect "the real world" better than binary logic, by allowing gradient values to be assigned to given qualities or situations. Thus, in fuzzy logic, a person can be "fairly short or fairly tall" whereas in simple logic a person could only be described as "tall" or "short". Fuzzy logic provides a way of taking our common sense knowledge that most things are matter of degree into account when a computer is automatically making a decision.

This is a difficult task to handle in imperative programming languages (C, Java, Pascal) that are set in a world of strict true and false states. However, declarative languages (Lisp, Prolog, ML) are better at handling this type of situation. Most of them supply all possible responses in no particular order, but it is possible to provide some order, which in turn provides some sense of certainty.

Statistical methods provide a means for representing beliefs that are not certain but for which there may be some supporting evidence. Statistical methods deal with uncertainty via probability and logic. Probability allows us to determine the chances of an event occurring. Through probability we can also describe how combinations of events influence each other. Probabilistic analysis offers advantages in two broad scenarios: Genuine Randomness and Exceptions. Card games are a good example of genuine randomness. We may not be able to predict any outcomes with certainty but we have the knowledge about the likelihood of certain items. A further use of probability is to summarize large exceptions. The statistical approach groups all exceptions to a relationship together and then uses this measure to describe how often an exception of any type might be expected to occur.

As expected, an implementation of statistical reasoning will require a database of statistics. This process may be completed by a team separate from the programming team. Until the project is near completion, a complete set of statistics is not required. In implementation and testing, only a primary subset of the statistics database will be needed.

Natural Language Processing (NLP) is an example of a specific area of artificial intelligence where the statistical approach is applied. A natural language is any of the languages naturally used by humans; the language acquired by humans in the normal socialization process. For a computer to process natural languages, it would have to mimic what a human does. That is, it would have to recognize the sequence of words spoken by a person or another computer, understand the syntax or grammar of the words, and then extract the meaning of the words. The goal of natural language processing is to design and build a computer system that will analyze, understand, and generate natural human-languages^[3]. There are at least three distinct 'levels' in processing natural languages: phonics, grammar, and meaning. Natural Languages are massively ambiguous at every level. To resolve ambiguity, humans need not only a detailed knowledge of the language itself (its sounds, rules about sound combinations, and rules about word combinations and ordering), but also a large detailed knowledge of the world, the ability to follow a story by connecting sentences, and the ability to infer what a speaker means, even if he, she, or it did not actually provide a complete statement. These particular factors make natural languages difficult to process by computer.

The statistical approach to natural language processing suggests that a general language model be applied to statistical and pattern recognition methodology. Statistical systems are effective in modeling language performance though training based on most frequent language use. They are useful in modeling linguistic phenomena that are not well understood from a competence perspective, e.g., speech. One of the easiest tasks for a natural language processing system is to parse a sentence to determine its syntax. Statistical techniques have already successfully attacked its first syntactic problem, part-of-speech tagging. This is where either the input string is viewed as a string of parts of speech or the parser is guided by statistical information about the particular words in a sentence. A more difficult task is determining the semantic meaning of a sentence. One of the most difficult tasks is the analysis of the context to determine the true meaning and comparing that with other text. This involves the design and implementation of common formulas for proper grammar.

A Bayesian network is a modeling technique that provides a mathematically sound formula for representing and reasoning uncertainty, imprecision, or unpredictability in our knowledge. The basic idea surrounding Bayesian networks are as follows:

• Most events are conditionally independent of each other.
• Only interactions between events that affect one other should be modeled.
• Events may be either unidirectional or bi-directional.
• Events may be causal and thus get chained together in a network.
• Bayesian networks are represented as directed graphs.

Unfortunately, testing such a program is rather difficult. Given a simple problem containing only three possible parts, there are many different responses available based on the value of each part and the order the parts appear in. In a true/false world, there are 27 possible value combinations (true, false, and not considered). For instance, Part A could be True, Part B could be Not Considered, and Part C could be False. There are also 6 orderings for the parts. The computer may check Part C first, then Part B, then Part A. In Boolean logic, having A True, B Not Considered, and C False is identical to any situation that is True, Not Considered, and False. In proper reasoning, checking that C is False first is a completely different situation to checking if A is False first. It may be thought of as a double-permutation that, in the end, creates 162 possible responses to be checked. In a real-world situation, there will be many more than three parts to a situation and only three states for each part. That makes a complete test not only difficult, but also impossible to complete in a reasonable amount of time.

This model of artificial intelligence is plagued with assumptions. Assuming a reasonable test of the system is completed, it is assumed that the reactions are reasonable for all situations that the system will be confronted with. This method of solving the artificial intelligence problem assumes that a one-size-fits-all set of reactions is acceptable. It also assumes that the designers are capable of predicting all possible situations that the program will be confronted with. Even then, an error in the initial database of statistics or a change in the real world that invalidates the database of statistics will cause apparent errors in the program's ability to accurately make decisions.

2.5. One Thousand Monkeys

Let's bring the "One Thousand Monkeys" paradigm up to date. Consider one thousand monkeys picking one thousand stocks to buy or sell every day. One monkey will eventually make some money. It may not happen consistently, but it is probable that it will happen at some point.

Now, try to improve the situation by altering the survival of the fittest to fit your own needs. Those monkeys who make money are allowed to breed. Those that do not are not allowed to breed. This will weed out whatever genetic trait is causing some monkeys to lose money and possibly enhance whatever genetic trait is causing other monkeys to make money.

As each generation is born, this directed evolution should enhance the moneymaking ability of the monkeys. Given enough generations, the moneymaking ability will become dependable. It is possible that it may advance to the point that the monkey brokers out-perform all of Wall Street. Best of all, it is not important or even necessary to know how the monkeys are making their decisions. It is only important to know which monkeys to breed and which monkeys to get rid of.

2.6. The Evolutionary Approach

Many well-defined problems have been solved using various techniques of artificial intelligence. These solutions tend to be specific in nature and may apply to some, but definitely not all, future problems. They also require large amounts of time-consuming research and human insight. However, when a problem that needs artificial intelligence arises naturally in the workplace, it may not be possible to perform such extensive research. Therefore, techniques have been developed to rapidly solve such problems while minimizing the human insight factor. Some formulated techniques are based on Darwin's theory of evolution. These methods are very similar and together make up the evolutionary approach to artificial intelligence engineering.

Although many names have been given to specific evolutionary techniques such as genetic programming, evolutionary programming, genetic algorithms, etc., researchers have not generally agreed on their exact meanings^[4]. These methods all have the same underlying idea. Potential solutions to the problem at hand are encoded genetically in some type of data structure or binary medium. The structures come in many different types. Some structures are modeled from the human brain and include computer models of neurons and synapses. Another structure is a linear program using a very small instruction set similar to assembly language. There are even mixed models that use nodes and connections similar to the neural net but include small programs inside each node. Regardless of the structure, it is important to have some sort of way to describe each variation of the structure, called a genotype.

Figure 3: Two structures representing A+B-C=D

Initially, many random genotypes will be developed. These genotypes will be tested but will likely fail miserably. Over time, some random genotype will display some form of minimal intelligence. The initial goal is to develop a small number of semi-intelligent genotypes.

These genotypes are then combined during an artificial reproduction, which often allows random mutations. An important step is then to pick out the best possible encoded solutions. This stage is analogous to natural selection, also known as "survival of the fittest". These survivors are then recombined and the best are again chosen. This process of reproduction and selection is repeated as many times as necessary. At each stage, the set of genotypes, often called the population, should improve. The method of selection performed is perhaps the most important process to consider. For example, the implementers may choose to supervise the selection by hand picking the survivors, or write another program to control the process.

Each testing method has its advantages and disadvantages. An advantage of supervised selection (supervised learning) is that a human can often pick out promising genotypes much more efficiently than a computer program. A disadvantage is the amount of time and effort required to perform this task, as there may be thousands of genotypes and many iterations. A computer program can test many genotypes quickly, but it will be highly limited and unable to rationalize which genotypes are displaying the highest form of intelligence. This creates a sort of paradox. A good artificial intelligence is required to quickly develop a good artificial intelligence.

When enough cycles have been performed, the final, most promising genotypes are then decoded back into the actual solutions. If actual assembly-level programs were used for the encoding, then there really is no decoding. In this case the genotypes, the assembly-level programs, would be the phenotypes, the decoded genotypes.

Evolutionary techniques have been used successfully to solve many complicated problems. In fact they have recently "gained considerable popularity as tools for searching vast, complex, deceptive, and multimodal search spaces"^[4]. They are especially good at reproducing or imitating natural behavior. For example, evolutionary algorithms have produced good digital representations of animals (artificial life) and have developed algorithms used by robots. These life-like solutions are well adapted to the evolutionary approach. However, genetic algorithms have also been used to solve other problems such as sorting numbers. On the other hand, evolutionary techniques would probably not be well suited for problems easily solved using monotonic logic (i.e. state-based search problems like chess). Such problems are based upon many understood rules, which may be difficult for a genetic solution to determine and/or utilize.

Evolutionary algorithms will no doubt radically alter the generic software engineering life cycle and therefore impact most of the current software engineering models. For one thing, it may require an artificial intelligence expert just to determine the feasibility of an evolutionary solution to a given software engineering problem. If such a solution is deemed useful, it is certainly the case that several members of the development team should have extensive experience, skill, and expertise in the area of designing evolutionary solutions. The encoding of potential solutions (programs) and the reverse process of decoding should be determined during the design phase. They are absolutely required and do not represent an implementation detail. Of course, the method(s) of selection should also be determined during the design phase. The advantages of different strategies, for example supervised verses unsupervised learning, should be explored and considered. Like any programming project, some up-front thought and consideration during the design phase will save an incalculable amount of time and effort in the long run.

The most significant impact of evolutionary techniques to the software engineering life cycle is upon the testing, implementation, and maintenance phases. That is, the use of genetic programming greatly affects the latter software engineering stages. A most notable change is the merging of the implementation and testing phases. There is now much less distinction between them. The continued process of genetic reproduction (recombination and mutation) and selection is definitely a fundamental step in the implementation of evolutionary algorithms. However, the performance evaluation used to select genotypes may strongly resemble methods often deployed in the testing phase.

For example, when developing an algorithm to "control the locomotion of an 8-legged robot"^[4], the selection criterion used was the performance, as perceived by a human tester, of the robot at each stage of the evolution. This fact may lead some people to conclude (not entirely incorrectly) that during the development of an evolutionary solution there is really no implementation stage, but only that of design and testing. Of course, when the final genetic solution is chosen, additional tests should be performed to ensure the correctness of that solution. This end testing should proceed normally, where many broad test cases are fed to the program and solutions analyzed for correctness.

One technique that can be used to provide the best possible solution to the user is to include not one but many of the best possible genetic solutions with the final product. When the program then runs some set of data it can respond with the most common weighted answer among the genetic population. It may also indicate a level of confidence of that answer with the percentage of genetic agents that produced that answer. Alternative answers, if necessary, are available from the genetic agents that did not provide the most common answer. This is often much more useful to the user then just a single output.

There exists a similar situation with respect to the maintenance phase. Again, the distinction between the maintenance phase and the testing phase is not as strong. Traditionally, the maintenance phase consisted of correcting any errors or bugs detected by the user and adding additional features. This is still the case. However, if the program does not work correctly in certain cases a programmer will usually not be able to go in and "fix the bug". Perhaps the problem is in the code that was derived from the genetic algorithm, and that code may be almost completely unintelligible to any programmer. Therefore, the solution may require the evolution to be continued. Of course, one simple hack may be to hardcode in certain simple cases that the genetic solution just doesn't handle correctly, although this may not always be a possibility. Furthermore, the addition of features will often require additional evolution to occur. This is especially true in the case of E-systems.

An E-system is defined by Pfleeger in her book Software Engineering: Theory and Practice as a system "that is embedded in the real world and changes as the world does"^[5]. This is a system that interacts with the real world in a way not completely known and therefore the system is subject to change. When new information is discovered which needs to be incorporated into the software, the whole process of artificial evolution used to derive the original solution must be repeated (although the previous results are used) to provide the modified software. The general theme here is that during the use of evolutionary artificial intelligence, pure programming is drastically de-emphasized and a modified form of testing, which involves a performance criterion among several possibilities and a reproduction algorithm among these possibilities, is greatly emphasized.

2.7. The Rat Race

Assume we have a little rat named Paul. This rat is a maze-racing rat. He spends all day finding his way through mazes. After a few times through a maze, he memorizes the quickest path and can race through the maze at high speed without any errors.

One day, Paul is put up against Mary. Mary is as good, if not better than Paul at memorizing mazes and racing through them. So, the researchers decide to mate Paul and Mary. They get little Peter.

Now, since Paul and Mary have memorized many mazes in their lifetime, should Peter know his way through any of the mazes? That is not possible. Memories do not pass from parents to child. Only the DNA, which has no concept of what happens after birth, is passed to the offspring. So, Peter must learn his own way through the mazes. If he's lucky, though, his parents will give him a few pointers along the way.

2.8. The Learning Computer

Many problems come up every day that fall in the domain of artificial intelligence. Some are basically well defined although complicated, such as chess, where generally it is the case that computer ability can easily surpass human intelligence. However, most problems are not as easily solved. It may be the case that an adequate solution to many of these other problems can be found using evolutionary techniques. However, evolutionary techniques may not always work and often produce a solution that is very specific. One of the main "intelligent" activities of humans is the ability to learn. Therefore, learning is a fundamental step in the development of artificially intelligent software. That is, a software solution may be required that not only works with some specific problem but also has the ability to easily adapt to a changing environment. The fact that humans find learning easy has lead to an artificial intelligence approach that attempts to mimic the human brain. Projects that use this approach require some specific software engineering considerations.

The connectionist approach is an approach to the implementation of artificial intelligence by modeling the human brain. A specific instance or program that uses the connectionist approach is called a neural network. A neural network consists of neurons. Each neuron has input connections (from other neurons), output connections (to other neurons), and generally some function to compute the output from the input. The connection between two neurons occurs at a synapse. Neurons can receive excitatory or inhibitory signals from other neurons. A neural sends a signal when it fires, and it fires when it reaches its threshold. There are two methods of using neural networks to solve problems. The first is called feed-forward, and in this case the input is run through the neural network with no nodes (neurons) repeated. The second is called recurrent and in this case the neural network contains loops, which is the output of the neural network is fed back into the neural network as input. If the output of the recurrent neural network converges to something, a result is formed. Generally, neural networks are "taught" the task for which they were designed using some type of evolutionary technique. In this case, the genetic encoding used in the evolutionary process would be a description of the connections and the strength of the connections between neurons (the neural pathways). The individual neural networks that perform better at each stage of the evolution are selected for survival. Ideally, this technique should produce a successful and even optimal configuration of the neurons. Evolutionary techniques are often used in this manner because of the difficulty of custom configuring a neural network. Besides evolutionary techniques there are other techniques meant to train a neural network, often relying on a mathematically sound way of altering the weights (and possibly connections) between neurons^[3].

The connectionist approach of AI is radically different from traditional approaches. In this way it lends itself well to problems where traditional approaches have failed. Neural networks are basically different from traditional approaches is that they "de-emphasize the explicit use of symbols in problem solving"^[3]. For one thing, data is not stored in specified data structures; instead it is propagated throughout the neural network. Therefore, neural networks can tolerate some internal damage (i.e. several nodes failing). Neural networks are especially well suited to learning. They are good at identifying patterns, handling noisy input, predicting the future, etc.^[3]. This is expected since they are modeled after the human brain. Neural networks have performed successful in many areas, including face recognition, even in developing intelligent agents: According to Patel et al, "artificial neural networks offer an attractive paradigm of computation for the synthesis of agent programs for a variety of reasons including their potential for massively parallel computation, robustness in the presence of noise, resilience to the failure of components, and amenability to adaptation and learning via the modification of computational structures, among others"^[4].

Although neural networks have performed well in some areas, there are others where they give disappointing results. Fortunately, the areas that neural networks perform poorly in are exactly the areas that traditional AI approaches have excelled in. For example, problems that require large amounts of conditional statements (expert systems), and problems that require large space-state searches are not well suited for neural networks.

The development of neural networks to solve real world problems will probably present a paradigm shift to the typical programmer. The typical programmer views computer code as an entity that accepts well-defined input, conforms exactly to some formal specification, and performs its operations in a well-defined manner. That is, typically when the same program is given the same or similar data it will generally perform the same operations and often will produce the same output. Of course the program may change, i.e. adding something to its database, but it will always change in a well-known way. This whole way of thinking is completely invalid when considering neural networks. Neural networks can accept noisy input that does not necessarily conform exactly to some specification. The same neural network will often produce a completely different result when given similar data. In fact, neural networks often make mistakes, like humans, and if allowed to, a neural network will continually improve or alter itself in an unknown fashion. Therefore, the skills needed to develop neural network software are different from those of the traditional programmer. Firstly, some of the developers need a high level of mathematical training. The theory of neural networks is still young and based largely on mathematics. To understand and implement the results of research, and perhaps to develop some additional theory when needed, math will be needed. In the future, neural network libraries or packages may be developed that eliminate this need, but for now it is not the case. Secondly, many of the developers should have some knowledge and training in the areas of psychology and biology. Neural networks are models of the brain. Psychologists study the brain from one realm, the mental, and biologists study the brain from another viewpoint, the physical. Both viewpoints are necessary to the development of neural networks. Psychology can be used to train and test the neural networks as well as understand the thought processes of the neural networks. Biological training can be used in the design of the neural networks and the refinement of the evolutionary process that is usually used to train the neural network.

Besides having an impact on the skill set of the staff, the use of neural networks also has other impacts on the software engineering life cycle. For one thing, during the design phase, one should be very careful to separate the code that implements the neural network from other parts of the program. This should be a completely isolated module. If for no other reason, it may be necessary to completely replace the neural network at some point in the future. However, as was the case with the evolutionary approach to artificial intelligence, the use of neural networks most greatly impacts the latter stages of development, including implementation, testing, and maintenance.

Since it is most often the case that evolutionary techniques are, in fact, used to their fullest ability while developing neural networks, all the effects of evolutionary techniques on the life cycle are relevant with neural networks as well. However, neural networks have the ability to learn and this requires additional considerations among software engineers. The biggest concern is the testing/training of a neural network. There is absolutely no way to ensure a neural network will always work correctly. Not only will the programmer have little clue of how the neural network is working, but it is also the case that the neural network can change itself over time. Furthermore, unlike traditional programs, neural networks cannot always be classified as working and not working. They may work near perfection in some situations, falter in other situations, and completely fail in a few situations. Therefore, more than normal amounts of test data needs to be run through the neural network. Not only will this help assign some statistical level of confidence to the correctness of the neural network, but it will also simultaneously train the neural network.

What happens if the neural network works well during the development stage, but becomes corrupted after deployment? This is a serious problem that needs to be addressed. One possible solution is to ship the neural network with a reset function that reverts the configuration of the neural network back to some default configuration. How would one detect that a neural network is going bad? One possible answer would be to allow the user to do a monthly or weekly test of the neural network. The software packaged with the neural network could read a test file and run the data though the neural network, evaluating its response with the correct answers. The test data should not, however, be the same each month, less the neural network will learn the correct answers to this specific data, rendering the test useless. The test data could be downloaded by the user every month or so. This may be inconvenient but it is not much different from the user downloading ".dat" files for his/her virus software every month.

Neural networks are designed to behave like the human brain. Therefore, the development of such systems, perhaps, requires more ethical considerations than normal software. Furthermore, neural networks must be considered thinking entities, possibly with intentions in conflict with the purposes for which they were designed. Would it be ethical to design a neural network on a robot with a physical instantiated model of a biological nervous system, hence allowing the robot to possibly experience pain? This is a very relevant consideration. Would it be advisable to allow an agent powered by a sophisticated neural network to have unsupervised access to the Internet for a long period of time? In summary, the learning ability of neural networks needs to be considered during their development and deployment as software products.

3. CONCLUSIONS

The inclusion of artificial intelligence in a project invariably alters the common software engineering models taught and used in modern computer science. In choosing to include artificial intelligence, the project leaders must understand the benefits and limitations of the many forms of artificial intelligence as well as the effect each design has on the project development cycle. Failing to do so, and treating the artificial intelligence development as a common programming project, the project leaders will certainly encounter problems in allocating simple resources such as time and manpower.

Monotonic Logic, the most basic form of artificial intelligence, may be implemented without much trouble. However, determining the correct logical rules may take a significant amount of effort, as is the case with expert systems. In any case the design requires, at least, an understanding of Boolean logic. Otherwise, the resulting program will not be reduced to the simplest and most efficient form. The programming phase is equivalent to any other programming project and should be treated as such. Testing and maintenance will both demonstrate the limitation of this form of artificial intelligence, which is the inability to surprise the user with a new decision. As the limitations become well known, it may be necessary to return to the design phase and improve the decision algorithm.

Non-Monotonic Reasoning is often developed in the same manner as monotonic logic. Reasoning, as well as any non-monotonic logic, requires more design than most common programming projects because the data used will not be discrete. Uncertainty must be taken into account before the programming takes place. Many times, an entire programming project is required to build and test a database of expected reactions before the programming of the artificial intelligence begins. Then, because of the uncertainty, testing the non-monotonic logic will take longer than a similar monotonic logic project. Even with full testing, non-monotonic logic is often based on statistical information that changes over time, requiring further testing and updates after the project is completed.

Generational and Evolutionary programming does not usually fit within the constraints of a standard software engineering model. There are many engines available to evolve code over generations. This makes the programming phase minute unless one considers the evolution to be part of programming and not testing. If so, the evolution process may take anywhere from a few hours to a few years depending on the level of artificial intelligence desired. Testing is by far the most difficult part of this sort of project. Most failures in programs evolved over many generations are the result of failures in the testing procedures. Often, a human can judge when responses are correct, but the speed of development is greatly hindered when a human has to judge each generation for correctness. Therefore, programs are written to test the responses of each generation. Any error in the testing program will likely result in an error in the final artificial intelligence product. It should also be noted that this form of artificial intelligence is rarely considered complete. It can always evolve further to provide more correct responses in a shorter amount of time with little effort on the part of the project team. Further evolution only requires another pass through some large number of generations.

Learning agents are, by far, the most difficult artificial intelligence products to develop. They have the same problems as generational and evolutionary projects because they are developed in the same way. The main difference is that the final product must be taught before it can perform. No software engineering models include a teaching phase. It may also be necessary to include a re-teaching phase after release when a product has learned incorrect responses and begins performing poorly. However, this is the most beneficial form of artificial intelligence. Assuming that the project leaders can alter their engineering model as necessary, a learning agent can easily customize itself to each end user. Therefore, one project period results in a multitude of products that may fit many different needs. If enough foresight is available, the products may even update themselves, reducing the need for extensive manpower in the end user support department.

Four major areas of artificial intelligence have been analyzed, each demonstrating an increased change in the five common parts of the software engineering life cycle. Therefore, it is apparent that as the research in artificial intelligence techniques advances, the software engineering models must also advance to account for new challenges and concepts throughout the engineering process. However, it is possible to plan ahead and avoid the artificial intelligence pitfalls.

4. GRAPHS AND CHARTS

Much of the information in this paper may be relayed quickly through graphs and charts. These are intended as quick reference summaries to aid in the software engineering process. They are not derived from test data. All results are purely theoretical, but expected to be accurate enough to help increase the success of artificial intelligence laden software engineering projects.

4.1. Model Change Table

The following is a quick reference chart for software project managers. It clearly shows the increasing level of alteration to the general software engineering model as the complexity of artificial intelligence increases. A brief summary of the alterations follows the table.

Table 1: Model Change Table

	Monotonic Logic	Non-Monotonic Reasoning	Evolutionary Development	Learning Programs
Analysis				A
Design			B	C
Implementation		D	E	F
Testing	G	H	I	J
Maintenance	K	L	M	N

A. During the Requirements Analysis phase, it is important to determine what knowledge is intrinsic to the program, what the program is intended to learn, and how the program should react to new knowledge.

B. In Evolutionary Development, the Design phase is not normally focused on how to solve the problem, but how to rate a solution. Careful attention to detail is required to design a sound method of coding and decoding solutions as well as how to test those solutions as they evolve.

C. Similar to 'B', the learning program usually evolves through solution testing, not problem solving. It will be helpful to develop the education tools for the program to learn from at this point. Otherwise, the implementation may evolve without regard for proper learning.

D. The implementation of many Non-Monotonic programs will include a long period of data entry. Initially, the program should be tested on a small set of critical data, and then advanced to the full data set.

E. Expect to spend many hours awaiting the first signs of intelligence from the evolutionary solutions. It is very difficult to determine the amount of time required for a proper solution to evolve. Therefore, the schedule may easily jump ahead of schedule or fall far behind schedule in this phase.

F. Similar to 'E', it is very difficult to predict the amount of time required for evolution. Also expect an extended time for the teaching phase in between generations, drastically increasing the implementation time.

G. Testing Monotonic Logic is difficult because the testers will often expect more out of the solution than it has been programmed to handle. This results in repeated trips back to the Design and Implementation phases.

H. It is not possible to test every possible reaction from most Non-Monotonic solutions. Therefore, an agreed upon certainty level must be reached. That certainty level should be described in detail before testing begins or it will not be possible to tell when testing ends.

I. Similar to 'H', an evolutionary solution can only be tested to a level of certainty. Also take into account the ability to quickly send solutions back to the Implementation phase for further evolutionary development at much less cost in time and effort than a standard programming project. Due to this, the Testing and Implementation phases become less distinguishable.

J. Similar to 'I', this can only be tested to a level of certainty and it is nearly effortless to evolve solutions further. However, the level of learning ability must also be tested to a level of certainty to weed out bad 'learned' behavior. Learning solutions tend to form habits quickly, good or bad.

K. Expect to be required to come out with a more advanced AI rather quickly as the users discover the limits of the program's ability.

L. There are two major problems to deal with: change of data and typos in initial data. Both require a new database for the AI to use for making decisions.

M. After development, it is possible (if not expected) to have a more evolved solution in a short period of time. Expect to continually roll out a newer version that will update the older ones. An auto-update feature would be very handy.

N. Similar to 'M', updates will happen often. Also, there will be a need to 'rehab' products that have learned bad habits.

4.2. Team Resource Charts

As different forms of artificial intelligence are implemented, the team resources required will change. The following are pie charts that provide a general basis for deciding which parts of the overall software engineering process will require the most resources (and attention). The parts of the software engineering process are labeled as:
A: Analysis
D: Design
I: Implementation
T: Testing
M: Maintenance

Figure 4: Team Resource Charts

The division between Implementation and Testing in Evolutionary Development and the Learning Computer is purposely fuzzy. As described in the paper, these two phases of development are run in conjunction with one another through many iterations during the engineering project.

Please note that the implementation time for Evolutionary Development and the Learning Computer is very volatile. By pure chance, it may be completed in a very short period of time. If luck is not with the programming team, implementation could take far longer than expected.

5. REFERENCES

1. Anthony Aaby. http://cs.wwc.edu/~cs_dept/KU/Logic/Nonmonotonic.html
2. Knuth, Donald E. The Art of Computer Programming Volume 1: Fundamental Algorithms (Third Edition). Addison-Wesley, Reading MA, 1997.
3. Luger, George F. and Stubblefield, William A. Artificial Intelligence. Addison-Wesley, Reading MA, 1997.
4. Patel, Honavar, and Balakrishnan. Advances in the Evolutionary Synthesis of Intelligent Agents. MIT Press, Cambridge MA, 2001.
5. Pfleeger, Shari Lawrence. Software Engineering: Theory and Practice (Second Edition). Prentice Hall, Upper Saddle River NJ, 2001.
6. Rich, Elaine. Artificial Intelligence. McGraw-Hill, New York NY, 1983.