Forio's Forum

I’m a big science fiction buff. I like thinking (and reading) about the future. So perhaps it is no surprise that my favorite way of thinking of computer simulations is as a dynamic story. Similar to a good novel (but with an infinite number of possible endings), a solid story makes a simulation exciting, credible, and memorable.

Once Upon a Time…

A simulation story contains all the standard elements: setting, plot, theme and even a moral.

Setting – Getting the setting right is key to an engaging simulation. Users must be able to get up to speed quickly so that they can begin play. This means that they must understand what the simulation is about. The setting includes the type of business the simulation is about, what role the user plays and (most importantly) what type of decisions users make. Without a good setting, users will find the simulation abstract, difficult to understand, and disengaging.

Plot – Users usually remember the plot of their simulation experience. The specifics will vary from user to user, but it usually starts in the beginning with an exposition of the case study. Drama and tension build as the user struggles with the business case. Frequently, it concludes with a dramatic twist either positive (high score!) or more frequently– negative (crashed the business!) One interesting design issue how often will users be able to run the game and still enjoy it? (In other words, are their a multitude of possible plots, or only one main one with a few variations).

Theme - The sponsors of a simulation are often concerned with the theme, i.e. the learning objectives and business issues built into the simulation. Often these involve trade-offs between conflicting business goals. An e-business simulation might allow users to struggle with “bricks or clicks”, that is the tension between opening up an online channel to sell goods and cannibalization that implied of traditional retail outlets. A common conflict is between the cost of an intangible investment (for example, in customer service) and its return. Without one or more strong themes the simulation will be entertaining but have little value.

Moral - The moral of the story are the take-home points from the simulation. What did the user do? What should the user have done differently? What are the points they should have learned?

A Simulation Game in One Act

A colleague of mine once played a business simulation game in a micro-economics college course where his grade was partially based on our score in the game. As a budding professional in simulation development, he was determined to beat the system.

The game involved offering a price for a product with a goal of maximizing profit. The game was played for 10 rounds (with 5 initial practice rounds) using a simple set of decisions and results.

My colleague surmised that there was a model behind the scenes that looked something like this:

After playing a few practice rounds he graphed the price demand curve and confirmed that the relationship was a smooth curve. He then determined the optimum price and offered the same price again and again. By the end of the game, he was thoroughly bored, although he had the highest score in the class. However, the next time there was a simulation assignment, he forgot to go to the lab and play the game!

My point is not to illustrate the study habits of my friend, but to note that simulations with simple or absent storylines can be deadly dull and (often) either are too hard to understand or (in this case) too easy.

A Simulation with Zest

Clearly, the previous simulation described could have been more engaging with a better story. Imagine a different scenario…

The student “Frank” sits at the computer, and reads a memo, flashed on the screen on corporate letterhead from “SafePath Grocery Store”. It’s from Bob Smith, store manager.

“Frank, I’m going on vacation for the next 8 weeks and am handing management of the store over to you in the meantime. As you know, it’s ‘Back to School’ time our second most important season. It’s important you make the right decisions, or our store will lose market share to the competition just before Christmas. I’ve asked Jack Johnson, our manager from the store down the road, to drop in from time to time and offer suggestions. I’ll award you a bonus of 10% of any increase in profits while I am gone. I’m sure you’ll do well. Good luck!”

As Frank examines the various simulation screens, he notices two things. The “Decisions” screen has only one item “BlixCreem Price”. A recent financial statement lists BlixCreem toothpaste sales as 500 units per week. A “post-it” note on the report suggests, “HQ suggests we might be able to raise the volume of these if we lower the price. What do you think? Bob.”

Frank runs the simulation for two weeks, playing with different prices on the BlixCreem toothpaste. He successfully raises the profit on that item by 25%.

On Day 10, a picture of store manager “Jack Johnson” appears on the screen and a written note appears: “Frank, heard that you had some success boosting sales of BlixCreem. Nice job we’ve followed your lead and have done the same. Unfortunately, our main competitor “A-Mart” has also lowered their price on their main toothpaste brand. We can expect some rough times ahead. Our regional marketing department should be getting in touch with you immediately.”

Frank immediately reviews the latest day’s sales reports and sees that BlixCreem sales have dropped down from the day before. He then notices that his decision page has more than just the Price decision he had previously. Now he has “Marketing Budget” as well.

And the simulation continues

At the end of the simulation, Frank submits a report to his instructor containing a write-up of his business strategy, a printout of the simulation listing his high score, and a diagram drawing his view of the key business relationships in the simulation. (Which looked something like the following)

Guidelines for Writing A Simulation Story

As the previous anecdote illustrates, creating a good simulation story involves both content and technical design. The following characteristics are important:

• Quick to get started. Right at the beginning of the sim, there should be a brief overview explaining the case with a charge (and specific goal) for the user.
• A model of increasing complexity. The cause-and effect relationships should be easy to enough to understand and initially master. However, the user should discover unexpected results as things progress.
• All decisions should have tradeoffs. There’s a reason the user is running the simulation in the real world there is rarely an obvious “right answer”. There should never be a decision that doesn’t have a cost or other negative potential consequence.
• User feedback is provided along the way. Messages should appear (based on the simulation results) that provides coaching and ideas, giving insight into the cause-and-effect relationships behind the simulation.

Conclude with a Debriefing

Not described in the above example is the debriefing. After every simulation, an analysis of the simulation helps the learner to process his experiences and turn them into a deeper understanding of the business issues. In a classroom setting, this might be instructor driven. However, online simulations can do this as well through one or more of the following:

• Analysis of user simulation play, highlight areas that brought success or poor performance
• Written discussion of business issues in both the simulation and real world
• Questionnaire asking the user to reflect on their strategy and performance
• Discussion groups for online class sharing their simulation experiences

A successful simulation will engage the learner, address the relevant content, and help the user to understand the cause-and-effect relationships that are driving the business simulation. Of these items the last is the most critical. In the end a simulation is not about presentation, but is about interactive learning. A good story is a key part of making this happen.

A person in the formal role of a leader may not possess leadership skills nor be capable of leading. Leadership is essentially related to a person’s skills, abilities and degree of influence.

A good deal of leadership can come from people who are not formal leaders. Leading is the result of using one’s role and leadership ability to influence others in some way. True leaders are not “bosses” or “commanders”. Instead of power, true leadership comes from influence, congruence and integrity.

Successful leaders are committed to “creating a world to which people want to belong”. Successful leadership involves managing relationships and communicating within a team to move towards a specific goal. Leadership is the ability to: “express a vision, influence others to achieve results, encourage team cooperation, and be an example.”

Strengthening your leadership ability can help you improve your capacity to achieve results and reach personal or organizational outcomes. Leadership is not the same thing as management. Management is “getting things done through others.” Leadership is “getting others to want to do things.” Leadership is intimately tied up with motivating and influencing others.

The Four Basic Actions of World-Class Leaders
Although leadership is a complex capability, successful leadership is linked to four core actions: stretching, empowering, sharing, and coaching. These four actions support and expand the core managerial competence of the leader.

Stretching is the ability to challenge a teams habits and to take risks. Stretching involves the capacity to create challenging situations, to compel, to push towards doing more, to go beyond. Stretching is necessary to promote change and achieve results.

Empowering is the ability to help others achieve their individual potential in order to obtain more effective organizational behavior. Empowering requires the capacity to facilitate conditions which allow people to express themselves better, recognizing the value of their work and stimulating personal and professional growth as well as self esteem. Empowering is necessary to achieve results and develop people.

Coaching is the ability to be a guide and a trainer. Coaching is based on the capacity to respect people, to listen attentively, willingly, and considerately. It requires the recognition of individual potential and taking responsibility for the development of these competencies as assets in order to harvest underutilized potential. Coaching is essential in order to develop people and realize values.

Sharing is the ability to exchange information and know-how. Sharing involves the capacity to involve people with respect to objectives, including them in meetings in which ideas and information are exchanged, in order to achieve true collaboration, and permitting easy access to resources and acknowledging that they are to be enjoyed by all. Sharing is required for realizing values and promoting change.

The Four Actions of World-Class Leaders diagram and description is used here with permission from Isvor-Dilts Leadership Systems. © 2003 Isvor-Dilts. All rights reserved.

Picture the following: A group of managers are playing a simulation game in which they run a chain of retail convenience stores. This particular chain was growing rapidly until the last few years when sales have slacked dramatically. The managers decide to offer a major promotion and price discount to boost sales. Unexpectedly, the resulting sales show little increase (and in some regions a decrease) and are accompanied by a sharp drop in profits. “This doesn’t make sense!” says one frustrated manager. “Our customers look for bargains. Sales should have gone up. Where’s the increase?”

People have a variety of reactions to a computer simulation that provides an unexpected result. Some will immediately argue that the model is biased or inaccurate. (“garbage in, garbage out”). Others will accept the result as a voice of authority, without critical thought. A third group will try to understand the assumptions driving the unexpected result (this may or may not be possible) and judge if it makes better sense than their previously held assumptions.

We are all familiar with arguments of questionable validity supported by computer models. Some of the conflicting headlines of the past few years are examples:

• Y2k is Worse than Anyone Thought (December 14, 1998)
• Overkilling Y2K (January 3, 2001)

Or on a different subject…

• Everybody Into the Venture Capital Pool (July 26, 1999)
• The Tech Slump: 2 More Years (April 30, 2001)

A Question of Purpose

The number one thing to remember when discussing model validity is that there is no such thing as a valid model. Instead models are more (or less) useful given a set of objectives and boundary conditions. It all depends on the purpose of the model.

Typically, simulations that focus on strategy issues tend to cover a broad range of strategic issues but not include a high level of operational detail. Conversely, simulations that focus on operational issues tend to be very precise regarding the detail, but leave out business factors not relevant to the area of operational focus.

A version of PDA Sim that was intended as a market analysis tool for a specific company would be built around a model that was calibrated to a finer level of accuracy. A PDA Sim that was used by marketing staff in Germany to test specific promotions would have a detailed model of the German market but a very high level model on broader corporate issues.

One of my favorite writings on this issue is A Skeptic’s Guide to Computer Models (PDF), by MIT Professor John Sterman. In this article he notes: “Beware the analyst who proposes to model an entire social or economic system rather than a problem. For the model to be useful, it must address a specific problem and must simplify rather than attempting to mirror in detail an entire system… The art of model building is knowing what to cut out, and the purpose of the model acts as the logical knife.”

Best Practices in Model Validation

Experienced simulation designers go through several phases of model validation.

1. Verify the model assumptions

Working with subject matter experts, articulate and confirm each cause and effect relationship. Modeling disciplines such as System Dynamics help tremendously by representing assumptions in a visual diagram that can be easily verified. This is an iterative process that is performed simultaneously with the construction of the model. It begins with design meetings and interviews of company managers, includes reviews of high level structure with company generalists, and detailed reviews of model assumptions with subject matter experts.

2. Verify the model technical structure

Check the simulation model against a set of technical rules. These rules, which vary depending on the type of model, ensure that the assumptions are consistent and the equations doesn’t violate requirements in the modeling discipline. For example, when I was hiking a few years ago, I started at a trail head with a sign that said “8.6 miles to Highpoint Falls”. When I got to HighPoint Falls, a similar sign pointing back read “8.1 miles to trailhead”. These signs violated an obvious technical rule. (it’s worth noting that even if the signs had passed the “technical structure” test, other validation needs to be done to ensure they represent the right distance!).

3. Validate the model behavior

Once the model is built, test to see that the results of the simulation match expected behavior. There are a number of useful techniques that modelers can use. There is a significant body of literature that provides more information on best practices in this area.

• Extreme condition testing: Run the simulation with parameters set at extreme levels (price of $0, price of $1,000) checking the model results for consistency (for example, with a price of $0, the revenue should be $0 and the profit should be negative).
• Sensitivity testing: Run the simulation multiple times varying each parameter a bit higher and a bit lower. Look for parameters that cause the results to change significantly (and then pay extra attention to validating those parameters).
• Calibration & optimization: Use automated tools that apply algorithms such as Hill-Climbing or Genetic Algorithms to adjust each parameter until the result matches a predetermined value or time series.

4. Test business policies

In this final stage, modelers test alternative sets of decisions and policies, confirming that simulation produces results that are realistic and make sense. (Often this step confirms that the scope of the simulation is appropriate to the problem). If further revisions are needed, more verification or validation may be done.

Management Training Simulations

Management training simulations are often built with a semi-fictional case study or scenario rather than a detailed predictive model. This allows managers to learn about the key issues in a short amount of time while not being distracted by the need to analyze a detailed forecast of the real business. In addition, the simulation is run as a game, managers may try strategies that they would never implement in the real business. Consequently, training simulations have less detail but broader possible outcomes than a decision-support or predictive model.

The simulation can also present the logic built into a simulation as a means of helping managers to understand how their decisions lead to their business results. PDA Sim has several “help” screens that show a typical product lifecycle curve and a diagram of the key cause and effect relationships in the model. It also features an “advisor” that highlights key business issues. (for example, “Your price is lower than your cost of goods. You are losing money on every sale.”) More complex simulation can have advisors with multiple perspectives, for example a CFO, VP of Marketing, and a customer.

Conclusion

As the opening scenario notes, model validity is often questioned in a training simulation when results are encountered that are unexpected. As the point of most training simulations is to help managers experience new strategies and ideas, this is a common occurrence!

Simulation designers and instructional facilitators need to represent a simulation for what it is: an approximation of reality. Good rules of thumb: Articulate the purpose and learning objectives before building the simulation, apply best practices to model design and validation, then ensure the simulation is used appropriately. When questions about simulation assumptions arise (as they almost always will), guide learners in considering the impact of their decisions in the real world, and help them look for clues as in the simulation to find information that can provide additional insight.

In the retail simulation described above, the managers found a screen that described a customer survey of their chain and their competitors. They discovered that immediately after they lowered their prices, a major competitor did the same. Comparing notes on the impact of this price war, they found out that market shares had barely changed, while profits (and management morale) dropped at both sets of stores. This was very similar to a real-world dynamic that had occurred in their industry about 5 years before.

As a final thought, let me refer back to Professor Sterman in A Skeptic’s Guide (PDF) :

This paper discusses the introduction of system dynamics modeling and scenario planning into the planning process at Chevron. These techniques can help planners by providing an integrated, quantitative perspective, filling a gap between the detailed micro-analysis of spreadsheets and the holistic, but static forms of decision analysis. Specifically, system dynamics simulations can help planners to manage uncertainty and view the consequences of investment decisions in alternative futures.

In this paper we discuss lessons learned from the creation of a pilot simulation focused on gas asset management in Chevron’s Nigeria operations. We briefly look at the process by which we built the system dynamics model. We discuss possible ways these types of simulations might be used in the future at Chevron and in the oil & gas industry. Finally we close by presenting several key lessons learned from the pilot at Chevron that could contribute to the success of disseminating these methods across an organization.

Download the PDF article: Managing Gas Assets through Simulation and Scenario Planning

Clothing and toys are manufactured overseas. So why not make software there too, where labor is cheaper?

Many U.S. technology companies have outsourced their software development to India. Last year Hewlett-Packard became India’s largest multinational IT employer, with more than 10,000 employees.

The enthusiasm for overseas outsourcing and offshoring, mirrors the enthusiasm for Internet companies in the Nineties. In a recent article, Ravi Chiruvolu, a partner at Charter Venture Capital wrote that “Venture Capitalists decided that because of cheap engineering talent in countries like India it would be more cost effective to outsource software development. If Nike could outsource sneaker manufacturing, we could do the same with code.” Following similar logic, Oracle has announced it will more than double the number of software engineers it employs in India to 6,000.

Although the offshoring trend has resulted in a net transfer of jobs outside of the US, this article isn’t about job losses in the United States. We live in a global economy and people in India deserve jobs as much as people in the United States or anywhere else. It’s worrisome when companies are criticized solely because they have hired people overseas.

Offshoring is a mistake when technology companies confuse operational effectiveness and strategy. Operational effectiveness is about working cheaper or faster. Strategy is about the creation of a long-term competitive advantage, which for technology companies is usually the ability to create innovative software.

Outsourcing programmers works when the software developed isn’t a key part of the pipeline of innovation for products a company actually sells. For example, when website design or back-office software such as payroll or inventory control is outsourced, that can be good because it improves operational effectiveness.

But writing innovative software cannot be done on an assembly line. It requires hard-to-find development and design skills. Farming out development to legions of programmers overseas will not create a differentiation advantage. When a technology company outsources software development, that company loses its capacity to innovate and its competitive advantage.

Why Some Software Companies are Confusing the Box for the Chocolates

Recently, I bought some chocolates as a gift for some friends from a specialty shop. These chocolates are remarkable. Owner Jean-Marc Gorce makes them by-hand and his small shop has been rated as one of the top ten in the United States. In addition to being a chef, Jean-Marc is also an entrepreneur and an innovator.

Jean-Marc recently started selling his chocolates in gold and blue boxes. I told him I liked the new boxes. He explained that his wife designed the boxes and he found a company in the Philippines that could produce the boxes in the small volume they needed for a good price.

Jean-Marc’s gold and blue boxes are an example of successful outsourcing. Jean-Marc sells chocolates, not boxes. The design and production of chocolates is his core competency. Jean-Marc can outsource box production to improve his operational efficiency without sacrificing his reputation as a maker of superlative chocolates.

While outsourcing boxes improves chocolatier Jean-Marc’s operational effectiveness, he would never consider outsourcing chocolate production because he would lose his core differentiation advantage. Yet, in their enthusiasm for cost savings, several US technology companies have done precisely that– outsourcing their core technology and key strategic differentiator.

Design and Assembly are Different

This isn’t the first time companies have tried to commoditize software development. In the eighties, Japanese companies unsuccessfully attempted to set up software factories to manufacture programs. They discovered that just throwing a lot of programmers together doesn’t create innovative software.

Unlike software, it makes sense to outsource the manufacture of clothing and toys. Most of the cost of clothing and toy manufacturing is in the assembly, not the design. Those products can still be designed close to corporate headquarters but assembled elsewhere to keep costs low.

Programming is like design and nearly all of the costs of creating software come from writing the program, not the assembly. The assembly stage for software is really just copying the final program onto a disk and enclosing it with a manual in a box.

Harvard Business School’s Michael Porter, a world expert on strategy and competitive advantage, nicely summarized the problem with competing solely on operational effectiveness:

“If all you’re trying to do is essentially the same thing as your rivals, then it’s unlikely that you’ll be very successful. It’s incredibly arrogant for a company to believe that it can deliver the same sort of product that its rivals do and actually do better for very long. That’s especially true today, when the flow of information and capital is incredibly fast. It’s extremely dangerous to bet on the incompetence of your competitors — and that’s what you’re doing when you’re competing on operational effectiveness.”

Ultimately, the offshoring fad is bad for companies not because of the short-term programmer layoffs but because technology companies will lose their capacity to innovate. Tech companies that outsource their programming talent will ultimately be replaced by competition, and then everyone will be losing their jobs.

View Reader Comments on This Article