Knowledge Management Success/Effectiveness Models

Introduction

The first two articles in this series discussed Knowledge Management (KM) and Knowledge Management Systems (KMS). Summarizing these two articles we concluded that KM is the retention of experience, knowledge, information, and data about events in an organization that are then applied to future events to support decision-making. The KMS is the system an organization builds to implement KM by supporting the capture, storage, search, retrieval, and application of knowledge. This includes the management support, processes, and IT applications and components necessary to support these activities. A holistic, Churchmanian (1979) view of KM and the KMS reveals these two concepts to be closely linked. So closely linked that it is unlikely we can have one without the other or that one can succeed without the other being successful.

The third article expanded this discussion by identifying those critical success factors necessary for Knowledge Management and/or Knowledge Management Systems to succeed. It also defines KM and KMS success as encouraging or helping users to use the KMS to effectively perform KM functions.

This leads to the topic for this article, how to model KM and KMS success. This article surveys KM/KMS success/effectiveness models that combine theory and critical success factors to further refine the concept of KM/KMS success and to explain how to implementing critical success factors leads to KM/KMS success.

Why is it important to determine KM/KMS success? Turban and Aronson (2001) list three reasons for measuring the success of a Knowledge Management System, KMS:

• To provide a basis for company valuation
• To stimulate management to focus on what is important
• To justify investments in KM activities.

All are good reasons from an organizational perspective. Additionally, from the perspective of KM academics and practitioners, the measurement of KMS success is crucial to understanding how these systems should be built and implemented.

To meet this need several KM and/or KMS success models are found in the literature. Models of KM success are included as a Churchman (1979) view of a KMS can be defined to include the KM initiative driving the implementation of a KMS (also the counter view is valid as looking at KM can also include looking at the KMS).

We previously defined KM/KMS success as encouraging or helping users to use the KMS to effectively perform KM functions. This isn't a sufficient definition as researchers are finding it difficult to quantify results of KM/KMS efforts. This article presents several KM/KMS success models. Two basic approaches are used to determine success. The first looks at the effective implementation of KM processes as the indicator of a successful implementation with the expectation that effective processes will lead to successful knowledge use. These models identify KM processes by looking at KM/KMS success factors. The second approach looks at identifying impacts from the KM/KMS implementation with the expectation that if there are impacts from using knowledge then the KM/KMS implementation is successful. These models consider success a dependent variable and seek to identify the factors that lead to generating impacts from using knowledge. The following models, found through a review of the literature, use one or both of these approaches to determine KM/KMS success.

Knowledge Management Success Models

Bots and de Bruijn: Knowledge Value Chain

Bots and de Bruijn (2002) assessed KM and determined that the best way to judge good KM was through a knowledge value chain. Good KM is defined as using KM to improve organizational competitiveness. However, measuring KM impact on competitiveness is considered difficult so ultimately it was concluded that good KM is when the KM initiative matches the model provided in figure 1 and the KM processes are implemented well. KM is assessed for effectiveness at each step of the knowledge process and is good if each of the indicated activities is performed well with the ultimate factor being if the KM enhances competitiveness. Figure 1 illustrates the KM value chain. The model was developed by viewing and contrasting KM through an analytical (technical) perspective and a actor (user) perspective. These perspectives are conflicting and KM assessment occurs by determining how well the KMS meets each perspective at each step.

Massey, Montoya-Weiss, and Driscoll KM Success Model

Massey, et al. (2002) present a KM success model based on their Nortel case study. The model is based on the framework proposed by Holsapple and Joshi (2001) and reflects that KM success is based on understanding the organization, its knowledge users, and how they use knowledge. It recognizes that KM is an organizational change process and KM success cannot separate itself from the organizational change success with the result the KM success is essentially defined as improving organizational or process performance. The model is presented in Figure 2. Key components of the model are:

• KM Strategy — defines the processes using knowledge and what that knowledge is, the sources, users, and form of the knowledge, and the technology infrastructure for storing the knowledge.
• Key Managerial Influences — defines management support through leadership, allocation and management of project resources, and oversight of the KMS through coordination and control of resources and the application of metrics for assessing KMS success.
• Key Resource Influences — these are the financial resources and knowledge sources needed to build the KMS.
• Key Environmental Influences — describe the external forces that drive the organization to exploit its knowledge to maintain its competitive position.

Lindsey KM Success Model

Lindsey (2002) considered KM success as being defined by Kaplan and Norton's (1992) Balanced Scorecard approach and propose a KM effectiveness model based on combining Organizational Capability Perspective theory (Gold, et al., 2001) and Contingency Perspective Theory (Becerra-Fernandez and Sabherwal, 2001). The model defines KM effectiveness in terms of two main constructs: Knowledge Infrastructure Capability and Knowledge Process Capability, with the Knowledge Process Capability construct being influenced by a Knowledge Task. Knowledge infrastructure capability represents social capital, the relationships between knowledge sources and users, and is operationalized by technology (the network itself), structure (the relationship), and culture (the context in which the knowledge is created and used). Knowledge process capability represents the integration of KM processes into the organization, and is operationalized by acquisition (the capturing of knowledge), conversion (making captured knowledge available), application (degree to which knowledge is useful), and protection (security of the knowledge). Tasks are activities performed by organizational units and indicate the type and domain of the knowledge being used. Tasks ensure the right knowledge is being captured and used. KM success is measured as a combination of satisfaction with the KMS and the effectiveness of KM processes. Figure 3 illustrates the Lindsey model.

Jennex Olfman KMS Success Model

Jennex and Olfman (2004) present a KMS Success model that is based on the DeLone and McLean (1992, 2003) IS Success Model. Figure 4 shows the KMS Success Model. This model evaluates success as an improvement in organizational effectiveness based on use of and impacts from the KMS. Descriptions of the dimensions of the model follow:

• System quality — defines how well the KMS performs the functions of knowledge creation, storage/retrieval, transfer, and application; how much of the OM is codified and included in the computerized portion of the OM, and how the KMS is supported by the IS staff and infrastructure.
• Knowledge/Information quality — ensures that the right knowledge/OM with sufficient context is captured and available for the right users at the right time.
• Use/User Satisfaction — indicates actual levels of KMS use as well as the satisfaction of the KMS users. Actual use is most applicable as a success measure when the use of a system is required. User satisfaction is a construct that measures satisfaction with the KMS by users. It is considered a good complementary measure of KMS use when use of the KMS is required, and effectiveness of use depends on users being satisfied with the KMS.
• Perceived Benefit — measures perceptions of the benefits and impacts of the KMS by users and is based on Thompson, Higgins, and Howell's (1991) Perceived Benefit Model. It is good for predicting continued KMS use when use of the KMS is voluntary, and amount and/or effectiveness of KMS use depends on meeting current and future user needs.
• Net Impact — An individual's use of a KMS will produce an impact on that person's performance in the workplace. Each individual impact will in turn have an effect on the performance of the whole organization. Organizational impacts are typically not the summation of individual impacts, so the association between individual and organizational impacts is often difficult to draw that is why this construct combines all impacts into a single construct. This model recognizes that the use of knowledge/OM may have good or bad benefits and allows for feedback from these benefits to drive the organization to either use more knowledge/OM or to forget specific knowledge/OM.

Maier KMS Success Model

Maier (2002) also proposes a KMS success model based on the DeLone and McLean IS Success Model (1992). This model is similar to the Jennex Olfman model. Breakdown of the dimensions into constructs is not provided but specific measures for each dimension are identified. This model is illustrated in Figure 5 and uses the following dimensions:

• System Quality — taken directly from DeLone and McLean (1992) and refers to overall quality of the hardware and software.
• Information, Communication, And Knowledge Quality — refers to the quality of the stored data, information, and knowledge and to the quality of knowledge flow methods.
• Knowledge-Specific Service — refers to how well subject matter experts and KMS managers support the KMS.
• System Use/User Satisfaction — taken directly from DeLone and McLean (1992) and refer to actual KMS use and the satisfaction users have with that use.
• Individual Impact — taken directly from DeLone and McLean (1992) and refers to the impacts KMS use has on the individual's effectiveness.
• Impact On Collectives Of Peoples — refers directly to the improved effectiveness within teams, work groups, and/or communities that comes from using the KMS.
• Organizational Impacts — taken directly from DeLone and McLean (1992) and refers to improved overall organizational effectiveness as a result of KMS use.

Future Trends

There are two areas needing research. The first is in defining KM/KMS success and quantifying the factors that define or reflect this success. Two promising approaches are in identifying success measurements with respect to DeLone and McLeans (2002) IS Success Model or to Kaplan and Norton's (1992) Balanced Scorecard approach.

The second area is to improve generalizability of the models by establishing quantitative support across a broad range of organizations and users. This is necessary for showing that the models are not just reflective of the conditions observed in the case study leading to their generation.

Conclusions

Many KM/KMS Success/Effectiveness models have been proposed. Most are based on case study research that looked in depth at KM or a KMS in an organizational setting. This type of research yields good insight into organizational and user processes and mechanics giving researchers excellent perspective from which to build models that explain their observations. Additionally, all of these models have some level of theoretical foundation. Use of established theory coupled with observation yields useful models. The models presented in this chapter are all useful to researchers exploring KM and KMS success/effectiveness. Also, these models are useful to practitioners and KMS designers as they provide guidance into what to look at when determining KM and KMS success and what to include in a KMS.

REFERNCES

1. Alavi, M. and Leidner, D.E. (2001). Review: Knowledge Management and Knowledge Management Systems: Conceptual Foundations and Research Issues. MIS Quarterly, 25(1), 107-136.
2. Becerra-Fernandez, I., and Sabherwal, R. (2001). Organizational Knowledge Management: A contingency perspective. Journal of Management Information Systems, 18(1), 23-55.
3. Binney D. (2001). The Knowledge Management Spectrum: Understanding the KM Landscape. The Journal of Knowledge Management, 5(1), 33-42.
4. Borghoff, U.M. and Pareschi, R. (1998). Information Technology for Knowledge Management. Berlin: Springer-Verlag.
5. Bots, P.W.G. and de Bruiin, H. (2002). Effective Knowledge Management in Professional Organizations: Going by the rules. 35th Hawaii International Conference on System Sciences, IEEE Computer Society Press.
6. Churchman, C. W. (1979) The Systems Approach (revised and updated) New York: Dell Publishing.
7. DeLone, W.H. and McLean, E.R. (1992). Information Systems Success: The Quest for the Dependent Variable. Information Systems Research, 3, 60-95.
8. DeLone, W.H. and McLean, E.R. (2003). The DeLone and McLean Model of Information Systems Success: A Ten-Year Update. Journal of Management Information Systems, 19(4), 9-30.
9. Gold, A.H., Malhotra, A. and Segars, A.H. (2001). Knowledge Management: An Organizational Capabilities Perspective. Journal of Management Information Systems, 18(1), 185-214.
10. Hahn, J. and Subramani, M.R., (2000). A Framework of Knowledge Management Systems: Issues and Challenges for Theory and Practice. Proceedings of the Twenty-first International Conference on Information Systems, Association for Information Systems, 302-312.
11. Holsapple, C. W., and Joshi, K.D. (2001). Knowledge Management: A Three-Fold Framework. The Information Society, 18(1), 47-64.
12. Jennex, M.E. and Olfman, L. (2002). Organizational Memory/Knowledge Effects on Productivity, A Longitudinal Study. 35th Hawaii International Conference on System Sciences, HICSS35, IEEE Computer Society.
13. Jennex, M. E. and Olfman, L. (2004). Modeling Knowledge Management Success. Conference on Information Science and Technology Management, CISTM.
14. Kaplan, R.S. and Norton, D.P. (1992) The Balanced Scorecard, The Measures That Drive Performance. Harvard Business Review.
15. Lindsey, K. (2002). Measuring Knowledge Management Effectiveness: A Task-Contingent Organizational Capabilities Perspective. Eighth Americas Conference on Information Systems, pp. 2085-2090.
16. Maier, R. (2002). Knowledge Management Systems: Information and Communication Technologies for Knowledge Management. Berlin: Springer-Verlag.
17. Marwick, A.D., (2001). Knowledge Management Technology. IBM Systems Journal, 40(4), 814-830.
18. Massey, A.P., Montoya-Weiss, M.M., and O'Driscoll, T.M. (2002). Knowledge Management in Pursuit of Performance: Insights from Nortel Networks. MIS Quarterly, 26(3), 269-289.
19. Nonaka, I., (1994). A Dynamic Theory of Organizational Knowledge Creation. Organization Science, (5)1, 14-37.
20. Stein, E.W. and Zwass, V., (1995). Actualizing Organizational Memory with Information Systems. Information Systems Research, 6(2), 85-117.
21. Thompson, R.L., Higgins, C.A., and Howell, J.M. (1991). Personal Computing: Toward a Conceptual Model of Utilization. MIS Quarterly, 15(1), 125-143.
22. Turban, E. and Aronson, J.E. (2001). Decision Support Systems and Intelligent Systems. Sixth Edition, Prentice Hall.
23. Zack, M.H., (1999). Managing Codified Knowledge. Sloan Management Review, 40(4), 45-58.