Exploring Success Factors, Failure Factors, and Barriers to Establishing eLearning Programs in Membership Associations

ASAE Foundation

Scholarly Research in Association Management Grant Program – 2019

Abstract

Association leaders invest in eLearning programs for a variety of reasons including increased non-dues revenue and improved membership learning opportunities. eLearning provides convenience, flexibility and reduced learning costs for members who are often busy and have significant barriers to attending traditional classroom programs. The use of technology has influenced how we learn as adults in a variety of industries. But technology has not been widely adopted in the field of continuing professional education (CPE) outside of an academic environment. Membership associations lag behind their pure academic counterparts in using technology to provide online learning programs, thus limiting educational options for adults who must gain industry certifications to remain relevant in the workforce.

The first phase of the ASAE Foundation Scholarly Research in Association Management Grant Program grant funds revealed critical success factors of eLearning program implementations. However, most of the successes were within the larger associations with ample budgets and a significant membership base, leaving the majority of associations yet to be studied. The purpose of this phase of the research is to gain an understanding of state of eLearning within the association industry in general, and to understand not only success factors, but failure factors, and barriers to entry as well. In order to achieve this goal, the principal investigator (PI) will interview as many associations’ executives as possible to determine the state of eLearning in each organization. Using the success factors discovered in the first round of research, a course entitled Organizational Readiness for eLearning (Org Readiness) will be developed and offered free to association executives willing to participate in the research project. The Org Readiness course will be facilitated by the PI and participants will be presented with a set of success factors that can be used as benchmarks. After determining the current situation in each association, a gap analysis can easily be conducted to determine barriers to entry for those who have not yet established a successful program and also for those who have tried and failed.

The Org Readiness course is essential for this research because, unlike success factors, associations that have failed to establish a successful program do not understand the reasons for failure, or barriers to entering eLearning. The Org Readiness program will provide guidelines so association executives can gain an understanding of where they are in relationship to the essential elements of a successful program.

Statement of the Problem

The problem addressed in the first phase of the research project was the lack of understanding of success factors among associations implementing a eLearning program. The research also revealed that mid-sized to smaller associations were less likely to attempt an eLearning program even though education was significant to the mission of the association. Using the data from the first phase to create an Organizational Readiness for eLearning program (Org Readiness) will enable the principal investigator (PI) to study associations that have not been successful in launching an eLearning initiative. The second phase of research will address the failure factors among associations that attempted to launch a successful program along with barriers to entry for the remainder. By combining this data with the success factors in the first phase of the research, a complete understanding of the state of eLearning among membership associations will be gained.

Because all associations that have educational programs can be studied, rather than only those with successful eLearning programs, the need for qualifying individuals for participation will be greatly reduced. A survey will also be administered to help gauge the characteristics of associations that have been successful along with the perceived barriers to success. The Org Readiness program will help benchmark the current state of ASAE members against the success factors to determine where associations of different sizes lie in the eLearning spectrum. Additional information such as size of the educational department, type of current educational offerings, and membership characteristics will be collected. The overarching objective of the second phase of the research project is to gain a full understanding of the association industry in terms of offering education offerings through the use of technology.

The Need for Continuing Research on eLearning Program Development

Association leaders invest in eLearning for a variety of reasons. eLearning lowers costs by reducing travel associated with training and development, and increases revenue by attracting distance and busy adult learners (Cox & Radwan, 2015). The use of technology has influenced how we learn as adults in a variety of industries. Higher education has adapted eLearning to the extent that most every adult college student in the United States will experience blended or fully online learning programs in the next decade (Allen & Seaman, 2015). But technology has not been widely adopted in the field of continuing professional education (CPE) outside an academic environment. Membership associations lag behind their pure academic counterparts in using technology to provide online learning programs, thus limiting educational options for adults who must gain industry certifications to remain relevant in the workforce.

Online college degrees were first offered in the 1990s, and today 96% of major universities offer some sort of online education (Allen & Seaman, 2010). Corporations followed higher education institutions into online learning creating a multi-billion dollar e-learning industry. The federal government currently offers technology-based training to employees at all levels (Berbary & Malinchak, 2011). Countless books have been written about establishing e-learning programs for corporations, and academic journals are bulging with articles that explain the various aspects of using technology in higher education (Joo, Lim, & Kim,  2012; Li, Dong, & Huang, 2011; Safar 2012). However, an overview of eLearning program success and failure factors yielded information on implementations primarily within the academic sector which leaves a gap in our understanding of why associations have been slow to adopt eLearning.

This study is necessary because research concerning what works, what does not and why is lacking. Studies conducted within the educational sector recommended extending research concerning CSFs of eLearning program development to organizations outside the academic sector (Alhomod & Shafi, 2013). Bitzer, Menschner, and Leimeister (2013) mentioned that the researchers of many eLearning studies focus on technology and other factors such as content and marketing. Examining and reporting on the actual scientific process of establishing success and failure factors, especially within a given industry is also important; and in this study, I will draw on the recent past research concerning critical success factors, and extend it to include associations who have not, for whatever reason embraced eLearning as a revenue stream.

Significance

Adult education is a core competency of both credentialing associations and universities. However a survey of the literature surrounding the subject shows a wealth of information about using technology to provide learning opportunities in higher education, but very little for professional education delivered outside an academic institution. The American Society of Association Executives’ database shows that more than 3,000 organizations offer professional certification programs. A random survey of 100 of these organizations, showed that about half of the associations offer only traditional face-to-face programs, and just less than 30% offer some sort of webinar training, and promote these programs as online learning. Only 15 of the 100 organizations surveyed have online, on-demand programs administered by a learning management system. The first round of research confirmed that larger associations with amble resources tended to have more successful eLearning programs.

One possible barrier to using educational technology is that associations consider traditional stand-up courses to be tested and reliable, and using technology to deliver learning programs can represent a risk in terms of establishing learning outcomes (Kisielnicki & Sobolewska, 2010). There are also issues of selecting technology to build and deploy online courses, and gaining new skills among staff to implement and manage technology-based educational programs. These success factors have been distilled and are presented in the Organizational Readiness for eLearning course to be offered free to all associations participating in the study. Now that we understand the experiences of association executives that have produced successful programs, the research must be extended to include all associations regardless of budget, size, or membership composition. The components of a successful technology-based education program are a set of interlocking, interdependent sub-systems (Saba, 2012) that have been well-explored in literature concerning launching technology-based programs in higher education. Using systems theory, the Org Readiness course is designed to inform association professionals of the various aspects to creating a successful program.

Reaching, educating, and certifying more individuals will create a positive impact on industries served by credentialing associations, while providing additional avenues for career growth for millions of association members. The study will also benefit associations needing to produce non-dues revenue, in a budget-conscious economy. The process of exploring what works, what does not, and why in developing technology-based professional education will provide far reaching benefits.

Conceptual and Theoretical Foundation

The concept of success and failure factors for complex implementations originated in the 1960s and links requirements to successful outcomes. Critical success factors (CSFs) ensure the proper performance of the end user, the department, and the institution, which together form a general system, thus requiring the need to underpin the study with general systems theory (Ram & Corkindale, 2014). Of the known success factors Rockart (1982) determined the end-user experience was the most important factor, and this fact was verified by Whitcomb (2016) in a study among association managers who implemented a successful learning management system. In the first round of funding for this study, Organizational Commitment manifested itself as the most important success factor. Kull, Ellis, and Narasimhan (2013), explained that exploring human resource considerations in program implementations constitutes a need to incorporate sociotechnical systems (STS) theory in establishing a foundation for studying CSFs. STS theory provides a foundation for the study of how humans interact with technology.

Specific Aims – Second Round Research

The first round of research provided an insight into success factors for eLearning programs among associations. The results indicated that large associations with ample budgets were most successful in launching a successful eLearning education program. If associations did not have an eLearning program, they were excluded from the research criteria. This round of research will extend the study to include all associations with an education program regardless of the state of an eLearning program. This will provide a comprehensive look at the association industry as a whole with respect to learning with technology.

Research Design

The first phase of the research project produced a set of critical success factors for eLearning programs. The second phase, addressed in this proposal, will be to expand the research to include all associations that currently offer educational programs regardless of the state of eLearning. This will enable the student workers at S.U. to eliminate the cumbersome activity of qualifying an association for inclusion in the study. The only criteria will be an educational program, and most associations offer educational opportunities. The literature has yielded both a set of success and failure factors along with barriers to eLearning implementations but not within the association industry. The critical success factors discovered in the first phase of the research will be coupled with information obtained in the literature review to form interview questions, surveys, and other empirical investigation tools that contribute to a study of an implementation process researchers have yet to explore in the literature (Ahlan, Kartiwi, & Sukmana, 2015). While associations executives that experienced successful implementations have a good understanding of the success factors, other executives with either a bad experience or no experience may not know what caused a failure or what barriers exist. The Organizational Readiness for eLearning (Org Readiness) course will be available to any association executive that wishes to benchmark his or her organization against the success factors discovered in the first round of grant funding.

NOTE: There were several barriers to concluding the original second round of funding proposal “The Economics of Online Professional Education Exploring the Delta between Design, Delivery, and Demand”. One significant factor was the lack of qualified S.U. students to aid in the research. Building three separate online courses required skills beyond the capability of our students or current curriculum and the training curve was significant. This modified proposal, however, contains research protocol that enables one or more students from our marketing and professional sales program to fully participate in the research. The other barriers such as where to house the course(s) and how to gain exposure for the research project are addressed in this revised proposal. The scaled down Org Readiness course is appropriate for the timeline and resources in this proposal.

The target population will consist of education directors of associations, who manage educational programs regardless of their experience with eLearning. Many of the American Society of Association Executives (ASAE) membership organizations provide continuing professional education for their members. An ASAE membership list will be loaded into a contact management system and one or more student researchers will a) canvas each association’s website to determine the probable education director, b) contact the executive by phone or email, c) invite them to participate in the online survey, d) obtain an appointment for an interview and e) offer the opportunity for one or more individuals from the association to enroll in the Org Readiness course at no cost should they agree to participate in the research. The research instruments (survey, interview questions, and Org Readiness course) will be developed using the data collected from the first round of research, and the resulting report of the findings.

Expected Outcomes

The first phase of the research yielded information concerning critical success factors of eLearning implementations including the following:

The CSFs discovered as a result of this study are as follows:

  1. Understand your Learners
  • Membership Characteristics
  • Familiarity with Technology
  • Demographics
  1. Master the Technology
  • Delivery Technology
  • Course-Building Technology
  • Vendors
  • Integration
  • Video Production
  1. Managing the Content Conversion Process
  • Traditional Classroom Delivery
  • Workshops
  • Webinars
  • Book (Print) to Content
  • Build from Scratch
  1. Develop Knowledge, Skills and Abilities In-House
  • Written Communication
  • Instructional Design
  • Video Production
  • Administration and Support
  • Project Management
  • Course Building Skills

The objective of this phase of the research is to reach beyond the successful associations to determine why so many eLearning programs are not considered successful and what barriers exist in launching an eLearning program. This is necessary because so few associations were qualified to participate in the first round of research due to the fact that there were very few successful programs; and those that were, had been launched by very large associations with ample resources.

Dissemination Plan

The research will result in a report concerning the “State of eLearning in Membership Associations” and include information concerning the status of eLearning projects by a) size of association, b) membership base, c) demographic of members, d) resources, and other concerns. The Org Readiness course will be provided for any association executives who participate in the study and results will provide an insight into the gaps in various aspects required for success. These results can be aggregated and organized by type and size of association as well. The resulting report will be available to all that participate and to ASAE Foundation.

Organizational Readiness for eLearning Course Development – The Org Readiness course will be developed using the equipment and supplies ordered under the original second phase grant. The instructor-led course will be offered through the researcher’s private instance of Instructure Canvas.

Informational Website – The researcher will develop a web presence that explains the study and produce all accompanying collateral so students can contact association executives and encourage participation.

Quantitative Study – A Qualtrics survey will be developed so associations executives can participate whether or not they intend to either a) be interviewed, or b) take the Org Readiness course. The questions on the survey will be informed by the results of the first phase research.

Qualitative Study – Open-ended interview questions will be developed based on the results of the first phase of research and these questions will be applicable to any association whether or not they have been successful in launching an eLearning program.

Bibliography

Ahlan, A. R., Kartiwi, M., & Sukmana, H. T. (2015). Influences of the input factors towards success of an information system project. TELKOMNIKA (Telecommunication Computing Electronics and Control), 13(2) doi:Retrieved from: http://www.journal.uad.ac.id/

Alhomod, S., Alsadhan, A. O., & Shafi, M. M. (2014). The 3C’s model: A framework for development of E-learning courses. Computer and Information Science, 7(1), p87. doi:10.5539/cis.v7n1p87

Alhomod, S., & Shafi, M. M. (2013). Success factors of E-learning projects: A technical perspective. Turkish Online Journal of Educational Technology-TOJET, 12(2), 247-253. doi:Retrieved from http://eric.ed.gov/?id=EJ1015421

Allen, I. E., & Seaman, J. (2010). Learning on demand: Online education in the united states, 2009. ERIC.

Allen, I., & Seaman, J. (2015). Grade level: Tracking online education in the united states. Babson Survey Research Group,

Berbary, D., & Malinchak, A. (2011). Connected and engaged: The value of government learning. The Public Manager, 40(3), 55-59. doi:Retrieved from http://search.proquest.com/docview/914990952?accountid=28711

Bhuasiri, W., Xaymoungkhoun, O., Zo, H., Rho, J. J., & Ciganek, A. P. (2012). Critical success factors for e-learning in developing countries: A comparative analysis between ICT experts and faculty. Computers & Education,58(2), 843-855. doi:10.1016/j.compedu.2011.10.010

Bitzer, P., Menschner, P., & Leimeister, J. M. (2013). Success factors for technology mediated learning services-first results of a delphi study. Die e-Learning Fachtagung Informatik (DeLFI), Braunschweig, Germany. 251-256. doi:Retrieved from http://pubs.wi-kassel.de/wp-content/uploads/2015/04/JML_527.pdf

Chih, Y., & Zwikael, O. (2015). Project benefit management: A conceptual framework of target benefit formulation. International Journal of Project Management, 33(2), 352-362. doi:10.1016/j.ijproman.2014.06.002

Cox, J. B., & Radwan, S. S. (2015). ASAE handbook of professional practices in association management John Wiley & Sons.

Hailu, A., & Rahman, S. (2012). Evaluation of key success factors influencing ERP implementation success. 2012 IEEE Eighth World Congress on Services, 88-91. doi:10.1109/SERVICES.2012.74

Joo, Y. J., Lim, K. Y., & Kim, S. M. (2012). A model for predicting learning flow and achievement in corporate e-learning. Educational Technology & Society, 15(1), 313-325.

Kisielnicki, J., & Sobolewska, O. (2010). E-learning as a strategy of acquiring a Company’s intellectual capital. Interdisciplinary Journal of E-Learning and Learning Objects, 6(1), 153-174.

Kull, T. J., Ellis, S. C., & Narasimhan, R. (2013). Reducing behavioral constraints to supplier integration: A Socio‐Technical systems perspective. Journal of Supply Chain Management, 49(1), 64-86. doi:10.1111/jscm.12002

Li, Y., Dong, M., & Huang, R. (2011). Designing collaborative E-learning environments based upon semantic wiki: From design models to application scenarios. Educational Technology & Society, 14(4), 49-63.

Miranda, P., Isaias, P., & Costa, C. J. (2014). From information systems to e-learning 3.0 Systems’s critical success factors: A framework proposal. Learning and collaboration technologies. designing and developing novel learning experiences (pp. 180-191) Springer.

Moustakas, C. (1994). Phenomenological research methods Sage Publications.

Ram, J., & Corkindale, D. (2014). How “critical” are the critical success factors (CSFs)? examining the role of CSFs for ERP. Business Process Management Journal, 20(1), 151-174. doi:10.1108/bpmj-11-2012-0127

Ram, J., Wu, M., & Tagg, R. (2014). Competitive advantage from ERP projects: Examining the role of key implementation drivers. International Journal of Project Management, 32(4), 663-675. doi:10.1016/j.ijproman.2013.08.004

Rockart, J. F. (1982). The changing role of the information systems executive: A critical success factors perspective. Sloan Management Review, Fall(24), 3-13. doi:10.2307/248989

Saba, F. (2012). A systems approach to the future of distance education in colleges and universities: Research, development, and implementation. Continuing Higher Education Review, 76, 30-37.

Safar, A. H. (2012). The students’ perspectives of online training at Kuwait University. College Student Journal, 46(2), 436-458.

Whitcomb, V. J. (2016). Exploring critical success factors of learning management systems implementations in membership associations. ProQuest Dissertations and Theses

Addendum – Critical Success Factor Research Concerning eLearning in other Industries

Critical success factor (CSF) research concerning eLearning programs has been studied extensively in the higher education industry, but little is known concerning eLearning program success among membership associations. Frimpon (2012) explained that even as online learning in higher education expands, failure rates remain very high and he conducted a study to determine probable causes. Beginning with a literature review, Basak, Wotto and Belanger (2016) explored successful eLearning implementations in higher education to help universities understand relevant issues that could contribute to successful programs. Factors of effective eLearning programs were aggregated and defined by AlShoshan and Zargham (2013) who provided a foundation for further research. And Mosa, Mahrin and Ibrrahim (2016) conducted a study that contributed eLearning readiness to the body of knowledge concerning CSFs for online programs. Universities have similar structures to one another as do membership associations although they provide services in various settings and for different purposes. Research methods to discover and report against CSFs are well defined in a number of industries and for a variety of high-risk, high-reward initiatives.

Critical success factors of eLearning implementations in higher education have been studied from a number of perspectives. Musa and Othman (2012) contributed to the body of knowledge by conducting a study concerning learner perceptions of CSFs in eLearning. Institution involvement, one of the most important factors (Masrom, Zainon & Rahiman, 2008; Parsazadeh, Zainuddin, Ali, & Hematian, 2013) includes all the various resources required to bring a successful eLearning program to market. Sridharan, Deng and Corbitt (2008) explained that the many facets of a successful program are intertwined and there are dependent variables that should be explored to the extent possible. Economic and environmental concerns also contribute to eLearning program success or failure and should be taken into account as well (Sela & Sivan, 2009). Orindaru, Bumbac, and Pamfilie (2013) explored CSFs of eLearning programs that formed as a result of partnerships between universities and sponsors in the private sector. These various research perspectives provide a solid framework for the study of factors that may influence the success of eLearning programs offered by associations because of the varied constituents and industries they serve.

CSFs of eLearning program implementations were also studied in a variety of countries including those in Western Europe and in developing nations. Lin, Ma, and Lin (2011) explored CSFs of eLearning programs in higher education by comparing and contrasting success factors in United Kingdom institutions with those of Taiwan universities and determined that cultural differences must be taken into consideration when launching eLearning programs. Considered essential to the success of higher education in developing countries, Dorobat (2014), Caporarello and Sarchioni (2014), and others studied eLearning program CSFs from the perspective of country involvement and support. Although studying programs launched in India, Puri (2012) produced research that has a bearing on the overall profitability and sustainability of eLearning programs that is applicable in a variety of settings. These studies provide a foundation from which to launch an investigation into successful membership association eLearning initiatives.

Addendum References

AlShoshan, A. A., & Zargham, M. (2013). E-learning factors and definitions. Proceedings of the International Conference on e-Learning, e-Business, Enterprise Information Systems, and e-Government (EEE), 146.

Basak, S. K., Wotto, M., & Bélanger, P. (2016). A framework on the critical success factors of E-learning implementation in higher education: A review of the literature. World Academy of Science, Engineering and Technology, International Journal of Social, Behavioral, Educational, Economic, Business and Industrial Engineering, 10(7), 2075-2080.

Caporarello, L., & Sarchioni, G. (2014). E-learning: The recipe for success. Journal of e-Learning and Knowledge Society, 10(1)

Dorobat, I. (2014). Models for measuring E-learning success in universities: A literature review. Informatica Economica, 18(3), 77.

Frimpon, M. F. (2012). A re-structuring of the critical success factors for e-learning deployment. American International Journal of Contemporary Research, 2(3), 115-127.

Lin, C., Ma, Z., & Lin, R. C. (2011). Re-examining the critical success factors of e-learning from the EU perspective. International Journal of Management in Education, 5(1), 44-62. doi:10.1504/IJMIE.2011.037754

Masrom, M., Zainon, O., & Rahiman, R. (2008). Critical success in e-learning: An examination of technological and institutional support factors. International Journal of Cyber Society and Education, 1(2), 131-142.

Mosa, A. A., Naz’ri bin Mahrin, M., & Ibrrahim, R. (2016). Technological aspects of E-learning readiness in higher education: A review of the literature. Computer and Information Science, 9(1), 113.

Musa, M. A., & Othman, M. S. (2012). Critical success factor in e-learning: An examination of technology and student factors. International Journal of Advances in Engineering & Technology, 3(2), 140-148.

Orȋndaru, A., Bumbac, R., & Pamfilie, R. (2013). Innovative e-learning benefits for universities and companies. Conference Proceedings of eLearning and Software for Education, (02) 398-405.

Parsazadeh, N., Zainuddin, N. M. M., Ali, R., & Hematian, A. (2013). A review on the success facors of e-learning. The Second International Conference on e-Technologies and Networks for Development (ICeND2013), Malaysia. 42-49. doi:Retrieved from http://sdiwc.net/digital-library/a-review-on-the-success-factors-of-elearning.html

Puri, G. (2012). Critical success factors in e-Learning–An empirical study. International Journal of Multidisciplinary Research, 2(1), 149-161.

Sela, E., & Sivan, Y. Y. (2009). Enterprise e-learning success factors: An analysis of practitioners’ perspective (with a downturn addendum). Interdisciplinary Journal of E-Learning and Learning Objects, 5(1), 335-343.

Sridharan, B., Deng, H., & Corbitt, B. (2008). Evaluating intertwined critical success factors for sustainable e-learning. ACIS 2008 Proceedings, 102.

Wong, W., & Huang, N. N. (2015). The effects of e-learning system service quality and users’ acceptance on organizational learning. International Journal of Business and Information, 6(2)