University of Maryland University College: UMUC Digital Repository
Not a member yet
6660 research outputs found
Sort by
Investigation of the value to students of improving their knowledge of quantitative methods in class performance
No full textThe University of Maryland University College (UMUC) student body is growing dramatically. As an open access university, UMUC accepts all qualified students. Thus these students have a broad range of skills in terms of knowledge, experience, and practice. A small percentage of our students need additional help to enhance their ability to effectively use quantitative methods associated with finance, accounting, and statistical analysis. In the spring 2012 semester, we provided information, reference sources, and tools related to these methods to students in three existing classes. The classes were in Project Management (PMAN) and Technology Management (TMAN). The purpose of the study was to quantitatively test the associated value and benefits of the interventions.1
INVESTIGATION OF THE VALUE TO STUDENTS OF IMPROVING THEIR KNOWLEDGE OF QUANTITATIVE METHODS IN CLASS PERFORMANCE
Funded by Faculty Research Grant Program - SPRING 2012
GOAL
The University of Maryland University College (UMUC) student body is growing dramatically. As an open access university, UMUC accepts all qualified students. Thus these students have a broad range of skills in terms of knowledge, experience, and practice. A small percentage of our students need additional help to enhance their ability to effectively use quantitative methods associated with finance, accounting, and statistical analysis. In the spring 2012 semester, we provided information, reference sources, and tools related to these methods to students in three existing classes. The classes were in Project Management (PMAN) and Technology Management (TMAN). The purpose of the study was to quantitatively test the associated value and benefits of the interventions.
Specific Objectives
Perform a literature review to document a summary of current knowledge.
Identify resources, tools, and material to acquaint students with the knowledge and quantitative skills needed for successful completion of PMAN 639, PMAN 650, and TMAN 625.
Post the identified material in each course and have the faculty of each section direct the students to make maximum use of the material.
During week 1, survey the students to baseline their understanding of the required skills and to assess their perception of their own skills.
At the end of the semester survey the students to collect information on their use of the material and the perceived benefits derived. In addition, solicit additional material and resources that would be useful in future classes.
Background
In the Graduate School of the University of Maryland University College, we have wonderful students interested in learning, committed to completing their degree, and willing to work hard individually and cooperate in teams. They are highly motivated to learn from the faculty and class colleagues. In spite of that, when it comes to the use of quantitative techniques, a few students are challenged by the syllabus contents, the assignments, or the degree of required knowledge and their state of preparedness. The fact that our classes are predominately taught on-line add to the complexity of addressing the issue. Although the university could provide tutors in these classes, this is a large and expensive undertaking. We sought a potential solution that was economical, easy to implement on a large scale, and contributed major impacts or benefits to the students.
In a 2008 study using data that was self-reported by students entering into their first year of college, 15 to 22% required remedial math instruction (Planty et al. 2008). Another study during the same time frame stated that 65% of new students took a remedial math course and approximately one half passed the course (Fike and Fike, 2008). Currently at our institution, we are using anecdotal information and heuristics to estimate the number of students who require remedial work in quantitative concepts.
To successfully complete the Technology Management program and/or the Project Management specialization requires a certain level of skill in quantitative concepts and methods. It is speculated that many 2
current students do not possess the necessary quantitative skills and therefore may be in need of remedial support. Studies have concluded that in addition to manipulation and interpretive skills, competence in the demonstration of quantitative skills also require the student having the confidence to exercise their knowledge (Taylor and Galligan, 2006). Taylor and Galligan also make reference to a number of possible solutions. One of the solutions involves demonstrating more than one way to solve quantitative problems. Another involves providing the student with tools to perform a self- assessment of their own improvement.
In a study conducted at the University of North Carolina, Ashville, the researcher postulated that math skills are more than an understanding of the concepts (Bahls, 2009). Although this study was about the pursuit of a math-based degree and not specifically about the demonstration of mathematical skills, the conclusions reached are similar to the previously referenced bodies of work. This study concluded that student confidence mattered more than mathematical skill. In addition to self-assessments, another important contributor to student confidence is instructor interaction (Bjorklund, Parente, and Sathianathan, 2004). Using data from more than 1,500 students, the researchers showed a positive correlation between faculty interaction and feedback to the students’ perceptions of improvement in their own skills.
An important aspect to note is that the interventions utilized should be focused on building the student’s confidence in their ability to demonstrate quantitative skills. (Pajares and Miller, 1995) stated, “Students' confidence to solve mathematics problems was a more powerful predictor of their ability to solve those problems than was their confidence to perform math-related tasks or their confidence to earn As or Bs in math-related courses.”
Results
We invited all students enrolled in the three selected courses to participate in the questionnaire. This included 21 sections of courses with a total of 465 students. The result for week 1 was an average participation response rate of 58%. At week 12 the enrollment was somewhat decreased with 434 present and 192 responding for a response rate of 45%. This decrease in response rate from week 1 to week 12 was expected since students’ enthusiasm and energy level tends to decline from the first day of class compared to the final week. Selected questions with finite answers and associated responses from week 1 and week 12 are presented below. Open ended questions were also asked and are incorporated within the conclusions section. The finite response data are summarized in a table by each course, by all finance and accounting courses, and then by all courses. All results were then analyzed and interpreted. The results for the week 1 and the week 12 number and percentage of survey responses are summarized in Table 1.
Table 1.
Summary of Respondents Courses/subject matter Week1 Week1 Week 12 Week 12 Enrollment Respondent Enrollment Respondents Number % Number % TMAN 625 277 151 55 258 115 45 PMAN 650 78 46 59 73 34 47 PMAN 639 110 68 62 101 43 43 Finance/accounting 355 197 56 331 149 45 All 465 268 58 434 192 45
3
Week 1 Survey
Table 2.
Question 1: What level of quantitative skills do you think are a necessary requirement for an adequate graduate education?
Question 2: Currently, at what level would you rate your own quantitative skills? Excellent Good Fair Required Own Required Own Required Own PMAN 639 43% 7% 50% 56% 7% 37% PMAN 650 39% 26% 54% 61% 7% 13% TMAN 625 48% 20% 49% 63% 5% 17% All Finance/Accounting 46% 23% 49% 68% 5% 17% All 45% 18% 49% 61% 6% 22%
From the response to question 1 in Table 2, clearly a majority of the students felt that they were well or very well prepared for the class in question. According to the responses to the responses to question 2 of Table 2, on average, students believed that their quantitative skills are good. Students taking the finance and accounting courses (TMAN 625 and PMAN 650) felt that their quantitative skills were better than those taking a course that required the use of standard statistical tests (PMAN 639). Note that TMAN 625 is part of the core degree program and is taken among the first courses. PMAN 650 is part of the specialization and is taken later in the students’ academic career. For most students it is the second course in finance and accounting. This may explain why students rated their quantitative skills higher in PMAN 650 than in TMAN 625.
It is interesting to note that more than 59% had already taken a course in accounting, finance or statistics mostly in undergraduate school. 21% of the students had taken such a course in the graduate school. That left 18% with no preparation in finance, accounting or statistics. One question addressed the knowledge the students acquired and the grade they secured in taking a prior course in accounting, finance or statistics. 84% of the students who took such a course reported excellent or passing grades.
After reviewing the syllabus, 95% of the students felt they would pass the course based upon their current knowledge. This level of confidence may be influenced by the fact that a majority of the students had a prior knowledge of statistics, accounting or finance. What they needed was a refresher or some form of specific assistance. The university offers short preparatory courses in accounting, finance, and statistics. As at other universities, students at UMUC are focused on completing their degree in the shortest possible time and will not take a detour to take a non-required course. This attitude may be more prevalent in UMUC students because on average, many are enrolled in a master’s degree program after an approximately ten-year gap in their formal education. The focus on earliest completion is also true when the reason for seeking a degree is associated with the desire for advancement, career change, or for starting a business. Although the responses indicate that on average, 95% of the students had not taken a preparation course, responses to another question indicates that 60% would not consider doing so.
Students indicated having access to quantitative resources and support outside the course. We did not attempt to identify such resources but we postulate that they include family members, relatives, other students 4
in the program, and access to resources at their work location. It is clear that a majority of the students would like a tutor available in the class. Discussing problems, issues and approach with a knowledgeable person is clearly a preferred approach. Unfortunately the cost of such a solution is probably prohibitive.
Week 12 Survey
At the completion of the course, the students concluded that they did very well (35%) or reasonably well (57%). It is interesting, but not easily explained why the PMAN 639 felt less confident in their performance that the other students. As mentioned earlier, PMAN 650 students should feel more confident since this is their second course in accounting and finance. However, a lower percentage of the PMAN 639 students indicated a detailed knowledge of the concepts than students in the other courses.
Of all the resources made available to the students, the generic information (examples, definitions, cases) was the most utilized followed by tutorials and software packages. Students reported a dramatic improvement in their confidence and ability to perform the quantitative assignments. It should be noted that improvement in confidence is probably as valuable as improvements in skills and performance. The students that took advantage of the posted material reported that they were useful. This result encourages us to look for more varied, extensive, and easier to use resources for the future.
Conclusions
From the students’ perspective, the identified interventions provided benefit and value to those who took advantage of the resources. Given this benefit, students wanted more examples from the professor and from the textbooks. They also thought that an in class mentor would be more beneficial. The conclusion derived from this information and other open ended comments is that even when the student feels confident in their ability to perform the calculations, they often need help in determining when to use specific formulas. Although sometimes, the math itself was an issue, knowing when to use the formulas and which formula to use for solving different problems was of greater concern. Tutorials supplemented by videos showing step by step usage of the formulas may be of great benefit and should be investigated as an investment by the university. The videos should not be stand-alone but put into context via incorporation into real life scenarios.
Future
Follow-up to this research can be to correlate the student results to actual performance outcomes or grades. This would provide an indication of what effect the expression of student confidence has on performance. In the absence of this correlation, it was concluded that given the low cost and scalability, this solution was practical and effective based upon student confidence and their self- assessments.
5
References
Bahls, P., (2009). Math and Metaphor: Using Poetry to Teach College Mathematics. The WAC Journal. Pages 75-90
Bjorklund, S., Parente, J., Sathianathan, D.,(2004) Effects of Faculty Interaction and Feedback on Gains in Student Skills, Journal of Engineering Education. April. Pages, 153-160
Fike, D.S., and R. Fike. (2008). Predictors of first-year student retention in the community college. CommunityCollege Review 36 (2): 68–88.
Taylor, J., Galligan, L., (2006). Mathematics for Maths Anxious Tertiary Students: Integrating the cognitive and affective domains using interactive multimedia. Literacy & Numeracy Studies. Vol 15. No 1. Pages 23-43.
Pajares, F., Miller., M.D. (1995). Mathematics Self-Efficacy and Mathematics Performances: The Need for Specificity of Assessment. Journal of Counseling Psychology, Vol. 42, No. 2, 190-198
Planty, M., Hussar W., Snyder, T., Provasnik, S., Kena , G., Dinkes, R., Kewal Ramani, A., and Kemp, J. (2008). The Condition of Education 2008 (NCES 2008-031). Washington, DC: U.S. Department of Education, National Center for Education Statistics
Development of a reliable, valid, multi-dimensional measure of student engagement in group projects
No full textThe Study Purpose: Assess SGPEQ reliability and internal consistency; establish criterion-related and construct validity, for measure to be usable with confidence for quantitative analysis.
Data: All students form 25 ITEC 620 sections and 15 TLMN sections from two semesters in 2012 were invited to participate. Data was gathered from 260 students, from which four declined to fill in the survey and 24 did not answer the questions.
Method:
• Study on SGPEQ initial data reliability and validity.
• Initial item reduction and factors identification through exploratory factor analysis
• Further verification of validity through a study on relationships between factors and students’ self-reported engagement and endorsement of self-theories.Development of a Reliable, Valid, Multi-Dimensional Measure of Student Engagement in Group Projects
Irena Bojanova
Program Director Telecommunications Management, Graduate School, UMUC
[email protected], 240-684-2428
Introduction
In Summer 2011, student experiences in immersive vs. traditional group projects were examined and “student engagement in group projects” was identified as a distinguishable construct; research on the identified construct and the available measurement instruments were reviewed; and a preliminary scale for a new instrument to measure students engagement in group projects – Student Group Project Engagement Questionnaire (SGPEQ) – was developed, and face and content validated.
The Spring 2012 project focused on the theoretical development of SGPEQ as a reliable, valid, multi-dimensional measure of student engagement in group projects. A study on the SGPEQ initial data reliability and validity was conducted in accordance with established psychometric principles for use in survey research. Initial item reduction and factor identification on SGPEQ was performed through exploratory factor analysis and examination of reliability estimates. The validity of the measure was further verified through a study on relationships between the SGPEQ factors and students’ self-reported engagement, endorsement of self-theories, goal orientation and self-reported grades.
The Study
Purpose: Assess SGPEQ reliability and internal consistency; establish criterion-related and construct validity, for measure to be usable with confidence for quantitative analysis.
Data: All students form 25 ITEC 620 sections and 15 TLMN sections from two semesters in 2012 were invited to participate. Data was gathered from 260 students, from which four declined to fill in the survey and 24 did not answer the questions – please refer Appendix A.
Method:
• Study on SGPEQ initial data reliability and validity.
• Initial item reduction and factors identification through exploratory factor analysis
• Further verification of validity through a study on relationships between factors and students’ self-reported engagement and endorsement of self-theories.
Initial Data Reliability and Internal Consistency
Reliability refers to the ability of the measurement instrument to give similar results for similar inputs. Reliability analysis was conducted based on the following models: Cronbach’s Alpha (model of internal consistency, based on the average inter-item correlation) and Split-Half (model splits the scale into two parts and examines the correlation between the parts). The Cronbach's alpha (0.940) and Split Half (0.894 vs 0. .884) values for the analyzed survey instrument confirms high level of reliability – please refer Appendix C. The Inter-Item correlations (783 of the 1225 are larger than 0.30) also confirm suitability of the data for factor analysis – please refer Appendix D.
Explanatory Factor Analysis and Reliability Estimates
Item reduction and factors identification: Principal axis factoring with Varimax rotation was performed on the 35 student engagement in group projects items. The questions were: "How many components (factors) are needed to represent the variables?" and "What do these components represent?" The analysis of a six-factor (please refer Appendix E) and a four-factor (please refer Appendix F) solution and the inspection of a scree plot (please refer Appendix G), four factors were retained. Interpretability was difficult after four factors and the scree plot confirmed that the slope decreases a little after four factors. The four factors accounted for 59.15% of the variance. Appendix I shows the factor solution and the items (questions) for each factor. Only one question, “Stepped in when a teammate was not performing”, was dropped due to low communality and low loading on all factors. The four factors counted for 53.40% of the variance; each factor comprised 19.69%, 14.04%, 11.69%, and 8.23% of the variance correspondingly. They were titled: Effort, Teamwork, Motivation, and Organization.
Factors reliability and preliminary evidence of discriminant validity: The four student engagement in group projects factors show reasonable reliability (.926, .838, .889, .766 correspondingly); and the inter-item correlations (the highest is .088) support the discriminant validity of the SGPEQ measure.
Further Verification of Validity
SGPEQ validity was further verified through a study on the relationship between the identified factors and the students’ self-reported engagement and endorsement of self-theories – please refer Appendix B and Appendix K.
• Self-reported engagement: Factors predicting absolute engagement (engagement in this group project) and relative engagement (engagement compared with other group projects) were determined. For that two analyses that regressed each of the two dependent variables on the four factors of SGPEQ were performed.
Absolute engagement (“How engaged were you in this group project How engaged were you in this group project How engaged were you in this group projectHow engaged were you in this group projectHow engaged were you in this group projectHow engaged were you in this group project How engaged were you in this group project How engaged were you in this group project How engaged were you in this group projectHow engaged were you in this group projectHow engaged were you in this group project How engaged were you in this group project How engaged were you in this group project How engaged were you in this group project How engaged were you in this group project How engaged were you in this group projectHow engaged were you in this group project?”): Regression of relative engagement on the four factors shows that the factors account for 63.1% (R Square from Model Summary table) of the variance in absolute engagement; 70% of absolute engagement was explained by the model: F(4, 166)=70.81, p < .001 (F and Sig. from ANOVA table). Factor 1 (β=.611); Factor 2 (β=.174), Factor 3 (β=.325), Factor 4 (β=.236) are all positive predictors of absolute engagement.
Relative engagement (“How engaged were you in this group project compared to other How engaged were you in this group project compared to other How engaged were you in this group project compared to other How engaged were you in this group project compared to other How engaged were you in this group project compared to other How engaged were you in this group project compared to other How engaged were you in this group project compared to other How engaged were you in this group project compared to other How engaged were you in this group project compared to other How engaged were you in this group project compared to other How engaged were you in this group project compared to other How engaged were you in this group project compared to other How engaged were you in this group project compared to other How engaged were you in this group project compared to other How engaged were you in this group project compared to other How engaged were you in this group project compared to other How engaged were you in this group project compared to other How engaged were you in this group project compared to other How engaged were you in this group project compared to other How engaged were you in this group project compared to other How engaged were you in this group project compared to other How engaged were you in this group project compared to other How engaged were you in this group project compared to other How engaged were you in this group project compared to other group projects you worked on during the same semester?group projects you worked on during the same semester? group projects you worked on during the same semester?group projects you worked on during the same semester? group projects you worked on during the same semester?group projects you worked on during the same semester? group projects you worked on during the same semester?group projects you worked on during the same semester? group projects you worked on during the same semester?group projects you worked on during the same semester?group projects you worked on during the same semester? group projects you worked on during the same semester? group projects you worked on during the same semester?group projects you worked on during the same semester? group projects you worked on during the same semester? group projects you worked on during the same semester? group projects you worked on during the same semester? ”): Regression of relative engagement on the four factors shows that the factors account for 44% of the variance in absolute engagement; 33% of relative engagement was explained by the model: F(4, 166)=32.54, p < .05. Factor 1 (β=.387); Factor 2 (β=.259), Factor 3 (β=.326), Factor 4 (β=.233) are all positive predictors of absolute engagement.
• Incremental and entity self-theories: Regression of belief in incremental theory (“You You have a certain amount of intelligence and you cannot do much to change it. have a certain amount of intelligence and you cannot do much to change it. have a certain amount of intelligence and you cannot do much to change it. have a certain amount of intelligence and you cannot do much to change it.have a certain amount of intelligence and you cannot do much to change it.have a certain amount of intelligence and you cannot do much to change it.have a certain amount of intelligence and you cannot do much to change it. have a certain amount of intelligence and you cannot do much to change it. have a certain amount of intelligence and you cannot do much to change it.have a certain amount of intelligence and you cannot do much to change it. have a certain amount of intelligence and you cannot do much to change it. have a certain amount of intelligence and you cannot do much to change it. have a certain amount of intelligence and you cannot do much to change it.have a certain amount of intelligence and you cannot do much to change it. have a certain amount of intelligence and you cannot do much to change it. have a certain amount of intelligence and you cannot do much to change it.have a certain amount of intelligence and you cannot do much to change it. have a certain amount of intelligence and you cannot do much to change it.have a certain amount of intelligence and you cannot do much to change it.have a certain amount of intelligence and you cannot do much to change it.have a certain amount of intelligence and you cannot do much to change it. have a certain amount of intelligence and you cannot do much to change it.have a certain amount of intelligence and you cannot do much to change it. have a certain amount of intelligence and you cannot do much to change it.have a certain amount of intelligence and you cannot do much to change it.have a certain amount of intelligence and you cannot do much to change it. have a certain amount of intelligence and you cannot do much to change it. have a certain amount of intelligence and you cannot do much to change it. have a certain amount of intelligence and you cannot do much to change it.have a certain amount of intelligence and you cannot do much to change it.have a certain amount of intelligence and you cannot do much to change it.have a certain amount of intelligence and you cannot do much to change it. have a certain amount of intelligence and you cannot do much to change it.”) on the four factors shows that the factors account for 59% of the variance in absolute engagement; 3% of belief in incremental theory was explained by the model: F(4,166)=2.6, p < .001. Factor 1 (β=.024); Factor 2 (β=.141), and Factor 4 (β=.171) are positive predictors of belief in incremental theory.
Conclusion
The conducted in Spring and Summer 2012 research resulted in the creation of a reliable, valid, multi-dimensional measure of student engagement in group projects. Empirical evidence of the usefulness of SGPEQ was provided, so it can be easily administrated while providing a comprehensive snapshot of students’ engagement in group projects. References
[1] AERA, APA, NCME (1999). Standards for Educational and Psychological Testing, http://teststandards.org
[3] Carini, R., Kuh, G., Klein, S. (2006). Student Engagement and Student Learning: Testing the Linkages, Research in Higher Education, Vol. 47, No. 1,
http://gov.alaska.edu/faculty/StudentSuccess/TintoReview-Carini-Kuh-Klein.pdf
[4] Clark, L. & Watson, D. (1995). Constructing validity: Basic issues in objective scale development. Psychological Assessment, 7, 309-319.
[5] Community College Survey of Student Engagement. University of Texas at Austin, www.ccsse.org.
[6] Cooperative Institutional Research Program (CIRP) Surveys. Higher Education Research Institute (HERI), http://heri.ucla.edu/herisurveys.php
[7] DeVellis, R. (2003). Scale development: Theory and applications. Thousand Oaks, CA: Sage Publications.
[8] Dweck, C. (1999). Self-Theories: Their role in motivation, personality, and development. Philadelphia: The Psychology Press.
[9] Handelsman, M, Briggs, W., Sullivan, N., & Towler, A. (2005). A Measure of College Student Engagement. Journal of Educational Research, Vol. 98, No. 3, pp. 184-191; http://www.stanford.edu/dept/SUSE/projects/ireport/articles/self-regulation/self-course%20engement%20measure.pdf
[10] Hinkin, T. (1998). A brief tutorial on the development of measures for use in survey questionnaires. Organizational Research Methods, 1, 104–121.
[12] Kerlinger, 1F.N., (1986). Foundations of Behavioral Research. Holt, Rinehart and Winston.
[13] Likert, R. (1931). A technique for measurement of attitudes. Archives of Psycology. New York: Columbia University Press.
[14] Litwin, M. S. (2003). How to assess and interpret survey psychometrics, 2nd edition. Thousand Oaks, CA: Sage Publications.
[15] McMillan, J. & Schumacher, S. (2001). Research in education: A conceptual introduction. New York: Longman.
[16] Molinari, J. & Huonker, J. (2010). Diagnosing student engagement in the business school classroom. Journal of the Academy of Business Education.
[17] National Survey of Student Engagement (NSSE). Center for Postsecondary Research, Indiana University in Bloomington, http://nsse.iub.edu/
[18] O'Donnell, A., Reeve, J., and Smith, J. (2009) Educational Psychology: Reflection for Action, Chapter 11. Wiley.
[19] Pike, G., Kuh, G. (2005) A Typology of Student Engagement for American Colleges and Universities, Research in Higher Education, Vol. 46, No. 2,,
http://cpr.iub.edu/uploads/Pike,%20Kuh%20(2005)%20A%20Typology%20of%20Student%20Engagement%20for%20American%20Colleges%20and%20Universities.pdf
[20] Robinson C. &Hullinger, H. (2008). New Benchmarks in Higher Education: Student Engagement in Online Learning. Journal of Education for Business.
http://cyber.law.harvard.edu/communia2010/sites/communia2010/images/Robinson_et_al_2008_New_Benchmarks_in_Higher_Education_Student_Engagement_in_Online_
[21] Schoenfeldt, L. F. (1984). Psychometric properties of organizational research instruments. In T. S. Bateman & G. R. Ferris (Eds.), Method & analysis in organizational research (pp.68-80). Reston, VA: Reston.
[22] Skinner, E.A., and Belmont, M.J. (1993). .Motivation in the classroom: Reciprocal effects of teacher behavior and student engagement across the school year. Journal of Educational Psychology, 85(4). p. 572.
Appendix A: Preliminary SGPEQ Scale
To what extent do the following behaviors, thought, and feelings described you in this group project? Please rate each of them on the following scale: 1 = Very little; 2 = Some; 3 = Quite a bit’; 4 = Very much
Worked on the project on a regular basis
Put forth effort
Preferred to work on my own
Took detailed notes during discussion meetings
Wished my teammates were working harder than me
Completed all assigned tasks on time
Rehearsed for project presentation
Was motivated and enthusiastic
Found project activities relevant to my life
Thought about project activities between meetings
Found ways to make project interesting to me
Was inspired to learn and contribute
Felt presence of team members during meetings/ presentations (as if in person)
Found project academically challenging
Would like to have similar projects in other classes
Fulfilled the assigned role
Contributed to discussions with ideas and opinions
Got to know teammates' strengths
Had fun during team activities
Incorporated teammates ideas and opinions
Helped/ tutored teammates during project activities
Preferred team-work than working on my own
Experienced sense of discovery and accomplishment
Trusted teammates will do well on their project parts
Found ways to involve non-participating team members
Stepped in when a teammate was not performing (+ N/A option if everybody participated)
Was organized and prepared
Communicated clearly and effectively
Attended all group meetings
Applied critical thinking and problem solving
Did good work on my part
Was creative and productive
Developed leadership skills
Presented the final research product clearly and effectively
Was confident that we can learn and do well on the project
Appendix B: Further validity of the developed measure
Global Engagement
How engaged were you in this group project? (1=not at all, 6=extremely)
How engaged were you in this group project compared to other group projects you worked on during the same semester? (1=less engaged, 6=more engaged)
Incremental theory
You have a certain amount of intelligence and you cannot do much to change it. (1=do not agree, 6=strongly agree
Appendix C: SGPEQ Reliability
Ideally, in order to obtain a good estimate of the reliability of a survey, we would like to administer the survey twice to the same group of people and then correlate the two sets of results. However, this is often impractical because bias may be introduced in the second set of answers or because respondents may be unwilling or unable to take the survey a second time. One solution is to compute Cronbach's alpha. Another is to split the items into two groups and then to compare these groups as if they were two separate administrations of the same survey.
1) Cronbach's Alpha
Case Processing Summary N % Cases Valid 185 71.2 Excludeda 75 28.8 Total 260 100.0 a. Listwise deletion based on all variables in the procedure.
Reliability Statistics Cronbach's Alpha Cronbach's Alpha Based on Standardized Items N of Items .941 .945 35
Item Statistics Mean Std. Deviation N Worked on the project on a regular basis 2.9351 .83803 185 Put forth effort 3.3514 .70001 185 Preferred to work on my own 2.6973 .96978 185 Took detailed notes during discussion meetings 2.3189 1.01131 185 Wished my teammates were working harder than me 1.8703 .99697 185 Completed all assigned tasks on time 3.4811 .70781 185 Rehearsed for project presentation 2.1568 1.08461 185 Was motivated and enthusiastic 2.9405 .92180 185 Found project activities relevant to my life 2.6108 1.08342 185 Thought about project activities between meetings 2.7676 .94122 185 Found ways to make project interesting to me 2.7676 .89990 185 Was inspired to learn and contribute 3.0054 .92371 185 Felt presence of team members during meetings/ presentations (as if in person) 2.3892 1.01612 185 Found project academically challenging 2.7568 .97817 185 Would like to have similar projects in other classes 2.3730 1.12110 185 Fulfilled the assigned role 3.4811 .62634 185 Contributed to discussions with ideas and opinions 3.3946 .73793 185 Got to know teammates' strengths 2.7135 1.00492 185 Had fun during team activities 2.3027 1.09109 185 Incorporated teammates ideas and opinions 3.1405 .76024 185 Helped/ tutored teammates during project activities 2.2432 1.01632 185 Preferred team-work than working on my own 2.0649 1.01944 185 Experienced sense of discovery and accomplishment 2.6162 .99932 185 Trusted teammates will do well on their project parts 2.8216 1.00843 185 Found ways to involve non-participating team members 1.9946 1.01348 185 Stepped in when a teammate was not performing 3.4324 1.37407 185 Was organized and prepared 3.2108 .72524 185 Communicated clearly and effectively 3.3297 .67117 185 Attended all group meetings 3.2649 .82759 185 Applied critical thinking and problem solving 3.2486 .73189 185 Did good work on my part 3.4595 .61656 185 Was creative and productive 3.3081 .69719 185 Developed leadership skills 2.7351 1.06327 185 Presented the final research product clearly and effectively 3.2432 .77338 185 Was confident that we can learn and do well on the project 3.1946 .89380 185
2) Split Half
Note: Different splits of the items will produce different estimates of the reliability coefficient. When they are split so that each group contains items that are highly correlated within the group but not between groups (as has been done here), the split-half coefficients will be close to their lowest values. When highly correlated items are paired off and placed into separate groups, then the split-half coefficients will reach their highest values.
The following table shows the Split Half-Coefficient:
• Correlation between forms - the correlation between the sum
Investigating of the value to students of improving their knowledge of quantitative methods in class performance
No full textA small percentage of our students need additional help to enhance their ability to effectively use quantitative methods associated with finance, accounting, and statistical analysis
Pedagogical Purposes
•Provide access to additional information, reference resources, and tools
•Quantify the associated value and benefits of the interventions
Results
•Solution was
–Economical
–Scalable
–EffectiveRobert Ouellette and Rudy Watson University of Maryland University College
Abstract
A small percentage of our students need additional help to enhance their ability to effectively use quantitative methods associated with finance, accounting, and statistical analysis
Pedagogical Purposes
•Provide access to additional information, reference resources, and tools
•Quantify the associated value and benefits of the interventions
Results
•Solution was
–Economical
–Scalable
–Effective
Introduction
•U.S. Department of Education reports that 15 to 22% of first year college students need math remediation
•Anecdotal information and heuristics says this applies to UMUC
•Self recognition of limitations is important to participatory learning
•Self recognition of skills leads to a higher level of student confidence
•Student confidence matters more to performance than a skill set deficiency
Hypothesis
Facilitating students in assessing the level of their own quantitative skills and providing resources to help close the gaps will improve student confidence
Purpose
Create benchmark of students’ understanding of the need for quantitative skills required to be successful
Document students’ opinions of their own skill set and readiness to perform in course requiring quantitative skills
Assess the effectiveness of supplied interventions
Methodology
Literature review
Identify sources, tools, and materials
Post to classrooms and faculty encourage use
Pre and post semester surveys
Context
•PMAN 639, PMAN 650, TMAN 625
•21 sections
•465 students
Data Gathering
•Student Surveys
•Week 1
•Week 12
Survey Results
Robert Ouellette [email protected] Rudy Watson [email protected]
Course Descriptions
Key Findings
18% of the students had no preparation in finance, accounting or statistics
95% of the students felt they would pass the course based upon their current knowledge
95% of the students indicated that they had not taken a prep course
60% of the students would not consider taking a prep course
Students reported a dramatic improvement in their confidence and ability to perform the quantitative assignments. It should be noted that improvement in confidence is probably as valuable as improvements in skills and performance. The students that took advantage of the posted material reported that they were useful.
PMAN 639: A study of the policy, processes, and procedures involved in assuring that projects will satisfy the objectives for which they were undertaken. Emphasis is on quality planning, quality assurance, quality control, and process improvement. Discussion covers all the activities that determine quality objectives, policies, and responsibilities. The importance of customer satisfaction, prevention over inspection, management responsibility, and continuous improvement is recognized. Topics include control charts, cause and effect diagrams, Pareto charts, failure mode and effect analysis, design reviews, and cost of quality. Course content and approach are compatible with the International Organization for Standardization.
PMAN 650: An investigation of financial and strategic decision making in the management of projects. Topics include estimating project costs from work breakdown structure; formulating, monitoring, and controlling project budgets; monitoring, evaluating, and forecasting project costs, schedule, results, and performance using earned value management; and deriving project cash flows. Discussion also covers the impact of project scope, schedule, and changes; management reserves to cover risks and contingencies; top-down and bottom-up budgeting; investment project analysis; discounted cash flow, internal rate of return, and net present value methodologies; cost of capital; and capital budgeting. Broader issues (such as links between project and corporate financial performance, business ethics, corporate social responsibility, project and organizational culture, information flow, and project sustainability) are also examined.
TMAN 625: A study of the financial tools that managers use to find answers to four important questions: What is the financial condition of the firm? What long-term investment should the firm make? How can the money be raised for the investments? And how will the firm meet its daily financial requirements? Topics include accounting statements, tax implications, types of costs, profit recognition, financial markets, investment decision tools, net present value, free cash flows, project financing, valuation of firms, risk-return, cost of capital, long-term financing, short-term financing, and equity financing for entrepreneurs. Discussion also covers mergers and acquisition activities, governance and ethics, and international aspects. Business cases from contemporary firms and readings relevant to technology management are used to illustrate the application of financial concepts.
The identified interventions provided benefit and value to students who took advantage of the resources
Given this benefit, students wanted more examples from the professor and from the textbooks
Students needed help in determining when to use specific formulas for solving different problems
Videos should not be stand-alone but put into context via incorporation into real life scenarios
Conclusions
Quantitative Techniques Assistance
Investigation of the Value to Students of Improving their Knowledge of Quantitative Methods in Class Performance
Contact Info
Tutorial Support
Free Accounting Software
Accounting and Finance for Dummies
Tutorial Quiz
Free Accounting/Finance Information
Courses/subject matter
Week1
Week1
Week 12
Week 12
Enrollment
Respondent
Enrollment
Respondents
Number
%
Number
%
TMAN 625
277
151
55
258
115
45
PMAN 650
78
46
59
73
34
47
PMAN 639
110
68
62
101
43
43
Finance/accounting
355
197
56
331
149
45
All
465
268
58
434
192
45
Summary of Respondents
Excellent
Good
Fair
Required
Own
Required
Own
Required
Own
PMAN 639
43%
7%
50%
56%
7%
37%
PMAN 650
39%
26%
54%
61%
7%
13%
TMAN 625
48%
20%
49%
63%
5%
17%
All Finance/Accounting
46%
23%
49%
68%
5%
17%
All
45%
18%
49%
61%
6%
22%
Rate Your Own Skills
Yes- in undergrad school
Yes- in grad school
Never
PMAN 639
54%
41%
4%
PMAN 650
35%
54%
11%
TMAN 625
61%
63%
17%
All Finance/Accounting
56%
21%
22%
All
55%
21%
18%
Taken Course in Accounting, Finance, or Statistics
UCSP 630 statistics only
UCSP 621 Economics only
UCSP 620 Financial accounting
Both statistics and finance/economic courses
Never taken one of these prep courses
PMAN 639
0%
0%
1%
4%
94%
PMAN 650
2%
0%
4%
4%
89%
TMAN 625
0%
0%
0%
2%
98%
All Finance/Accounting
0%
0%
2%
3%
96%
All
0%
0%
2%
3%
95%
Taken Prep Course
Yes
No
I took a prep course
PMAN 639
49%
48%
3%
PMAN 650
26%
61%
13%
TMAN 625
33%
65%
2%
All Finance/Accounting
31%
64%
5%
All
36%
60%
4%
Consider a Prep Course
Not at all
Moderate knowledge of concepts
Detailed knowledge of concepts and procedures
PMAN 639
19%
74%
7%
PMAN 650
21%
76%
13%
TMAN 625
69%
73%
8%
All Finance/Accounting
67%
74%
9%
All
68%
74%
8%
Familiar with Concepts, Terminology, & Computations
Generic
Tutorial
Software
None
PMAN 639
50%
27%
2%
20%
PMAN 650
47%
12%
6%
35%
TMAN 625
52%
34%
2%
13%
All Finance/Accounting
50%
29%
2%
18%
All
50%
29%
3%
18%
Week 12: Resources Utilized
Improve Confidence
Helpful in Assignments
Yes
No
Did Not
Yes
Yes, but not worth the additional work
No
Did not
PMAN 639
61%
20%
18%
55%
11%
14%
20%
PMAN 650
53%
18%
29%
48%
12%
6%
33%
TMAN 625
80%
12%
8%
78%
9%
7%
6%
Finance / Accounting
74%
14%
13%
71%
10%
7%
12%
All
71%
15%
14%
68%
10%
8%
14%
Week 12: Student Perception of Effectivenes
Estimating framework for technology-driven projects: A parametric estimating technique for the concept phase
No full textThis paper documents the production of a conceptual estimating framework which combines the knowledge of technology with that of business process and organization in an effort to improve productivity. The research is focused solely on deployment, an area that in the past has primarily been combined with software application development. This study examines parametric project cost estimating practices by investigating how parametric estimating techniques impact a technology-driven project.Estimating Framework for Technology-Driven Projects: A Parametric Estimating
Technique for the Concept Phase
Rudy J. Watson, Ph.D.
Information and Technology Systems
Graduate School
University of Maryland University College
3501 University Boulevard East
Adelphi, MD 20783
USA
[email protected]
1
Introduction
Services Science, Math and Engineering (SSME) is a discipline that focuses on
fundamental science, models, theories and applications to drive innovation, competition,
and quality of life through services (Bitner & Brown, 2006). It is a multidisciplinary
research approach, combining technology with an understanding of the underlying
business processes to improve performance in the service industry. Services science
research yield models, methodologies, processes, and software tools that create and
deliver services more efficiently.
This paper documents the production of a conceptual estimating framework which
combines the knowledge of technology with that of business process and organization in
an effort to improve productivity. The research is focused solely on deployment, an area
that in the past has primarily been combined with software application development. This
study examines parametric project cost estimating practices by investigating how
parametric estimating techniques impact a technology-driven project.
The subject projects consisted of the deployment of new or enhanced processes
and software applications across user communities within a single large international
company. Deployment activities encompass user organization interface-related items,
such as data migration, user testing, end-user education, organizational communications,
project management, and system/process move to production. The output of this
conceptual estimating framework provides estimated costs of a project represented as the
human resource hours required to execute these deployment activities that can be used to
prioritize the various deployments.
An important factor in understanding a conceptual estimate for deployment is the
time frame in which it is created. Figure 1 shows that a conceptual estimate must be
completed prior to the concept phase exit decision checkpoint (DCP) or a Stage-Gate®1
go or no-go decision (Cooper, 2000). The timing of a conceptual estimate severely limits
the amount of information available upon which to base estimates.
1Stage-Gate® is a trademark of Product Development Institute, Inc.
2
Figure 1. Conceptual estimating timeline2
A premise of this study is that the accuracy of the resulting estimate was not the
most important reason for performing the conceptual estimate. Stewart (1991), who
stated that the knowledge learned during the process of estimating often provides more
value than the estimate itself, supported this premise. Although the conceptual estimate
is primarily based upon assumptions, thinking through and documenting these
assumptions provide value to the project team responsible for the deployment. By
analyzing different scenarios and selecting among the most likely, the project manager
begins to identify potential issues.
Methodology
In the development of a conceptual estimating framework for deployment costs,
the primary research question is, “What software application deployment project
characteristics can be identified and quantified to perform conceptual estimating of
human resource hours required for deployment activities?”
A modified Delphi Technique was used to identify the project characteristics, the
cost-estimating relationships, and the values for the range of estimates used to predict the
deployment costs. The technique is considered modified; because, in some of the first
rounds, external data was utilized from other sources besides the subject-matter experts.
2 SOURCE: Adapted from Watson, R. J., & Kwak, Y. H. (2004). Development of a parametric estimating
model for technology-driven deployment projects. Proceedings of the 2004 Winter Simulation Conference.
3
The use of historical data in addition to subject-matter expert input can improve the first
round response rate and ensures a solid grounding in previously developed work (Custer
et al., 1999).
The fifty-two subject-matter experts who participated in this study are all Project
Management Institute certified Project Management Professionals. Recertification “give
back” credit was the incentive.
The research method executed the following six steps of framework development.
1. Construct Definition
2. Selection of Cost-Estimating Relationships
3. Determination of Estimates
4. Development of the Estimating Framework
5. Execution of the Framework
6. Documentation of Assessments and Findings
The first step of the research method was construct definition providing a
consistent set of terms and definitions.
The second step of the research method was the selection of cost-estimating
relationships (CERs) using a modified Delphi Technique to identify the CERs used as the
independent variables of the framework.
The third step of the research method was the determination of an established
range of estimates for each CER based upon the explicit knowledge of subject-matter
experts.
The fourth step of the research method was the development of the estimation
framework. Rules for use of the framework were established as well as guidelines for
assessment of the cost-estimating independent variables.
The fifth step of the research method involved executing the framework . Using
project definition documents from the concept phase of various projects, the framework
was exercised to produce output.
The final step of the research method was the performance and documentation of
the assessment. The output was analyzed with findings and recommendations
documented. Figure 2 depicts the development of the framework using these steps.
4
Figure 2. Conceptual estimating framework.
Findings
The findings of the study include the results of executing the framework using a
sample of four deployment projects. The projects were deployments of a centrally
developed integrated enterprise application that was deployed in various geographic
locations. The conclusion includes feedback from a project manager and a senior
manager to demonstrate that the framework is meaningful to practitioners. Their
comments serve as a qualitative validation of the framework.
There are four steps involved in the use of the framework. Step one is completed
by the estimator to assess project complexity for a specific project. The purpose is to
determine which resource template to use for the estimation in conjunction with the task
complexity assessments, which are determined in step two. Step three in the execution of
the framework involves using the complexity ratings to obtain estimates from tables. The
fourth step of the process is called validation. The purpose of validation is to quickly
5
highlight any significant variations from the expected result that may be inherent in the
project and need to be investigated further or may have resulted from manual overrides.
The four projects chosen for which to execute the framework to obtain results
represented four different geographic locations: Asia Pacific South, China, European
Union, and Latin America. With the execution of each step, the output of the framework
yields four different associated perspectives on the project. Subsets of sample reports are
depicted in the following tables.
Table 1 represents the assessment of project complexity. Each characteristic
being assessed by the user of the framework is listed in the first column. The second
through fourth columns contain a description or criteria for the characteristic that would
respectively designate the complexity rating as high, medium or low. The final column is
reserved for the complexity assessment.
Table 1
Step One: Example of Project Complexity
Project Criteria High Medium Low Complexity for Project
H-M-L
Number of
Stakeholder
Organizations
3 or More 2 1 L
Duration/Timeslot
Required for
Implementation of
Project
Eight
Months or
Greater
Five to Seven
Months
Four Months
or Less
M
Architecture/
Complexity of
Solution
Integrated
Enterprise
Application
Client/Server Stand Alone M
Overall Project
Complexity
Overall
Project
Complexity
M
Table 2 depicts the assessment of task complexity. Under the results column, an
acronym is presented corresponding to a summary rating for each group of tasks. In the
example, “PM” represents project management activities, and “DA” represents data
conversion and migration activities.
6
Table 2
Step Two: Example of Task Complexity
Task/Complexity/Criteria High Medium Low Response Results
PM—Project Management PM L
Use of Experienced Project
Manager
No - Yes L
Use of Project Office
Resources
No - Yes L
DA—Data Conversion and
Migration
DA H
Use of Centralized Skills No Partial Yes H
Size/Volume of Data to be
Converted
Much Little None H
Within the framework, there is a table of estimates corresponding to the overall
project complexity assessment. The example being shown here had a project complexity
assessment of medium in step one, therefore a subset of the medium range of estimates is
shown in Table 3. Using the assessments from step two, the corresponding hours
associated with that rating is entered into column six. These hours are totaled to become
the deployment project conceptual estimate.
The framework allows for the estimator to make adjustments to the output. In the
example, the estimator decided to decrement the suggested number of hours by 500. This
was done because the estimator knew that in a previous period of time, this organization
had engaged in major data clean-up in association with another project.
Table 3
Step Three: Example of Range of Estimates—Medium
Category Low Medium High
Step 2
Results
Hours
Adjustment
to Hours
PM—Project Management 800 1700 2600 L 800
DA—Data Conversion and
Migration
500 1500 2600 H 2600 -500
Total Hours 3400
7
Total Hours Plus Adjusted Hours 2900
The validation step is used to assess the reasonableness of the percentage that
each deployment category is of the total. In Table 4, all of the categories are not shown;
therefore, the percentages do not add up to 100% of the total project estimate. What can
be seen is the effect of adjusting data conversion and migration. This is evident when the
next to the last column of entitled “adjusted percentage of total” is compared to the
column entitled “percentage of total deployment estimated cost within a category,” the
original calculated percentage before adjustments. The project management percentage
of total cost was adjusted from 9.88% to 10.89%, and the data conversion and migration
percentage of total cost dropped from 6.17% to 5.22%.
Table 4
Step Four: Example of Validation
Deployment
Category
Estimated
Deployment
Cost in
Hours
% of Total
Deployment
Estimated
Cost
Within a
Category
Adjusted
Hours
Adjusted %
of Total
Recommended
% of Total
Deployment
Cost Within
Category
PM—Project
Management
800 9.88% 0 10.89% 8-12
DA—Data
Conversion and
Migration
2600 6.17% -500 5.22% 3-15
Following are the results of the use of the framework on the four projects.
Statements on each project contain a project description and a narrative derived from
analysis of the output. To substantiate the validity of the model, included is a comparison
of the framework output to the original project conceptual estimates.
Asia Pacific South
The software application was developed by a centralized organization with
worldwide responsibilities. The local deployment activities included the development of
an upgrade plan that interlocked with all external dependencies to ensure a successful
software system upgrade, the development and implementation of a data migration plan,
solution validation and testing, as well as end-user communications and education.
Some of the key components of the software application being delivered were
new interfaces with other billing and invoicing applications, function to support new
processes such as direct shipments to geographic locations outside of Asia Pacific South,
8
enhanced taxing support, special pricing override capabilities, and multiple language
support.
This software application was one of many in a portfolio of applications
supporting particular business processes. The geographic location test team was
responsible for conducting a systems integration test of this application with the existing
applications contained within the portfolio. This included testing the flow of data and
function from two applications up the process and two applications down the process
from the application being changed. It also involved testing any output to systems
outside of the portfolio of applications. Systems identified as outside of the portfolio
normally consisted of older applications that were in the process of being phased out of
the process but had not yet been sunset.
The overall project complexity for the Asia Pacific South deployment was rated
as high. This was determined through assessing the responses to the predetermined
criteria contained in the framework. This particular project met the threshold for being
rating high in complexity in seven of the eight overall project criteria. The only criterion
rated as medium was duration.
The next step in the process is the rating of criteria to determine the complexity of
specified tasks. With one exception, all of the tasks were rated as low complexity. The
testing task was rated as medium. There was partial use of centralized testing resources,
partial use of automated testing, and associated volumes and other requirements that are
not in the extreme. The ratings for these tasks are input into step three to determine the
estimated hours.
For the Asia Pacific South project, all of the task estimates fell within the
recommended ranges except testing. The testing task would normally fall into a range of
20-30% of the total estimate. In this example, it was over 51%.
A review of the responses in step 2 showed that for this specific project, testing
was rated the most complex of any of the tasks. This alone may not account for the
estimates being so far outside of the recommended percentage of total. However, if one
considers that the overall project complexity was high and that every task complexity
with the exception of testing was rated low, it is not an unreasonable assertion that testing
could account for more than half of the cost of the deployment project.
A cursory review of this project, absent performing a conceptual estimate using
the framework, could lead one to overlook the impact of the overall project complexity
and simply focus on the low task complexity ratings. Another implication of the use of
this framework for assessing the deployment to this particular geographic location was
seen in the percentages associated with the validation step. There were two category
areas outside of the upper range of the validation recommendations. The two areas were
testing along with education and training. This emphasis is consistent with a project in
which the tasks are fairly non-complex, but the impact is great due to the numbers of
9
stakeholders affected by the processes being changed. The focus of the resources can be
directed to the most appropriate tasks to minimize the impact of problems incurred after
going live.
China
This project involved the deployment of application software function to an
administrative organization in China to automate the order fulfillment processing that
was previously done with manually intensive applications. The actual business processes
were only slightly enhanced. The primary objective was to increase the productivity and
controls around special bid orders. This involved automating the look-up functions and
ensuring the flow of order status information to the appropriate front-end systems.
The application function was already being utilized by other organizations in
other countries and geographic locations. The primary scope of this project was a
deployment of existing function to an organization using the application for the first time.
Each project characteristic was assessed for complexity. The assessment
indicated that the application would be deployed to one set of stakeholders or users.
Although they are not all in the same physical location, unlike many of the other
geographic locations, they do reside within the same country. This eliminated any
multiple language considerations.
The existing business processes would primarily remain in place but are modified
by a few enhancements. This lessens the complexity in terms of user education. The
new functionality of the application increases the complexity of the software application
from an information systems perspective. In addition, the number of new interfaces adds
to the technical complexity.
There were an equal number of high and medium ratings of the project
characteristics. Using the detailed guidelines and taking into account the additional low
ratings of two characteristics, the results were an overall project assessment of medium
complexity.
The tasks associated with the project management, communications management,
and information technology activities resulted in a rating of high in complexity. The
activities in which an assessment of the tasks resulted in a low complexity rating were
measurement systems, data conversion and migration, and move to production. The
remaining categories were rated as medium complexity. The framework was then used to
select the appropriate hours for each task. The last step of the process was to perform a
validation to determine if the task estimates were reasonable according to the expected
percentage of total project cost for a particular task.
The China project was an example of how the estimating framework lends itself
to gaining a better understanding of the project. The use of the framework was not
10
simply about determining a number for a total estimate, but to understand why certain
results were derived and how the task estimates related to each other and to highlight
areas of potential concern. The documented description and scope of the project could
lead one to believe that this is a typical deployment that placed heavy emphasis on end-user
education and training. However, the results of the execution of the framework
indicated that this project had a very significant information technology component. This
knowledge could lead to more focus on project management to ensure increased
awareness of the nature of the project and more effective execution of the IT activities.
European Union
This particular project was the first deployment of significantly enhanced
functionality for an existing application used within the EU. The enhancement to the
application allowed for the support of additional order types. The basic function
remained the same, but there was a significant change to the business processes. At one
point in time, specific products began to be ordered within the application being updated
by this project and ceased to be ordered via another system.
Coordination was required among the teams deploying and supporting the
enhanced application and the teams supporting the older application, which was no
longer used for ordering these products. The two primarily involved applications were
hosted and supported by teams located in different parts of the world. There also was a
major coordination effort associated with the upstream and downstream interfaces. In
effect, one application was seamlessly substituted for another within the business process
application portfolio.
After the execution of step one the overall project complexity was rated as high.
Although there was a single stakeholder organization and the deployment was within a
single country, the technical aspects and business process changes associated with the
project accounted for the complexity rating. The number of interfaces, the duration,
architecture, distribution of the teams and business process changes were each rated high
in complexity.
The tasks rated as high complexity were those related to communications
management, measurement systems and test. This is consistent with what can be
expected to be the major focus activities of a deployment project of this scope. Due to
the multiple support teams involved, communications management was highly complex.
Because an application is replacing another application within an existing business
process that could not be significantly interrupted, the complexity of measurement
systems was expected. Noting the number of interfaces and the fact that this application
is a part of a portfolio of applications supporting certain business processes, it is not
unusual that testing was a major item in terms of complexity.
The deployment resource estimates for EU were amended by the estimator in
certain areas. The testing hours were decreased; the education and training was
11
decreased; and the move to production was decreased. These adjustments were made
from assessing the estimated hours and deciding that they were too large based upon the
knowledge and experience of the estimator. The documented assumptions fell within the
valid assumptions referenced in the guidelines.
After making the adjustments to the hours, the validation step was analyzed. The
percentage of the t
Virtual reference at a global university: an analysis of patron and question type
No full textSince August of 2003, UMUC Information and Library Services have had a partially outsourced reference service that provides 24 hour online chat and after hours e-mail service. The service is continually monitored for quality purposes. This study tries to determine the following:Do the two services have different types of questions asked? What is the difference in the quality of in house service vs. outsourced service?Virtual Reference at a Global University: An Analysis of Patron and Question Type
Joseph Rawson, Megan A. Davis, Julie Harding, and Clare Miller
University of Maryland University College Information and Library Services
Introduction
References
Methods Results Conclusion/Recommendations
Since August of 2003, we have had a partially
outsourced reference service that provides 24 hour
online chat and after hours e-mail service. The
service is continually monitored for quality
purposes.
Follow up occurs for about 4% of the questions
answered by the outsourced service.
Overall, the approval rating from students is
above 90%.
We provide instant messaging (IM) reference in
house (staffed by UMUC librarians):
Monday-Thursday: 9am-9pm
Friday: 9am-5pm
Saturday-Sunday: 1pm-5pm
We conducted our study to determine the
following:
Do the two services have different types of
questions asked?
What is the difference in the quality of in house
service vs. outsourced service?
Collecting surveys:
Chat – Optional survey pops up automatically
IM – Survey had to be pushed out to patrons.
Some left chat before survey link sent. Most did
not click on the survey link. Out of 88 that clicked
on the link, 37 completed the survey.
Coding categories:
We coded transcripts using predetermined
categories. All categories, except “How Do I Cite?”
were taken from Arnold & Kaske (2005).
Transcripts reviewed:
Chat: 1,308
IM: 37
Time period: July and October 2011*
*reflects “typical library use”
Transcript coding categories:
• Directional: User is looking for the location of a
resource or library.
• How do I find the Academic Search Complete
database?
• Ready Reference: Require a single,
straightforward answer; usually uncomplicated.
• What is the capital of China?
• Specific Search: Differs from ready reference
query in that information is gathered in the form of
“a list of citations, a book, or a report” (Katz,
1997).
• I need “x” number of scholarly articles on the
effectiveness of police patrols.
• Research: Complicated, time-intensive questions,
usually from experts in field, requiring in-depth
searching. We get very few of this question type.
• Policy and Procedural: Questions pertaining to
policies or procedures within the library system.
• How do I borrow books?
• Holdings/Do You Own?: Questions about specific
holdings of a library.
• Do you have the book The Color Purple by
Alice Walker?
• How Do I Cite?: Questions about citation (APA,
MLA, Chicago, etc.)
• How do I cite this scholarly article in APA
style?
Staffing of Chat and IM service is sufficient.
A significant percentage of the questions asked
fall into the policy/procedural category,
therefore:
Reference staff must be familiar with the
library’s circulation and interlibrary loan
policies and procedures.
Special consideration should be given for
training outsourced chat operators.
The online reference manual needs to
contain all library policy and procedures
and must be kept up to date
IM and chat survey results indicate a high level
of satisfaction with the services.
The library should increase awareness of the
services though the following:
Open houses
Online visits to classrooms
LIBS 150 and UCSP 611
Arnold, J., & Kaske, N. K. (2005). Evaluating the
quality of a chat service. Portal: Libraries And
The Academy, 5(2), 177-193. Retrieved from
http://muse.jhu.edu/journals/portal_libraries_a
nd_the_academy/
Katz, W. A. (1997). Introduction to reference work,
vol. 1, Reference services and reference
processes, (7th ed.). New York: McGraw Hill.
How do I Cite?
6% Directional
10%
Ready Reference
6%
Specific Search
48%
Research
1%
Policy and
Procedural
16%
Holdings/Do you
Own?
13%
Types of Questions Asked
Chat Service
0
10
20
30
40
50
60
Extremely
Helpful
Very Helpful Helpful Somewhat
Helpful
Not Helpful
Perceived Level of
Helpfulness Chat Service
Types of Questions Asked
IM Service
How do I Cite?
16%
Directional
21%
Ready Reference
0%
Specific Search
53%
Research
0%
Policy and
Procedural
5%
Holdings/Do you
Own?
5%
Perceived Level of
Helpfulness IM Service
0
5
10
15
20
25
30
35
40
45
Extremely
Helpful
Very Helpful Helpful Somewhat
Helpful
Not Helpfu
Helpful tips in preparing faculty research grant proposals: An editor's perspective
No full textTo assist with preparing faculty research grant proposals for our next round of grant proposal submissions (the Call for Faculty Research Grant Proposals for Spring 2013 and Summer 2013 will be issued on October 1, 2012), Dr. Liebowitz will present some helpful tips for aiding faculty in developing and writing their faculty research grant proposals. These will include best practices developed from both the experiences of the UMUC Faculty Research Grant Committee, as well as useful heuristics from the experiences of others. Those who are interested in submitting grant proposals for the next round are strongly encouraged to attend the WebEx.1
Helpful Tips in Preparing Faculty Research Grant Proposals: An Editor’s Perspective
Jay Liebowitz, D.Sc.
Orkand Endowed Chair in Management and Technology
Chair of the Faculty Research Grant Committee
University of Maryland University College
[email protected]
Speaking on Behalf of the UMUC Faculty Research Grant Committee
•Faculty Research Grant Committee Members: Bob Goodwin, Cynthia Davis, John Aje, Theo Stone, Denise Nadasen, Jay Liebowitz (Chair)
2 3
My Editor’s Experience
•Founding and Current Editor-in-Chief, Expert Systems With Applications: An Int. Journal (Elsevier); entering Vol. 40; Ranked top-tier journal via most recent 2011 Thomson impact factors; 1.8 million articles downloaded worldwide in 2011; 10-15 manuscripts submitted daily worldwide
•Associate Editor, Int. Journal of Teaching and Case Studies (Inderscience)
•Former Series Editor, Pergamon Press/Elsevier CS-related journals
•Former Associate Editor, Telematics and Informatics: An Int. Journal (Elsevier)
•Former Founding Editor-in-Chief, Failure & Lessons Learned in IT Management: An Int. Journal (Cognizant Communication Corp.) 4
What Do I Look for as a Journal Editor?
•Fits within the aims and scope of the journal
•Follows manuscript guidelines
•Well-written
•The research provides a significant contribution to the existing literature
•Thorough literature review (with most recent references in the field) and appropriate research methods employed
•Detailed analysis, discussions, and conclusions
•Lessons learned, limitations, and future work 5
Thomson ISI Master Journal List
•About 9,000 journals
•Top 2 reasons for manuscript rejection:
–Flawed or poorly planned study design
–Lack of detail in methods Faculty Research Grant Program at UMUC
•Started in 2010 out of the Provost’s Office
•4K for single-PI; $5K for Co-PIs to be split evenly; normally for one semester research work)
•20-30% acceptance rate
•Each round, we typically receive anywhere from 30-60 proposals worldwide
•Open to all UMUC adjunct, NOL, and Collegiate faculty in US and Worldwide
6 Next Round of Faculty Grant Research Proposals for Spring/Summer 2013
•Opens October 2, 2012
•Grant Proposals Due: November 15, 2012
•Notification of Awards: December 20, 2012
•Proposals limited to 3 single-spaced pages, not including the reference list, letter of support from your supervisor, letter/email of permission from Office of Institutional Planning, Research, and Accountability, copy of IRB form submission, and current CV of PI and any co-PIs
7 Proposal
•Title and abstract of the research proposal
•Faculty member’s name and contact info
•Proposed research semester
•Description and goals for proposed research
•Brief literature review and summary of current knowledge in the topic area to be investigated
•Timeline of research tasks
•Expected deliverable outcomes of the research
•Expected benefits to you, your program, and/or to UMUC
•Target publication or conference for disseminating the research results
8 Selection Criteria
•Alignment with areas of interest to UMUC
•Expected impact and benefits of the research to UMUC
•Originality, that is, research that has not already been published
•Sound research design
•Feasibility of the research design for completion during the time proposed and with facilities and/or equipment available
9 Deliverables Expected
•Summary report, of no more than 2 single-spaced pages, to be sent no later than one month after the conclusion of the research semester
•Share results of your research via a UMUC forum (e.g., Lunch & Learn faculty research seminar (F2F or virtual—e.g., WebEx), ShareFair, etc.)
•Include any deliverables (conference/journal papers/presentations, etc.) resulting from your funded research
10 The Research Process pshsresearchunit.webs.com/Modules/BSICR%20Ch%204.pdf Good Applied Research Design: Five Components
1. Goals: Why is your study worth doing? What issues do you want it to clarify,
and what practices and policies do you want it to influence? Why do you want to
conduct this study, and why should we care about the results?
2. Conceptual framework: What do you think is going on with the issues, settings,
or people you plan to study? What theories, beliefs, and prior research findings
will guide or inform your research, and what literature, preliminary studies,
and personal experiences will you draw on for understanding the people or issues
you are studying?
3. Research questions: What, specifically, do you want to learn or understand by
doing this study?
4. Methods: What will you actually do in conducting this study? What
approaches and techniques will you use to collect and analyze your data, and how
do these constitute an integrated strategy?
5. Validity: How might your results and conclusions be wrong? What are the
plausible alternative interpretations and validity threats to these, and how will you
deal with these? How can the data that you have, or that you could potentially collect,
support or challenge your ideas about what’s going on? Why should we believe
your results?
Joseph Maxwell (2008), “Chapter 7: Designing a Qualitative Study”, Applied Research Designs: Sage Publications. http://www.corwin.com/upm-data/23772_Ch7.pdf Sample Observations
•“We've seen a lot of proposals that have had weak research questions (i.e., questions that are not easily answerable or otherwise non-directional, for example: ‘We propose to explore how rubrics can lead to inconsistent grading across sections’ vs. a more directional research question such as ‘Through the use of multiple readers vs. one reader, we propose to identify points of inconsistent scoring in the rubrics across sections.’)”
•“We’ve also seen methodologies that don’t seem to address answering the question, are vague, or seem overly ambitious for the scope of the funding period.”
13 Helpful Tips
•Be sure to cite your research question, the need for your work, and explain your research methodology
•Include a short description of other research you have previously conducted, as well as a brief literature review (including recent sources) from others
•Make sure you have a representative sample size
•We allow for various research methodologies (e.g., quantitative, qualitative, exploratory, etc.)
•Discuss the value of your research to the field and specifically its impact to UMUC
•Highlight how you will disseminate your results
•If using human subjects (UMUC students or others), you must get approval from the Office of Institutional Research
•Be sure that your proposal is well-written, free of grammatical errors, and abides by the page limit
14 15
Failure Breeds Success (Sir Winston Churchill)
•A pessimist finds the difficulty in opportunities, and the optimist finds the opportunity in difficulties
•The optimist finds the success between the failures 16
Summary
•Look at UMUC Research and Scholarship site: http://www.umuc.edu/library/libresearch/
•Sign onto an EBSCO Alert through the Library to get latest articles in your research area sent to you each week
•Do your homework!
•Perseverance will get you far
•Set goals and objectives
•Think “continuous improvement” (Working paper, conference paper, journal article
Healthcare reform: Innovative strategies for successfully managing and implementing organizational change
No full textThe purpose and scope of this study is to provide innovative change strategies that will assist healthcare organizations with implementing successful change. Also, explore healthcare trends, healthcare innovation, organizational change theory, and innovation theory in an effort to gain a better understanding of how organizations successfully implement organizational change.Healthcare Reform: Innovative Strategies for Successfully Managing and Implementing Organizational Change
Lisa M. Pinkney
Doctor of Management, UMUC, 2013
U.S. Food and Drug Administration
[email protected]
Purpose and Scope of the Research
The purpose and scope of this study is to:
•
provide innovative change strategies that will assist healthcare organizations with implementing successful change.
•
explore healthcare trends, healthcare innovation, organizational change theory, and innovation theory in an effort to gain a better understanding of how organizations successfully implement organizational change. Management Problem
•
The United States Healthcare System is in crisis.
•
Healthcare expenditures in the United States neared 256 billion spent in 1980 (Centers for Medicare and Medicaid Services, 2012).
•
Present day healthcare costs are escalating, diminishing access to care has left millions of Americans without healthcare, and the quality of care is decreasing. Research Questions
1.
What innovative strategies can management in the healthcare organizations use to successfully achieve needed change and reform?
2.
What role does being open to innovation play on the successfulness of organizational change initiatives?
3.
How can organizations utilize best practices when undergoing change?
4.
How does trend analysis benefit an organization when embarking upon organizational change?
Significance of this Research for Management
The ability of leaders to utilize innovative strategies to manage organizational change is critical to an organizations’ sustainability.
The use of innovative change strategies can result in a better functioning healthcare industry.
Main Literature Themes and Authors
•
Theme 1 – Healthcare Trends
Authors: International Federation of Healthcare Plans, 2011; Lazarus, 2011; Lazarus & Fell, 2011; Mann & Gotz, 2007
•
Theme 2 – Healthcare Innovation
Authors: Topol, 2012; Thakur, Hsu, & Fontenot, 2011; Morse, 2010; Omachonu & Einspruch, 2010
Main Literature Themes and Authors
•
Theme 3 – Organizational Change Theory
Authors: Young, 2009; Kotter, 2007, 1996; Mintzberg & Waters, 1985; Kolb, 1984; Deming, 1982; Juran, 1964; Dewey (1896, 1910, 1916); Lewin (1942, 1943/1944, 1946, 1947)
•
Theme 4 – Innovation Theory
Authors: Tidd, Bessant, & Pavitt, 2009; Birkinshaw, Hamel and Mol, 2008; ; Tushman & O’Reilly, 1996; Utterback, 1996; Rogers, 1995, 1983; Damanpour, 1991, 1984; Henderson & Clark, 1990; ; Schroeder, Van de Ven, Scudder, & Polley,1989; Mohr, 1969; Schumpeter, 1950
Key Propositions Distilled from the Literature
•
Proposition 1: Healthcare trends have had significant influence over the current state of the United States healthcare system.
•
Proposition 2: The implementation of innovative information technology initiatives into healthcare organizations can positively influence healthcare cost.
•
Proposition 3: By considering best practices in healthcare innovation, best practices may be generalizable to other organizations.
•
Proposition 4: Examining organizational change theory may provide leadership with a better understanding of the challenges of achieving success.
•
Proposition 5: Innovation is an effective mechanism for incorporating organizational change in healthcare reform.
Conceptual Framework Thesis statement
The thesis statement is:
Healthcare organizations within the United States can successfully implement organizational change and positively affect healthcare quality and cost by utilizing innovative change strategies. Research Approach & Formation of Argument
•
The research approach involved an Evidence Based Research approach consisting of a comprehensive literature review that provided a foundation of knowledge and identified major themes and constructs specific to healthcare trends, healthcare innovation, organizational change theory, and innovation theory.
•
In order to conceptualize the literature, three lenses for viewing the problem of the United States healthcare system in crisis were employed. The lenses are 1) system lens; 2) practical lens and 3) theoretical lens.
•
The literature supports the argument that management scholars and practitioners can successfully implement organizational change in healthcare with the use of innovative change strategies. Literature Review Convergence via the Viewing Lenses Findings
Finding #1: Healthcare industry trends provide critical information in organizational strategic planning and continual improvement.
Identifying trends provide a useful way for organizations to see what changes are likely to occur in a firm’s industry in the future within a specific timeframe.
Using trend information can assist managers in analyzing the likelihood of the change affecting their company’s competitive position and business prospects.
Finding #2: There are an abundant amount of healthcare innovation that has been identified and implemented successfully.
There are many healthcare innovations including: checklist; behavioral economics; payment innovation; evidence-based decision-making; virtual visits; surgical robots; use of best practices and technological advances.
The use of information technology innovation in healthcare is aimed at reducing costs, increasing the quality of care, and increasing efficiency and the access to care. Findings
Finding #3: Organizational change and innovation can be viewed as a process resulting in successful outcomes if designed, managed, and effectively implemented.
Organizational change must consider the change in individual behavior as organizational change occurs.
The interrelationship of variables such as action planning, skill development and delivery, incentives, monitoring and control, and implementation success are critical during organizational change.
Identification of the type of innovation is critical when exploring innovation as a process with a focus on adoption.
Other findings include: Nine Social Dynamics, Healthcare Trends, Healthcare Innovation, Best Practices, Technological Advances, Change Theory, Environmental Change, and Innovation Theory. Implication for Management Practice
This study illustrates that:
•
Organizational change and innovation play a critical role in determining the success of today’s business environment.
•
There are key healthcare innovations and best practices that may enhance the success of the U.S. healthcare reform.
•
Exploration of relevant classical and contemporary organizational change and innovation literature demonstrates a systematic and process approach and provides management a better understanding of each construct. Future Research Agendas
1.
Expand the exploration of the Nine Social Dynamics in and effort to identify additional key findings that may advance management theory.
2.
Conduct a study to measure the efficacy of various innovation change theories.
3.
Research if innovative change strategies can be generalized to other fields of management (i.e. education, social science)
Enron ethics: A hermeneutic study seen through a multi-theoretical lens
No full textThe purpose of this study is to examine the influence managers have on the organizational culture and the effect that the organizational culture may have on individual inclinations that may affect a group decision-making culture.
The goal of this research is to present a model to minimize the risk of unethical behavior within organizations, and to present a framework as a best-practice guide to decision-making activities conducted in a way that is morally and legally acceptable to the greater community that it serves.
This study highlights factors of a decision-making culture that is inherently unethical.Enron Ethics: A Hermeneutic Study seen through a Multi-Theoretical Lens
Leverne Moore Laws
Advisors: Dr. Michael Evanchik and Dr. Dennis Winters
SOARS: Fall 2012Purpose and Scope of Research
The purpose of this study is to examine the influence managers have on the organizational culture and the effect that the organizational culture may have on individual inclinations that may affect a group decision-making culture.
The goal of this research is to present a model to minimize the risk of unethical behavior within organizations, and to present a framework as a best-practice guide to decision-making activities conducted in a way that is morally and legally acceptable to the greater community that it serves.
This study highlights factors of a decision-making culture that is inherently unethical.Personal Reasons for Research Topic
This journey started in DMGT 810 (Leadership and Change) with a discussion about Saul Gellerman’s article “Managing Ethics from the Top Down.” As a manager I was not completely sold on the belief that managers had this burden alone. The research has led me back to the idea that managers do influence the ethical culture of the organization in significant ways.Importance to Management Practice
Unethical behavior in an organization can have a devastating impact on every stakeholder.
Individual ethical inclinations are often introduced into a group decision-making culture by managers’ everyday ethics, their ethical or unethical choices, and behaviors.
Enron continues to stand out as an example and reminder of what goes wrong when an unethical culture is set into motion when one or more individuals’ unethical behavior seeps into the organizational culture. Line of Argument
When managers create an unethical ethos through their decisions, communicated values, and everyday behaviors within the organization, it permeates the culture and negatively affects employees’ behavior and decision-making.Most Important Theoretical Frameworks
Cognitive Moral Development – suggests people make moral judgments in varied distinct ways based on three levels (six stages) of moral development (Kohlberg)
Groupthink – study of decision-making behavior that occurs within groups (Janis)
Symbolic Interactionism – explains how individuals view the world and take action based on their individual perspectives and the interactional sharing of these perspectives (Blumer, Charon, Mead)
Bounded Ethicality – explains the process that occurs when individuals participate in unethical behavior that is contrary to their personal ethical belief (Bazerman)Research Question
RQ1 – what influence do managers have on employee ethical behavior within the organizational culture?
RQ2 – how do individual ethical inclinations affect a group decision-making culture? Research Propositions
Proposition 1: Managers highly influence the organizational culture. They have the responsibility to ensure that the organizational culture remains ethical, but all stakeholders have a role.
Proposition 2: Organizational cultures will be ethical when managers are ethical role models for employees to mirror, make ethics a priority, communicate clearly their position on ethics, reinforce its policies, and create transparent ethical plans that can help guide the people through the process.Primary Practitioner Scenario
The management group at Enron influenced the organizational culture by guiding it through and supporting a carefully managed web of deception, which ultimately affected the organization’s bottom line.
Some organizations continue to get it right, such as United Technologies Corporation (UTC).
Like Enron purported to do, UTC actually did: it developed a code of conduct, provided ethical guidelines, and held all employees accountable. Senior managers modeled the behavior they expected of employees.
Enron managers also provided ethical guidelines to employees, but unlike United Technologies, they failed to address and often responded positively to unethical behavior, or aligned their own behavior with these same policies.
United Technologies saw the ethics policy and procedures as a method to protect the bottom line. Enron saw the ethics policy as an obstacle to override in order to inflate the bottom line and benefit from that deception.Main Literature Themes & Authors
Organizational Culture (Ciulla, 2005; Javidan, Dorfman, DeLuque & House, 2006 Schein, 1990; Sims, 2004; Sims & Brinkman, 2003; Trevino & Brown, 2004, Trevino, Weaver, & Brown, 2008; Trevino & Nelson, 2011; Yukl, 2006)
Managers’ Ethical Influence on Employees (De Hoogh & Den Hartog, 2008; Gellerman, 1989; Gini,2004; Quaquebeke, Kerschreiter, Buxton & Dick, 2010; Schminke, Ambrose & Neubaum, 2005; Storr, 2004)
Ethical Behavior and Decision-making (Banerjee, Cronan & Jones, 1998; Burger, 2009; Janis, 1971; Trevino, 1986; Trevino & Nelson, 2011; Bazerman & Moore, 2009)
Whistleblowing (Dandekar; 1991; Glazer & Glazer, 1989; Jensen, 1987; Miethe & Rothschild, 1994)
Factors of an Ethical Decision-making Culture (Hosmer, 1994; James, 2000; McDonald, Simpson & O’Brien, 2008; Thomas, Schermerhorn & Dienhart, 2004)Warrants
There is a correlation between a manager’s ethical decision-making and the organization’s effectiveness (De Hoogh & Den Hartog, 2008; Trevino, 1986).
In an ethical decision-making organizational culture, managers have the responsibility to lead, take appropriate corrective actions, and demonstrate that they are held to the same standard, not just pontificate, but demonstrate ethical behavior through their actions (Banerjee, Cronan, & Jones, 1998; Brown, Trevino & Harrison, 2005; Cohen, 1993; De Hoogh & Den Hartog; Meyers, 2004; Thomas, Schermerhorn & Dienhart, 2004). Conceptual Framework
Conclusions
It is possible to create, sustain, and maintain an ethical decision-making culture (Ardichvile et al., 2009; Gini, 2004; Hosmer, 1994; McDonald, Simpson, O’Brien, 2008; Thomas, Schermerhorn, & Dienhart, 2004; Trevino, 1986) despite those behavioral components that may affect it.
Behavioral factors are an important consideration when establishing or maintaining an ethical group decision-making culture. Awareness of these factors may help managers lead and promote an ethical environment.
Ethical awareness should continue to be a part of organizational conversations, its strategic plans, and performance awards and punishment processes
Tending the Garden: Repairing Organizational Trust Damaged by Turmoil
No full textThis study’s argument is that greater understanding of damaged organizational trust will help managers better understand how to repair trust. Greater understanding of the issue of repairing damaged organizational trust will help managers understand the importance of maintaining trust and will help make clear the consequences of not repairing damaged trust within organizations.Tending the Garden: Repairing Organizational Trust Damaged by Turmoil
Joanne L. Tritsch
Doctor of Management, UMUC, 2012
New Jersey Department of Health & Senior Services
[email protected]
https://sites.google.com/site/researchbytritsch/
Background of Management Problem
•Management events from the last five years, including the dissolution of the Daimler Chrysler AG merger, the bankruptcy of Lehman Brothers and the phone hacking and bribery scandals of Murdoch’s News Corporation, make clear the importance of maintaining organizational trust, as well as the consequences for not repairing trust damaged by workplace turmoil.
•Organizational trust is a foundational construct for business. However, once trust is established it must be maintained. Workplace turmoil can damage and even destroy trust. Realizing the fundamental importance of trust, management theorists have increasingly turned their attention to trust repair research (Gillespie & Dietz, 2009; Kim, Dirks & Cooper, 2009; Kramer & Lewicki, 2010; Tomlinson & Mayer, 2009).
Significance of Management Problem
•“Given the prevalence of trust failures and the seriousness of the consequences, knowing how to repair trust is seen as a critical management competency” (Lewicki & Bunker, 1996, p. 114). Studying the issue of repairing organizational trust will help managers increase their understanding about the importance of maintaining trust and will make clear the consequences of not repairing damaged organizational trust.
•Additional research on the repairing damaged organizational trust may allow managers to understand more fully how trust can be damaged, lost and repaired. Research Question & Thesis
•The research question for this study is “To what extent can organizational trust damaged by workplace turmoil be repaired to its pre-turmoil state?” Four associated propositions were crafted to further support the research question.
•This study’s argument is that greater understanding of damaged organizational trust will help managers better understand how to repair trust. Greater understanding of the issue of repairing damaged organizational trust will help managers understand the importance of maintaining trust and will help make clear the consequences of not repairing damaged trust within organizations Key Literature Themes & Authors
•This study’s literature review begins with an overview, categorizes the literature into four perspectives, and concludes with a general summary. When woven together, the literature review provides a foundation for the subsequent discussion of the research propositions as well as for the conceptual framework.
•The study begins with the Origins of Trust Research and examines the basic construct of trust and its multiple definitions. Drawing from the work of seminal theorists from Barnard to Kramer and Lewicki, the foundations of trust research are explored.
Key Literature Themes & Authors
• The first perspective, Varied Perspectives on Trust, examines the theory along with empirical studies concerning the foundations of trust, the process of building trust, and the maintenance of trust.
•The second perspective, Losing Trust, examines the theory along with empirical studies concerning the turmoil caused by unmet expectations, broken promises and unplanned crises.
•The third perspective, Repairing Damaged Organizational Trust, examines the theory along with empirical studies concerning apologies, fresh starts and resultant levels of trust. Establish
Afirm Build
Acknowledge
Apologize Damage
Rebuilding
Damaged
Trust
Establish
Build Afirm
Acknwlde
Damage Apologize
Rebuild
Trust
Estab
lish
Build Afirm
Ack
Dam
Apolo
gize
Rbild
Theoretical & Conceptual Framework
Research Propositions
The research question for this study is “To what extent can organizational trust damaged by workplace turmoil be repaired to its pre-turmoil levels?”
•Proposition 1 -- Ongoing attention may be required to maintain optimal levels of organizational trust.
•Proposition 2 -- Workplace trust is damaged in proportion to the magnitude of the workplace turmoil.
•Proposition 3 – The process of acknowledgement, apology and affirmation aids in the repair of damaged trust.
•Proposition 4-- The longevity of repaired trust may be dependent upon the quality of the trust repair process.
EBRS Methodology
•The primary research method for this study, Evidence-Based Research Synthesis (EBRS), is used to select the literature for analysis and to help explore both the study’s research question and its four associated propositions.
•In addition to extensive documentation, the evaluation of the quality of the research elements is an important step. This study uses a variation of Reay, Berta and Kohn’s (2009) evidence matrix expanded to include Chenail’s Matrix and a weighted calculation of model validity and citation volume. Findings & Results
•RQ: To what extent can damaged organizational trust be repaired to its pre-turmoil state?”
•Argument: Greater understanding of damaged organizational trust will lead to greater understanding of repairing trust and the consequences of not repairing damaged organizational trust.
•Finding 1: Organizations need to take care to honor trust in relationships with workers.
•Finding 2: When damaged, organizations should move quickly to attempt trust repair with workers.
•Finding 3: Acknowledgement, apology and affirmation facilitates trust repair between organizations and workers. Management Practice Implications
•Trust is most often fully appreciated once it is lost or damaged.
•Trust is a fundamental business construct.
•Trust enables businesses to function at higher levels.
•Greater understanding of repairing organizational trust is important to both organizations and workers.
•Trust exists between organizations and workers.
•Once broken, trust can be restored, but not always to the pre-turmoil level. Future Research Potential
•Conduct primary and empirical research
•Strengthen link between trust, turmoil and trust repair
•Cognitive neuroscience – Trust and the Brain
•Explore trust in different types of organizations with different categories of workers
•Gather feedback – you can help! https://sites.google.com/site/researchbytritsch
Demands culture makes on successful quality management implementation: A study at the senior management level
No full textAchieving the desired benefits of Quality Management System implementation is reported to occur in only one-third to one-half of organizational attempts (Burdett, 1994; Garvin, 1986; Grant, Shani, and Krishnan, 1994; Hindo, 2007; Yandrick, 1994; Naor, 2006). Even with this imperfect record of success, senior managerial leadership spends considerable energy and investment maintaining the current Quality Management System or chasing an alternative perceived superior. Consistently, implementation failures are blamed on the lack of a cultural change for quality in the target organization or the choice of an inadequate quality management system underpinning the implementation effort. This quantitative study of 44 senior managers fills a gap in the literature by combining a previously validated cultural assessment tool with a previously validated Quality Management System benefits assessment tool to show how cultural context relates to successful implementation. Results of this study show that some qualitative benefits of Quality Management System implementation correlate with the gap between current organizational culture and the organizational culture perceived by senior management as most supportive of their current Quality Management System. Quantitative benefits, while reported by the senior manager participants to move in a direction management would consider advantageous, showed only one correlation. Additional insight came from delineating current organizational culture and the organizational culture perceived by management as most supportive of their current Quality Management System into one of four cultural typologies (Clan, Adhocracy, Market, and Hierarchy). Some qualitative benefits of Quality Management System implementation correlate with the gap between current organizational culture and the organizational culture perceived by senior management as most supportive of their current Quality Management System for Clan and Adhocracy cultural typologies. Market culture typology correlated with one quantitative benefit, reduction in external nonconformances.ABSTRACT
Title of Dissertation:
THE DEMANDS CULTURE MAKES ON SUCCESSFUL QUALITY MANAGEMENT IMPLEMENTATION: A STUDY AT THE SENIOR MANAGEMENT LEVEL v5 Oct. 2010
Mark Douglas Green
Doctor of Management, 2010
Dissertation Directed By:
Dr. Claudine SchWeber and Dr. Ann Herd, Co-Chairs
Graduate School of Management and Technology
University of Maryland University College
Achieving the desired benefits of Quality Management System implementation is reported to occur in only one-third to one-half of organizational attempts (Burdett, 1994; Garvin, 1986; Grant, Shani, and Krishnan, 1994; Hindo, 2007; Yandrick, 1994; Naor, 2006). Even with this imperfect record of success, senior managerial leadership spends considerable energy and investment maintaining the current Quality Management System or chasing an alternative perceived superior. Consistently, implementation failures are blamed on the lack of a cultural change for quality in the target organization or the choice of an inadequate quality management system underpinning the implementation effort. This quantitative study of 44 senior managers fills a gap in the literature by combining a previously validated cultural assessment tool with a previously validated Quality Management System benefits assessment tool to show how cultural context relates to successful implementation.
Results of this study show that some qualitative benefits of Quality Management System implementation correlate with the gap between current organizational culture and the organizational culture perceived by senior management as most supportive of their current Quality Management System. Quantitative benefits, while reported by the senior manager participants to move in a direction management would consider advantageous, did not show the same correlation.THE DEMANDS CULTURE MAKES ON SUCCESSFUL QUALITY MANAGEMENT IMPLEMENTATION: A STUDY AT THE SENIOR MANAGEMENT LEVEL
By
Mark Douglas Green
Dissertation submitted to the Faculty of the Graduate School of the University of Maryland University College, in partial fulfillment of the requirements for the degree of
Doctor of Management
2010
Advisory Committee:
Dr. Claudine SchWeber, Co-chair
Dr. Ann Herd, Co-chair
Dr. Hasan Sayani
Dr. Leslie Dinauer
Version 5 Oct. 2010©Copyright by
Mark Douglas Green
2010i
Acknowledgements
Family support is critical for a working adult attempting to complete any degree and especially so for something as intense and demanding as a doctorate. Not always a smooth journey, my wife Allister, and daughter Blair, found patience and understanding as I pursued my dream. Parents too, play a crucial role in life success. I was blessed with parents who instilled in me the values of hard work, accomplishment, and a passion for continual learning. I owe these people much.
No successful candidate completes a degree like a Doctor of Management without the inspiration, patience, and support of a number of academic mentors, who along the way had to deal with the eccentricities of a working adult student. Early in my career I was fortunate enough to be mentored, during my pursuit of an MBA, by Doctor Burton Lowe who encouraged me to embrace critical thinking and showed me that it was possible to be both a successful business person and an academic. More recently, I have had the good fortune to have been both taught and mentored by Dr. Claudine SchWeber, my dissertation co-chair, who despite my occasional frustrations, always kept me focused and moving towards my goal. My first class was with Dr. SchWeber and she created a standard and expectation I very much admire. I also want to thank Dr. Ann Herd, my dissertation co-chair, who taught me new ways to think about statistics and how to apply and explain them to others, Dr. Hasan Sayani, whose candor and insights helped bridge the occasional significant gap between business application and academic theory, and Dr. Leslie Dinauer, who always challenged my thinking.
I would also like to thank Dr. Krys Bledowski of Manufacturers Alliance and Process Improvement (MAPI) who helped me gain access to the senior leaders of the prestigious ii
companies that make up MAPI membership and whose knowledge and council on survey design proved significant.
We build knowledge standing on the shoulders of those who have gone before us. Their work inspires, feeds and directs new attempts at scholarship. For me, in the area of quality, the work of Dr. Barbara Flynn and Dr. Roger Schroeder stands out above all others. Dr. Flynn was gracious enough to speak with me by phone as I was developing my research and Dr. Schroeder was kind enough to exchange emails with me. In addition, though I have never met them, as both a longtime practitioner and a nascent academic, I was inspired by the work of Dr. Kim Cameron and Dr. Robert Quinn and their competing values framework for diagnosing culture.
Lastly, I must thank the members of my cohort who helped make the class time together something I very much looked forward to and will never forget. Specifically, I must thank Marcia Bouchard, Al Holliman, and Rich Feyerabend. These are three people I could count on not only academically, but also personally, and I am very proud to call you colleague and friend.iii
Table of Contents
Acknowledgements………………………………………………………………………………i
Table of Contents………………………………………………………………………………..iii
List of Tables………………………………………………………………………………….…vi
List of Figures……………………………………………………………………………….......vii
List of Acronyms………………………………………………………………………………..viii
Chapter One: Introduction………………………………………………………………………...9
Historical Context for Quality Management Systems and Organizational Culture….9
Management Problem and Purpose Statement…………………………………………….10
Management Problem………………………………………..……………………...10
Purpose Statement…………………………………………………………………..11
Relevance to the Study of Management…………………………………………………...11
Academic Perspective……………………………………………………………….11
Practitioner Perspective…………………………………………………...………...12
Chapter Two: Literature Review and Argument………………………………………………...13
Purpose and Focus Restated……………………………………………………...………..13
Purpose Statement…………………………………………………………………..13
Focus and Scope of Work…………………………………………………………...13
Background of Literature Review…………………………………………………………13
Literature Review Part I: Quality Management and Organizational Success…..................15
History and Development of Quality Management……………………………...….15
Theories of Quality Management…………………………………...………………24
Strategy and Quality Management………………………………………………….29
Strategic Leadership and Quality Management…………………………………….32
Human Resources and Quality Management……………………………………….34
Innovation and Change and Quality Management………………………………….34
Organizational Structure and Quality Management………………………………...38
Conflict and Quality Management…………………………………………………..41
Organizational Success and Quality Management………………………………….43
Literature Part II: Culture and Its Context for Successful Quality Management Implementation…………………………………………………………………………….45
Models for Understanding and Defining Culture…………………………………...45
Organizational Culture and Quality Management…………………………………..48
Summary…………………………………………………………………………………..56
Argument………………………………………………………………………………......58
Stated Argument…………………………………………………………………….58
Research Questions and Hypothesis……………………………………………………….59iv
Research Question 1……………………………………………………………...…59
Research Question 2…………………………………………………………..…….59
Hypothesis 1………………………………………………………………………....59
Hypothesis 2………………………………………………………………………....59
Chapter Three: Conceptual Framework and Research Method………………………………….60
Conceptual Framework and Supporting Literature………………………………………..60
Data Collection…………………………………………………………………………….61
Recruiting of Participants, Sample Technique and Size…………………………….62
Description of Population…………………………………………………………...62
Research Methods…………………………………………………………………………63
Method………………………………………………………………………………63
Vehicle for Assessing Culture………………………………………………………65
Measures…………………………………………………………………………….70
Data Collection Procedures………………………………………………………………..83
Data Preparation for Analysis……………………………………………………………..84
Raw Data Coding……………………………………………………………………84
Recombining Results………………………………………………………………..84
Reliability Analysis…………………………………………………………………84
Creating the Variable Actual Congruency (Ideal Culture versus Actual Culture)….85
Calculating Average Qualitative Benefits………………………………..................87
Calculating Internal and External Nonconformance Improvement…………………87
Chapter Four: Data Analysis and Discussion……………………………………………………89
Hypothesis: Data Analysis and Findings………………………………………………….89
Congruency Measures……………………………………………………………....93
Hypothesis 1………………………………………………………………………...94
Hypothesis 2……………………………………………………………………….106
Additional Analysis………………………………………………………………………107
Culture Typologies and Benefits…………………………………………………..110
Discussion of Findings…………………………………………………………………...112
Summary of Findings……………………………………………………………...112
Chapter Five: Conclusion, Implications, Recommendations for Future Study………………...115
Revisiting Study Basics………………………………………….....................................115
Purpose Revisited………………………………………………………………….115
Argument Revisited………………………………………..………………………115
Summary of Method Employed……………………………………………..……..115
Findings from Study……………………………………………………………….116
Implications………………………………………………………………………………118
Understanding, Identifying, and Considering Cultural Context for Success………118
Importance of Cultural Typologies to Successful Implementation….………….…119
Understanding Why Manipulating Culture Inappropriately Can Lead to Failure....121
A Measured Approach to Quality Management Acculturation…………………....123
Longitudinal Consolidation of Culture Oriented Change…………………………126v
Limitations of the Study………………………………………………………………….126
Future Research Opportunities…………………………………………………………...128
Summary…………………………………………………………………………………130
References………………………………………………………………………………………132
Appendix A: Definition of Terms……………………………………………………………...140
Appendix B: Survey Instrument…………………………………………………………….....144
Appendix C: Calculating Congruency Between Ideal and Current Culture…………………...174
Appendix D: Hypothetical Assessment of a Number of Quality Management System’s Cultural Leanings………………………………………………………………………..176
Appendix E: Frequency Distribution of Survey Partial Titles…………………………………183
Appendix F: Descriptive Statistics of Measures……………………………………………….185vi
List of Tables
Table 1: Responsibility for Quality Over Time…………………………………………………16
Table 2: Overcoming Cultural Conflict in Organizations………………………………………50
Table 3: Literature Themes……………………………………………………………………...57
Table 4: MAPI Council Membership…………………………………………………………...64
Table 5: Definitions of Four (4) Cultural Typologies…………………………………………...66
Table 6: Definitions of Quality Management System Implementation Qualitative Benefits …...80
Table 7: Defintions of Six Quantitative Benefits Measures………………………………….....83
Table 8: Cronbach’s Alpha of Cultural Typology Variables……………………………………85
Table 9: Cronbach’s Alpha of Qualitative Benefits Measures……………………………….....86
Table 10: Correlations between Congruency and Qualitative Benefits………………………..100
Table 11: Correlations between Congruency and Quantitative Benefits………………………105
Table 12: Correlations between Cultural Typology Congruency and Qualitative Benefits ......110
Table 13: Correlations between Diagnosed Ideal Clan VS Actual Clan………………………112vii
List of Figures
Figure 1: Current ISO 9000 Model for Quality Management………………………………..…20
Figure 2: Baldridge Criteria for Performance Excellence……………………………………....21
Figure 3: Theory of Quality Management Underlying Deming……………………………..….26
Figure 4: Quality Management Breadth & Scope……………………………………………....28
Figure 5: Process Model for Re-thinking the Relationship Between Quality and Culture……..37
Figure 6: Probability of Change Acceptance…………………………………………………...38
Figure 7: Delineating Relationships and Responsibilities of Managers………………………...40
Figure 8: Competing Values Framework…………………………………………………….…48
Figure 9: Competing Values Framework and Typologies………………………………………48
Figure 10: Competing Values Framework and Quality Management………………………….54
Figure 11: Overall Ideal Cultural Profile for Quality…………………………………………..55
Figure 12: Framework for Research…………………………………………………………….61
Figure 13: Hypothetical Cultural Profile of Organization A ………………………………….125
Figure 14: Hypothetical Cultural Assessment of Six Sigma ………………………………….126viii
List of Acronyms
ASQ – American Society for Quality
CEO – Chief Executive Officer
ISO – International Standards Organization
MAPI – Manufacturers Alliance for Productivity Improvement
QM – Quality Management
QMS – Quality Management System
TQ – Total Quality
TQM – Total Quality Management
VP – Vice President9
Chapter One: Introduction
Introduction
Quality Management Systems1
existing organizational culture toward supporting a particular QMS, perhaps the type of system(s) chosen should support the existing organizational culture. A review of the history of (QMS) were developed to help organizations ensure the efficiency and effectiveness of all activities necessary to, design, develop, specify, plan, purchase, manufacture, check, control, and deliver quality products and services (Juran, Gryna, & Bingham, 1951). Over time, a number of systems have been proposed by various experts to guide and underpin an organization’s efforts toward successful Quality Management (QM) development and implementation (i.e. ISO 9000, Malcolm Baldridge, Total Quality Management, Six Sigma, etc.). Each form of QMS has seen its devoted proponents and vocal detractors, with each side in the debate pointing to examples of grand success and grand failure to make its case for a preferred system or combination of preferred systems. Among organizational leaders and quality practitioners, the debate continues over which system or combination of systems is best. Left unresolved, such disputes detract from the real objective, which is to provide products and services that meet customer needs. When someone obtains a good or service, regardless of the source (i.e. business, non-profit, government) that good or service is required to meet certain needs and expectations. Regardless of QMS preference, proponents explicitly state or strongly imply the need to move the existing organizational culture to an organizational culture supporting quality by aligning the organizational culture to the QMS chosen. Historically, missing from the debate is consideration that rather than moving the ixixixixixixix
1 Many different terms are used in the literature to describe the activities of management necessary to consistently provide quality products and services including: 1) Total Quality Management (TQM), 2) Total Quality (TQ), 3) Zero Defects, 4) Six Sigma , 5) ISO 9000, 6) Malcolm Baldridge, etc. For this research proposal the terms Quality Management System, Quality Management Model and/or Quality Management will be used to refer generically to these various approaches.10
the QMS debate is important in order to understand the management problems that standard orthodoxy creates.
Historical Context for Quality Management Systems and Organizational Culture
Edgar Schein (2004, p.11) writes, “It can be argued that the only thing of real importance that leaders do is to create and manage culture; that the unique talent of leaders is their ability to understand and work with culture”. Historically, the major thinkers and proponents of various QMS write about creating a new culture for quality within organizations (Kolesar, 1995; Juran, et al., 1951; Crosby, 1979; Deming, 1982; Peters & Waterman, 1982; Fuentes, Montes, & Fernandez, 2006; Fuchs, 1993; Buch & Rivers, 2001). Generations of quality practitioners have been taught that the default objective is to change current organizational culture to support the chosen QMS. Literature concerned with managing, understanding, and working within existing culture is scarce.
Substantial attempts to move the existing culture away from its equilibrium and more toward supporting a chosen QMS could, predictably, produce resistance to such change. The literature on threat rigidity responses by organizations supports this possibility (Barnett & Pratt, 2000; Staw, Sanderlands, & Dutton, 1981). The realization of benefits from implementing a Quality Management System could be negatively affected. The problem this presents for management can now be fully articulated and the purpose of this research established.
Management Problem and Purpose Statement
Management Problem
Achieving the desired benefits of Quality Management System implementation is reported to occur in only one-third to one-half of organizational attempts (Burdett, 1994; Garvin, 1986; Grant, Shani, and Krishnan, 1994; Hindo, 2007; Yandrick, 1994; Naor, 2006). Despite 11
such a deficient record of success, managerial leadership constantly spends considerable energy and investment maintaining the current QMS or chasing an alternative perceived superior. Consistently, implementation failures are blamed either on the lack of a cultural change for quality in the target organization or the choice of an inadequate QMS that influences the efforts at implementation. Assessing how cultural context and the choice of a quality management system relates to successful implementation will help management direct precious resources to their best use within the organization.
Purpose Statement
The purpose of this research is to investigate how senior management’s perception of congruency, between an organization’s current cultural characteristics and the organization’s current Quality Management System, correlates with the benefits achieved from implementing the Quality Management System.
Relevance to the Study of Management
Academic Perspective
Consistently missing from QMS implementation discussions is consideration that working with the culture that already exists may be more beneficial than detrimental to the ultimate goal of profitably delivering quality products and services to customers (Darling, 1992; Schroeder, Linderman, & Zhang, 2005; Taylor & Wright, 2003; Calingo, 1996; Morgan & Piercy, 1998, Huq, 2005; Hoang, Igel, & Laosirihongthong, 2006; Lagrosen,S., Lagrosen,Y., 2003). One possible explanation for the lack of research in the area of QMS implementation within the existing culture may be that the preponderance of literature related to QM focuses on QMS implementation success at the manufacturing plant level. The focus at the manufacturing plant level tends to keep investigations confined to tactical issues. Senior managerial leadership 12
cultural perspectives on successful QMS implementations are scarce. This dissertation adds to the literature by exploring how Senior Management perceptions of congruency between current organizational culture and the organizational culture ideally are needed to support the organization’s current QMS correlates with benefits associated with QM implementation.
Practitioner’s Perspective
Organizational leaders and quality practitioners need to optimize resources in their attempts to provide quality products and services to customers. Inappropriate attempts to change culture detract from the efficient and effective use of resources. This study provides management with a better understanding of the contextual demands culture makes on successful QMS implementation.
In addition, inappropriate selection of a QMS underpinning the organization’s implementation effort, affects the efficient and effective use of resources. If the current organizational culture is not congruent with the type of QMS implemented (i.e. ISO 9000, Malcolm Baldridge, Total Quality Management, Six Sig