Education And Language

Construct Validity And Reliability Of Outdoor Education Evaluation Instrument

Assoc. Prof Dr Ahmad Hashim, Alvin Alvin Raj,
Article Date Published : 23 May 2018 | Page No.: EL-2018-341-344 | Google Scholar

Downloads

Download data is not yet available.

Abstract

This research aims to obtain the validity and reliability of a survey instrument to evaluate outdoor education. It is a survey which is divided into three components namely first; strategic knowledge, second; efficient behaviour and third; affective. Development of the survey instrument is based on Tennessee Self Concept Questionnaire, Multifactor Leadership Questionnaire (MLQ), Participant Motivation Questionnaire (PMQ), Cooper Smith Inventory, Group Environment Questionnaire (GEQ) and Outdoor Education Manual (MPL). There are 95 items prepared based on the five-point Likert scale which were tested at the pilot test level. It involved 100 research subjects aged between 15 and 16 years old who were selected based on purposive random sampling.  In addition, the research method applied is in the form of pre-experimental one group pre-test–post-test. Results of the analysis showed that overall field expert validity is r = .89 and language expert validity is r = .90, while the Cronbach alpha reliability correlation value of outdoor education instrument evaluation survey is  r = .89. Next, this survey was tested again for construct validity using the factor analysis method for statistical analysis  which would validate each item tested was supposed to be in the right component.  From the analysis, results show that Bartlett’s test is significant p < .05 and Kaiser-Meyer-Olkin index range is r = .62. 81.16 percent variant is explained from  18 components which have been analyzed with more than one eigenvalue. 51 survey items are produced out of 95 items of the survey based on this factor analysis method. Research has shown that the survey instrument developed  is valid and reliable to be used for Sabah’s Co-curricular Centre.


 

Introduction

Co-curricular centre is a training hub which is responsible for handling planned and systematic outdoor education activities guided by Co-curricular centre curriculum that is drafted by the Malaysian Ministry of Education. The aim of State Co-curricular centre is to provide learning experiences through actual activities while emphasizing the concept of knowledge enhancement, skills, disciplines, self-confidence and reinforcement of moral values which contribute to an individual’s success. This study aims to increase quality of outcome as well as the quality of outdoor education. Some of the importance of this research are it could provide feedbacks about students’ achievements and the acquisition of skills, knowledge along with social values from the aspect of affective domain, strategic knowledge and efficient behaviour. It is useful to the Sabah co-curricular centre coach who can make the outdoor education instrument as guidelines in assessing students’ skills, knowledge and social values. In addition, results of this study are expected to contribute to Sabah co-curricular centre which offers outdoor education activities in order to monitor. The centre also evaluate outdoor education camping program activities to find out whether or not they need adjustments so that its execution can be improved to find its effectiveness on the students. It is also hoped that the research findings are able to provide encouragement to the Ministry of Education and State Education Department in planning, arranging and improving outdoor education program activities.

Outdoor education organized by Sabah co-curricular centre is gaining positive responses among the schools in Sabah. Laporan Tahunan Pusat Kokurikulum, Sabah ( 2012), indicated positive results towards motivation, group integration, self-concept and generic development among the participants who participated in the programs at the centre. Nevertheless, the report did not state the performance of participants’ strategic knowledge, efficient behaviour and affective domain by using a valid instrument, systematic and consistent in measuring the participants’ performance which covers the outdoor education program at the co-curricular centre. Until there is a development of effective instrument in the aspect of validity and reliability, the evaluation cannot be carried Out validly and accurately. This view is supported ( Ahmad Hashim, 2004), which stated that an instrument which is credible is not necessarily valid. Other than that, up to now, there is still no outdoor education instrument which is complete and accurate as well as able to measure the three domains namely affective, strategic knowledge and efficient behaviour which can be used by the co-curricular centres and the Malaysian Ministry of Education to study the effects of the program conducted. If this issue is not overcome, the aim and objective of national education philosophy that stresses on JERIS is not going to be attained ( Kementerian Pendidikan Malaysia, 1990).

Outdoor education assists in enhancing an individual’s social values such as one’s dignity, integration, motivation self-concept and confidence (Brewing, O'Connell, Todd, Anderson, & Young, 2010). Through these outdoor education activities, efficient behaviour domain can be achieved and applied in any outdoor education. Affective domain is able to produce learners who are all-rounders and provide life experiences to them. Affective domain can also be practiced in their daily life. This is not merely education but it is a lifelong learning. The aspect of strategic knowledge also provides positive impact as well as offers new knowledge in the learning process. Learning from the aspect of strategic knowledge is not limited to mastering facts but learners have to understand and internalize strategic knowledge skills so that they could analyse and solve problems.

Research Objective

The objective of this research is to find the outdoor education camping instrument through several change factors that are related to the learners who completely participated in the camping activity. In order to attain this goal, the researchers have underlined several research objectives such as, determining the instrument validity based on field and language experts, outdoor education instrument’s reliability based on pilot test and validity of outdoor education instrument isolated construct based on pilot test.

Methodology

This research aims to evaluate the validity and reliability of outdoor education survey instrument. In order to achieve this objective, a research is conducted in the form of pre-experimental method for one group pre-test–post-test. Ary, Jacobs, & Razavieh ( 2002) explained that one group pre-test–post-test research is used to make comparisons between pre and post-test scores after an intervention is conducted on a certain group before it is tested again during post-test.

3.1 Subject

100 students (50 males and 50 females) aged between 15 and 16 years old, were selected based on purposive random sampling to participate in this research at Perak Co-curricular Centre.

3.2 Test Instrument

The researcher has adapted and changed several questionnaires taken from other studies in order to develop the questionnaire for aspects of strategic knowledge, efficient behaviour and affective domain such as the survey on Tennessee self-concept, multifactor leadership questionnaire, participant motivation questionnaire Cooper Smith inventory and group environment questionnaire.. As for outdoor education is used to develop a questionnaire for strategic knowledge and affective domains. All of these questionnaires are tested by using the five-point scale. 95 items for the questionnaire are developed to be tested during the outdoor education camping. The amount of questionnaire items for strategic knowledge component is 18, efficient behaviour has 27 items and affective is given 50 items.

3.3 Analysis

For data collection, the researcher uses SPSS version 17.00. Some of the analysis used are field and language experts validity analysis in order to enable every item asked fulfil the criteria required and focused on the right component. Next, Cronbach Alpha statistic is used to obtain reliability while Isolated construct validity is used with factor analysis method. Extraction technique applied in this analysis is Principal Components Analysis. Orthogonal rotation factor which is the varimax rotation method is also used. This factor analysis method is carried out according to three main steps. Firstly, it involves evaluating suitable data for factor analysis method. Secondly, extraction factor and thirdly , rotation factor and interpretation are applied. To evaluate whether the data is suitable, the researcher has to examine the correlation coefficient matrix values 0.3 and above. Bartlett’s test (Barlett’s test of sphericity) is used to see whether the sample used is sufficient. Factor analysis method is suitable to be carried out when the results from Bartlett’s test is significant (p <.05) and Kaiser-Meyer-Olkin (KMO) index range is in between 0 to 1 with r =.3 is suggested as the minimum value for a good factor analysis.

Research Findings

Results from the analysis of field experts correlation validity value for outdoor education survey instrument from expert one is .83, expert two is .92 and expert three is .9. As for the whole expert validity, it was at r = .89. According to Sidek Mohd Noah ( 2000) and Tuckman and Waheed ( 1981) the value r = .70 was considered as mastery or attainment of the highest achievement level. As for analysis of language expert correlation validity value for outdoor education survey instrument, the details were as follows; language expert one, r = .92, language expert two, r = .88 and the third expert, r = .90. Total value of reliability for language expert validity was r = .90. In addition, total value of Cronbach Alpha for strategic knowledge component was r = .78, efficient behavior r = .86 and affective r = .91. The whole outdoor education survey instrument had the Cronbach Alpha correlation value of r =.89. This shows that the survey instrument which is adapted and developed can be used. However, factor analysis was conducted to ensure that each item which was developed were able to test the right component and non-repetitive. This research analysis showed that all questionnaire items in this study had correlation coefficient value of r =.3 and above. Table 1.0 featured KMO value obtained which was r = .62 and Bartlett’s test was significant (p =.000), therefore factor analysis was suitable to be used in the next test.

Table 1 Barlett’s and Kaiser-Meyer-Olkin test results

Kaiser-Meyer- Olkin Measure of Sampling Adequency .618
Barlett s Test of Sphericity Approx. Chi SquareDfSig. 11386.2664465.000

Kaiser’s criterion technique was used to determine the number of components. Components with only eigenvalue one or more were selected in this analysis. There were 18 analysis components which had more than one eigenvalue: all these 18 analysis component explained 81.16 per cent variance. Matrix component showed loading in each line expressed each survey item’s correlation with strategic knowledge, efficient behaviour and affective domains.

In order to maintain three components for the next analysis, the researchers used varimax rotation method to minimize number of survey items which had high correlation on each factor. According to Tabachnick and Fidell ( 2007) results based on orthogonal rotation was easier to be translated and reported.

Table 2, showed results from the three components rotation using the varimax rotation method. Results showed that the first component explained 11.255 per cent of variance, the second component explained 10.927 per cent of variance and the third component explain 10.544 per cent variance. Total amount of variant available which could be explained by all three components was 66.16 per cent variance and remained after rotation.

Table 2 Total Variance Explained

Component Rotation Sums of Squared Loadings
Total % of Variance Cumulative %
1 10.692 11.255 11.255
2 10.380 10.927 22.182
3 10.016 10.544 32.726

Table 3 shows the loading for each survey item for strategic knowledge, efficient behaviour and affective domains. Based on Principal Component Analysis, from 95 survey items only 51 showed high communality score. Component line one, represented outdoor evaluation instrument which measures strategic knowledge, the second line represented outdoor evaluation instrument which measures affective domain and the third line represented outdoor education evaluation instrument which measured efficient behaviour.

Forty four from 95 survey items which consisted of strategic knowledge, efficient behaviour and affective components which were disqualified during the factor analysis processes . This was due to loading factor which was lower than .30 and there was no relation between survey item and the component.

Selection for construct component for strategic knowledge, efficient behaviour and affective domain in this study was based on high main loading and it exceeded the correlation coefficient value r = 0.50. This was because high correlation value of a test on a measured factor indicated close relation with the factor. According to Pallant ( 2005) based on this significant finding, 51 strategic knowledge, efficient behaviour and affective survey items in this analysis are accepted as valid for outdoor education survey items in this research.

Discussion

A certain test instrument must be measured by using several statistical method so that the test instrument which was measured was definitely consistent and reliable. This is supported by Baumgartner and Chung ( 2001) who stated that an instrument which has construct validity is a valid and reliable instrument and could be used on any population that is being tested. Having formed 51 out of 95 outdoor education evaluation survey instrument that is valid and reliable, it could now provide the right and accurate information and feedback to the Sabah Co-curricular Centre in order to make improvement for whatever that is lacking now and for the future. One of the common issues is the inability to provide accurate and correct feedback before from the camping activity organized.

Conclusion

The aim of this research is to obtain validity and reliability of outdoor education evaluation instrument. Research has shown that the questionnaire developed is valid and reliable. This indicates that the questionnaire which is constructed can be used to measure the implication and effects of outdoor education on learners

Table 3 Construct Validity for Rotated Component Matrix Component

Item Component
Strategic knowledge (1) Affective 2 Efficient behavior (3)
S63 .835
S68 .812
S64 .808
S62 .793
S61 .791
S67 .786
S59 .757
S65 .741
S60 .730
S66 .718
S9 -.654
S6 -.645
S5 -.607
S7 -.565
S10 -.553
S11 -.546
S12 -.546
S8 -.540
S43 .672
S42 .671
S46 .644
S44 .628
S47 .609
S39 .592
S22 .587
S41 .581
S27 .580
S20 .565
S40 .548
S45 .548
S29 .525
S30 .522
S23 .516
S19 .507
S48 .502
S31 .500
S79 .834
S70 .816
S83 .801
S81 .799
S78 .758
S74 .755
S82 .744
S80 .738
S75 .720
S71 .708
S76 .699
S69 .679
S72 .658
S77 .658
S73 .654

References

  1. Book Reviews: Cardiovascular Pharmacotherapeutics Manual, 2nd EditionCardiovascular Pharmacotherapeutics Manual, 2nd Edition Edited by FrishmanWilliam H MD, SonnenblickEdmund H MD, and SicaDomenic A MD. Published by McGraw-Hill Medical Publishing Division, New York, NY, 2004. ISBN 0-07-137363-2. Paperbound, xi $\mathplus$ 802 pp. (21 × 12 cm), $39.95. www.accessMedBooks.com Nappi JeanM. Annals of Pharmacotherapy.2004-oct;:1754-1754. CrossRef Google Scholar
  2. New York Amish, 2nd Edition Johnson-Weiner Karen. .2018. CrossRef Google Scholar
  3. Multivariate Analysis of Variance (MANOVA) Tabachnick BarbaraG, Fidell LindaS. International Encyclopedia of Statistical Science.2011;:902-904. CrossRef Google Scholar
  4. Seroprevalence of Bovine Viral Diarrhea Virus Infection and Associated Risk Factors in Cattle in Selangor, Malaysia Daves Larry, Yimer andNurhusien, Arshad SitiSuri, Sarsaifi Kazhal, Omar MohdAriff, Yusoff Rosnina, Haron AbdWahid, Abdullah FaezFirdausJesse, and and and. Veterinary Medicine - Open Journal.2016-jul;:22-28. CrossRef Google Scholar
  5. Evaluating an individualized science program for community college students Tuckman BruceW, Waheed MohammedA. Journal of Research in Science Teaching.1981-nov;:489-495. CrossRef Google Scholar
  6. Book Review: Timothy A. Brown. (2006). Confirmatory factor analysis for applied research. New York: Guilford Edwards BryanD. Organizational Research Methods.2008-aug;:214-217. CrossRef Google Scholar
  7. Exploratory Factor Analysis Methods of Multivariate Analysis.2012;:435-477. CrossRef Google Scholar
  8. Development and validation of scales to measure environmental responsibility, character development, and attitudes toward school Powell RobertBaxter, Stern MarcJ, Krohn BrianD, Ardoin Nicole. Environmental Education Research.2011-feb;:91-111. CrossRef Google Scholar
  9. Confidence Limits for Intraclass Reliability Coefficients Baumgartner TedA, Chung Hyuk. Measurement in Physical Education and Exercise Science.2001-sep;:179-188. CrossRef Google Scholar

Author's Affiliation

Copyrights & License

International Journal of Scientific Research and Management, 2018.
Creative Commons License

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.

Article Details


Issue: Vol. 6 No. 05 (2018)
Page No.: EL-2018-341-344
Section: Education And Language
DOI: https://doi.org/10.18535/ijsrm/v6i5.el05

How to Cite

Hashim, A. P. D. A., & Alvin Raj, A. (2018). Construct Validity And Reliability Of Outdoor Education Evaluation Instrument. International Journal of Scientific Research and Management, 6(05), EL-2018. https://doi.org/10.18535/ijsrm/v6i5.el05

Download Citation

  • HTML Viewed - 87 Times
  • PDF Downloaded - 60 Times
  • XML Downloaded - 19 Times