Sunday, 8 January 2017

Corporal Punishment in Our Schools

Every now and then there is a news of brutal punishment in a school. The innocent children get their bones broken, skin torn and nerves shattered by their so called teachers. 

National Education Assessment System (NEAS) through its study NAT asked the question of giving corporal punishment to the students, for which the answers were shocking. Almost 60% of the students reported about receiving such punishment. It is not the figure gathered through few hundred students survey but it is result of a study over 12000 students.
 
Several other surveys present similar reports where considerable number of teachers and even the parents believe that corporal punishment is necessary for teaching. 


The reason behind such thinking seems to be the teachers' own experiences during their own school time. The reports that come out on media are of extreme cases but there are many who never report such things out of so called respect and mostly because of the fear of vendetta. 

It has to end somewhere. A great number of students don't like their school and feel scared of going to the school for the same reason. Only law making is not enough but there should be some mechanism to stop all these scary teaching practices. If such practices are not stopped in any way, we will keep witnessing the same stories from our schools. 

   

Tuesday, 15 November 2016

Selection of Samples of Schools and Students for the National Achievement Testing; NAT-2016

In National Achievement Testing NAT, (Mathematics, Urdu reading & writing, Science, and English reading & writing) a sample of schools and students is selected to represent all strata of the country. This year 30,000 students from 1500 schools are to be assessed for learning achievement on Grade 4 and Grade 8 levels.

The selection process for schools uses stratified random sampling method with urban, rural, male and female population from the concerned grade. For NAT-2016 following sample has be drawn:
Schools Sample for NAT-2016
Province / Area
Sample Schools
Grade 4
Grade 8
Total
Azad Jammu & Kashmir
51
55
106
Balochistan
68
61
129
FATA
51
38
89
Gilgit-Baltistan
27
30
57
Islamabad Capital Territory
32
38
70
Khyber Pakhtunkhwa
145
147
292
Punjab
224
244
468
Sindh
152
137
289
PAKISTAN
750
750
1500

For the national achievement testing, 20 students are assessed from each sample school. In this way the target students are 30,000. Selection of these students is done by using Random Number Table, where all the students have equal chance of getting selected. By using the random stratified sample method for sampling, the schools and students are selected to represent the diverse student population in Pakistan. 

Wednesday, 9 November 2016

Test Administrators Training at Gilgit-Baltistan


Just before the large scale assessment National Achievement Testing, NAT-2016, one of the most important activities is training of the test administrators. 

The main purpose of TAs training is to ensure standardized procedures of training and assessment so that validity of the procedures is ensured. Right training of these administrators ensures the smooth conduct of national assessment that is conducted simultaneously in many regions. 


In NAT-2016, there is a plan to train 3300 test administrators across the country for 1500 schools. For this purpose Lead Master trainers were already trained who were expected to train the actual test administrators in their respective areas. 

33 participants attended the sessions where there were 27 males and 6 females. These invited test administrators were the teachers from different schools. 

After recitation by a participant, Mr. Faisal Shakir informed the participants about the purpose of the workshop. Mr. Suhail Bin Aziz, Test Development Specialist, NEAS discussed the role of NEAS throughout Pakistan for uplifting the quality of education by diagnosing the better and weak practices. 

The Director Education, Mr. Faizullah Lone, thanked NEAS for arranging the program in such organized manner and also for sending the representative from Islamabad to ensure smooth conduct of the training activities. He also assured of his complete assistance for this national cause. Mr. Lone requested to pursue the case of their seats, pending in Finance Department.  
  
The resource persons were the Lead Master Trainers who were trained at NEAS  in the month of June 2016. Mr. Faisal and Mr. Ismail conducted the sessions skillfully. 

All the planned sessions were conducted as per program. The participants were allowed to ask any question so that no ambiguity remain before going into the field. On first day, the Test Material was shown, the manual was discussed in detail and the sample questions from all four subjects were also shared with the participants. 

On the second day, the session started with recitation from the Holy Quran. Later the Random Number Table was discussed for random sampling within the school to identify 20 students for assessment purpose. The participants had practice of filling the random number table and attendance sheets. 

After the practical session, all three background questionnaires (student’s & parents’ questionnaires, teacher’s questionnaires, and head teacher’s questionnaires) were discussed in detail and the participants also gave their feedback to improve the quality of the documents. 

Role play was another important session where the participants had hands on practice of two days assessment activities in school. The participants performed the roles in head teacher’s office, in the assessment room, filling up the TA manual and receiving/ dispatching assessment material. These role plays provided a nice visual guide for all the participants. 

For closing session, Haji Shahid Hussain, the President of Gilgit-Baltistan Teachers Association and Mr. Johar Nafees, Secretary General of the association were invited as the guests of honour. Haji Shahid Hussain addressed the audience and shared his feelings on seeing such a well-organized session. He compared the session with those conducted by different Non-governmental organizations including British Council, GIZ and UNICEF and termed the present activity much more organized and impressive. He also assured of his full cooperation towards the conduct of assessment activities in the whole region. 

Some participants also shared their feelings and ideas about the workshop. They expressed their satisfaction over the conduct of the training session and hoped that it would help them in conducting the assessment activities as per the given directions. 

Mr. Suhail Bin Aziz from NEAS Islamabad thanked the administration and participants for their keen interest in all the activities.  
 
At the end the assessment material and stationary packs were distributed among the participants and group photographs were taken.  

Smooth and well managed activities of the training session proves the level of seriousness on the part of Mr. Faisal Shakir who kept in touch during all arrangements. It is heartening to see such hardworking fellows, working restlessly for a national cause. 

It is hoped that the assessment activities in Gilgit-Baltistan will be conducted in a smooth manner and without any problems. 

Tuesday, 18 October 2016

What is TIMSS? Useful information on TIMSS 2015


TIMSS 2015 in Brief
The Trends in International Mathematics and Science Study (TIMSS) is an international comparative study of student achievement. TIMSS 2015 represents the sixth such study since TIMSS was first conducted in 1995. Developed and implemented at the international level by the International Association for the Evaluation of Educational Achievement (IEA), an international organization of national research institutions and governmental research agencies, TIMSS is used to measure the mathematics and science knowledge and skills of 4th- and 8th-graders over time.
TIMSS is designed to align broadly with mathematics and science curricula in the participating countries and education systems. The results, therefore, suggest the degree to which students have learned the mathematics and science concepts and skills likely to have been taught in school. TIMSS also collects background information on students, teachers, schools, curricula, and official education policies to allow cross-national comparisons of educational contexts that may be related to student achievement.

Participation
In 2015, TIMSS was administered in 49 IEA member countries and 6 other education systems at grade 4, and in 38 IEA member countries and 6 other education systems at grade 8.1
IEA member countries include both "countries," which are complete, independent political entities and "other education systems," or non-national entities (e.g., the Flemish community of Belgium, England). Non-national entities that are not IEA member countries (i.e., Florida, Abu Dhabi) are designated as "benchmarking participants." "Other education systems" are indicated in the tables and figures with the three-letter international abbreviation for their nation following their name (e.g., England-GBR, Florida-USA). (For more information on terminology, see the Textbox below.) For convenience, the generic term "education systems" is used when summarizing across results.
For a number of education systems, changes in achievement can be documented over the last 20 years, from 1995 to 2015. For those that began participating in TIMSS data collections after 1995, changes can be documented over a shorter period of time. The TIMSS participation table shows the education systems that participated in TIMSS 2015 as well as their participation status in the earlier TIMSS data collections. The TIMSS 4th-grade assessment was implemented in five TIMSS cycles (1995, 2003, 2007, 2011, and 2015), while the 8th-grade assessment was implemented in all six cycles (including 1999).
Countries or Education Systems?
The international bodies that coordinate international assessments vary in the labels they apply to participating entities. For example, the IEA, which coordinates TIMSS and the Progress in International Reading Literacy Study (PIRLS), differentiates between IEA members, which the IEA refers to as "countries" in all cases, and "benchmarking participants." IEA members include countries such as the United States and Japan, as well as subnational entities, such as England and Northern Ireland (which are both part of the United Kingdom); the Flemish community of Belgium; Hong Kong, which is a Special Administrative Region of China; and Chinese Taipei, which is also a part of China. IEA benchmarking participants—typically all subnational entities—included the U.S. state of Florida and sub-national entities from five other countries in 2015. The Organization for Economic Cooperation and Development (OECD), which coordinates the Program for International Student Assessment (PISA), differentiates between OECD member countries and all other participating entities (called "partner countries" or "partner economies"), which include countries and subnational entities. In PISA, the United Kingdom and Belgium are reported as whole countries.


The TIMSS Mathematics and Science Assessments
The TIMSS Mathematics Assessment
The TIMSS mathematics assessment is organized around two dimensions: (1) a content dimension specifying the subject matter to be assessed and (2) a cognitive dimension specifying the cognitive or thinking processes to be assessed. At grade 4, TIMSS assesses student knowledge in three content domains: number, geometric shapes and measures, and data display. At grade 8, TIMSS assesses student knowledge in four content domains: number, algebra, geometry, and data and chance. At both grades (and across all content domains), TIMSS assesses students' mathematical thinking in three cognitive domains: knowing, applying, and reasoning.
The proportion of item score points devoted to a content domain—and, therefore, the contribution of the content domain to the overall mathematics scale score—differs somewhat across grades (see the table below). For example, in 2015, 52 percent of the TIMSS mathematics assessment at grade 4 focused on the number content domain, while the analogous percentage at grade 8 was 31 percent. The proportion of items devoted to each cognitive domain was similar across grades.
In order to give students the best opportunity to operate in settings that mirror their classroom experience, TIMSS permits calculator use at the eighth grade (as it has since 2003). Thus, in education systems in which calculators are typically used in classroom activities, students can be encouraged to use them on TIMSS. However, if calculators are not typically used or permitted in the classroom, an education system need not permit their use on TIMSS.
The TIMSS Science Assessment
The TIMSS science assessment is similarly organized around two dimensions: (1) a content dimension specifying the subject matter to be assessed and (2) a cognitive dimension specifying the cognitive or thinking processes to be assessed. At grade 4, TIMSS assesses student knowledge in three content domains: life science, physical science, and Earth science. At grade 8, TIMSS assesses student knowledge in four content domains: biology, chemistry, physics, and Earth science. At both grades (and across all content domains), TIMSS assesses students' scientific thinking in three cognitive domains: knowing, applying, and reasoning.
The proportion of item score points devoted to a content domain—and, therefore, the contribution of the content domain to the overall science scale score—differs somewhat across grades (see the table below). For example, in 2015, 46 percent of the TIMSS science assessment at grade 4 focused on the life science domain, while the analogous percentage at grade 8 was 36 percent. The proportion of items devoted to each cognitive domain was similar across grades.
For more detailed descriptions of the range of content and cognitive domains assessed in TIMSS, see the TIMSS 2015 Assessment Frameworks. The development and validation of the mathematics cognitive domains are detailed in IEA's TIMSS 2003 International Report on Achievement in the Mathematics Cognitive Domains: Findings From a Developmental Project.
Table A. Percentage of TIMSS mathematics and science assessment score points devoted to content and cognitive domains, by grade: 2015
Mathematics content and cognitive domains
Grade 4
Grade 8
Content domains
Percent of 
assessment
Content domains
Percent of 
assessment
Number
52
Number
31
Geometric shapes and measures
32
Algebra
28
Data display
15
Geometry
21


Data and chance
21
Cognitive domains
Percent of 
assessment
Cognitive domains
Percent of 
assessment
Knowing
36
Knowing
31
Applying
44
Applying
45
Reasoning
20
Reasoning
24
Science content and cognitive domains
Grade 4
Grade 8
Content domains
Percent of 
assessment
Content domains
Percent of 
assessment
Life science
46
Biology
36
Physical science
35
Chemistry
19
Earth science
19
Physics
24


Earth science
21
Cognitive domains
Percent of 
assessment
Cognitive domains
Percent of 
assessment
Knowing
41
Knowing
36
Applying
38
Applying
41
Reasoning
21
Reasoning
23
NOTE: The percentages in this table are based on the number of score points, not the number of items. The number of score points and the number of items are not the same because some constructed-response items are worth more than one score point. (For the corresponding percentages based on the number of items, see the Technical Notes [forthcoming].) The content domains define the specific mathematics and science subject matter covered by the assessment, and the cognitive domains define the sets of thinking processes students are likely to use as they engage with the respective subject's content. Each content domain has several topic areas. Each topic area is presented as a list of objectives covered in a majority of participating education systems, at either grade 4 or 8. However, the cognitive domains of mathematics and science are defined by the same three sets of expected processing behaviors—knowing, applying, and reasoning. Detail may not sum to totals because of rounding.
SOURCE: Martin, M.O., Mullis, I.V.S., and Hooper, M. (Eds.). (2016).
 Methods and Procedures in TIMSS 2015. Chestnut Hill, MA: TIMSS & PIRLS International Study Center, Boston College.

Item Format
In both the mathematics and science assessments, items vary in terms of item format (see the table below) and item difficulty. They also differ across grade levels to reflect the nature, difficulty, and emphasis of the subject matter encountered in school at each grade.
Table B. Percentage of TIMSS mathematics and science assessment score points, by grade and item format: 2015
Item format
Mathematics
Science
Grade 4
Grade 8
Grade 4
Grade 8
Multiple choice
49
51
52
52
Constructed response
51
49
48
48
NOTE: The percentages in this table are based on the number of score points, not the number of items. The number of score points and the number of items are not the same because some constructed-response items are worth more than one score point. (For the corresponding percentages based on the number of items, see the Methodology and Technical Notes. Multiple-choice items require students to select the single correct response from four response options. Constructed-response items require students to generate a written response. 
SOURCE: Martin, M.O., Mullis, I.V.S., and Hooper, M. (Eds.). (2016).
 Methods and Procedures in TIMSS 2015. Chestnut Hill, MA: TIMSS & PIRLS International Study Center, Boston College.

Methods in Brief

TIMSS and TIMSS Advanced are sample-based assessments, meaning that only a sample of students take the assessments but that they are selected in such a way as to allow the results to be generalizable to a larger target population. The TIMSS and TIMSS Advanced target populations are based on standardized definitions, and the sampling is conducted based on standardized and refereed procedures.
TIMSS required participating countries and other education systems to draw probability samples of students who were nearing the end of their fourth or eighth year of formal schooling, counting from the first year of the International Standard Classification of Education (ISCED) Level 1.3 In the United States, one sample was drawn to represent the nation at grade 4 and another at grade 8.4 In addition to these two national samples, separate state public school samples were drawn for Florida at both grades in order to benchmark that state's student performance internationally.
TIMSS Advanced required participating countries and other education systems to draw samples of students in their final year of secondary school—International Standard Classification of Education (ISCED) Level 3—who have taken courses in advanced mathematics or who have taken courses in physics. In the United States, two samples were drawn to represent the nation—one for advanced mathematics and one for physics.5 The courses that define the target populations have to cover most, if not all, of the advanced mathematics and physics topics that were outlined in the TIMSS Advanced 2015 Assessment Frameworks. In the United States, this was defined as a calculus course for eligibility for advanced mathematics and an advanced physics course similar to AP physics for physics.
For additional details on the methods used in TIMSS and TIMSS Advanced 2015, see the Methodology and Technical Notes.

Reporting in Brief
TIMSS and TIMSS Advanced achievement results are reported on a scale from 0 to 1,000, with a TIMSS scale average of 500 and standard deviation of 100. TIMSS provides an overall mathematics scale score and an overall science scale score, as well as subscale scores for the content and cognitive domains in each subject at each grade level. TIMSS Advanced provides an overall advanced mathematics scale score and an overall physics scale score, as well as subscale scores for the content and cognitive domains in each subject.
The scaling of data is conducted separately for each subject and grade. Data are also scaled separately for each of the content and cognitive domains. Because the level of difficulty of items necessarily differs between subject, grade, and domains, direct comparisons between scores across subjects, grades, and different domain types should not be made. In contrast, scores within a subject, grade, and domain are comparable over time.
In addition to scale scores, TIMSS and TIMSS Advanced have developed international benchmarks for each subject and grade. The international benchmarks provide a way to interpret the scale scores and to understand how students' proficiency in mathematics and science varies along the TIMSS scale. TheTIMSS benchmarks describe four levels of student achievement (Advanced, High, Intermediate, andLow) for each subject and grade, based on the kinds of skills and knowledge that students at each score cutpoint would need to successfully answer the mathematics and science items. The TIMSS Advanced benchmarks similarly describe three levels of student achievement (Advanced, High, and Intermediate).
The TIMSS 2015 and TIMSS Advanced 2015 Results present the performance of U.S. students relative to their peers in other countries and other education systems and describe changes in mathematics and science achievement since 1995. Most of the TIMSS 2015 findings are based on the results presented in two international reports published by the IEA (forthcoming):
Most of the TIMSS Advanced 2015 findings are based on the results presented in the international report published by the IEA (forthcoming):
Besides findings based on the international reports, the TIMSS 2015 and TIMSS Advanced 2015 Results provide details about the achievement of subgroups of U.S. students that are not available in the international reports (e.g., scores of students of different racial and ethnic and socioeconomic backgrounds).
It is important to note that comparisons presented here treat all participating education systems equally, as is done in the international reports. Thus, the United States is compared with some education systems that participated in TIMSS without a complete national sample (e.g., Northern Ireland-GBR participated but there was no national United Kingdom sample) as well as with some education systems that participated as part of a complete national sample (e.g., Florida-USA participated as a separate state sample of public schools and as part of the U.S. national sample of all schools).
In addition to describing performance in 2015, the results also document changes in mathematics and science over time. TIMSS has been administered six times (every 4 years) since the first assessment in 1995. In each administration, the framework is reviewed and updated to reflect developments in the field and in curricula, while at the same time ensuring comparability in sampling procedures and assessment items. Additionally, each successive administration of TIMSS since 1995 has been scaled so that the mean of all education systems is 500, as it was originally set in 1995, and thus comparable across years. This report focuses on comparing the 2015 results with those from the prior TIMSS assessment in 2011 and, for a long-term perspective, the first TIMSS assessment in 1995.
Changes in advanced mathematics and physics achievement between 1995 and 2015 are reported to the extent possible. Six countries, including the United States, participated in TIMSS Advanced in both years. However, because of changes in the framework and sampling procedures between the 1995 and 2008/2015 administrations, results should be interpreted with caution, as described in the Methodology and Technical Notes. Performance changes between 2008 and 2015 (which are not subject to such concerns about comparability) are not presented because the United States did not participate in TIMSS Advanced in 2008.

All results are presented in tables, figures, and brief text summaries of key findings. In the interest of brevity, in most cases, only the names of education systems (including benchmarking participants) scoring higher than or not measurably different from the United States (not those scoring lower than the United States) are reported. Results also include data on the one U.S. state—Florida—that participated as a benchmarking education system.
All statistically significant differences described in this report are at the .05 level. Differences that are statistically significant are discussed using comparative terms such as "higher" and "lower." Differences that are not statistically significant are either not discussed or referred to as "not measurably different." In almost all instances, the tests for significance used were standard t tests. No adjustments were made for multiple comparisons.
For additional information on scaling, reporting, and statistical procedures, see the Methodology and Technical Notes.