Logo Utrecht University

Coloring Book: Testing Language Comprehension

Home

A NEW, INNOVATIVE METHOD FOR ASSESSING LANGUAGE COMPREHENSION THROUGH A PLAYFUL DIGITAL GAME

 

The Coloring Book method has the goal of assessing language comprehension, mainly of young children (aged 4 to 7). It can test both syntax and lexicon, in children with and without language acquisition problems. Furthermore, it can be used for both monolingual and multilingual children. Previous research has shown it is an ecologically valid method, and children who participated in these studies have indicated that they found it fun to work with.

The Coloring Book method has emerged
from the need for an ecologically valid method
to test implicit linguistic knowledge

Pages from the Coloring Book. Left: monkey is being scratched by another monkey. Right: scene of a farm; the tractor is colored red. © Coloring Book project

In linguistic research, we want to determine the most intuitive interpretation that a subject assigns to a word or to a sentence. Actually, this is precisely what happens in real life. While reading a text or listening to speech, we automatically assign meaning to the linguistic stimulus offered to us. How to unveil that piece of information in an experimental setting? On the one hand, we want to recreate a natural situation (ecological validity). On the other, we want to have an objective measure of the phenomenon under examination.

©Aslan Alphan

Language comprehension is normally assessed by using the so-called pointing task (Picture Selection Task). Subjects receive an auditory or written prompt and show their interpretation by pointing to a picture out of a set of three or four that best represents their intended meaning. Many tools for assessing language proficiency (both in first and second language acquisition, as well as in clinical linguistic research) are based on the pointing-task method. This method, however, is not sufficiently precise for testing subjects’ first intuitions (implicit knowledge of grammar). In addition, having to choose out of some explicit alternatives may induce subjects to guess and provide chance answers.

The Coloring Book Method stems from two crucial prerequisites:

  • It must tap the first intuition that comes to mind (not the most accepted out of a set of possible answers);
  • It must be both ecologically valid and scientifically measurable.

The idea is simple: Instead of pointing to the correct picture, subjects color the correct item on a coloring plate. The prompt that the subject hears contains coloring instructions.

For instance, the red monkey is being scratched by the green monkey (for assessment of passive voice), and the tractor is red (for assessment of receptive vocabulary). The subject then colors the item in the coloring plate that they thinks corresponds to the word or to the construction in the prompt. Note that each coloring plate contains several items (for vocabulary assessment up to 10-12 different items) all placed in a natural context (a classroom, a farm, a birthday party, etc.), so that it is most ecologically valid, guessing is minimized and word and sentence recognition reflects the way they have been learned (i.e., in context).

Language research

The Coloring Book method has now been used in a number of studies (vocabulary, syntax, morphology, semantics and discourse pragmatics) and with a large variety of populations (first and second language learners, atypical language learners, adults and children). This new method has proven to be particularly successful with young children. The playful format and the familiar activity of coloring has a great appeal on children and it enables experimenters to collect objective, ecologically valid and – most importantly – novel data on language comprehension.

Education tools

© KleurenSchat

The positive responses to the Coloring Book Method led us to the creation of KleurenSchat, a new observation tool for receptive vocabulary in Dutch. KleurenSchat helps teachers and education professionals to observe and to measure vocabulary development in Dutch in preschoolers and first and second graders. KleurenSchat has been developed in collaboration with Boom Uitgevers Amsterdam.

You must be logged in to post a comment.