Abstract: Die Erfahrungen der Vergangenheit zeigen, dass internationale Indizes, auch wenn sie sich auf einige wenige zentrale Indikatoren stützen, ein sinnvolles Instrument sein können, Entwicklungserfordernisse sowie Erfolge aufzuzeigen und einen konstruktiven Wettbewerb zwischen Ländern anzuregen. Der Erfolg internationaler empirischer Studien, wie PISA oder Indizes wie der „Education For All (EFA) Development Index (EDI)“ der UNESCO, wirft die berechtigte Frage auf, ob ein solches empirisches Instrument im Bereich der Kulturellen Bildung auch mehr internationalen Wettbewerb in der Kulturellen Bildung hervorrufen könnte.

Der Artikel fasst die Hintergründe für die Entwicklung eines solchen international-vergleichenden empirischen Forschungsansatzes, mit dem Arbeitstitel „Arts Education Development Index“ (AEDI), zusammen und zeigt die Notwendigkeit der Nachbesserung sowie mögliche künftige Nutzen des AEDI anhand der Ergebnisse eines ersten internationalen Pretests in acht Ländern auf fünf Kontinenten auf. Der Forschungsansatz wurde von Prof. Dr. Susanne Keuchel, Prof. Dr. Eckart Liebau und Dr. Ernst Wagner in Zusammenarbeit mit Fachleuten des internationalen Netzwerks zur Förderung der Forschung auf dem Gebiet der Kulturellen Bildung INREA (International Network for Research in Arts Education) 2012 zur Unterstützung der Umsetzung der Seoul Agenda entwickelt.

Arts Education Development Index (AEDI) – Aims and Background for the conception

It is extremely difficult to compare national arts education activities on international level. This difficulty arises in part as a result of different institutionalised structures of arts education. This can be illustrated for example by the international music school statistics provided by the European Music School Union. As the Union points out, its data mainly relates to the public, extracurricular music schools, whereas generally neither private institutions nor freelance music educators are included in the statistics (see European Music School Union 2011:13f.). So it is not possible to present a complete picture of music activities generally in countries such as Germany with a higher amount of private music schools and freelance music educators or England, where music services usually don’t take place outside schools but are included in school offers.

Adding to the difficulty are different national school and education systems, such as half-time or all-day schools, single or multiple school systems – which affect the organisation of arts education in curricular and extracurricular contexts. If you look at the Eurydice Report (Education, Audiovisual and Culture Executive Agency 2009) for example, which focuses on schools and excludes non-formal arts education outside schools, problems of comparing different school systems in arts education become obvious: How is it possible to compare countries with an all-day-school system with countries with a half-day school and probably a few school arts educational offers, but a broad non-formal arts educational sector outside school?

Different national political systems of cultural funding further complicate international comparisons. The difficulties in comparing arts education "investment" within different political systems of cultural funding were recently shown in a national research project on arts education in Germany called mapping//arts education (Keuchel 2013a:220-236), in which several structural indicators were compared for 16 federal states in Germany.

This research approach was adapted to free researchers from the various institutions and allow them to work with the different 'fields of education', as defined by UNESCO: the formal, non-formal and informal education fields, which are the main focal points of AEDI. The simultaneous integration of all three fields enables international compatibility of various education systems.

Moreover, the informal education field is also important for an international comparison in arts education – especially under the presumption that in some countries cultural and artistic traditions are maintained intensely but possibly are not represented in formal or non-formal arts education services. It is also interesting to analyse whether there is a relation between the existence of formal and non-formal arts education services and arts education activities within the informal fields.

Methods and research design

Specific national statistics about arts education in the fields of education hardly exist. So it will be necessary to use several different sources such as analysis of the existing curriculum, existing statistics from national unions, population surveys or time-budget studies. Because of these different sources or as an alternative estimates, which are needed, there was the idea to work with an expert survey.

By a standardized questionnaire the experts will be questioned in a first step about two arts education indicators for the education fields described above: 'reached target groups' and 'educators’ qualification'. For a future design it will be necessary to develop further indicators connected to the Soul Agenda, like intensive of mediation or other indicators.

Concerning the available data, especially curriculums, the formal field will be subdivided twice: into arts education within school classes (curricular) and arts educational offers outside classes (extracurricular). Elsewhere existing national analyses prove that it usually is easier to give statements about curricular and extracurricular offers, e.g. school orchestra or drama club (Keuchel 2013b).

To gain an overview of the scope of the different national fields of arts education, as other surveys e.g. PISA do (PISA aims to evaluate education systems worldwide every three years by assessing 15-year-olds' competencies in three key subjects: reading, mathematics and science), the target group of each field of education will be measured by different age groups: one primary school age and the other secondary school age. In order to identify art education structures for conducting an initial questionnaire design, arts education will be refined in a first step to creative activities in traditional cultural sections such as playing music, making/taking pictures, dancing or acting. In later surveys there is an option to a) either limit the creative activities to further due to practicability or b) extend them to other artistic activities, such as media art or receptive activities, e.g. visits of theatres etc.

 

To determine the reach of target groups, the percentage of the various age groups in the respective fields of activity and learning was gathered in the questionnaire. For the indicator 'educators’ qualification' the percentage of the active educators was captured in each field with pedagogical and/or artistic qualification.

Because at the present time structural data and empirical studies in the field of arts education hardly exist on an international level, the instrument presented here is also intended to be used to improve the empirical data on arts education internationally. So for all the indicators respondents are asked whether the answers are based on existing data sources or whether they are based on estimates.

Background and function of the pre-test

To ensure the practical application of AEDI on an international level INRAE conducted an international pre-test with INRAE-Experts (many thanks for their hard work in supporting the pre-test to: Robin Pascoe, John O’ Toole (Australia), Liane Hentschke (Brazil), Larry O'Farell (Canada), Eckart Liebau, Richard G. Whitbread, Samuel Leong (Hong Kong), Shifra Schonmann (Israel), Ralph Buck (New Zealand), Aud Berggraf SæBø (Norway) and Chee Hoo Lum, Shuxia Tai (Singapore)) in eight different countries in five continents with the indicators described above in 2012 and 2013.

The primary function of the pre-test was to test the instruments for practicability in an international context and to collect initial data about some countries, even if they are not significant. Completion of the standardized questionnaire was not the only intention of the pre-test. The experts were also asked to give a precise feedback on the model’s suitability for international comparisons, on the mentioned validity and on specific suggestions for improvements.

In the process of the pre-test it turned out that research to identify the requested data needs far more time than estimated because it requires extensive secondary analyses, especially in federal state systems. These are the reasons why the significance of the collected present data is very limited. In the following they will just be used to demonstrate opportunities for analyses.

Analysis opportunities

The research approach is characterized in its name as an index with the goal of enabling international comparison. But AEDI provides more analysis opportunities, which can be located in four focal relevant themes as demonstrated in the following:

Monitoring of International Data

AEDI in its approach and basic idea is designed not just with the intention of collecting structural data, but also of producing an overview of existing data and empirical studies. So one analysis option is the possibility of monitoring the empirical data situation of arts education worldwide.

 

Generally, the feedback of experts, who participated in the pre-test, from countries with comparatively high educational levels point out the insufficient state of statistical data in arts education. Partial exceptions are the formal and the non-formal fields, for which in some cases data exist, presumably from school statistics or institutional statistics. If this feedback would be verified by valid data it becomes obvious that there is a real lack of empirical research in arts education.

General Findings about Organisation Structures

AEDI also could help to identify general findings on arts education organisational structures. For instance, on the basis of the experts’ feedback there is a correlation between the reach of target group within the formal curricular education on the one hand and the reach of target groups as well as the educators’ qualification within the non-formal education on the other hand, as it is presented in the following figure:

 

If this finding can be verified based on valid data, that countries with a highly developed curricular arts education sector also provide this level within their non-formal offers, this would be revealing. It would not just dispel the prior outlined presumption, according to which a wide range of non-formal arts education would relieve a less developed curricular one, but also support the perspective, that a comprehensive curricular arts education sector is rather the premise for a manifold non-formal arts education sector.

It would also be enlightening if the following diagnosis could be proved with valid data: The reach of the curricular artistic sector in the pre-test is internationally much higher for the 8-year-olds than it is e.g. for the 15-year-olds. The offers for the 15-year-olds in this context are also not compensated by intensified non-formal structures. This leads to the question, why the 15-year-olds should be served less in arts education than the 8-year-olds.

 

Working as Index for International Comparison

Designed as an index AEDI can be used for international comparisons. Generally it can be argued that a few quantitative data as indicators are not sufficient to draw a conclusion on the efficiency of arts educational systems and especially the quality of offers. However, in practice there can be found examples where this happens in other contexts for the purpose of contributing international comparisons and by this international competition.

When dealing with indices, it is essential to be able to estimate the significance of the implemented indicators in a realistic manner and to avoid generalizations or over-interpretations of any kind. The way in which the use of various indicators leads to different results, can be demonstrated strikingly with the few data of AEDI.

For example, by comparing the national arts educational structures with AEDI it is necessary to take the predefined research design into account and think about how to handle the formal field that was – in favour of a better characterization – separated into the curricular and the extracurricular fields. 

Including the data of both mentioned sub fields equally in the index without weighting, the formal educational field is predominant compared to the non-formal and informal contexts. Taking the corresponding effects into account, New Zealand ranks first with its far reach of target groups within the formal fields of education. If one stays with the original idea that the three fields are weighted equally – which involves the necessity to divide the formal educational field by two, since it is composed of the curricular and the extracurricular field –, Canada ranks first due to the fact that in this country a high reach of target groups within the non-formal and the informal fields of education can be observed.   

Another modified ranking amongst the countries that included the educators’ qualifications in addition to the target groups (unfortunately, while translating the AEDI questionnaire from German into English the translator made a serious translation error. Originally, it was asked how many qualified school teachers and how many other educators are participating in arts educational programmes within the respected educational fields, and what kinds of artistic trainings and pedagogical qualifications they have. Due to the translation error this crucial information wasn’t available. For that reason, the qualification of school teachers and educators was combined to one group in which the data of the teachers’ qualification was weighted higher within the formal educational field (2/3). Outside school, it wasn’t possible to analyse data concerning the pedagogical qualification; the data simply consists of the information whether the educators have or have not an artistic training) illustrates how crucial it is to use as many indicators as possible in order to describe the outcome of a system. So in a ranking list Germany is in seventh place concerning the reach of target groups, but shifts to fourth place when both aspects are taken into consideration.

Due to the increase of indicators, the ranking amongst the countries could change continuously. Hence, researchers are encouraged to establish more indicators for this context in a long-term. In doing so, one must keep in mind that the model has to stay manageable for the experts.

Using Index for Linking to Other Studies

AEDI can also be linked to other arts education studies. Conceivable is a connection to qualitative international research projects, for which AEDI as an index can offer a basis to group the participating countries into different arts education structural types and then put this grouping directly into the context of the qualitative results of the particular studies.

Another possible link is the connection with existing international indices on different topics such as PISA, Human-Development-Index, Global Peace Index, Corruption Perception Index, Democracy Index etc. This can be used to outline the value of arts education for national systems according to the Seoul Agenda, and at the same time show potential links between the national significance of arts education and other social developments. An interesting research question in this context could be for example, if countries which succeeded in PISA also invest in a broad infrastructure of arts educational offers. Is the key to success here perhaps not the focus on core subjects, but rather the support of a wide range of educational offers?

Feedback of the expert for modification

All participating experts, involved in the first pre-test, stressed the necessity of reworking AEDI and this also pointed out the difficulty of developing international research designs. In three areas it was possible to frame some important ideas for improvement as well as to point out problems which still need to be solved regarding AEDI, as listed up in following table:

Conclusion – Use and future

Initial, useful advice to further development of AEDI was provided by the experts in the pre-test. But it became rather obvious as well that a continued exchange of ideas and further pre-tests are still necessary for the development of a final index. 

Even if there is a need to invest more time and work into AEDI the investment could be useful because there are many advantages to a continuous monitoring of arts education, including the following aspects:

1. Collecting more expertise about structural relations (organization of arts education depending on different educational systems, correlations between arts education in formal, non-formal and informal contexts, etc.)
2. Analysing interdependencies between the national development of arts educational structures and other national topics (e.g. PISA, different political systems, national aspects like corruption, economy, etc.)
3. Establishing a basis with AEDI to systematise different national arts educational systems and link them to other studies concerning arts education, especially qualitative ones
4. Providing a political rationale for implementing the goals of the Seoul Agenda on a national level.