Generic Learning Outcomes in BalticMuseums: LoveIT!

Introduction to Generic Learning Outcomes

In the UK, the Generic Learning Outcomes (GLO) model was developed for museums, libraries and archives to measure the impact these institutions have on their users. GLOs can be used to describe what and how people learn in museums, help to identify the bene ts of people taking part in the activities of the institution and has proved to be a successful method for measuring outcomes. The use of GLOs has spread to other countries (for instance Nordic countries and the Netherlands). Examples of other tools motivated by outcomes, results and impact are ‘Outcome- based evaluation (OBE)’ and ‘The Heritage Learning Framework – Heritage Learning Outcomes (HLOs)’.This handbook has been developed in connection with the project BalticMuseums: LoveIT. It gives a background to Generic Learning Outcomes (GLOs), describes and guides you through the steps in a work process in which GLOs were used and shows how you can apply it. The handbook is based on the results of a workshop at NatrurBornholm that introduced GLOs to the participants. Workshop facilitators were Henrik Wulf de la Motte, Learning Officer and Johanna Rylander, Curator from Malmö Museums. Malmö Museums, Sweden, is a cultural institution within the project that uses GLOs in planning and evaluating its work and in its user dialogue.

This handbook draws on insights gained from experiences of using GLOs as a tool but it is only one of many User Experience (UX) tools for measuring and evaluating the effects cultural organisations have on their users.

DOWNLOAD

Full report on Generic Learning Outcomes – including practical example of applying GLO at Naturbornholm, facilitated by Malmö Museums

Generic Learning Outcomes -Malmö Museums

Background of Generic Learning Outcomes (GLO)

Cultural institutions (such as archives, libraries, museums, galleries, experimentariums, science centres, theatres and so on) operate within a political climate that gradually has become more concerned with and driven by outcomes. For various political, social and economic reasons, cultural and heritage institutions need to communicate their worth, impact and relevance. During the 1990s many British cities and areas were a ected by problems of high unemployment, criminality, low education, poverty and bad public health.

The government sought different ways to solve these problems. A report by the Department of Culture, Media and Sports (DCMS) was published in 2001 in which it was suggested that culture ought to be a natural part of the solution to these social problems. In 2008 the Museums, Libraries and Archives Council (MLA) in the UK introduced the digital learning resource Inspiring Learning for All Framework (ILFA) (http://www.inspiringlearningforall.uk.gov) alongside a set of Generic Learning Outcomes (GLOs) and a set of Generic Social Outcomes (GSOs). Initially the framework was created as a self-help tool to enable museums, libraries, archives and other cultural organisations to become more relevant through developing and improving their practice and to evaluate and demonstrate their value and impact.

An important part of the work that resulted in the ILFA, GLO and GSO models was to create a uni ed vocabulary or language for professionals in the culture sector. The ILFA framework was transferred to the Arts Council in 2011 and many organisations across the UK have used GLOs successfully, including the Tate, English Heritage, BBC, the British Museum and the national museums of Wales and Scotland.

The overall purpose of using GLOs is to improve practice in order to become more relevant to your users by investigating what they know, think, feel, do and need. This means to work in continuance driven by outcomes and the knowledge the fulfilled outcomes give.

GLO can be used in planning activities in cultural settings taking into account what the users bring regarding needs, values, attitudes, knowledge, and motivations and so on.

It can help you formulate what you want your users to experience, feel, do and learn when taking part in your activities. It can help you to measure and assess what happened during the activity. You can also use GLOs internally in your organisation in order to help the sta to understand what it is you learn in dialogue with your partners and collaborators. In this sense it is an evaluation tool and a way to identify clear, achievable goals and how to meet them, which can help you to make strategic decisions for the future regarding your practice (Jönsson & Peterson 2011).

If you want to understand the motivations behind your users’ behaviour, demo- graphic studies of your users’ age, gender, educational background and so on is very often an instrument that is too blunt if used in isolation. Other ways of segmenting users have proved more relevant if we want to gain knowledge about the reasons for visits or use.

Measuring visitor numbers is a common way to start evaluating your work; however, it tells you nothing regarding the quality of the user experience or if the users have experienced and learned what you intended. Visitor studies have shown that a person from one demographic group – say, women in their thirties – can have very di erent motives for their visit than another person belonging to the same demographic group. A woman of 35 might, in other words, have more in common with a man of 65 than another woman of her own age as regards the motives
for their visit.

The generic learning outcomes are grounded in a broad understanding of learning:

Learning is a process of active engagement with experience. It is what people do when they want to make sense of the world. It may involve increase in or deepening of skills, knowledge, understanding, values, feelings, attitudes and the capacity to re ect. Effective learning leads to change, development and the desire to learn more.

Hooper-Greenhill, 2002:5,
see also http://www.inspiringlearningforall.uk.gov

The emphasis of ‘learning’ in the GLOs might seem farfetched or odd to some – why all this talk about learning? Cultural institutions are open and exible learning environments and have a well-known potential for encouraging and supporting creativity and in-depth learning. Autonomous, individual and voluntary learning is common in cultural institutions but they can be used for formal as well as informal forms of learning (Hooper-Greenhill, 2002). Learning processes are continuous. What we learn is not always educational or useful. We learn without intending to do so – many times we learn things without it being planned or desired (see also Hooper-Greenhill, 2002).

Learning does not only involve the intellect, it involves the emotions and the body as well; it is both tacit (felt) and verbal. Learning can be experiental/performative; it may have depth or be quite shallow

Eilean Hooper-Greenhill, 2007, p. 9.

The GLO framework reflects this broad view of learning and in focus for it is learning effects. GLOs measure the individual’s perception of what she or he has learned. Therefore they are a subjective way of measuring learning and not an objective test of learning that has taken place. The outcomes are commonly measured by asking the users about their experiences.1

1 http://nckultur.org/wp-content/uploads/2013/06/HLO_Final.pdf

DOWNLOAD

Full report on Generic Learning Outcomes – including practical example of applying GLO at Naturbornholm, facilitated by Malmö Museums

Generic Learning Outcomes -Malmö Museums

The GLOs

The GLOs are organised in ve di erent categories, or themes, that re ect di erent aspects of learning that are equally important and often overlap. In a particular setting (program, project, exhibition, activity, practice improvement, sta develop- ment and so on) you might focus more on one or two of these learning aspects.

1. Knowledge and Understanding – includes for example learning facts or information and the deepening of understanding, for instance how things relate to each other.
2. Skills – includes know-how, knowing how to do something and the development of skills in order to be able to do new things. It encompasses intellectual skills, so- cial skills, communication skills, physical skills and information management skills.
3. Attitudes and Values – includes feelings, perceptions, opinions about ourselves or attitudes towards other people or an organisation, increased capacity for tolerance and empathy, increased motivation, and also attitudes towards an organisation and attitudes in relation to an experience.
4. Enjoyment, Inspiration, Creativity – includes having fun, feeling happy, being surprised, inspired, innovative or creative and exploring, experimenting and making things.
5. Activity, Behaviour, Progression – includes what people do, intend to do, or have done and changes in people’s behaviour as a result of the learning experience.2

2 http://nckultur.org/wp-content/uploads/2013/06/HLO_Final.pdf http://www.inspiringlearningforall.uk.gov

“The important thing in relation to these headings is not to fit in something under each of them but to use them in organising your thought on one thing or another

– Interview with Henrik Wulff de la Motte

DOWNLOAD

Full report on Generic Learning Outcomes – including practical example of applying GLO at Naturbornholm, facilitated by Malmö Museums

Generic Learning Outcomes -Malmö Museums

The work process

When applying GLOs it is important to do things in a certain order. It might seem nat- ural to start deciding on what method to use when addressing the users: ‘We’re going to do an interview/questionnaire…’ and continue from there. However, if you start the process with deciding on method – which tends to coincide with one’s own preferences of interacting with others – you determine what kind of answers you will get.

We now move on to describing each step of the work process.

Step 1 – Outcomes

Which are your desired outcomes? What do you want your users to feel, do, learn etc.?

The first step of the work process is to identify your goals and formulate your desired outcomes by crafting GLOs. When you know what your desired outcomes are you can formulate strategic questions in order for you to get the answers, knowledge and insights you need to reach your desired outcomes.

Step 2 – Questions

What do you need to nd out in order to fulfill your outcomes and reach your goal? You have formulated your desired outcomes – what you want your users to feel, do, learn etc.

The next step is to find out what you need to know in order to fulfil your outcomes by formulating questions.

The formulation of these questions is essential. A common mistake is to mix up these strategic questions with the questions you ask in dialogue with the users (in questionnaires, interviews or focus groups).

Step 3 – Advisors

Who do you need to ask to find out what you need to know to fulfil your desired outcomes? You have identi ed and formulated your strategic questions – what you need to know in order to reach your desired outcomes.

The next step is to figure out who can best answer your questions and decide who will become your advisors. Whom you need to ask or get advice from in order to get useful answers that will help you to reach your desired outcomes. You can begin with asking yourself who your target group is and if it is available and whom you need to ask if your preferred target group is not available. One place to start from when choosing advisors – users to enter into dialogue with – is in the target group, often speci ed in project descriptions.

Step 4 – Method

How do you need to ask our advisors to get useful answers to your strategic questions in order to fulfil your outcomes?

You have decided who will be your advisors to get the knowledge that you need in order to reach your desired outcomes.

The next step is to decide what method you are going to use when asking your advisors. The method you use is dependent on what questions you choose and on the target group. Sometimes a combination of di erent methods can be used, for instance to ask both before, during and after a project is completed. In order to get answers that give you insights about your users, you may have to phrase your questions in di erent ways and try out di erent methods of user dialogue.

DOWNLOAD

Full report on Generic Learning Outcomes – including practical example of applying GLO at Naturbornholm, facilitated by Malmö Museums

Generic Learning Outcomes -Malmö Museums

Experiences from using GLOs

Emily Johnson, consultant in the eld of Arts and Education, has developed a model for outcome driven and insight guided practice improvement. In this model the GLO terms and categories are used to describe desired and actual outcomes or e ects of museum exhibitions. As we have seen, however, GLOs can also be used in projects, services and other services and products of cultural organisations.
For a cultural organisation to work outcome driven and insight guided means to work driven by the outcomes (or e ects) you want to ful l and that you let the insight or knowledge you have of your users and their needs guide your work
(c.f. Jönsson & Peterson). Ful lling your outcomes in turn gives you new insights
to continue to work from.

Malmö Museums, Sweden, is a cultural institution within the project that has used GLO as a tool for planning and evaluating its work since 2009, when members of the staff participated in a course led by Emily Johnson. Since then Malmö Museums has increased its number of visitors from 170,000 to 440,000 per year. Josefine Floberg, head of exhibitions at Malmö Museums, sees this positive development in part as a result of the Museum’s work with GLOs. In what follows Josefine develops her thoughts on her experiences of working with GLOs.

When asked about the benefits of working outcome driven and insight guided by formulating and evaluating GLOs, she says that she sees it as a quality assurance: ‘Some mistakes you do not have to make’. To Josefine, to work in this manner is a question of attitude in relation to your users:

‘It is not about giving the audience what it wants – that is a common misunderstanding – it is about knowing more about the people you want to reach and becoming more relevant to them. It is about finding out what the starting point is for the people you want to reach – what they know and what they need. Do they for instance have any previous knowledge about the theme addressed in e.g. an exhibition and do they have any special needs mentally, physically or culturally and so on?

Josefine Floberg

Having knowledge about such issues makes it possible for cultural organisations to communicate more e ciently with their users through their various programmes and platforms:

‘Central to using GLOs in your work, as I see it, is to stop presuming things about the users and instead to nd out what we need to know in order for us to become better and more relevant. What I as a museum professional think is good is not necessarily, or very seldom I would say, what the users think is good,’ Jose ne says. She gives as an example a previous project:

‘The 52 persons in the group of advisors from our target group thought that one prototype was the best while all of the museum professionals involved in the project thought that another prototype was best. It cannot be any clearer than that. In the end it comes down to being prepared to, for instance, go against your own taste values if they do not correspond with the users’ taste.’

Josefine also points out that working with GLOs is a slow learning process for cultural organisations internally. That said, this does not mean that the work process in various projects does not have a starting point and an end point:

‘GLOs can help you to get from A to B. Start by asking yourselves what do we want our users to feel, think and do?’

Generic Learning Outcomes are useful in the process of making something new, or finding out how something works or learning about what needs to change

– Johanna Rylander.

Links

Arts Council UK: http://inspiringlearningforall.uk.gov

Danish television interview with René Larsen and Johanna Rylander about the workshop at Bornholm: http://play.tv2bornholm.dk/?area=searchTV&id=286758#286758

Morris, Hargreaves & McIntyre: http://mhminsight.com

The Heritage Learning Framework – Heritage Learning Outcomes (HLO): http://nckultur.org/wp-content/uploads/2013/06/HLO_Final.pdf

Outcome based evaluation (OBE): https://www.imls.gov/grants/outcome-based-evaluation/basics