Monthly Mentor

Karen Keifer-Boyd (October)
Each month, a different member and NAEA awardee is the guest writer for the NAEA Monthly Mentor Blog. This month’s mentor is Karen Keifer-Boyd, PhD, Professor of Art Education and Women's, Gender, and Sexuality Studies at The Pennsylvania State University. She is a co-founder and editor of the journal Visual Culture & Gender. Her research on feminist pedagogy, visual culture, inclusion, disability justice, transdisciplinary creativity, cyberart activism, transcultural dialogue, and social justice arts-based research is published in 60+ publications and translated into several languages. She is a past president of the NAEA Women’s Caucus, a former Fulbright Scholar, and recipient of many awards and grants in the areas of social justice art education, gender barriers in technology, and more. She also serves on the NAEA Equity, Diversity & Inclusion Task Force. Click "GO" to read her full bio.

Go

Membership

Join the largest creative community established exclusively for visual arts educators, college professors, researchers, administrators, and museum educators.

Join NAEA Renew Membership

« December 2017 | Main | February 2018 »

Wednesday 01.31.18

You Can’t Fatten a Chicken by Weighing It

From: Robin Schnur

It’s a slightly gruesome yet apposite aphorism. The least effective way to help something grow healthy and strong is to focus on measuring it. Nurturing, understanding, tending to needs— this creates an environment for something to develop and thrive.

Yet, in every learning context measurement is an essential component. We dedicate significant time to designing and conducting evaluation, and then synthesizing and communicating our learnings. We have to—how else will we understand learner growth and the efficacy of our learning approaches? How else would we measure reach and impact? How will we able to assess whether resources are appropriately allocated.

So, what’s the right formula between doing the thing and measuring the thing?

Within our department, we have many different approaches to this question. In some areas, we have rigorous, multi-year studies conducted by external evaluators. In others, we have surveys and observations forms that we’ve devised ourselves and use with some regularity. It often feels, though, that we’re re-inventing the wheel; that this is not an integrated part of work but an add-on, ad hoc. My quest this year is to crack the nut on evaluation and to make it a reasonable, regular, resource-efficient, and rewarding part of our work.

One area of our work is leading the way, and I am hopeful it will provide a model for evaluating a broader range of activities.

Our Interactive Gallery is a multigenerational participatory learning space within the Ryan Learning Center of the museum. Over time, this room has worn many hats, most recently as an interpretive exhibition space with long-term installations tied to special exhibitions or areas of the permanent collection. Earlier this year, I and my colleague Mary Erbach, assistant director of learning environments, decided to put a hold on our larger scale installations and instead use this space to mount a series of shorter experimental installations that would help us learn about our audiences and their orientations to creative learning environments in the museum—by observing their interactions in novel participatory spaces, by visibly soliciting their feedback within the space, and/or by studying the things they produced and left for others to see.

Image 1_Overview

The first experiment is Drawing Room (November 5, 2017–January 28, 2018). In this installation, we aimed to learn about the linkage between the museum experience and creative production.

Image 2_Zone 1
The space is divided into three zones. Zone 1 invites visitors to look outside and be inspired by our garden and the works of art installed therein. Participants can use artist’s horses and basic drawing materials to sketch what they see out these windows.

Image 3_ Still LIfe

In Zone 2, visitors are invited to create still-lifes from an array of traditional studio objects—vessels, simple forms, flowers, articulated figures and hands. Visitors may choose to display their drawings in a clip-up gallery in the space.

Image 4_Response Wall

Zone 3 is a space for visitor feedback. We offer a set of prompts that query dimensions of creative response, and visitors may post their responses for others to see.

Image 3_ Still LIfe

Drawing Room closed last week, and we have not yet sorted through the hundreds of drawings and response cards that were left behind. But, we have been observing behavior and collecting and reviewing artifacts every day for nearly three months and have noted some broad trends and patterns:

- Many people signed, dated, or wrote other messages on the drawings they clipped up in the display area.
- Some people drew other members of their family drawing in the space, or incorporated elements of the space's design into their drawings?
- Many of the drawings clearly show that the maker spent a significant amount of time in the space.
- Many of the feedback cards express surprise or admiration for other family members’ willingness or skill in trying their hands at drawing.
- Some of the feedback cards express gratitude for or simply note the chance to slow down, reflect, have a go at something unexpected, or think about their own relationship to creativity.
- Many response cards tell us about those areas of the museum that are most meaningful to the writer.

This integrated yet informal approach certainly has its limits: we don’t know much about the people who visited the space and left behind their feedback or drawings. We don’t know whether people came to the Drawing Room at the beginning or end of their museum visit, and how that might have affected their experience. We know nothing about the quality or length of the entire museum visit. We don’t know whether Drawing Room attracted a representative sample of museum attendees or if it drew a new or different audiences. These are things we would also like to know but require a more structured, less integrated approach— timing and tracking, intercepts, etc. For these experimental installations, the integrated, informal approach made the most sense: we now have a rich trove of qualitative data that indicates the strengths and weaknesses of the installation overall, and which can inform our decisions for participatory learning environments in the museum moving forward.

-RS

***

For an interesting angle on the relationship between experimentation, failure, and evaluation, read the December post of the Museum Questions blog.

Monday 01.29.18

TBD

From: Robin Schnur

It’s a conundrum in museum programming. We want to be thoughtful about and well prepared for the learning experiences we create. We want to be able to communicate with our public about opportunities for participation. This requires careful planning and, most often, a long timetable. At the same time, we aim to be of-the-moment, to respond to events as they arise in the media or in our communities. We need to be responsive to the questions and concerns that audiences carry with them, including when they come through the museum doors.

How do we negotiate between the long horizon of program planning and the necessity to be in active and responsive dialog with our audiences and the world as it unfolds around us?  

Apropos of this big question, in our annual planning process in youth and family programs we asked ourselves: What are we obliged to do? What do we know we want to do and can plan for in advance? And, where are the open spaces to capitalize on or connect with what’s happening around us?

I invited my colleague Nenette Luarca-Shoaf, director of adult learning and associate curator of interpretation to join me in addressing this question. Not long ago, Nenette and her team confronted this conundrum with fresh eyes and launched a new public program called Intersections. Within the fixed structure of the public gallery talk schedule, Intersections opens up a space to address social issues and developing current events.

RS: Nenette, Intersections is a big departure from the traditional gallery talk. How did you approach shaking up the public programs structure to create a setting for conversations about social questions, while not abandoning the core premise that works of art are the focus of such conversations.

NLS: There was comforting simplicity in the schedule of thematic or collections-based gallery talks given by educators 4 times a week at noon. Deadlines had us publishing our topics 4-5 months in advance. In summer and fall 2016, though, a series of events (the Pulse nightclub shooting, police violence against African Americans, the presidential election) spurred some of us to ask how we might capitalize on the museum's potential, and that of the artwork in our collection, to prompt reflection and dialogue for both staff and visitors. It was essential that we were able to respond to events as they happened, however, and the current system was not compatible with that flexibility. 

I figured out a way to situate a new program format within the existing framework of noontime talks. Intersections happens on the second Friday of each month at noon. It's still publicized in print but now we use the website to post the topic about a week in advance. This has allowed us to tackle issues such as patriotism, privacy, gender stereotypes, and neutrality in ways that felt timely and urgent, and also opened up space for our visitors and staff to shape the content of the program. In order to determine the topic, the two facilitators have to ask themselves, "What are people talking about outside the museum?" It makes our work and the artwork at its center feel relevant and meaningful in new ways.

RS: Intersections addresses social questions that are of-the-moment, and that means that most people don't have much distance from the issue at hand. For instance, when you and I chose patriotism as our topic this fall, we were responding to that week’s uproar around NFL players taking a knee during the national anthem. The debate around #TakeAKnee was politicized and racially charged. We knew that calling the question might provoke strong reactions among group members, and we chose focal artworks that would we thought would elicit a range of responses. So, how do you create a space in Intersections that promotes a respectful exchange of divergent perspectives?

NLS: It helps to set expectations for the type of program this will be—topical, participatory, welcoming, personal—from the very beginning and then being okay if someone chooses not to participate. We begin the program with everyone sharing their first name and a response to a related prompt. We ask participants to use "I" statements rather than speak in general ways about others. The dynamic it sets up is one of individuals coming together to add their perspective on a topic rather than a lecturer and an anonymous audience.

RS: I am wondering if 'breaking' the form shook things loose in a broader way and allowed you to see more possibilities for open-ended and responsive public programming?

NLS: I have encouraged our educators to experiment with existing program slots, pedagogically and format-wise, within the constraint of the hour-long tour (though I'm not wedded to that duration if a program idea calls for something else). We have established other series such as "New on View," which allows us to highlight installations or rotations that were not planned or publicized in advance, woven guest artists into our programming, including poets and musicians, and next month will be doing a "pop-up exhibition," working with curators to bring artwork out of storage for the hour. While the number of traditional gallery talks has decreased a bit, the diversity of program offerings and our ability to respond, experiment, and collaborate in new ways has been a worthwhile change.

***

In Intersections, people process their own reactions to current events—as they emerge in a rapidly increasing media cycle—with one another and in the context of human expression across time and culture. We realize that our questions belong to this moment and also have been asked and answered by artists in other times and places. Its success has shed light on the urgency to carve out space for personal reflection, response to the moment, and critical inquiry for all of the audiences museums serve.

-RS

Monday 01. 8.18

Happy New Year!

From: Robin Schnur

This is the time of year that many people make personal resolutions for the year ahead. Perhaps more pressingly, this is also the time of year that many of us in museums and non-profits lay plans and craft budgets for the next fiscal year. That’s where my head is on this January morning: when I get back to the office on Monday, January 8th, I and my team need to finish up the process of outlining our fiscal year 2019 plan and determining how much everything will cost.

As we near the end of our planning period, I thought I’d share a few of the big questions we’ve asked ourselves as part of the process, which began back in September and will be all tied up in early February when I have to submit our official budget. In this blog post and subsequent ones, I’ll write about those questions and how we endeavored to answer them.

Question #1: What’s the ultimate goal for our work?

That’s a big one. I’ve worked at the same museum for 16 years and for most of those years I have been involved in leading efforts to create programs, resources, spaces, and leadership opportunities for K-12 students, teens, and families. Annual planning has always involved shaping a strategy, priorities, and actions for the year ahead.

This year we looked further out, asking ourselves what the ultimate outcome of our work should be. This entailed getting two divisions of our department together—School Programs and Youth and Family Programs—to craft an impact statement for our shared efforts to reach and engage young people. An impact statement is a one-sentence articulation of what an organization (or group within an organization, as in our case) intends to deliver at the highest level.

Crafting an impact statement can be tricky. We often propose our work in future-facing language about what the museum will do for/with a group of people (e.g. This new digital initiative will empower families to…). An impact statement centers the community, group of people, or system and the intended effect it expresses is much deeper and broader than any one program, resource, initiative, or group can achieve on its own.

A good example of a succinct statement of intended impact comes from the Santa Cruz Museum of Art and History: “Our Community Grows Stronger and More Connected.” Nina Simon writes about the process of defining the museum’s impact and theory of change in a series of blog posts from 2015.

In our process, we found it was sometimes challenging to not have our impact statement include an indication of the changes our museum needs to make in order to be a more welcoming, inclusive, and equitable place for young people. Ultimately, though, an impact statement is not an articulation of organizational transformation. It is an expression of the positive change that will happen for people if your organization makes the right plans and executes them well. So, it sits out there at the very end of your theory of change or logic model; it’s the beacon on the horizon toward which you point all of your efforts, including those aimed at internal change.

This a basic theory of change model:

If we do these things (strategy, actions) >>> then these immediate outcomes will be achieved (e.g. teens will be involved in enriching arts-centered activities after school) >>> and then these intermediate outcomes will be achieved (e.g. teens will develop their creative voice in an inclusive environment) >>> and then ultimately this impact will be achieved (this is where the impact statement lives).

The distance between outcomes and impact is somewhat indeterminate. Generally, immediate outcomes are things we can observe in the space of a program or learning opportunity. But we know that transformation doesn’t happen right before our eyes. It can take years for an idea, an inclination, a way of seeing one’s self to take shape. The long horizon and aspirational focus of impact makes it very difficult to evaluate. I’ll try to come back to this in a subsequent post.

Back to our process. We began our exercise by using a series of sentence stems to keep us focused on the people rather than the programs or the museum. These were the ones we used:

I wish for every child and teen to feel…

Through an encounter in the museum every child and teen should be able to…

Every child and teen should walk away with…

To be honest, these are flawed sentence stems, and these flaws were well debated during the process. The second two were particularly problematic. Many people bristled at the word “should.” Rightly so, it does feel a bit like we intend to impose our will upon young people in these sentences. Once we wrestled with and moved beyond the imperfectness of language, we generated dozens of sentences that surfaced our aspirations for the affective, cognitive, social, and other dimensions of our work with youth. I should note that we did this first within our divisions—School Programs and Youth and Family Programs—separately, in two sessions each.

Then, we got this big group together to winnow pages of aspirational language down to one statement, using this protocol:

- first, people worked in pairs (one person from each division) to synthesize and craft a provisional statement
- then, pairs grouped together into 4s to share and synthesize their respective ideas
- then, we moved into two large groups of 8 to further synthesize ideas

At the end of the hour, we had a small handful of working ideas that were fairly close together. Subsequently, the head of School Programs, Sarah Alvarez, and I got together to merge and prune our language into one draft impact statement, which I’ll share in my next post after it has been reviewed and accepted by our teams.

At times, I and others have questioned the merit of spending much time on this effort of defining impact in one sentence. Can you really sum up one’s aspirations in 20 words or less? Have we approached this as thoughtfully as possible? What haven’t we considered yet? With all these questions, I remain optimistic that this process has positioned us to better align the efforts of a talented staff to the needs of youth and with the resources at hand and will enable us to communicate the value of the museum for and to young people in Chicago.

If you’re interested, here’s another good story in Medium about a museum seeking to define its social impact, by Kelly McKinley, Deputy Director of the Oakland Museum of California.

-RS