Learning While We Build

My son loves LEGO. Give him a new set and he can spend hours sorting pieces, testing connections, taking sections apart, and rebuilding them when something doesn’t quite work. And the finished product rarely matches the blueprint—but he’s usually pretty excited about it.

I love seeing the learning that happens while he’s building. It reminds me of Donald Schön’s idea of reflection-in-action: thinking about what we’re doing while we’re doing it, experimenting, and adjusting as we go.

The concept of learning while you build stands out for me when I think about a recent evaluation of an emerging cross-sector initiative. Although the evaluation is finished, the experience reinforced something for me: when the thing you’re evaluating is still being built, evaluation has to build alongside it.

When the Pieces Are Still Moving

The Inland Empire Cybersecurity Initiative (IECI) brings together education, workforce, and industry partners around a shared challenge: developing a stronger regional cybersecurity workforce that can contribute to the region’s economic development and social well-being. Like many cross-sector initiatives, its work depends not on a single program or organization, but on a system of organizations with different roles, perspectives, and ways of working.

That creates an interesting challenge for evaluation.

We often design evaluation as though the thing being evaluated is relatively stable and easily contained: define the program, determine what success looks like, collect evidence, assess progress, and report what we learned.

But what happens when the program itself is still taking shape?

In an emerging initiative, questions are evolving. Relationships are developing. Partners are figuring out their roles. New opportunities and challenges appear. Decisions made in one part of the initiative can affect what happens somewhere else.

In other words, the pieces are still being sorted, tested, and rebuilt.

And that changes what evaluation needs to do.

Learning as the Work Unfolds

For IECI, we used a developmental approach to evaluation—an approach designed to support learning and adaptation as initiatives develop in dynamic and complex environments (Patton, 2010). We gathered evidence through interviews, surveys, and dialogue and provided feedback as the initiative developed.

Looking back, what stands out to me isn’t simply what evidence we collected. It is when that evidence became available and what people could do with it.

If evaluation is intended to support learning and improvement, a beautifully constructed report delivered after an important decision has already been made may not be particularly useful. In developmental evaluation, change often happens incrementally. Sometimes what people need is credible information at the right moment to see something differently, ask another question, make a small adjustment, and learn from what happens next.

That means creating shorter loops between listening, making sense, sharing, adapting—and listening again.

It also means accepting that we may not always know in advance which information will prove most useful. We have to pay attention to what is emerging and draw on different ways of knowing: what the evidence tells us, what people experiencing the work know, what we bring from our own experience and disciplinary knowledge, and what questions of equity and the common good ask us to notice.

The work becomes less about following a predetermined evaluation blueprint and more about learning our way forward.

Different Partners Hold Different Pieces

The LEGO metaphor also has me thinking about perspective.

In a cross-sector initiative, no one person or organization can see the whole picture.

Universities bring knowledge of curriculum, instruction, and how students develop the foundational knowledge and skills needed for cybersecurity work. Employers understand the skills their workplaces require and see how emerging professionals apply what they have learned in real-world settings. Workforce partners understand the pathways, opportunities, and barriers between education and employment. Students and apprentices experience these different parts of the system—and the connections between them—from yet another vantage point.

Each holds a piece of knowledge that matters.

One example from the IECI work involved the relationship between employers and universities. Both play a role in preparing apprentices, but they provide different kinds of learning. Universities help apprentices develop the theoretical and conceptual foundations of cybersecurity—understanding not only how to perform particular tasks, but the principles and concepts behind them. Employers provide opportunities to apply that knowledge in real workplace settings, where apprentices encounter practical problems, organizational constraints, and situations that can’t always be replicated in a classroom.

Neither form of learning stands entirely on its own. Conceptual knowledge can deepen what apprentices understand from workplace experience, while hands-on experience can give meaning and context to what they learn in the classroom. Together, they can support a deeper understanding than either could provide alone.

But that requires more than simply having both pieces in place. It requires connection between them. What employers learn about apprentices’ skills and training needs can inform university education, while universities can help address concepts or knowledge gaps that aren’t easily taught through workplace experience alone.

Seen this way, the evaluation question becomes bigger than whether each individual component is working. The connections between the pieces are part of what strengthens the initiative and keeps it moving forward.

How well are the pieces connecting?

And perhaps just as importantly: Is learning moving between them?

Evaluation While We Build

Developmental evaluation is often discussed in relation to innovation, adaptation, and complexity. All of those ideas are relevant here. But this experience brought something more basic into sharper focus for me:

Timing matters. Connections matter. And learning needs somewhere to go.

Collecting perspectives from across an initiative is useful. Bringing those perspectives together can be even more useful. But the real opportunity comes when what we learn can travel back through the relationships and structures that shape the work—and influence what happens next.

For me, that is one of the possibilities of evaluation in an emerging initiative: not simply documenting what is happening, but helping people notice what is happening, connect perspectives that might otherwise remain separate, and make adjustments while the work is still developing.

Which brings me back to the LEGO scattered across my house.

You don’t have to wait until the last brick is in place to ask whether the pieces are fitting together. In fact, by then, you may have missed some of the best opportunities to learn. And when you wait until the structure is nearly complete to make a change, pulling it apart and rebuilding can require far more effort than making small adjustments along the way.

Perhaps evaluation is most useful not when it tells us whether we built the right thing, but when it helps us learn while we still have the opportunity to build differently.

Explore Further

• Patton, M. Q. (2010). Developmental Evaluation: Applying Complexity Concepts to Enhance Innovation and Use. Guilford Press.
• Schön, D. A. (1983). The Reflective Practitioner: How Professionals Think in Action. Basic Books.
• BetterEvaluation: Developmental Evaluation — a practical overview of developmental evaluation and additional resources.