Know what your work changed, and show it.

Surveys that measure what they claim to, any program you run evaluated against a comparison, and arguments built on how people actually judge. Research and evaluation for organisations that are asked for evidence of what they do.

Research and evaluation works on its own, and as one part of your organisation’s system (its knowledge and your people’s judgement at the centre, with training and workflow automation beside it): the part that shows what changed, so the next decision rests on what holds up. How it fits together

You’re asked for evidence. You’re rarely resourced to produce it.

Most organisations are asked for evidence: by a funder, a board, a regulator or the public. Few have a psychometrician, a data analyst or someone to appraise the research on staff. Three things follow.

  1. Research that isn’t really research.

    Member surveys and community consultations without sound design behind them, and analysis that doesn’t hold up when someone looks closely.

    Instruments built for the question and tested before you rely on them, with findings at the strength the evidence carries.

  2. Programs funded, but never truly evaluated.

    Evaluations rarely include a comparison, so nobody can say what the program itself changed.

    Evaluation designed in from the start, with a comparison wherever one is possible.

  3. Good intent lost along the way.

    A program built from what the community asked for loses that intent somewhere between the funding and the rollout.

    The program’s theory of change written at the start, from the evidence and from what the community asked for, and the program measured against it, so any drift shows.

The research cycle.

Whether it’s a single member survey or a whole program, the work follows the same four steps. The one most often skipped is the comparison, so we plan for it from the start.

The research cycle in four steps: Design, Instrument, Run and measure with a comparison, and Report, with what is learned feeding back into the next design.

  1. 01 Design

    The program and its theory of change, from what the evidence says works.

  2. 02 Instrument

    Measures built for the question, not borrowed because they exist.

  3. 03 Run and measure

    With a comparison wherever one is possible, which many funded evaluations skip.

  4. 04 Report

    Clear findings for boards, funders and members, at the strength the evidence carries.

The same four steps for a survey or a whole program. Step three is the one that lets you say the program, and not something else, made the difference.

Two schematic charts. On the left, one group improves from before to after, but there is no way to tell why. On the right, a second group without the program improves a little too, and the gap between the two groups is what the program changed.

Without a comparison, an evaluation tells you that things changed. With one, it can tell you whether the program changed them.
What does “at the strength the evidence carries” mean?

A small program with no comparison can still tell you something: how people found it, and what they say changed. It can’t tell you the program caused the change. We say which kind of finding each one is, plainly, so a board or a funder can weigh it, and so nobody is caught out later by a claim the evidence couldn’t carry. And when the question is about one person, one team or one change, a careful small trial often beats a confident guess.

An illustrative survey with a few questions, each answered on a five-point scale, beside a private dashboard. For each question, a pair of bars sets last year’s answers beside this year’s. The survey and its answers are made up.

Illustrative: a made-up survey and its private dashboard. The answers are invented; the pattern is what we build, so this year’s answers sit beside last year’s.

Built and in use: A participant survey with a private dashboard, built to measure a client’s training series.

Instruments you can run yourself.

Measuring soundly what your members, clients or the public think takes skills few small teams have on staff. We build the surveys, scales and rubrics you need: adapted from established measures where they exist, built for your question where they don’t, and tested before they’re used, so the numbers mean something. They’re built for you to run yourself, year after year, so this year’s answers can be set beside last year’s.

What makes a measure sound?

Two questions, asked before anyone relies on it: does it measure what it says it measures, and does it give the same answer twice when nothing has changed? A survey written in a hurry often fails both, and nobody finds out until the numbers are quoted. We test both before you use it.

The science of judgement, applied.

Ryan’s research is in how people judge: how they reason, decide, and tell real insight from claims that only sound profound. Many consequential decisions run on intuition nobody checked.

Well-earned instinct is powerful; the trouble is the answer that sounds right because it’s compelling, not because it’s correct.

Here, that science goes into the arguments you make.

The alluring intuition, on Philosophy

For peak bodies and member organisations

Submissions that persuade the people who read them.

A submission is read by people deciding what to fund and what to change, and there’s research on what makes an argument compelling to them. We bring it to your submissions: their structure, the evidence behind each claim, and the language decision-makers need, in your organisation’s own voice, with every claim checked against its source. The same evidence on what makes a submission persuasive is built into the submission drafting in our workflow automation.

How the agents work, on Workflow automation

The same science applies to the decisions at the top: boards and senior leaders are often cocooned from the honest feedback good intuition is built on, so we train the knowledge their decisions draw on, rather than asking them to think harder. Judgement for leaders and boards

Building the skills that stay human.

We design programs as well as evaluate them. As AI takes on more of the routine, the skills that stay human matter more: thinking, reasoning, and the social and emotional skills of living and working well with others. Many organisations exist to build capability in the people they serve. We design programs that build those skills on the ground, inside what people already do. Alongside the skills, we teach people to use AI in ways that support them rather than replace them.

Thinking skills taught in the abstract transfer poorly. Programs work when they’re built into what people already do.

An example of a program we would design, not one already delivered

A program for men’s sheds.

Picture older men at a men’s shed, where they already come to make things and to talk. A program run through the shed would build thinking, reasoning and the social and emotional skills that matter more as AI takes on more of the routine, around the projects and conversations already happening there. Alongside it, the men would learn to use AI in ways that support those skills, for everyday things like planning a project at home or a week’s meals. And it would be measured, with a comparison where one is possible, so the funder and the peak could see what it changed.

For peak bodies and member organisations

Funded through grants, with your organisation applying.

Capacity-building grants can often fund this kind of work. The route is a partnership: your organisation applies, and we’re named as the delivery partner, shaping the application, designing and delivering the program, and measuring what it changed.

A funder gives a grant to a peak body, which partners with Tailored Ideas as delivery partner. Tailored Ideas runs the program through the peak’s member organisations, and evidence of what changed returns to the peak and the funder.

The peak applies and leads; evidence of what changed flows back to the peak and the funder.

Scientists who measure their own work too.

  • Ryan · Research in reasoning and insight, PhD, UQ
  • Ryan · Supervised honours research teams

Dr David Maij, a cognitive neuroscientist, did his PhD on how people learn, and his work since has been helping organisations change what they actually do by putting the science of learning to work. We hold our own programs to the standard we’d set for yours. Meet the team

Start with a conversation.

Bring the program whose effect you’d most like to know, or the survey you’re not sure you can trust.