Melbourne · Evaluation and evidence practice
Know what
your evidence is
really telling you.
We help organisations understand what their evidence actually supports, and use it well in evaluation, decisions and reporting. Our work is strongest today across health, community services, government and funding.
Claim trace · Illustrative example
CLM-0031v3
The programme improved young people’s mental wellbeing.
Under review. Five sources are linked. One is present, two are partial, one reading is contested and one source is missing.
SRC-0142 · PROGRAMME RECORDS · PRESENT
Attendance and session logs
- What it supports
- Young people attended sessions across both programme years, and the logs show what was delivered and when.
- Where it falls short
- Attendance shows contact, not change. On its own it says nothing about wellbeing.
This is how we read every claim we’re asked to stand behind.
Record IDs, dates and sources are illustrative. Lines show connection only, not evidence strength.
How people arrive
Sound familiar?
Nobody asks for an impact intelligence system. They arrive with a live pressure.
- We have loads of data but don't know what it means.
- The funder wants us to demonstrate outcomes.
- We think the programme works, but we can't prove it.
- We've got three reports saying different things.
- We're collecting heaps of information nobody uses.
- We need an evaluation, but we're not even sure what we should measure.
- Our board wants to know whether we're actually making a difference.
Services
What we do
Evaluation is part of the work. So are the evidence systems, reporting decisions and organisational capability around it.
Evaluation design and delivery
Finding out what worked, for whom and under what conditions.
Outcomes frameworks and theory of change
Agreeing what you're trying to change and how you'd know.
Measurement and evidence improvement
Fixing data you collect but can't use.
Impact reporting and decision support
Saying what the evidence supports to boards and funders, no more and no less.
Evidence governance and capability
Helping your team keep evidence traceable and use it without us.
Practice first
A connected evidence environment
Impact Intelligence Lab is the practice. CIIS is evidence infrastructure used inside some engagements. It does not decide what is true or turn weak evidence into certainty.
Evidence surfaces. People decide. Limits stay visible.
Evidence coverage · Illustrative example
| Claim | Records | Survey | Interviews | Independent review | Own accounts |
|---|---|---|---|---|---|
| CLM-0031Under reviewThe programme improved young people’s mental wellbeing. | |||||
| CLM-0028ReviewedYoung people attended regularly across both programme years. | |||||
| CLM-0029ReviewedThe programme was delivered as designed. | |||||
| CLM-0035DraftFamilies noticed changes at home. |
CLM-0031 × Survey · SRC-0157 · Partial coverageFewer people completed the exit survey than the entry survey, and there is no comparison group.
Shapes show coverage only. They are not a score or a strength rating.
How we work
Experience can be evidence.Lived Experience is expertise.Participation must carry influence.
Did people merely provide information, or could they influence what it meant and what happened next?
We don't only ask whether there is evidence. We ask whose evidence counts, how it has been interpreted, and whether the people closest to an issue had any influence over what happened next.
- Preserve people's meaning, context and consent.
- Distinguish what someone said from our interpretation of it.
- Keep disagreement and missing voices visible.
- Allow people to challenge interpretations.
- Where possible, show how their contribution influenced what happened next.
Human Meaning Plane
One account, held in four roles: evidence, context, interpretation and governance. The words carry the meaning. Lived Experience stays in the context and influence of the account. It is not a checkbox.
- 01
Voice
Evidence
What was said, kept distinct from what a reviewer later made of it.
- 02
Context
Context
The setting the account came from, including what the record leaves out.
- 03
Consent
Governance
Whether the account can be used, and for what. Consent is not assumed.
- 04
Provenance
Evidence
Who the account is connected to, and how it entered the record.
- 05
Disagreement
Interpretation
Different perspectives stay visible. They are not averaged into one view.
- 06
Missing voices
Context
People closest to the issue who are not yet in the account.
- 07
Who interpreted
Interpretation
A named human reader. Interpretation is not an automated score.
- 08
Influence
Governance
Whether people could change what the evidence meant and what happened next.
Audiences
Who we help
- Community organisations and NGOs
- Councils and local government
- Primary Health Networks and health programmes
- Funders, philanthropy and impact investors
- Evaluation and strategy teams
Facing the same evidence problem in another sector? Talk to us.
Who you'd work with
Alan McCarthy
Founder
- Close to 20 years across community mental health, youth services, primary health commissioning and local government.
- Supported commissioning, provider performance and outcomes reporting across six headspace services within a primary health portfolio.
- Established and evaluated a youth outreach programme that delivered more than 1,000 occasions of service in its first year.
- Led national training and a whole-of-community youth needs assessment.
Bring the question. We work out the right response. CIIS only if it fits.
Learning horizon
What we know, then the next question
What we know → Emerging questions → Worth examining next
What we know
An illustrative claim is on the table, with some sources, context and limits attached. It is still under review.
Emerging questions
Which voices are not in the record? What would a different reader make of the same material?
Worth examining next
A consented way to hear from people the current records do not include.
The path stays open. A new question does not raise evidence strength, reporting permission or claim authority.
Discuss the evidence in front of you.
Impact Intelligence Lab is the practice. CIIS supports some of that work. The next step is a conversation.