Subscribe for tips & insights Subscribe for tips & insights
Get in Touch
Insights & Resources

What should my sample size be?

How to determine sample size

Whether you are conducting a formal evaluation or review, engaging with stakeholders will help you understand what is happening ‘on the ground’.

It is very easy to say you need to ‘collect data from a sample’, but what does this really mean? How many people should you be engaging with to collect something meaningful?

Question no more! We’ve put together some quick and easy ‘rules of engagement’ to help you understand what your sample size needs to be.

A good maximum sample size is usually around 10% of the population, if this does not exceed 1000.

For example, in a population of 5000, 10% would be 500. In a population of 200,000, 10% would be 20,000. This exceeds 1000, so in this case the maximum would be 1000

Even in a population of 200,000, sampling 1000 people will normally give an accurate result.

Sampling more than 1000 people won’t add much to the accuracy and could potentially be unnecessarily costly

Choose a number closer to the minimum if:

  • Time and money are an issue
  • You only need a rough estimate of the results
  • You don’t plan to divide the sample into different groups during the analysis, or you only plan to use a few large subgroups (e.g. males / females)
  • You think most people will give similar answers
  • The decisions that will be made based on the results do not have significant consequences.

Choose a number closer to the maximum if:

  • Time and money are not an issue
  • It is very important to get accurate results. Typically you will want to lean towards larger samples to increase accuracy.
  • You plan to divide the sample into many different groups during the analysis (e.g. different age groups, socio-economic levels, etc)
  • You think people are likely to give very different answers
  • The decisions that will be made based on the results of the survey are important, expensive or have serious consequences.
  • But remember over sampling could be problematic in terms of creating more work and effort than required.

These rules provide a useful reference point for understanding what is required and are applicable for basic surveys such as feedback forms, opinion surveys or needs assessments that utilise random sampling. Complex surveys such as national surveys or non random sampling would use different criteria, however this guide is appropriate for the overwhelming majority of sampling required.

Use the table below as a reference when deciding on your sample size.

Sample Size Reference Table
The table indicates the margin of error present in a given sample and this will allow you to communicate that margin of error when reporting results, providing greater credibility and transparency.

 

If in doubt, contact us – we would be more than happy to help you calculate the sample size you need.


What is the recommended sample size for public sector evaluations in Australia?

In Australia, it’s generally advised to sample up to 10% of your target population, with a maximum of 1,000 participants. For instance, in a population of 5,000, a sample of 500 is appropriate. However, for larger populations, such as 200,000, sampling 1,000 participants is typically sufficient to achieve reliable results.

To determine your sample size, consider the following factors:

  • Population size: The total number of individuals in your target group.
  • Confidence level: Commonly set at 95%, indicating the probability that your sample accurately reflects the population.
  • Margin of error: Typically between 5% and 10%, representing the range within which your results are expected to fall.

    Utilising online sample size calculators can simplify this process.

An appropriately determined sample size ensures that your findings are statistically valid and representative of the broader population. This is crucial for informing policy decisions, program evaluations, and public service improvements.

Key considerations include:

  • Population variability: Greater diversity within the population may require a larger sample size.
  • Desired confidence level and margin of error: Higher confidence levels and smaller margins of error necessitate larger samples.
  • Resource constraints: Budget and time limitations can impact the feasible sample size.
  • Balancing these factors is essential for effective research design.

To achieve a representative sample:

  • Define your target population: Clearly identify the group you wish to study.
  • Use random sampling methods: Employ techniques like simple random sampling or stratified sampling to minimise bias.
  • Consider demographic factors: Ensure your sample reflects the diversity of the population in terms of age, gender, location, etc.

    These steps enhance the generalisability of your findings.

A small sample size can lead to:

  • Increased margin of error: Results may not accurately reflect the population.
  • Lower confidence in findings: Decisions based on such data may be less reliable.
  • Potential for bias: Limited diversity within the sample can skew results.

    It’s crucial to balance feasibility with statistical requirements when determining sample size.

To account for non-responses:

  • Estimate expected response rate: Historical data or pilot studies can provide insights.
  • Increase initial sample size: Compensate for anticipated non-responses by oversampling.
  • Implement follow-up strategies: Encourage participation through reminders or incentives.

    These measures help maintain the integrity of your sample.

The confidence level indicates the probability that your sample accurately represents the population, while the margin of error reflects the range within which the true values lie. Together, they define the precision and reliability of your survey results.

Yes, various Australian resources provide guidelines:

Creative Victoria: Offers tools and examples for calculating sample sizes in community surveys. 
Australian Bureau of Statistics: Provides sample size calculators and design standards for statistical surveys. 

These resources can assist in tailoring sample sizes to specific public sector needs.

To maximise efficiency:

  • Prioritise key subgroups: Focus on the most critical segments of your population.
  • Use cost-effective sampling methods: Techniques like online surveys can reduce expenses.
  • Leverage existing data: Utilise available information to inform your sampling strategy.

    These approaches help balance statistical rigor with practical limitations.
DIY Program Evaluation Kit

DIY Program Evaluation Kit

In "Plain English"

Not sure how to monitor your program’s performance?

Don’t know where to start to judge whether your program is on track?

A monitoring and evaluation framework is an integral part of understanding how well your program is performing.

It helps you set out and record the activities you need to complete to assess your program’s performance over time to assess whether your program is on the right track.

We created this M&E framework guide to help you build your own. The guide gives you the right structure and useful explanation of what is typically required in each section

Download our comprehensive guide to build an M&E framework for your own program today.

DOWNLOAD YOUR FREE COPY
Get in Touch
X

Whether you are looking to get more out of your projects, are interested in collaboration or would like to talk about joining the TargetedBrains team, please get in touch.