Introduction

Creating a sample pack or library is a long and arduous journey. It’s a labour of love and because of that, we can sometimes be blinded by problems with our creations in the same vein as being told your angel of a child has been naughty at school. We need colleagues to tell us how amazing our creations are for validation, but we also need to be told when we’ve created something that sounds pants.

So here’s a step-by-step guide on how to beta test sample packs successfully from someone who beta tests Spitfire Audio’s sample libraries.

1. Choose Your (BETA) Fighter

Choosing the person you work with is the most important part to get right when it comes to developing sample libraries. Choose someone who doesn’t know what they’re doing, the might miss something vital. Choose someone who’s a close friend, and they might not give you honest feedback.

The best beta testers are the people who would actually go on to use your sample pack. If you’re making a library full of hip-hop beats, it might not be a good idea to get someone who specialises in classical music to test. Equally, you need someone with a good level of experience who will weed out the issues. You may want to also choose someone with less experience to check how easy your sample pack is to use.

The person(s) you pick in the end should be able to give you honest feedback. Even to the extent where they feel confident enough to tell you the whole project is awash. These are the the type of people who will look after your best interests.

So in summary:

Pick at least one person with sample pack development experience (there’s plenty of those on our Forum)

Find someone with very little experience to check how easy it is to use your pack (especially if you have a complicated user interface)

Try and find people with different operating systems (OS) on both Mac and Windows

2. The Release Candidate

If developing a sample pack was like going to a car wash, beta testing is essentially the rag at the end for the final polish. We’re not reinventing the product at this stage (although beta testing might cause us to rethink a detrimental element), merely we’re polishing off any imperfections.

At Spitfire, the pack we release to final beta testers is known as the “release candidate”. This is essentially the finished article. A product you are completely happy with. If you still have major changes to enact after beta testing, all the weeds that were found in the testing round may come a-crop again. So make sure the sample pack is finished and working before you distribute it. That way, any issues found should be minor imperfections that are easy to make squeaky clean.

3. Create a Test Grid or Feedback Form

Use Google Sheets to create a test grid that you and your testers can fill out. This also gives the testers a plan of action, and is a good way of making sure nothing is missed. Additionally, forms are also a great way of collecting data and making sure that testers follow a particular process, but be weary of influencing their testing too much. If you are too controlling over the testing process, things might get missed.

4. The Download Test

Upload the library to a WeTransfer, Dropbox, or Google Drive link where the file can be downloaded as if it was being downloaded from the Pianobook site. Make sure everything is packaged correctly and zipped. Your beta testers should then download the file and try and drop it into the sampler you have developed the library in.

At this stage we’re checking for:

Missing Sample Files

Missing Resource Files (Picture, Images, GUI Elements)

Corrupt files

If the library doesn’t work in Kontakt, a quick fix from your end would be to Batch Resave the library before re-uploading for the testers.

6. Play Every Note

I like to start by playing the notes in quick succession to check for tuning issues, missing notes, and an consistent volume level across the board. This will give you a very quick interpretation of whether the current patch you have open works. If there are multiple microphones or multiple layers which can be solo, be sure to check these individually, and don’t forget about round robins (multiple samples on each note).

Another method is to use a MIDI file of different note values and velocities, listening out for imperfections whilst being able to carry on with other tasks. I prefer to test manually as I find the mind does wonder and you don’t get a proper feel for the instrument.

7. Play it in Context

This is the most important thing- does the instrument actually play well when using it to compose. Whilst the individual notes might sound great, if it doesn’t stack up well against other sample libraries, the sample pack will be unusable. The main thing being tested here is overall volume levels, the different note lengths, and any other issues that might occur (such as CPU dropouts and RAM usage).

8. Report Any Issues

Make sure you have a good system for reporting issues. As suggested above, Google Sheets is a great way of tracking this, as the issues are updated and anyone who has access to the form can see these. This will stop bugs being reported twice.

9. Rinse and Repeat?

If you have made fundamental changes the inner workings of the sample pack, you may want to consider another round of beta testing. Usually, this would be sent to individuals who reported the issues to sanity check whether it has been fixed with the latest version.

10. Aftercare

Be prepared to look after your sample pack once its out in the wild. With more people using the library, more issues might be found. You should also keep an eye on the reviews section for any feedback or feature request. Whilst you might not choose to implement this in a updated version of the sample pack, it may feed into your creative process for your future releases.