Case Study: Saving Remote Test Results in SDK

Goal - We needed a way to save test results for future analysis + we wanted to explore our own API library while away on a recent team PBH (product building holiday, of course...) Background - In the past, we've been able to see 'real time' test outcomes via terminal, but there is no way to go back in time to see test results and determine: which ones passed? which ones failed? when was the last time a test passed? how have test results changed over time? etc. PBH Solution - Using a generic dataset, we were able to successfully run a feature test and confirm the data saves to Secure Data Kit (SDK), therefore creating a 'test outcome log'. Here's how it works:

And here's why it's awesome: 1. With this tool, we took advantage of the SDK API by adapting a separate test suite for a completely different project and saving its results to an SDK dataset. By doing so, it shows how any app using any language can do the same thing - essentially, we can remotely execute tests in whatever framework we want and remotely add the results to an SDK dataset! 2. Another advantage to this is we can report on 'pass/fail rates' or 'date last passed' for all of our projects, plugins, clients, etc. in a single dataset, which allows us to gather insights over a period of time (no matter the framework). Because code bases change over time and tests can start failing, if unmaintained, a log of test results is incredibly valuable for analysis + understanding the clear picture.