We came, we saw, we Rumbled
RunCodeRun sponsored Rails Rumble this past weekend and offered continuous integration to participating teams. We knew going in that this would be a great way to give a useful service for test-driven teams and learn a lot about how RunCodeRun stands up to load.
Things started well on Friday night and we got some great feedback from the initial teams who signed up. As development heated up on Saturday, we realized that keeping up with the rapid commits was going to be a challenge. Before the Rumble, we’d never had more than 10 builds queue up at any one time. By Saturday night we saw several teams committing every few minutes, and the queue backed up to thirty or forty builds.
We fired up additional builders to run more builds in parallel, and we started pruning old builds that had been queued too long to be useful. On Sunday we automated a job to prune old builds and mark them as “cancelled”, which kept things moving at a reasonable pace until the teams raced to the end on Sunday night.
By the end of the Rumble, about 30 teams had their projects building on RunCodeRun, with around 1,000 successful builds and 800 failing builds in all. The average build time toward the end of the competition was about five minutes.
This was a great experience for us, and we think the service was useful for participating teams despite the backlog. We now have a much better idea of more steps to take to scale and where the pain points are.
The RunCodeRun team is all about releasing early and often and gathering feedback as early as possible. Our thanks go out to the Rails Rumble organizers and to the teams who took RunCodeRun for a spin. We look forward to improving the service based on the lessons we learned at the Rumble. Don’t hesitate to contact us if you have specific feedback, or want to learn more.
