On all fronts it’s been a great year for Paperless Post and we’re really proud of the way we’ve grown our team, our product, and our business. We thought we’d wrap up 2012 by sharing some lessons we’ve learned, along with a dose of some fun numbers we dug up during our end-of-the year retrospectives.

Ops is Dev and Dev is Ops

This was our first full year with a subteam of Operations developers on our Engineering team. Johnny, Bethany, Dan, Alex, and Vipul have tackled some gigantic projects in 2012, including a data center migration aided by the automation framework they have been building for over a year. Other highlights of their hard work include the full automation of our development environment with Vagrant, an amazingly solid and scalable metrics infrastructure with Sensu and Graphite, and, perhaps most importantly, a collaborative workflow for Operations work that looks and feels very similar to how we develop Product features for our web application. Our Ops team members write Ruby and shell code, participate in Code Reviews with Pull Requests, maintain Issues on GitHub for long running projects, and manage features with great aplomb.

As a consequence of the insane gravitational force of our Ops team’s awesomeness, Engineers on other subteams are taking Operational concerns into consideration when developing features. They are making changes to the Chef repo for configuration file changes, coordinating after-hours data migrations, and generally sharing the responsibilities that have traditionally been Ops-team-only burdens. This is both a reflection of the excellence of our Engineers and the great work Ops has pulled off. Major props.

Internal Feedback Rules

Besides an investment in a full Operations team in 2012, one of our other biggest bets was on a suite of internal tools that our CTO Aaron Quint started writing basically as soon as he started working at Paperless Post in 2010. The approach we have taken will be covered in some detail as some members of our Ops team present at DevOps Days 2013 in NYC, but the big lesson is pretty simple, and our statement of it here is hardly novel: Internal Feedback Rules. Do whatever you need to do to shape the data that your application and its environment generate in order to make it more consumable. If your logs are too verbose, fix your logs. If your alerting system is broken, fix it. If you can’t find the right one, fix your process, or write a new one that fits.

During the last weeks of 2012, we now have alerts in our Development Campfire room if our acceptance tests aren’t passing in a given browser, alerts in our Operations room if the latency on key pages exceeds a certain threshold, and daily emails sent containing graphs and KPI numbers to anyone in the company who is interested. Our management and Product leadership rely on our developers to record the right numbers, and our developers trust management and Product leadership to help them interpret the results. Data is our common ground, and as we all work together to record, present, and understand it, the entire process improves.

Community Involvement Works

Working hard on awesome internal tools is more fun if you can share the outcomes. We have been really happy to be able to send our Engineers to great conferences such as Strange Loop, Rocky Mountain Ruby Conf, JSConf, GoRuCo, RubyConf, and more. We encourage our developers to submit proposals and we send people to conferences whether they’re speaking or not. The result has been a lot of information sharing, some great networking, and a chance for us to spread the word about our Engineering team.

Besides sending our team all over the place, we have been using our office space as a meeting ground for groups like NYC Ruby Women, the NYC Chef meetup, and a variety of our own internal tech talks. Having people come to our office for beer and pizza to hear people we admire talk about whatever they want has been an awesome experience. Since we are a group of insanely passionate and dedicated developers our enthusiasm rubs off. It’s been a lot of fun to see young developers learn from our guests, and we’re psyched that people now think of us as a company that gives back to the dev community.

Interns Do Not Fetch Coffee

Our third annual summer internship program helped reinforce what is hopefully by now accepted wisdom: hire interns and get them to actually do things, and both parties will benefit greatly. In addition to travelling to recruit interns, we spend a lot of time putting together the program that they go through once they get here. It’s easy to grind on the recruiting aspect and say, “eh, we’ll figure it out when they get here” but that’s simply a waste of your early investment in finding good interns. Having a program ready that involves as much of your company (yes, not just your engineering team) as possible is the easiest way to reach a productive place as quickly as possible. We find promising, passionate students, teach them as much as we can, give them as many chances to be successful as they can, and then sit back and watch them ship. We blogged about the interns experiences here and we can’t wait to get our new crop in in early Summer 2013.

Embrace New Languages and Frameworks

Our main web application codebase celebrated its 4th birthday this year (oh commit 65ac5ab, you seem so long ago). Since four internet years equals around 400 human years, a lot has changed in the interim. Frameworks change, traffic changes, products change, developers change. One of the most important lessons we re-learned in 2012 was that new is not bad. We’ve tried to embrace technologies that will make it easier for us to maneuver our way towards our goals in 2013. This year we’ve introduced a network service using the Go programming language, moved towards Grunt and other modern JS frameworks for managing our JavaScript code, employed KSS and other modern style frameworks for our CSS, and of course hacked lovingly on our Campfire bot that is written in Node.js. We’ve managed to avoid ‘the big rewrite’ by integrating new tools as they are necessary, and it has paid off. Do yourself a favor and kill that three-year-old code that is causing you and your team a lot of stress. It will pay off in the end.

2012: The Year In Some Numbers

Finally, here are some great figures that sum up our awesome year. Thanks for reading our Engineering blog this year, we hope to hear from you soon!

• Unique animated gifs posted in Campfire: 3,208
• Unique metrics stored in Graphite: ~250,000
• Most individual user-generated cards sent in a single week: ~2 million
• Registered user growth over December 2011: 211% (3.1x)
• Revenue growth over December 2011: 135% (2.3x)
• Number of commits to main app git repo: 44,545
• Number of commits to chef git repo: 4,447
• Peak traffic served: ~300 Mbps
• Number of production deploys across applications: 1,631
• Number of staging deploys accross applications: 10,600

Thanks to David Stamm, Michael Hansen, Tarun Chakravorty, and Alexa Hirschfeld for help with the above numbers