When I used to embark on a new software project, I remember feeling overwhelmed with how to make my app successful. I wondered if people would even like the product, let alone pay for it.
I was lucky that with my experience as a software developer in a larger company, I could take a lot of knowledge learned from previous projects to use for my personal projects, including which tools I needed to use to hit the ground running.
I quickly discovered error tracking is among the most important tools. (In this article, I’ll use the terms “errors” and “bugs” interchangeably, but I mean the same thing.)
Error tracking software automatically catalogs the bugs and crashes that your software triggers, which is necessary so that you can release good, bug-free software—which we all agree is the holy grail of production. You might also hear error tracking tools referred to as error monitoring or crash reporting tools.
The reason you need to start error tracking as early in your project as possible, instead of waiting until you’re years down the line, is that at the precious startup phase, you need to gather as much feedback about the state of your app as you can so that you are able to pivot, tweak, prioritize, and evolve based on your needs and those of your customers.
In order for you to benefit from my time spent researching the best way to manage error tracking, I’m going to share with you a few questions that went through my head at the beginning of my development career:
- Why do I even need error tracking?
- Do I need to buy an error tracking tool or can I build one myself?
- Once I have an error tracking tool, how should I use the metrics it provides to ensure great software quality?
I’ll go ahead and answer these questions below.
Why do I need error tracking?
Until launch, you probably have your head buried in the codebase, focusing on the ship date. But it’s the quality of the final product that will determine your success, not how fast you build it.
Error tracking surfaces bugs in production, giving you a chance to fix them and make sure they never get into users’ hands, which is an important part of ensuring your project is as high-quality as possible.
You may have spent hours testing your software in different environments, but bugs are a natural part of software life—whether you’re Microsoft or just a solo developer trying to make it in the gaming world. Users will always find quirky ways of navigating your app, version releases will introduce unseen errors, and third-party scripts will cause performance problems.
While you can’t stop all bugs in their tracks, you can minimize the risk of serious damage by detecting them early and addressing them quickly. But you can’t be everywhere at once, and that’s where error tracking can do the heavy lifting.
Error tracking will automate the most time-intensive task (hunting for problems), so your most valuable asset (you) can spend time on decision-making tasks that give your app value (like coding software).
Remember, slow and buggy software isn’t an option.
Should I build or buy?
Build versus buy is a question I asked at first. It’s a common question, particularly when you are first seeking out tools to support your projects. If you’re anything like I was, you are probably resource-poor—both time and money are slim—so you want the best option for your project and your own sanity.
So you have to make a choice. As an entrepreneur, you may have an “if you want a job done well, do it yourself” mentality. But don’t forget the flipside of that—recognizing that you are the most valuable asset and you shouldn’t be spending time on menial tasks that don’t add value to the business.
What’s a busy yet poor developer to do? Here are some balanced arguments for each side; in the end, you have to go with what’s best for you.
When to build
Specific requirements
Off-the-shelf software may not allow you to be as flexible as you’d like; for example, you may see yourself needing custom features to allow for industry-specific integrations to talk to each other. Most software is built with the needs of the many in mind. If you fall outside that area in a special way, you may need to assess a custom build.
Complex integrations
If you have complex integrations, you don’t want communication problems affecting your future team. Build your own error tracking system if your integrations will grind to a halt with an external tool.
You genuinely have no other option
Something is better than nothing. If you are strapped for cash, a free tool like Google Analytics or an error logging tool like elmah.io might be suitable at this stage.
When to buy
Lack of developer resources
If your development team is stretched for time, you shouldn’t be building software. Your priority needs to lie in giving your users a great experience with your app, especially if it is less than two years old. You’ll have a lot of kinks to iron out, and the more help with that you can get, the better.
Lack of time
Creating is just part of the story—you’ll be spending most of your time maintaining and improving the design of your epic codebase, hopefully for many years to come. Martin Fowler has an excellent book on the topic.
Lack of technical expertise
Do you have experience in building internal tooling? If there is a skill gap, you could be setting your team up for failure and frustration—all wasted time when you could be creating features for your customer base.
More detail needed
If you build a tool that allows you to track errors, you might not get the level of detail without a lot of hard work. For example, to diagnose an error quickly, you’ll need the stack trace and the exact environment it occured in.
So, what should I measure for great app performance?
Error tracking tools are not just for managing your software errors at a granular level. You can use the metrics they provide to set benchmarks around how your software is performing for your end users.
Of course, metrics like how many people adopt your app are really important, and should be measured, but they move all the time. And, as our industry matures, so do people’s expectations for excellent software.
To have a chance in the software market, at the very minimum, your software needs to:
- Be of high quality so your app doesn’t crash
- Load quickly in many different environments (under three seconds is ideal)
- Have a repeatable and scalable delivery process
- Let the user enjoy the software for the intended purpose
Sounds like a lot, right?
Well, after taking many learning experiences from my own projects, and working on a larger team, I have found that you can use four metrics to give you an accurate picture of your app’s performance. Use your error tracking tool to report on these metrics daily, and you’ll be on your way to building software that will give your users a better experience.
Users affected by bugs
Forget error counts. If 10,000 errors affect one customer, it’s not as bad as 500 errors affecting 250 customers. On my team, we measure affected customers on a monthly basis, always aiming to reduce that number.
Median application response time
Better performance will make money, and the best way of tracking that is by measuring the median response time—that is, what 50 percent of your customers experience.
P99 application response time
P99 is important for understanding the upper limits of your app’s response time. It’s a metric that refers to the time it takes for the app to load for the 99th percentile of users. This will usually be over three seconds, but start to worry if your app takes 25 seconds to load. I use the 99th percentile, not the 100th, because the 100th often represents timeouts and bots.
Resolved bugs greater than created bugs
Instead of tracking crash count, measure bug count. As a guide, you should be solving more bugs than you’re creating, and in a brownfield project (a project addressing a problem space like replacing legacy systems), probably even more.
Keeping an eye on these four metrics will:
- Improve user experiences with fewer crashes and faster software
- Reduce technical debt accumulation (making sure you are not leaving a trail of destruction behind every release)
As you gather more data after the launch of your app using error tracking software, you’ll start to build a picture of how you can guide the success of your app and where to focus your resources for maximum effect.
Get started with error tracking
Error tracking can provide security and peace of mind that your early customers get the best possible experience with your app, while you focus on the important work of building new features.
Start tracking your errors from the beginning so that you can fix things as they happen, and not leave them to accumulate over time. Consider all the factors in your decision to build your own error tracking software or buy something off the shelf. And remember that users affected by bugs is just one metric you should track throughout the entire life of your application.
Errors and crashes are bound to happen, but if you keep track of them and fix them quickly, your customers will have a great experience, while you focus resources on what matters most. It’s a win-win!
So, which tools should I use to make sure that my software project succeeds from the start?
At this early stage of your project, you may be running a tight budget. But error tracking is only one of the things you need to worry about. If you are going to build a successful application, you need the support of a complete toolkit. The right tools are critical to your growth as they will do the heavy lifting.
After five years in the industry, I’ve had the privilege of looking after thousands of developers and their software performance, so I’m just passing these tools along.
This list was recommended to me by Kunal Johar, who is the co-founder of a successful technology company and mentor to many young software projects. It represents the minimum that you need if you are serious about giving customers great experiences.
- Jira Software – for project management
- GitHub – repository management
- Slack – team communication and raising problems quickly
- Intercom – a customer service management platform
- Stripe.com– to receive and organize payments
- Raygun – for crash and error tracking
That’s it! The thing about this list is they all integrate with each other, always talking and providing feedback about the quality of your software.
Yes, some of these tools do have a cash investment. But your software project deserves to succeed. Most have start-up plans available which scale with you as you grow. It’s never been easier to manage user expectations, give awesome customer experiences, and monitor your application for performance problems.