I Tested 4 Different Error-Reporting Services for PHP

And why I switched to the self-hosted version of Sentry

Photo by Portuguese Gravity on Unsplash

In case you don’t know what an “error-reporting-service” is, let me try to explain this shortly:

Say you want a notification in case your application throws any kind of error, more important, when we speak about PHP: Exceptions.

PHP offers methods to handle these Exceptions. If you feel lazy to build your own notification and monitoring system, the Internet offers a lot of different services that do this job for you.

Many of these services handle more than one programming language, others have a focus on a specific language, it really depends on your needs.

These services usually offer you a package (composer require….) that registers an exception-handler that pushes the error via HTTP to that service.

In the Dashboard of this service, Errors/Exceptions get grouped and displayed
like in a Ticket-System (i.e. Zendesk). With that, you get a very good overview of what’s going wrong inside your system. When working in a Team you can assign these Errors to Teammates, add comments, and so on.
But most importantly: You get a Notification if something goes wrong!

I hate Clickbaiting, so let’s start with the important findings first:

The winner (for me) is Sentry!

Why?

1. Because sentry is the only service that shows the contents of $variables
   that are involved in the Error! (please correct me, if I am wrong here)
   I am going to explain this…
2. Sentry offers flexible pricing and you get a lot for your money
3. Because Sentry is open-source, you can run it on your own server with all features (please correct me, if I am wrong here)
4. Sentry has tons of Features (I have never used and supports a lot of different languages and offers packages for most of them)

Disclaimer: The posting here is extremely subjective because on the one hand I have been using Sentry for years and on the other hand I have only taken a very superficial look at the other services. (because of the lack of time)
So it can’t be ruled out that I missed some great features of other services — with that said: Please test everything yourself if you are also looking for a (new) service!

So which services did I look into?

• sentry.io (offers a Free Plan but without 3rd Party Integrations)
• bugsnag.com (offers a Free Plan with full Alerting)
• raygun.com (no free plan)
• flareapp.io (no free plan)

What leads to me checking others Services than Sentry after all that years?

Sentry recently made an announcement:

And this is a bummer! ;-(

Let me explain. In Sentry the structure is like this:

• you have an account (as usual)
• inside your account, you have “organizations”
• within each “organization” you have projects

So of course, to keep things clean, people who work for different clients/companies create for each client an “organization” inside Sentry and in here all projects of that client — makes sense, right?

For a very long time, Sentry offered Slack Notifications with their “Legacy Integration” — even in the Free Plan — fantastic!

As you can read in that Announcement, this “Legacy Slack Integration” is no longer usable and if you want to go on with Slack Notifications, you are now forced to upgrade to a paid plan. To be clear: This is 100% fair and okay — but here’s the big problem with their concept:

You don’t pay inside your “Account”— you pay “for each organization” in your Account.

So let’s assume we have 5 clients:

To keep Slack Notifications alive, you would have to pay 312$/year * 5,
which makes 1560$/year just to have Slack Notifications !!!

….yes, and some extra stuff you get with your upgrade, but that´s not the point here.

Isn’t that crazy ?!? 1560$/year just because you pay by “organization” and not by your global account?

I tried to explain to the sentry this way:

If you live in a house or flat: Do you have a “water delivery contract” for every single water tap in your house/flat or do you have a contract for your complete house/flat? If your local water supply company would tell you, you need a contract for every water tap in your house you would start thinking they are kidding you.

That’s how I felt after reading that announcement. And that’s why I checked other services because having Slack notifications is nothing special these days and I must be stupid paying all that money just because of that.

The Sentry support told me:

Also, if you’d like, you could consider transferring your projects to one organization, and cancelling the then unused subscriptions. This way all of your projects will be under one organization, and you’d only be billed once. Just keep in mind that you’d need to invite all members from each org to the main one, and set up new teams.

For me: Not acceptable!

Again: That’s why I quickly checked some other services. I sent the same “Exception” to all services and checked the look and feel of the Dashboard and how much information they provide.

First: Here’s the code that produces the error:

It’s very simple, but the important part is:

It can happen all the time that your app is calling a function with the wrong types. So let’s say $this->aConfig[‘error_reporting’][‘service’] is a “string”.

In this case, we will get the error:

Fatal error: Uncaught TypeError: Argument 1 passed to Mode_test_exception::throwerror() must be of the type array, string given

Now let’s compare how this looks in our different error reporting services:

Raygun

Full-Size Image: https://i.imgur.com/Qtbnx3R.png

Is that helpful to you? The stack trace is not even clickable it’s nearly the same info you see on the bash. You can’t see the code that is involved here. To be honest, this looks so damn “old-fashioned” and there are so many things missing here, but I won’t go into details here.

Bugsnag

Full-Size Image: https://i.imgur.com/yYxsyK2.png

Darn! This looks so much cleaner, you get a full stack trace and you see the code where the error happened. Much much better. It has a lot in common with Sentry. Bugsnag has no “organizations”, there are only Projects and Teams and Team-Members.

To keep track of which Project belongs to which of your clients, it makes sense to use a prefix in your project names here, i.e. CustomerA-webinterface, CustomerA-backend, CustomerB-webinterface, you get the point.

Here’s what the slack notification looks like:

By the way: Sentry notifications don’t have a “Snooze” Button but hey, you cannot have everything ;-)

Flarapp

Full-Size Image: https://i.imgur.com/ELaWHwh.png

I don’t know what to say. Flare tries to address Laravel Developers and if someone is paying $319/year for — this, I am very sorry, I cannot express this in more friendly words. For me, it is an absolute impudence to charge the same money as Sentry but only offer 10% of the functionality. Yes, Flare offers “Deep Laravel application insights”, which I have never seen, but I can assure you, that you won’t miss anything if you use Sentry with Laravel.

Sentry

Full-Size Image: https://i.imgur.com/7xCAlLK.png

This doesn’t look only cool because of the beautiful dark mode, there are so many features, options, actions, information — it’s really overwhelming. The number of features and possibilities are endless! I could write a whole story about Sentry, maybe I will!

This is an example of a Slack Notification:

So Why Is Sentry Ahead of All the Others?

I was very close to switching to Bugsnag, but while testing my very first Exception, I noticed that the most valuable Information in the Error-Report is missing:

“The contents of $variables”

Let´s remember the code:

private function throwerror(array $params): void

so when $parms is not an array, we will get the the following error:

Argument 1 passed to Mode_test_exception::throwerror() must be of the type array

okay, getting notified about an error is great, knowing which code is involved, is great as well… but do you know what is even better?

“Knowing the value OF $params in the moment the error occurred”

How the heck shall I debug that error, if I have no clue what data has raised the error? Shall I guess? Shall I add logging just to find out what $params might have been in that moment?

I tell you something: Error-Reporting-Services — in my view — are futile if you don’t see the data that is involved in the error.

Let’s say you get an error, that a DB-INSERT failed, what is that information worth if you don’t see the data you tried to insert? Nothing !

None of these Error-Reporting-Services do show this super information, only Sentry does! (again, please correct me, if I am wrong here)

To be clear: It´s easy to add data to an Exception like:

throw new RuntimeException(print_r($somedata,true));

But seeing the contents of $paramters that are used inside my stack trace
from the beginning till the moment when the error happens, this is awesome!

Again, let’s look into our example:

Please don’t be confused: The TypeError happened because the variable ($params) has been a “string” with the value of “sentry” at that moment.

And exactly knowing this, makes debugging and fixing so incredibly fast!

I have no idea why someone should use any Error-Reporting-Service/Tool without having these pieces of information. That makes no sense (to me).

Photo by Mufid Majnun on Unsplash

Okay, at this point you hopefully understand why Sentry is the winner here,
but you may ask how I solved the “money problem” without paying for each of my “organizations”.

Simple as that: I switched to the self-hosted version of Sentry.

Running Sentry on your own server gives you full control over everything.
It´s incredibly easy with the help of Docker.

So instead of paying >1500$/year I am now going to pay 0$ !

I am running different servers anyway, so the server costs don’t play a role for me here.

With that simple solution, I can stay with my organizations and go on with my well-loved Slack Notifications.

Before you start considering a self-hosted solution, think about it:

At the moment you get a paid plan for “only” $26/month — considering what a server with 8GB RAM costs, this is a fantastic deal!

Thanks for reading.