We launched an app last week.

(Sounds like an opening line for a monologue on how minimally viable products and iterations allowed us to get from one user, to SnapChat status overnight.) 

Not so much.

This is a lesson in framing a problem and understanding its root cause, so as not to immediately pull the fire alarm, proclaim the world is flat, and assume the system is broken.

Thursday morning my phone blew up and my inbox was climbing like a stock ticker. 

“All of the data is gone!”
“The application is broken!”
“We have lost all of our client data!”

The app we had just launched was a prospecting tool to help our clients digitally capture leads and automatically follow up with them using an email drip campaign. 

There was a mobile application component and a dashboard/administration user interface. This dashboard allowed clients to see overall sales and conversion numbers, compare their staff performance and search through prospects based on several criteria; number of kids, age, interests, etc. 

On that most pleasant Thursday, that search page was blank. 

Panic ensued. 

It’s important to mention that a few days earlier, we’d launched a major update to the application and clients were just getting used to it.

With the rollout, everyone, by default, was waiting for the bottom to fall out because we are human and that is what we do. 

We expect things to blow up, in our faces, in the most dramatic and embarrassing way possible. We fear the worst.

As I read through the panicked emails, I tried to think through them logically. What could have possibly happened? Our server got completely wiped out? Our database got deleted? Browser caching issue? All pretty unlikely. 

Instead of trying to diagnose the problem sight-unseen and sending out a mass reply email asserting blame, I did the unspeakable.

I called one of my sales guys (that had sent a panic-stricken email) and started a conversation.

“What is happening Bob, I saw your email?”
“All of the data is gone,” he said. “The results say '0' on the search screen.”
“Who are you logged in as, Bob… what browser are you using”... the usual IT-based questioning ensued. 

Once I understood how he had logged in and how his data parameters were set, I asked him to expand the parameters to a few days prior.

“Oh… I see some now,” he said.

The date parameter was just filtering everyone out. We could change the default to be the previous week, so clients could always see people and wouldn’t immediately assume the system was broken.

That night, we updated the date parameter. Problem solved.

We took the time to have a conversation, identify the problem and fix it quickly. 

My sales team doesn't really understand databases or web applications or default variables, and they don't need to. We operate in two different frames of reference. That is why I need them and they need me and my team. 

It’s important for the entire team to communicate. Have a conversation.

Sales people need to take the time to provide as much information as possible back to the support team. And then be patient.

In turn, developers need to talk to the sales team. It’s easy to be defensive when it feels like someone is questioning the functionality of something you built. 

Stop that way of thinking. It isn't healthy or beneficial. It isn’t personal.

Analyze the problem slowly and as a team. Then act quickly to fix it.

You never know, it might just be a date filter. 


- Nate

Have you had a similar experience in application development or any other industry? Leave a comment below if so; I would love to hear from you.