Starting this site/blog wasn’t an easy decision for me. I’ve been doing some kind of data analysis going on 20 years now. I love numbers – you should know that about me. And believe it or not, lots of people don’t love numbers. Even worse, some people believe numbers and data can be twisted to suit their purpose. And that dichotomy has created some challenges for me. Nevertheless, I decided to start this blog to share my experiences, my knowledge, and maybe a bit of advice here and there.
When I started working professionally in the analytics field, we were called Business Analysts (a title that now belongs to IT). Back then, if you knew how to use Microsoft Excel and Access you were considered a good Analyst. If you knew how to use Access to ODBC into an Oracle database and use Excel to pivot table some data, you were a rock star.
The world has changed considerably in the last 5 to 10 years or so. Many of the underlying technologies that are popular in data analytics and data science have actually been around longer than I’ve been in the field. And yes, I’m looking at you Python. Many others are new or more recently becoming mainstream.
I remember when I realized how much “big data” was gaining in popularity. I just returned from my first maternity leave and compiled a list of all the tools that were being used by Analysts in my company. Just as a point of reference, when I left on maternity leave I used about 6 different tools. These included a strong focus (still) on Microsoft Excel, followed by some of the basic tools in what is now known as Adobe Analytics. There were also some SQL tools and then throw in PowerPoint and Word for some presentation needs. When I returned almost a year later (yup, one year maternity leave in Canada!), those 6 tools became close to 20!
That’s when I realized it was going to be impossible to keep up with the tech. I could no longer know everything happening in my field and had to determine my strengths and focus on them. Data scientists are doing super cool things with machine learning and mathematical models. Data analysts are helping businesses make better decisions by turning all their data into useful dashboards and visualizations. I belong somewhere between those two worlds.
I’m not a computer scientist but I know my statistics and can do some pretty fun mathematical models. I also go beyond dashboards and visualizations to try and understand the why in the data. After all, the underlying data generally represents the psychology of human beings and they don’t always act in the most predictable ways.
Recently, I came across a term called “data whisperer” and it really resonated with me. It seemed to be a great combination of the algorithmic approach used by data scientists with the decision-making and presentation abilities of a data analyst and just a pinch of cognitive psychology thrown in for good measure. “Big data” should not just be about trends, patterns and predictive modeling. Somewhere along the way, the quest for “why” people behave this way seems to have gotten lost in the field of data analysis. Sure, people like Dan Ariely and Daniel Pink have done some awesome research but generally on “small data”.
There will always be the human element in data because as skilled as we have become at coding computers to help detect patterns, they will never understand (or it will take a really long time) the motivation behind that behavior. And when the computer is wrong, what impact does that have on the customer – the person – on the other end of that algorithm? This is where the data whisperer comes in. At the crossroads between computers and humans and is where I’d like to continue to develop my skills and focus on building my strengths.