Data Science blog

Characterized by its volume, velocity, value, and variety; big data is being produced at a rate of over 2.8 zettabytes (ZB), or 2.8 trillion gigabytes, each year. Every day, 2 million blogs are posted, 172 million users visit Facebook (spending a combined 4.7 billion minutes on a single social networking site), 51 million minutes of video are uploaded, and 250 million digital photos are shared. We continue to generate 294 billion emails each day, even though many consider email an outdated form of communication. It is expected to explode to over 40 ZB per year by 2020; and to stay ahead of the pack, businesses need to start tackling big data today. Investments are being made faster than ever before to improve productivity, create value, stay competitive, spot new business trends, and to generate exciting analytic solutions. Big data is becoming a hallmark of the start of the 21st century where it is being consumed and utilized by more and more businesses.

You can generally split big data into two different types, structured and unstructured. The 294 billion emails being sent per day can be considered structured text and one of the simplest forms of big data. Financial transactions including movie ticket sales, gasoline sales, restaurant sales, etc., are generally structured and make up a small fraction of the data running around the global networks today. Other forms of structured data include click stream activity, log data, and network security alerts. Unstructured data is a primary source of growth in big data as well. Music is an ever increasing variety of data and we are streaming nearly 19 million hours of music each day over the free music service, Pandora. Old television shows and movies are another source of variety in the non-structured realm. There are over 864,000 hours of video uploaded to YouTube each day. MBAOnline.com even found that we could pump 98 years of non-stop cat videos into everyone's home for endless hours of boredom, fun, or insanity!

Beyond technology in general, big data is going to require changes in most business' processes to ensure decisions with proper analytic judgments are made. In order for them to recognize these requirements, two main ideas will need to be focused on more closely. First, exploration of how businesses can leverage current technological solutions to both segment and then dissect the data is required; and second, the presentation and then prediction of the ways in which businesses have and will use the data to form strategies to create, maintain, and then enhance their different revenue streams will need to occur occur.

Businesses have been segmenting customer markets for decades, but the era of big data is making segmentation more essential and even more sophisticated. The challenge is not just to gather the information; rather it is a race to understand customers more intimately. Segmentation is a foundational element of understanding customers. In its simplest form, customers are grouped based on similar characteristics. As the data improves (demographic, attitudinal, and behavioral), the approaches to segmentation become more sophisticated. Right now, enterprises are practically drowning in all the data being collected and if they are not careful, they can spend all their time staring at it and not putting it to good use to make better business decisions. The dissection time can be limitless without yielding actual results, so having a proven and scalable analytics system in place can drastically cut down this segmentation time.Data Science blog

Businesses from all sectors recognize that knowing your customer well leads to improved and personalized service for the buyer and this results in a more loyal customer. In the effort to know their customers better, businesses have traditionally employed advanced analytics systems such as Google Analytics to segment their customers into groups based on demographics, geography, and more. Although this type of segmentation helps, it often fails to not only define important differences between customers, but lacks in offering consistent innovative features. For example, a basic traveler segmentation from an airline might define a customer as a male, 37 years old, lives and works in Raleigh, and makes frequent Business trips to London.

A better approach is to classify by the customer's choices, preferences and tastes based on all his interactions with the business. But to accurately micro-segment their customers, businesses need to recognize a broader range of customer characteristics many of which are found beyond the structured information in Reservation, Departure Control and Loyalty systems of an airline. A rich set of additional information about customers can be found in customer interaction like emails, call transcripts, chat, SMS, social media and more. Businesses should have the ability to understand the meaning in customer dialog, and can do so automatically through newer types of analytics systems.

Big data has the potential to fundamentally change how marketers relate to their customers -all of them - not just the small percentage that actively participate in a loyalty program. Business can leverage the vast amounts of information available in their customer interactions and online marketing paths (such as social media, blogs, and websites) to finely segment, maintain, and grow relationships with their customers.

It is commonly known that big data is both a critical challenge and an opportunity for businesses. Having technologies designed to address the explosive growth of the volume, variety and velocity of information is critical for their success. Luckily, today's alternative hardware delivery models, cloud architectures and open source software bring big data processing within reach. In the end, the big story behind big data may be very small - the capability to create and serve very small micro segments of customers - with a significantly higher accuracy and achieving more with less. Segmenting is the mere tip of the big data iceberg, and the strategies that companies have already formed and will continue to form in order to leverage it is incredible.

There are currently four main strategies companies use to leverage big data to their advantage: performance management, decision science, social analytics, and data exploration. Performance management is where all things start. By understanding the meaning of big data in company databases using pre-determined queries, managers can ask questions such as where the most profitable market segments are. It can be extremely complex and require a lot of resources; however, things are beginning to get easier. Most business intelligence tools today provide a dashboard capability. The user, often the manager or analyst, can choose which queries to run, and can filter and rank the report output by certain dimensions (e.g., region) as well as drill down/up on the data. Multiple types of reports and graphs make it easy for managers to look at trends. With functional and "easy"-to-use dashboards, companies are starting to be able to do more with less; but we have yet to see a solution that offers a clean design with simple functionality, that offers even higher insights then what currently exists.

Data exploration is the second strategy that is currently in play by businesses. This strategy makes heavy use of statistics to experiment and get answers to questions that managers might not have thought of previously. This approach leverages predictive modeling techniques to predict user behavior based on their previous transactions and preferences. Cluster analysis can be used to segment customers into groups based on similar attributes that may not have been originally planned. Once these groups are discovered, managers can perform targeted actions such as customizing marketing messages, upgrading service, and cross/up-selling to each unique group. Another popular use case is to predict what group of users may "drop out." Armed with this information, managers can proactively devise strategies to retain this user segment and lower the churn rate.
The well-known retailer Target used big data mining techniques to predict the buying habits of clusters of customers that were going through a major life event. Target was able to identify roughly 25 products, such as unscented lotion and vitamin supplements, that when analyzed together, helped determine a "pregnancy prediction" score. Target then sent promotions focused on baby-related products to women based on their pregnancy prediction score. This resulted in the sales of Target's Baby and Mother products sharply increased soon after the launch their new advertising campaigns.

The next strategy companies' use is leveraging social media sites such as Facebook, Twitter, Yelp, or Instagram. Social analytics measure the vast amount of non-transactional data that exists today. Much of this data exists on social media platforms, such as conversations and reviews on Facebook, Twitter, and Yelp. Social analytics measure three broad categories: awareness, engagement, and word-of-mouth or reach. Awareness looks at the exposure or mentions of social content and often involves metrics such as the number of video views and the number of followers or community members. Engagement measures the level of activity and interaction among platform members, such as the frequency of user-generated content. Finally, reach measures the extent to which content is disseminated to other users across social platforms. Reach can be measured with variables such as the number of retweets on Twitter and shared likes on Facebook.

Social analyzers need a clear understanding of what they are measuring. For example, a viral video that has been viewed 10 million times is a good indicator of high awareness, but it is not necessarily a good measure of engagement and interaction. Furthermore, social metrics consist of intermediate, non-financial measures. To determine a business impact, analysts often need to collect web traffic and business metrics, in addition to social metrics, and then look for correlations. In the case of viral videos, analysts need to determine if, after viewing the YouTube videos, there is traffic to the company web site followed by eventual product purchases.

The final strategy companies' use has been given the name "Decision Science". It generally involves experiments and analysis of non-transactional data, such as consumer-generated product ideas and product reviews, to improve the decision-making process. Unlike social analyzers who focus on social analytics to measure known objectives, decision scientists explore social big data as a way to conduct "field research" and to test hypotheses. Crowdsourcing, including idea generation and polling, enables companies to pose questions to the community about its products and brands. Decision scientists, in conjunction with community feedback, determine the value, validity, feasibility and fit of these ideas and eventually report on if/how they plan to put these ideas in action. For example, the My Starbucks Idea program enables consumers to share, vote, and submit ideas regarding Starbuck's products, customer experience, and community involvement. Over 100,000 ideas have been collected to date. Starbucks has an "Ideas in Action" section to discuss where ideas sit in the review process.

Many of the techniques used by decision scientists involve listening tools that perform text and sentiment analysis. By leveraging these tools, companies can measure specific topics of interest around its products, as well as who is saying what about these topics. For example, before a new product is launched, marketers can measure how consumers feel about price, the impact that demographics may have on sentiment, and how price sentiment changes over time. Managers can then adjust prices based on these tests.

The future of strategies is hard to predict, however, based on how things are growing, companies are betting that it will be in new types of technology leveraged within analytics systems with a focus in big data. As a founder of a company that focuses in web and data analytics, we are betting the future is in big data processing. By creating an analytics platform accessible online, with an emphasis in beautiful design and a simple interface that is easily used, we are combining powerful analytics with beautiful results. By leveraging all four current strategies and adding our own technology to the mix, the results should push the boundaries between non-fiction and science fiction.