How to collect and leverage big data for business insights

Businesses today have data on every touch point of a transaction, whether it is among their employees internally, or with suppliers, vendors or clients. As a result, organizations are collecting a wealth of information and putting it to use.

“The challenge that management faces is to harvest big data and then leverage it as knowledge to make more informed decisions,” says Sassan Hejazi, Ph.D., director of Technology Solutions at Kreischer Miller.

“There is a lot of data available, so much so that it can be overwhelming. If it can’t be managed well, an organization is not really leveraging that asset. It becomes a matter of effectively managing the information overload.”

Smart Business spoke with Hejazi about harnessing the power of data to create knowledge.

What is the initial step in creating knowledge about a customer base?

The first step is to collect the data. A lot is already being collected, but you have to make sure it is good, clean data, with no mistakes or errors in it. For example, at a supermarket checkout, if the cashier can’t work with the UPC code on the product, the cashier just types in a miscellaneous code. That is an example of an error that has been entered in the data. While there is no perfect data, the degree of data quality is very important.

After collecting the data, how does an organization process it?

The data then needs to be organized. If it is not organized in a proper fashion, it is going to be difficult to massage that data to achieve higher levels of business intelligence.

The data comes from different systems, and every system has its own format.

To overcome that issue, organizations can implement a process called data normalization, which involves data scrubbing so management can use the data assets to support knowledge creation. The data needs to be normalized and stored in an environment called a data warehouse so it can be easily retrieved and analyzed.

There also are smaller warehouses that are referred to as data marts, and these can be useful for departments like sales, financial and production.

How can management then analyze the data?

One of the most popular trends is building a management software dashboard as a tool to present the information to support decision-making. Dashboards are highly customizable, based on each manager’s preference and depending upon the function and nature of the data the manager is reviewing.

A good dashboard tool will allow flexibility. As managers become more proficient in using the dashboard and their needs become more sophisticated, they will want to adapt it to address their needs. The ability to modify the dashboard with minimal help from IT is important because it will allow managers to experiment.

What are the key advantages of these tools?

When managers make a decision, whether individually or in a collective fashion, dashboard tools can help ensure that everyone is on the same page. Stakeholders can see an up-to-date picture of issues, performance levels and key performance indicators within their organization. They will become more knowledgeable about trends and challenges, what’s working and what’s not working. The organization then will reach a higher level of knowledge, leading to improved performance, better decision-making and the ability to develop new products and services.

Are management dashboards difficult to implement?

Dashboard tools are becoming more powerful and easier to use. There are a lot of new data visualization and analysis tools, and companies can test drive them via cloud-based applications.

Previously, these technologies were very expensive and required extensive resources. It is becoming more affordable for middle market organizations to experiment with business intelligence-type solutions without investing in major hardware and software platforms. It allows them to gradually build their business intelligence capabilities using a crawl, walk and run approach.

Insights Accounting & Consulting is brought to you by Kreischer Miller



Data quantity doesn’t overshadow quality of consumer facts and figures

In the 20th century, instinct and an understanding of human nature drove marketing and advertising. Quantitative data simply did not exist the way it does today, and that left advertisers/marketers with the tool of qualitative research — speaking to the target market directly. Back then, just as today, qualitative research told brands the why behind a decision to buy a product, choose one brand over another and more.

So what happened? How did we get to the point where quantitative data has become the darling of data-crunchers and marketers, while qualitative data has fallen by the wayside?

Marketers, e-commerce leaders and product managers seem to have all but forgotten the power of the qualitative data they used to rely on so heavily, and we’re starting to see a tipping point. Is focusing solely on big data a big mistake?

When big data controls marketing strategy, companies risk treating their customers like numbers. Today, marketers all over the globe are so wrapped up in numbers that they are neglecting the human side of data.

Further, big data is destined to eventually plateau — and we may already be there. You see this happening when looking at something like “standard” conversion rates.

Enter qualitative research. To make a real difference — to move the dial beyond “standard” conversion rates or any other conventional figure you’re used to seeing — we need to enact a different approach when it comes to creating and marketing new products and positioning strategies.

You need insight into what customers really want. And to obtain this, you need qualitative data.

Leveraging both kinds of data

Don’t get me wrong, quantitative research is essential for our product and marketing strategies. Still, qualitative research is the missing link to getting the whole picture. While quantitative data can tell you how many people are abandoning your e-commerce shopping cart, qualitative can tell you why.

In these instances, marketers begin to see the benefits of layering qualitative insights with quantitative data. And now that brands are becoming more interested in creating stories that align with the “language” of their customers, qualitative research is once again becoming an interesting avenue.

Investing only in big data is not going to take your brand to the top. A layered approach to bringing quantitative and qualitative together is well worth the effort in the coming year for any product or marketing initiative you have planned.

A few takeaways

  • Real customer insights are necessary to turn quantitative data into qualitative information. Use qualitative insights to direct the questions of any quantitative study, and use qualitative data to help answer the “why” behind the numbers.
  • Aim for moving the dial beyond the inch-by-inch growth and optimization we’re used to seeing with pure quantitative-based actions.
  • Take advantage of the efficient, affordable methods available for obtaining qualitative customer insight.

Finally, don’t make the mistake of getting too wrapped up in big data. The real stories lie in the holistic set of data that’s available to us no matter what type of campaign we’re running or product we’re developing.

How big data is changing the way business decisions are made

Satyendra Rana, Ph.D., vice president, HTC Global

Satyendra Rana, Ph.D., vice president, HTC Global

More information leads to better decisions, and big data is providing companies with enough background to take much of the guesswork out of decision making.

“The larger the data set, the greater the context. Big data promises the tools to make observations much sharper and guide decisions based on facts or highly likely predictions, as opposed to intuition or sheer courage,” says Satyendra Rana, vice president at HTC Global.

Smart Business spoke with Rana about how companies can take advantage of big data and its potential for improvement and innovation.

What is big data?

Big data is a paradigm shift in the way businesses view and use data. For a long time, businesses focused on people, process and technology; data was considered a pain rather than an asset or an opportunity. Companies need to innovate and can no longer do so with the old approach. The new triangle is people, process and data, with technology as a substratum enabling all of those.

The first step to using big data is to look at what business outcomes are desired, then work backward and determine what data needs to be captured to glean those insights. A lot of that data might be internal, but business is not conducted in a vacuum — it needs to be understood in the context of markets, customers and suppliers. So there may be a need to gather external data to be correlated with the internal data. People have conceptions that big data initiatives have to necessarily use outside data, or that it’s only about outside data such as social media. It’s really more about what is done with the data than the source.

There are opportunities to collect data through various applications and sensors. Historically, it was a problem to collect data because it wasn’t readily available. For example, surveys were the main mechanism for collecting market data; you would need to approach 100 people to get 10 to respond. Now, people are volunteering information through mobile platforms and social media. Data is also being collected through instruments such as sensors on cars. Technology has made it cheaper to collect and store data, but businesses still have to take another step and leverage that data.

What are some of the applications?

The applications are everywhere, even though the most frequent uses are seen in marketing. Understanding customers better leads to improved relationships and more cross-selling and upselling. But big data insights can also improve operational efficiencies. For example, supply chain decisions about what products to stock in the warehouse can be influenced by big data. Insights could also lead to entering into new lines of business that weren’t considered. Further, a consumer using his or her credit card at a large retailer might be sent an alert offering a coupon for lunch at a partnering restaurant. The credit card company knows from its data that the customer eats lunch at this time and one of its restaurant partners is nearby, so it tries to predict behavior in real time. When the person uses the coupon, the credit card company gets a share. That’s a new line of business based on information the company had and was not utilizing.

How can companies get started on big data initiatives?

That’s an issue companies are struggling with. A data governance strategy is needed to deal with the amount of data that is received. You have to understand what is coming in and how it can be used. The most important step is to realize that big data is not just a technology issue, which can be a difficult task internally. Big data requires the business and IT sides to work together more closely than in the past. If big data is approached as an IT issue, its full benefit will not be realized. If it’s a business process and IT is involved only in terms of what storage to buy or application to install, companies may not quite understand what is possible.

Big data is changing the way businesses approach the fundamental need to innovate and create differentiation. For the past 20 years, innovation was about streamlining processes such as supply chains. Big data provides a new field for innovation by providing insights quickly and in more creative ways. Eventually, businesses will not have a choice; they will have to deal with big data in order to innovate and survive.

Satyendra Rana, Ph.D. is vice president at HTC Global. Reach him at (512) 773-0357 or [email protected]

Insights Technology is brought to you by HTC Global Services

How commonly found information can transform your business

Pervez Delawalla, President and CEO, Net2EZ

Pervez Delawalla, president and CEO, Net2EZ

In the past 20 years, companies have been generating an increasing amount of data. The growth of social media has also created a massive pool of information that any company can access, mine and use.

“Utilizing big data can help a company uncover the relationships it has with consumers and businesses that perhaps it didn’t previously realize it had,” says Pervez Delawalla, president and CEO of Net2EZ. “In many ways, that data can help a company gain a better understanding of its clients’ needs and formulate its products to win more business.”

Smart Business spoke with Delawalla about big data and how to effectively store and utilize it to the benefit of your business.

Where can companies find big data, and how can they use it?

With the advent and proliferation of social media, there is information that companies can collect called ‘big data,’ which can be used to analyze, in a cost-effective and time-efficient way, the social habits of consumers. This information allows them to devise targeted marketing campaigns and develop products.

Data about consumers is being collected from social media outlets such as Facebook and Twitter, data about businesses can be collected from sources such as LinkedIn and Foursquare, and there is data contained in emails coming into a company.

Do all companies have access to big data?

In today’s world, any company that uses computers has a big data resource or is collecting it without realizing it. For example, most salespeople have a contact database that includes people they’ve met through work, in their personal lives and through networking. If you are going to meet with the CFO of a potential client company and you learn that someone on your sales team knows that CFO, that is an invaluable personal connection. Knowing about that relationship allows you to bring the person to the meeting and quickly establish a connection.

What challenges come with big data?

Storing big data was traditionally cost prohibitive, which is why only large companies could do it. However, solutions such as new, lower-cost hardware have recently hit the market, which has given smaller companies the ability to have large sets of storage devices to store big data. At the same time, cloud computing allows a company to rent storage on a monthly or short-term basis, meaning more companies can collect, store and mine big data.

Indexing this data so that it can be used to benefit the company is a challenge, but there are plenty of tools available from major software manufacturers that can be used to mine it.

What methods are available to companies to help store this data?

Big data can be stored privately or on servers that host multiple clients. Which option a company chooses depends on how important it is to keep information secure.

Private cloud services give companies a certain amount of secure storage on a server that only belongs to them. The type of data being stored determines which tools are applied to extract it, such as a dashboard through which a company can query or search its data. There are also data feeds that provide ticker updates as data comes in, giving fast access to information.

Public cloud services are available, but are less secure than private services.

How can companies efficiently navigate such large data sets to get the most use out of the information being retained?

It takes some time to understand which data is going to be useful and to learn which tools are available to store and sort it. For example, you could buy and deploy big data-mining tools to start collecting various sets of data from multiple sources, then create a dashboard that puts that information at your fingertips. However, you can’t simply keep storing information and expect results. You need to better understand your company’s demographics and understand what is going to help your company grow. You have to know your end result and employ the tools necessary to achieve it.

Many companies don’t realize what they have beyond their traditional database and that is sometimes where the treasure trove of data exists. Accessing that data will open a world of opportunities.

Pervez Delawalla is president and CEO of Net2EZ. Reach him at (310) 426-6700 or [email protected]

Insights Technology is brought to you by Net2EZ



How to use big data to make better business decisions

Dr. Chongqi Wu, Assistant professor of management, College of Business & Economics, California State University, East Bay

Dr. Chongqi Wu, Assistant professor of management, College of Business & Economics, California State University, East Bay

Business leaders often rely on intuition when making critical decisions, but according to The Economist Intelligence Unit, executives dramatically increase their chances of success when they bring facts and data into the decision-making process.

“Although beliefs and instincts help executives make expedient decisions, they aren’t always good decisions,” says Dr. Chongqi Wu, assistant professor of management for the College of Business & Economics at California State University, East Bay. “Business leaders become better decision makers when they take advantage of the facts derived from data analysis.”

Smart Business spoke with Wu about the benefits of incorporating big data and analytics into the decision-making process.

Why is fact-based decision making superior?

Although intuitive decision making is simplistic and quick, a lack of underlying data makes it hard for executives to diagnose and correct problems when something goes wrong. Instead of compounding the problem by making another bad decision, executives can drill down into the data to determine the cause of misfires and use factual analysis to set a new course. Actually, studies show that cumulative improvement is hard to obtain when executives react to problems instead of using facts to make prudent business decisions. And since most of your competitors are probably using data, companies that base decisions on gut feel or instinct are at a competitive disadvantage.

What types of decisions or problems are best solved by big data?

In general, data-driven decision making works better at an operational or tactical level since there are relatively fewer risks involved. In fact, when aided by technology, data makes it easy to automate rudimentary tasks and decisions.

Conversely, strategic decisions still require intuition and judgment, but injecting data analysis and modeling into the process can significantly improve the odds of success. Don’t think of gut-based and fact-based decision making as competing concepts because they actually complement each other. For instance, cross-functional teams often use data to project outcomes and validate the return on proposed programs or new products. It also helps diverse teams build consensus by using facts instead of politics and personal preferences to reach conclusions. Strategic decision making still requires risk taking, and success may hinge on market timing, execution and luck. Data just makes executives better gamblers.

What’s the best way to incorporate data into the decision-making process?

First, executives need to lead the way in supporting cultural change by acknowledging the importance of data in the decision-making process. Next, use data modeling to project probable outcomes and evaluate ideas, since facts and knowledge generated from analyzing big data provide a common ground on which ideas can be debated. Finally, force your team to analyze data by asking questions during the evaluation process so they learn how to marry facts and instincts.

Do executives need copious amounts of data to conduct modeling and analysis?

It’s hard to estimate, but simply put, gather as much relevant data as possible. However, there’s no reason to wait; start small and start immediately because there’s no need to invest in expensive systems or software. Purchase information from third parties, tap free sources to validate ideas, use economical cloud services and software as a service programs to analyze information, and begin collecting in-house data. Finally, run an experiment or test to see how much data you actually need to project the return on a small marketing project or idea.

How can executives gain the confidence to make data-backed decisions?

Even though great decisions don’t always produce great outcomes, you’ll gain confidence by realizing that great decision gives you the best chance to succeed. For example, it’s a great decision to have Kobe Bryant take the final shot when the Lakers are behind because, with a career field goal percentage of 45.4 percent, he gives the team the best chance to win. But data also shows he’ll miss about 55 percent of the time. Luck and timing still play a key role in determining success.

Dr. Chongqi Wu is assistant professor of management, College of Business & Economics, at California State University, East Bay. Reach him at (510) 885-3568 or [email protected]

Event: See a calendar of upcoming seminars hosted by the Department of Economics.

Insights Executive Education is brought to you by California State University, East Bay


IBM unveils software and new services to exploit petabytes of data

ARMONK, N.Y. ― As companies seek to gain real-time insight from diverse types of data, IBM has unveiled new software and services to help clients more effectively gain competitive insight, optimize infrastructure and better manage resources to address Internet-scale data.

Organizations can integrate and analyze tens-of-petabytes of data in its native format and gain critical intelligence in sub-second response times.

IBM also announced a $100 million investment for continued research on technologies and services that will enable clients to manage and exploit data as it continues to grow in diversity, speed and volume.

The initiative will focus on research to drive the future of massive scale analytics, through advancing software, systems and services capabilities.

The news comes on the heels of the 2011 IBM Global CIO Study where 83 percent of 3,000 CIOs surveyed said applying analytics and business intelligence to their IT operations is the most important element of their strategic growth plans over the next three to five years.The news further enables Smarter Computing innovations realized by designing systems that incorporate Big Data for better decision making, and optimized systems tuned to the task and managed in a cloud.

According to recent IT industry analyst reports, enterprise data growth over the next five years is estimated to increase by more than 650 percent. Eighty percent of this data is expected to be unstructured.

The new analytics capabilities pioneered by IBM Research will enable chief information officers to construct specific, fact-based financial and business models for their IT operations. Traditionally, CIOs have had to make decisions about their IT operations without the benefit of tools that can help interpret and model data.