Big data has entered the lexicon of supply chain managers in recent years, with greater access to larger data sets opening the door for sharper decision making. Some of the biggest, and simplest gains, however, can often be answered by small data. Chris Michel, Director of Digital Solutions at TMX Transform explains how mastering your small data is a key step in achieving visibility and efficiency in supply chains.
Data is king in today’s supply chain, driving improvements in critical functions such as forecasting, inventory management, customer service and responses to market changes. With datasets increasing in scale and complexity, organizations have invested heavily in data science and analytics to lead them to the promise land.
Regardless of the scale or size of datasets, many companies aren’t getting what they need from the data sitting inside their organization. Supply chain leaders are starting to look beyond the impressive-looking information dashboards to ask, what is the actual business return from my digital assets?
More organizations are considering restarting their digital journey, so they can reorganize their assets to drive business outcomes rather than just content.
Advanced analytics and data science teams are required for large scale projects, but many businesses are missing out on the potential to leverage some of their smaller assets to drive more sustainable change.
“Dashboards and visualizations tell a lot of information, but businesses are struggling to convert that into ROI. To be effective, data needs to activate something within the organization to create a better outcome.”
While big data has been heavily promoted, small data often presents a greater opportunity for organizations, especially in the short term. By combining smaller, diverse datasets, businesses can create a comprehensive view of their operations, leading to more actionable insights and practical improvements. Rather than relying solely on massive amounts of transactional data and hoping for the best outcomes, organizations are using the power of small data to drive impactful solutions to reoccurring problems.
Small Data vs Big Data
Big data is characterized by its scale, involving large volumes of information that change rapidly and come in various forms. This type of data often requires specialized technology for analysis, beyond the capabilities of a typical laptop. Big data is frequently associated with advanced analytics, machine learning, and AI, involving the input of enormous datasets into complex models with the expectation of generating significant insights and ultimately financial results. The strategy often delves deep into a specific domain, such as analyzing billions of point-of-sale transactions.
In contrast, small data involves more manageable datasets and focuses on connecting different information nodes across the business. It aims to provide a wider, more holistic view of operations, such as linking the customer journey to product development and the supply chain. Small data can often be analyzed using more conventional tools and methods, emphasizing the quality and relevance of data over sheer quantity.
Big data relies heavily on extensive technology and infrastructure, requiring powerful computing tools to handle massive datasets. Small data, however, presents a different challenge focused on people and domain knowledge instead of technology.
For small data, the emphasis is on organization and expertise. A knowledgeable individual with a laptop can drive significant change without needing any of the intricate tools, leveraging small existing digital assets through skilled analysis rather than investing in complex technological solutions.
How to make the most out of your small data
Organizations often make mistakes when dealing with small data, primarily due to a mismatch between what technology teams aim to deliver and what the business needs.
“A common error is the tendency to collect and process vast amounts of raw data without a clear purpose. For instance, while point-of-sale systems generate detailed transaction data, businesses often only need aggregated information by store and day. This focus on data depth rather than breadth can lead to inefficiencies and missed opportunities,” Chris notes.
“Another frequent misstep is the disconnect between technology and business teams. Tech teams may pursue complex data solutions, while business teams have different expectations, resulting in projects that fail to meet actual business needs. This misalignment can lead to investments in overly complicated systems that don't deliver the desired outcomes.”
To effectively leverage small data, Chris recommends organizations consider the following steps:
1. Develop a clear, short-term strategy: Focus on what you want to achieve in the next six months. This timeframe provides a balance between immediate needs and long-term planning, allowing for a "Goldilocks zone" where technology and business teams can align their goals and capabilities.
2. Evaluate existing resources: Take a fresh look at your current data assets and tools. Often, organizations already possess valuable data that can be repurposed or analyzed differently to yield new insights.
3. Shift from reactive to proactive analysis: Instead of relying solely on historical dashboards, use existing data to predict future trends. This approach can help manage issues proactively rather than reactively.
4. Prioritize cross-functional collaboration: Ensure that technology and business teams work together to create a roadmap that addresses real business needs while considering technical feasibility.
5. Focus on connecting data points: Rather than diving deep into one area, aim to stitch together various nodes of your supply chain or business operations.
6. Invest in the right skills: Ensure you have people with the necessary domain knowledge and analytical skills to interpret and act on small data effectively.
Simulation as a small data opportunity
At the forefront of the new digitally enabled strategy is simulation – a powerful validation, visibility and communication tool for businesses looking to assess automation feasibility, improve network design and optimize distribution centers.
Simulation helps organisations to model hypothetical scenarios without having to physically change the asset and is a great example of using small data to make big changes.
Simulation is oftentimes a small data exercise, taking diverse slices of a company's data and combining them with extensive domain experience and analytics.
“This process integrates people, processes, and technology to achieve the best results, taking snapshots and segments of data to generate recommendations and outcomes that drive business changes. Whether implementing an Automated Storage and Retrieval System (ASRS) or adopting a different network strategy, the simulation approach is designed to look to the future to prepare for today’s fast-changing business environments.”
To find out more about TMX Simulation solutions here.
This blog is the first in our series on leveraging data in supply chain management. If you're interested in learning more about the power of data, sign up to our mailing list for Part 2, where we'll dive into practical ways to use data sets to optimize your supply chain decisions.
Chris Michel is Director of Digital Solutions at TMX Transform.
