Big data concepts are being utilised by many industries to reveal opportunities to improve their business. In health, there is so much more to gain in our population’s health, experience of care and the total cost of our health systems. Leveraging big data is a mechanism to improve all of these, a universal ambition of all health systems.

Other industries that already operate digitally have been able to forge ahead in extracting insights from big data that have led to profitable changes in the way they do business. In these industries, the digital revolution has come earlier, because, in part, the number and type of data elements are significantly less than in health.

Healthcare is far more complicated. The volume and variety of data that is relevant to a person’s health continues to expand and become available as health gradually becomes more digital. Many organisations are drowning under the weight of all this data and struggling with the cost of applications, people and infrastructure to support it.

The challenge is that in order to provide a comprehensive view of the patient, the data needs to include a range of new and emerging data types, like social determinants of health which are vital in assessing health risk, and an aggregation of data from a plethora of disjointed systems. This is why big data concepts are so vital in health. We need to learn from other industries and adapt for Health. Until you can provide the full context of a patient, including their social determinants of health and genome for example, clinicians only have partial context for decision-making. A more complete patient record – extending beyond just clinical data – could help drive new insights through AI, or efficiencies through automation.

We need only look to our recent experience in the genomic revolution to understand the perils of ignoring context. For a long time, about 97% of the genome was considered to have no purpose, which scientists labelled as “junk DNA.” We know now that most of the regulation of genes came from this so called “junk DNA.” As an industry, we shouldn’t make the same mistakes ignoring the patient’s context, their social, familial, geographic and economic context, in clinical decision-making.

In addition, there are many sources of data that we know to be rich but that are difficult to access. Much of healthcare is still done on paper, meaning things such as hand-written notes from a clinician make it very difficult to capture this data because of a lack of NLP tools. Where it is digital, the data sets are messy and fragmented. These challenges and complexities in the health sector mean that the best way to leverage big data will be in incremental steps.

There have been many attempts by health organisations to aggregate all their data into data lakes or swamps, which is a good first step, but unless we can go further and begin to extract valuable insights, this first step is just a cost. Taking that next step is a challenge when the data from sources is not normalised and available in a way that allows them to generate insights and make better decisions. The healthcare organisations making most gains in big data are those tackling the aggregation and normalisation challenge concomitantly, so they can present their data in meaningful ways. This will allow humans and machines that are designed to augment intelligence, to make better decisions.

Orion Health has built a highly scalable platform that can aggregate all types of health data, from both traditional and non-traditional sources. Aggregating huge volumes of different data and surfacing it in a complete patient record is what our open platform, Orion Health Amadeus does best. Click below to learn more.