As big data platforms become more popular, two models of data management \u2013 aggregated and federated \u2013 are emerging as two of the most common patterns, but which one is better?<\/p>\n\n\n\n
The main difference between the two models is how they store, access, and surface data. The aggregated model (also known as a centralised model) stores data within its system and allows users access to the information in one centralised repository to manipulate the data, normalise it, combine it and create new insights.<\/p>\n\n\n\n
While this model is popular, it doesn\u2019t necessarily meet every organisation\u2019s requirements. There can be legal and technical barriers when aggregating data in one centralised location, which is where the federated model becomes attractive.<\/p>\n\n\n\n
In contrast, the federated approach surfaces just enough information about the data to tell users where the complete information sits, in the same way that a phonebook will show a person\u2019s name, address and phone number but not much more. This adds an extra step in retrieving full files such as lab results, but if enough summary information is available a user can potentially avoid ever using the full file in the first place.<\/p>\n\n\n\n
In the case of lab results, the portal that users interact with could surface enough metadata to display key pieces of information \u2013 such as the date of a result, type of test, and whether it\u2019s conclusive or not \u2013 to avoid having to open the full file. If a user was needing to drill into a result then they would be able to then go to the original information. The federated model is essentially a well-informed index that provides a path to the original data.<\/p>\n\n\n\n
Aggregated Data Model:<\/p>\n\n\n\n
Pros:<\/p>\n\n\n\n
Cons:<\/p>\n\n\n\n
Federated Data Model:<\/p>\n\n\n\n
Pros:<\/p>\n\n\n\n
Cons:<\/p>\n\n\n\n
So, which of the models is better? It\u2019s subjective and really depends on a user\u2019s situation. Each model offers a number of pros and cons, and act as different sides of the same coin. Federated data is lighter weight but can have latency issues, aggregated data takes up a lot of room but is typically faster \u2013 each one is similar but also very different, one model\u2019s pros are the others\u2019 cons.<\/p>\n\n\n\n
While, there may be different approaches to storing, distributing and accessing data, there\u2019s no single right way to do it as each have their benefits and shortfalls. Ultimately, it\u2019s about ensuring secure and timely access to the right information, wherever that data may reside.<\/p>\n\n\n\n
The market is starting to demand that large-scale data platforms incorporate the best aspects of both models. To capitalise on this, developers need to incorporate ideas from both models to truly meet the needs of their users both now and in the future.<\/p>\n\n\n\n