“Why create a data warehouse in the cloud?” I have been answering this question many a times when I go to networking events.
There are a number of cloud data warehousing advantages:
- Cost savings
- Total cost of ownership
- Mergers and acquisitions
- Better understanding of competitive landscape
- Improving product and service quality.
Many enterprises are finding that data warehouse appliances can significantly reduce the total cost of ownership. All resources, including expensive networking equipment, servers, IT personnel, etc are shared, resulting in reduced costs, especially for small to mid-sized applications and prototypes.
In the case of mergers and acquisitions, duplicate financial data warehouses need to be rationalized. An enterprise or government agency may want to rationalize various warehouses based on disparate warehouse technology platforms by moving to a standard platform. Cloud computing enables companies to shift money from capital expenses to operating expenses, enabling the customer to focus on adding value in their areas of core competence, such as business and process insight, instead of building and maintaining IT infrastructure.
In short, cloud computing allows you to focus your money and resources on innovating. “Regardless of the reason for the migration, in every case the reporting and analysis supported by the migrating data warehouse must continue to run seamlessly.” This is a common need expressed by my professional network. Cloud computing platforms such as Microsoft Azure, provide many of the core services that, under traditional development models, would normally be built in house. These services, plus templates and other tools can significantly accelerate the development cycle.
“What should be the solution?” is the subsequent question that arises during conversation. “Use data visualization to insulate reporting users during data warehouse migrations” is my to-the-point answer. Additionally, data virtualization to the cloud provides a more agile integration approach that overcomes data complexity and disparate silos to provide business with the timely data it needs to meet today’s ever-changing business requirements. Provisioning-on-demand enables faster set-up and tear-down of resources on an as-needed basis. When a project is funded, you initiate service, then if the project is killed, you simply terminate the cloud contract.
Data Virtualization using cloud computing is the opposite of traditional data warehousing. The cloud is more vibrant and valuable in that it’s something elastic that customers can scale up or down with dynamic resource allocation ensuring efficient use of shared processing and storage resources. It wants data to be location independent, transparent and function shippable; whereas, the traditional data warehouse is a centralized, persistent data store. There will be a need for a run-time metadata in order to register and access data sources as a service. Growing data volumes are winning. Still, with cloud computing (as with web services), the service, not the database, is the primary data integration method.
The ability of the cloud to load data quickly allows the vendor to work with larger data sets during smaller time windows as dictated by the customer, as well as service more customers at any one time. While smaller companies may only refresh the data in their data mart weekly, larger customers typically refresh data daily. Moreover, the transactional database of such larger customers can only stay offline for short time periods during which data must be transferred to the data marts.
Data loaded in the data warehouse has to be organized in the best possible way to enable the optimal execution of queries. The managers of on-premise data warehouses and marts constantly look at the queries executed against their databases to determine how to best organize the data to achieve best query execution times. Some of these optimizations can be performed automatically by the database management system but most require manual intervention. Having full control of the database management system enables the SaaS BI vendor to better optimize the organization of the stored data.
Finally, the emotion of the language used to query the data mart in the cloud determines the range of data analyses that can be performed and reports that can be created by the business intelligence tools (like Microsoft SSRS). Cloud infrastructure also contributes to the speed with which queries are executed that generates quick analytics reports.
As a business intelligence vendor in Vancouver, OptimusAnalytics would be happy to share our experiences with you. To learn more about cloud based data warehouse solutions, contact me directly at firstname.lastname@example.org
(image credit: Goma on Flickr)