Tuesday, 24 March 2015

Multiple Source Data : Conversation of Consolidation


Recently one of client came up with a requirement which was not uncommon in the projects we do. The nature of the situation prompted me to write this post which could be of use to many of you out there.  




Requirement :

So this client of ours is a Manager who is interested in Reports of the various performance dimensions of his supply chain team. Currently the data comes from different sources, like share point sites, spreadsheets, flat files, Database, etc. And this data is collected using HTML/JS compilation and stored in the spreadsheet using VBA summarized views. Now the customer is wondering whether to go for SQL to have centralized data collection and more fool proof reporting.

Our Suggestion:

The above is the model we suggested for the client. There could be two different scenario based on the type of ccustomizationthat is needed to extract data. If the data extraction process is straight forward we can use SSIS. SSIS is a platform for data integration and workflow applications. It features a fast and flexible data warehousing tool used for data extraction, transformation, and loading (ETL). 


On the other hand, PsiberTech tool Tokenizer can be used specifically for the customized Data Collection. Tokenizer is a data extraction; transformation and monitoring tool that allows the extraction of data from file formats like flat files, databases, and excel files including un-formatted files. 

Just by configuring few XML files you will be able to extract data from a source, apply your business logics and then transform and load the data into a destination in your desired format.  Assuming your source data is from a text file and you want to apply certain business rules before the data is cleansed, transformed and loaded into your destination, let’s say a SQL database table; Tokenizer is the right tool to perform this process. 

One killer feature about Tokenizer is that it allows automation of data import or transformation instantaneously, and can perform additional functions such as emailing and FTPing files. Now once the data is collected it can be stored in the central SQL Database and can be further processed using Data Mining to create OLAP cubes. Data Mining is an analytic process designed to explore Data (usually large amounts of data - typically business or market related ones like our customers. Data mining automates the process of finding predictive information in a large database.Using Data mining we can establish sophisticated mathematical algorithms to segment the data and evaluate the probability of future eventsUsing this Data Mining Data is then fed into OLAP. OLAP is a set of operations that one can do on a data set, for example pivoting, slicing, dicing, drilling. This is particularly useful to gain more insights into the data collected.

This is where PsiberTech Voyager comes in. Voyager IIG  allows the users to extract business information  from disconnected backend systems and turn that into meaningful Charts, Graphs, Gauges, Maps and tables. This easy to use tool allows interactive analysis of large amounts of  data from many perspectives.Users can analyze data, perform ad-hoc queries, and create their own reports.Voyager reports can be published to any Web portal including  SharePoint Portal.

I hope this post has helped to understand the Data extraction from multiple sources.Also take a look at the 3 Common Issues of Data Warehouse projectIts important to engage the right consultant to help you Pre plan, Analyse, and Implement the Data Warehouse Solution with reduced effort and disruption to users. Contact us to know more about Data Warehouse and how it can benefit your organisation. Thanks for dropping by and Have a fantastic day!

Best
Srivats



Srivatsan Aravamudan
Srivatsan Aravamudan

Public Relations and Social Media Manager

Srivatsan Aravamudan
General Enquiries:info@psibertech.com.sg
Sales Enquiries:sales@psibertech.com.sg
Telephone:+65 62689551
Fax:+65 62689858
Public Relations and Social Media Manager

Related Posts Plugin for WordPress, Blogger...

Share Buttons