Bigdata,cloud , business Intelligence and Analytics

There huge amount of data being generated by BigData Chractersized by 3V (Variety,Volume,Velocity) of different variety (audio, video, text, ) huge volumes (large video feeds, audio feeds etc), and velocity ( rapid change in data , and rapid changes in new delta data being large than existing data each day…) Like facebook keep special software which keep latest data feeds posts on first layer storage server Memcached (memory caching) server bandwidth so that its not clogged and fetched quickly and posted in real time speed the old archive data stored not in front storage servers but second layer of the servers.
Bigdata 3V characteristic data likewise stored in huge (Storage Area Network) SAN of cloud storage can be controlled by IAAS (infrastucture as service) component software like Eucalyptus to create public or private cloud. PAAS (platform as service) provide platform API to control package and integrate to other components using code. while SAAS provide seamless Integration.
Now Bigdata stored in cloud can analyzed using hardtop clusters using business Intelligence and Analytic Software.
Datawahouse DW: in RDBMS database to in Hadoop Hive. Using ETL tools (like Informatica, datastage , SSIS) data can be fetched operational systems into data ware house either Hive  for unstructured data or RDBMS for more structured data.

BI over cloud DW: BI can create very user friendly intuitive reports by giving user access to layer of SQL generating software layer called semantic layer which can generate SQL queries on fly depending on what user drag and drop. This like noSQL and HIVE help in analyzing unstructured data faster like data of social media long text, sentences, video feeds.At same time due to parallelism in Hadoop clusters and use of map reduce algorithm the calculations and processing can be lot quicker..which is fulling the Entry of Hadoop and cloud there.
Analytics and data mining is expension to BI. The social media data mostly being unstructured and hence cannot be analysed without categorization and hence quantification then running other algorithm for analysis..hence Analytics is the only way to get meaning from terabyte of data being populated in social media sites each day.

Even simple assumptions like test of hypothesis cannot be done with analytics on the vast unstructured data without using Analytics. Analytics differentiate itself from datawarehouse as it require much lower granularity data..or like base/raw data..which is were traditional warehouses differ. some provide a workaround by having a staging datawarehouse but still  data storage here has limits and its only possible for structured data. So traditional datawarehouse solution is not fit in new 3V data analysis. here new Hadoop take position with Hive and HBase and noSQL and mining with mahout.

18 Comments

  1. Pingback: Next generation Application developement | sandyclassic

  2. Pingback: Design and Analysis of Algorithm- Practical Example | sandyclassic

  3. Pingback: A day in Life of Business Intelligence Engineer | sandyclassic

  4. Pingback: A Day in Life of Business Intelligence (BI) Architect- part 1 | sandyclassic

  5. Pingback: Coke Vs Pepsi of :Datawarehousing ETL Vs ELT | sandyclassic

  6. Pingback: A Day in Life of Business Intelligence (BI) Architect- part 1 | sandyclassic

  7. Pingback: A Day in Life of Business Intelligence (BI) Architect- part 1 | Business Intelligence Technology Trends

  8. Pingback: Coke Vs Pepsi of :Datawarehousing ETL Vs ELT | Datawarehouse View

  9. Pingback: A Day in Life of Business Intelligence (BI) Architect- part 1 | Datawarehouse View

  10. Pingback: A day in Life of Business Intelligence Engineer | Datawarehouse View

  11. Pingback: Next generation Application development | The Security View

  12. Pingback: Use Case: Big data Analytics: Cisco,Machine to Machine,image processing,economics | Trends from Data Science

  13. Pingback: Architecting analysis of Structured and unstructured data | sandyclassic

  14. Pingback: Data war noSQL vs SQL vs NewSQL | Datawarehouse View

  15. Pingback: Data war SQL vs noSQL vs newSQL | Data Architect View

  16. Pingback: CLoud war: SQL vs noSQL vs newSQL | The Cloud View

  17. Pingback: Cloud BI/DW war : SQL vs noSQL vs newSQL | Business Intelligence Technology Trends

  18. Pingback: RDBMS war: SQL vs noSQL vs newSQL | sandyclassic

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s