Data Integration , map Reduce algorithm , virtualisation relation and trends

In year 2011 This reply i did to a discussion. would later structure it into proper article.

As of 2010 data virtualization had begun to advance ETL processing. The application of data virtualization to ETL allowed solving the most common ETL tasks of data migration and application integration for multiple dispersed data sources. So-called Virtual ETL operates with the abstracted representation of the objects or entities gathered from the variety of relational, semi-structured and unstructured data sources. ETL tools can leverage object-oriented modeling and work with entities’ representations persistently stored in a centrally located hub-and-spoke architecture. Such a collection that contains representations of the entities or objects gathered from the data sources for ETL processing is called a metadata repository and it can reside in memory[1] or be made persistent. By using a persistent metadata repository, ETL tools can transition from one-time projects to persistent middleware, performing data harmonization and data profiling consistently and in near-real time.

———————————————————————————————————————————————-
– More then colmunar databases i see probalistic databases : link:http://en.wikipedia.org/wiki/Probabilistic_database

probabilistic database is an uncertain database in which the possible worlds have associated probabilities. Probabilistic database management systems are currently an active area of research. “While there are currently no commercial probabilistic database systems, several research prototypes exist…”[1]

Probabilistic databases distinguish between the logical data model and the physical representation of the data much like relational databases do in the ANSI-SPARC Architecture. In probabilistic databases this is even more crucial since such databases have to represent very large numbers of possible worlds, often exponential in the size of one world (a classical database), succinctly.

————————————————————————————————————————————————
For Bigdata analysis the software which is getting popular today is IBM big data analytics
I am writing about this too..already written some possible case study where and how to implement.
Understanding Big data PDF attached.
———————————————————————————————————————————————–
There are lot of other vendors which are also moving in products for cloud computing..in next release on SSIS hadoop feed will be available as source.
— Microstraegy and informatica already have it.
— this whole concept is based on mapreduce algorithm from google..There are online tutorials on mapreduce.(ppt attached)
—————————————————————————————————————————————–

Without a doubt, data analytics have a powerful new tool with the “map/reduce” development model, which has recently surged in popularity as open source solutions such as Hadoop have helped raise awareness.

Tool: You may be surprised to learn that the map/reduce pattern dates back to pioneering work in the 1980s which originally demonstrated the power of data parallel computing. Having proven its value to accelerate “time to insight,” map/reduce takes many forms and is now being offered in several competing frameworks.

If you are interested in adopting map/reduce within your organization, why not choose the easiest and best performing solution? ScaleOut StateServer’s in-memory data grid offers important advantages, such as industry-leading map/reduce performance and an extremely easy to use programming model that minimizes development time.

Here’s how ScaleOut map/reduce can give your data analysis the ideal map/reduce framework:

Industry-Leading Performance

  • ScaleOut StateServer’s in-memory data grids provide extremely fast data access for map/reduce. This avoids the overhead of staging data from disk and keeps the network from becoming a bottleneck.
  • ScaleOut StateServer eliminates unnecessary data motion by load-balancing the distributed data grid and accessing data in place. This gives your map/reduce consistently fast data access.
  • Automatic parallel speed-up takes full advantage of all servers, processors, and cores.
  • Integrated, easy-to-use APIs enable on-demand analytics; there’s no need to wait for batch jobs.

oracle interview questions

Oracle Interview Question covering
Oracle Interview Questions covering. 
Oracle Architecuture 
Oracle SQL 
Oracle PL/SQL 
performance tuning

Oracle discoverer vs sap business objects

BI Tool Comparision, 
Oracle Discoverer Vs SAP Business Objects 
Semantic Layer comparision
SAP BO universe Vs oracle discoverer EUL,
oracle discoverer end user layer vs sap business objects Universe,
semantic layer comparision

 

Informatica Tranformation Example

Informatica Tranformation Example  For Peoplesoft ERP HR data.
1. Informatica Expression (Source to Target) 
2. Source Qualifier transformation 
3. Expression Transformation 
4. Informatica Language example use of IFF syntax. 
5. performance optimisation using Oracle DECODE function inside Expression transformation 
6. Hexadecimal to Decimal Conversion Logic

A day in Life of ETL Consultant

Consultant Analyses the business deeper to come up with star-schema design and further ETL load design,
Working as datawarehouse consultant most important task is to fix granularity of fact across dimensions to be analysed in FACT-DIMENSION Star schema design.
Granularity depends on business requirement and key drivers for business to be analysed for having its impact on Topline and Bottomline of Company. For Clinical Research key driver is No. of patient Enrolled, For banking key driver is cost of adding new customer,
Now patient is analyzed across geography dimension, against time dimension. But at what level of Granularity.
(#no of patient, day)   OR
(#no of patient, year)  OR
(#no of patient, hour)
This depends on business need and level of criticality to time. For Stock trading Every second is crucial but not for clinical trails but if trial involve enrollment of large public it may required a drill down to per day figure in BI reports hence provisions must be there in star schema.
Besides this The Other task per day can be taken based on stage of project
https://sandyclassic.wordpress.com/2014/02/19/a-day-in-life-of-datawarehouse-architect-part-1/
For datawarehouse Engineer involved with task the day look like
https://sandyclassic.wordpress.com/2014/02/19/a-day-in-life-of-datawarehousing-engineer/
For Unstructured data analysis you can look at
https://sandyclassic.wordpress.com/2011/10/19/hadoop-its-relation-to-new-architecture-enterprise-datawarehouse/

Then data Transformation are applied for
Example in Informatica and SSIS:

https://sandyclassic.wordpress.com/2014/01/15/eaten-tv-from-partly-eaten-apple-part-2-artificial-intelligence/

Two Sets of documents are There LLD and HLD to look at what needs transformation to be applied.
Like in Informatica Transformation Types are :

http://www.techtiks.com/informatica/beginners-guide/transformations/transformation-types/

Look at all transformations available in Informatica version 9

http://www.folkstalk.com/2011/12/transformations-in-informatica-9.html

These can be customized according to logic required.
Next step is Loading to datawarehouse dimension tables  and then to Fact table.
Read: https://sandyclassic.wordpress.com/2014/02/06/coke-vs-pepsi-of-datawarehousing-etl-vs-elt/
And more

https://sandyclassic.wordpress.com/2013/07/02/data-warehousing-business-intelligence-and-cloud-computing/

A day in life of datawarehouse Consultant

Consultant Analyses the business deeper to come up with star-schema design and further ETL load design,
Working as datawarehouse consultant most important task is to fix granularity of fact across dimensions to be analysed in FACT-DIMENSION Star schema design.
Granularity depends on business requirement and key drivers for business to be analysed for having its impact on Topline and Bottomline of Company. For Clinical Research key driver is No. of patient Enrolled, For banking key driver is cost of adding new customer,
Now patient is analyzed across geography dimension, against time dimension. But at what level of Granularity.
(#no of patient, day)   OR
(#no of patient, year)  OR
(#no of patient, hour)
This depends on business need and level of criticality to time. For Stock trading Every second is crucial but not for clinical trails but if trial involve enrollment of large public it may required a drill down to per day figure in BI reports hence provisions must be there in star schema.
Besides this The Other task per day can be taken based on stage of project
https://sandyclassic.wordpress.com/2014/02/19/a-day-in-life-of-datawarehouse-architect-part-1/
For datawarehouse Engineer involved with task the day look like
https://sandyclassic.wordpress.com/2014/02/19/a-day-in-life-of-datawarehousing-engineer/
For Unstructured data analysis you can look at
https://sandyclassic.wordpress.com/2011/10/19/hadoop-its-relation-to-new-architecture-enterprise-datawarehouse/

Then data Transformation are applied for
Example in Informatica and SSIS:

https://sandyclassic.wordpress.com/2014/01/15/eaten-tv-from-partly-eaten-apple-part-2-artificial-intelligence/

Two Sets of documents are There LLD and HLD to look at what needs transformation to be applied.
Like in Informatica Transformation Types are :

http://www.techtiks.com/informatica/beginners-guide/transformations/transformation-types/

Look at all transformations available in Informatica version 9

http://www.folkstalk.com/2011/12/transformations-in-informatica-9.html

These can be customized according to logic required.
Next step is Loading to datawarehouse dimension tables  and then to Fact table.
Read: https://sandyclassic.wordpress.com/2014/02/06/coke-vs-pepsi-of-datawarehousing-etl-vs-elt/
And more

https://sandyclassic.wordpress.com/2013/07/02/data-warehousing-business-intelligence-and-cloud-computing/

Architecture sap hana vs oracle exadata competitive analysis part -2

READ part 1:
https://sandyclassic.wordpress.com/2011/11/04/architecture-and-sap-hana-vs-oracle-exadata-competitive-analysis/
This debate of SAP Vs Oracle or last 2 yrs buzz SAP HANA vs Oracle Exalytics
Every year in Enterprise Software space Competition of SAP Vs Oracle Hots up with new announcements and New technology comparisons of SAP new Tech Vs Oracle New Tech.
The Enterprise Software stack built up by No.1 and No.2  world’s top most valued companies in Enterprise Technology Space.
So SAP Vs Oracle comparison will never go out of picture only every year it will up the ante. 
In Memory Technology: SAP Vs Oracle (New tech now 2 yrs old).

Now a day In-Memory Technology is the Hottest area especially in Business Intelligence (BI). But its not limited to BI it goes into ERP, Application development of any Kind.
The processing infrastructure in form of in Memory Systems is much faster than any other form. As Cost of RAM is coming down and capacity of server is expanding So most data to be processed can be pulled inside memory at once instead of using Locality of reference to pull and process from secondary memory.
Also Server can hold lot more data to processed in memory at once.
SAP HANA Vs Oracle Exadata
So SAP Came with SAP HANA and Oracle has its own in memory systems. At same time Oracle released high performance machine Exadata which brook many performance records. So There was running comparisons of SAP Vs Oracle in Enterprise Technology space.
innovation at SAP : ABAP, BSP and BAPI 
SAP customized by using  Advanced Business Application Programming (ABAP) Language.
Using which reports are customized, forms are modified, business processes are written to reflect business Logic. ILE, BAPI, IDOC can be used to interface with external software or integration or developing Adaptor. Custom Exits and User Exits are written to customize forms and reports.
For SAP datawarehouse technology SAP BIW or SAP BI datawarehouse is created using
Extraction Legacy System Migration Workbench (LSMW) or Cost and profitability Analysis (COPA) Extrations then transformation can be applied using ABAP user exits.
Problem with ABAP is compared to mordern langauges it has not evolved much over time.
Most of ABAP constructs are similar to COBOL constructs. There is Object oriented ABAP also like JSP or ASP  it came up with BSP (Business Server Pages) to Expose ABAP code directly to Web like JSP does for Java or ASP does for Microsoft Technology.

SAP Netweaver  Vs SAP ABAP BSP
SAP began adopting Java in 2003 and came up with Netweaver product which was J2EE server for ABAP code. Now you can code using either java or ABAP in netweaver. For Application requiring functionality to exposed to Web Java was natural choice.
But Oracle having acquired Java in year 2010 Since then there was continuous Effort on part of SAP to move away from its dependence of Java.  So BSP came into picture as first step.
Benefits of In-memory in SAP HANA
next was paradigm shift using Columar database instead of Row oriented databases which consumed less storage (on account of reduction in repetitions of similar column values as well compression of data).
Also it integrates:
BI and datawarehousing system or OLAP with
Operational systems or OLTP Systems as one.
Even Analytic requiring lowest level of granularity can be queried on same server.
Effect of which was Whole data can be pulled and kept in In-memory system offering faster response time to multiple user connected at same time rather than
Regular database Query processing logic
parsing request , making parse tree,
comparing with already fetched query parse tree in cache
if not available
then
fetching data from secondary memory
when depending on the request into

Improvement in data processing in SAP HANA and column oriented database
Now since whole data can be kept in-memory So Every query can directly fetch data quicker.
Future Technology Like SAP HANA
Enhancement over this Technology are Probabilistic databases and Graph databases.
Graph databases are available commercially since long time.
Index free storage. Every element has direct pointer to adjacent element, hence no lookup needed.
Here is list:
http://en.wikipedia.org/wiki/Graph_database
And
Probabilistic databases : Are active area of research as discussed above as well.
http://en.wikipedia.org/wiki/Probabilistic_database

A day in life of Peoplesoft Functional Consultant part 1

Depending on ERP there can be many modules inside..
Major modules of peoplesoft ERP are :
1. Human Resource Management Systems HRMS
Submodules: (payroll, Core HR, benefits, recruitment, performance management)
2. Financials
(Account payable AP, Account Receivable AR, General Ledger GL etc..)
3. Supply chain Management SCM
4. Customer Relationship Management CRM

Functional consultant major work is during GAP Analysis To find As-IS state and identify work required to achieve TO-BE state.
Find all those forms which needs to be modified, and all those reports which needs to be modified And Exactly what field to change, What button to remove, What functionality to be disabled, What pre-delivered business logic behind components to be modified and What formulae to be used during modification.
There is Another Module which is consider Techno-functional Peopesoft EPM. EPM is datawarehouse part which has four major components for Each of four above there is a data mart for each area of finance, HR, SCM, CRM.
Each contains data from its own module transferred using ETL tools and reporting using BI Tools and analytics can be applied over formulae in BI systems.
-Also functional consultant decide based on feedback from local management to modify field like in HRMS a rejoining of employee should get ID as previous from old record or new ID based of policy in management.
if Payroll of local market like irish payroll or indian payroll does not exits then it should existing global payroll should be modified to achieve local labour laws and saving rules , income tax based customisation.

BI App development using Cognos SDK

Read previous article written before on Topic Link below:
Business Intelligence reports are generally created to un-structured or semi structured business Problems which covers Decision support system DSS, Management support system MIS.
To Know more : Read:  https://sandyclassic.wordpress.com/2013/01/31/strategic-information-systems-will-be-in-focus-again-next-5-yrs/

Business Intelligence Reporting solution can develop reports of varying requirements from Nowadays from operation to BI due to pervasive nature of BI to Exit at level of  Transaction processing System TPS at Knowledge level, to Office Automation system OAS at lowest operational level where operational reports about daily status is gerally used from ERP.
MIS Typical hierarchy of Information systems from Decision Support system at top to lowest Office automation system.
Customising Cognos Authentication mechanism for integration is first step see details

1. https://sandyclassic.wordpress.com/2014/03/08/authentication-using-cognos-java-sdk/

How to Customize Cognos to any specific non-existent customisation using SDK. Read
2. https://sandyclassic.wordpress.com/2014/03/08/cognos-software-development-kit/

If data is completely unstructured data which cannot be analysed by traditional BI system but requiring Hadoop, Hive , HBase then customisation has to integrate Big data system Read:
3. https://sandyclassic.wordpress.com/2011/10/26/big-data-and-data-integration/

BI App development using BO Java and ASP SDK

How to develop BI App using Java or ASP?
standard SDK tutorial presentation by Raphael Geoffroy, Marc Labouze Alastair Gulland’s bou training material.

A Day in Life of Datawarehouse Architect part 1

A data warehouse Architect generally help to design datawarehouse , requirement gathering in ETL Low level design LLD, and HLD high level design, setting up database infrastructure design for datawarehouse like Storage Area Network requirements, Rapid application Clusters for database of datawarehouse more details read
Datawarehousing consists of three main area :
1. ETL(data migration, data cleansing, data scrubbing, data loading )
2. Datawarehouse design
3. Business Intelligence (BI) Reporting infrastructure.
BI
Read These Two part article for BI
– https://sandyclassic.wordpress.com/2014/01/26/a-day-in-life-of-bi-engineer-part-2/
– https://sandyclassic.wordpress.com/2014/01/26/a-day-in-life-of-business-intelligence-engineer/
And Architect
https://sandyclassic.wordpress.com/2014/02/02/a-day-in-life-of-business-intelligence-bi-architect-part-1/

Design : Now Coming to part 2 (is generally work of Data warehouse architect)
Read Some details More would be covered in future articles
https://sandyclassic.wordpress.com/2013/07/02/data-warehousing-business-intelligence-and-cloud-computing/
9:00-9:30 Read and reply mails.
9:30-10:30 Scrum Meeting
10:30-11:30 update documents According to Scrum meeting like burn down chart etc..update all stake holders.
11:30-12:00 Meeting with Client to understand new requirements. create/update design specification from requirement gathered.
12:00-13:30 create HLD/LLD from the required user stories according to customer Landscape of technology used.
13:30-14:00 Lunch Break.
14:00-14:30 Update the Estimations ,coding standards , best practises for project.
14:30-15-30 Code walk through update team on coding standards.
15-30-16:30Defect call with Testing and development Team to understand defects, reasons of defects, scope creep, defect issuse with defect manager, look at issue/defect register
16:30-17:30 Work on specification of Design of datawarehouse modelling Star or Snow flake schema design according to business requirements granularity requirements.
17:30-18:30 Look at Technical Challenges requiring Out of Box thinking, thought leadership issue, Proof of concept of leading Edge and Breeding Edge technologies fitment from project prospective.
18:30-19:30  onwards Code for POC and Look a ways of tweaking , achieving technology POC code.
19:30- 20:30 onwards Forward thinking issue might be faced ahead by using a particular technology is continuous never ending process as there can be multiple combination possible to achieve as well as using particular component or technology should not create vendor lock in, cost issues, make/buy cost decisions, usability, scalability, security issues (like PL/SQL injection, SQL injection using AJAX or web services may be affected by (XSS attack or web services Schema poisoning), Environmental network scalability issues. Affect due to new upcoming technology on Existing code.
20:30 Dinner
Available on Call.. for any deployment, production emergency problems.

A day in Life of datawarehousing Engineer Part-2

Read Previous part
https://sandyclassic.wordpress.com/2014/02/19/a-day-in-life-of-datawarehousing-engineer/
Normal Schedule for development role :
9:00-9:30 Check all mail communications of late night loads Etc.
9:30-10:30 Attend Scrum meeting to discuss update status of completed task mappings and mapping for New user stories requirements, understand big picture of work completed by other staff status.
10:30 am -1:30 pm Look at LLD, HLD to create source to target transformations after understanding business logic and coding that in transformations available with tool.
1:30-2:00 Lunch break
2:00-3:00 Unit test data set to validate as required between source and target.
3:00-3:30 Documentation requirements of completed work.
3:30-4:30 Attend defect Call To look into new defects in code and convey back if defects not acceptable as out of scope or not according to specifications.
4:30-5:00 Status update daily work to Team Lead.
5:00-5:30 sit with Team lead, architect code walk through and update with team.
5:30-6:30 Take up any defects raised in Defect meting and Code walk through.

A day in Life of datawarehousing Engineer Part-1

Datawarehousing consists of three main area :
1. ETL(data migration, data cleansing, data scrubbing, data loading )
2. Datawarehouse design
3. Business Intelligence (BI) Reporting infrastructure.
BI
Read These Two part article for BI
https://sandyclassic.wordpress.com/2014/01/26/a-day-in-life-of-bi-engineer-part-2/
– https://sandyclassic.wordpress.com/2014/01/26/a-day-in-life-of-business-intelligence-engineer/
And Architect
https://sandyclassic.wordpress.com/2014/02/02/a-day-in-life-of-business-intelligence-bi-architect-part-1/
Design : Now Coming to part 2 (is generally work of Datawarehouse architect)
Read Some details More would be covered in future articles
https://sandyclassic.wordpress.com/2013/07/02/data-warehousing-business-intelligence-and-cloud-computing/
Part 1: ETL Engineer:
Most common task of ETL (Extract data- Transform data -Load to target).
Most Common ETL Tool being
Independent Tool: Informatica, IBM data stage, Ab-initio, Terradata ETL utilities,
Tool within ERP: SAP BIW ABAP based transformations , LSMW, peoplesoft EPM (internally uses other tools though).
Tool within Databases: Oracle SQL loader, Teradata ETL utilities,(Tricle pump, multi-load, fast load),
Microsoft BI Stack with SQL server had : SSIS SQL server integration services.
Cloud based Tool: Apache Hadoop Hive datawarehouse ( here requirement is different from un-structured realtime data analysis.
First data modelling have to completed to have level of granularity to represent requirements of business key drivers.
Once datawarehouse Structure in completed to ascertain level of granularity required.
The data loading Cycle Starts With:
Extraction from desperate data sources in Stagging area
On Stagging area data is cleansed.
Then data Transformation are applied for
Example in Informatica and SSIS:
https://sandyclassic.wordpress.com/2014/01/15/eaten-tv-from-partly-eaten-apple-part-2-artificial-intelligence/
Two Sets of documents are There LLD and HLD to look at what needs transformation to be applied.
Like in Informatica Transformation Types are :

Informatica Transformation Types


Look at all transformations available in Informatica version 9
http://www.folkstalk.com/2011/12/transformations-in-informatica-9.html
These can be customized according to logic required.
Next step is Loading to datawarehouse dimension tables  and then to Fact table.
Read: https://sandyclassic.wordpress.com/2014/02/06/coke-vs-pepsi-of-datawarehousing-etl-vs-elt/
And more
https://sandyclassic.wordpress.com/2013/07/02/data-warehousing-business-intelligence-and-cloud-computing/

 

Case Study Artificial Intelligence,ETL and Datawarehousing Examples part 1

Read : IPTV and Augmented Reality using Artificial Intelligence.
https://sandyclassic.wordpress.com/2012/06/27/future-of-flex-flash-gamification-of-erp-enterprise-software-augmented-reality-on-mobile-apps-iptv/

AI is there in many place like one area of AI Fuzzy Set there is already Fuzzy Transformation in SQL Server Integration Services since year 2010.
What it does Fuzzy logic Transformation achieve?
So when we match two records we do it by checking each alphabet using regular matches.
But when we use fuzzy logic it brings out similar sounding and combination matches although alphabet may not be same also it checks meaning is same. Even it can override spelling mistakes to get right results How?
Example Fuzzy logic in SSIS:
USA,us, united states – For country Any person can enter any of these combination.
Usually its taken up for Data cleansing.
If data is not cleaned using De-dup it may not show many of these records in result for matches.
But Fuzzy logic we use Fuzzy set from all records it creates fuzzy set of record with
Set A { ElementA, membershipOfElementA}
membershipOfElementA define in percentage terms the possibility of it being in the similarly grouped set.
{us,0.97} {united states,0.98} {usa,0.99} {united states of America,1} so we can set tolerance level to 3% then all of these matches are there in result.
code you can see at http://www.codeproject.com/Tips/528243/SSIS-Fuzzy-lookup-for-cleaning-dirty-data
SIRI:  Speech Recognition Search Which was introduced in iPhone long back takes speech.
Speech input to pressure sensor –> generate Waveform –> Then Compare wave form
That’s process but.
AI in SIRI
The Waveform may be amplitude modulated but yet same thing let suppose we say
Apple the Two Waveform compared may have boundary level aberrations which can be defined by membership function Then same result within same Tolerance limit can be deemed to be similar. This membership can be calculated each time person do a search dynamically when it says something in on Mike which repeat same process again.
There can be lots of image processing and AI search algorithm can be built to make better.
Like A* search etc.
Already if the words are linked can be understood by Neural Network. Similar way Neural Network is used to predict The  traffic congestion aggregating data paths from street light sensors in japan Tokyo.
Aggregation of words can be achieved by neural Network in not exact but similar way to some Extent. Thus completing the search.
This aggregation may be used in text, covariance matrix of images or covariance of sound score or speech search.
Using Laplace Transform’s Cross correlation. Read (http://en.wikipedia.org/wiki/Cross-correlation512px-Comparison_convolution_correlation.svg
Now TV is large platform just like difference between watching movie on laptop or TV Vs on 70 mm screen. Each of those has there own market.
Costly Miniaturisation
What effect you can provide on TV may not be provided on mobile until there is technical break through in miniaturisation. I am not saying it cannot be provided but it will require relative less technical  break through compared with miniaturized chips or may be less costly.
Second TV is like we have last mile connectivity in Telecom.
So When you have something to watch in any storage device you can just throw that on TV Ubiquitously . As TV would be there in every house so you need not carry screen to watch. Just like Last mile wireless connectivity using HotSpot.

Coke Vs Pepsi of :Datawarehousing ETL Vs ELT

The Coke and Pepsi are always fighting to have bigger pie in international drinks market.
Both are present in 180+ countries agressively pursuing the pie of market share.

The Datwarehouses are different animals on block. They are databases But they are not normalized. They do not follow all 12 Codd Rules. But yet source and Target are RDBMS.
The Structure Where its saved Whether in Star Schema or Snow-flake is denormalized as possible like flat file structures. More Constraints slows down the join process.
Read more: https://sandyclassic.wordpress.com/2014/01/26/a-day-in-life-of-business-intelligence-engineer/
So there are less restrained much Faster file based alternatives for databases which Emerged for need to store unstructured data and achieve 5V massive volume, variety, velocity etc.. Read below links:
https://sandyclassic.wordpress.com/2013/07/02/data-warehousing-business-intelligence-and-cloud-computing/
Which are have also found favour in ETL world with Hadoop. Now Every ETL allows hadoop connector or adapter to Extract data from hadoop HDFS so service in HDFS and similar.
https://sandyclassic.wordpress.com/2013/06/18/bigdatacloud-business-intelligence-and-analytics/
(
Adapters use-case for  product offering Read:https://sandyclassic.wordpress.com/2014/02/05/design-pattern-in-real-world/)
ETL process

ETL Extract-Transform-Load
ETL where transformation happens in staging area.
Extract data from sources , put in staging area cleanse it, transform data and Then Load in Target Datawarehouse. So popular Tools like informatica, datastage or ab-initio use this approach. Like in Informatica for fetching data or Extract Phase we can use fast source-qualifier transformation OR use Joiner transformation when we have multiple different databases like both SQL Server and Oracle although may be slow but can take both input but Source qualifier may require single vendor but is fast.
After Extracting We can use Filter transformation to filter out unwanted rows in staging area. Then load into target Databases.

ELT Extract Load and then Transform.
Extract data from disparate sources , Load the data into RDBMS engine first after . Then use RDBMS facility to Cleanse and Transform Data. This Approach was popularised By Oracle because Oracle Already had Database Intellectual property and was motivated to increase its usage.So Why does cleansing and Transformation outside the RDBMS into staging area rather within RDBMS engine. Oracle ODI Oracle Data integrator uses this concept of ELT not ETL bit reversal from routine.

So like Pepsi Vs Cola wave of Advertisement and gorilla marketing or To showcase Each other products strengths and hide weakness Games continue here Also in ETL world of data warehousing. Each one has its own merits and demerits.

Cloud Computing relation to Business Intelligence and Datawarehousing
Read :
1. 
https://sandyclassic.wordpress.com/2013/07/02/data-warehousing-business-intelligence-and-cloud-computing/
2.
 https://sandyclassic.wordpress.com/2013/06/18/bigdatacloud-business-intelligence-and-analytics/

Cloud Computing and Unstructured Data Analysis Using
Apache Hadoop Hive
Read: 
https://sandyclassic.wordpress.com/2013/10/02/architecture-difference-between-sap-business-objects-and-ibm-cognos/
Also it compares Architecture of 2 Popular BI Tools.

Cloud Data warehouse Architecture:
https://sandyclassic.wordpress.com/2011/10/19/hadoop-its-relation-to-new-architecture-enterprise-datawarehouse/

Future of BI
No one can predict future but these are directions where it moving in BI.
https://sandyclassic.wordpress.com/2012/10/23/future-cloud-will-convergence-bisoaapp-dev-and-security/

SAP Business Objects reporting Errors and resolutions

#multivalue
usually when section is created on a column having multiple values for respective data set.
Solution : By looking why multiple values are coming issue can be resolved.
#error:
#compute : Computation Error
comes due to Errors in formulae and Also due to objects used in computation not present in data block.
Solution 1: if summary data is needed from detailed block then keep copy of objects and use Fold at break level in Desktop reporting. But this option is not present in Web Intelligence WebI Reports XI but version XI Release 3 onwards introduced again in WebI.

#sync
Synchronisation between data provider.
List of Error Codes:
http://help.sap.com/businessobject/product_guides/boexir3/en/xi3_error_message_guide_en.pdf

Universe Development issue:
https://sandyclassic.wordpress.com/2013/09/18/how-to-solve-fan-trap-and-chasm-trap/