A day in life of datawarehouse Consultant

Consultant Analyses the business deeper to come up with star-schema design and further ETL load design,
Working as datawarehouse consultant most important task is to fix granularity of fact across dimensions to be analysed in FACT-DIMENSION Star schema design.
Granularity depends on business requirement and key drivers for business to be analysed for having its impact on Topline and Bottomline of Company. For Clinical Research key driver is No. of patient Enrolled, For banking key driver is cost of adding new customer,
Now patient is analyzed across geography dimension, against time dimension. But at what level of Granularity.
(#no of patient, day)   OR
(#no of patient, year)  OR
(#no of patient, hour)
This depends on business need and level of criticality to time. For Stock trading Every second is crucial but not for clinical trails but if trial involve enrollment of large public it may required a drill down to per day figure in BI reports hence provisions must be there in star schema.
Besides this The Other task per day can be taken based on stage of project
https://sandyclassic.wordpress.com/2014/02/19/a-day-in-life-of-datawarehouse-architect-part-1/
For datawarehouse Engineer involved with task the day look like
https://sandyclassic.wordpress.com/2014/02/19/a-day-in-life-of-datawarehousing-engineer/
For Unstructured data analysis you can look at
https://sandyclassic.wordpress.com/2011/10/19/hadoop-its-relation-to-new-architecture-enterprise-datawarehouse/

Then data Transformation are applied for
Example in Informatica and SSIS:

https://sandyclassic.wordpress.com/2014/01/15/eaten-tv-from-partly-eaten-apple-part-2-artificial-intelligence/

Two Sets of documents are There LLD and HLD to look at what needs transformation to be applied.
Like in Informatica Transformation Types are :

http://www.techtiks.com/informatica/beginners-guide/transformations/transformation-types/

Look at all transformations available in Informatica version 9

http://www.folkstalk.com/2011/12/transformations-in-informatica-9.html

These can be customized according to logic required.
Next step is Loading to datawarehouse dimension tables  and then to Fact table.
Read: https://sandyclassic.wordpress.com/2014/02/06/coke-vs-pepsi-of-datawarehousing-etl-vs-elt/
And more

https://sandyclassic.wordpress.com/2013/07/02/data-warehousing-business-intelligence-and-cloud-computing/

A Day in Life of Business Intelligence (BI) Architect- part 1

BI Architect most important responsibility is maintaining semantic Layer between Datawarehouse and BI Reports.
There are basically Two Roles of Architect: BI Architect or ETL Architect in data warehousing and BI. (ETL Architect in Future posts).
Semantic Layer Creation
Once data-warehouse is built and BI reports Needs to created. Then requirement gathering phase HLD High level design and LLD Low Level design are made.
Using HLD and LLD BI semantic layer is built in SAP BO its called Universe, in IBM Cognos using framework manager create Framework old version called catalogue, In Micro strategy its called project.
Once this semantic layer is built according to report data SQL requirements.
Note: Using semantic layer saves lot of time in adjustment of changed Business Logic in future change requests.
Real issues Example: Problems in semantic Layer creation like in SAP BO: Read
https://sandyclassic.wordpress.com/2013/09/18/how-to-solve-fan-trap-and-chasm-trap/
Report Development:
Reports are created using objects created by semantic layer.Complex reporting requirement for
1. UI require decision on flavour of reporting Tool like within
There are sets of reporting tool to choose from Like in IBM Cognos choose from Query Studio, Report Studio, Event Studio, Analysis Studio, Metric Studio.
2. Tool modification using SDK features are not enough then need to modify using Java/.net of VC++ API. At html level using AJAX javascript API or integrating with 3rd party API.
3. Report level macros/API for better UI.
4. Most important is data requirement my require Coding procedure at database or consolidations of various databases. Join Excel data with RDBMS and unstructured data using report level features. Data features may be more complex than UI.
5. user/data level security,LDAP integration.
6. Complex Scheduling of reports or bursting of reports may require modification using rarely Shell script or mostly Scheduling tool.
List is endless
Read More:
details of
https://sandyclassic.wordpress.com/2014/01/26/a-day-in-life-of-bi-engineer-part-2/

Integration with Third party and Security

After This BI’s UI has to fixed to reflect customer requirement. There might be integration with other products and seamless integration of users By LDAP. And hence Objects level security, User level security of report data according to User roles.
Like a Manager see report with data The same data may not be visible to clerk when he sees same report. Due filtering of data by user roles using User Level security.

BI over Cloud
setting BI over cloud Read blog.
Cloud Computing relation to Business Intelligence and Datawarehousing

Read :
1. https://sandyclassic.wordpress.com/2013/07/02/data-warehousing-business-intelligence-and-cloud-computing/

2. https://sandyclassic.wordpress.com/2013/06/18/bigdatacloud-business-intelligence-and-analytics/

Cloud Computing and Unstructured Data Analysis Using
Apache Hadoop Hive
Read: 
https://sandyclassic.wordpress.com/2013/10/02/architecture-difference-between-sap-business-objects-and-ibm-cognos/
Also it compares Architecture of 2 Popular BI Tools.

Cloud Data warehouse Architecture:
https://sandyclassic.wordpress.com/2011/10/19/hadoop-its-relation-to-new-architecture-enterprise-datawarehouse/

Future of BI
No one can predict future but these are directions where it moving in BI.
https://sandyclassic.wordpress.com/2012/10/23/future-cloud-will-convergence-bisoaapp-dev-and-security/

A day in Life of Software Architect – part 2

Read part 1:
https://sandyclassic.wordpress.com/2014/02/02/a-day-in-life-of-software-architect-part-1/
Work scheduled of a day depends on which phase projects are in currently. As typically architect work across multiple projects and help Project Manager PM aligning with organisation strategy and standards, reduce risk, liaising with stakeholder to full fill expectations develop blueprint for project.
different classes of architect have different skills and expectations:
Read More details: https://sandyclassic.wordpress.com/2011/09/25/enterprise-architecture-togafitilzachmanetomngoss/
Suppose a project is using a agile methodology: Then Architect day starts with.
Business Analyst gather requirement from clients, Architect work with them in putting technical prospective to each business Case.
In case of software services company Architect Even help sales team in estimations during bidding process like total man hours required for the projects under bidding.
Architect Also work with domain specialist along with business Analyst to find implications of changes in business environment, regulations, etc and its affect on software implemented or under implementations.
Architect Help Higher management in giving directions to roadmap of implementions or future implications on use of lets suppoe third party Charts in our BI product (license fees has to given for each chart (Make or Buy decision).Other options is make yourself it may take time (speed vs flexibility : Time to Market is crucial). Making takes time but gives flexibility, IP intellectual property, and may reduce cost in long term But Buying already developed third party may not give flexibility but can give speed and reduce cost in short term.
Adjusting solution to landscape of Client a Telecom case study.
Read This case Study for more details of similar process in telecom Stack development:
https://sandyclassic.wordpress.com/2013/10/26/telecom-technology-stack/

What happens At Enterprise Architect level ? see this presentation:
https://sandyclassic.wordpress.com/2013/02/28/498/

So A day typically depends on stage of projects like (initiation, development, implementation or maintenance), Domain of projects (insurance, finance, healthcare) and Technology used (J2EE, ERP (peoplesoft, oracle apps, SAP), .net, COBOL etc. data warehousing and BI).
Example Peoplesoft ERP Architect
e.g. Peoplesoft architect is using already dilivered set of programs, reports, table, forms, processes etc Has to take decisions Based on GAP Analysis. Plan ERP customisation either using peoplecode for UI and Forms or (SQR,nvision, Crystal reports) for report customisation, Or Administrative tools for customizing background processes.
Then plan for implementation of say suppose customising supply chain module to use some KABAN process or customising General ledger to introduce new regulation of mandatory approvals and Archiving records Or customising payroll according to country specific labour laws and company specific compensation policies.
Once functional Experts points what changes are needed Peoplesoft Architect has to draw a map immediately what Changes would be required in technology.
Then work along with team on implementations, stabilization of ERP and further supporting the ERP along with team figure out technical challenges and POC alternative solutions by using different levels of tools/technology in ERP or by integrating third party products.
To be continued…..