Group items tagged

1. Project Sponsorship and Governance 2. Project Management 3. Development Team (Core Team) 4. Extended Project Team

1. Project Sponsorship and Governance IT and the business should form a BI steering committee to sponsor and govern design, development, deployment, and ongoing support. It needs both the CIO and a business executive, such as CFO, COO, or a senior VP of marketing/sales to commit budget, time, and resources. The business sponsor needs the project to succeed. The CIO is committed to what is being built and how.

2. Project Management Project management includes managing daily tasks, reporting status, and communicating to the extended project team, steering committee, and affected business users. The project management team needs extensive business knowledge, BI expertise, DW architecture background, and people management, project management, and communications skills. The project management team includes three functions or members: Project development manager - Responsible for deliverables, managing team resources, monitoring tasks, reporting status, and communications. Requires a hands-on IT manager with a background in iterative development. Must understand the changes caused by this approach and the impact on the business, project resources, schedule and the trade-offs. Business advisor - Works within the sponsoring business organization. Responsible for the deliverables of the business resources on the project's extended team. Serves as the business advocate on the project team and the project advocate within the business community. Often, the business advocate is a project co-manager who defers to the IT project manager the daily IT tasks but oversees the budget and business deliverables. BI/DW project advisor - Has enough expertise with architectures and technologies to guides the project team on their use. Ensures that architecture, data models, databases, ETL code, and BI tools are all being used effectively and conform to best practices and standards.

Mistake 1.  Not putting a strong interdisciplinary team together. It is impossible to put together an analytics platform without understanding the needs of the customers who will use it.  Sounds simple, right?  Who wouldn’t do that?  You’d be surprised how many analytics projects are wrapped up by IT because “they think” they know the customer needs.  Not assembling the right team is clearly the biggest mistake companies make.  Many times what is on your mind (and if you’re an IT person willing to admit it) is that you are considering converting all those favorite company reports.  Your goal should not be that.  Your goal is to create a system—human engineered with customers, financial people, IT folks, analysts, and others—that give people new and exciting ways to look at information.  It should give you new insights. New competitive information.  If you don’t get the right team put together, you’ll find someone longing for the good old days and their old dusty reports.  Or worse yet, still finding ways to generate those old dusty reports. Mistake 2.  Not having the right talent to design, build, run and update your analytics system.  It is undeniable that there is now high demand for business analytics specialists.  There are not a lot of them out there that really know what to do unless they’ve been burned a few times and have survived and then built successful BA systems.  This is reflected by the fact you see so many analytics vendors offer, or often recommend, third-party consulting and training to help the organization develop their business analytic skills.  Work hard to build a three-way partnership between the vendor, your own team, and an implementation partner.  If you develop those relationships, risk of failure goes way down.

Mistake 3.  Putting the wrong kind of analyst or designer on the project. This is somewhat related to Mistake 2 but with some subtle differences.  People have different skillsets so you need to make sure the person you’re considering to put on the project is the right “kind.”  For example, when you put the design together you need both drill-down and summary models.  Both have different types of users.  Does this person know how to do both?  Or, for example, inexperience in an analyst might lead to them believing vendor claims and not be able to verify them as to functionality or time to implement. Mistake 4.  Not understanding how clean the data is you are getting and the time frame to get it clean.  Profile your data to understand the quality of your source data.  This will allow you to adjust your system accordingly to compensate for some of those issues or more importantly push data fixes to your source systems.  Ensure high quality data or your risk upsetting your customers.  If you don’t have a good understanding of the quality of your data, you could easily find yourself way behind schedule even though the actual analytics and business intelligence framework you are building is coming along fine. Mistake 5.  Picking the wrong tools.  How often do organizations buy software tools that just sit on the shelve?  This often comes from management rushing into a quick decision based on a few demos they have seen.  Picking the right analytics tools requires an in-depth understanding of your requirements as well as the strengths and weaknesses of the tools you are evaluating.  The best way to achieve this understanding is by getting an unbiased implementation partner to build a proof of concept with a subset of your own data and prove out the functionality of the tools you are considering. Bottom Line.  Think things through carefully. Make sure you put the right team together.  Have a data cleansing plan.  If the hype sounds too good to be true—have someone prove it to you.

What this means in practice is that when the BI Server component starts up, it creates and reserves a number of threads in advance, determined by a number of parameters including SERVER_THREAD_RANGE.

You can see these threads running and ready to perform tasks for the BI Server component by using a tool such as Process Explorer for Windows

Thinking it through a bit, any given single query is, to a certain extent, only really going to use a small part of the total amount of CPUs available on a server, because it’s not the BI Server that runs queries in parallel, it’s the underlying database. For example, a single analysis against a single Oracle Database datasource would only really need a single BI Server thread to handle the query request, but when the underlying database receives the query, it might use a large number of its CPUs to process the query, returning results back to the BI Server to then pass back to the Presentation Server for display to the user.

Integration BI infrastructure: All tools in the platform use the same security, metadata, administration, portal integration, object model and query engine, and should share the same look and feel. Metadata management: Tools should leverage the same metadata, and the tools should provide a robust way to search, capture, store, reuse and publish metadata objects, such as dimensions, hierarchies, measures, performance metrics and report layout objects. Development tools: The platform should provide a set of programmatic and visual tools, coupled with a software developer's kit for creating analytic applications, integrating them into a business process, and/or embedding them in another application. Collaboration: Enables users to share and discuss information and analytic content, and/or to manage hierarchies and metrics via discussion threads, chat and annotations.

Information Delivery Reporting: Provides the ability to create formatted and interactive reports, with or without parameters, with highly scalable distribution and scheduling capabilities. Dashboards: Includes the ability to publish Web-based or mobile reports with intuitive interactive displays that indicate the state of a performance metric compared with a goal or target value. Increasingly, dashboards are used to disseminate real-time data from operational applications, or in conjunction with a complex-event processing engine. Ad hoc query: Enables users to ask their own questions of the data, without relying on IT to create a report. In particular, the tools must have a robust semantic layer to enable users to navigate available data sources. Microsoft Office integration: Sometimes, Microsoft Office (particularly Excel) acts as the reporting or analytics client. In these cases, it is vital that the tool provides integration with Microsoft Office, including support for document and presentation formats, formulas, data "refreshes" and pivot tables. Advanced integration includes cell locking and write-back. Search-based BI: Applies a search index to structured and unstructured data sources and maps them into a classification structure of dimensions and measures that users can easily navigate and explore using a search interface. Mobile BI: Enables organizations to deliver analytic content to mobile devices in a publishing and/or interactive mode, and takes advantage of the mobile client's location awareness.

Analysis Online analytical processing (OLAP): Enables users to analyze data with fast query and calculation performance, enabling a style of analysis known as "slicing and dicing." Users are able to navigate multidimensional drill paths. They also have the ability to write back values to a proprietary database for planning and "what if" modeling purposes. This capability could span a variety of data architectures (such as relational or multidimensional) and storage architectures (such as disk-based or in-memory). Interactive visualization: Gives users the ability to display numerous aspects of the data more efficiently by using interactive pictures and charts, instead of rows and columns. Predictive modeling and data mining: Enables organizations to classify categorical variables, and to estimate continuous variables using mathematical algorithms. Scorecards: These take the metrics displayed in a dashboard a step further by applying them to a strategy map that aligns key performance indicators (KPIs) with a strategic objective. Prescriptive modeling, simulation and optimization: Supports decision making by enabling organizations to select the correct value of a variable based on a set of constraints for deterministic processes, and by modeling outcomes for stochastic processes.

Some of the Hadoop projects we will talk about are: HDFS : A distributed filesystem that runs on large clusters of commodity machines. Map Reduce: A distributed data processing model and execution environment that runs on large clusters of commodity machines. Pig: A data flow language and execution environment for exploring very large datasets. Pig runs on HDFS and MapReduce clusters. HBase: A distributed, column-oriented database. HBase uses HDFS for its underlying storage, and supports both batch-style computations using MapReduce and point queries (random reads). ZooKeeper: A distributed, highly available coordination service. ZooKeeper provides primitives such as distributed locks that can be used for building distributed applications. Oozie: Oozie is a workflow scheduler system to manage Apache Hadoop jobs.