Group items tagged

Business intelligence is a data analysis process aimed at boosting business performance by helping corporate executives and other end users make more informed decisions.

Business intelligence (BI) is a technology-driven process for analyzing data and presenting actionable information to help corporate executives, business managers and other end users make more informed business decisions.

BI encompasses a variety of tools, applications and methodologies that enable organizations to collect data from internal systems and external sources, prepare it for analysis, develop and run queries against the data, and create reports, dashboards and data visualizations to make the analytical results available to corporate decision makers as well as operational workers.

Schema on write This is the traditional approach for Business Intelligence. A model, often dimensional, is built as part of the design process. This model is an abstraction of the complexity of the underlying systems, put in business terms. The purpose of the model is to allow the business users to interrogate the data in a way they understand.

The model is instantiated through physical database tables and the date is loaded through an ETL (extract, transform and load) process that takes data from one or more source systems and transforms it to fit the model, then loads it into the model.

The key thing is that the model is determined before the data is finally written and the users are very much guided or driven by the model in how they query the data and what results they can get from the system. The designer must anticipate the queries and requests in advance of the user asking the questions.

Big Data has been all the rage. But fast data, even if it is small, can be more valuable than complicated masses of information.

Here's why: All the focus on "bigger is better" has overlooked the fact that most Big Data segments have not been validated with a business application or value.

Those kinds of analytics can help you find the right streams to access and work with, and also can help you build out robust programs that identify valuable customers.

Integration BI infrastructure: All tools in the platform use the same security, metadata, administration, portal integration, object model and query engine, and should share the same look and feel. Metadata management: Tools should leverage the same metadata, and the tools should provide a robust way to search, capture, store, reuse and publish metadata objects, such as dimensions, hierarchies, measures, performance metrics and report layout objects. Development tools: The platform should provide a set of programmatic and visual tools, coupled with a software developer's kit for creating analytic applications, integrating them into a business process, and/or embedding them in another application. Collaboration: Enables users to share and discuss information and analytic content, and/or to manage hierarchies and metrics via discussion threads, chat and annotations.

Information Delivery Reporting: Provides the ability to create formatted and interactive reports, with or without parameters, with highly scalable distribution and scheduling capabilities. Dashboards: Includes the ability to publish Web-based or mobile reports with intuitive interactive displays that indicate the state of a performance metric compared with a goal or target value. Increasingly, dashboards are used to disseminate real-time data from operational applications, or in conjunction with a complex-event processing engine. Ad hoc query: Enables users to ask their own questions of the data, without relying on IT to create a report. In particular, the tools must have a robust semantic layer to enable users to navigate available data sources. Microsoft Office integration: Sometimes, Microsoft Office (particularly Excel) acts as the reporting or analytics client. In these cases, it is vital that the tool provides integration with Microsoft Office, including support for document and presentation formats, formulas, data "refreshes" and pivot tables. Advanced integration includes cell locking and write-back. Search-based BI: Applies a search index to structured and unstructured data sources and maps them into a classification structure of dimensions and measures that users can easily navigate and explore using a search interface. Mobile BI: Enables organizations to deliver analytic content to mobile devices in a publishing and/or interactive mode, and takes advantage of the mobile client's location awareness.

Analysis Online analytical processing (OLAP): Enables users to analyze data with fast query and calculation performance, enabling a style of analysis known as "slicing and dicing." Users are able to navigate multidimensional drill paths. They also have the ability to write back values to a proprietary database for planning and "what if" modeling purposes. This capability could span a variety of data architectures (such as relational or multidimensional) and storage architectures (such as disk-based or in-memory). Interactive visualization: Gives users the ability to display numerous aspects of the data more efficiently by using interactive pictures and charts, instead of rows and columns. Predictive modeling and data mining: Enables organizations to classify categorical variables, and to estimate continuous variables using mathematical algorithms. Scorecards: These take the metrics displayed in a dashboard a step further by applying them to a strategy map that aligns key performance indicators (KPIs) with a strategic objective. Prescriptive modeling, simulation and optimization: Supports decision making by enabling organizations to select the correct value of a variable based on a set of constraints for deterministic processes, and by modeling outcomes for stochastic processes.

You cannot accept problems as handed to you in the business environment. Never allow yourself to be the analyst to whom problems are “thrown over the fence.” Engage with the people whose challenges you’re tackling to make sure you’re solving the right problem. Learn the business’s processes and the data that’s generated and saved. Learn how folks are handling the problem now, and what metrics they use (or ignore) to gauge success.

Solve the correct, yet often misrepresented, problem. This is something no mathematical model will ever say to you. No mathematical model can ever say, “Hey, good job formulating this optimization model, but I think you should take a step back and change your business a little instead.” And that leads me to my next point: Learn how to communicate.

In today’s business environment, it is often unacceptable to be skilled at only one thing. Data scientists are expected to be polyglots who understand math, code, and the plain-speak (or sports analogy-ridden speak . . . ugh) of business. And the only way to get good at speaking to other folks, just like the only way to get good at math, is through practice.

1. Project Sponsorship and Governance 2. Project Management 3. Development Team (Core Team) 4. Extended Project Team

1. Project Sponsorship and Governance IT and the business should form a BI steering committee to sponsor and govern design, development, deployment, and ongoing support. It needs both the CIO and a business executive, such as CFO, COO, or a senior VP of marketing/sales to commit budget, time, and resources. The business sponsor needs the project to succeed. The CIO is committed to what is being built and how.

2. Project Management Project management includes managing daily tasks, reporting status, and communicating to the extended project team, steering committee, and affected business users. The project management team needs extensive business knowledge, BI expertise, DW architecture background, and people management, project management, and communications skills. The project management team includes three functions or members: Project development manager - Responsible for deliverables, managing team resources, monitoring tasks, reporting status, and communications. Requires a hands-on IT manager with a background in iterative development. Must understand the changes caused by this approach and the impact on the business, project resources, schedule and the trade-offs. Business advisor - Works within the sponsoring business organization. Responsible for the deliverables of the business resources on the project's extended team. Serves as the business advocate on the project team and the project advocate within the business community. Often, the business advocate is a project co-manager who defers to the IT project manager the daily IT tasks but oversees the budget and business deliverables. BI/DW project advisor - Has enough expertise with architectures and technologies to guides the project team on their use. Ensures that architecture, data models, databases, ETL code, and BI tools are all being used effectively and conform to best practices and standards.

Data scientists are the magicians of the Big Data era. They crunch the data, use mathematical models to analyze it and create narratives or visualizations to explain it, then suggest how to use the information to make decisions.

Rachel Schutt, a senior research scientist at Johnson Research Labs, taught “Introduction to Data Science” last semester at Columbia (its first course with “data science” in the title). She described the data scientist this way: “a hybrid computer scientist software engineer statistician.” And added: “The best tend to be really curious people, thinkers who ask good questions and are O.K. dealing with unstructured situations and trying to find structure in them.”

Excel Power Pivot is targeted for Personal and Team Business Intelligence (BI) solution use cases.

Power Pivot also is excellent for quick prototypes and proofs-of-concept.

no row level security,

Mistake 1.  Not putting a strong interdisciplinary team together. It is impossible to put together an analytics platform without understanding the needs of the customers who will use it.  Sounds simple, right?  Who wouldn’t do that?  You’d be surprised how many analytics projects are wrapped up by IT because “they think” they know the customer needs.  Not assembling the right team is clearly the biggest mistake companies make.  Many times what is on your mind (and if you’re an IT person willing to admit it) is that you are considering converting all those favorite company reports.  Your goal should not be that.  Your goal is to create a system—human engineered with customers, financial people, IT folks, analysts, and others—that give people new and exciting ways to look at information.  It should give you new insights. New competitive information.  If you don’t get the right team put together, you’ll find someone longing for the good old days and their old dusty reports.  Or worse yet, still finding ways to generate those old dusty reports. Mistake 2.  Not having the right talent to design, build, run and update your analytics system.  It is undeniable that there is now high demand for business analytics specialists.  There are not a lot of them out there that really know what to do unless they’ve been burned a few times and have survived and then built successful BA systems.  This is reflected by the fact you see so many analytics vendors offer, or often recommend, third-party consulting and training to help the organization develop their business analytic skills.  Work hard to build a three-way partnership between the vendor, your own team, and an implementation partner.  If you develop those relationships, risk of failure goes way down.

Mistake 3.  Putting the wrong kind of analyst or designer on the project. This is somewhat related to Mistake 2 but with some subtle differences.  People have different skillsets so you need to make sure the person you’re considering to put on the project is the right “kind.”  For example, when you put the design together you need both drill-down and summary models.  Both have different types of users.  Does this person know how to do both?  Or, for example, inexperience in an analyst might lead to them believing vendor claims and not be able to verify them as to functionality or time to implement. Mistake 4.  Not understanding how clean the data is you are getting and the time frame to get it clean.  Profile your data to understand the quality of your source data.  This will allow you to adjust your system accordingly to compensate for some of those issues or more importantly push data fixes to your source systems.  Ensure high quality data or your risk upsetting your customers.  If you don’t have a good understanding of the quality of your data, you could easily find yourself way behind schedule even though the actual analytics and business intelligence framework you are building is coming along fine. Mistake 5.  Picking the wrong tools.  How often do organizations buy software tools that just sit on the shelve?  This often comes from management rushing into a quick decision based on a few demos they have seen.  Picking the right analytics tools requires an in-depth understanding of your requirements as well as the strengths and weaknesses of the tools you are evaluating.  The best way to achieve this understanding is by getting an unbiased implementation partner to build a proof of concept with a subset of your own data and prove out the functionality of the tools you are considering. Bottom Line.  Think things through carefully. Make sure you put the right team together.  Have a data cleansing plan.  If the hype sounds too good to be true—have someone prove it to you.

Channel Optimization. Many marketers struggle to optimize each individual channel, let alone optimizing at a customer level across many channels. To the extent that Big Data can help marketers understand what is important in the moment and across touch points, that could be valuable, but it seems more of us need stronger attribution models and analytics methodologies more than access to data. Big Data does seem to be valuable if you want to understand which customers are highest value within each channel and across channels, because the platforms that manage Big Data can handle both structured and unstructured data - which is what you need to truly include Web/clickstream and social data in your analysis.

The best practice in business intelligence delivery is always to build a data warehouse.

Pure transactional reporting is problematic. There are, of course, the usual performance issues. Equally troublesome is the difficulty in distilling a physical model down to a format that is easy for business users to understand. Dimensional models are typically the way business users envision their business: simple, inclusive structures for each entity. The standard OLTP data model that takes two of the four walls in the conference room to display will never make sense to your average business user.

What this means in practice is that when the BI Server component starts up, it creates and reserves a number of threads in advance, determined by a number of parameters including SERVER_THREAD_RANGE.

You can see these threads running and ready to perform tasks for the BI Server component by using a tool such as Process Explorer for Windows

Thinking it through a bit, any given single query is, to a certain extent, only really going to use a small part of the total amount of CPUs available on a server, because it’s not the BI Server that runs queries in parallel, it’s the underlying database. For example, a single analysis against a single Oracle Database datasource would only really need a single BI Server thread to handle the query request, but when the underlying database receives the query, it might use a large number of its CPUs to process the query, returning results back to the BI Server to then pass back to the Presentation Server for display to the user.

Some of the Hadoop projects we will talk about are: HDFS : A distributed filesystem that runs on large clusters of commodity machines. Map Reduce: A distributed data processing model and execution environment that runs on large clusters of commodity machines. Pig: A data flow language and execution environment for exploring very large datasets. Pig runs on HDFS and MapReduce clusters. HBase: A distributed, column-oriented database. HBase uses HDFS for its underlying storage, and supports both batch-style computations using MapReduce and point queries (random reads). ZooKeeper: A distributed, highly available coordination service. ZooKeeper provides primitives such as distributed locks that can be used for building distributed applications. Oozie: Oozie is a workflow scheduler system to manage Apache Hadoop jobs.

OBIEE 11g skills, both in terms of new functionality (mapping, analyses, KPIs and Scorecards etc) and new infrastructure (WebLogic, EM, OPSS etc) A smattering of Essbase skills, focused mainly on the integration with OBIEE and Essbase (and the many workarounds and gotchas) Good ODI skills, both in terms of the basics, but also being able to write knowledge modules, integrate with OBIEE, deployment and migration Solid database skills – OBIEE gave the illusion through aggregates etc that database tuning was redundant, but time has shown it’s by far the biggest success factor in a project – get the database design and optimisation wrong, and your project is toast. You need to know partitioning, materialized views, index types, and increasingly, you need to get yourself on an Exadata project as customers are buying the technology but you can’t teach it to yourself at home BI Apps skills, but watch out for everything changing when BI Apps 11g comes out, and be prepared to learn the Fusion Apps and JDeveloper if you want to stay in the game Looking to the future, keep an eye on technologies such as in-memory (TimesTen), mid-tier caching (Coherence), plus technologies such as Business Activity Monitoring (BAM), “big data” (Hadoop, large data sets, NoSQL), complex event processing and maybe products such as Qlikview, just in case Oracle buys them, or at least to know what the competition are up to, or more importantly pitching to your boss

The other thing to bear in mind of course, if you’re an Oracle BI developer, is that you need to have great business, communication and data modeling skills.

In the past, prospects primarily accessed information about a company by interacting directly with a salesperson.

As media evolved, mass ads, events, direct mail, and more recently, email, have been the primary tools for engagement.